Transcript #67: Result of moving Python to Github

00:00 Hello and welcome to Python bites where we deliver Python news and headlines directly to your earbuds This is episode 67 recorded February 28th 2018 I'm Michael Kennedy and I'm Brian Harkin and we have a ton of great stuff for you as always It's getting harder and harder to narrow it down to just keep this like bite-sized stuff because there's so much amazing stuff out there Isn't there Brian? Yeah, there is and we got a lot this week. Oh, yeah for sure So before we get on to those I just wanna say thanks to digital ocean They're our sponsor for this week.

00:30 It's where our code runs.

00:31 So when you download the audio or you go visit the website, you're running a little slice of that ocean.

00:37 So check them out at do.co/python.

00:40 Now, Brian, I would have a better performing site and be able to run on a much smaller server if I literally had just files.

00:50 No logic, no database queries, no database server, just files, right?

00:55 Yeah, and you can host it cheaper there sometimes too for static sites.

01:00 Yeah, absolutely.

01:01 So you've got a, the first item is something on Pelican, which is one of the more popular static site generators, right?

01:08 Right.

01:09 So somebody on Twitter actually asked us, they said they were going to, I forget who it was, but they said, "Hey, I'm going to build a new blog.

01:16 What are the Python options now?" A few people came back with different static site generators.

01:24 One of the people that just found this out today is a guy named Peter.

01:29 I believe he's a professor at Portland Community College that has a whole series, a blog series, on how to build a blog with Pelican and host it up as a GitHub page.

01:40 You don't even have to pay for your hosting.

01:43 Like the pages.github.io or something like that?

01:46 Yeah.

01:47 Yeah, nice.

01:48 The focus of his site, it's Python for Undergrad Engineers.

01:53 Dot com, what a mouthful.

01:56 But cool focus, he's not assuming really much of anything ahead of time, so he even walks through installing Python and making sure people are using Git appropriately and saving their work.

02:09 So even though it's a seven part series, if you know what you're doing, you can probably go through that pretty fast.

02:15 I thought it was a nice tutorial on how to get a blog up.

02:18 - That's really cool, and I love the fact that he takes it all the way to deployment, because it's one thing to show you like, well here, now you've got a bunch of static files, and good luck with that.

02:28 So, like getting it online is like a nice final step for his article there.

02:33 - One of the other places I know uses Pelican is Fullstack, Matt McKay's site.

02:38 - Yeah, and he's doing amazing, amazing stuff, and his site is just so super fast.

02:42 He really thinks a whole lot about that.

02:44 That's really cool.

02:46 There's actually a number of options around this kind of stuff, but Pelican seems to have a lot of mind share in the Python space for sure.

02:52 - Yeah. - Yeah, cool.

02:53 So how would you like to get notified in a whole bunch of different ways if something went wrong with your static site, right?

02:58 - I would.

02:59 - You could use a service like Rollbar or something, but I'm thinking more like, hey, we've processed your order and now it's put onto a ship and it's headed your way, or the build server has built some version of your app and the build is ready for deployment, or, you know, those kinds of notifications, right?

03:17 And there's just this explosion of places where you might want to notify people of these things.

03:23 And they all have their own API.

03:26 You know, if you're lucky, there's a pip install thing.

03:29 And you can at least use that.

03:30 But right, sometimes there's not even that there's just a API you got to sign up for.

03:34 So I'm thinking things like Slack, email, Gmail, Telegram, getter, you name it, there's a whole bunch of different places where you might want to push notifications that something like this happened, right?

03:47 And these grow every day.

03:48 Yeah, yeah.

03:49 So there's this thing called notifiers.

03:51 And what notifiers does is, it's like this abstraction API front end for all these different other API's.

04:00 So you can work with the notifier API and say, I want to notify the things.

04:06 And you can like register a slack provider, register a getter provider, a telegram provider, email provider and just check those off and then use the same API against all of them.

04:17 That's actually really cool.

04:18 Yeah, it's really cool.

04:19 I can imagine like individual like a selection on your site for different users.

04:24 So some people would rather get a Slack notification and some people would rather get a text message and some people wanted it in Slack or something.

04:32 Yeah, and so you could just list off all the providers that you support, check them off, but your code doesn't really change.

04:38 it just feeds that on through this notifiers thing.

04:40 So like you could send a text message 'cause they have integration with Twilio and then Twilio would let that push on through and so on.

04:47 There's a bunch of stuff.

04:48 There's some things that I don't even know what they are like push bullet.

04:49 I have no idea what that is.

04:50 It's probably cool.

04:51 But it's cool that I don't have to learn its API to integrate.

04:55 Somebody comes and says, "Hey, Michael, your site's cool, "but could you have push bullet integration?" Yeah, sure.

05:00 Here's the checkbox.

05:01 - Yeah, that's actually awesome.

05:02 I'm gonna have to play with some of that.

05:04 - Yeah, I don't really have a great reason to notify people, but I thought it was cool.

05:08 One final thought on notifiers was we've talked for quite some time.

05:12 We talk less about it these days, I think, but we've talked about how one of the points of resistance to moving to Python 3 was, well, there's this one or two libraries I really need, and it's only Python 2 and I can't leave it behind.

05:24 We're now in a world where unless you move to Python 3, these things are inaccessible to you.

05:30 And one of them is notifiers, which is Python 3 only.

05:33 Pretty cool.

05:34 Yeah.

05:34 So it's becoming more and more actually.

05:36 It's sort of reversing that, what used to be friction resistance, which is now pushing people.

05:41 The gravity is increasing, obviously, which is great.

05:44 Yeah.

05:44 It's been a long time since I've used a Makefile, and I think it was in C++ when I last really cared about it.

05:50 Yeah, actually. So I hadn't really even thought about Makefiles too much, but I got notified today about KidPixo. He's contributed to us before, But he notified me today that said that combination of information from my book for on Pytest and also a blog post by I'm going to go with Christoph is about using make files and Python projects.

06:15 And he's got all of his now all of his testing is driven through make files as well make files through Pytest.

06:22 And so there's, it's actually kind of a neat idea.

06:25 He, this article talks about starting up in a new work group and being surprised by make files in Python projects because there's not that much compiling, right?

06:34 Yeah.

06:35 And it's not for compiling.

06:36 It's just, there's a lot of extra little, little, utility work that people do and make files are an easy way to keep them all in one place.

06:45 And some of the examples are like cleaning out all of your pyc files or if you normally create Packages, you'll have egg directories lying around and clean those out running your linter or you know Running your test and you could have like multiple targets to start different types of testing if you wanted to like, you know Start your smoke test or start with them, you know your full test or something or even up to like deploying your code up onto a server possibly. So anyway, I thought that was cool.

07:15 Yeah, it's quite cool and definitely a nice article.

07:18 So it's cool to bundle all these actions.

07:21 The more you can automate and just bundle up and take that knowledge of managing and working with your project and make it one CLI command or maybe even something less than that, something automatic just like say a git trigger or something like that. It's just it's all good, right? The article also assumes that you've not really worked with makefiles before. So it tells you some of the gotchas. And one of the ones that bites me all the time is you really do want tabs, not spaces when you're working with makefiles. And that's the opposite of how I use Python. So something to watch out for.

07:56 - Yeah, definitely opposite, interesting.

07:59 Okay, so speaking of things that are good, DigitalOcean, they're pretty good.

08:02 They're hosting our code and our apps here.

08:06 I just wanna repeat something that I talked about last time, which is that they've basically doubled all of the things for the same price, which is a really big deal.

08:15 They've been super competitive and really good and compared to the other hosting places, small and large, right?

08:23 You go up there, 30 seconds, you are able from the time you decide to click the button till your server's up and running, you can log in.

08:30 And that's all great.

08:31 But they basically said everything that you're gonna pay for for five bucks, we're gonna double that, right?

08:36 Double the RAM, you're gonna pay 20 bucks, we'll double the RAM and the CPU.

08:40 And there's all sorts of high computational stuff they've added.

08:44 So it's really a great place.

08:45 And it's gotten a lot better recently with like this big upgrade or price deduction and availability of higher ones.

08:51 So if you're thinking about checking them out, go up there at do.co/python, help support the show.

08:58 And also, if you already are a customer, consider upgrading your servers and stuff you have because you're still paying the $10 price for half the thing.

09:08 So if you're willing to take it down for two minutes and upgrade it, you'll get something better for very, very little work.

09:12 So check them out, definitely good stuff.

09:15 Thanks Digital Ocean for supporting the show.

09:16 So we talked a while ago about Python, CPython in particular, moving to GitHub, right?

09:24 - Yes.

09:25 - I can't remember exactly when that was, but it was in 2016, so it must have been one of our early episodes.

09:31 Can you remember when?

09:31 - We can probably look it up, but yeah, I don't remember.

09:34 - But so what I wanted to feature next was there's this post, and I don't remember if it was actually just on Reddit, or if it's actually, yeah, it's just on Reddit.

09:46 It's literally a picture on Reddit.

09:48 But the conversation is interesting, the picture is interesting. And the title is the result of moving Python to GitHub.

09:56 And so they put this graph up there, which the graph is not super well labeled. So it's kind of hard to tell. But it looks like there's a lot more activity since Python moved to GitHub from material, right. And they do call out a couple of numbers saying quick couple of numbers, including PRS from 2017, compared to 2016. The number of commits increased by 190%. Wow, that's a big deal, right? The inserted lines of code have increased by 140%. The number of unique contributors and this is probably the biggest. Actually, this is probably the biggest thing right here. And I think this is the biggest reason to go to GitHub.

10:35 The number of unique contributors has increased by 13 times. That's wonderful. Isn't that crazy? Yeah. Our new Python releases that come out are like there's way more stuff in than there used to be.

10:47 - Yeah, that's for sure.

10:48 This is really cool.

10:49 And finally, not just new or unique contributors, but we now have more returning contributors.

10:55 There are nine times as many returning contributors as previously.

10:58 So people tried it and they liked it.

11:00 So that was cool.

11:01 I guess that's really the news, right?

11:03 You can see the picture.

11:04 The picture's actually not that valuable, but those numbers are.

11:07 And the conversation under it is more of a, it's more of a conversation of like, what is the value of statistics?

11:15 So, you know, that's an interesting conversation to have on the internet around graphs.

11:20 But one comment was, when somebody said, "Oh, I would like them to move to GitLab instead," but this is still good news.

11:26 What do you think?

11:27 Do you think we'd have this 13 times the number of contributors if it was on GitLab?

11:30 I don't think so on GitLab, although I use both GitHub and GitLab, and they're both great products.

11:36 But the GitHub is definitely more of a social place to have more contributors.

11:42 I tend to agree.

11:43 I think GitLab is great.

11:45 I think also places like Bitbucket are great, but they just don't have that social momentum.

11:51 It's like, well, I could have posted the question on Stack Overflow, but there's another forum that some developers go to, so I'm gonna put it over there.

11:58 It's like, well, you can, but it's probably not gonna have the same reach and effect, right?

12:02 So I think it's really great, and really, thanks to Brett Cannon, who spearheaded this, one of the core developers, who got Python over there in the first place.

12:12 - Yeah, it's pretty cool.

12:14 - Yeah, yeah, definitely.

12:15 So as developers, it's easy for us to have kind of a bad self-image, look around and go, oh, all these people are so smart, I'm not very good, I'm stuck.

12:24 I think two days ago, I was more frustrated than I have been in probably months trying to get something to work.

12:31 I was really frustrated.

12:33 So you've got an article kind of around this, right?

12:35 - Actually, somebody posted an article called Self-Deprecation Needs to Stop.

12:39 And really, this was Maurice Hayward, And it was inspired by some tweets by Stephanie Hurlburt.

12:45 And the comments apparently started with, she was trying to get more women to blog about what they're doing, their technology stuff, and that she would retweet the blog posts.

12:57 And many of the ones that she was told about were comments like, well, this project isn't that good, but here it is anyway.

13:05 And I guess I'm paying attention to this too, 'cause there's a fine line between being honest about what your skill level is and how you can help, and also trying not to be arrogant and act like an expert when you're not.

13:20 But there's, I mean, if you spent some time working on something and learning it, there are people out there that know less than you.

13:26 And I think people need to be, I like this article 'cause people need to be better about understanding that they have learned a lot and they can contribute.

13:35 This also resonated with me because as a parent, I really try to stop my kids from self-deprecating speech and not say things like, well, I'm dumb, this was dumb, because it can give you a negative self-image and that's bad, and we do see it a lot in the tech field, I think partly because we like to solve problems, so we're always looking at people that know the answer to our problems who clearly know more than us because they've already figured it out.

14:03 So this is good.

14:04 - Yeah, it's definitely good, and no matter how experienced you are, you can find something that will frustrate you and make you feel stupid in tech and programming.

14:11 I'm sure of it.

14:12 - This was repeated in the article, but it was from Stephanie.

14:15 He has really like three points, which I numbered them one, three, four, which is interesting.

14:21 I'll have to fix that.

14:22 But first off, stop saying words like, my project is not good, it's just a thing I wrote, or it's just by a newbie, or it's silly, or it's not useful.

14:31 Just share it and let others be the judge.

14:33 Second point is really think about the value you bring and let everybody know what that is.

14:39 This is advertising for yourself and your public image also might reflect on a job interview sometimes, so don't be too hard on yourself.

14:47 And also just be proud of the things that you've done.

14:50 It's completely okay to say, man, I did this thing, it was really cool, and here it is.

14:55 - Yeah, I think that's great advice.

14:56 People should definitely put what they do out there and just let people know what they're up to.

15:01 Very nice.

15:02 All right, so we wouldn't be a Python Bytes if we didn't somehow feature our friend, Anthony Shaw's work, would it?

15:10 - Yeah.

15:11 - It seems like he is so prolific in doing such cool stuff.

15:14 The last item I wanna cover is something called five speed improvements to Python 3.7.

15:19 - Oh, this is nice.

15:20 - Yeah, yeah, so Anthony's digging deep into Python 3.7 for a project that he's working on, and I'm sure he'll talk about that later.

15:28 But he's been trying to explore this through blogging and so on.

15:31 And so, this is a really nice article for a couple reasons. One, it's nice to call out the things that are going to be faster in June.

15:38 That's great when Python 3.7 comes out. It's also cool because you see him doing a lot of like internal introspection and looking into Python like with the disassembler and the opcodes and seeing how different methods actually turn into operations in the CPython interpreter, which is really cool. So let's talk about the five things. So one is calling methods will be faster.

16:04 kind of maybe sometimes in Python 3.7. Okay. Okay. So the reason is, there's actually two new op codes that are being added. And under certain circumstances, these can either remove the slowness of methods, you know, functions tied to classes, or generally other ones, I think, if you have a bound method with no arguments, it's going to be faster than it was before.

16:31 So he has some numbers. I don't remember exactly what the numbers on this one are.

16:34 String.find is faster for some characters by a lot. So some of the time you're working with standard ASCII characters and everything's fine on string.find. But for some reason, the way Unicode is laid out, there's certain types of characters that can make it up to 25 times slower to call string.find based on the input. That's crazy, right?

16:57 Yeah.

16:58 Yeah, so they said these are still slower than standard characters in Python 3.7, but now they're only three times as slow, not 25 times as slow. So that's still a really good speed up, right? Close to 8x. That's good. Let's see really quickly os.fwalk, which creates like a tree directory structure type thing, which you can point it at a folder, and it'll create like a subtree of files and folders now twice as fast. Regular expressions are faster, but only in like a weird way.

17:30 Like they're less, they were changed in Python three, six, basically for, I don't actually know why they were changed, but they were changed in Python three, six, and that slowed them down. So now this fix kind of puts them almost back to as fast as they were in three, five. So take that for what it is. And finally, regular expressions that are case insensitive are now 20 times faster. That's really good. So basically, the internal implementation of that, it used to call lower on every character, which I guess was slow. So now I can just like, work on them in a different way. It's great. How did I not know about f walk before? Yeah, I got to look that looks neat. It does look neat, right? Like, I've traditionally used like recursion and stuff like that to do all this. But yeah, I don't think you need it. That's to check that out. Maybe you still do. But it definitely it returns like a tuple with four items with like the directory, the sub directories, the files, things like that. It's pretty cool.

18:25 So does he have like some, I haven't read this article, is there, are there like a benchmark of a, you know, a little application that runs in Python three, six versus three, seven?

18:35 And that is a good question. I did not see that it may be in there. He was just quoting the numbers of his testing, but I don't know if he actually had a demo. He may and I just didn't catch it.

18:46 Hi Brian, that's it for our news this week, but you've got some follow-up stuff.

18:51 First of all, we talked about your pytest, PyCharm webinar, and that was in the past at this point, but the recording just came out, didn't it?

18:59 It did just come up, and so I went back and this last week I listened to it, and there's a couple things that I noticed, especially with respect to this, you know, don't be hard on yourself and self-deprecating speech because my memory of it was I felt really embarrassed about the times where I didn't know the answer because people are asking in real time and I don't want to go and look it up or something.

19:24 You're on the spot, hundreds of people are there and it's being recorded.

19:26 Yeah, so it just gave me a like, and then I listened back to it, it's really not that bad and so there are some pauses that I have there which are fine, but I think I'm going I'm gonna try to be easier on myself.

19:39 I think it'd be a useful thing to watch if people aren't familiar with pytest or using it in PyCharm.

19:44 But the other thing was also just, on my to-do list is try to come up with better ways to, or some way to say, you know what, I don't know the answer to that right off the bat.

19:55 I'll get back to you.

19:56 Or maybe that's it.

19:58 - Yeah, I don't know the answer.

19:59 I have no, yeah, I mean, the last one I did, people asked me about Solr, S-O-L-R, which is a full-text search type of thing.

20:06 I've learned since then.

20:08 How does this work with Solr?

20:09 I have no idea what you're talking about, I'm sorry.

20:12 - Yeah, yeah, it's okay.

20:14 We don't have to be experts on everything.

20:16 The other thing I wanted to bring up is there's a, I don't think it really deserved an entire story, but the PyPA has put up a post that says they'd like help from package maintainers to test out the new PyPI, because they've gotten funding to do some fixes and they want to see how they work.

20:35 - Oh, that's really cool.

20:36 And a couple of cool things about it is I think people should go help them out.

20:40 But the link that I put up, which apparently is broken, I'll fix it by the time this goes out.

20:45 If you're asking people for help in beta testing something, this format that they put up is a really nice format to kind of use of these are the kind of things that we'd like you to try out some of the workflows and let us know and how to how to let us know.

20:58 So there's a good template.

21:00 Here's 10 things we would love for you to test.

21:02 Here's how you check on security.

21:03 Here's how we have live chat hours if you find something you want to talk to us about that's great Alright. Well, I don't really have too much news. I've been working on a whole bunch of projects but Nothing to announce at this point. So just have to keep those under wraps for a little while. All right. All right, Brian Thanks for finding all these things. Like I said every week. These are just so much fun to explore them with you So appreciate you being here. I appreciate it. Also. Thanks. Yep. Thanks. Bye everyone Thank you for listening to Python Bytes. Follow the show on Twitter via @pythonbytes.

21:35 That's Python Bytes as in B-Y-T-E-S. And get the full show notes at pythonbytes.fm.

21:41 If you have a news item you want featured, just visit pythonbytes.fm and send it our way.

21:45 We're always on the lookout for sharing something cool. On behalf of myself and Brian Auchin, this is Michael Kennedy.

21:52 Thank you for listening and sharing this podcast with your friends and colleagues.