Transcript #67: Result of moving Python to Github

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 67, recorded February 28th, 2018.

00:10 I'm Michael Kennedy.

00:11 And I'm Brian Okken.

00:11 And we have a ton of great stuff for you.

00:13 As always, it's getting harder and harder to narrow it down to just keep this, like, bite-sized stuff,

00:18 because there's so much amazing stuff out there, isn't there, Brian?

00:20 Yeah, there is. And we got a lot this week.

00:23 Oh, yeah, for sure.

00:24 So before we get on to those, I just want to say thanks to DigitalOcean.

00:27 They're our sponsor for this week.

00:30 It's where our code runs.

00:31 So when you download the audio or you go visit the website, you're running a little slice of that ocean.

00:36 So check them out at do.co.python.

00:39 Now, Brian, I would have a better performing site and be able to run on a much smaller server if I literally had just files.

00:50 No logic, no database queries, no database server, just files, right?

00:55 Yeah. And you can host it cheaper there sometimes, too, for statics.

00:59 Yeah, absolutely.

01:00 So you've got a—the first item is something on Pelican, which is one of the more popular static site generators, right?

01:08 Right.

01:08 So somebody on Twitter actually asked us.

01:11 They said they were going to—I forget who it was, but they said, hey, I'm going to build a new blog.

01:16 What are the Python options now?

01:18 And a few people came back with different static site generators.

01:23 And one of the people that I just found this out today is a guy named Peter, and he's—I believe he's a professor at Portland Community College that has a whole series, a blog series, on how to build a blog with Pelican and host it up on—as a GitHub page.

01:40 So you don't even have to pay for your hosting.

01:42 At, like, the pages.github.io or something like that?

01:46 Yeah.

01:46 Yeah, nice.

01:47 And the focus of his site, it's pythonforundergradengineers.com.

01:54 What a mouthful.

01:55 But cool focus.

01:57 He's not assuming really much of anything ahead of time.

02:01 So he even walks through installing Python and making sure people are using Git appropriately and saving their work.

02:09 So even though it's a seven-part series, if you know what you're doing, you can probably go through that pretty fast.

02:14 I thought it was a nice tutorial on how to get a blog up.

02:18 Yeah, that's really cool.

02:19 And I love the fact that he takes it all the way to deployment because it's one thing to show you, like, well, here, now you've got a bunch of static files, and good luck with that.

02:29 Yeah, like, getting it online is, like, a nice final step for his article there.

02:33 One of the other places I know uses Pelican is Fullstack, Matt McKay's site.

02:38 Yeah, and he's doing amazing, amazing stuff.

02:40 And his site is just so super fast.

02:42 He really thinks a whole lot about that.

02:44 That's really cool.

02:45 There's actually a number of options around this kind of stuff, but Pelican seems to have a lot of mindshare in the Python space, for sure.

02:52 Yeah.

02:52 Yeah, cool.

02:53 So how would you like to get notified in a whole bunch of different ways if something went wrong with your static site, Brian?

02:58 I would.

02:58 You know, you could use a service like Rollbar or something, but I'm thinking more like, hey, we've processed your order, and now it's put onto a ship, and it's headed your way.

03:08 Or the build server has built some version of your app, and the build is ready for deployment.

03:14 Or, you know, those kinds of notifications, right?

03:17 And there's just this explosion of places where you might want to notify people of these things.

03:23 And they all have their own API.

03:25 You know, if you're lucky, there's a pip install thing, and you can at least use that.

03:30 But, right, sometimes there's not even that.

03:32 There's just an API you've got to sign up for.

03:34 So I'm thinking things like Slack, email, Gmail, Telegram, Gitter, you name it.

03:40 There's a whole bunch of different places where you might want to push notifications that something like this happened, right?

03:47 And these grow every day.

03:47 Yeah.

03:48 Yeah.

03:48 So there's this thing called notifiers.

03:50 And what notifiers does is it's like this abstraction API front end for all these different other APIs.

04:00 So you can work with the notifier API and say, I want to notify the things.

04:05 And you can, like, register a Slack provider, register a Gitter provider, a Telegram provider, Gmail provider, and just check those off.

04:13 And then use the same API against all of them.

04:16 That's actually really cool.

04:18 Yeah.

04:18 Yeah, it's really cool.

04:19 I can imagine, like, an individual, like, a selection on your site for different users.

04:24 So some people would rather get a Slack notification.

04:27 Some people would rather get a text message.

04:30 And some people wanted it in Slack or something.

04:32 Yeah.

04:32 And so you could just list off all the providers that you support, check them off.

04:35 But your code doesn't really change.

04:37 It just feeds that on through this notifiers thing.

04:40 So, like, you could send a text message because they have integration with Twilio.

04:43 And then Twilio would let that push on through and so on.

04:46 There's a bunch of stuff.

04:47 There's some things that I don't even know what they are, like Pushbullet.

04:49 I have no idea what that is.

04:50 It's probably cool.

04:51 But it's cool that I don't have to learn its API to integrate.

04:55 Somebody comes and says, hey, Michael, your site's cool, but could you have Pushbullet integration?

04:59 Yeah, sure.

05:00 Here's a checkbox.

05:01 Yeah, that's actually awesome.

05:02 I'm going to have to play with some of that.

05:03 Yeah.

05:04 I don't really have a great reason to notify people, but I thought it was cool.

05:07 One final thought on notifiers was we've talked for quite some time.

05:12 We talk less about it these days, I think.

05:14 But we've talked about how one of the points of resistance to moving to Python 3 was, well, there's this one or two libraries I really need.

05:22 It's only Python 2, and I can't leave it behind.

05:24 We're now in a world where unless you move to Python 3, these things are inaccessible to you.

05:30 And one of them is notifiers, which is Python 3 only.

05:33 Pretty cool.

05:33 Yeah.

05:34 So it's becoming more and more, actually.

05:36 It's sort of reversing that what used to be friction resistance, which is now pushing people.

05:41 The gravity is increasing, obviously, which is great.

05:44 Yeah.

05:44 It's been a long time since I've used a makefile.

05:46 And I think it was in C++ when I last really cared about it.

05:50 Yeah, actually, so I hadn't really even thought about makefiles too much.

05:54 But I got notified today about KidPixo.

05:58 He's contributed to us before.

06:00 But he notified me today that said that a combination of information from my book on pytest and also a blog post by, I'm going to go with Christoph,

06:12 is about using makefiles in Python projects.

06:15 And he's got all of his, now all of his testing is driven through makefiles as well, makefiles through pytest.

06:21 And so there's, it's actually kind of a neat idea.

06:25 This article talks about starting up in a new work group and being surprised by makefiles in Python projects.

06:32 Because there's not that much compiling, right?

06:34 Yeah.

06:35 And it's not for compiling.

06:36 It's just there's a lot of extra little utility work that people do.

06:41 And makefiles are an easy way to keep them all in one place.

06:45 And some of the examples are like cleaning out all of your PYC files.

06:49 Or if you normally create packages, you'll have egg directories lying around and clean those out.

06:55 Running your linter or, you know, running your test.

06:58 And you could have like multiple targets to start different types of testing if you wanted to.

07:03 Like, you know, start your smoke test or start with, you know, your full test or something.

07:08 Or even up to like deploying your code up onto a server, possibly.

07:14 So anyway, thought that was cool.

07:15 Yeah, it's quite cool.

07:16 And definitely a nice article.

07:18 So it's cool to bundle all these actions.

07:21 The more you can automate and just bundle up and take that knowledge of managing and working with your project and make it one CLI command or maybe even something less than that.

07:33 Something automatic.

07:34 It's like, say, a git trigger or something like that.

07:37 It's just, it's all good, right?

07:38 The article also assumes that you've not really worked with make files before.

07:42 So it tells you some of the gotchas.

07:44 And one of the ones that bites me all the time is you really do want tabs, not spaces when you're working with make files.

07:52 And that's the opposite of how I use Python.

07:55 So something to watch out for.

07:56 Yeah, definitely opposite.

07:57 Interesting.

07:58 Okay.

07:59 So speaking of things that are good, DigitalOcean, they're pretty good.

08:02 They're hosting our code and our apps here.

08:06 I just want to repeat something that I talked about last time, which is that they've basically doubled all of the things for the same price, which is a really big deal.

08:15 They've been super competitive and really good compared to the other hosting places, small and large, right?

08:23 You go up there, 30 seconds, you are able, from the time you decide to click the button until your server is up and running, you can log in.

08:30 And that's all great.

08:30 But they basically said, everything that you're going to pay for, for five bucks, we're going to double that, right?

08:36 Double the RAM, you're going to pay 20 bucks, we'll double the RAM and the CPU.

08:40 And there's all sorts of high computational stuff they've added.

08:44 So it's really a great place.

08:45 And it's gotten a lot better recently with this big upgrade or price deduction and availability of higher ones.

08:51 So if you're thinking about checking them out, go up there at do.co slash Python, help support the show.

08:58 And also, if you already are a customer, consider upgrading your servers and stuff you have, because you're still paying the $10 price for half the thing.

09:08 So if you're willing to take it down for two minutes and upgrade it, you'll get something better for very, very little work.

09:12 So check them out.

09:14 Definitely good stuff.

09:15 Thanks, DigitalOcean, for supporting the show.

09:16 So we talked a while ago about Python, CPython in particular, moving to GitHub, right?

09:24 Yes.

09:24 I can't remember exactly when that was, but it was in 2016.

09:27 So it must have been one of our early episodes.

09:30 Can you remember when?

09:31 We could probably look it up, but I don't remember.

09:34 But so what I wanted to feature next was there's this post.

09:39 And I don't remember if it was actually just on Reddit or if it's actually, yeah, it's just on Reddit.

09:46 It's literally a picture on Reddit.

09:48 But the conversation is interesting and the picture is interesting.

09:51 And the title is The Result of Moving Python to GitHub.

09:55 And so they put this graph up there, which the graph is not super well labeled.

10:01 So it's kind of hard to tell, but it looks like there's a lot more activity since Python moved to GitHub from Mercurial, right?

10:09 And they do call out a couple of numbers saying, quick couple of numbers, including PRs from 2017 compared to 2016.

10:16 The number of commits increased by 190%.

10:21 Wow.

10:21 That's a big deal, right?

10:22 The inserted lines of code have increased by 140%.

10:26 The number of unique contributors, and this is probably the biggest, actually, this is probably the biggest thing right here.

10:32 And I think this is the biggest reason to go to GitHub.

10:34 The number of unique contributors has increased by 13 times.

10:39 That's wonderful.

10:40 Isn't that crazy?

10:41 Yeah.

10:41 Our new Python releases that come out are like there's way more stuff in them now than there used to be.

10:47 Yeah, that's for sure.

10:48 This is really cool.

10:49 And finally, not just new or unique contributors, but we now have more returning contributors.

10:55 There are nine times as many returning contributors as previously.

10:58 So people tried it and they liked it.

11:00 So that was cool.

11:01 I guess that's really the news, right?

11:03 You can see the picture.

11:04 The picture is actually not that valuable, but those numbers are.

11:07 And the conversation under it is more of a conversation of like, what is the value of statistics?

11:14 So, you know, that's an interesting conversation to have on the internet around graphs.

11:19 But one comment was when somebody said, oh, I would like them to move to GitLab instead.

11:25 But this is still good news.

11:25 What do you think?

11:26 Do you think we'd have this 13 times the number of contributors if it was on GitLab?

11:30 I don't think so on GitLab.

11:32 Although I use both GitHub and GitLab and they're both great products.

11:35 But the GitHub is definitely more of a social place to have more contributors.

11:42 I tend to agree.

11:43 I think GitLab is great.

11:44 I think also places like Bitbucket are great.

11:47 But they just don't have that social momentum.

11:51 It's like, well, I could have posted the question on Stack Overflow.

11:54 But there's another forum that some developers go to.

11:56 So I put it over there.

11:58 It's like, well, you can.

11:59 But it's probably not going to have the same reach and effect.

12:02 Right.

12:02 So I think it's really great.

12:04 And, you know, really, thanks to Brett Cannon, who spearheaded this, one of the core developers,

12:09 who got Python over there in the first place.

12:12 Yeah.

12:13 That's pretty cool.

12:14 Yeah, yeah, definitely.

12:14 So as developers, it's easy for us to have like kind of a bad self-image.

12:19 Look around and go, oh, all these people are so smart.

12:22 I'm not very good.

12:23 I'm stuck.

12:24 Like, I think two days ago, I was more frustrated than I have been in like probably months trying to get something to work.

12:31 I was really frustrated.

12:32 So you've got an article kind of around this, right?

12:34 Actually, somebody posted an article called Self-Deprecation Needs to Stop.

12:39 And really, this was Maurice Hayward, and it was inspired by some tweets by Stephanie Hurlbert.

12:45 And the comments apparently started with she was trying to get more women to blog about what they're doing, their technology stuff,

12:54 and that she would retweet the blog posts.

12:57 And many of the ones that she was told about were comments like, well, this project isn't that good, but here it is anyway.

13:05 And I guess I'm paying attention to this too because there's a fine line between like being honest about what your skill level is and how you can help,

13:14 and also trying not to be arrogant and act like an expert when you're not.

13:19 But there's, I mean, if you spent some time working on something and learning it, there are people out there that know less than you.

13:26 And I think people need to be, I like this article because people need to be better about understanding that they have learned a lot and they can contribute.

13:34 This also resonated with me because as a parent, I really try to stop my kids from self-deprecation speech and not say things like, well, I'm dumb, this was dumb.

13:45 Because it can give you a negative self-image and that's bad.

13:49 And we do see it a lot in the tech field.

13:52 I think partly because we are, we like to solve problems.

13:55 So we're always looking at people that know the answer to our problems who clearly know more than us because they've already figured it out.

14:02 So this is good.

14:03 Yeah, it's definitely good.

14:05 And no matter how experienced you are, you can find something that will frustrate you and make you feel stupid in tech and programming.

14:11 I'm sure of it.

14:12 This was repeated in the article, but it was from Stephanie.

14:14 He has really like three points, which I numbered them one, three, four, which is interesting.

14:21 I'll have to fix that.

14:22 But first off, stop saying words like my project is not good.

14:26 It's just a thing I wrote or it's just by a newbie or it's silly or it's not useful.

14:31 Just share it and let others be the judge.

14:33 The second point is really think about the value you bring and let everybody know what that is.

14:38 This is advertising for yourself and your public image also might reflect on a job interview sometimes.

14:45 So don't be too hard on yourself.

14:46 And also just like be proud of the things that you've done.

14:50 It's completely okay to say, man, I did this thing.

14:53 It was really cool.

14:54 Here it is.

14:55 Yeah, I think that's great advice.

14:56 People should definitely put what they do out there and just let people know what they're up to.

15:01 Very nice.

15:03 All right.

15:03 So it wouldn't be a Python byte if we didn't somehow feature our friend Anthony Shaw's work, would it?

15:09 Yeah.

15:10 It seems like he is so prolific and doing such cool stuff.

15:14 The last item I want to cover is something called five speed improvements to Python 3.7.

15:19 Oh, this is nice.

15:20 Yeah.

15:20 Yeah.

15:20 Yeah.

15:20 So Anthony's digging deep into Python 3.7 for a project that he's working on.

15:25 And I'm sure he'll talk about that later.

15:27 But he's been trying to explore this through blogging and so on.

15:31 And so this is a really nice article for a couple of reasons.

15:35 One, it's nice to call out the things that are going to be faster in June.

15:38 That's great.

15:39 When Python 3.7 comes out.

15:41 It's also cool because you see him doing a lot of internal introspection and looking into Python, like with the disassembler and the opcodes and seeing how different methods actually turn into operations in the CPython interpreter, which is really cool.

15:59 So let's talk about the five things.

16:01 So one is calling methods will be faster, kind of maybe, sometimes, in Python 3.7.

16:07 Okay.

16:07 Okay.

16:07 So the reason is there's actually two new opcodes that are being added.

16:12 And under certain circumstances, these can either remove the slowness of methods, functions tied to classes, or generally other ones.

16:25 I think if you have a bound method with no arguments, it's going to be faster than it was before.

16:30 So he has some numbers.

16:32 I don't remember exactly what the numbers on this one were.

16:34 String.find is faster for some characters by a lot.

16:38 So some of the time you're working with standard ASCII characters and everything is fine on string.find.

16:45 But for some reason, the way Unicode is laid out, there's certain types of characters that can make it up to 25 times slower to call string.find based on the input.

16:56 That's crazy, right?

16:57 Yeah.

16:58 Yeah.

16:58 So they said these are still slower than standard characters in Python 3.7.

17:05 But now they're only three times as slow, not 25 times as slow.

17:09 So that's still a really good speed up, right?

17:11 Close to 8x.

17:13 That's good.

17:13 Let's see.

17:14 Really quickly, os.fwalk, which creates like a tree directory structure type thing, which you can point it at a folder and it'll create like a subtree of files and folders.

17:24 It's now twice as fast.

17:25 Regular expressions are faster, but only in like a weird way.

17:30 Like they're less, they were changed in Python 3.6 basically for, I don't actually know why they were changed, but they were changed in Python 3.6 and that slowed them down.

17:41 So now this fix kind of puts them almost back to as fast as they were in 3.5.

17:46 So take that for what it is.

17:48 Okay.

17:48 And finally, regular expressions that are case insensitive are now 20 times faster.

17:53 Oh, awesome.

17:54 That's really good.

17:55 So basically the internal implementation of that, it used to call lower on every character.

18:00 which I guess was slow.

18:01 So now it can just like work on them in a different way, which is great.

18:05 How did I not know about fwalk before?

18:07 Yeah.

18:07 I got to look, that looks neat.

18:08 It does look neat, right?

18:10 Like I've traditionally used like recursion and stuff like that to deal with this, but yeah, I don't think you need it.

18:17 You have to check that out.

18:17 Maybe you still do, but it definitely, it returns like a tuple with four items with like the directory, the subdirectories, the files, things like that.

18:24 It's pretty cool.

18:25 So does he have like some, I haven't read this article.

18:28 Is there, are there like a benchmark of a, you know, a little application that runs in Python 3.6 versus 3.7?

18:35 That is a good question.

18:36 I did not see that.

18:38 It may be in there.

18:39 He was just quoting the numbers of his testing, but I don't know if he actually had a demo.

18:44 He may, and I just didn't catch it.

18:45 Okay, cool.

18:46 All right, Brian.

18:46 That's it for our news this week.

18:48 But you've got some follow-up stuff.

18:50 First of all, we talked about your PiTest, PiCharm webinar, and that was in the past at this point.

18:57 But the recording just came out, didn't it?

18:59 It did just come up.

19:00 And so I went back in this last week.

19:03 I listened to it.

19:04 And there's a couple things that I noticed, especially with respect to this, you know, don't be hard on yourself and self-deprecating speech.

19:13 Because my memory of it was I felt really embarrassed about the times where I didn't know the answer because people are asking in real time, and I don't want to, like, you know, go and look it up or something.

19:23 You're on the spot.

19:24 Hundreds of people are there, and it's being recorded.

19:26 Yeah.

19:27 So it just gave me, like, and then I listened back to it.

19:30 It's really not that bad.

19:31 And so there are some pauses that I have there, which are fine.

19:35 But I think I'm going to try to be easier on myself.

19:38 I think it would be a useful thing to watch if people aren't familiar with pytest or using it in PyCharm.

19:44 But the other thing was also just on my to-do list is try to come up with better ways to, or some way, to say, you know what?

19:53 I don't know the answer to that right off the bat.

19:55 I'll get back to you.

19:56 Or maybe that's it.

19:57 Yeah.

19:58 I don't know the answer.

19:59 I have no.

19:59 Yeah.

19:59 I mean, the last one I did, people asked me about SOLR, S-O-L-R, which is a full-text search type of thing I've learned since then.

20:08 They're like, how does this work with SOLR?

20:09 I have no idea what you're talking about.

20:11 I'm sorry.

20:11 Yeah.

20:12 Yeah, it's okay.

20:14 We don't have to be experts on everything.

20:16 The other thing I wanted to bring up is there's a, I don't think it really deserved an entire story,

20:21 but the PyPA has put up a post that says they'd like help from package maintainers to test out the new PyPI

20:30 because they've gotten funding to do some fixes and they want to see how they work.

20:35 Oh, that's really cool.

20:36 And a couple of cool things about it is I think people should go help them out.

20:39 But the link that I put up, which apparently is broken, I'll fix it by the time this goes out.

20:44 If you're asking people for help in beta testing something, this format that they put up is a really nice format to kind of use of,

20:52 these are the kind of things that we'd like you to try out, some of the workflows and let us know and how to let us know.

20:58 So there's a good template.

20:59 Here's 10 things we would love for you to test.

21:01 Here's how you check on security.

21:03 And here's how we have live chat hours if you find something you want to talk to us about.

21:07 That's great.

21:07 All right.

21:08 Well, I don't really have too much news.

21:10 I've been working on a whole bunch of projects, but nothing to announce at this point.

21:16 So I just have to keep those under wraps for a little while.

21:19 All right.

21:19 All right, Brian.

21:20 Thanks for finding all these things.

21:21 Like I said, every week, these are just so much fun to explore them with you.

21:24 So I appreciate you being here.

21:25 I appreciate it also.

21:26 Thanks.

21:27 Yep.

21:27 Thanks and bye, everyone.

21:28 Thank you for listening to Python Bytes.

21:32 Follow the show on Twitter via at Python Bytes.

21:35 That's Python Bytes as in B-Y-T-E-S.

21:38 And get the full show notes at pythonbytes.fm.

21:41 If you have a news item you want featured, just visit pythonbytes.fm and send it our way.

21:45 We're always on the lookout for sharing something cool.

21:49 On behalf of myself and Brian Okken, this is Michael Kennedy.

21:52 Thank you for listening and sharing this podcast with your friends and colleagues.