Transcript #28: The meaning of _ in Python

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:06 It's Tuesday, May 30th, 2017, and this is episode 28 of Python Bytes.

00:12 And I'm Michael Kennedy.

00:13 And I'm Brian Okken.

00:14 Brian, can you believe that we've done 28 episodes already?

00:17 No, I can't. I was telling somebody about the show recently and I said, "Oh, we've only done like, oh my gosh, we've done like 27 episodes already." Yeah, we've done over half a year in the history of Python podcasts.

00:27 I think over half a year's worth, over 20 episodes, makes it like one of the longest running podcasts of all time, actually.

00:34 A lot of the other ones, they sort of cut it out early 20s, things like that.

00:38 >> Yeah, that's true.

00:39 >> Yeah, I mean, Talk Python and Podcasting.net, they've been going for a while, but still.

00:43 One of the primary rules that, or documents or enhancements that tells us how we should write our code is PEP 8, right?

00:51 >> Yeah, definitely everybody that's really into Python knows about PEP 8.

00:56 But sometimes when you mention, "Hey, you should follow Pep8" to somebody new, it's not really that easy to get them up to speed.

01:05 You know, it's not that hard to read.

01:06 But I didn't know about this thing called Pep8.org, and so I just stumbled across this recently.

01:13 And it's from Kenneth Reitz, and it's just an easy to use... it is just Pep8, the style guide.

01:22 It's a nice webpage with... it's styled nicely.

01:26 It's got red for bad examples and green for good examples.

01:29 And it's got a nice index on the left that you can click through.

01:32 - Yeah, it's definitely, yeah, it's a really nice and navigable thing.

01:35 And you're right, like the green do this, red don't do that.

01:39 And, you know, even little touches like the code samples that demonstrate good and bad, just basically any of the code samples that demonstrate anything are color-coded, which is like surprisingly delightful.

01:51 - Yeah, and I also like, because of those index there, They're all anchors in the webpage.

01:57 So you can take one of those and if you wanna mention a particular guideline to somebody, maybe a coworker that's not following it, you can email them that link and say, hey, could you read this?

02:08 - Exactly, yeah, put that in your issue response.

02:12 Like, I'm willing to take your PR, but hold on, you gotta fix this first.

02:16 - Yeah, but one of the things that I always wanna mention when I talk about Pep8 is don't be a Pep8 bully.

02:23 the first line of Pep8 talks about hobgoblins or something like that.

02:28 And basically-- - Yes, blindly following rules is the hobgoblins of little minds or something to this effect, yeah?

02:35 - Yeah, so somebody might have a decent reason for not following Pep8, so tread carefully when you try to get somebody to follow it.

02:44 If you haven't stumbled across Pep8 before, which is kind of hard to think that's possible if you're listening to this podcast, It does describe itself as being the style guide for the standard library for Python and for, but it's a lot more than that.

03:01 Most open source Python projects follow it to some degree, don't you think?

03:05 - Right, yeah, and you can set up linters that will like mention violations.

03:10 So for example, I think cookie cutter, if you do a PR to back to them and you do something like say write an 81 character line, it will fail the automated build and say, we're not going to accept it because it fails the flake eight or something to this effect.

03:27 - Right, okay.

03:28 - Yeah, I suspect everyone, pretty much everyone's heard of PEP 8, but I'm not sure everyone has actually taken the time to read through PEP 8.

03:35 And I think that this thing is more readable, so it's probably, if you're in that group, then go check it out, pep8.org.

03:41 - Okay, so what's next?

03:43 - I want to talk about asynchronous stuff.

03:45 Is it surprising?

03:46 I always talk about asynchronous stuff.

03:47 (laughing)

03:49 So there's this new project called TOKIO.

03:55 And at the heart of AsyncIO and Async and Await stuff are these things called Async event loops and whatnot.

04:03 And this is one of those.

04:05 We've previously spoken about Sanic and some of these faster web frameworks and their magic was that they run on top of this thing called UV loop.

04:14 And UV loop is I think implemented in C and is an alternate implementation of this internal async IO loop.

04:21 While tcoy is a similar concept but implemented in Rust.

04:27 So Rust is a compiled low level language such as, you know, like C. I mean, you can debate with this low level.

04:34 I feel like it is.

04:35 It's closer to the metal anyway.

04:36 It's a compiled language and it's supposed to be a little better for threading, a little better for memory management and safety.

04:44 So it's a little bit of a safer, more modern, low-level way to implement this.

04:49 I just thought it was kind of an interesting example that you could go check out and say like, "Here's integration of Python with Rust." Okay.

04:58 So it's a Python thing?

05:00 Yeah, so you use it from Python, but instead of having like a C extension, you have a Rust extension.

05:05 Oh, okay.

05:06 That makes sense.

05:07 Took me a while to catch up.

05:08 Yeah, yeah, yeah.

05:09 event loop is like written in C when you're looking at a UV loop, here it's written in Rust. And so here's like an interesting internal bit but you basically plug it into your Python code and then you can you know work with the async event loop like you normally would. Okay and I'm guessing it might be pronounced Tokyo but just guessing. Tokyo yeah yeah okay Tokyo you're probably right. You're probably right about that. So yeah cool and the project's still super early stage development. So if you're into Rust or you want to play with Rust and you would like a lot of people asking me like, "Hey, I want to do something open source. Is there an approachable small project?" This thing's pretty early days and so it's probably not too complicated to jump in and play around with if people are looking for some project to get involved with.

05:55 Well, that's great.

05:56 Yeah, absolutely. Tell me about some boilerplate.

05:59 Well, this is, I've got another project that is fairly early. I guess I was just assuming it's fairly early in the project, but I stumbled across... stumbled... I'm saying that a lot today.

06:09 I came across this uh a website called Python Boilerplate. It's python-boilerplate.com and it's addictively fun, at least right at first. I wouldn't try it on a phone. Actually, I haven't tried it on a phone, but it's uh it's got a a whole bunch of push buttons and stuff, and you can say which sort of boilerplate you want to have it spit out.

06:32 And you can select Python 2 even, or Python 3.

06:36 Luckily, it does default to Python 3.

06:39 And you can say whether or not you want an executable, whether it's an executable script, and whether or not you want arg parse stuff in there, which actually, somebody that wants to add arg parse stuff, it's kind of nice to have some built-in boilerplate of examples.

06:55 - Yeah, so it's a little bit like what Cookie Cutter does for large projects where you can say, I wanna generate kind of a starter code, a starter skeleton, but it's more like that, but for command line apps, CLI apps, right?

07:10 - Yeah, smaller little projects.

07:11 And it even like puts together a preliminary, like for instance, if you add Flask or pytest, for instance, it'll add your requirements.txt to it.

07:22 So it's a similar vein as Cookie Cutter, but a little more interactive and it's fun.

07:28 - Yeah, no, I like it.

07:29 People should check that out.

07:30 So that one's fun.

07:32 We recently went to PyCon, it was awesome, right?

07:34 - It was very awesome, yeah.

07:35 - Hopefully this year people will take us seriously when we say get your tickets before it sells out.

07:40 'Cause...

07:41 Anyway, one of the things that's really cool that I really liked from PyCon was the presentation that Instagram, the folks from Instagram did.

07:51 And luckily this is all on YouTube, so people can check it out.

07:54 So Instagram, they're probably not the biggest, but they're one of the biggest by traffic number of users or whatnot, consumers or users of Python for their web apps and their backend services.

08:06 Like they seriously use Python, they use Django and many, many places.

08:12 And they had found themselves in a bit of a situation, like many, many of our listeners probably find themselves, but into a degree that is way, way higher, right?

08:23 So they've got hundreds of millions of users, many, many, who knows how many thousand lines of Python, and they were stuck on Python 2.7 and Django, I think they said 1.3, which is years out of date.

08:38 It's quite out of date.

08:40 Like the current one is 1.8.

08:42 The one people are working on is 2.

08:44 The big news about Django 2 is it's only gonna support Python 3, right?

08:49 That was the big news there.

08:51 And so they said, "We somehow need to get onto Python 3." A lot of companies say, "We have this huge bit of code, and so, well, I'll throw my hands up in the air, we just couldn't possibly switch.

09:04 It's just too much work, it's too much risk." If you watch this presentation, there's almost no way you can look at this and go, "Oh, there's no way we can switch to Python 3 after you see what they went through." So it's really intense.

09:18 So for example, they don't believe in branches.

09:21 in their Git repo.

09:22 - Wow, I must have missed that.

09:23 That's interesting.

09:24 - Yeah, it's interesting, right?

09:26 Because I think to some degree rightly feel that the farther you are away from master or yeah, from master, what they're calling it, the more it's gonna be hard to integrate that back in and the more you could introduce trouble by merging that back in.

09:40 So what they do is they use feature switches.

09:43 So if you're working on a new feature, there's a config on your dev server that says enable this feature, but in production, it might be turned off.

09:51 So they're shipping, they said they check-in to the master branch quite often, like many times a day, and within an hour of that check-in, it's pushed to production, even when the features are not done, right?

10:04 So now they have this huge, huge branch of Python code where lots of the libraries and modules and stuff are shared across different services and different servers.

10:14 It runs on our old version of Django and Python 2.7, and they're like, "All right, What we're going to do is we're going to upgrade to Python 3, and we're not going to branch.

10:22 We're going to keep shipping on this main branch and convert it to Python 3.

10:27 We're going to upgrade like years worth of Django.

10:28 I'm going to keep shipping on Python to production and so on.

10:33 The process that they went through and the little steps that they took, and even for you, the testing procedures they went through to make this work was really, really interesting.

10:46 Basically I'll give you guys the summary if you want to watch the video it's highly recommended but it's really a concrete roadmap for every large project every large company there was lots of users and traffic to switch.

10:58 So they said they got between 12 and 40 percent performance improvement depending on where you look and like say the web tier versus the async queue tier and so on and I think like 30 percent better memory usage so just like straight across the board everything was better, and they haven't even started using the async IO stuff.

11:18 And they're super happy now that they can switch to a new version of Django.

11:22 Something cool comes out in terms of performance, say like Python 3.7, some function call becomes much faster, they just get that.

11:29 They don't have to like, oh, well, that'd be nice, right?

11:31 They basically, so they get to be part of the forefront of Python once again, and they're happy about it.

11:37 So then didn't they like start converting their code base so that it could run on Python 3 but they were still deploying on Python 2.7.

11:45 Yes, yes exactly.

11:47 And then eventually one of the steps they made was we're going to make all the developers run and develop on Python 3 but still deploy to Python 2.

11:57 And so that was one of the main like checks and balances was the developers will start hitting these bugs locally, will never push it out.

12:06 And then they didn't push it out to production fully then, they push it out to just Facebook employees and then they pushed it out to 0.1% of their users to see how that worked.

12:15 And then finally they started, they switched to like 20% and then like they pushed it all the way out, something like that.

12:20 Yeah.

12:21 So it was really interesting rollout.

12:22 And on their testing, they had like, at first there was a bunch of tests that just couldn't be fixed.

12:27 And so they set up a like, these are the tests that are included in the automated build.

12:33 And slowly as they migrated tests, they added them to the also include this so they would not regress back to be broken.

12:40 And they kind of chopped their way through that.

12:41 And then eventually they said, okay, the default is new tests all have to pass on Python 3.

12:45 And the old ones are going to go into this exclude list until we get the failing ones to zero.

12:49 And I really liked the presentation.

12:50 And I liked how they talked about actually a lot of the points that they went through.

12:54 And it's a lot, I think the example is a good example for any project.

12:59 But the, I also really liked the, just the reality that it's not trivial.

13:05 Yeah, no, it wasn't trivial.

13:06 It was like six months or four months or something huge.

13:09 But very doable and while in production.

13:11 That's pretty cool.

13:12 Yeah, and they also kept shipping new features, right?

13:14 That was one of their requirements.

13:15 Right.

13:16 So they shipped a bunch of new features at the same time.

13:18 I bet they use at least one underscore in their code when they did that.

13:21 Probably several underscores, yeah.

13:23 We've featured Dan Bader on the show a lot, but this is a pretty cool article.

13:28 The meaning of underscores in Python.

13:31 And again-

13:32 I think this falls into the realm of like what you said with PEP 8, right?

13:34 Like a lot of people are seeing these, but maybe they haven't really looked specifically at what it's about, right?

13:39 Yeah, and also he goes through pretty much all the different meanings, but one of the things that I, it took me a while to understand is that when people say dunder, they mean something with two underscores on both ends. Shortened of double underscore. So, it's a good list. He talks about the leading underscore meaning that it's sort of by convention private for, or internal use only.

14:04 But, and I added a note in our show notes that that doesn't apply to named tuple.

14:10 Named tuple has its functions starting with leading underscore, but you can still use them.

14:16 That's what they're for.

14:17 Yeah, nice.

14:18 Nice.

14:19 Yeah, I was just pairing with somebody today and said, what is class underscore mean?

14:22 Like, why is the underscore there?

14:24 Oh, yeah, right.

14:25 So if that's the trailing underscore, it's because somebody doesn't want to collide with a Python keyword or something.

14:34 - Yeah, exactly, like, it's class, so class_.

14:36 - Yeah, and I do like, I actually like that convention a lot better than having, like, my class, or this class, or something like that.

14:44 - I know, that really bugs me.

14:45 - Yeah, and one of the things that a lot of people don't, that I was surprised by when I first learned about it, is a single_by itself.

14:52 - Yes, that's one of my favorites.

14:53 - Yeah, and that's a I don't care variable that you can, like, if you have to assign a variable, something to a variable, but you're not gonna use it anymore or use it at all, that's a decent one.

15:03 And the warning, you don't get a warning for that if it's something assigned and never used.

15:08 - Exactly, it's meant to not be used.

15:10 Yeah, yeah, I mean, the couple of places I see that, like a tuple unpacking, when you don't care about all the values, it's kind of good.

15:17 If you say, I just wanna do a loop from one to 10, but I don't care about the number, I just wanna do it 10 times, you could say four underscore in range, you know, zero, 10, whatever, things like that.

15:28 - That's a good use for that.

15:29 - Yeah, yeah.

15:30 The other place that I really like this is where I have function signatures I have to match, but I don't care about the variables.

15:36 So for example, in the web methods, a lot of them take request as a parameter.

15:41 And if you don't put the request parameter there, like the framework will fail to like try to pass it and won't like match and it'll crash and all that.

15:47 But I don't care about calling the request variable or working with it.

15:51 So you'll get like a flake eight violation you're not using it.

15:54 So underscore.

15:55 - In that case, can you use it multiple times?

15:58 Like can you have multiple parameter names that are underscore?

16:01 - I think so, yeah.

16:02 you just like blast away the previous value stored in underscore.

16:05 But if you're really using it as like, I don't care.

16:07 Cool.

16:08 Okay.

16:09 Yeah, absolutely.

16:10 So the underscore is definitely good.

16:12 If you're not familiar with that, check out Dan's article.

16:14 It's a good one.

16:15 So let's close this out with a little bit of a look towards the future.

16:19 Okay.

16:20 And we talked about recently, the Stack Overflow tools, trends tool.

16:24 And when Stack Overflow released it, one of the things that they featured right up in the front was like language popularity for a few languages including Python.

16:33 It looks pretty good, right?

16:34 Yeah.

16:35 Yeah.

16:36 So I want to come back.

16:37 There's a guy who wrote a Medium post that actually dug into this quite a bit and played around with it and said, "The future is looking bright for Python." And basically took that tool and did a little better analysis with it than we did.

16:50 And there's one chart that they put up as a standard chart at Stack Overflow since we we covered it called the most popular languages trend chart.

16:59 And that's got like 15 languages or something still.

17:01 It's really completely overwhelming.

17:03 But if you hover over the language names, it highlights them and dims the others.

17:06 You can actually see it.

17:07 Okay, cool.

17:08 What's really interesting is if you go through that chart, and you hover over the languages, there are only like two languages that have upward trends.

17:17 And recently, and which one is the most steeply upward has the most positive derivative, probably Python.

17:25 It's of course it does actually by quite a quite a large margin and there's also a trend chart on there for Python 2 versus Python 3.

17:35 And last year Python 3 overtook Python 2.

17:37 Oh nice.

17:38 So just a little the same same kind of ideas with this trends but just digging in a little more detail and it's kind of interesting.

17:44 I thought it'd be fun to tie that together with the Instagram story so you can go back to your company your boss whatever and go into life 2020 for Python 2.

17:54 is no longer supporting Python 2.

17:56 Look, Python 3 is growing here.

17:58 Just like all the good stuff, right?

18:00 I'll bring it all together to help get a little more Python versus legacy Python in the world.

18:05 Yeah, and I definitely don't hear...

18:07 A couple years ago, we still heard people coming up with good reasons to stay on 2.7.

18:13 I don't hear anybody talking about that anymore.

18:16 Nope. Definitely, we've crossed over some sort of boundary, for sure.

18:21 Hopefully also in the future, it'll bring us at least another half year of Python Bytes.

18:27 I'm pretty sure we can at least get another half year of this.

18:30 Yeah.

18:31 So, it's fun.

18:32 That's all of our topics.

18:33 Any news for us, Michael?

18:35 No real news for me.

18:36 I'm just recording podcasts like crazy trying to get ahead for the summer.

18:40 So I have, like I was just telling you in the opening of the show before, hit record, like eight episodes of Talk Python to Me I'm recording over the next six, seven days.

18:49 So there's going to be a whole bunch of good stuff lining up, and I'm excited about what's coming.

18:53 How about you?

18:54 I'm kicking off some more recording also.

18:57 Definitely the book is still going strong, and doing a lot of work on that.

19:03 But I recorded an interview this morning, and I'll have at least a couple coming out in the next couple weeks.

19:08 So test and code is not dead.

19:10 It will come back.

19:11 What was the topic?

19:12 What did you record on today?

19:13 Actually, I recorded somebody that does mobile testing.

19:17 So mobile application and mobile and responsive web page testing.

19:21 And it's a company that offers a service that has, like, you can like rent a device.

19:27 Like, so somebody like tells you that that your, your website or your, your web application or something doesn't work on this particular device.

19:36 You don't have to go out and buy it.

19:38 Yeah.

19:38 Android Oreo, whatever, right?

19:41 Yeah.

19:41 It's on this.

19:42 Yeah.

19:42 You can just, you can go and rent it for like a, I don't know, I think, I think their prices are like 10 cents a minute or something like that.

19:49 But you can, I have no idea what their prices are actually.

19:52 I can't remember, but, but it's a, it's a commercial product, but I, I want to, I want to make sure that testing code isn't just about open source stuff.

19:59 It's also companies that are helping out that area.

20:02 So it was good.

20:03 Yeah.

20:03 It's cool to cover the whole, whole spectrum.

20:05 And can you imagine what their data center must be like with a thousand different phones?

20:09 Yeah.

20:10 And I, and I also asked him silly things like, okay, so I operate my phone with my finger do you have somebody are you paying somebody to move their finger or something and they said no they've got that all figured out automated so perfect perfect all right well Brian thanks for sharing the news this week and meeting with me and as always you know yeah thank you talk to you later you bet bye bye everyone thank you for listening to Python bites follow the show on Twitter via at Python bites that's Python bites as in B Y T E S And get the full show notes at PythonBytes.fm.

20:45 If you have a news item you want featured, just visit PythonBytes.fm and send it our way.

20:50 We're always on the lookout for sharing something cool.

20:52 On behalf of myself and Brian Okken, this is Michael Kennedy.

20:56 Thank you for listening and sharing this podcast with your friends and colleagues.