Transcript #28: The meaning of _ in Python

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:06 It's Tuesday, May 30th, 2017, and this is episode 28 of Python Bytes.

00:12 And I'm Michael Kennedy.

00:13 And I'm Brian Okken.

00:14 Brian, can you believe that we've done 28 episodes already?

00:17 No, I can't. I was telling somebody about the show recently and I said, oh, we've only done like, oh my gosh, we've done like 27 episodes already.

00:24 Yeah, we've done over half a year in the history of Python Podcast.

00:27 I think over half a year's worth, over 20 episodes, makes it like one of the longest running podcasts of all time, actually.

00:34 Right? A lot of the other ones, they sort of cut out early 20s, you know, things like that.

00:38 Oh yeah, that's true.

00:39 Yeah, I mean, Talk Python and Podcasting, they've been going for a while, but still.

00:42 One of the primary rules that, or documents, or enhancements, that tells us how we should write our code is PEP 8, right?

00:50 Yeah, definitely everybody that's really into Python knows about PEP 8.

00:56 But sometimes when you mention, hey, you should follow PEP 8 to somebody new, it's not really that easy to get them up to speed.

01:04 You know, it's not that hard to read.

01:06 But I didn't know about this thing called PEP 8.org.

01:10 And so I just stumbled across this recently.

01:13 And it's from Kenneth Reitz.

01:15 And it's just an easy to use, it is the, just PEP 8, the style guide.

01:21 It's a nice webpage with, styled nicely.

01:25 It's got red for bad examples and green for good examples.

01:29 And it's got a nice index on the left that you can click through.

01:32 Yeah, it's definitely, yeah, it's a really nice and navigable thing.

01:35 And you're right, like, the green do this, red don't do that.

01:39 And, you know, even little touches like the code samples that demonstrate good and bad, just basically any of the code samples that demonstrate anything.

01:47 Are color-coded, which is, like, surprisingly delightful.

01:51 Yeah.

01:51 And I also like, because there's the index there, they're all anchors in the webpage.

01:57 So you can take one of those.

01:59 And if you want to mention a particular guideline to somebody, maybe a coworker that's not following it, you can email them that link and say, hey, could you read this?

02:08 Exactly.

02:09 Yeah, put that in your get issue response.

02:12 Like, I'm willing to take your PR, but hold on, you got to fix this first.

02:15 Yeah.

02:16 And, but one of the things that I always want to mention when I talk about Pep 8 is don't be a Pep 8 bully.

02:22 The first line of Pep 8 talks about hobgoblins or something like that.

02:27 And basically...

02:29 Yes.

02:30 Blondly following rules is the hobgoblins of little minds or something to this effect, yeah?

02:34 Yeah.

02:35 So somebody might have a decent reason for not following Pep 8.

02:40 So tread carefully when you try to get somebody to follow it.

02:44 If you haven't stumbled across Pep 8 before, which is kind of hard to think that's possible if you're listening to this podcast, it does describe itself as being the style guide for the standard library for Python.

02:57 But it's a lot more than that.

03:01 Most open source Python projects follow it to some degree, don't you think?

03:05 Right, yeah.

03:06 And you can set up linters that will, like, mention violations.

03:09 So, for example, I think cookie cutter, if you do a PR back to them and you do something like, say, write an 81 character line, it will fail the automated build and say we're not going to accept it because it fails Flake 8 or something to this effect.

03:27 Right.

03:27 Okay.

03:28 Yeah.

03:28 You know, I suspect everyone, pretty much everyone's heard of Pep 8, but I'm not sure everyone has actually taken the time to read through Pep 8.

03:35 And I think that this thing is more readable.

03:37 So it's probably, you know, if you're in that group, then go check it out, pepe.org.

03:41 Okay.

03:41 So what's next?

03:43 I want to talk about asynchronous stuff.

03:45 Is it surprising?

03:45 I always talk about asynchronous stuff.

03:48 So there's this new project called Tokio, I think is how you might say it, T-O-K-I-O.

03:54 And at the heart of AsyncIO and the Async and Await stuff are these things called Async Event Loops and whatnot.

04:02 And this is one of those.

04:05 We'd previously spoken about Sanic and some of these faster web frameworks.

04:09 And their magic was that they run on top of this thing called UV Loop.

04:13 And UV Loop is, I think, implemented in C and is an alternate implementation of this internal AsyncIO loop.

04:20 Well, Tokoy is a similar concept, but implemented in Rust.

04:25 So Rust is a compiled low-level language such as, you know, like C.

04:32 I mean, you can debate with this low-level.

04:34 I feel like it is.

04:35 It's closer to the metal anyway.

04:36 It's a compiled language.

04:38 And it's supposed to be a little better for threading, a little better for memory management and safety.

04:44 So it's a little bit of a safer, more modern, low-level way to implement this.

04:49 And I just thought it was kind of an interesting example that you could go check out and say, like, here's integration of Python with Rust.

04:57 Okay.

04:57 So it's a Python thing?

05:00 Yeah.

05:00 So you use it from Python.

05:01 But instead of having, like, a C extension, you have a Rust extension.

05:04 Oh, okay.

05:05 That makes sense.

05:05 Took me a while to catch up.

05:06 Yeah, yeah, yeah.

05:07 So, like, the heart of the asyncIO event loop is, like, written in C when you're looking at a UV loop.

05:14 Here it's written in Rust.

05:15 And so here's, like, an interesting internal bit.

05:18 But you basically plug it into your Python code, and then you can, you know, work with the async event loop like you normally would.

05:24 Okay.

05:24 And I'm guessing it might be pronounced Tokyo, but just guessing.

05:29 Tokyo?

05:29 Tokyo?

05:29 Yeah, yeah.

05:29 Okay.

05:30 Tokyo.

05:30 You're probably right.

05:31 You're probably right about that.

05:34 So, yeah, cool.

05:37 And the project's still super early stage development.

05:40 So if you're into Rust or you want to play with Rust and you would like it, a lot of people ask me, like, hey, I want to do something open source.

05:46 Is there an approachable small project?

05:48 This thing's pretty early days.

05:50 And so it's probably not too complicated to jump in and play around with if people are looking for some project to get involved with.

05:55 Well, that's great.

05:56 Yeah, absolutely.

05:57 Tell me about some boilerplate.

05:59 Well, this is – I've got another project that is fairly early.

06:02 I guess I was just assuming it's fairly early in the project.

06:05 But I stumbled across – stumbled.

06:08 I'm saying that a lot today.

06:09 I came across this – a website called Python Boilerplate.

06:13 It's python-boilerplate.com.

06:16 And it's addictively fun, at least right at first.

06:21 I wouldn't try it on a phone.

06:23 Actually, I haven't tried it on a phone.

06:25 But it's got a whole bunch of push buttons and stuff.

06:29 And you can say which sort of boilerplate you want to have it spit out.

06:33 And you can select Python 2 even or Python 3.

06:36 Luckily, it does default to Python 3.

06:39 And you can say whether or not you want an executable – whether it's an executable script or – and whether or not you want arg parse stuff in there.

06:49 Which actually, somebody that wants to add arg parse stuff, it's kind of nice to have some built-in boilerplate of examples.

06:55 Yeah.

06:56 So it's a little bit like what Cookie Cutter does for large projects where you can say, I want to generate kind of a starter code, a starter skeleton.

07:04 But it's more like that but for command line apps, CLI apps, right?

07:10 Yeah.

07:10 Smaller little projects.

07:11 And it even, like, puts together a preliminary – like, for instance, if you add Flask or pytest, for instance, it'll add your requirements.txt to it.

07:22 So it's a similar vein as Cookie Cutter but a little more interactive and it's fun.

07:28 Yeah, I know.

07:28 I like it.

07:29 People should check that out.

07:30 So that one's fun.

07:31 We recently went to PyCon.

07:33 It was awesome, right?

07:34 It was very awesome, yeah.

07:35 Hopefully this year people will take us seriously when we say, get your tickets before it sells out.

07:40 Because – anyway, one of the things that's really cool that I really liked from PyCon was the presentation that Instagram – the folks from Instagram did.

07:51 And luckily this is all on YouTube so people can check it out.

07:54 So Instagram, they're probably not the biggest but they're one of the biggest by traffic number of users and whatnot, consumers or users of Python for their web apps and their backend services.

08:06 Like, they seriously use Python.

08:09 They use Django in many, many places.

08:12 And they had found themselves in a bit of a situation like many, many of our listeners probably find themselves.

08:20 But to a degree that is way, way higher, right?

08:23 So they've got hundreds of millions of users.

08:26 Many, many, who knows how many thousand lines of Python.

08:30 And they were stuck on Python 2.7 and Django, I think they said 1.3, which is years out of date.

08:38 It's quite out of date.

08:40 Like, the current one is 1.8.

08:42 The one people are working on is 2.

08:44 The big news about Django 2 is it's only going to support Python 3, right?

08:49 That was the big news there.

08:50 And so they said, we somehow need to get onto Python 3, right?

08:57 A lot of companies say we have this huge bit of code.

09:00 And so, well, I'll throw my hands up in the air.

09:02 We just couldn't possibly switch, right?

09:04 It's just too much work.

09:05 It's too much risk.

09:06 If you watch this presentation, there's almost no way you can look at this and go,

09:12 oh, there's no way we can switch to Python 3 after you see what they went through.

09:16 So it's really intense.

09:18 So, for example, they don't believe in branches in their Git repo.

09:22 Wow, I must have missed that.

09:23 That's interesting.

09:24 Yeah.

09:24 It's interesting, right?

09:25 Because I think to some degree rightly feel that the farther you are away from master or, yeah, from master,

09:33 what they're calling it, the more it's going to be hard to integrate that back in.

09:38 And the more you could introduce trouble by merging that back in.

09:40 So what they do is they use feature switches.

09:43 So if you're working on a new feature, there's a config on your dev server that says,

09:47 enable this feature.

09:48 But in production, it might be turned off.

09:50 So they're shipping.

09:52 They said they check in to the master branch quite often, like many times a day.

09:59 And with an hour of that check-in, it's pushed to production, even when the features are not done.

10:03 Right?

10:04 So now they have this huge, huge branch of Python code where lots of the libraries and modules and stuff are shared across different services and different servers.

10:14 It runs on our old version of Django on Python 2.7.

10:17 And they're like, all right, what we're going to do is we're going to upgrade to Python 3.

10:20 And we're not going to branch.

10:22 We're going to keep shipping on this main branch and convert it to Python 3.

10:26 We're going to upgrade like years worth of Django.

10:29 I'm going to keep shipping on Python to production and so on.

10:33 And so the process that they went through and the little steps that they took, and even for you, the testing procedures they went through to make this work was really, really interesting.

10:44 So basically, I'll give you guys the summary.

10:47 If you want to watch the video, it's highly recommended.

10:49 But it's really a concrete roadmap for every large project, every large company with lots of users and traffic to switch.

10:57 So they said they got between 12% and 40% performance improvement, depending on where you look, like, say, the web tier versus the async queue tier and so on.

11:08 And I think like 30% better memory usage.

11:12 So just like straight across the board, everything was better.

11:14 And they haven't even started using the asyncio stuff.

11:18 And they're super happy now that they can switch to a new version of Django.

11:22 Something cool comes out in terms of performance, say, like Python 3.7.

11:26 Some function call becomes much faster.

11:28 They just get that.

11:29 They don't have to, like, go, well, that'd be nice, right?

11:31 They basically, so they get to be part of the forefront of Python once again, and they're happy about it.

11:36 So then they, like, start converting their code base so that it could run on Python 3.

11:41 But they were still deploying on Python 2.7.

11:44 Yes.

11:45 Yes, exactly.

11:46 And then eventually one of the changes, one of the steps they made was we're going to make all the developers run and develop on Python 3, but still deploy to Python 2.

11:57 And so that was one of the main, like, checks and balances was the developers will start hitting these bugs locally.

12:04 I will never push it out.

12:06 And then they didn't push it out to production fully then.

12:08 They pushed it out to just Facebook employees.

12:11 And then they pushed it out to 0.1% of their users.

12:14 Yeah.

12:14 To see how that worked.

12:15 And then finally they started, they switched to, like, 20% and then, like, they pushed it all the way out.

12:19 Something like that.

12:20 Yeah.

12:20 Yeah.

12:20 So it was a really interesting rollout.

12:22 And on their testing, they had, like, at first there was a bunch of tests that just couldn't be fixed.

12:27 And so they set up a, like, these are the tests that are included in the automated build.

12:32 And slowly as they migrated tests, they added them to also include this so they would not regress back to be broken.

12:38 Yeah.

12:38 And they kind of chopped their way through that.

12:41 And then eventually they said, okay, the default is new tests all have to pass on Python 3.

12:45 And the old ones are going to go into this exclude list until we get the failing ones to 0.

12:49 And I really liked the presentation.

12:51 And I liked how they talked about actually a lot of the points that they went through.

12:54 And it's a lot, I think the example is a good example for any project.

12:59 But the, I also, I really liked the, just the reality that it's not trivial.

13:04 Yeah.

13:05 No, it wasn't trivial.

13:06 It was like six months or four months or something huge.

13:09 Yeah.

13:09 But very doable and while in production.

13:11 That's pretty cool.

13:12 Yeah.

13:12 And they also kept shipping new features, right?

13:14 That was one of their requirements.

13:15 Right.

13:15 So they shipped a bunch of new features at the same time.

13:17 I bet they used at least one underscore in their code when they did that.

13:20 Probably several underscores.

13:22 Yeah.

13:22 We've featured Dan Bader on the show a lot, but this is a pretty cool article.

13:27 The meaning of underscores in Python.

13:30 And again, I think this falls into the realm of like what you said with PEP 8, right?

13:34 Like a lot of people are seeing these, but maybe they haven't really looked specifically at what it's about, right?

13:39 Yeah.

13:39 And also he goes through pretty much all the different meanings.

13:42 But one of the things that I, it took me a while to understand is that when people say dunder, they mean something with two underscores on both ends.

13:51 Yes.

13:51 Shortened of double underscore.

13:53 So it's a good list.

13:55 He talks about the leading underscore, meaning that it's sort of by convention, private for, or internal use only.

14:04 But, and I added a note in our show notes that that doesn't apply to named tuple.

14:10 Named tuple has its functions starting with leading underscore, but you can still use them.

14:15 That's what they're for.

14:16 Yeah.

14:16 Nice.

14:16 Nice.

14:17 Yeah.

14:17 I was just pairing with somebody today and said, what does class underscore mean?

14:22 Like why is the underscore there?

14:23 Oh yeah.

14:24 Right.

14:24 So if that's the trailing underscore, it's because somebody doesn't want to collide with a Python keyword or something.

14:34 Yeah, exactly.

14:35 Like it's class.

14:36 So class underscore.

14:36 Yeah.

14:37 And I do like, I actually like that convention a lot better than having like my class or this class or something like that.

14:44 I know that really.

14:45 Yeah.

14:46 And one of the things that a lot of people don't, that I was surprised by when I first learned about it is a single underscore by itself.

14:52 Yes.

14:52 That's my, one of my favorites.

14:53 And that's a, that's a, I don't care variable that you can, like, if you have to assign a variable, something to a variable, but you're not going to use it anymore or use it at all.

15:02 That's a decent one.

15:03 And the warning, you don't get a warning for that.

15:05 If it's a, something assigned and never used.

15:08 Exactly.

15:09 It's meant to not be used.

15:10 Yeah.

15:10 Yeah.

15:10 I mean, the couple of places I see that like a tuple unpacking, when you don't care about all the

15:15 values is kind of good.

15:17 If you say, I just want to do a loop from one to 10, but I don't care about the number.

15:22 I just want to do it 10 times.

15:23 You could say four underscore in range, you know, zero, 10, whatever, things like that.

15:28 That's a good use for that.

15:29 Yeah.

15:29 Yeah.

15:30 The other place that I really liked this is where I have function signatures.

15:33 I have to match, but I don't care about the variables.

15:35 So for example, in the web methods, a lot of them take request as a parameter.

15:40 And if you don't put the request parameter there, like the framework will fail to like try to pass it and won't like match and it'll crash and all that.

15:47 But I don't care about calling the request variable or working with it.

15:50 So you'll get like a flake eight violation.

15:53 You're not using it.

15:54 So underscore.

15:55 In that case, can you use it multiple times?

15:57 Like, can you have multiple parameter names that are underscore?

16:00 I think so.

16:01 Yeah.

16:01 I mean, you just like blast away the previous value stored in underscore, but if you're really using it as like, I don't care, then it's fine.

16:07 Cool.

16:07 Okay.

16:08 Yeah, absolutely.

16:09 So the underscore is definitely good.

16:11 If you're not familiar with that, check out Dan's article.

16:13 It's a good one.

16:14 So let's close this out with a little bit of a look towards the future.

16:18 Okay.

16:19 And we talked about recently the Stack Overflow tools, trends tool.

16:24 And when Stack Overflow released it, one of the things that they featured right up in the front was like language popularity for a few languages, including Python.

16:33 And it looked pretty good, right?

16:34 Yeah.

16:34 Yeah.

16:34 So I want to come back.

16:35 There's a guy who wrote a Medium post that actually dug into this quite a bit and played around with it and said, the future is looking bright for Python.

16:44 And basically took that tool and did a little better analysis with it than we did.

16:50 And there's one chart that they put up as a standard chart at Stack Overflow since we covered it called the most popular languages trend chart.

16:59 And that's got like 15 languages or something still.

17:01 It's really completely overwhelming.

17:03 But if you hover over the language names, it highlights them and dims the other so you can actually see it.

17:07 Okay.

17:07 Cool.

17:08 What's really interesting is if you go through that chart and you hover over the languages, there are only like two languages that have upward trends and recently.

17:18 And which one is the most steeply upward?

17:22 Has the most positive derivative?

17:24 Well, probably Python.

17:26 Of course it does.

17:27 Actually, by quite a large margin.

17:29 And there's also a trend chart on there for Python 2 versus Python 3.

17:34 And last year, Python 3 overtook Python 2.

17:37 Oh, nice.

17:38 So just a little, same kind of ideas with this trends, but just digging in a little more detail on it.

17:43 It's kind of interesting.

17:44 I thought it'd be fun to tie that together with the Instagram story so you can go back to your company, your boss, whatever, and go into life 2020 for Python 2.

17:53 Django is no longer supporting Python 2.

17:56 Look, the Python 3 is growing here.

17:58 Just like all the good stuff, right?

17:59 Bring it all together to help get a little more Python versus legacy Python in the world.

18:05 Yeah, and I definitely don't hear, a couple years ago, we still heard people coming up with good reasons to stay on 2.7.

18:13 And I don't hear anybody talking about that anymore.

18:15 Nope.

18:16 It's definitely, definitely, we've crossed over some sort of boundary, for sure.

18:21 Well, hopefully also in the future, it'll bring us at least another half year of Python bytes.

18:25 I'm pretty sure we can at least get another half year of this.

18:29 Yeah, so.

18:30 It's fun.

18:31 That's all of our topics.

18:33 Any news for us, Michael?

18:35 No real news for me.

18:36 I'm just recording podcasts like crazy, trying to get ahead for the summer.

18:40 So I have, like I was just telling you, the opening of the show before we hit record, like eight episodes of Talk Python to Me.

18:47 I'm recording over the next six, seven days.

18:49 So there's going to be a whole bunch of good stuff lining up, and I'm excited about what's coming.

18:53 How about you?

18:54 I'm kicking off some more recording also.

18:57 Definitely the book is still going strong.

18:59 Yeah.

18:59 And doing a lot of work on that.

19:03 But I recorded an interview this morning, and I'll have at least a couple coming out in the next couple weeks.

19:08 So Test and Code is not dead.

19:09 It will come back.

19:11 What was the topic?

19:12 What did you record on today?

19:13 Actually, I recorded somebody that does mobile testing.

19:17 So mobile application and mobile and responsive web page testing.

19:21 And it's a company that offers a service that has, like you can rent a device.

19:28 So somebody, like, tells you that your website or your web application or something doesn't work on this particular device.

19:36 You don't have to go out and buy it.

19:38 Yeah.

19:38 Android, Oreo, whatever, right?

19:41 Yeah.

19:41 It's on this?

19:42 Yeah.

19:42 You can go and rent it for, like, I think their prices are, like, 10 cents a minute or something like that.

19:49 But you can – I have no idea what their prices are, actually.

19:52 I can't remember.

19:52 But it's a commercial product, but I want to make sure that Test and Code isn't just about open source stuff.

19:59 It's also companies that are helping out that area.

20:02 So it's good.

20:03 Yeah.

20:03 It's cool to cover the whole spectrum.

20:05 And can you imagine what their data center must be like with a thousand different phones running in there?

20:10 Yeah.

20:10 And I also asked them silly things like, okay, so I operate my phone with my finger.

20:16 Do you have somebody – are you paying somebody to move their finger or something?

20:19 And they said, no, they've got that all figured out, automated.

20:23 So – Perfect.

20:25 Perfect.

20:25 All right.

20:26 Well, Brian, thanks for sharing the news this week and meeting with me.

20:29 And as always, you know.

20:30 Yeah.

20:30 Thank you.

20:31 Talk to you later.

20:32 You bet.

20:32 Bye.

20:32 Bye, everyone.

20:33 Thank you for listening to Python Bytes.

20:36 Follow the show on Twitter via at Python Bytes.

20:38 That's Python Bytes as in B-Y-T-E-S.

20:41 And get the full show notes at Pythonbytes.fm.

20:45 If you have a news item you want featured, just visit Pythonbytes.fm and send it our way.

20:49 We're always on the lookout for sharing something cool.

20:52 On behalf of myself and Brian Okken, this is Michael Kennedy.

20:55 Thank you for listening and sharing this podcast with your friends and colleagues.