#121: python2 becomes self-aware, enters fifth stage of grief

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to

00:04 your earbuds. This is episode 121, recorded March 13th, 2019. I'm Michael Kennedy. And I'm

00:11 Brian Okken. And this episode is brought to you by Datadog. Check them out at pythonbytes.fm

00:14 slash Datadog. More on that later. Brian, how you been? I'm, well, actually I had a rough week

00:20 last week, but I think I'm getting back to it. Where'd you get in a fight with a dentist?

00:23 Yeah, I went to like two dentists and two doctors just to fix one tooth, but yeah,

00:29 it's all done now. Well, glad to hear you're doing better at least.

00:31 You know, I would say the future is looking bright. What do you think?

00:35 Yeah, especially with Python 3 coming everywhere.

00:37 Yeah, absolutely. All right. So what's your first item here?

00:40 Well, the first item is, oh yeah, you were doing like a whole transition thing. So yeah,

00:46 I get it now. Futurize and auto-futurize. So Futurize is, there's a website called pythonfuture.org

00:54 and they have this thing called Futurize that you can turn it on in stages. So it has a whole bunch of

01:00 different modular switches. And the idea is to automate your conversion from Python 2 to Python 3.

01:06 So like one thing you can do is like a fix the print statement, but nothing else, things like that?

01:11 Is that the idea?

01:12 Yeah. There's a whole bunch of different little modular things you can do. And I don't, I actually

01:16 didn't dig into it enough to know if you can really, really go really small and granular and only change

01:22 a few things at once. But one of the things they do have is a staged conversion. So they have split up

01:28 a bunch of their stuff into relatively safe fixes, things like changing the exception syntax using print

01:35 function. So the print function is easy to change, making sure that your classes have an object-based

01:41 class. And apparently the iterator syntax changed, but I apparently that I didn't realize that.

01:46 And a handful of other things that are really safe to just change. So there's a stage one that just

01:52 changes all that stuff. And then the stage two where there's some more risky items, possibly

01:58 they've do more kind of wrap some of the Python 2 style stuff with Python 3 style code. And, you know,

02:06 of course all of this would only be for large projects because small projects, just go do it yourself, man.

02:11 Anyway, if you're going to try to do all these modular things, it'd be kind of neat if you could test

02:17 between hand and save all your changes. Timothy Hopper put together something called

02:22 auto-futurize that is a shell script that uses Git to save all your changes and then talks to run your,

02:30 is assuming you're using talks to check your code. And it goes and go ahead, does all of these changes

02:38 to your code and test it in between. And if it does a little bit of a change and your tests pass,

02:44 it does the rest of it. And then it doesn't check in things that don't work. So this is kind of fun

02:49 and it'd be fun to try on a project. Yeah. I like the idea of both of these. I can totally envision

02:54 like some high paid consultants sitting down to do the Python 2 conversion for the companies that put

03:01 their head in the sand until January. And then they come in and they just, you know, run this for the

03:07 morning, go have a long lunch, come back. Everyone will be super happy how quickly they did it. And

03:14 it'll be great. The machine will do all the conversion. Yeah. There'll be like two files

03:18 that they have to hand edit to fix. And exactly. It's going to take a few weeks, but we'll see if

03:23 we can get it done without too much of a problem. Hey, that's a new business idea. Maybe I should

03:28 start another side business. It's a little shady, but you know, it's all right.

03:33 So thanks Tim for, for Christmas money for next year.

03:35 Yeah. Thank you, Tim. And that's a cool, actually, you know, in all seriousness,

03:38 it's a cool project and I think it's going to help a lot of folks.

03:40 Speaking of helping folks, Anthony Shaw, he's putting a lot of content out there for everyone.

03:45 And the reason we talk about him so often is often he's doing some kind of blog post,

03:51 he's written some kind of cool article. And he went kind of meta recently and did a live stream.

03:57 I think it was on YouTube of how he goes about writing these posts, how he does research,

04:02 how he thinks about the different formats, all that kind of stuff.

04:05 Yeah. So I was just, it's still on my to-do list to watch. So you watched it and it was pretty good.

04:09 I watched it. Yeah. About 20 minutes of it. Didn't have a chance to watch the whole thing,

04:12 but it was, it's good. Him just writing a post and you just watch him write it. He goes through

04:17 and talks about the different posts he's written, some of the trade-offs and how he works and things

04:22 like that. So he talks about one of his posts called modifying the Python language in six minutes,

04:26 which is like a deep code focused article. He talks about listicles. And so he talks about five

04:32 easy coding projects to do with kids. He did, he talks about what is popular and like why those

04:38 are popular, which is pretty interesting. He says one type of article he likes to write is like

04:43 question articles. So why is Python slow, for example? And of course to, you know, disprove that or

04:47 whatever. And then also the tourist, like I'm going to take you on a tour or something. So a tourist guide

04:51 to the CPython source code. So if you're looking to up your writing game or get inspired, you know,

04:56 if you sit down and you're like, Oh, we got to write about something. What am I going to do? Well,

04:59 here's a bunch of ideas like, do I want to write a listicle? Do I want to write a tour? Do you want

05:02 to write a question? And so it really helps focus the mind and get you going, I think.

05:06 Yeah, definitely going to check this out because he's pretty much crushing it. And I'd like to

05:10 kind of learn from that technique. So yeah, very, very well done. Nice job, Anthony. I feel like

05:16 you've chosen a very controversial topic for the next one. Not so much, but, but a pretty interesting.

05:21 Yeah, I guess I just totally forgot that it was controversial. And I know it was like big thing at the

05:28 time, but it's the walrus operator or peps 572. And I kind of love the walrus operator because

05:34 it's the, what semi or the colon with the equal sign. And it's basically excitement expression

05:42 that's coming in Python three eight. And it's kind of hard to describe on audio, but it's a,

05:48 so a lot of the times, if you were to say, I do a lot with dictionaries, or if I'm looking into

05:53 some data database or something, I want to find out if an element is there. And if it is there,

05:58 I want the value. And it's always this two stage thing. And it'd be great. And pattern matching is

06:04 another one, regular expressions. If there is a match, then search for it and get this stuff.

06:09 Otherwise don't do something. And, being able to do this as one assignment expression is going to,

06:15 I think, help out quite a bit. Anyway, you can play with it now. So that that's the whole point of this

06:20 is, Alexander Holtner. Yes. I wrote an article saying that now that the alphas,

06:26 some of the alpha releases for Python three, eight are out, you can try these out. And he wrote a post

06:31 demonstrating exactly what I was talking about, getting elements out of a dig, iterating through

06:36 dictionaries and grabbing keys out of their elements. If they, if they exist, he used pyenv and, to

06:45 grab the new releases. I don't know why, but I'm just one that I like to just download the stuff.

06:50 So I've added the links to the, the two, three, eight alphas available so far.

06:55 People want to try it out. So. Yeah. The one that I really like here is the

06:59 list expression, which could be a generator or it could be set expression or whatever, but

07:04 a list comprehension, because those have to be one line. And if you want to assign a variable and

07:10 then test it and then do other stuff with it, you can't really get really do that.

07:14 And without the walrus operator, if you're going to like apply like a function to the elements that

07:20 you're looping over and then you want to use those, you have to basically get access or compute that

07:26 twice for each element. And the walrus lets you do it all in one. So these little expressions are kind

07:30 of like the lambda has the, you know, the colon and then the body type of syntax. It's a little bit

07:35 like that kind of in line and that definitely seems nice to me. Yeah. That's a pretty cool use.

07:39 I hadn't, hadn't really seen that before this article. I guess I didn't pay much attention,

07:44 but using it as a, in a comprehension is pretty cool. Yeah. Cool. Yeah. Also your little nested

07:49 for loop if test, it's pretty nice. You know, I guess I'm coming around to the walrus operator,

07:55 honestly. Yeah. I mean, once I start using three, eight, I'll be using it all the time. It'd be like,

07:59 be like using f-strings, I think to say, okay, well now it's three, eight and above because I don't want to

08:05 use anything else. I'm not there yet, but awesome. All right. Well, before we get onto the next one,

08:12 which I'm pretty thrilled about, cause it's simple and amazing. I just want to tell you quickly about

08:16 Datadog. So they're sponsoring this episode. Of course, Datadog is a monitoring and analytics

08:21 service that brings all your metrics and logs and distributed traces together. And their client

08:26 automatically instruments things like asynchronous libraries, such as Async IO and popular frameworks

08:32 like Django or tornado to help you visualize your performance. If you want to trace your requests

08:37 across service boundaries and figure out where your app is slow or find errors, you know, go over to

08:41 pythonbytes.fm/Datadog. Got a cool free trial there and you get a nice little Datadog t-shirt too.

08:47 So check them out. It helps support the show. Now to the awesome thing. We talked about click and we

08:54 talked recently about argparse and other cool ways to build command line applications, right? Yeah.

08:59 And there's of course the tried and true input, right? You got input. What is your question? And

09:07 you set the value and then you maybe have to test it. We'll see whether it's an integer. And then like,

09:11 if there's an error, you say, no, there's an error. You can't say that you got to say an integer.

09:14 And there's all these, these challenges, right? So maybe I've got a list of things I'd like them to

09:19 pick from like what kind of, you know, maybe you're building a site and like through a generator says,

09:23 well, what kind of data backend do you want? I want Django ORM. I want SQLAlchemy. I want raw SQLite,

09:28 whatever. You might give them a list and say, well, pick one, two, three, or four, right?

09:32 Which box do you want? Well, with bullet, have you seen bullet?

09:35 I'm just looking at it right now and it's pretty exciting.

09:38 It's incredible, right? So what you get is imagine like a dropdown combo box,

09:42 like you would have on the web that is the list and you can click, but in the command line as a

09:47 command line or even, so it says choose something and you can arrow through it. It has like little

09:52 indicator of which one you're on. Oh man, it is slick. It supports colors and emojis and it has a

09:58 scroll bar. You can like even scroll through them if there's too many choices to fit on the screen.

10:05 Yeah. It's, it's pretty cool and it's easy to use. So that's nice. There's also other types of inputs.

10:11 Like you can have a password. It says enter your password or what is the password? And it does the

10:14 star marking, which is cool. It has yes, no questions and numbers. It only, you know, a number

10:20 input. You can only enter numbers. They won't accept, you know, junk that won't parse. So you

10:26 don't have to go through all those steps. it's pretty incredible, right? Yeah. So we need somebody

10:32 to write cookie cutter with, with bullet prompts. Exactly. I was thinking of cookie cutter.

10:36 Exactly. Cause the cookie cutter asks all those types of questions all the time. Yeah. It would be

10:41 beautiful to have bullet just, you know, beautifying all these things. So there's one idea people want

10:47 to contribute to open source. Another one is right here on bullet. you know, there's probably other

10:51 types of input besides passwords. Yes, no. And numbers like maybe multi-select. I don't know,

10:57 but also looking for windows support. I think right now this does not work on windows because the way

11:02 the terminal works versus on, you know, a POSIX system. So pretty cool though. Still, still digging it.

11:08 So go bullet. Go bullet. Yeah. So we actually have a couple of items to do with pip and packages and

11:13 installing. You go first. Okay. Well, last week, I think it was last week. Maybe it was the week

11:17 before we talked a little bit about, maybe you want to try something different than, Travis for your,

11:23 pipelines, for your continuous integration. The exodus after all the layoffs and all that

11:30 business, huh? Yeah. Well, I mean, or just maybe you're just wanting to try Azure anyway,

11:34 Azure just because it looks neat. There's an article that I thought was super cool. That was,

11:39 hosting private pip packages using Azure artifacts. And the idea is for the, of the article

11:48 is let's say if I'm using Azure pipelines to do actually a data pipeline, doing a whole bunch of

11:53 different stages of changing the data or whatever, whatever you do with data pipelines, some of the

11:58 work can be packaged up and used, in later stages with just like a pip install. That'd be

12:04 cool. And then how, how do you do that? There's a few gotchas that they get through and they, for

12:10 instance, in one part, you can't just use the normal pipelines just by itself, but you can use

12:16 the CI pipelines from Azure DevOps tool to get the packages into an artifact form. And basically it's

12:24 all the hacks that you need to do to make this work. I just thought that was cool. It was neat.

12:28 Yeah. I think that sounds really cool as well. And now go ahead and leverage those artifacts to,

12:33 to make your pipelines, which is cool.

12:35 I don't think you can make these public. They're mostly, you know, you're using it for your own

12:39 stuff. So they're a little bit private. Like they're talking about using a credential,

12:44 hooking up your credentials so that a pip install can get those correctly. and then I also wanted,

12:50 one of the things we did, did talk about last week was, Anthony Shaw's, pytest Azure

12:55 pipelines, pytest plugin. We've already covered that last week, but one of the things that's cool that

13:01 happened since then is that plugin has now part of the recommended pipeline recommended setup from

13:08 Microsoft itself. Yeah, that's pretty cool. Well done. So it's definitely going to be sort of the

13:13 de facto way over there now. Yeah. So I want to talk about something pretty unique. maybe we

13:17 haven't touched on this before. It has to do with GUIs. So, last week we spoke about mixing in

13:25 the asyncio. I know this was two weeks ago, the asyncio pipeline, sort of event loop and mixing that

13:32 in with cute. So cute for Python and PyQ and things like that. So all GUIs have event loops, right?

13:39 They're always just going around and around going, did something happen? Did they click a button? I'm

13:42 looking for, you know, like mouse move. I'm looking for key down, resize events. And it just, you know,

13:48 passes it on to like the various event handlers. Well, asyncio is another loop that kind of goes

13:54 around and around process those things, but like, how do we put those together? So a guy named Andy

13:59 Bolka sent us a message after hearing the first conversation about what was it? He was async

14:05 QT cute was the name of the project. After hearing about that, he's like, Hey, I work on WX Python.

14:11 And, you know, there's a cool thing called WX async as well. And it does a really similar thing.

14:17 And it basically merges those two event loops into one thing, which is pretty cool. So he wrote a

14:23 really nice in-depth medium article about it called async and await for WX Python.

14:28 Oh, very nice. Yeah. So one of the challenges with WX Python apparently is doing background work,

14:35 right? So you try to like call service or you try to do something. And then, you know,

14:39 if you don't take extra measures, it's going to block up that event loop. That is the GUI thread.

14:45 And it's just like your app's going to freeze and not respond and do all sorts of bad stuff. You're

14:50 not supposed to do in desktop apps. So with this, you can have async handlers that, you know,

14:55 run in the background. So if you're like waiting on a call to go to the service, you just await that.

15:00 And like the event loop keeps on chugging. So it's pretty cool. Obviously you can do threading

15:05 and multiprocessing and that kind of stuff, but this is a little bit cleaner. And if you want to

15:08 use async and await. Yeah. Nice. The other thing that he talks about in his article is about this

15:13 sort of viral nature of using async and await. So if I define a function, I say async def something,

15:20 the function that's going to call it may well have to itself be async so it can await it. And then like

15:27 the things that call that have to become asyncified and so on. It creates this kind of

15:32 like from the bottom up spread of async, like invading all of your code, which that can be a

15:39 challenge. And it talks a little bit about that, but I would just wanted to throw a shout out, which

15:44 to something I think we've spoken about before. If I haven't, we should definitely make it in full

15:48 featured item later, but unsync. Have I mentioned unsync on the show? I don't remember.

15:53 All right. Well, that's probably going to be a new thing if we haven't. I'll double check. But

15:56 unsync is a super simple library that puts a unifying API on multiprocessing, threading,

16:04 and asynchronous methods. And it manages its own little background queue running somewhere else on

16:10 another thread. It allows you to do a lot of things like just blocking on those running sub method

16:16 calls to get the result so that you don't actually have the problem. You can sort of use it to stop

16:22 this async spread of these methods. So anyway, you know, it's an interesting article. It's interesting

16:28 it covers that. And I want to throw unsync out there as one of the cool solutions for it.

16:32 Yeah. That's a good thing to bring up. I kind of didn't think about that. I mean, I'm used to it

16:37 in C++ world that if you're running multi-threaded, everything's got to be multi-threadable.

16:42 Yeah. It starts to go crazy. And it's just like, yeah, similarly. Yeah. Yeah. Anyway. Okay.

16:46 Pretty cool. All right. Well, that's it for our main items. I have a couple of follow-up ones and I

16:50 have a set of jokes for you. I have a joke generator for you even. How's that?

16:55 Oh, that's great. Yeah. All right. You got anything you want to let people know about?

16:58 No. So you just released a new testing code episode.

17:01 Yeah. That released an episode talking about a fun alternative to our version of TDD that's called

17:09 TCR test commit revert with the idea of what happens if every time your tests fail, you just throw away

17:18 all your code that you've changed since the last good commit. And it seems absolutely crazy, but it's a fun

17:23 thing to talk about. Yeah. It does seem a little crazy, but also, yeah, it seems fun. Yeah. And then

17:27 there are a whole bunch of cool people lined up. I've got like six interviews lined up. So there's

17:31 going to be lots of good content coming up soon. Yeah. Sweet, sweet. All right. Well, I have a couple

17:36 of things, a correction, some self-aware acknowledgements, and then a joke. So correction, we spoke about the

17:44 MongoDB licensing and the AWS kerfuffle where AWS said, well, you're going to change your license. So we

17:50 can't use MongoDB as our own service. Well, fine. We'll rewrite a new implementation that has nothing to do

17:56 with you, but is, you know, wire protocol identical and things like that. Well, mostly I got that right.

18:02 However, Will S sent me a message said, Hey, I actually posted a message on the comment section

18:07 of that episode and said, Hey, you said they switched to the AGPL. They actually had already been on the

18:12 AGPL. And what they switched to is something called the SSPL server side license. And it turns out that

18:21 the OSI open source initiative doesn't even recognize SSPL as an open source

18:26 license because the way some of its conditions. And so it's interesting in some sense that like

18:33 MongoDB might not even be open source anymore based on its license, not in the traditional,

18:38 what is its license mean sense anyway.

18:40 Right. I took the perspective of, let's say I've got a service out there. I'm just like a,

18:45 to have a side project where I've got a service out there where I just run my own instance of

18:49 Mongo and have amusing that. Do I have to worry about this license change? And the answer is no.

18:56 Yeah. That's what I saw as well. It's like, but if you want to be a cloud provider and you want to

19:00 offer the service to others, then you do.

19:02 Right. But if the service you're offering isn't Mongo, it's just some other service that happens to use

19:08 Mongo. Yeah. You're good.

19:09 Yeah. Like businessinsider.com or whatever.

19:12 Yeah.

19:14 Yep. Okay. So MongoDB license, sorry, it's SSPL. That's the change. It was already AGPL. Thank you,

19:19 Will, for that. So Brian, do you know that legacy Python is self-aware and has actually entered the

19:26 fifth stage of grief?

19:27 No.

19:29 So people out there can try this. So if you type Python 2 dash M and you run any module,

19:34 so like Python 2 dash M pip list, like that's a meaningless command. You'll get the output that

19:41 says deprecation. Python 2.7 will reach its end of life on January 1st, 2020. Please upgrade your

19:48 Python as Python 2.7 won't be maintained after that date. Moreover, my word, a future version of

19:55 pip will drop support for Python 2.7.

19:57 Oh, wow. Yeah. Well, that makes sense.

19:59 Yeah. So, I mean, I feel like that's acceptance, right? It's like, yeah, I'm going away. Sorry.

20:03 Anyway, there's a funny Twitter thread as well that I threw out when I first saw that going by. So that's cool.

20:10 And then finally, we talked about pip and packaging and stuff. There's this thing that Dan Bader from

20:14 RealPython shot me over and said, hey, have you seen this? This is kind of cool. Called PyDist.

20:19 So this is a thing, a service, I guess, that is in beta. And the idea is you can use it as your

20:28 package source, right, for pip and whatnot. And you can have public and private dependencies there.

20:35 It'll mirror the public ones and keep a copy of them and never, ever delete them, even if they were

20:40 deleted from pip, PyPI, so that you always have them there stable. It's pretty cool. It's supposed to do a

20:46 bunch of other stuff, like show you, like, if you use this package, here are all the packages that

20:50 depend upon it and what you're going to get, which is not really available right now and things like

20:55 that. But it looks like it maybe at some point become a paid service. I don't know. It's in free

21:00 beta. They don't say one way or the other. So anyway, maybe that's helpful to people out there.

21:04 Interesting though. Cool.

21:05 All right. You ready for a joke?

21:06 Definitely.

21:07 This is super hilarious. All right. Let's do this. pip install --user.

21:11 PyJokes.

21:12 Okay.

21:12 All right. You got to do it.

21:14 Or better. I'm super loving PipX. So PipX install PyJokes if you got that, because that

21:20 that is the business these days. PipX is awesome. It's like homebrew, but for Python executables.

21:25 Okay. This is going to take two then.

21:26 All right. Let me, I'll do a couple and then you can give me one at the end. So what this is,

21:30 this is a package that you can install that gives you a command line access to developer jokes.

21:34 Really?

21:35 Yeah. Okay. So once you install, I'm going to type PyJoke. There are two ways to write error-free

21:39 programs. Only the third one works. All right. Let's see what else we got. This one, I think

21:44 you'll like this one. A QA engineer walks into a bar, runs into a bar, crawls into a bar, dances

21:50 into a bar, tiptoes into a bar, rams a bar, jumps into a bar. Okay. That's it. How many programmers

21:57 does it take to change a light bulb? None. They just make darkness a standard. Let's see.

22:03 A good programmer is someone who always looks both ways before crossing a one-way street.

22:10 See, these are like, once you, once you install PipX and sell PyJokes, these are at your hand.

22:17 And you're like, I'm feeling a little down. What are we going to do? Why do Java programmers

22:20 wear sunglasses? Because they can't see sharp. Huh? So like there's, it just doesn't stop.

22:25 Apparently.

22:26 Well, we don't even have to have anybody submit any jokes for us anymore. We just use these.

22:33 We have the fountain of, the endless fountain of jokes. Although I'm sure there is some limit

22:36 to how many jokes are in there. You can go to their website and actually submit a joke to be

22:40 included in the app. Oh, okay. Which is all good. So I have to make sure I update it every once in a

22:45 moment. Yeah. That's why you need PipX because then you have to do PipX update all and it does it.

22:48 Okay. Cool. Sweet. Anyway, that's our joke. If people can go out there and install PyJokes,

22:54 the package and run it. Although they might not need us anymore, Brian. Yeah, they will. Because they'll need

23:02 our wonderful transitions from one thing to another. That's right. That's right. All right. Well,

23:06 a lot of fun as always and I'll chat with you later. Okay. Bye.

23:09 Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's

23:14 Python Bytes as in B-Y-T-E-S. And get the full show notes at pythonbytes.fm. If you have a news item

23:21 you want featured, just visit pythonbytes.fm and send it our way. We're always on the lookout for

23:25 sharing something cool. On behalf of myself and Brian Okken, this is Michael Kennedy. Thank you for

23:30 listening and sharing this podcast with your friends and colleagues.