Transcript #121: python2 becomes self-aware, enters fifth stage of grief

00:00 Hello and welcome to Python Bytes where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 121, recorded March 13th, 2019.

00:10 I'm Michael Kennedy.

00:10 And I'm Brian Okken.

00:11 And this episode is brought to you by Datadog.

00:13 Check them out at pythonbytes.fm/datadog.

00:16 More on that later.

00:17 Brian, how you been?

00:18 I'm... well, actually I had a rough week last week, but I think I'm getting back to it.

00:22 What, did you get in a fight with a dentist?

00:24 Yeah, I went to like two dentists and two doctors just to fix one tooth.

00:28 But yeah, it's all done now.

00:29 - Well, glad to hear you're doing better at least.

00:32 You know, I would say the future is looking bright.

00:34 What do you think?

00:35 - Yeah, especially with Python 3 coming everywhere.

00:38 - Yeah, absolutely.

00:39 All right, so what's your first item here?

00:40 - Well, the first item is, oh yeah, you were doing like a whole transition thing.

00:45 So yeah, I get it now.

00:47 Futurize and auto-futurize.

00:50 So, Futurize is, there's a website called pythonfuture.org, and they have this thing called Futurize that you can turn it on in stages.

00:59 So it has a whole bunch of different modular switches.

01:02 And the idea is to automate your conversion from Python 2 to Python 3.

01:06 - So like one thing you can do is like fix the print statement, but nothing else, things like that?

01:11 Is that the idea? - Yeah.

01:12 There's a whole bunch of different little modular things you can do.

01:15 And I don't, I actually didn't dig into it enough to know if you can really, really go really small and granular and only change a few things at once.

01:23 But one of the things they do have is a staged conversion.

01:26 So they have split up a bunch of their stuff into relatively safe fixes.

01:31 Things like changing the exception syntax using print function.

01:36 So the print function is easy to change.

01:38 Making sure that your classes have an object-based class.

01:42 And apparently the iterator syntax changed, but I didn't realize that.

01:47 And a handful of other things that are really safe to just change.

01:50 So there's a stage one that just changes all that stuff.

01:53 and then the stage 2 where there's some more risky items possibly they do more, kind of wrap some of the Python 2 style stuff with Python 3 style code and you know of course all of this would only be for large projects because small projects, just go do it yourself man anyway, if you're going to try to do all these modular things it'd be kind of neat if you could test between hand and save all your changes Timothy Hopper put together something called auto-futurize that is a shell script that uses git to save all your changes and then talks to run your is assuming you're using talks to check your code and it it goes and go ahead because it does all of these changes to your code and test it in between and if it does a little bit of a change and your tests pass it does the rest of and then it doesn't check in things that don't work.

02:48 So this is kind of fun and it'd be fun to try on a project.

02:52 - Yeah, I like the idea of both of these.

02:53 I can totally envision some high paid consultants sitting down to do the Python 2 conversion for the companies that put their head in the sand until January and then they come in and they just run this for the morning, go have a long lunch, come back.

03:10 Everyone will be super happy how quickly they did it and it'll be great.

03:15 The machine will do all the conversion.

03:16 - Yeah, there'll be like two files that they have to hand edit to fix.

03:20 - Exactly.

03:21 It's gonna take a few weeks, but we'll see if we can get it done without too much of a problem.

03:25 (laughing)

03:26 - Hey, that's a new business idea.

03:28 Maybe I should start another side business.

03:30 - It's a little shady, but you know, it's all right.

03:33 - So thanks, Tim, for Christmas money for next year.

03:36 - Yeah, thank you, Tim.

03:36 And that's a cool, actually, in all seriousness, it's a cool project, and I think it's gonna help a lot of folks.

03:41 Speaking of helping folks, Anthony Shaw, he's putting a lot of content out there for everyone.

03:45 And the reason we talk about him so often is often he's doing some kind of blog post, or he's written some kind of cool article.

03:53 And he went kind of meta recently and did a live stream, I think it was on YouTube, of how he goes about writing these posts, how he does research, how he thinks about the different formats, all that kind of stuff.

04:05 - Yeah, so I was just, it's still on my to-do list to watch.

04:08 So you watched it and it was pretty good?

04:09 - I watched, yeah, about 20 minutes of it.

04:10 I didn't have a chance to watch the whole thing, but it was good.

04:14 Him just writing a post and you just watch him write it.

04:16 He goes through and talks about the different posts he's written, some of the trade-offs and how he works and things like that.

04:22 So he talks about one of his posts called "Modifying the Python Language in Six Minutes," which is like a deep code-focused article.

04:29 He talks about listicles.

04:31 And so he talks about five easy coding projects he did with kids he did.

04:35 He talks about what is popular and why those are popular, which is pretty interesting.

04:40 He says one type of article he likes to write is like question articles.

04:44 So why is Python slow, for example, and of course to disprove that or whatever.

04:48 And then also the tourist, like I'm gonna take you on a tour or something.

04:51 So tourist guide to the CPython source code.

04:53 So if you're looking to up your writing game or get inspired, if you sit down and you're like, oh, I gotta write about something, what am I gonna do?

04:58 Well, here's a bunch of ideas like, do I wanna write a listicle?

05:01 Do I wanna write a tour?

05:02 Do I wanna write a question?

05:03 And so it really helps focus the mind and get you going, I think.

05:06 - Yeah, definitely gonna check this out 'cause he's pretty much crushing it and I'd like to kind of learn from that technique.

05:12 - Yeah, very well done.

05:14 Nice job, Anthony.

05:15 I feel like you've chosen a very controversial topic for the next one.

05:20 Not so much, but pretty interesting.

05:22 - Yeah, I guess I just totally forgot that it was controversial.

05:25 And I know it was like big thing at the time, but it's the walrus operator or PEPS 572.

05:32 And I kind of love the walrus operator because it's the what, semicolon with the equal sign.

05:39 and it's basically an excitement expression that's coming in Python 3.8.

05:45 It's kind of hard to describe on audio, but it's a...

05:48 So a lot of times if you were to say, I do a lot with dictionaries or if I'm looking into some database or something, I want to find out if an element is there, and if it is there, I want the value.

05:59 And it's always this two-stage thing, and it'd be great, and pattern matching is another one, regular expressions. If there is a match, then search for it and get the stuff otherwise don't do something and being able to do this is one assignment expression is going to help out quite a bit anyway you can play with it now so that that's the whole point of this is alexander holder holder yes Wrote an article saying that now that the alphas some of the alpha releases for Python 3 8 are out You can try these out and he wrote a post demonstrating exactly what I was talking about getting elements out of a dick iterating through dictionaries and grabbing keys out of their elements if they if they exist he used pie e and V and To grab the new releases. I don't know why but I'm just one that I like to just download this stuff So I've added the links to the two, three, eight alphas available so far.

06:55 People want to try it out.

06:57 Yeah, the one that I really like here is the list expression, which could be a generator or it could be set expression or whatever, but a list comprehension because those have to be one line.

07:08 And if you want to assign a variable and then test it and do other stuff with it, you can't really do that.

07:15 And without the walrus operator, if you're going to like apply like a function to the elements that you're looping over and then you want to use those, you have to basically get access or compute that twice for each element and the walrus lets you do it all in one. So these little expressions are kind of like the lambda has the you know the colon and then the body type of syntax. It's a little bit like that kind of in line and that definitely seems nice to me. Yeah that's a pretty cool use. I hadn't really seen that before this article. I guess I didn't pay much attention but using it as a in a comprehension is pretty cool. Yeah cool yeah also you're little nested for loop if test.

07:52 It's pretty nice.

07:53 You know, I guess I'm coming around to the walrus operator, honestly.

07:56 Yeah, I mean, once I start using 3.8, I'll be using it all the time.

07:59 It'll be like using f strings, I think, to say, "Okay, well, now it's 3.8 and above because I don't want to use anything else." I'm not there yet, but...

08:08 Awesome.

08:09 All right, well, before we get on to the next one, which I'm pretty thrilled about because it's simple and amazing, I just want to tell you quickly about Datadog.

08:17 So they're sponsoring this episode.

08:18 Of course, Datadog is a monitoring and analytics service that brings all your metrics and logs and distributed traces together.

08:25 And their client automatically instruments things like async host libraries, such as async I/O and popular frameworks like Django or Tornado to help you visualize your performance.

08:36 If you want to trace your requests across service boundaries and figure out where your app is slow or find errors, you know, go over to pythonbytes.fm/datadog.

08:43 Got a cool free trial there and you get a nice little Datadog t-shirt too.

08:47 So check them out, it helps support the show.

08:50 Now, to the awesome thing.

08:53 We talked about Click and we talked recently about argpars and other cool ways to build command line applications.

09:00 And there's of course the tried and true input.

09:03 You got input, what is your question?

09:07 And you set the value and then you maybe have to test it, see whether it's an integer.

09:11 And then if there's an error, you say, no, there's an error.

09:13 you can't say that, you gotta say an integer, and there's all these challenges, right?

09:16 So maybe I've got a list of things I'd like them to pick from, like what kind of, maybe you're building a site and like through a generator, it says, well, what kind of data backend do you want?

09:25 I want Django ORM, I want SQLAlchemy, I want raw SQLite, whatever.

09:29 You might give them a list and say, well, pick one, two, three, or four, right?

09:32 Which box do you want?

09:34 Well, with Bullet, have you seen Bullet?

09:35 - I'm just looking at it right now and it's pretty exciting.

09:38 - It's incredible, right?

09:39 So what you get is, imagine like a dropdown combo box, like you would have on the web, that is the list and you can click, but in the command line, as a command line argument.

09:48 So it says choose something, and you can arrow through it, it has like a little indicator of which one you're on.

09:54 Oh man, it is slick.

09:55 - It supports colors and emojis and--

09:58 - It has a scroll bar.

09:59 It has a scroll bar.

10:02 You can like even scroll through them if there's too many choices to fit on the screen.

10:05 Yeah, it's pretty cool and it's easy to use, so that's nice.

10:09 There's also other types of inputs, like you can have a password, It says enter your password or what is the password, and it does the star marking.

10:15 - Wow, okay.

10:16 - Which is cool.

10:17 It has yes/no questions, numbers, it'll only, you know, a number input.

10:21 You can only enter numbers.

10:23 It won't accept, you know, junk that won't parse, so you don't have to go through all those steps.

10:28 It's pretty incredible, right?

10:31 - Yeah, so we need somebody to write cookie cutter with bullet prompts.

10:35 - Exactly, I was thinking of cookie cutter, exactly, 'cause cookie cutter asks all those types of questions all the time.

10:41 It would be beautiful to have Bullet just beautifying all these things.

10:45 So there's one idea if people want to contribute to open source.

10:48 Another one is right here on Bullet, there's probably other types of input besides passwords, yes, no, and numbers, like maybe multi-select, I don't know.

10:57 But also looking for Windows support.

10:59 I think right now this does not work on Windows 'cause the way the terminal works versus on a POSIX system.

11:05 So pretty cool though.

11:07 Still digging it.

11:08 So go Bullet.

11:09 - Go Bullet.

11:10 So we actually have a couple items to do with pip and packages and installing.

11:14 You go first.

11:15 Okay. Well, last week, I think it was last week, maybe it was the week before, we talked a little bit about maybe you want to try something different than Travis for your pipelines, for your continuous integration.

11:27 The Exodus after all the layoffs and all that business, huh?

11:30 Yeah, well, I mean, or just maybe you just want to try Azure anyway, or Azure just because it looks neat.

11:37 There's an article that I thought was super cool that was hosting private pip packages using Azure artifacts.

11:45 The idea of the article is, let's say if I'm using Azure pipelines to do actually a data pipeline, doing a whole bunch of different stages of manipulating changing the data or whatever, whatever you do with data pipelines.

11:58 Some of the work can be packaged up and used in later stages with just like a pip install, that'd be cool.

12:04 Then how do you do that?

12:06 There's a few gotchas that they get through.

12:09 And they, for instance, in one part, you can't just use the normal pipelines just by itself, but you can use the CI pipelines from Azure DevOps tool to get the packages into an artifact form.

12:23 And basically, it's all the hacks that you need to do to make this work.

12:27 I just thought that was cool.

12:28 Isn't it?

12:29 - Yeah, I think that sounds really cool as well.

12:30 And, you know, go ahead and leverage those artifacts to make your pipelines, which is cool.

12:35 I don't think you can make these public.

12:36 They're mostly, you know, you're using it for your own stuff.

12:40 So they're a little bit private, like they're talking about using credentials, hooking up your credentials so that pip install can get those correctly.

12:48 And then I also wanted, one of the things we did talk about last week was Anthony Shaw's pytest Azure Pipelines, pytest plugin.

12:57 We've already covered that last week, but one of the things that's cool that happened since then is that plugin is now part of the recommended pipeline, recommended setup for Microsoft itself.

13:09 - Yeah, that's pretty cool.

13:11 Well done, so it's definitely gonna be sort of the de facto way over there now.

13:15 So I wanna talk about something pretty unique.

13:17 Maybe we haven't touched on this before.

13:18 It has to do with GUIs.

13:19 So last week we spoke about mixing in the async IO, I know this was two weeks ago, the async IO pipeline sort of event loop and mixing that in with Qt.

13:34 So Qt for Python and PyQt and things like that.

13:36 So all GUIs have event loops, right?

13:39 They're always just going around and around, going, did something happen?

13:41 Did they click a button?

13:42 I'm looking for, you know, like mouse move.

13:44 I'm looking for key down, resize events, and it just, you know, passes it on to like the various event handlers.

13:51 Well, asyncIO is another loop that kind of goes around and around, process things, but like, how do we put those together?

13:57 So a guy named Annie Bolka sent us a message after hearing the first conversation about, what was it, I think it was AsyncQt, Qt was the name of the project.

14:08 After hearing about that, he's like, "Hey, I work on WXPython, "and there's a cool thing called WXAsync as well, "and it does a really similar thing, "and it basically merges those two event loops "into one thing, which is pretty cool." So he wrote a really nice, in-depth, medium article about it called Async and Await for WXPython.

14:29 - Oh, very nice.

14:30 - Yeah, yeah.

14:31 So one of the challenges with WX Python, apparently, is doing background work, right?

14:36 So you try to like call a service, or you try to do something, and then, you know, if you don't take extra measures, it's going to block up that event loop that is the GUI thread, and it's just like your app's gonna freeze and not respond and do all sorts of bad stuff that you're not supposed to do in desktop apps.

14:52 So with this, you can have async handlers that, you know, run in the background.

14:56 So if you're like waiting on a call to go to the service, you just await that and like the event loop keeps on chugging.

15:02 So it's pretty cool.

15:03 Obviously you can do threading and multiprocessing and that kind of stuff, but this is a little bit cleaner if you want to use async and await.

15:09 The other thing that he talks about in his article is about this sort of viral nature of using async and await.

15:17 So if I define a function, I say async def something, the function that's going to call it may well have to itself be async so it can await it.

15:26 and then like the things that call that have to become a sync of ID and so on.

15:31 It creates this kind of like from the bottom up spread of a sink, like invading all of your code, which that can be a challenge.

15:39 And it talks a little bit about that, but I would just want to throw a shout out, which is something I think we've spoken about before.

15:46 If I haven't, we should definitely make it a full featured item later, but unsync.

15:50 Have I mentioned unsync on the show?

15:52 I don't remember.

15:53 All right.

15:53 Well, that's probably going to be a new thing if we haven't.

15:55 I'll double check, but Unsync is a super simple library that puts a unifying API on multiprocessing, threading, and asynchronous methods.

16:06 And it manages its own little background queue running somewhere else on another thread.

16:11 It allows you to do a lot of things like just blocking on those running sub-method calls to get the result, so that you don't actually have the problem.

16:21 You can sort of use it to stop this async spread of these methods.

16:25 Anyway, it's an interesting article, it's interesting it covers that, and I wanted to throw it out there as one of the cool solutions for it.

16:32 - Yeah, that's a good thing to bring up.

16:34 I kind of didn't think about that.

16:36 I mean, I'm used to it in C++ world that if you're running multithreaded, everything's gotta be multithreadable.

16:42 - Yeah, it starts to go crazy, and it's just like, yeah, similarly.

16:45 - Yeah, anyway, okay.

16:46 - Pretty cool.

16:47 All right, well, that's it for our main items.

16:49 I have a couple of follow-up ones, and I have a set of jokes for you.

16:53 I have a joke generator for you, even.

16:54 How's that?

16:55 - That's great, yeah.

16:56 - All right, you got anything you wanna let people know about?

16:58 - No.

16:59 - So you just released a new testing code episode.

17:01 - Yeah, that released an episode talking about a fun alternative to, or version of TDD that's called TCR, test, commit, revert, with the idea of what happens if every time your tests fail, you just throw away all your code that you've changed since the last good commit.

17:21 And it seems absolutely crazy, but it's a fun thing to talk about.

17:24 - Yeah, it does seem a little crazy, but also, yeah, it seems fun.

17:27 - Yeah, and then there are a whole bunch of cool people lined up.

17:29 I've got like six interviews lined up, so there's gonna be lots of good content coming up soon.

17:34 - Yeah, sweet, sweet.

17:35 All right, well, I have a couple of things.

17:37 A correction, some self-aware acknowledgments, and then a joke.

17:42 So correction, we spoke about the MongoDB licensing and the AWS kerfuffle, where AWS said, well, you're gonna change your license, so we can't use MongoDB as our own service?

17:52 well fine we'll rewrite a new implementation that has nothing to do with you but is you know wire protocol identical and things like that.

17:59 Well mostly I got that right however will s sent me a message said hey actually posted a message on the comment section of that episode.

18:07 And said hey you said they switch to the AGPL they actually had already been on the AGPL what they switched to is something called the SS PL server side license.

18:19 And it turns out that the OSI Open Source Initiative doesn't even recognize SSPL as an open source license because of the way, some of its conditions.

18:31 And so it's interesting in some sense that like MongoDB might not even be open source anymore based on its license, not in the traditional what does its license mean sense anyway.

18:40 - Right, I took the perspective of, let's say I've got a service out there, I'm just like to have a side project where I've got a service out there where I just run my own instance of Mongo and I'm using that.

18:53 Do I have to worry about this license change?

18:55 And the answer is no.

18:56 - Yeah, that's what I saw as well.

18:57 It's like, but if you wanna be a cloud provider and you wanna offer the service to others, then you do.

19:02 - Right, but if the service you're offering isn't Mongo, it's just some other service that happens to use Mongo, yeah, you're good.

19:09 - Yeah, like businessinsider.com or whatever.

19:12 - Yeah.

19:13 - Yep, okay, so MongoDB license, sorry, it's SSPL.

19:17 That's the change, it was already a GPL.

19:19 Thank you, Will, for that.

19:21 So Brian, do you know that legacy Python is self-aware and has actually entered the fifth stage of grief?

19:27 (Brian laughs)

19:28 - No.

19:29 - So people out there can try this.

19:30 So if you type Python2-M and you run any module, so like Python2-M pip list, that's a meaningless command, you'll get the output that says, deprecation Python 2.7 will reach its end of life on January 1st, 2020.

19:47 please upgrade your Python as Python 2.7 won't be maintained after that date.

19:52 Moreover, my word, a future version of pip will drop support for Python 2.7.

19:57 - Oh wow, yeah, well that makes sense.

20:00 - I mean, I feel like that's acceptance, right?

20:02 It's like, yeah, I'm going away, sorry.

20:04 (laughing)

20:05 Here, there's a funny Twitter thread as well that I threw out when I first saw that going by, so that's cool.

20:10 And then finally, we talked about pip and packaging and stuff, there's this thing that Dan Bader from real python shot me over and said, hey, have you seen this?

20:18 This is kind of cool, called Pydist.

20:19 So this is a thing, a service, I guess, that is in beta.

20:24 And the idea is you can use it as your package source, right, for pip and whatnot, and you can have public and private dependencies there.

20:35 It'll mirror the public ones and keep a copy of them and never ever delete them, even if they were deleted from pip, PyPI, so that you always have them there stable.

20:45 It's pretty cool.

20:46 It's supposed to do a bunch of other stuff, like show you, like if you use this package, here are all the packages that depend upon it and what you're going to get, which is not really available right now, and things like that.

20:55 But it looks like it may be, at some point, become a paid service.

20:59 I don't know.

21:00 It's in free beta.

21:01 They don't say one way or the other.

21:02 So anyway, maybe that's helpful to people out there.

21:04 - Interesting though.

21:05 Cool.

21:06 - All right, you ready for a joke?

21:07 - Definitely.

21:07 - This is super hilarious.

21:08 All right, let's do this.

21:10 pip install --user pyjokes.

21:12 - Okay.

21:13 - All right, you gotta do it.

21:14 Or better, I'm super loving PipX, so PipX install PyJokes if you got that, because that is the business these days.

21:21 PipX is awesome.

21:23 It's like homebrew, but for Python executables.

21:25 - Okay, this is gonna take too long.

21:26 - All right, I'll do a couple, then you can give me one at the end.

21:29 So what this is, this is a package that you can install that gives you a command line access to developer jokes.

21:34 - Really? - Yeah, okay.

21:35 So once you install, I'm gonna type PyJoke.

21:37 There are two ways to write error-free programs.

21:40 Only the third one works.

21:42 Let's see what else we got.

21:43 This one, I think you'll like this one.

21:44 A QA engineer walks into a bar, runs into a bar, crawls into a bar, dances into a bar, tiptoes into a bar, rams a bar, jumps into a bar.

21:54 Okay.

21:55 That's it.

21:56 How many programmers does it take to change a light bulb?

21:59 None.

22:00 They just make darkness a standard.

22:03 Let's see.

22:05 A good programmer is someone who always looks both ways before crossing a one-way street.

22:11 (laughing)

22:12 See, these are, like, once you install PIPX and sell PyJokes, these are at your hand.

22:17 And you're like, I'm feeling a little down, what are we gonna do?

22:19 Why do Java programmers wear sunglasses?

22:21 Because they can't see sharp, huh?

22:23 So, like, it just doesn't stop, apparently.

22:26 (laughing)

22:28 - Well, we don't even have to have anybody submit any jokes for us anymore, we can just use these.

22:33 - We have the fountain of, the endless fountain of jokes.

22:35 Although I'm sure there is some limit to how many jokes are in there.

22:38 You can go to their website and actually submit a joke to be included in the app.

22:41 - Oh, okay.

22:42 - Which is all good.

22:43 - So I have to make sure I update it every once in a while.

22:45 - Yeah, that's why you need pipx, 'cause then you have to do pipx update all, and it does it.

22:49 - Okay, cool.

22:50 - Sweet, anyway, that's our joke.

22:52 If people can go out there and install PyJokes, the package, and run it.

22:56 Although they might not need us anymore, Brian.

22:58 - Yeah, they will, 'cause they'll need our wonderful transitions from one thing to another.

23:04 - That's right, that's right.

23:05 All right, well, a lot of fun as always, and I'll chat with you later.

23:09 - Okay, bye.

23:10 Thank you for listening to Python Bytes.

23:11 Follow the show on Twitter via @PythonBytes.

23:14 That's Python Bytes as in B-Y-T-E-S.

23:17 And get the full show notes at PythonBytes.fm.

23:20 If you have a news item you want featured, just visit PythonBytes.fm and send it our way.

23:25 We're always on the lookout for sharing something cool.

23:27 On behalf of myself and Brian Okken, this is Michael Kennedy.

23:30 Thank you for listening and sharing this podcast with your friends and colleagues.