Transcript #275: Airspeed velocity of an unladen astropy

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly This is episode 275, recorded March 15th.

00:10 And I am Brian Okken.

00:12 I am Michael Kennedy.

00:13 And I'm Emily Morehouse.

00:14 Yay, thanks for coming.

00:17 I also want to thank, give a shout out to Microsoft for Startup Founders Hub, and we'll learn more about them later in the show.

00:26 So welcome, Emily.

00:28 if people aren't familiar with who you are, who are you?

00:31 - Yeah, of course.

00:32 So I'm Emily, I am the Director of Engineering and one of the co-founders of CuddleSoft.

00:37 So we are a creative product development agency focused on web, mobile, IoT, and the cloud.

00:43 I'm also a Python core developer and the PyCon conference chair for this year.

00:49 - Awesome.

00:50 - That is awesome.

00:51 Said another way, you're quite busy.

00:53 - Yeah, I also have a 10 month old, so not a lot going on in my life.

00:57 - Yeah, no, a lot of time to binge watch Netflix.

01:00 - 10 months.

01:01 - So are you pretty busy for PyCon already?

01:05 - So interestingly enough, this is kind of a time that goes into autopilot in a way, you know, like most things are already set in motion that need to be set in motion.

01:13 So it's really, we're working on the fun stuff right now, like, you know, speaker and staff gifts and stuff like that.

01:20 But otherwise it's pretty smooth sailing and just sitting back and watching COVID numbers and hoping that we don't get another spike before April 9th.

01:28 - Yeah, fingers crossed.

01:30 This'll be the first PyCon after COVID hit.

01:34 So hopefully everything goes great.

01:35 I know people are excited.

01:36 - Yeah. - Yeah.

01:37 - Well, Brian, do I have the first one?

01:39 - You do. - Do we have to wait?

01:39 Do we have to wait for me or can I talk about this?

01:42 - Yeah, so I'll wait.

01:44 - Well, let's, oh, wait.

01:46 I'm very excited to talk about this one, actually.

01:47 This one comes from Frederick Averpill.

01:50 I believe listens to the show a lot.

01:51 So hello, Frederick, nice work on this.

01:54 I was working on the Python Bytes website of all things, and I wanted to do some stuff with like uploading MP3s and having a bunch of automation happen behind the scenes.

02:04 And one of the things I did not too long ago is switch over to an async database.

02:10 I think we talked about moving to Beanie and some of the cool stuff I'm doing to like halfway move to async, but not all the way yet, not till we're quite ready.

02:17 But as part of that, I'm like, well, all this file stuff, this is kind of slow, like this is a couple of seconds.

02:22 is there a way to sort of set the web server free while we're doing this work, right?

02:28 And some of that involved calling subprocesses.

02:31 I thought, well, maybe there's some third-party package like AIO files that I could use that would allow me to asynchronously do subprocesses instead of the subprocess module.

02:43 So I did a quick Google search and I came across Frederick's article here and much to my surprise, I don't know if you all are aware of this, but built in asyncio has async subprocess management already.

02:54 - That's pretty cool. - Isn't that cool?

02:56 - Yeah. - Yeah.

02:58 Emily, have you played with this any?

03:00 - Yeah, you know, I actually think I've used this exact blog post, which is super funny.

03:04 I was actually just recently writing like CLI regression test and pytest.

03:11 And you basically like test running two different servers.

03:14 And I was like fighting with subprocess to get it to work.

03:17 I don't think they were using a new enough version that I could use async await, but I definitely remember referencing this.

03:23 So, yeah.

03:24 - Yeah, very cool.

03:25 So you can just say async io, the module, dot create subprocess exec for just running it.

03:31 Or if you need to sort of follow along and see what's going on, you can use even the shell one that requires shell commands, like a CD or an LS type of thing.

03:41 And then you could just grab the standard out from set that to be async.io subprocess.pipe and so on, you can get the results out of it and everything.

03:51 So you just do things like await, creating a subprocess with a shell or execing it and so on.

03:58 Then you can await communicating with it, which I think is pretty cool and so on.

04:03 So not a whole lot more to say than other than if you're doing stuff with subprocess and you're moving already into other areas where async and await are very doable, Think FastAPI, SQL model, the SQLAlchemy 1.4 or later, where you're already doing a bunch of other async stuff, go ahead and write this in the async way so it just sort of flows into the rest of what you're doing.

04:27 - That's pretty cool.

04:28 This is from like 2017, it's an older article, isn't it?

04:33 - Yeah, it looks like it, yeah.

04:34 - Yeah.

04:35 - It's news to me, maybe not news to the world.

04:37 Like Emily said, she was already working with it previously, but yeah, I think it's great.

04:42 >> Right. Well, the Subprocess Communicate is people often shifted over to just run.

04:48 I'm hoping there's a run version of that too.

04:51 >> Yeah, probably is.

04:53 >> Anyway, cool.

04:54 >> Indeed. All right. Well, I'm going to explain some stuff to us, Brian.

04:58 >> I must.

04:59 >> I see the author of what you're about to talk about out in the audience as well.

05:03 >> Really? That's very cool.

05:05 Well, this is pretty fun.

05:06 >> Definitely an exciting one.

05:07 >> Yeah. This is TypeSplainer and let me explain it to you.

05:11 So I don't know, it's just this cool thing.

05:16 It popped up last week.

05:17 We saw it.

05:18 This is from Arian, sorry, Arian Malik Wasi.

05:23 It's a pretty cool name by the way.

05:26 But so this is this little neat Heroku app that has, it's pretty simple.

05:33 Well, I don't know how simple it is behind the scenes, but to use it's very simple.

05:38 you pop in any sort of Python code that has type hints in it.

05:43 And so this one has, like, for instance, we've got an example up that the default with like a callable that takes a std and an int and a generator.

05:53 And yeah, so there's a bunch of type hints in here.

05:56 This is even like kind of more than most people use all the time, but, and then you hit typesplain and it will show you what the different type hints mean and translate them into English for you.

06:09 It's just pretty cool. I like it.

06:14 One of the things that Wasi said that he was also, when he was developing this, he was on his way to developing a Visual Studio Code plugin.

06:26 If you search for TypeSplainer as a VS Code plugin, that functionality is available to you right in your editor as well.

06:36 This is pretty neat.

06:37 >> Yeah, this is really cool. This explanation you have there, like dictionary of list of set of frozen set of it.

06:42 Like, oh my gosh, the description is something like a dictionary that maps a list of sets, of frozen sets of integers onto a string.

06:50 That's way more legible and internalizable than.

06:54 >> Yeah.

06:55 >> How many brackets deeper we were?

06:57 Four brackets deep in that type information there?

07:00 >> Yeah. It's interesting on Twitter with the announcement of it, or we heard about it through Will McGugan, or at least I did.

07:10 Some of the comments were like, not that this isn't cool, it was like, "Oh, yeah, this is cool." But maybe Python shouldn't be this complicated if you have to explain it.

07:22 >> Have these people done C++?

07:24 Let me just ask them.

07:25 >> I know. You don't have to use types if you don't want to, but there's a lot of places where types are helping out a lot.

07:33 If you're running into somebody else's code that has some types on there that you're not quite sure what that's going on, throw it in TypeSplainer and you'll be able to figure it out.

07:43 >> Absolutely.

07:44 >> Yeah, and I did actually take a look.

07:46 >> Emily, what do you think about this?

07:47 >> I think this is awesome. I absolutely agree.

07:50 mypy has allowed us to do gradual typing and all that, but a lot of times you do jump into somebody else's code base and you're like, "Whoa, these are more types than I've ever seen." Being able to convert it really easily is nice.

08:02 And I did actually take a look at how it works under the hood.

08:05 There's a really big if statement of like serialization, but then it also, I'm a nerd for anything AST related.

08:14 And so it uses like the mypyparse under the hood, which is actually relatively complex for what it needs to handle based on, you know, different Python versions and whatnot.

08:23 Wow, that's pretty awesome.

08:25 Yeah, the very first time you were on talkbythenomy was to talk about the AST, the abstract syntax tree, right?

08:31 It was right around my first conference talk back in the day.

08:35 >> Yeah, awesome. Way back in the day.

08:37 I think we met in Vancouver to set that up.

08:39 When you were up, we met at PyCascades there.

08:41 >> I generally think of myself as a smart person, but people that can handle doing AST work in Python, I'm like, "Oh my gosh, it's over my head." >> Yeah, that's pretty awesome. One thing really quick before we move on, Brian.

08:57 If you go to the homepage of the TypeSplainer, - Yeah. - Which is basically back one.

09:01 So it's got this fairly pathological insane example to show you like you can take crazy stuff and explain it, yeah? - Yeah.

09:08 - But you can type in there.

09:09 Like you can take that code away and put whatever you want in there and then hit type spline.

09:13 So if you just made like a function that just took, you know, yeah, exactly.

09:18 Give it a name and do a dot, dot, dot, yeah.

09:20 Give the function a name, you'll be good.

09:22 - Type spline. - And then you hit type spline.

09:24 Oh, oh, oh. - Compiler, nevermind.

09:25 - It needs a function name.

09:27 - Oh, I didn't.

09:28 - It's not JavaScript, come on.

09:30 (both laughing)

09:31 - Try and use Python. - Ooh, I love it.

09:34 Yeah, see, I mean, it's not a super huge explanation of what an integer is, but like you can take some random code and drop it and then go explain it to me.

09:41 I guess now you can also use the VS Code extension, which I haven't seen.

09:44 Anyway, I thought this was cool as well and certainly saw a lot of people talking about it on Twitter when it came out.

09:50 So, but final thing, I think this is noteworthy as well.

09:54 And I think it's worth mentioning, WASI is just 14.

09:57 So speaking of people like Emily, who can do like AST stuff, like crazy, like also, if you write this when you're 14, well done, you're on your way.

10:05 Plus it's, it's really good looking for something right out of the gate.

10:10 So it looks nice.

10:11 Yeah.

10:11 Awesome.

10:12 Yeah.

10:12 I also think the, the like architecture of it's really great too.

10:16 So I really like that.

10:17 He embraced sort of building out the tool itself and then building a CLI and a web interface and a VX code extension.

10:25 So I think that is a really great example of how to structure a project like this, yeah.

10:31 - Nice. - Yeah, that's awesome.

10:33 Hey, Emily, we lost your screen, if you wanna share it back, or I can just add in. - Oh, yeah.

10:39 - While you're working on that, let me just follow up with Sam real quick, who pointed out, "Be super aware of the limitations of your hardware "when you try to write files in async environments.

10:47 "I had this project that ground to a halt "because too many workers were trying to run at once." Yes, absolutely, good point, Sam.

10:54 That is generally true for any limited resource that you point async things at, right?

10:59 If you're going like async against a database and a couple of queries will max it out, and if you end up hitting it with 100 queries at the same time, it's not gonna go faster.

11:08 It's only gonna get worse as it like fights for contention and resources and stuff.

11:13 And then on this one, on the TypeSplainer, Brian Skin says, "Agreed, very nice work." So awesome.

11:20 - All right, so this is another--

11:21 - All right, Emily, you're up for the first one.

11:23 - Yeah, so this is another one of those new to me things, but Marlene's article just came out and that's how I actually found out about it.

11:30 So Marlene wrote this really excellent introduction to using IBIS for Python programmers.

11:36 IBIS itself has been around for like seven years or so.

11:39 It's a project I think that was started by Wes McKinney, but they are a productivity-centered Python data analysis framework for SQL engines and Hadoop.

11:49 So you get a ton of different backends.

11:52 going to compile to basically any flavor of SQL database and then a bunch of more data science-focused backends. But this popped up on my Twitter feed from Marlene. And it's just a really great introduction. Also, just a really interesting sort of application.

12:10 So she went through and wanted to pull some art information about a city that she was going to visit because she likes the experience, the culture of a new city. So it just walks through like, how to get data into it and then how to interact with it with IDIS. So I'll actually switch over to the IDIS documentation. Oh, and this is now just different because it's small. But, yeah, I think I was really interested in it because we have, like, a pseudo legacy system that we're moving all the migrations out of Django and we're actually managing it with a tool called Hasura. We're so used to having Django that's going to use SQLAlchemy and ORM and everything is magic from there. You give it a YAML file and you get seed data.

13:03 So we're trying to figure out how to manage seed data in a wildly different environment where you have to load it in via the Hasura CLI tool. And you need SQL. And I don't want generate SQL like anything I can possibly do to avoid that.

13:18 So this was a really neat way for getting around needing that modeling.

13:25 So let's see if I can get super cool.

13:27 Yeah, it also looks like it integrates with Hadoop and other things that maybe are not direct SQL.

13:33 It might need a slightly different query language anyway, right?

13:35 Yeah, and it's super interesting.

13:37 So they have a few different ways that it works.

13:39 So it directly will execute pandas and then it compiled in a few different ways to either those SQL databases, Dask, Hadoop, BigQuery, a bunch of different stuff.

13:51 But yeah, it's not necessarily just going to be straight SQL, but it's going to handle that for you.

13:56 So you're really future-proofing yourself away from that.

14:00 But yeah, they just got a ton of really intelligent ways to filter data and interact with data in a really performant way.

14:08 I'm actually gonna go back to Marlene's blog post real quick and do some quick scrolls.

14:14 It's also one of the most Pythonic tools to integrate with SQL that I've seen. So she gets to the point where she has this database table. So you just tell it, you know, your table name and you set the variable. And then you can interact with it as if it's just a dictionary. So you've got your table.

14:33 Oh, that is cool.

14:34 you want to just pull these columns and you've got it and it's there. So I thought it was like a really beautiful.

14:40 So you would say something like dv dot table of quote art and then you say art, you know, art equals the art bracket, quote artists and display and then boom, you get those back, right? That's awesome as a dictionary, I guess, or something like that. Yeah, that's cool.

14:55 So yeah, there's a ton of different things that you can do with it. I highly recommend checking out their tutorials. They've got a ton of different options. My favorite one One is the geospatial analysis. So, if you check out their example, they're going to show you how to pull information out of a geospatial database and then a really quick way of actually mapping out the data. So, if you check out these examples, I know it's not going to come through necessarily on audio, but you can pull information out of these land plots and then tell it to graph it. And it gives you a really nice looking graph with of all the data there in like a whopping 10 lines of code.

15:37 So, yeah.

15:39 - Generating that picture in 10 lines of code, that's awesome.

15:43 - Yeah, it's pretty neat.

15:44 - It makes me think I should be doing more with geospatial stuff.

15:46 Like I don't do very much 'cause I'm always afraid like, ah, how am I gonna graph it?

15:49 What am I gonna do?

15:51 But there's a lot of cool layers in that graph and everything, that's neat.

15:54 - Yep. - Yeah, yeah.

15:56 The API reminds me a little bit of PyMongo actually, where you kind of just say, you know, dot and give it the name of things and it's really kind of dynamic in that sense and you get dictionaries back and stuff.

16:07 - Yeah, and any--

16:08 - But it's against different databases, right?

16:10 - Right, yeah, but I do like that perspective.

16:12 Like it really is kind of taking any database, but especially taking a relational database and giving it more of a document-oriented interface to it, which is pretty cool.

16:24 - Yeah, this is cool.

16:25 I definitely want to check this out for, especially for exploration.

16:29 It feels like it's really got a lot of advantages for data scientists.

16:32 like they're going to fire up a notebook and like I just need to start looking at this and playing with it.

16:36 They don't really want to write queries and then convert that right.

16:39 Well it also looks like as far as I can tell it looks like both in this article and in one of the tutorials on the main web page is that there's a good relation one to almost a one to one relationship between SQL things you can do in SQL and things you can do here.

16:57 So, so the people familiar, already familiar with SQL can transfer over pretty easily.

17:04 So that's pretty neat.

17:05 - Absolutely.

17:06 Yeah, this is a nice find.

17:07 All right, Brian, before we move on, can I take a second to tell you about our sponsor?

17:10 - Yes. - Very excited about this.

17:11 I think it's a big opportunity for people.

17:14 So let me tell you about Microsoft for Startups Founders Hub.

17:17 This episode of Python Bytes is brought to you by Microsoft for Startups Founders Hub.

17:22 Starting a business is hard.

17:24 By some estimates, over 90% of startups will go out of business in just their first year.

17:29 With that in mind, Microsoft for Startups set out to understand what startups need to be successful and to create a digital platform to help them overcome those challenges.

17:38 Microsoft for Startups Founders Hub was born.

17:41 Founders Hub provides all founders at any stage with free resources to solve their startup challenges.

17:48 The platform provides technology benefits, access to expert guidance and skilled resources, mentorship and networking connections, and much more.

17:56 Unlike others in the industry, Microsoft for Startups Founders Hub doesn't require startups to be investor backed or third party validated to participate.

18:06 Founders Hub is truly open to all.

18:08 So what do you get if you join them?

18:10 You speed up your development with free access to GitHub and Microsoft Cloud computing resources and the ability to unlock more credits over time.

18:18 To help your startup innovate, Founders Hub is partnering with innovative companies like OpenAI, a global leader in AI research and development to provide exclusive benefits and discounts.

18:28 Through Microsoft for Startups Founders Hub, becoming a founder is no longer about who you know.

18:33 You'll have access to their mentorship network, giving you a pool of hundreds of mentors across a range of disciplines and areas like idea validation, fundraising, management and coaching, sales and marketing, as well as specific technical stress points.

18:47 You'll be able to book a one-on-one meeting with the mentors, many of whom are former founders themselves.

18:52 Make your idea a reality.

18:53 today with a critical support you'll get from Founder's Hub.

18:57 To join the program, just visit pythonbytes.fm/foundertub, all one word, the link's in your show notes.

19:04 Thank you to Microsoft for supporting the show.

19:06 Yeah, so $150,000 credit people get.

19:08 So if you're doing a startup, you know, that would have been awesome when I was trying to do a startup.

19:13 Now, this next thing I wanna tell you about, I think this kind of lives in your wheelhouse as well.

19:18 And keeping with the theme of the show, this one is recommended by Will McGugan.

19:22 So thank you Will for all the good ideas.

19:25 I know you're out there scouring the internet for all sorts of cool things to use on textual and rich and whatnot.

19:32 And this is the one of the ones they said they are gonna start testing on and that has to do with performance.

19:37 So the topic is, or the library is airspeed, velocity or ASV and the pip nomenclature.

19:44 And the idea is basically setting up profiling and performance as a thing that you can measure over the lifetime of your project rather than a thing that you measure when you feel like, ah, it's slow, I need to go figure out why it's slow for now.

19:57 So as you automatically do, as you do check-ins, as you know, like CI runs and stuff like that.

20:02 So probably the best way to see this is to just like pick on an example.

20:07 So if you go to the link in the show notes, airspeed velocity, there's a thing that says, see examples for AstroPi, NumPy, SciPy, I'll pick up AstroPi, and you get all these graphs.

20:17 So each one of these graphs here is the performance of some aspect of Astropy over time.

20:24 How cool is this?

20:25 Look at that.

20:26 - That's pretty.

20:27 - And if you hover over it, it shows you the code that runs that scenario.

20:31 - Wow.

20:32 - Yeah.

20:33 And this is a sample, this is a sample, and then they did a huge, you can see there are like two massive refactorings on the sky coordinate benchmarks, scalar, whatever this is, right?

20:44 This particular test they did there, it goes along pretty steady state, and then there's a big drop, a little spike up, and then another big drop and then steady state for a long time.

20:53 So wouldn't it be cool to have these different views into your system about like how it's performing over time?

20:59 - Yeah, so lower is better, right?

21:01 - Yeah, I believe lower is better.

21:02 I think lower is better.

21:04 You can pull up regressions.

21:05 You can say, okay, well, what got worse?

21:07 Like for example, timetable out putter got 35 times slower.

21:12 So that might want some attention.

21:15 And it lists the GitHub command, or really technically I suppose it just lists the git commit, which is probably on GitHub, which is on GitHub, so that you can actually say, this is the code that changed that made it go 35 times slower.

21:28 - Wow. - That's neat, I think.

21:30 I think one of the other challenges here is, what about, what if you wanted this information, but you're only now learning about this project, right?

21:41 You're only like now realizing, wouldn't it be great to have these graphs?

21:46 How do you get those graphs back in history?

21:49 So Will pointed out that you can actually connect it with your Git repository and it will check out older versions and run it.

21:57 It'll like roll back in time and go across different versions and different versions of Python and generate those graphs for you, even if you just pick it up now.

22:07 - That's awesome.

22:08 Any idea of if it's like restricted to packages or if you can also apply this to general projects?

22:11 - I think you can apply it to general projects.

22:13 I don't remember where I saw it.

22:16 I had to pull it back up here.

22:17 Somehow I've escaped the main part.

22:19 But yeah, I think if you look at the using Airspeed, you basically come up with a configuration file that says, you know, this particular project with these settings, and then here's like the run command.

22:34 You come up with one of these test suites.

22:35 I don't think it has any tie-in to packages per se, 'cause I think it goes against Git, not against IPI.

22:42 Yeah, yeah, so pretty neat.

22:44 People can check that out, but like here you can specify like which versions of Python or is this two seven stuff, I don't know.

22:51 But yeah, so you can run it against all those old versions, you can configure how it runs and so on.

22:56 - Okay, so you can even, you can set up like, since you're defining what's being timed, you can time large things like a particular workflow through lots of bits of code first and things like that.

23:10 - Yeah, exactly.

23:11 So you basically come up with a couple of scenarios of what you would want to do that you're going to run against.

23:15 Here you can see you can benchmark against tags and things like that and get, or branches.

23:22 >> Yeah.

23:22 >> Yeah.

23:23 >> So Will says he ran it against two years worth of rich releases.

23:31 That's cool.

23:32 >> And found a performance regression. Nice work.

23:35 I love it. Optimizations that made rich slower.

23:38 Isn't that true? This is going to make it better.

23:40 - No.

23:41 - Yeah, so pretty cool.

23:45 - And I have to give a nice shout out to the full embracing of the mypython reference.

23:51 If you go back to the AstroPy version in the top left corner, it says airspeed velocity of an unladen AstroPy.

23:57 - Oh yeah, I did notice that, that's awesome.

24:00 - It's nice.

24:02 - Yeah, very cool.

24:03 Well, yeah, thanks for sending that over, Will.

24:06 - Yeah, I got some projects I'd like to do that on.

24:08 But speaking of testing things, this one comes from Anthony Shaw.

24:13 This is Perflint.

24:15 So this is a pilot extension to check for performance anti-patterns.

24:22 And it's Tony somewhere, Anthony.

24:26 - Some guy named Anthony Shaw.

24:29 - Anthony Shaw.

24:30 - I forgot if I even mentioned him.

24:30 - Tony Bologna.

24:32 Says, oh, here it is.

24:33 Project is in early beta.

24:35 it will likely raise many false positives.

24:38 So I'm thinking that might be why he went with an extension to Pylint instead of like an extension to PyFlex or Flake 8 because Pylint gives lots of false positives.

24:51 No, at least in my experience with Pylint, it takes some configuration to get happy with it because it will show you things that maybe you're okay with.

25:02 Like I threw Pylint against some demo code that I have for like teaching people stuff.

25:08 And I'm using short variable names like, you know, X and Y and things like that.

25:13 And one of the restrictions for Pylint is you have to have, most everything has to be three characters or longer.

25:21 And you know, for production code, that's probably fine.

25:24 But if you have different rules, you can change that.

25:26 But back to this, I really like, I like the idea of having something look over my shoulder and look at performance problems, because I'm an advocate for don't solve performance problems unless you find that there's a performance problem.

25:40 So don't do premature optimization.

25:42 However, having some things are just kind of slow that you should get out of the habit of doing like when using list in a for loop, if the thing that you're using a list of already is an iterable, that's a big performance hit if it's a huge thing, because that turns an iterable or a generator into an entire list.

26:07 It creates the list.

26:08 You don't need to do that.

26:09 So that's a big one.

26:11 Anyway, there's a whole bunch of different things it checks for.

26:14 And I like the idea of just, as you're writing code and as you're running this and trying to figure out if there's problems with it, you can kind of get out of the habit of doing some of these things.

26:27 - Yeah, these are nice.

26:28 catch just some of the things you might think you need to do, you're not super experienced with or whatever.

26:35 >> Yeah. One of the things here is error W801, which is loop invariant statement.

26:43 This is one that's interesting is, there is an example of taking the length of something within a loop.

26:51 If that never changes within the loop, don't do the length in the loop, take it out of the loop.

26:56 >> Yeah, exactly.

26:58 >> There's a few examples that you might not notice right away, especially if you've taken something that was some linear code that you added it inside of a loop and indented it over, and now it's in a loop.

27:13 You might forget that some of the stuff inside might maybe shouldn't be in the loop.

27:18 >> Yeah. This example here, you're doing a loop 10,000 times, and every time you're asking the length of this thing that is to find outside the loop and is unchanging.

27:27 So you're basically doing it 10,000, 9,999 times more than necessary.

27:32 Yeah.

27:33 Yep.

27:33 So kind of fun.

27:35 I'm going to give it a shot, see what I think in as using it.

27:39 So yeah, definitely.

27:40 Emily, do you use some of these linters or any things like this to give you warnings?

27:46 Yeah.

27:47 Yeah.

27:48 I mean, I think we mostly use Playgate, but I'm definitely curious to try this out too.

27:51 I can see how this would be tricky to get really consistent errors for these things.

27:58 So props to Tony Bologna for taking it on.

28:01 - Well done, yeah, this is exciting.

28:03 I'm glad to see this coming out.

28:04 I know he was talking about it, but I didn't see actually anything on GitHub yet or anything.

28:09 So yeah, very well done.

28:10 - Yeah, this is cool.

28:12 I like stuff like this that really like takes you to that next level of like, this is something that somebody would hopefully notice in like a code review, but if you can automate it.

28:20 - Yeah, I think that's a great point.

28:21 I think a lot of these things that would have to be a discussion during a code review, if they could be automated and you could save the code review for meaningful stuff like security or, you know, like how are we going to version this over time?

28:35 It's going to be tricky.

28:36 Like, are you really storing pickles in the database?

28:39 Let's not, you know, stuff like that.

28:40 Yeah.

28:41 All right.

28:42 PEP 594 has been accepted, which is super exciting.

28:47 So PEP594, if you don't know what that is, it's a Python enhancement proposal.

28:53 So a proposed change to the Python language itself.

28:58 And so this one is removing dead batteries from the standard library.

29:02 It was written by Christian Himes and Brett Cannon.

29:05 I think I saw a tweet from Brett saying that it had been accepted.

29:10 So this is just really exciting for anyone who's followed along with any of this discussion.

29:16 It's been a long time coming. I think there was a major discussion about it at PyCon US 2019, it must have been. And shortly after that, there was a pep. But it's been since then that it's kind of been off and on in discussion and finally figuring out what is going to be the thing that really works for everyone and for the future of the language.

29:41 So this is going to be targeting version 3.11.

29:45 So just a quick recap of the release plan for that.

29:50 Development on 3.11 will start this May, so May 2021.

29:54 The final release, even for 3.11, is not until October 2022.

29:59 And even then, this is just going to be deprecating modules.

30:03 So it'll be deprecations in 3.11 and 3.12.

30:06 And it's not until 3/13 that it will actually be fully removed from the language itself.

30:12 So you can kind of get a glimpse into how long of a process this is and how big of a decision it was to get everyone on board.

30:21 Yeah, it didn't look at all like anything rushed.

30:24 When I went through and read this, it was like, here's the things that we think we can take out.

30:30 why there's a table in there that shows third party alternatives to certain things.

30:36 Mostly, yeah, that's the one.

30:38 So there's certain things in here just like, "No, that probably isn't needed or it's really superseded." So there's pipes, but then we also have subprocess, which will take care of that, and that's a built-in one.

30:51 Then async core, just use asyncio, but then there's other ones.

30:57 - There's a bunch in here I've never even heard of.

31:00 - Yeah, that's the thing, right?

31:01 There's one called Crypt and it's like, look, just use Passlib or Argon or Hashlib or anything that is better and modern.

31:08 You know, this was from 1994.

31:10 Cryptography is not exactly the same as it was then.

31:14 So, you know, maybe it makes sense to take it out, right?

31:17 - Yeah.

31:18 - I guess.

31:20 - Yeah, yeah, I think it's a really like, it's a thin line to walk, right?

31:24 Like some people are using these and some of these modules maybe didn't have a lot of like maintenance over time.

31:31 But that also meant that there wasn't somebody watching it for bugs or security vulnerabilities or anything like that.

31:37 So the balance of is it worth pulling it out if somebody was relying on it versus the maintenance cost or the lack of maintenance that could be a liability.

31:48 There's a CGI library. That's something else that takes you back from '95.

31:54 - Yeah, that's how I started, but not with Python.

31:57 I was doing CGI with Perl way back in '95.

32:00 - Yeah, that does go back.

32:02 It also talks about whether that bit of code has a maintainer and whether that maintainer is active.

32:07 For example, CGI has no maintainer.

32:10 Like, no one wants that.

32:11 One of the things that's interesting here is you could take this code and you could still use it.

32:15 You could vendor it into your code, right?

32:18 Just instead of using-- - Yeah, now you're the maintainer.

32:20 Yeah. - Yeah, exactly.

32:21 It's all yours, you can have that.

32:23 But you could just go to CPython on GitHub, get that module, copy it over, and now you kind of still have that functionality, just you're taking it on.

32:33 I expect maybe one or two of these might end up in their own GitHub repository as a package that is maintained.

32:39 They did talk about that, right, Emily?

32:40 About that being one of the possible paths they decided against?

32:43 - Yeah, yeah, yeah.

32:45 That was like the big conversation back at the Language Summit in 2019 was, you know, could we get libraries on a more independent release schedule and pull them out of the standard library entirely and just have them be sort of their own standalone thing, which as I have briefly outlaunched the release schedule for 3.11, you can see that it is on a very long scale time frame.

33:12 So I definitely agree.

33:14 I think that's some of the use that people are still using.

33:16 People are either going to go in there and grab the code and hopefully grab the license with it as well.

33:22 Or they're just gonna become, you know, modules that enough people care about that live on their own in PyPI.

33:28 - I don't see anything here that I would miss, but that doesn't mean that there's not people using them.

33:33 - So on the good side, I mean, it totally makes sense to like remove things, especially stuff that's not getting maintained and there's no maintainer and does, possibly has bugs in it now, nobody knows.

33:46 But like, what are some of the good aspects, other good aspects?

33:51 Is it gonna make the library or the Python standard, the install smaller or, I mean, you'd think, anybody know the numbers on that?

34:01 Okay.

34:01 - I don't know the numbers on that, but that is something interesting to look at.

34:04 - I would say the biggest change is like maintenance.

34:06 Just no one has to worry about whether there's a bug in CGI that someone discovers 'cause it's just not there.

34:13 - Yeah.

34:14 - Yeah.

34:14 And especially with CPython, there's often a very big barrier to entry. So if a CGI bug was even filed by somebody, where would you start to take that off? And then the other thing too, is maybe somebody else goes through the effort to fix it. But it always takes the core dev to review that PR and get it merged in. And so a lot of times, if you don't have an owner of a module, It's just not going to get a lot of attention.

34:43 So as a whole, it should be hopefully an impact on how we interpret core developer time.

34:52 Because right now I think we're at like over a thousand PRs open on GitHub.

34:56 So a lot of times, you know, it's not just core developers writing code.

35:00 And a lot of times you can have even more of an impact being that person that, you know, tries to review PRs and keep that number down.

35:07 - Brian out in the audience points out that the comment threads on discuss.python and elsewhere are really interesting if you wanna see examples of these old modules still in use, yeah.

35:17 - Yeah, I've got a couple of them here.

35:19 I think I linked them in the show notes, but if they're not there, I'll make sure it's in there.

35:22 - Nice, yeah, you got a link to Brett's, or the Brett's discussion there, that's cool.

35:26 No, I think this is good, I think this is good.

35:27 And quick shout out to a new theme, right?

35:30 - Yeah, so it's a brand new PEP site.

35:33 So it's peps.python.org, and there's this really lovely theme on it.

35:37 It's really clean and modern.

35:39 You've got a nice dark theme here as well.

35:41 - Yeah, I noticed the dark theme, that was cool.

35:43 And I think it even auto adapts to the time of day, which is great.

35:46 Right, is that it for all of our main items?

35:47 - I think it is.

35:49 - It is.

35:50 - Do you have anything extra for us?

35:52 - Would it surprise you if I said no?

35:53 - Yeah, what? - Nothing extra.

35:55 I know, I always have like 10 extra things.

35:57 No, I don't have anything extra this week.

35:58 - Oh, really?

35:59 - Yeah.

36:00 - Nice, nice, okay.

36:01 How about you, Emily?

36:02 - Cool.

36:03 Yeah, I've got a couple of extra things.

36:06 So as I was prepping for this, I looked at, I think it was just the most recent episode before this one.

36:11 There was a blog post that I think Brian shared on like a better Git flow that basically was saying like, commit all your stuff, reset everything, and then recommit everything once you're like ready to make a clean PR.

36:27 And so I wanted to share this as well.

36:30 This is one of my favorite tools that I learned about probably a few months ago.

36:34 Again, it's 2015.

36:36 Not a new thing, but new to me.

36:38 So you can do auto-squashing of Git commits when you're interactive rebasing.

36:45 So essentially, if you've got a ton of different commits and you realize, oh, like I had a style commit for styling all my new stuff a few commits back, but like I wanna make this one more change, instead of needing to rebase immediately or remember to stage it in a certain way in the future, you can actually go ahead and just commit one more time.

37:06 And then you flag that commit that you're making with the fix up flag.

37:10 So it's just --fix up.

37:12 And then you tell it the commit that you're wanting to sort of amend.

37:18 So you can just keep working like that, make your fix up commits.

37:21 And then the only thing that you do right before you PR is you tell it to rebase with auto-squashing.

37:27 So once you do that interactive rebase with auto-squash, it's gonna find all those fix up commits And when you interrupt a rebase, you often have to move commits around and tell it to squash it in the previous commit.

37:39 You've gotta get it in the right order.

37:42 This handles all of it for you.

37:43 And anything that's flagged with a fix up, it finds that commit ID and auto squashes it back in.

37:49 So you get a really, really clean history without having to redo all of your commit work that you had done a long way.

37:57 - Yeah, this is really nice.

37:58 And this looks built in to Git.

37:59 Yeah.

38:01 I've never heard of auto squashing.

38:02 I've definitely never used it, but it looks really useful.

38:05 - Yeah. - Cool, thank you.

38:06 All right, what's your next one?

38:07 - Yeah, yeah, and then a couple other cool ones.

38:11 There was a tweet from Dustin Ingram about an award that the Python Software Foundation actually received.

38:18 It's from the Annie Awards, which is, you know, animation version of the Academy Awards sort of thing.

38:27 And it was for Python's use in animation.

38:31 And so I think this is just super cool.

38:33 It's one of those like applications that you don't necessarily think about for Python all the time.

38:39 I don't think it gets talked about enough.

38:41 I actually tried to find, Paul Hildebrandt had a talk at PyCon Montreal, but I think it was back before we were recording these.

38:51 So if you ever see Paul at a conference, you've got to ask him about, you know, how Python is used in animation and at studios.

38:59 - Oh yeah, that's really neat.

39:00 So exciting.

39:01 I would have never expected that, but that's great.

39:03 And congrats, Guido, for getting the award.

39:05 - And two more quick ones.

39:07 The PSF Spring Fundraiser launched yesterday and they're having a ton of fun with it.

39:12 Launched on at least Pi Day in the United States.

39:16 So if you donate with some sort of contribution that is related to the number Pi, you get like a free swag bag.

39:25 So just a fun twist on this.

39:28 - You can donate $3.14 $31 and 41 cents or $314 and 116 cents.

39:35 And yeah, like it goes pretty far out if I remember pi, there's a lot of numbers in there.

39:41 So yeah, just keep going.

39:43 Yeah.

39:44 Cause whatever your bank account will allow.

39:46 All right.

39:49 Anything else you want to throw out?

39:51 yeah, just one last quick one.

39:54 Just a small plug for, for us.

39:56 Kettlesoft is hiring.

39:58 We have a bunch of different positions open, but we're especially always looking for Python engineers.

40:04 We're a small team. We're a team of about eight people right now.

40:08 Predominantly female engineering team. And just the pride of what I have done in the last few years of building this team.

40:17 So if you're looking for some place that is always innovating, always focused on really high-quality tested code, but you wanna work in a small team environment, get hands-on with clients, get hands-on with product, come check us out.

40:31 - Codalsoft looks really cool.

40:32 You seem to be doing a lot of, bunch of different small fun projects instead of just getting stuck in like one huge legacy code.

40:38 So if you're looking to kind of bounce around from project to project and learn a lot, I think that'd be a good place, right?

40:44 - Yeah, absolutely. - All right.

40:46 Well, I have two jokes for us, even though I have no extras, so I'm making up for it there, I guess.

40:51 - Nice.

40:52 So Aaron Patterson said, "I heard Microsoft is trying to change "the file separator in Windows, "but it received tons of backslash from the community." (imitates drum roll)

41:03 That's pretty funny, right?

41:04 - But the forward slash works fine in Windows.

41:06 People just forget to use it.

41:08 - It actually does, it totally does.

41:11 And following along there, oh, Emily, I think this is the perfect follow-on for you as well.

41:16 Do you ever look at people's GitHub profiles if they apply?

41:21 Like if they say, yeah, right, of course.

41:22 I mean, that would be crazy not to, right?

41:25 So this person here, you know, if you go to your GitHub profile, it will show you your public activity over time.

41:33 And it'll say like on this day, you know, in September on Monday, you had this much work and then on Tuesday that much, and it'll color like different colors of green.

41:40 So if you all check out the link here, we have a GitHub activity for a year that spells out, please hire me in like the exact amount of commits on just the right day.

41:51 And I think that's.

41:52 - I think there's some history manipulation going on here, but.

41:55 - There are probably some auto squashing, I don't know.

41:58 (laughing)

42:01 - I mean, hey, I would look at that and think that they had some decent enough Git skills to manage that. - Yeah, exactly.

42:07 It does mean that you're probably not doing like normal Git work on one hand, but on the other, like, I'd have to think for a while to figure out how to get it to draw that out.

42:16 So that's pretty cool too.

42:19 That's one of the main reasons why I switched my blog to Hugo, so that blog posts count as Git commits.

42:25 >> Exactly, double dip.

42:26 >> Yeah.

42:27 >> Nice.

42:28 Well, that's what I brought for the jokes.

42:32 >> Nice.

42:33 Well, thanks, everybody, for showing up.

42:35 Thanks, Emily, for showing up here, and also for the Walrus operator.

42:39 Love it.

42:40 >> Yeah.

42:41 >> And we'll see everybody next week.