Transcript #386: Major releases abound

00:00 Hello and welcome to Python Bytes where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 386, recorded June 4th, 2024. I am Michael Kennedy.

00:12 And I am Brian Ruckin.

00:14 This episode is brought to you by Mailtrap. I want to tell you more about them later.

00:20 And I also want to let you know that you can find us over on Fosstodon.

00:24 We are Fosstodonians, as we've said before.

00:27 If you want to talk to us on Mastodon, the links are in the top of your podcast player show notes.

00:32 We're also on X, but do a little bit more stuff on Mastodon.

00:36 So check us out there.

00:37 Also, you can find us streaming live at exactly this time right now, Brian.

00:43 Finally, which is usually 10am Pacific time on a Tuesday.

00:49 So just pythonbytes.fm/live and you can be part of the live show.

00:52 If you want, watch the older videos there if you like, or just subscribe in your podcast player. We really appreciate that.

00:57 Finally, interest in our email that you've been writing.

01:00 There's a lot of people signing up for it, Brian.

01:02 So if you want an artisanal handcrafted digest of the show in written form,

01:06 just visit pythonbytes.fm.

01:08 Click on newsletter right there in the center and enter your email address.

01:13 We won't share it, but we will send you an email after every show about what we covered.

01:18 So you have it saved if you like.

01:19 Yeah.

01:20 All right. You want to kick us off, Brian?

01:22 I will. So this comes as a suggestion from somebody and I'm sorry I didn't write the name down.

01:28 So thanks. We do love it.

01:30 I love it when people share important news with us and say, "Hey, you really should cover this because it's important."

01:37 Thank you for that.

01:38 So what am I talking about today?

01:39 I'm talking about NumPy 2.0.

01:41 And I was searching for it.

01:43 I thought we'd covered some of the changes in NumPy 2.0, but I don't know if we have.

01:48 So NumPy 2.0 has been in the works for a long time.

01:52 The reason why I'm bringing it up right now is because the release date is June 16th.

01:56 It's right around the corner.

01:58 And NumPy is both, you're probably using it.

02:02 I guess I'll just say that.

02:04 If you're using any data science or any machine learning or something, you're probably using it, even if you don't use it directly.

02:13 But you probably are using it directly.

02:15 Anyway, lots of changes to 2.0.

02:17 We're linking to an announcement.

02:20 We've got the main page, numpy.org, announces it.

02:24 You can just go there if you want.

02:25 But there's also a news item for the release for the June 16th.

02:29 There is some cool migration guides.

02:31 So I'm not going to go over a lot of the details, but there are some breaking changes with this.

02:37 So I'll go over a few things.

02:40 First off, went over and looked at the migration guide.

02:43 Let's see the release note.

02:44 Migration guide, I thought this was super cool.

02:46 So the migration guide is around to help you go through, navigate some of the changes that might be the most problematic.

02:56 But one of the cool things is Ruff is helping out.

03:00 So if you're using Ruff greater than version 0.2.0, you added a rule for Npy 201.

03:11 But you can just look at the migration guide and find this.

03:14 You select this rule and run Ruff and it will, and they can just run check or you can just run Ruff to correct.

03:23 Ruff is around to help you with a lot of the stuff in this migration guide

03:28 to try to modify your code from older NumPy to the NumPy 2.0.

03:34 So that's really cool.

03:36 I would probably start there.

03:38 Also, making sure you have tests around your code to make sure your code is working before and after the migration.

03:44 And then the release notes gets down to a lot of the nitty gritty stuff.

03:49 There's new string stuff, there's performance improvements, but there's also hopping down, there's a lot of breaking changes.

03:57 And we need to allow projects to do this.

04:02 So I think this is a good thing that they're moving towards some changes in NumPy.

04:07 Yeah, quite a big deal for a 2.0 of NumPy.

04:10 Yeah.

04:11 It's been around a super long time, 15 years or something like that.

04:15 So they must take it seriously.

04:17 And that was released in 2006.

04:19 Yeah.

04:20 And like I said, this 2.0 migration to make sure that they were making, doing the right deprecations and omissions and changes necessary for the breaking,

04:31 breaking some backwards compatibility.

04:35 But for good reasons, I think it's, yeah, just heads up, everybody.

04:40 So.

04:42 Yeah. Awesome.

04:43 Well, that's a good one to cover for sure.

04:45 I have another exciting one.

04:46 Let's jump over to that.

04:48 So if you are a fan of uvicorn, so uvicorn is one of the ways you can run ASGI, ASGI web app.

04:59 So things like Quartz, FastAPI, many others.

05:04 You can use uvicorn and it's definitely sort of the recommended way or the,

05:09 at least suggested way that you run FastAPI for development.

05:14 But then when you go into production, well, you've got challenges.

05:18 So for example, you want your web server to look and see if your web app crashes

05:25 for whatever reason, like something went wrong, excuse me, something went wrong with it.

05:29 Something, maybe it ran out of memory.

05:32 I don't know.

05:33 All these things, right?

05:34 Yeah.

05:35 You need the app to sort of look at that and say, well, we're going to restart it

05:38 because it looks like it's hung up or it looks like it crashed.

05:42 Bring it back.

05:43 The other thing you need to be able to do is you need to be able to potentially

05:46 scale out across processes.

05:49 So maybe I've, well, not maybe, at Python Bytes, we have an eight CPU server.

05:55 If we just create one process, you know, because of the GIL and other things,

05:58 it might be kind of blocked up.

06:01 If it's processing some slow requests, we don't want it to have to just queue up behind it.

06:06 You want to be able to say, well, this one is busy, but there's three other processes

06:09 that are clones of this thing running and hand it off to one of them, right?

06:14 Typically that's done in production by using g-unicorn with u-vehicorn workers, which is

06:20 like, why am I using two servers?

06:21 Like to kind of manage the CEO.

06:23 So like g-unicorn manages u-vehicorn running this stuff.

06:26 Like what a mess, right?

06:27 So the news of this item is u-vehicorn now does all of that stuff itself.

06:34 It no longer needs g-unicorn.

06:36 And this news comes from us from John Hagan.

06:39 So thanks for shooting this our way.

06:40 So this is filed under the new multiprocess manager.

06:45 And it says the multiprocess manager introduced by this includes a process keep alive and

06:51 process hung detection.

06:53 It also imitates g-unicorn and uses a bunch of different types of signals to control the

06:59 child processes.

07:00 And there's even, I haven't opened it up yet, but I will.

07:04 There we go.

07:04 I'll include a link to how you use this in production.

07:08 So it's pretty easy.

07:10 You just say u-vehicorn and then the module name and then the variable name of the app.

07:16 So like main colon app, super common.

07:18 But then the new thing is --workers, and you can specify how many workers you want.

07:23 The documentation has four.

07:25 It says unlike g-unicorn, u-vehicorn does not use pre-fork, but uses spawn, which allows

07:30 u-vehicorn's multiprocess manager to work well on Windows as well.

07:35 The default process manager monitors the status of child processes.

07:39 Those are actually the things handling your web requests.

07:42 And it automatically restarts child processes that die unexpectedly.

07:46 That's a little morbid.

07:48 Not only that, it'll also monitor the status of child processes through the pipeline.

07:53 And if it gets stuck, it'll be killed off and restarted and so on.

07:58 So that's the new thing.

08:00 And if you're already using u-vehicorn, especially if you're using u-vehicorn workers with g-unicorn,

08:06 this might be a pretty awesome way to simplify things.

08:08 Not only it might be, you kind of need to because the u-vehicorn workers have been

08:14 deprecated as well.

08:15 The g-unicorn workers, you mean?

08:17 The...

08:18 Oh, yeah.

08:19 Oh, yeah, that's right.

08:20 It is deprecated and will be removed in a future release.

08:24 So I guess get busy, people.

08:25 Yeah.

08:29 So exciting news.

08:31 Yeah, indeed.

08:32 Changes.

08:32 Very exciting.

08:32 Now, do you know what else is exciting, Brian?

08:36 Mailtrap.

08:37 Yeah.

08:37 Our sponsor.

08:38 Yeah, let's talk about them for just a second.

08:41 This episode is sponsored by Mailtrap, an email delivery platform that developers love.

08:45 An email sending solution with industry best analytics, SMTP and email API, as well as

08:53 SDKs for major programming languages and 24/7 human support.

08:57 Try for free at mailtrap.io.

09:00 Over to you.

09:01 All right.

09:01 What you got next?

09:02 We got Pixie.

09:03 This, I did write down who suggested this, suggested by Vic Kelson.

09:08 So thanks, Vic.

09:08 Pixie is a package management made easy.

09:13 So another package manager thing.

09:15 This is something that says Conda package management simplified.

09:21 So this is not trying to replace pip, but really trying to and things like that.

09:26 But it's trying to replace Conda or build on Conda.

09:29 So I haven't really dug too much into this, but I think it's an interesting project.

09:33 This is a cross platform written in Rust, but it's also not just about Python.

09:41 It's about other languages as well.

09:44 So the highlights are supports multiple languages, including Python.

09:49 C++ are using Conda packages.

09:52 And there's some exciting information around it.

09:58 It's cargo like.

10:00 I think that it's built thinking, trying to get Conda to kind of look like cargo.

10:04 Like I said, I haven't tried this, but it looks exciting.

10:10 There is a Pixie tutorial for doing Python development with Pixie.

10:16 This walks through creating a Pixie Pi project, which creates a Pi project tunnel.

10:24 I found this interesting that it is using setup tools as a back end.

10:30 And then there's some extra Pixie project stuff around Conda forward.

10:34 So I think this is a tool if you were publishing to the Conda space, this might be a tool worth looking into to try to manage your Conda environments.

10:45 >> Yeah, definitely.

10:47 It's a super cool project.

10:49 I feel like there was some collaboration between them and maybe the UV folks.

10:56 I can't remember.

10:57 But yeah, it's a super cool project.

10:59 I actually had Wolf Volprec and Ruben Arts on last year in October to talk about this

11:05 on Talk Python.

11:06 So if you want to hear them dive into why they built it and all the plans there.

11:10 And actually, we talked about some other stuff as well that they're doing.

11:13 We talked about like NixOS and so on.

11:15 So yeah.

11:16 >> Interesting.

11:17 >> Awesome.

11:17 Yeah, it's awesome.

11:18 Just, you know, there's a thousand flowers blooming in the packaging space right now.

11:22 It's very exciting.

11:22 >> Yeah.

11:23 So this was mentioned by Vic, but he also gave us a little bit of what his take on it is.

11:30 So I'm just going to quote from Vic right now.

11:34 He says, "Pixie is a project manager written in Rust that allows you to build Python projects

11:39 without having Python previously installed." That's interesting.

11:43 "It's installable via Homebrew.

11:45 There's support for supporting VS Code and PyCharm via plugins to deal with this.

11:51 By default, Pixie fetches packages from CondaForge.

11:55 So you get the scientific stack in a pretty reliable and performant build.

12:00 And if it's not there, it looks on PyPI, or if there's a possibility to look on PyPI."

12:07 So Vic tried to use it.

12:09 He was really impressed and got his Jupyter environment with QPy using NVIDIA GPUs working

12:17 really quickly.

12:18 So at least one person says it's a pretty cool thing to try.

12:21 >> Yeah.

12:23 Scroll up in that screen you got there.

12:25 >> Scroll up?

12:26 Or the other way?

12:27 >> Other up.

12:27 Yeah, to the top.

12:28 >> Yeah, 2,100 people think it's a pretty good way.

12:31 The number of stars there.

12:32 >> Yeah, yeah.

12:33 >> Exactly.

12:34 All right.

12:35 Final one for me is Jupyter.

12:39 So we already talked about NumPy, which also is the foundation of so many other data science

12:45 projects.

12:46 So here's maybe looking at it from the other direction of the data science space, looking

12:51 in from the top, from the UI.

12:52 So JupyterLab 4.2 and Jupyter Notebook 7.2 are now available.

12:59 So it's pretty awesome.

13:00 JupyterLab includes three new features.

13:03 I'll give you some shout outs on them.

13:06 20 enhancements, 33 bug fixes, and 29 maintenance tasks included by 39 contributors, which is

13:14 pretty awesome, right?

13:15 >> Yeah.

13:15 >> And Jupyter Notebook, I think, carries, you know, takes some of these forward as well.

13:21 So also has a bunch of fixes.

13:23 All right.

13:23 So what are some of the features?

13:24 Easier workspace management with the GUI.

13:28 So you can, instead of using the CLI or just grabbing a URL, for example, if you want to

13:33 have it open up a particular notebook when you launch it or something, you could create

13:37 a workspace with that open, I believe, instead of just pointing to a URL to that particular

13:43 notebook, right?

13:44 So now there's like a UI for workspaces that you can control, rename them, reset them,

13:50 delete them, and so on.

13:51 That's pretty cool.

13:52 You have recently opened and closed files.

13:55 So there used to be an extension called JupyterLab-recents to give you access to recent files and recent

14:01 directories.

14:02 And now that just comes built in.

14:04 So I guess if you have the extension still installed, you'll really know what recent

14:07 files you had, but now it probably disables it or something.

14:11 It has full notebook windowing mode by default.

14:15 That's not like a screen layout type thing.

14:18 Suppose you have a notebook with a thousand cells and their output, and those outputs

14:23 have interactive notebook widgets and Jupyter widgets, and they've got a bunch of graphs

14:28 and stuff.

14:29 If you open that thing up and it just goes, well, we're going to render all thousand items,

14:34 you know, it might take a moment, right?

14:35 Yeah.

14:36 So instead what it does is it basically renders only the cells visible on the window, which

14:41 should be for as it gets larger and larger, should be more important for performance.

14:47 Improved shortcut editor.

14:49 So you can do a bunch of different, better things.

14:51 In fact, like even change default bindings and stuff.

14:54 Dark, high contrast theme, more keyboard shortcuts or putting some of them back like control

15:01 D has been restored to allow users to delete a line and that type of stuff.

15:05 So there you have it.

15:06 If you are a notebook person, go forth and get the new one for you, whether JupyterLab

15:12 or Jupyter Notebook.

15:13 That's pretty exciting.

15:14 I'm glad to see this is continuing with the new releases.

15:18 So yeah, it's a really nice, really nice way to work with Python and data science and all

15:22 the things.

15:23 All the things.

15:24 How extra are you feeling today?

15:26 I got a few extras.

15:28 All right, do it.

15:28 Okay.

15:29 Tell us about them.

15:29 Okay.

15:30 This was this a couple of things came from Hugo Von Kinnam, I practiced this earlier.

15:36 Hugo Von Kaminada.

15:37 Sorry if I messed this up, Hugo.

15:41 But anyway, some encouragement for us to test Python 3.13.

15:46 So Python 3.13 is in the betas out and we're trying to, it's got some interesting stuff

15:54 in it, like the free threaded mode.

15:57 So a couple links here to encourage people to help test and some blog posts to help you

16:05 know how to test it.

16:06 So help test Python 3.13.

16:08 There's an announcement here, strongly encouraging people to test and then a specific test around

16:15 the free threading Python without the GIL and a reminder that some people call this

16:20 no gill, but we don't like to say that anymore.

16:23 So the free threading mode and how to test that.

16:27 And that one's a little tricky because you've got to, there are some official installers,

16:32 but you can build it yourself as well.

16:34 And it's available as well and get the dead snakes.

16:37 There's a question marker on GitHub Actions.

16:40 It'd be great if that works, like to hear back from people.

16:44 But we really want this to go well with this because it's a cool project.

16:49 So help test that.

16:51 So that's around the Python.

16:53 That's probably the biggest change, the biggest change facing people using 3.13 and packages

17:00 that you might be using that are on 3.13 and all those things, because those were not built

17:04 with this in mind.

17:05 Whereas everything else is built at least with the concept of backwards compatibility.

17:09 Right.

17:09 But this one.

17:10 Right.

17:10 But there's a break break.

17:12 I'm sure there's places where they've had to change this, the normal stuff as well, just

17:17 to keep get that ready for that.

17:19 So I encourage people, even if you're not planning on testing any part of the GIL versus

17:24 with free threading to test 3.13 anyway, just to make sure that there are no surprises with

17:30 your project and report them if there are.

17:32 Last thing, I have been fairly quiet on the Python test podcast recently, but there is

17:38 a an episode that came out the other day, how to get pytest to import your code under

17:43 test.

17:43 So 2.21 is out.

17:45 These are not going to come out very regularly, but occasionally they'll drop.

17:49 So I'll mention them.

17:50 And yeah, there's a new one.

17:52 So that's those are my extras.

17:54 Yeah, cool.

17:55 What's very excellent about podcast clients is even if something doesn't come out for

18:01 six weeks, when it does, boom, it just shows up right in your list.

18:04 Yeah, people will know.

18:06 Do you have extras?

18:07 I do.

18:08 I have just one quick bit of follow up.

18:11 So remember, Brian, we talked about the higher order company and we talked about bend, a

18:16 programming, a parallel programming language that is Python like it looks like Python.

18:22 That's the thing.

18:22 I feel like I maybe gave the impression that it is Python.

18:27 It's not Python.

18:28 It's just Python like.

18:31 So you could look at code and say, yeah, that is Python, but you might not be able to import

18:34 a bunch of things or there's certain things you can't do.

18:37 Right.

18:37 So it's a big time subset and maybe not exactly even a subset.

18:42 Right.

18:42 Anyway, I got some some follow up from Bernard Gabor.

18:46 I just wanted to read just so because I think it sounds like he's got more experience than

18:50 I do for sure.

18:51 So it says bend looks roughly like Python, but is nowhere, nowhere there actually.

18:57 For example, it has no for loops.

18:59 Instead, you're meant to use the bend keyword, hence the language name to expand calculations

19:04 and another keyword to join branches.

19:06 So basically think of something that resembles Python at a high level, but without being

19:11 compatible with that and without any of the standard library or packages that the Python

19:15 language provides.

19:16 That being said, it does an impressive job at parallelization, but essentially it's a

19:20 brand new language with new syntax and paradigms you'll have to learn.

19:24 It just shares the first look similar to Python.

19:26 So, yeah, if you want to check that out, I'm sure it's awesome, but it's going to be its

19:31 own kind of thing.

19:32 Yeah.

19:32 Yeah.

19:33 That's not funny, but you know what is funny?

19:35 What's funny?

19:37 Do while loops.

19:38 They're hilarious.

19:39 Do while loops are hilarious.

19:40 All right.

19:40 Describe what's going on in this picture without maybe the words here for us, Brian.

19:45 People dancing at a party with somebody in the corner looking at them.

19:48 Yeah.

19:48 So there's a couple, there's like a crowd of people dancing, having fun, just foot loose,

19:55 carefree, having a good time.

19:56 And there's someone in the corner with a party hat and a drink, but the most serious look on

20:00 their face as if the world is about to end and the people in the room don't know it.

20:05 And the quote says, they don't know today.

20:07 I finally ran into a situation that required a do while.

20:11 They're just going about it like life is normal.

20:16 And here it is, the do while.

20:18 We have do while in Python?

20:21 I was just thinking that I think no, but we have while else, which you could probably

20:26 end into some weird, I don't know.

20:28 I don't really think you ever need to do while.

20:30 I don't know why languages have this.

20:32 Amazing.

20:36 But it's a funny joke nonetheless.

20:38 Yeah, definitely.

20:38 Nice.

20:41 Do you know what's not funny?

20:44 It's the end of the podcast.

20:46 Oh, it is the end of the podcast.

20:48 I have to say goodbye, Brian, to everyone.

20:51 Until next week.

20:52 Yes, until next week.

20:54 Thank you, everyone, for listening.

20:55 We always appreciate it.

20:57 Thanks.

20:58 Bye.