Transcript #318: GIL, How We Will Miss You

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:06 This is episode 318, recorded January 10th, 2023.

00:11 I'm Michael Kennedy.

00:12 And I'm Brian Okken.

00:13 And this episode is brought to you by Microsoft for Startups Founders Hub.

00:17 Thanks to Microsoft for again supporting the show.

00:19 And I, Brian, just want to let people know real quick at the beginning here that they should join us here on YouTube.

00:25 Like right now, live if they want.

00:27 They could go to pythonbytes.fm/stream slash live or just go to the website and click live stream at the top.

00:32 We typically record on Tuesdays at 11 a.m. at Pacific Time.

00:37 And you can also see the older video versions.

00:39 People want to find them there.

00:41 So it's always nice to have people in the audience.

00:42 And we also put information in there about future episodes.

00:46 So if for some reason it changes, we keep that up to date.

00:49 Or you do.

00:50 Yeah, exactly.

00:50 Usually what I'll do is I'll just schedule whatever time we're going to record a week in advance.

00:55 And YouTube will notify you if you smash the bell, you know, that sort of thing.

00:59 Smash the bell.

01:00 Yeah.

01:00 What do you got to kick us off with today?

01:02 Well, I want to start with something like right out off the presses.

01:06 So we have 703.

01:08 This was suggested by Will Shanks.

01:11 And the author is Sam Gross.

01:14 And it really was the PEP was created just yesterday, January 9th.

01:19 Or at least.

01:20 Yeah.

01:20 So that's pretty cool.

01:21 So very recently.

01:23 That new PEP smell.

01:24 Yeah.

01:25 It has that new PEP smell.

01:26 And what also one of the things that the PEP thing, peps.python.org has a like a dark mode

01:36 and light mode thing.

01:37 I didn't know that.

01:38 I was playing with that today.

01:39 Anyway.

01:40 So the PEP 703 is making the gil optional in CPython.

01:45 I knew that there was talk about this.

01:48 I heard Guido talking about it.

01:50 Possibly going in 312.

01:52 But I didn't know if it really was.

01:55 And it does.

01:55 I mean, it's still in draft stage.

01:57 So it's not it's not promised yet.

01:59 But it's it's targeting 312, which is the one that's being worked on right now.

02:04 It's pretty exciting.

02:04 So, yeah, a couple of things that jump out from reading just the title.

02:08 Optional, not removal.

02:10 Yeah.

02:11 OK, that's interesting.

02:13 I have.

02:14 Like you said, it's still still brand new.

02:16 So I haven't got a chance to research this a lot yet to understand the nuances of what that is.

02:20 But, you know, a lot of the prior talk, we had the Gilectomy and other things like that.

02:26 Eric Snow's subinterpreters, all of those were about how do we remove the gil or effectively remove it.

02:32 And this is like, well, well, not really.

02:36 It is not really optionally selectable while you're running.

02:39 It's a the option is a compile time flag.

02:42 So, yeah, this is and we probably should link to that that that really long interview with Guido.

02:49 But there's a there's an interview with Guido Van Rossum that's that in it,

02:54 talks about some of the guild challenges.

02:57 And it's an interesting bit conversation anyway.

03:00 So there's a flag, a --without Gil flag that's going to be added for compiling CPython.

03:06 So and when I first heard this, I'm like, well, I don't really want to compile CPython.

03:12 I'm I mean, I could, but I'm not that kind of guy that likes to do that in their free time.

03:18 So we have good news.

03:19 I'll share with that with you later.

03:20 But the the idea around this is in there's a lot of a lot of interest because there's a lot more scientific computing and machine learning and all this sort of stuff that's running on Python.

03:34 Now that wasn't before or just I don't know what it was doing before.

03:38 But there's a lot of there's a lot of interest in this.

03:40 And scientific computing especially has a lot of concurrency that you can speed things up dramatically.

03:47 If you can run if you've got like these huge vectors that you can all compute at the same time.

03:52 So there's there's ways around it.

03:54 We have some really fast libraries that can do this.

03:56 But these are like engineering feats that people have to do to make this all work.

04:01 And they're usually in C API extensions and you and whatnot.

04:05 So one of some of the motivation is that the there's a there's a lot of obstacles for concurrency because the GIL is in place.

04:13 So what do we do?

04:14 So the I'm going to jump to the the the distribution part, which I'm pretty excited about distribution.

04:22 The distribution is going to be easier than I thought.

04:25 So I thought people are going to have to build it themselves.

04:27 It doesn't look like that's the case.

04:30 It looks like Sam is going to work with Anaconda to distribute a without GIL version of Python together with compatible packages.

04:40 And I imagine so this will take time.

04:42 One of the things that's is going to be important is to get this out is like quickly so that people can start building packages.

04:49 Because in one of the reasons why I'm glad this is like almost a different distribution.

04:54 I know this is going to be a lot of work, but people can't really use it until the packages that.

05:00 That like allow them to use it, you know, can are there straight pythons.

05:05 The C layer might if you're integrating with C or something, it might assume that there's a GIL and not the safety it needs to do or something.

05:13 Yeah, right.

05:13 And even even if there's no changes that have to be made, they're going to be they're going to be have to these packages are going to have to be recompiled.

05:21 But there probably are changes they have to make, but they're trying to keep it to a minimum.

05:25 But anyway, they'll at least have to get recompiled and they'll be not binary compatible.

05:30 With both versions of python.

05:31 So we'll have this to python world for a little bit, but it's really targeted.

05:35 And I think going through anaconda is a good way to do it.

05:38 It's targeted to people developing these C API extensions so that they can start developing it.

05:45 So that's pretty, pretty neat.

05:46 And this collection of packages that already support the GIL lists python will grow over time.

05:54 And I don't know, I'm okay with having that be coordinated through anaconda.

05:59 It seems reasonable.

06:00 Yeah.

06:01 One of the things I loved about this article is it talked about some of the challenges.

06:05 So zooming really fast.

06:07 Apologies for people watching the video.

06:08 But there's a lot of discussion around the motivation of all the lot of the difficulties of different types of parallelism that you are hard to express with with the GIL in place.

06:19 And it's just an interesting, this part's an interesting read.

06:24 There's effects on Python library usability.

06:27 There's effects.

06:28 You want to be able to GPU heavy workloads require multi-core processing to coordinate the whole thing together.

06:35 There's a lot of stuff that we've done engineering wise to work around the GIL.

06:41 And it's kind of exciting that this is, I didn't think, I actually didn't see this happening this quickly.

06:47 So this is pretty exciting.

06:48 Yeah, this came out of the blue for me too.

06:50 The, one of the things that the discussion, oh cool, I found it.

06:55 The, what's going to change?

06:57 So I had this surprise to me.

06:59 The surprise to me was that the, there's supposedly going to be relatively few changes to the public Python and C APIs.

07:07 So this isn't like a moving from Python two to Python three, you have to rewrite your Python code.

07:12 Hopefully you won't have to change your Python code at all.

07:15 It's going to be some of these extensions.

07:17 And these are, I mean, like the people writing numpy and things like that.

07:22 Pandas, they might, they're probably going to have to do some changes and pillow and, and, you know, types or what, like pie torch and things like that.

07:31 They really care about this and they're going to be working with it, but they, they're the ones that didn't want the GIL there.

07:37 In the first place.

07:37 So I think they're motivated to make these changes.

07:40 It's pretty exciting, but okay.

07:41 So what, what is changing?

07:43 And a lot of the internals of CPython.

07:44 So there's, and I think this is a fascinating thing just as a, like a computer science discussion of, of this is a big problem.

07:53 What are the problems?

07:54 Well, there's, and how do we, what are the changes to solve it?

07:57 And there's reference counting and memory management, container thread safety.

08:01 Like if you've got multiple, multiple threads using the same list, how do you deal with that?

08:06 Locking and atomic APIs.

08:08 And there's a great discussion in here on all those.

08:11 So I really loved this just as a refresher of some of these CS topics and how they're still relevant in, in today's programming.

08:18 So it's pretty cool.

08:19 Yeah.

08:19 Yeah.

08:20 The container one's pretty interesting, especially with Python's extreme dependency upon dictionaries.

08:27 Yes.

08:27 Even being like the foundation of classes and stuff.

08:31 The reference counting is the one that's always been held out as a holdup, right?

08:35 Like, well, we have reference counting and that can be so fast because we don't have to do any locking or any thread safety around it.

08:41 As soon as you remove the gill, all that stuff goes out the window.

08:45 It's, it talks about a couple of building on, on one of the things that was done previously, which is pretty interesting.

08:52 And, you know, Sam Gross behind this and a bunch of other folks have just put together a whole host.

08:57 You know, a lot of this comes out of Cinder, I believe at Meta.

09:00 There's a lot of small building blocks.

09:03 This is not just like, well, here's the change, right?

09:05 And one of the things that I saw in the reference counting section is immortal.

09:11 I think it was immortalizing.

09:13 Oh, yeah.

09:13 Right.

09:14 So one of the things was, can we make it so they basically, we're pretty sure that we don't want to ever remove these.

09:21 So we're just going to take them out of the reference counting checks and just assume that they're going to be here for the life.

09:27 Of the program.

09:28 Yeah.

09:28 Just for performance and memory caching and all that kind of stuff.

09:33 However, you also don't have to lock something that you're not incrementing and decrementing because you've, you've decided not ever to do that.

09:39 Right.

09:40 Right.

09:40 So there's like some interesting piling on effects.

09:43 And also that was talking about, I didn't, I don't remember the word, but something about a certain, a lock free thread, thread safe type of way to do the increment for the.

09:53 Well.

09:54 For the reference counting.

09:55 I mean, I was familiar with just like brute force reference counting.

09:58 Of course, it's a, it's a common way to deal with shared objects.

10:02 But the, one of the things I loved about this is the discussion around biased reference counting and deferred reference counting.

10:09 The notion that like statistically looking at how multi-threaded applications work.

10:14 Most objects really are only like most, most things are only, most objects are only accessed by one thread.

10:22 There's not that many objects that are shared.

10:25 So having like a, each object having a thread owner and the, the process for what reference counting is for the owner is different than the process for the, or, or different, you know, the non-owner, non-thread owner, you know, people accessing the object.

10:41 And, and all of this is, hopefully invisible to, the people using it.

10:47 I mean, clearly at some point we're going to have to have like some way in Python to be able to utilize this is like, it should, it, it can't be invisible forever.

10:57 so it'll be interesting.

10:59 Yeah.

11:00 Yeah.

11:00 The bias reference counting, that's the word I was looking for.

11:02 That's, that's pretty interesting.

11:04 Like the, the owner thread might not have to lock it.

11:06 It just does whatever.

11:07 And the others have to care.

11:09 Well, and this also, a lot of this is around, I mean, we do reference counting, but partly because of the reference counting and how our object model is.

11:18 I mean, one of the things I was surprised by when I moved from C++ to Python or picked up Python and with objects and everything is that like in C++, you totally know when the destructor happens.

11:30 But in Python, we don't really talk about destructors too much.

11:34 We have constructors, but we don't really put a ton of work in destructors because you don't really know when it's going to really go away.

11:39 and, I think that's a good thing because now we can say it's actually going to be a little bit longer.

11:46 You thought it was going to, you know, it probably would have disappeared earlier, but we're going to defer that and have a garbage collection phase where it also includes, making sure that the threads are concurrency is saved and stuff like that.

11:58 So.

11:59 Yeah.

11:59 Interesting.

12:00 It's a, it's a very fascinating read.

12:02 I just had a quick read of it this morning, but, uh.

12:05 It looks very clever and I'm, I'm all for it.

12:07 I'm here for it.

12:08 Let's do this.

12:08 I don't want to like paint it too, Rosie.

12:11 And everybody paying attention to this knows that there is a performance hit, for single threaded stuff.

12:17 So, and I thought it was going to be larger.

12:19 the reference in here talks about somewhere between a five and 10% performance hit, but we've already seen these like massive gains in performance.

12:29 So, and the, the work on improving the performance of Python is going to continue.

12:34 So I personally optimistic.

12:36 I think that the, the gains in, in performance elsewhere are going to far outweigh the possible slowdown of, you know, single threaded stuff.

12:46 I mean, a lot, a lot of, a lot of people don't really care about 5% difference.

12:51 I mean, the code they're running, that's more simple.

12:54 It, it's probably not performance sensitive a lot of the times.

12:57 And when it is, you, you might be trying to squeeze even more performance out of it.

13:02 Right.

13:02 And, you know, my little mini here has eight cores.

13:05 Python only cares about one of them.

13:07 Yeah.

13:08 If I could use all eight or six or something, I could do way more than 5% better, you know, so I could more than make up for that if I actually leverage this feature.

13:17 Yeah.

13:18 And if I don't care about performance plus or minus 5% probably doesn't matter.

13:21 Yeah.

13:22 I don't, I don't, I don't, I don't, I think they have to bring it up because there's going to be people saying, ah, but it's going to be slower.

13:28 And so I'm glad that.

13:29 Yeah.

13:29 And that's also, that's always been the reason we still have the GIL is it's like, we would like to make changes, but they make a con the most common use case less good.

13:38 So we're not doing it, but maybe this is close enough possible improvements even down the road.

13:43 So yeah.

13:43 I'm just, it's going to be, I'm excited to see how this, how this pans out.

13:48 So yeah, I'm excited as well.

13:50 Very, very cool.

13:50 I would say certainly the biggest news of the show is right here.

13:56 And we've got some good jokes and stuff.

13:58 People can stick around for those.

13:59 They should.

14:00 Yeah.

14:00 Stick around to the end.

14:01 What do you got for us next?

14:02 All right.

14:03 Well, I want to tell you about the ferret.

14:06 You heard of ferret DB?

14:08 I have not.

14:08 Like a mongoose, but it's not quite a, I honestly, if you showed me a mongoose and a ferret and said, which is which?

14:15 I don't know if I could tell.

14:16 Maybe the ferret mongooses are more intense.

14:18 I don't know.

14:19 But ferrets are more pet-like.

14:20 They're pretty similar, right?

14:21 Is Mongo from mongoose?

14:23 No, it's from humongous.

14:25 But there are some frameworks in the JavaScript space called mongoose from the Node.js side.

14:33 Anyway, so ferret DB is this thing that I think someone told me about.

14:38 Let me see if I got some credit.

14:40 Yeah, this comes from John Boltmeyer.

14:43 Boltmeyer, sorry.

14:44 And thanks for sending that in.

14:45 It said, ran across this.

14:47 So ferret DB is pretty interesting.

14:49 We'll see where it goes.

14:50 It's a truly open source mongo DB alternative.

14:54 By the way, I'm going to accept the cookies.

14:57 You know, a cool way you don't have to have cookie policy pop-ups is not to track other people.

15:02 That's even better.

15:03 But I'll go and accept it for them.

15:05 Our site doesn't have any, does it, Brian?

15:08 Because we don't have third-party cookies.

15:09 So 5,000 GitHub stars already.

15:12 That's pretty incredible.

15:14 And these folks are big fans of MongoDB and also really big fans of 100% true open source.

15:22 And that's why they decided to make this thing called ferret DB.

15:26 So here I'll read from their page a bit.

15:27 MongoDB is a life-changing technology for many developers, empowering them to build applications faster than using relational databases.

15:35 However, MongoDB abandoned its open source roots, changing the license to SSPL, making it unusable for many open source and commercial projects.

15:44 Now, MongoDB is still open source, but this is like, I'm pretty sure this is a license.

15:48 So basically, you can't make our thing as a service and then sell it without us.

15:53 Like AWS had a MongoDB as a service.

15:56 And Mongo is like, hello, we want to be, we didn't expect it to be used in this way.

16:02 So I don't think it's, I mean, it's not like a huge bash against Mongo, I don't really think.

16:07 But if you really wanted it to be unencumbered entirely, well, MongoDB is not it, right?

16:12 Right.

16:12 So here's the idea.

16:14 So ferret DB is built on top of Postgres SQL, but it is wire compatible with MongoDB.

16:22 That is, you can use the different libraries or drivers from different languages, you know, Python, JavaScript, .NET, whatever.

16:31 You should be able to just change the connection string over to the ferret DB and off it goes.

16:36 Wow.

16:37 And it exchanges the binary JSON encoding across the wire, just like MongoDB does.

16:44 I'm guessing they're using the JSON column stuff in Postgres SQL to match.

16:51 I couldn't imagine they're trying to do a relational thing over there.

16:54 They're probably just like stashing the blobs.

16:56 But yeah, it's pretty neat.

16:58 What do you think?

16:59 That's actually, the architecture of that sounds great for an open source project to not try to do the entire database backend, but built on top of Postgres.

17:08 Yeah.

17:09 I mean, there's Edge, there's EdgeDB, right?

17:12 Which is quite popular in the Python space.

17:15 And I believe they're doing basically the same thing.

17:17 Like build something.

17:18 Yeah.

17:18 Powered by Postgres.

17:20 Maybe just the query engine.

17:22 I'm not sure if the database engine as well for the EdgeDB folks.

17:25 I interviewed them, but still pretty, pretty neat.

17:27 I can get ahead with the joke.

17:28 Yeah.

17:29 So, FairDB, formerly MangoDB, is an open source proxy that translates MongoDB wire protocol queries to SQL with Postgres as a Postgres SQL as the database engine.

17:40 MangoDB.

17:41 That's a little bit too close, I think.

17:43 Yeah, a little bit.

17:44 Mango.

17:45 Yeah, that's maybe pushing it.

17:47 So, yeah, you pretty much write the regular MongoDB code that you would write, but it apparently works.

17:53 Now, somewhere, I don't know where the roadmap is.

17:56 Let's see.

17:57 They did a blog post.

17:59 Yeah, maybe that's where I should be looking.

18:00 And on the blog post, they have something about their roadmap.

18:04 I don't know.

18:04 I'll link to it if I can find it.

18:06 But there's a FairDB roadmap, and there's one really notable thing that people are going to need to be holding out for, and that's indexes.

18:16 Oh, okay.

18:17 And so if you don't have indexes, you have a real hard time doing fast database stuff.

18:24 So I think they said they're trying to get it kind of all finished up by the end of 2023.

18:29 So this is not like a, hey, you should just jump right on this and run with it.

18:34 Because I don't think it's far enough along.

18:37 It's written in Go, for what that's worth.

18:39 MongoDB itself is written in C, so it's not like it makes it more or less friendly to Python folks.

18:45 So, oh, here's the public roadmap.

18:47 Perfect.

18:47 So it's linked off their GitHub pages.

18:49 And so you can see the end of alpha is supposed to be in the past, and it does look like they're all done.

18:54 You can see their beta stuff they're working on.

18:57 But then for general availability, this index stuff that's supposed to come.

19:01 Actually, I said 2023.

19:03 This is supposed to be done January 2023, which felt like a long time ago.

19:08 Not long ago, but that's actually now.

19:10 I don't know how far along it is, but support for issues, for indexes.

19:15 Look, this issue here looks, I don't see a lot of traffic on it, but still, it would be basically required, right?

19:22 If you've got many, many gigs of data, then you have to have indexes.

19:27 Right.

19:27 However, the devil's advocate here, development work on a project that utilizes this can proceed before you get tons of users and stuff, knowing that it's coming soon, I guess.

19:42 Yeah, that's a good point.

19:43 If you trust him.

19:44 Yeah, although because it's wire protocol, you could just run MongoDB and then flip it theoretically, right?

19:50 You don't have to.

19:50 It's not like a complete new creation you've got to build upon, right?

19:54 It's meant to mirror something that exists.

19:56 So you could do that.

19:58 I think we've got like five, six, seven gigs of data in the database that backs Python bytes.

20:05 Like if you do queries against, you know, give me the 20 things out of this, these millions and millions of records that match this without an index.

20:12 Actually, I think MongoDB may refuse to do it.

20:17 There are certain situations where it's like too much data, no index, not going to be able to answer that question.

20:23 So it might be really slow or it might actually error out and say this, you need to create an index.

20:28 So anyway, long story short, like this is a big deal until this comes in.

20:33 I would not touch it in terms of a, I'm going to try to run on top of this.

20:37 Yeah.

20:38 But if that, if that comes through, then you already have Postgres, but you want some kind of document like experience for part of your app or one of your apps, but you don't want to switch to Mongo.

20:49 That would be pretty cool.

20:50 So one of the things notable in the menu bar of FairDB is careers.

20:57 So this is not, this is intended to be something commercial at some point, unless they just feel like hiring people for the fun of it.

21:05 Yeah, that is interesting.

21:07 So where are they going with it?

21:09 Right.

21:09 So I would imagine they have a FairDB as a service in mind out there, but you're right.

21:17 It's certainly, it has an ink and.

21:20 I mean, that's not necessarily bad.

21:22 That's probably a good thing that there's money behind it, but just interesting.

21:25 It is interesting.

21:26 Yes.

21:26 Very interesting.

21:27 All right.

21:27 Well, FairDB, something I put out there for people to keep their eye on, not necessarily to jump on right away, but quite interesting.

21:36 Yep.

21:36 Oh, by the way, we have a couple of excited folks out in the audience.

21:42 A GIL optional would be amazing, says Ciro.

21:44 And I know that Ciro is working on some real high performance web framework stuff.

21:49 So that would be cool.

21:50 And how about a Docker container prebuilt with CPython with no gill?

21:55 Pick your Docker container and run that one.

21:57 Sure.

21:57 That'd be cool.

21:58 Yeah, that's not a bad idea.

21:59 Yeah.

22:00 And Seth out there says, good thing we invested in faster CPython earlier.

22:04 So now we can roll back a little.

22:05 Yeah.

22:07 I mean, kind of, we knew that was coming.

22:08 So that's one of the reasons for the speed ups in 3.11 and planned for future.

22:12 So.

22:12 Yeah, indeed.

22:13 All right.

22:14 Let's switch over to real quickly talk about our sponsor before we move on here.

22:20 As I said at the beginning, this episode is brought to you by Microsoft for Startups.

22:23 We just got an email.

22:25 I don't know if you saw, Brian, from somebody who signed up for the Microsoft for Startups

22:28 program and said, this is a really great.

22:30 I was a little bit skeptical, but I signed up and this is a really great project, a really

22:34 great service and offering for them.

22:37 And that they actually, it's working really well.

22:39 So.

22:39 That's great.

22:40 Yeah.

22:41 It was really good to hear.

22:42 So, you know, Microsoft knows that starting a business is hard.

22:46 They want to help support startups that can then later, you know, be customers of theirs,

22:51 of course.

22:52 So they put together this Microsoft for Startups program to help them understand what startups

22:57 need to help them be more successful and to create a digital platform to help them overcome

23:02 those challenges.

23:03 And they call that Microsoft for Startups Founders Hub.

23:05 And it provides all founders at any stage with free resources to solve startup challenges,

23:11 provides tech benefits, access to expert guidance and skilling resources, mentorship, network

23:17 connections, and much more.

23:19 It doesn't require the startups to be investor backed or third party validated.

23:22 You just apply.

23:23 Microsoft says it looks like you would be a good fit for this program.

23:26 You're in.

23:27 And with that, you get access to a bunch of cloud resources, GitHub and Azure, Office 365.

23:33 I think you also get access to open AI, which, you know, chat GTP, pretty hopping these days

23:40 and might be something cool to leverage.

23:41 And through Microsoft for Startup Founders Hub, becoming a founder is also no longer about who

23:47 you know.

23:47 You have access to their mentorship network, which I think is one of the biggest values you

23:51 probably get.

23:52 Talk to people about idea validation, fundraising, management, coaching, sales and marketing, and

23:57 all those things.

23:58 And you get to book one-on-one meetings with those mentors to make your idea a reality

24:03 today with the critical support you'll get from Microsoft for Startups Founders Hub.

24:06 To join the program, just visit pythonbytes.fm/foundershub 2022.

24:11 The link's in your show notes.

24:12 So if you're interested, give it a click.

24:14 All right.

24:16 Over to you, Brian.

24:16 Well, I was, I want to guess transition is difficult for me apparently right now, as are

24:25 words.

24:26 Anyway, the next article I want to talk about is four tips for structuring your research

24:31 group's Python package.

24:33 So I think this is just a Python package, but research group, I think this is from David Aaron

24:40 Nicholson.

24:40 And I think the idea around it is a lot of people with scientific computing and various things

24:48 like that might not be, might be struggling with lots of Python code.

24:53 So they're putting it into subdirectories and stuff like that.

24:58 So this is talking about packages in that sense of it's a directory with a dunder init and you

25:04 can reference it by importing it through normal import.

25:08 The import mechanism, it's not about packaging for like IPI or something like that.

25:13 It's still some really great information.

25:15 So the four tips are one, give your packages and modules terse single word names whenever

25:22 possible.

25:23 And there's some great examples of what not to do later.

25:26 Number two, import modules internally instead of importing everything from modules.

25:31 Make use of sub packages.

25:34 And the last one is prefer modules with specific names instead of things like utils, helpers

25:41 and support.

25:41 And this is, this is everyone struggling with this, including me, even though I've been preaching

25:46 this for a long time.

25:47 It's hard.

25:48 There are some great examples of what not to do.

25:52 So the, the, what was this?

25:54 The give your modules terse single word names whenever possible.

25:57 And I guess there's a couple examples of what not to do.

26:00 So there's a really long directory name called electromyography toolkit.

26:07 Yeah, that's a bit long.

26:08 And then prefixing everything with EMG afterwards within it.

26:13 Yeah, don't do that.

26:14 You don't need to.

26:15 It's just some great examples of better versions.

26:18 Right.

26:18 It already has the, the, the top level part of the package as the namespace.

26:23 Yeah.

26:24 If you want to be explicit, just, you know, package dot sub package.

26:28 And so they're talking about how, how, what it looks like when you're importing it is different.

26:34 So I'm going to scroll down of like importing stuff.

26:37 I guess I'll just drop to one.

26:40 A better thing would be like EMG toolkit.

26:43 And I would say even like, is toolkit really adding any information?

26:47 Maybe just EMG or something.

26:49 And then having, instead of having like solvers and a whole bunch of prefixes, just put solvers

26:56 in a directory name and then put the types of solvers underneath.

26:59 You don't have to have EMGs under everything or yeah.

27:03 EMG underscore.

27:04 Or a brute force solver and a, a star solver and all that.

27:08 It's just solvers dot a star solvers dot brute force or something.

27:11 Right.

27:11 Yeah.

27:11 Right.

27:12 And I'd argue that you could even just probably drop the toolkit and it'd be EMG dot solvers dot

27:16 brute force.

27:17 It'd be a better name.

27:18 But I think, so I'm not going to go through this, this entire thing, but import modules is

27:25 something I've seen.

27:26 A lot of people will do like a from package solver import and then list all the things that

27:31 they're importing.

27:32 And I think, and I've been guilty of this to doing this before also, especially if I'm

27:38 only importing one.

27:39 One of the things that people will do this for is because the entire name is too long.

27:45 Well then, you know, make the name shorter.

27:47 So instead of doing that, just import the package solver and then you dot reference everything

27:52 else.

27:53 Right.

27:54 Or do what the data scientists do.

27:56 If you don't control the name, you know, import numpy as NP.

28:00 Yeah.

28:01 Then NP dot that you can shorten the name for you, even if you can't shorten it for others

28:05 at the source.

28:07 Right.

28:07 So one of the reminders is you can do from, if you're in the same directory, you can do

28:12 from dot import solvers, and then you can reference like solvers and entropy solver and things

28:17 like that.

28:18 Anyway, some great, even discussion of sub packages of, and this is something that I

28:25 always forget that is difficult.

28:26 It's a, it's a mind shift to say, oh, I'm going to throw things into a directory and then

28:30 have a package, but you can, you can do that again.

28:33 It can, it can keep going down and, and some people forget to do that.

28:37 So the reference is that as well.

28:39 So it's good.

28:40 You can have packages inside of packages inside of packages.

28:43 Nice.

28:44 I think Robert out in the audience, that's a great recommendation to simplify the code

28:48 we got to write.

28:49 Yeah.

28:50 Just how about we just extend it a little bit from star import star and like you just have

28:54 access to everything.

28:55 There's sure the startup time is 10 minutes, but it's fine.

28:58 Yes.

29:00 And then Pampho Roy is reminding us that, you know, one of the Zen of pythons is about these

29:05 namespaces.

29:06 Yeah.

29:06 Yeah.

29:07 And I think it's just also be nice to people.

29:10 Just, you know, people trying to write, write their code in Python.

29:14 They're not doing weird stuff because they want to do weird stuff.

29:17 They just don't know what the normal way to do it is.

29:20 So gentle reminders like this post are good.

29:22 Indeed.

29:22 What you got for us next?

29:24 I don't want to really quibble about it, but let's talk about it anyway.

29:28 Quibbler.

29:29 So quibbler actually is, is interesting keeping with the data science theme.

29:34 So this one here is a way to take matplotlib and make it more interactive.

29:41 So you probably know matplotlib will make a nice, like printable, publishable, high quality

29:48 image, but they're generally not as interactive as like bokeh, some of these other frameworks.

29:53 So quibbler is like your data, but interactive.

29:56 Let me just play a little video for you here and I'll just describe it.

29:59 But this way you'll kind of know what this is about if you haven't checked this out.

30:03 So the idea is you could go and plot something with matplotlib and you have some variable

30:09 you would like to make interactive.

30:10 You just wrap it in an iQuib and all of a sudden the entire UI becomes completely interactive,

30:18 which is insane.

30:19 So here's like a triangle with squares around each, based off each of the edge of the triangle.

30:26 And then if you just make the numbers, the inputs, this iQuib type, then all of a sudden

30:31 you can drag them around and interact with them.

30:33 And even, even works on like cells and zooms and other pictures.

30:38 What do you think?

30:39 This is amazing.

30:40 Also amazing use of video to show people how easy it is to switch.

30:44 Yes.

30:45 Thank you for doing that.

30:46 Thank you for doing that people.

30:47 But yeah, this is a super cool project.

30:50 I don't, I don't have a ton to say about it actually, but you know, if you're working

30:54 with matplotlib already and you're like, you know, what'd be really nice is if we could,

30:58 you know, move some of these things around and create interactive graphics a little bit

31:02 more than, you know, just wrapping the independent variables in iQuib interactive bits.

31:08 And it turns the whole, the whole graph into something that's, that's pretty excellent.

31:12 I, you know, I think this is, it looks great.

31:14 I don't see why you wouldn't, wouldn't want to use it.

31:17 Yeah.

31:17 That's, that's, that looks fun.

31:18 Yeah.

31:19 Cool.

31:19 And someone out of the audience says it loves Quib.

31:21 So they must have experience with as well.

31:24 All right.

31:24 So if you were doing matplotlib and you want to make it more interactive, check this out.

31:28 Awesome.

31:28 It's all of our items, isn't it, Brian?

31:30 Even though somehow I seem to have like a screen full of tabs still.

31:33 I don't know how that works.

31:34 Yeah.

31:34 So you've got some interesting Python related stuff.

31:37 And I thought I would go with the tradition of Python and say, and now for something completely

31:43 different.

31:44 I just ran across an article that I thought I'd share just quickly.

31:47 I'm not going to talk about it a lot.

31:49 But there was an article in Washington post about how turtle moms talk to their eggs before

31:55 they hatch.

31:55 And I just thought it was a fascinating read.

31:58 Plus it's got pictures of turtles.

32:00 So turtles are awesome.

32:02 I wasn't going to talk about it too much, but one of the things I thought was really cool

32:06 was somebody went out and thought, oh, we don't, we, there's the turtle moms hang out in

32:11 the rivers or somewhere near the, the, the, the, the hatch, the eggs on the sand.

32:18 And they just wait for him or something.

32:19 Well, are they talking?

32:21 It probably not, but they went out and recorded anyway.

32:23 And it's lower frequencies, frequencies that we don't normally hear.

32:28 And, that's also slower timeframes.

32:31 Like a conversation can be like hours for a reply.

32:35 So it takes a while to go back.

32:38 And then also like some of the eggs talk to each other before they hatch.

32:41 they can, they can talk to each other to help possibly coordinate when to hatch.

32:46 Because if there's predators around, more of them will survive if they all hatch at the

32:51 same time.

32:51 So anyway, just cool article.

32:53 Check it out.

32:54 If you guys want to look at turtles.

32:55 Yeah.

32:55 Science is neat, isn't it?

32:56 Yeah.

32:57 Yeah.

32:58 That's really cool.

32:58 So, kind of not one of my extras, but I'll go through this in is the next coming talk

33:05 Python episode is, around the, the story of imaging black holes with, Dr.

33:11 Sarah.

33:12 I, I said, yeah, so that's going to be really, really awesome.

33:16 And then there's a good chance, not a hundred percent chance, but a good chance that I also

33:20 will have a science show on the few, the fusion, national ignition breakthrough with Python

33:27 as well coming up.

33:28 So very cool.

33:29 Very cool stuff.

33:30 Very neat.

33:31 You get to talk to so many cool people through that.

33:33 That's neat.

33:34 I know it's, it's, I would never get to talk to these people.

33:37 They're like, Hey, I have a podcast.

33:38 Let's talk about it.

33:39 Amazing.

33:39 All right.

33:40 let's do your extras.

33:42 Yeah.

33:42 Let's do my extras.

33:43 So here we go.

33:46 So I've published the thing on RSS a little while ago and talked about a lot about RSS.

33:51 So I have more RSS stuff because I said reader R E E D E R is a really cool thing, but it's

33:59 Mac only.

33:59 Other people are like, Hey, guess what else I use?

34:01 This is neat.

34:01 So there's a fresh RSS, which is like your own hosted RSS web app.

34:07 I think it also does some read later type of things.

34:10 Feedly.

34:10 I knew about Feedly, but I thought it was just, I don't know, just some sort of web

34:14 app, type of thing.

34:16 But apparently it has like a cool AI engine to help kind of recommend if you get like

34:20 overwhelmed with all the stuff you subscribe to.

34:23 You're like, well, now I can't read it.

34:24 It's just insane.

34:25 You know, it has a kind of a, your own personal algorithmic feed, which is, I know algorithmic

34:30 feeds often get a bad name, but there are benefits too, you know?

34:32 Yeah.

34:33 Have you used Feedly lately?

34:34 I have not.

34:35 I've been using it for years.

34:37 So I didn't know about the AI bit, but I, yeah, I use it a lot.

34:41 Yeah.

34:42 I thought that was the one you had said you were using.

34:43 So, but yeah, the, the AI engine sounds at least interesting.

34:46 Flim on, Android.

34:49 People care about that.

34:50 And read wise, we talked about, but I've been playing the read wise and it continues to

34:54 be interesting.

34:54 Cool.

34:55 Can, not only when I first thought of that, oh, RSS and, you know, maybe read later, but

35:00 you can throw in like EPUB books, you can do YouTube videos and like, it'll let you

35:05 annotate the transcripts that it makes.

35:08 And it's, it's pretty wild.

35:09 Wow.

35:10 and then, okay.

35:11 Going for more RSS here is, just published all the talk Python courses as an RSS feed.

35:19 So if you want to subscribe to that, then soon as we publish a new Talk Python Training course,

35:25 like if we go down here, there's a, pie test one somewhere getting started with pie tests.

35:29 Right.

35:30 So that came out a couple months ago.

35:31 And so people, if they're subscribed to the RSS feed, another one comes out, they'll get

35:35 it straight away.

35:36 It's cool.

35:36 Right.

35:36 That is neat.

35:37 Yeah.

35:38 RSS seeing more of the things over at talk Python, the podcast, not the courses.

35:42 I, we were talking about this right before the show.

35:44 I did a ton of stuff for sort of helping people find when were, was a particular person, a guest

35:50 on the show.

35:51 When were they on the show?

35:52 If they're on one episode, what other episodes did they do?

35:56 So here's a great, huge list of tons of people who have been guests on the show.

36:02 And Brian, who's in number one right now?

36:04 I am with Brett as a close second.

36:07 So a close second.

36:08 Yeah.

36:08 So Brian's been on 10 times.

36:10 Brett Cannon, nine.

36:11 Anthony Shaw, nine.

36:12 Dan Bader, six.

36:13 Lucas Lenga, five.

36:15 Cecil down there at five.

36:18 It's Cecil Phillip as well.

36:19 Anyway, if this is a new resource on the podcast.

36:22 So go, if you want to explore the data for the last seven years in a slightly different

36:26 way, people can check that out.

36:28 And congrats, Brian, for leading the pack there.

36:30 Yes.

36:31 And last, I talked about how to do traveling around, just taking an iPad and still doing

36:38 development work.

36:39 And we talked about prompt and all those things.

36:40 So I basically wrote up an article on that discussion I had there.

36:44 So I'll link to that.

36:45 People can check that out if they want to.

36:47 So there's the written, more formal version called Dev on the Road on my website.

36:52 Cool.

36:53 All right.

36:53 Ready for a joke?

36:56 Oh, by the way, Robert out there is suggesting that we might need to update your photo.

37:01 Now you got your cool hair.

37:03 We might need to update you on that guest list there.

37:05 That was a seven-year-old photo or something.

37:08 Yeah.

37:08 That's from when you were on the show the first time, maybe six years ago.

37:10 No, I really do look like that other photo.

37:12 What you're seeing right now is an AI-generated bot.

37:15 Yeah.

37:15 This is like a...

37:16 Actually, Brian hasn't been here for a few years.

37:20 This is a deep fake and I invented him.

37:22 No, just kidding.

37:23 All right.

37:25 Speaking of jokes.

37:25 The real Brian is still embroiled in a lawsuit to try to get this to stop.

37:29 The real Brian will never defeat the AI.

37:33 Okay.

37:33 So I thought of you actually, Brian, when I saw this one.

37:39 This is on Program and Humor.

37:42 And it says it has this dog-looking thing kind of very deep in thought projected to the sky for reasons I don't understand.

37:51 But it says, am I testing this program or is it testing me?

37:56 I suspect a lot of people can relate to having that go both ways.

38:00 Deep thoughts.

38:01 Deep thoughts.

38:02 And we've got a name for the AI.

38:04 It's called Brian GPT.

38:08 All right.

38:08 I know you got a joke as well for us.

38:10 Maybe more involved than the floating dog.

38:12 Okay.

38:13 I can't remember where I got this, but every cloud architecture.

38:17 It's just, I don't know.

38:18 It's an image with some stuff that I really...

38:22 There's a lot.

38:22 I'm not going to cover everything, but one of my favorite bits of this is the managed services,

38:27 the mismanaged services, and the unmanageable services.

38:31 They're held together by duct tape and like some kind of like super glue.

38:35 Yeah.

38:35 Containers everywhere.

38:37 I do like the cool databases that drive resumes and the real database that's actually doing

38:44 all the actual work.

38:45 The plumbing.

38:47 There's a huge bunch of plumbing around it.

38:50 It just has...

38:50 The data engineers got very excited here.

38:52 Yeah.

38:53 It empties into a data lake.

38:54 Which also drips into a data swamp.

38:57 And there's a blame radius.

38:59 Services that stop stuff.

39:02 There's one tiny cron job that keeps everything from falling apart.

39:06 It's good.

39:07 VPN of sadness.

39:08 And of course, it's all backed by the freaking slick reverse proxy.

39:14 Yeah.

39:15 So this is found on Good Tech Things, which actually has a whole bunch of cool, like funny images.

39:22 So I might steal some more of these from...

39:24 Oh, we might hear from Good Tech Things again.

39:27 This is pretty awesome.

39:28 All right.

39:29 Well, is that everything for us?

39:30 That is.

39:31 That is all of it.

39:32 So...

39:33 Nice.

39:34 Thank you everyone for listening.

39:35 Brian, thanks as always.

39:36 Bye.