Transcript #255: Closember eve, the cure for Hacktoberfest?

00:00 Hey there, thanks for listening.

00:01 Before we jump into this episode, I just want to remind you that this episode is brought to you by us over at TalkBython Training and Brian through his pytest book.

00:10 So if you want to get hands on and learn something with Python, be sure to consider our courses over at TalkBython Training.

00:17 Visit them via pythonbytes.fm/courses.

00:21 And if you're looking to do testing and get better with pytest, check out Brian's book at pythonbytes.fm/pytest.

00:29 the episode. Hello and welcome to Python Bytes where we deliver Python news and headlines directly to your earbuds. It's episode 255 recorded October 20th, 2021. I'm Brian Okken.

00:41 I'm Michael Kennedy. And I'm Will McGuggin. Welcome Will. Thank you. Good to be here.

00:46 I'm sure people know who you are through all you do with Textual and Rich. Could you do a quick intro? Sure, yeah, I'm a software developer from Edinburgh, Scotland. Last couple of years I've been working quite heavily in open source. I built Rich and I started work on Textual, which is an application framework using Rich and I'm currently working exclusively on that. So I've taken a year off and probably more than that to work on open source projects. I'm very excited - We're excited about it too.

01:21 (laughing)

01:23 - Yeah, that's fantastic, Will.

01:24 I think we've talked about this offline as well, the success you're having with Rich and Textual and this opportunity you have to really just double down on this project you created.

01:35 And I know there must be thousands of maintainers of projects out there, like if I could just put all my energy into this and you're currently lucky enough to be in that situation, right?

01:44 That's fantastic.

01:44 - Yeah, I'm very fortunate actually.

01:47 I mean, I put some money aside, I planned for this year, but things are really looking up and I've been blown away by the level of interest from it.

01:58 I mean, it gradually ramped up with rich people, people like that.

02:02 I think there was a missing niche or something which did that.

02:07 But then with the textual, people were excited about it.

02:10 I mean, I put a disclaimer on the readme that said it's not quite ready for prime time yet.

02:16 It might break and is in active development, but it doesn't seem to discourage anyone.

02:22 Very busy building things with it.

02:25 So I'm excited.

02:26 I want to take it to the next level.

02:30 And to be honest, if I was doing it part-time, like I was doing rich, it would just take too long.

02:35 If it was evening and weekends, it would be two years before it was like 1.0.

02:42 - Yeah, and we're ready to use it now, so.

02:44 Yeah.

02:45 Most people want to use it yesterday.

02:47 Congrats again on that.

02:50 That's cool.

02:51 It's great stuff.

02:52 You know, we've talked about over on talk Python and people want to dive in.

02:55 We've certainly covered it many times over here as well.

02:58 So, or happy to spread the word on it.

02:59 Yeah.

03:00 Well, Michael kicked off the topics.

03:02 I do want to kick it off.

03:03 All right.

03:03 How about we start with some awesome Python topic like C++.

03:07 I like both of them.

03:09 You, this is right in your wheelhouse, Brian, a lot of C++.

03:13 So I want to talk about this tutorial article series, however you want to think about it, of wrapping C++ code with Cython.

03:22 So the interoperability story with C and Python, being C, Python as the runtime, is pretty straightforward, right?

03:29 But C++ is a little more interesting with classes and this pointers and all those kinds of things.

03:35 So the basic idea is Cython is this thing that allows us to write very nearly Python code and sometimes actually just Python code, sometimes like in a little extended language of Python that then compiles down to C.

03:51 And if that's the case, well, it's probably pretty easy to get that Cython code to work with C code.

03:57 And then Cython naturally is exposed as Python objects and variables and whatnot.

04:03 So that should be a good bridge between C++ and Python, right?

04:07 And it turns out it is.

04:08 So this person Anton Zidane Pushkin wrote an article or is working on a series of articles called "Wrapping C++ with Cython." And so there's this library called Yarkirl, yet another audio recognition library.

04:23 It's kind of like Shazam.

04:25 It'll, you give it a small fragment of audio and it'll say, "Oh, that's Pearl Jam Black," you know, "Black by Pearl Jam," or something like that.

04:32 Right?

04:33 Pretty cool.

04:34 And if you look at it, it's got some neat C++ features.

04:37 You know, Brian, feel free to jump in on this, but see that right there?

04:40 Namespace, so cool.

04:42 I love how they're writing like well-structured C++ code here.

04:47 But basically there's a couple of structures, like a wave file and an mp3 file, and then classes, which have like a fingerprint and public methods and storage and so on.

04:56 And so the idea is how could we take this and potentially make this a Python library, right?

05:01 Basically create a Python wrapper with Cython for it.

05:04 So you're gonna come down here and says, Well, what we're going to do is we're going to write some Cython code.

05:09 And Cython doesn't immediately know how to take a C++ header file, which is where stuff is defined in C++, and turn that into things that Python understands.

05:21 So you've got to write basically a little file, a PXD file that declares what the interface looks like.

05:28 So you write code like this.

05:30 Have you done this stuff before, Brian?

05:31 No, but this looks pretty straightforward.

05:34 Yeah, it's pretty straightforward.

05:35 How about you, Will?

05:36 - I've never wrapped a library, but I've used Cython quite successfully, so it's a really good system.

05:41 - Yeah, yeah, I agree.

05:42 I've done it, but not to wrap C++ code.

05:44 - No.

05:45 - So basically you do things like cdef extern from this header, create a namespace, and then you have cdef, a keyword cpp class, and then you get, what's interesting about this is you get to give it two names.

05:58 You get to say, here's the name, I want to talk about it in Python, So CPP wave file, and then here's its name in C, which is yarrr-control-colon-colon-wave-file.

06:10 And the value of this is they wanna have a thing called wave file in Python, but not the C++ one, a friendly Python one, but it needs to use the wave file from the C library.

06:19 So if you directly import it, then there's like this name clash, which I suppose you could fix with namespaces and all, but I think it's cool that you can give it this name, this kind of this internal name, and off it goes, right?

06:30 So then you def out its methods basically like just here are the functions of the class.

06:36 Same thing for the fingerprint and the storage and off it goes.

06:39 And so all of this stuff is pretty neat.

06:42 And yeah, this thing I'm talking about is called aliasing, which is pretty awesome.

06:47 It lets you reserve the name, wave file and storage and fingerprint and stuff like that for your Python library without, even though that's what the C names are as well.

06:58 So yeah, pretty straightforward.

07:00 What was the next thing I really want to highlight?

07:02 There's kind of this long article here.

07:03 So the next thing they talk about is using this thing called extension types.

07:08 So an extension is just a C structure or C++ library and you create some, some class that is kind of a proxy to it.

07:17 So here we say cdef Python class called storage and then internal it has in Python language, you have to say cdef, it has a C++ class called this.

07:29 And then from then on, you just go and write standard Python code.

07:32 And anytime you need to talk to the C library, you just work with the inner pointer thing that you've created, which is pretty awesome.

07:40 You just new one up in the constructor, and the C++ thing, and then it goes off to Python's memory management, so you don't have to worry about deleting it, stuff like that.

07:50 I guess you do have to sort of deallocate it, but that's, once you write that code, then Python will just take it from there, right?

07:56 So pretty neat way to do this.

07:58 And the library goes on to talk about how you use it and so on.

08:02 So there's a couple of interesting things about like dereferencing the pointer, like basically modeling reference types in Python.

08:08 But if you've got a C++ library that you want to integrate here, I think this is a pretty cool hands-on way to do it with Cython.

08:15 - Yeah, I think this looks fun.

08:17 I'd like to give it a try.

08:18 - Yeah, definitely.

08:19 Another one is PyBind11.

08:22 That might also be another option to look at.

08:24 So I saw Henry out in the live stream there.

08:27 So here's another way to operate seamlessly between C++ 11 and Python.

08:33 So another option in this realm, maybe I'll throw that link into the show notes as well.

08:39 But yeah, a lot of cool stuff for taking these libraries written in C++ and turning them into Python-friendly, feeling Python native libraries.

08:47 - Well, and you know, that's really how a lot of Python's taken off, right?

08:51 is because we've been able to take these super powerful C++ libraries and wrap a Python interface into it and have them stay up to date.

08:59 When you make updates to the C and C++ code, you can get updates to the Python.

09:05 - So you sometimes hear Python described as a glue language.

09:09 I think many years ago, that's probably what it was.

09:12 I think Python's grown, it's more than just a glue language, but it's very good at connecting other languages together.

09:19 - It's still good as a glue language.

09:21 - Still, yeah.

09:22 It's not just a glue language, it's a language of its own, I guess.

09:26 - Yeah. - Yeah.

09:27 - I was talking to somebody over on Talk Python, and I'm super sorry if I forgot which conversation this was, but they described Python as a glue language for web development.

09:38 I thought, okay, that's kind of a weird way to think of it, but all right, said, well, no, no, look, here's what you do with your web framework.

09:42 You glue things together.

09:43 You glue your database over to your network response.

09:47 you glue an API call into that.

09:50 And I'm like, yeah, actually that's kind of is what a website is.

09:53 It talks to databases, it talks to external APIs, it talks to the network in terms of like HTML responses and that's the entire web framework.

10:01 But yeah, you can kind of even think of those things in those terms there.

10:05 - It's like a party when no one's talking to each other and you need someone to like start conversations is what Python does.

10:12 - Yeah, yeah.

10:13 And I think also that that's why Python is so fast for web frameworks, you know, even though computationally it's not super fast, like it's mostly spending a little time in its own code, but a lot of time it's like, oh, I'm waiting on the database, I'm waiting on the network, I'm waiting on an API, and that's where web apps spend their time anyway, so it doesn't matter.

10:31 All right, Brian, you want to grab the next one?

10:33 - Yeah, sure.

10:34 - Bump it on to topic two.

10:36 - Bump it on.

10:37 So I've got, I just have a few packages that I support on PyPI, and then a whole bunch of internal packages I work on. And one of the things that is a checklist that I've got is what do I do when I bump the version? And I know that there have been some automated tools before, but they've kind of, I don't know, they make too many assumptions, I think, about how you structure your code. So I was really happy to see tbump come by. This was suggested by Sefi Beri.

11:10 So tbump is an open source package that was developed.

11:13 Looks like it was developed in-house by somebody, but then their employer said, "Hey, go for it, open source." So that's cool.

11:19 The idea really is it's just bump versions and that's it.

11:26 But it does a whole bunch of cool stuff.

11:29 Let's say I've got to initialize it.

11:31 So you initialize it as a little toml file that stores the information in the configuration.

11:37 But if you don't want yet another toml file or another configuration that can also append that to the PyProject.

11:44 That was a nice addition.

11:46 You can combine them or keep it separate, up to you.

11:49 And so for instance, I tried it on one of my projects, and I kept it separate because I didn't want to muck up my PyProject.toml file.

11:56 But once you initialize it, all you have to do when you want to add and bump a new version is just say tbump and then give it the new version.

12:06 It doesn't automatically count up.

12:08 I mean, you could probably write a wrapper that counts up, but looking at your own version and deciding what the new one is, is reasonable.

12:14 That's a reasonable way to do it.

12:16 And then it goes out and it patches any versions you've got and then in your code, in your code base, or your files, or config files, or wherever.

12:27 And then it commits those changes.

12:31 It adds a version tag, pushes your code, pushes the version tag.

12:35 And then also, you could have these optional run things, places where before you commit, you can run some stuff.

12:40 For instance, check to make sure that you've added that version to your change log or if you want to check your documentation.

12:48 Then also you can have post actions.

12:51 If you wanted to, I was thinking, a post action would be cool.

12:54 You could just automatically tweet out, "Hey, a new version is here." That somehow hook that up, that'd be fun.

12:59 >> Yeah, grab the first line out the release notes and just tweet that.

13:02 >> Yeah. Then the hard part really is, how does it know where to change the version?

13:10 That's where part of the configuration, I think it's really pretty cool.

13:13 It just has this file configuration setting, if I can find it on here, that you list the source, and then you can also list the configuration of it.

13:27 Let me grab one.

13:29 The source and then how to look for it.

13:34 It's a search string or something of what line to look for and then where to replace the version.

13:39 And that's pretty straight.

13:41 I mean, you kind of have to do some hand tweaking to get this to work.

13:45 But for instance, it's just a couple of lines.

13:47 It makes it pretty nice.

13:49 At first, I thought, well, it's not that much work anyway.

13:51 But it's way less work now.

13:53 And then, frankly, I usually forget.

13:55 I'll remember to push the version, but I'll forget to make sure that the version's in the changelog.

14:01 I'll forget to push the tags to GitHub because I don't really use the tags, the version tags in GitHub, but I know other people do.

14:09 So yeah, it's nice.

14:11 Yeah, Will, what do you think as someone who ships libraries frequently that matter?

14:15 I think it's useful. I think for my libraries, I've got the version in two places, two files.

14:20 So for me, it's like, edit 2000, done.

14:25 Probably wouldn't be like massive time saver, but I like the other things you can do with it, the actions you can attach to it, like creating a tag in GitHub.

14:38 So I do often, quite often forget that, especially for like minor releases.

14:42 I sometimes forget that.

14:44 So that's quite useful.

14:45 - Yeah, it's the extra stuff.

14:46 It's not just changing the files, but like Brian described, like creating a branch, creating a tag, pushing all that stuff over, making sure they're in sync.

14:53 That's pretty cool.

14:54 - Yeah.

14:54 - Yeah, good find.

14:55 This does more than I expected when I saw the title.

14:57 - What have we got next?

14:58 - Well, you can start off on your first one.

15:01 This is Closember, which is--

15:06 what's-- portmanteau is when you put two words together, November and close.

15:11 The idea is to help open source maintainers close issues and close PRs.

15:18 So is this like to recover from the hangover of Hacktober?

15:22 Hacktober, I think so.

15:24 I didn't do Hacktober this year.

15:26 I didn't either, no.

15:27 No.

15:28 Last year, I mean, I got a lot of PRs coming in.

15:33 Some of them were of dubious quality.

15:36 Some of them just--

15:38 some of them are very good, actually.

15:40 I did actually benefit a lot.

15:42 But it does actually generate extra work.

15:45 If you manage it, it's really great.

15:48 But this is-- it generates more work for you, even though it's in your benefit.

15:52 But Close Ember is purely to take work away from you, work away from maintainers.

15:59 You know, if there's lots of issues, I mean, I've been very busy lately and not kept an eye on the rich issues.

16:04 And they've just piled up.

16:07 Some of them can be closed with a little bit of effort.

16:11 So I think that's what this project, it's more of a movement than a project.

16:14 It's designed to do-- it's designed to take away some of that burden from maintainers.

16:20 And it's a very nice website here.

16:24 There's a leaderboard on different issues, and it describes what you should do to close issues and PRs.

16:35 The author, his name is Matthias Boussonnier.

16:39 I've probably mispronounced that.

16:41 He started this, and I think it's going to turn into a movement.

16:46 Possibly it's too soon to really get big this year, but I'm hoping that next year it'll be a big thing.

16:55 It'll be after October you can relax a bit because someone, you'll get lots of people coming in to fix your issues and clear some PRs and things like that.

17:06 I mean, sometimes it's maintenance, it's just tidying up, closing PRs which have been merged and closing issues which have been fixed, that kind of thing.

17:17 So I think it's a great thing.

17:19 - I guess I don't quite get what it is.

17:20 Is it a call out to people to help maintainers?

17:24 - Yeah, yeah, it's like a month long thing.

17:26 It was almost like a competition that they've--

17:30 - Yeah, they've got a leaderboard, right?

17:31 - Yeah.

17:32 - Yeah, yeah, Matias is a core developer of Jupyter and IPython, so he's definitely working on some of the main projects there.

17:40 - Yeah, he probably understands the burden of an open source maintainer.

17:46 Even if you love something, it can be hard work.

17:51 - Too much of a good thing, right?

17:53 - But no T-shirt for this, at least not this year.

17:56 - I don't think they offer T-shirts, no, maybe next year.

17:59 - I wonder if you can add your project to this.

18:02 - I think you can tag your project with Closember, I think that's how it works.

18:07 And then other people can search for it and decide which one they want to help with.

18:13 - All right, cool. - That's pretty cool.

18:14 So another Brian, Brian Skin sent over, thank you Brian, but sending a ton of stuff our way lately and we really appreciate it.

18:20 - Yeah, keep it coming.

18:21 - So this one is, the announcement is that scikit-learn goes 1.0.

18:27 And if you look at the version history, it's been zero for, zero ver for a long time with being, you know, 0.20, 0.21, 0.22, 0.

18:37 So this release is really a realization that the library has been super stable for a long time, but here's a signal to everyone consuming scikit-learn that in fact, we intended, they intended to be stable.

18:52 So there's certain groups and organizations that just perceive zero-over stuff as not finished, especially in the enterprise space, in the places that are not typically working in open source as much, but are bringing these libraries in.

19:07 You can see managers like, "We can't use scikit-learn, it's not even done.

19:10 "It's zero dot 24, come on." So this sort of closes that gap as well, signals that the API is pretty stable.

19:17 Will, Textual is not quite ready for this, is it yet?

19:22 - No, it's still on zero, 'cause I'm kind of advertising that I might change a signature next version and break your code.

19:31 Never do that lightly, but it's always a possibility.

19:34 So if you use a zero point version bit of anything, you should probably pin that, and just make sure that if there's an update that you check your code.

19:44 - But as a consumer of rich or a consumer of flask or consumer of whatever, if you're using Azure, you're recommending you pin that in your application or library that uses it, right?

19:54 - Yeah, exactly.

19:55 I mean, you might want to pin anyway, just to lots of bits of software working together.

20:01 There could be problems with one update here that breaks this bit of software here.

20:06 But when you got 1.0, that's the library developer is telling you I'm not going to break anything backwards compatibility without bumping that major version number.

20:16 If they're using SemVer, but because there's lots of other versioning schemes that have the pros and cons.

20:24 - Yeah, like calendar-based versioning and stuff like that, right?

20:26 - Yeah, yeah.

20:27 - I think that makes more sense in an application than it does in a library.

20:32 - Calendar versioning.

20:34 I think it might do actually.

20:35 - How much calendar versioning makes sense for libraries?

20:38 Maybe it does, I don't know.

20:39 - I think projects that, some projects that have shifted to Calvr have recognized that they really are almost never changing backwards compatibility.

20:48 So it doesn't, they're never going to go to a higher number.

20:53 - Yeah, it's strange that there's no one perfect system.

20:56 I quite like soundbar, and by and large, it does what I need of it, but there is no perfect system, really.

21:04 - Yeah, I like it as well.

21:06 Just the whole zero verb being for like, Something is on zero version, zero dot something for 15 years, like that doesn't make sense.

21:11 - Yeah.

21:12 - All right, so since we're talking about the 1.0 release of Scikit-Learn, let me give a quick shout out to some of the new features or some of the features they're highlighting.

21:20 So it exposes many functions and methods which take lots of parameters like hist gradient boosting regressor, use that all the time, no, not really.

21:29 But it takes, I don't know, what is that, 15 parameters?

21:32 Like 20, zero, 255, none, none, false.

21:36 What?

21:37 Like, what are these, right?

21:38 And so a lot of these are moving to require you to explicitly say min sample leaf is 20, L2 regularization is zero, max bins is 255, like keyword arguments to make it more readable and clear.

21:52 - I like to make virtually all my arguments keyword only.

21:56 I might have one or two positional arguments, but the rest, keyword only.

22:00 I think it makes code more descriptive.

22:03 You can look at that code, and then you know at a glance what this argument does.

22:08 - Yeah, absolutely.

22:10 - Yeah, it drives me nuts when there's like, I want all the defaults except for like something special at the last one.

22:16 And so I've got to like fill in all of them just to hit that.

22:19 - And also I would love to throw out that this is way better than star star KW args, way better, right?

22:26 If you've got 10 optional parameters that have maybe defaults or don't need to have a specified value, make them keyword arguments means that the tooling like PyCharm and VS Code will show you auto complete for these.

22:38 I mean, if it's truly open-ended and you don't know what could be passed, star star KWArgs. But if you do know what could be passed, something like this is way better as well.

22:47 That often means you have to type more.

22:50 If you've got like a signature which takes the same parameter as something else, you just have to type it all over again. That can be a bit tedious.

22:57 But it's very beneficial, I think, for the tooling, like you said.

23:01 - Indeed, also for typing, right?

23:04 You can say that this keyword argument thing is an integer and that one's a string, right?

23:08 And if it's star star KWR, you're just any any, great.

23:11 Okay, or string any.

23:12 Okay, so we also have new spline transformers.

23:16 So you can create spline Bezier curves, which is cool.

23:20 Quintile regressor is updated.

23:23 Featured name support.

23:25 So when you're doing an estimator past your pandas data frame during a fit, it will, Estimator will set up feature names and attribute containing the feature names, right?

23:34 So that's pretty cool.

23:35 Some examples of that, a more flexible plotting API, online one class SVM for all sorts of cool graphs.

23:42 Histogram based gradient boosting models are stable and new documentation.

23:46 And of course you can launch it in a binder and play with it, which is pretty sweet.

23:50 Congrats to the scikit-learn folks.

23:52 That's very nice.

23:53 And also kind of interesting to get your take on API changes and versioning and stuff, Will.

23:59 Oh, before we move on, Brian, I saw a quick question that maybe makes sense to throw over to Will from Android.

24:04 - Oh, God.

24:05 Everybody keeps asking this.

24:07 So I've ordered a Windows laptop.

24:10 - The question is, when will there be Windows support for Textual?

24:13 - Yeah, I've ordered a Windows laptop.

24:15 I've been working on a VM, but it's a pain to work on a VM.

24:19 I've ordered a Windows laptop, and that's gonna arrive end of this month.

24:22 And I don't know exactly when, But that'll definitely, I'll definitely need that to get started.

24:29 And in Citi, in Citi it should only be a week or two work.

24:34 So how about I say this year, this year.

24:37 - After the month of configuring your laptop.

24:40 - That's true, that's true.

24:41 I haven't used Windows in I don't know how long apart from a VM.

24:45 I need, I'm gonna test it with a new Windows Terminal, which is actually really, really good.

24:50 - Yeah, Windows Terminal's good.

24:51 - Yeah, I think it can be like a first class, like textual platform.

24:57 The Mac works great, Linux works great.

25:01 Windows has always been like a bit of a black sheep, but the new Windows Terminal is a godsend because the old terminal was frankly terrible.

25:09 It hadn't been updated in decades.

25:11 - Yeah, the old school one is no good, but the new Windows Terminal is really good.

25:15 Also, just a quick shout out for some support here.

25:18 Nice comment, Tushar.

25:19 Windows support will be provided when you click the pink button on Will's GitHub profile, AKA the sponsor button.

25:25 I'm sure you'll ransom.

25:29 I promise I do intend to do.

25:31 Alright, how about some server stuff?

25:36 I we've we talked.

25:37 I can't remember.

25:38 I think several times talked about how to use how to develop packages while you're offline.

25:42 Like let's say you're on an airplane or at the beach or something with no Wi-Fi.

25:47 I mean, maybe there's Wi-Fi at the beach, but not at the beaches I go to.

25:51 So that's because you live in Oregon and some of the most rural parts are the beach.

25:57 If this was California, you'd have 5G.

25:58 Yeah, well, I mean, I could tether my phone to it or something.

26:02 But anyway, so Jason Coombs sent over an article using DevPi as an offline PyPI cache, and I I had to tell you, to be honest, that I don't know if it's just the documentation for DevPi or the other tutorials.

26:17 It just like threw out a few commands and they're like, "You're good. That'll work." And I just never got it.

26:25 I've tried and it just didn't work for me, but this did.

26:28 So this tutorial is just a straightforward, "Okay, we're just going to walk you through exactly everything you do." It's really not that much.

26:36 For instance, he suggests using PipX to install DevPi server, which is nice.

26:43 >> The tbump package as well suggested installing itself with PipX.

26:47 PipX is gaining a lot of momentum.

26:49 - Well, especially things like, well, like, yeah, T-Bump or, well, or Dev-Pi.

26:55 I don't know if I'd do it with T-Bump because I want other package maintainers to be able to use it too.

26:59 But anyway, this is definitely something you're just using on your own machine, so why not let it sit there?

27:05 And then, so you install it, you init it, and it creates some stuff.

27:11 I don't know what it does when you init it.

27:13 But then hidden in here is you run dev by server also then.

27:18 There really is just a few commands and you get a server running, but there's nothing in it.

27:24 There's no cache in it yet.

27:25 So then you have to go somewhere else and then prime it.

27:30 So you've got a local host and it reports.

27:35 So you can export that as your pip index and then just create a virtual environment and start installing stuff.

27:41 That's all you gotta do.

27:42 And now it's all primed.

27:44 And then what you do is you turn off, next time when you don't have any wifi, you turn off, you can run the DevPi server as, where is it, DevPi offline mode.

27:59 And then there you have it.

28:00 You've got a cache of everything you need.

28:03 So I tried this out just on like, installing pytest from my plugins, and then set it in offline mode.

28:12 and then tried in the, all the installing the normal stuff that I just did, worked fine into a new virtual environment.

28:18 But then when I tried to do something like install requests that I didn't have yet or something else, it just said, oh, that's not, it's not of it.

28:26 I can't find it or something.

28:27 It's a happy failure.

28:29 So anyway, this instruction worked great.

28:31 I know DevPi can be do a whole bunch of other stuff, but I don't need it to do a whole bunch of stuff myself.

28:37 It just needed to be a IPI cache.

28:39 - Yeah, this is really neat.

28:41 of it looks like it creates a database schema as well as allows you to set up set up a user.

28:49 Okay, I guess you could you set up with some authentication that no one can mess with it and stuff like that. Apparently this works just fine for teams so you can set up set up a server on like a just like a computer that in your network that just runs as a cache and then you can point everybody can point to the same one. So I mean that that that would work as a really quick and dirty and not too dirty, just a fairly quick way for a local team to have a caching server. I'd probably even think about doing this for testing, even on one machine so that you can have multiple, like, you know, completely clean out your environments and still run, run the test machine and not hit the network so much if you're pulling a lot of different stuff.

29:32 Henry Schreiner on the live stream says, can we also mention that Jason, the article we're we're just talking about.

29:37 Also maintains 148 libraries, including setup tools on PyPI.

29:42 - That's awesome.

29:43 - So may know something about interaction with PyPI.

29:48 - That's phenomenal.

29:49 I don't know how he finds the time to be honest.

29:51 148 packages, he needs a close ember.

29:54 - He has a lot of close ember.

29:57 Awesome, all right, well, what's this last one you got for us here?

30:01 - Sure, so I found this project on Reddit.

30:06 it's called PyPy command line.

30:08 And I noticed it in particular because it used rich, but it is a pretty cool project.

30:12 It's notable because the author is 14 years old.

30:17 Like, that's blown me away.

30:19 To be that young, and he's done a very good job of it.

30:23 So it's a interface to PyPy from the command line.

30:26 You can do things like get the top 10 packages.

30:31 You can search for packages.

30:35 You can see here's, I think that's a search, oh, look, PyPy search.

30:41 Rich, and that's given all the packages that have got rich in the name.

30:45 It's got a description and everything and the date.

30:47 And here you can, PyPy info django, that gives you some nice information about the django package, which it pulls from PyPy.

30:55 - Like the GitHub stars, the download traffic, what it depends upon, meta information, like it's licensed and who owns it.

31:03 This is really cool.

31:04 Yeah, it's really nice.

31:06 Here we have the description, and that renders the markdown right in the terminal.

31:12 I wonder how it does that.

31:15 I couldn't hazard a guess.

31:17 It's got to use rich, right?

31:19 I think it might, yeah.

31:21 Yeah, so it makes good use of rich.

31:23 That's how I noticed it.

31:25 But it is a very cool project in its own right.

31:28 It also uses a questionnaire.

31:31 that's like a terminal thing for selecting stuff from the menu.

31:36 So it does a bit dynamically and also has like a command line to do it more from the terminal.

31:45 I think it's well worth checking out.

31:49 - I think I wanna check it out just for an example of using this sort of a workflow, not necessarily with PyPI, but with just sort of copying the codes.

31:57 - Yeah. - Yeah.

31:58 It's a really nice looking terminal user interface type thing.

32:02 I think it could be really interesting for you and me, Brian, to just do like info on the various things we're talking about, right?

32:08 That might be fun to pull up as well.

32:09 Yeah, there's actually tons of times where I don't really want to pull up a web browser just to look it up, but I do want more information, just the help gives me.

32:18 I love the web, but sometimes you have to do a context switch if you're in the terminal, you're writing commands and then you've got to like switch windows and find the bar and type everything in. It's just a little bit of effort but it can kind of like interrupt your flow when you are working.

32:35 Yeah, I mean especially when you got like the whole, I've got like a big monitor and I've got them all, everything placed exactly where I want it and there's no web browser.

32:42 So if I want to look something up I gotta like, you know, interrupt that.

32:46 Yeah, or the browser you want is there but it's behind a dozen other windows, a dozen other web browsers typically.

32:52 Exactly.

32:53 - Exactly.

32:55 - Yeah, that's a good find.

32:56 And well done to this guy who wrote it at such a young age, very cool.

33:01 - I was just gonna ask you if you have the extras thing.

33:04 - Do I have any extras?

33:06 Ta-da, here's my little banner extras.

33:08 I do have some actually, Ryan.

33:10 Quick shout out, Madison Sinover noticed, let us know that High Cascades 2022, their call for proposals is out.

33:19 So if people wanna sign up for that, It closes October 24th.

33:24 So, you know, make haste, you've got four days.

33:27 But yeah, still.

33:28 - EFP closes in four days?

33:30 - Yeah, so if you're thinking of preparing for something, you got three days.

33:33 Talks are 25 minutes long.

33:34 It was a lot of fun.

33:35 You know, we both attended this conference a few times.

33:37 In the before times, it was in Portland, Seattle, and Vancouver.

33:42 This, I'm not sure what the story is with this one.

33:44 If it's gonna be in person. - It's remote.

33:46 - I think it's remote, right, yeah.

33:47 - I think so, at least.

33:49 Hope I'm not wrong.

33:50 - Yeah, I think you're right.

33:51 - Then have you got your MacBook, your M1 Max?

33:55 You ordered that yet?

33:56 - I want one, but no.

33:57 (laughing)

33:58 The $3,000. - Well, how about you?

34:00 - I would love one, but I have no idea what I'd do with it.

34:02 You know, I just work in the terminal most of the time.

34:05 - Hey, you know, it has that new Pro, what is it, ProRes?

34:10 Something display where it has 120 adaptive display, Hertz display, so, you know, maybe.

34:16 - I think my monitor only does 60, so I don't know if I could use it, but I have actually got textual running at 120 frames per second, which is pretty crazy.

34:26 >> Yeah, that's pretty crazy. I did end up ordering one, and on my Apple account, I have this really cool message.

34:32 It says, "Your order will be available soon.

34:33 MacBook Pro available to ship null." We'll see where that goes.

34:39 See where that goes.

34:42 >> Did you put null as your address?

34:44 >> I should have. Think how many people's orders I would be getting.

34:47 I would get just a stack of boxes outside.

34:49 Did that at Amazon or something. Yeah.

34:51 And then I think also I want to give a quick shout out to this thing, this CodeWeavers crossover, which allows you to run Windows apps natively on macOS without a virtual machine.

35:03 It's like an intermediate layer.

35:05 So I think that that kind of stuff is going to get real popular, especially since the new M1s have like a super crappy story for Windows as a virtual machine because Windows has a crappy ARM story.

35:15 And you could only do ARM VMs over there.

35:17 So I think that things like this are going to become really popular.

35:20 There's a bunch of cool stuff.

35:21 People haven't checked out this crossover stuff.

35:23 I haven't really done much in it, but it looks super promising.

35:25 I've like been on the verge of like, I almost need this, but I just run into you.

35:29 That's that. Anyway, those are my extras.

35:31 OK, well, I've got a couple.

35:34 We've we've run up Starship once.

35:36 I just I've just broke down and I'm using Starship now.

35:40 It looks working nice.

35:41 And one of the things that installed when I when I grew installed Starship, It also installed pie and I'm not sure why.

35:48 So I started using pie of also it was still in pie and works great. I like it on my Mac, but I still don't think it belongs in Python tutorials.

35:58 Anyway, burning still out on me whether or not it's any better than just downloading off of org.

36:04 You're going to get tweets, Brian, you're going to get tweets, but I agree with you.

36:08 I support you in this.

36:09 And so one of the things that was announced today is VS Code.dev is a thing, so I thought it was already there, but apparently this is new.

36:18 If you go to VS Code.dev, it is just VS Code in the browser.

36:25 >> Oh, interesting.

36:26 >> I think it was already there.

36:27 >> Where does it execute and where is your file system and stuff like that?

36:30 >> Well, I think it's the same as the GitHub code spaces.

36:34 >> You press dot? Yeah. Okay, got it.

36:37 >> It can use the local file system though, which I think is a difference.

36:41 GitHub had this thing we you hit door and it brought a Vs code which worked with the files in your repo, but I think with this it can actually use your local File system Wow Which makes it more interesting is great if you work on another computer and just pop it open you've got all your settings there I'm home you ready to go. Yeah Yeah to use cases for me that that I think I would use this that seem really nice one is I'm working like say on my daughter's computer.

37:12 She's like, "Dad, help me with this file.

37:14 Help me with something and I've got to open some file in a way that has some form of structure." She doesn't have VS Code set up on her computer.

37:22 She's in middle school, she doesn't care.

37:23 But I could just fire this up and look at some file in a non-terrible way.

37:29 That would be great. The other is on my iPad.

37:31 >> Yeah.

37:32 >> Right? There's not a good, super good story for that, but this VS Code in the browser, other things in the browser, they seem really nice.

37:40 Or if I was on a Chromebook or something like that, right?

37:42 If I was trying to help somebody with code on a Chromebook, that'd be good.

37:45 - How about you, Will, do you have any extras for us?

37:47 - Here we go.

37:48 Python multithreading without the GIL.

37:51 GIL stands for Global Interpreter Lock, and it's something which prevents Python threads from truly running in parallel.

37:59 People have been talking about this for years, and I've got a bit kind of dismissive, 'cause every time it comes up, it never seems to happen, because there's quite a lot of trade-offs generally.

38:10 If you get rid of the GIL, you hurt single-threaded performance, and most things are single-threaded.

38:17 But this looks like the author, Sam Gross, has come up with a way of removing the GIL without hurting single-threaded performance.

38:28 I think they've got, it's to do with reference counting.

38:32 They've got two reference counts, one for the thread which owns the object and one for all the other threads.

38:39 And apparently it works quite well.

38:42 And the great thing about this is--

38:44 - That's super creative to basically think of like, well, let's treat the ref count as a thread local storage.

38:49 And probably when that hits zero, you're like, okay, well, let's go look at the other threads and see if they're also zero, right?

38:54 - Yeah, yeah.

38:56 And if this goes ahead, and it's got quite a lot of support, I think, in the core dev community, I don't keep a really strong eye on that, but from what I hear, it's got a lot of support.

39:08 And if that lands, then we can get fantastic performance out of multi-threaded code.

39:13 You know, if you've got 20 threads, you could get almost 20 times performance.

39:18 So that could be huge.

39:21 I've no doubt there'll be a lot of technical hurdles from C libraries and things, but I'm really excited about that.

39:29 I think performance improvements to single-threaded, they come in little fits and starts, you know, we get 5% here, 10% here, and it's all very welcome.

39:37 But if this lands, then we can get like 20 times for certain types of computing tasks.

39:43 So I'm really excited.

39:45 I hope this one lands.

39:47 - I mean, you're talking about this, oh, here, let's get this multi-thread stuff.

39:51 You know, you were just saying, what are we gonna do with these new M1 Pros and M1 Macs?

39:55 I mean, 10-core machines, 32-core GPUs, there's a lot of stuff that's significantly difficult to take advantage of with Python, unless something like this comes into existence, right?

40:07 - Exactly, if you have 10 cores, chances are you'll just use one of them.

40:11 I'm wondering if this goes in, whether it'll change, we'll need some other ways of taking advantage of that, because I think at the moment, for most tasks, you'd have to explicitly create and launch threads.

40:24 I wonder if there'll be advances where Python could just launch threads and things which could be easily parallelized.

40:32 Maybe I'm hoping for too much, but I've no doubt there'll be some kind of like software solution to help you just launch threads and like use all those cores in your shiny new Macs.

40:44 - There's a lot of interesting stuff that you can do with async and await.

40:47 And there's also some cool thread scheduler type things.

40:50 But I think the, you know, much like Python 3 when type annotations came along, there was a whole bunch of stuff that blossomed that took advantage of it, like Pydantic and FastAPI and stuff, I feel like that blossoming hasn't happened because you're really limited by the GIL of the CPU level, then you go multi-processing and you have like data exchange and compatibility issues, but if this were to go through, all of a sudden people were like, all right, now how do we create these libraries that we've wanted all along?

41:18 - Yeah, yeah, I think that's, I think once we've got over that technical hurdle, all the library authors will be looking for creative ways of using this for speeding code up and for just doing more with your Python.

41:36 - Yeah, I mean, with every programming language, the jump from single-threaded to multi-process is a huge overhead, so you don't do it lightly.

41:45 But you could do it lightly with multi-threads.

41:48 You don't have such a huge overhead burden with threads.

41:51 - That's very exciting.

41:52 I was also super excited about this, so I'm glad you gave it a shout out.

41:55 We'll probably come back and spend some more time on it at some point.

41:57 - Yeah, and where is it?

41:59 Somebody said one of the exciting things about it is Guido didn't say no immediately.

42:04 - That's a very good sign.

42:08 - Yeah, which has not been the case with some of these other ones, because they were willing to sacrifice single-threaded performance to get better multi-core performance.

42:15 They're like, you know, this is not a common enough use case that we're willing to do that.

42:18 - I think actually the solution the author came up with It did reduce single threaded performance, but he also added some unrelated optimizations, which speeded it back up again.

42:32 - Exactly.

42:32 I'm sorry, I fixed it, but yeah.

42:34 Interesting.

42:37 One more thought on this really quick.

42:38 David, pushing out in the live stream says, the Gilectomy is like nuclear fusion.

42:42 It's always 10 years away.

42:44 - Yeah, I hope it was.

42:45 Hopefully it's not 10.

42:46 - It's possible, but I think this is the biggest possibility since then, two interesting things, maybe already taking into account that Peter looked at it and didn't say no immediately.

42:56 Two, this is a project Sam's working on, but it's supported by Facebook where he works.

43:03 So there's like a lot of time and energy.

43:05 It's not just a side project.

43:07 Third, Larry Hastings, the guy who was doing the Gilectomy, commented on this thread saying, "You've made way more progress than I did.

43:15 "Well done, Sam." So these are all good signs.

43:18 - That's fantastic, yeah.

43:19 - Yeah, all right.

43:20 Well, Ryan, are you ready for our joke?

43:23 - Yeah. - Laugh?

43:24 - Definitely.

43:25 - See, this is optimistic 'cause they're not always that funny.

43:28 But I'm gonna give it a try.

43:29 This one is for the web developers out there, for those folks that work on APIs and probably have been working for a long time on them.

43:36 So the first one I got for us, I just found another one I'm gonna throw in from inspired by the live stream.

43:40 But this one is entitled, "The Torture Never Stops." All right?

43:45 - Okay.

43:46 - So it's a, every one of these, it's four different pictures in this cartoon.

43:50 There's a different developers up at the board describing some new way to talk to web servers from your app.

43:57 So way back in 2000, it says SOAP, Simple Object Access Protocol.

44:02 SOAP makes programming easier.

44:04 And the developer in the audience is like, WTF is SOAP?

44:08 Oh, come on.

44:09 What is this crazy namespaces in XML?

44:11 Skip ahead 10 years.

44:12 Now there's a developer up here saying, REST, Representational State Transfer.

44:17 REST is better than SOAP.

44:18 The developer now has, "WTF was wrong with SOAP?" (laughing)

44:23 2015, GraphQL.

44:25 GraphQL is more versatile than REST.

44:27 WTF, I was just kidding, I think of REST.

44:30 2018, gRPC.

44:32 gRPC is faster than GraphQL.

44:34 WTF, I thought you knew by now.

44:36 The torture never stops.

44:38 (laughing)

44:39 Says, like the guy next to the other developer that's been complaining for 20 years.

44:43 - I think that hits a bit too close to home, But if you're a JavaScript developer, that gets compressed into like the last six months, I think.

44:51 - That's right, you've lived it really hard, and really intensely.

44:54 Nick says, "Let's just start over with soap." Yeah, pretty good, pretty good.

44:57 All right, and then we were talking about vscode.dev, and how you just press dot in your browser in GitHub, or how you go to that URL, and so on, how cool it was.

45:08 And somebody said, "Oh, it doesn't work in Safari." So I wanna come back to this joke that used to be applied to IE.

45:15 But now I think should be applied to Safari.

45:19 Like genuinely, I think it should be.

45:21 It's the browser wars as a cartoon.

45:26 So there's Chrome and Firefox.

45:28 It's a little dated because Firefox is not as popular as it used to be, sadly.

45:30 But it's like Chrome and Firefox are fiercely fighting.

45:33 And like IE is in the corner eating glue.

45:36 I just feel like that needs a little Safari icon and we'd be good.

45:40 We'd be all up to date in 2021.

45:42 How do you know it's IE?

45:44 - It has a little E on it and a window symbol.

45:47 - Oh, okay.

45:48 - Well, the E of course is backwards 'cause the shirt's probably on backwards or something.

45:52 - Also, it's eaten glue, so.

45:54 (laughing)

45:56 - Yeah, true, funny.

45:59 So, thanks Will for joining us today.

46:02 This was a really fun show.

46:03 Thanks everybody in the chat for all the great comments.

46:06 - Thanks Brian, thanks Will.

46:07 See y'all later.

46:08 - Thanks guys, bye bye.

46:09 - Thanks for listening to Python Bytes.

46:12 the show on Twitter via @pythonbytes. That's Python bytes as in b y t e s. Get the full show notes over at Python bytes.fm. If you have a news item we should cover, just visit Python bytes.fm and click Submit in the navbar. We're always on the lookout for sharing something cool.

46:29 If you want to join us for the live recording, just visit the website and click live stream to get notified of when our next episode goes live. That's usually happening at noon Pacific on on Wednesdays over at YouTube.

46:42 On behalf of myself and Brian Okken, this is Michael Kennedy.

46:45 Thank you for listening and sharing this podcast with your friends and colleagues.