Transcript #206: Python dropping old operating systems is normal!

00:00 Hello and welcome to Python Bytes where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 206. Wow.

00:08 Recorded October 28th, 2020.

00:10 I am Brian Okken.

00:11 And I'm Michael Kennedy.

00:12 Yeah, and we have a special guest today, Steve Dower.

00:14 Hi, thanks for having me on.

00:15 Hey Steve. Thanks for coming. It's great to have you here.

00:18 I also want to throw out that this is sponsored by Tech Meme Ride Home podcast.

00:21 Check them out at pythonbytes.fm/ride.

00:24 Steve, I'm sure many listeners know you, but maybe just give us the quick rundown.

00:27 and you do cool stuff at Microsoft, getting Python better on Windows and you're a core developer.

00:32 Yeah, so my main day job is for Microsoft, where I work as basically a Python engineer, kind of a wide-ranging resource to the company. So I haven't shipped anything of my own in a while, but I've had my fingers in a lot of Python-related things that have gone out recently.

00:49 So it's a lot of fun, a lot of bouncing around between different teams, getting to work with a lot of different people. And yeah, as you say, I'm a CPython core developer, one of the Windows experts on the team, I'm responsible for the builds that go up on python.org, and just generally keeping Python running well on Windows.

01:04 Yeah, that's awesome.

01:06 And you've done some interesting talks, like, "Python is okay on Windows, actually," and talked about some of the popularity of it, and how we as a community shouldn't discount it just because we might happen to be on a Mac or use Linux or whatever, right?

01:18 A lot of people do Python on Windows.

01:20 Yeah, yeah. The estimates vary, and every time I get new numbers, they seem to show up slightly different. So it's real hard to get a good fix on how many Python developers there even are in the world. I did get some numbers recently that I had a few people double check because they were saying there's like 20 million installs of Python on Windows in the entire ecosystem. Wow.

01:40 Which sounded like too many to me. So I had them double check. And then I had someone else double check. And they all came back saying, yeah, it's about that. So I'm like, okay, there's a lot of Python on Windows out there. But yeah, it doesn't show up in conferences, doesn't show up on Twitter that much and a lot of people just look at the packages that don't work and go well I guess it doesn't exist on windows because otherwise this package would work and so you know chicken and egg problem right yeah there's a lot of chicken and egg problems in the python space I mean it's a beautiful place but there are some of these weird chicken and egg ones yeah it's weird I've been using python on windows since I started python so have I but one thing I haven't been using very much is enums so that was an attempt at a transition so why not Brian tell us more Actually, I've tried many times I've tried to use enums and I actually just to be honest, I don't very much and partly because I'm used to using enums in C and C++ and they're, they just act like symbols in C and C++, they work pretty good.

02:39 There is some weirdness with enums in Python and I'm going to highlight an article called Making Enums, as always, Arguably More Pythonic by Harry Percival.

02:49 He starts it off by saying, I hate enums.

02:52 So Harry's a funny guy and this is a fairly hilarious look at why enums are frustrating sometimes.

02:59 And then he presents a semi-reasonable workaround, I think, to make them more usable.

03:04 So what's wrong with enums?

03:06 Well, he gives an example and just as a simple enum with string values.

03:12 And you can't directly, if you try to compare one of the enum elements to the value, like the value gave, like a similar string, it's not equal.

03:22 It doesn't compare.

03:23 But you can use a dot value or an enum value, but that's just weird.

03:28 He also said, he kind of thinks it'd be neat if you could do a random choice of all the enum values.

03:33 I think that would be neat.

03:34 And you can't directly convert them to a list.

03:37 There's just interacting with the enum type itself or the class itself has problems.

03:43 In the documentation, there is a suggestion that you can, instead of strings, use int and do an int enum and it works a little better.

03:52 And if you like it like that, but want strings, you can make your own string enum class.

03:57 I'm not sure why they didn't just build this into the default or, you know, one of the standard types anyway, but string enum is not there, but there's an example.

04:07 And it sort of fixes a lot of stuff, but not everything.

04:11 It doesn't, still doesn't allow for those direct comparisons.

04:14 So the solution that Harry came up with is just kind of like the solution the documentation says, derived from both enum type and str when you're creating your enum class, but then also define this little snippet of a dunder str method, so that the str method works better.

04:35 And at that point, most of the stuff works that he wants to work.

04:38 It still doesn't do random choice, but apparently he's gotten over that a little bit.

04:43 So I actually think this is really, This is still just a really small snippet, it's like two lines of extra code to add to your enum types, make them a little bit more usable.

04:54 So I think it's reasonable.

04:55 - If you're judging by like value add per character, this is awesome because it makes working with the enumeration so much nicer.

05:03 You can do like the natural things that you would expect, especially around testing and comparison.

05:08 And it's like also add a drive from str and just add a dunder str method and you're good.

05:14 Steve, what do you think about this?

05:15 Yeah, I'd just add that the gotcha that you have by doing this is now you can have two values from different enums compare as equal, which as I recall from the original discussions was the reason this wasn't put in by default.

05:27 Say you've got a color enumeration and a fruit enumeration, is orange the same between the two?

05:34 And I think the decision was made for enums to say no, it's not.

05:37 If it's a fruit orange, it's different from the color orange.

05:41 Making this change or using an int enum is going to make them equal.

05:44 So as long as you're prepared to deal with that, which, to be quite honest, every time I've reached for enums, I am much happier with string literals and quite comfortable with them matching equal.

05:53 Yeah.

05:54 But that is just one thing to watch out for.

05:55 Usually it's about constraining the list of things.

05:58 Like, I know there's five things.

05:59 I want to make sure I don't, like, somehow mix up what those five are.

06:02 I just want to go, you know, class dot, and here's the five.

06:05 Let my editor tell me which one of those it is.

06:07 It seems like it's all there.

06:09 So what do you think about if you overrode, like if you added gender eq, gender neq, so like the comparison would say it has to be this type of enumeration and the str value has to match.

06:21 Yeah, that would certainly deal with that. Gotcha.

06:23 Again, when these were being designed, basically anything that gets designed and added to Python has a very large group of very smart people work through it.

06:31 And, you know, as a result, things always get missed.

06:33 So it's possible that one was just missed.

06:35 It's also possible that someone did figure out a reason why that was also risky.

06:40 And, you know, risky when you're developing one of the most popular languages in the world is just anything that might surprise anyone.

06:47 So someone has deliberately designed around using enums everywhere, they're probably not going to be surprised.

06:52 Someone who is using code and that developer has swapped out all of their...

06:57 You know, they had a class with static variables and they turned it into an enum, and now stuff breaks because of the defaults that were chosen for enum.

07:05 That's the kind of thing that you're trying to avoid in a language that has anywhere between 5 and 20 million kind of regular users.

07:14 But as a workaround, I mean, if you know where your enum is going to be used, there's a reason you can derive from string and it's exactly for stuff like this.

07:20 Yeah, thanks, Harry, for putting out there. That's quite a neat little bit of advice there.

07:24 And I'm so glad that we have Steve here because I picked some sort of semi-internal type of pieces and I'm going to make some statements about it.

07:33 And then Steve can correct me to make it more accurate.

07:36 And also we get the core developer's perspective.

07:38 Not that you represent all core developers, but at least a slice.

07:41 All right, so this next one I want to cover is that Python 3.10 will be 10% faster.

07:46 And this is 4.5 years in the making.

07:49 So Yuri Stelovanov long ago did some work on optimizing, I think, was it load attribute or was it load method, load call method?

08:02 So about some of these load operations down at the CPython CUVal level.

08:08 And then Pablo Galindo, who's also a core developer, and the Python 3.10.3.11 release manager, picked up this work, and now we have load method, call method, and load global going faster.

08:23 So basically, there's some optimizations around those opcodes that make this much faster.

08:30 And this idea apparently first originated in PyPy, P-Y, P-Y.

08:35 So I'm pretty excited to see that some simple internals that don't seem to change a whole lot of stuff might make this a lot faster.

08:43 What do you think, Steve?

08:44 This one is real. I was so excited about this when it was first being proposed.

08:48 The basis of the idea is that Python keeps everything in dictionaries, which means every time you look up .name of anything, it takes that name, makes it a string, turns it into a...

08:59 gets the hash value, looks it up in a dictionary, finds the value, maybe wraps that up in some extra stuff, Like if it's going to be a method, it's not stored as a method, you turn it into a method when you know what it's being kind of dotted through to get to, and then returns that. That's a whole lot of work.

09:14 And if you're regularly calling the same method over and over again, why not cache it?

09:19 That's the heart of this, right? It does that cache around load data, right?

09:22 Yeah, it does that cache. And the insight that Yuri had that he made work, and in fact, I think someone else had suggested it earlier and hadn't gotten it to work was what happens when things change?

09:32 Because again, as I say, it's we're designing language for many, many people do all sorts of weird things.

09:38 And if you cache a method lookup, and then someone tries to monkey patch it, you know, we've now broken their code for the sake of an optimization, which, you know, is a no-no in Pythons, like correctness beats performance.

09:49 In every case, that's just the trade off that the language chooses to make.

09:52 That's almost always what you want, right?

09:54 When would you want to be faster and wrong rather than slower and right?

09:57 I'd be happy with faster and no monkey patching.

10:00 But yes, yes, sure, like faster and fewer restricted capabilities might be a really good trade off, but faster and wrong. It's not a good one. Yeah, we did some benchmarking and basically found that there was a way to track all dictionary updates in the entire runtime with a version tag that was not going to instantly overflow and not going to break everything.

10:22 So it became really easy to say, has this dictionary changed since I last looked at it with one single value comparison.

10:29 And so it looks at that value.

10:31 If it has changed, it's going to do the full lockup again.

10:34 99.999% of the time it hasn't changed, so it can give you the cached value and you saved big dictionary lookup, possibly error handling, descriptor protocol, all of this extra stuff that just adds so much weight to every operation.

10:49 Yeah, and that's everywhere. I mean, that's everywhere in the language.

10:51 Absolutely everywhere.

10:52 Fantastic. One of the things when I was first getting into Python that made me sort of have a sad face was realizing that having function calls was pretty expensive.

11:02 Right? Like having a big call chain, like actually the act of calling a function has a fair amount of overhead.

11:09 When I wanted to break my code into like a bunch of small functions to make it real readable, this part needs to go a little faster.

11:14 Maybe that's not what I want.

11:17 You know, and so hopefully this helps with that as well.

11:20 Yeah, that and vector call is another optimization that we got in recently.

11:25 I think that might have been the pet 509 actually was a vector call also designed to make that faster just removing some of the internal steps for doing a function call fantastic and like Brian said this is everywhere so that's everyone's gonna benefit this is fantastic.

11:40 Well we would like to thank our sponsor.

11:43 So this episode is brought to you by tech meme ride home podcast is a great podcast for more than two years and nearly seven hundred episodes the tech meme ride home.

11:54 Has been silicon valley's favorite tech news podcast the tech meme ride home is a daily podcast only fifteen to twenty minutes long and even and every day by five p.m. eastern it's all the latest tech news.

12:08 But it's more than just headlines. You could get a robot to read your headlines.

12:11 The Techmeme Ride Home is all the context around the latest news of the day.

12:16 It's all the top stories, the top posts and tweets, and conversations about those stories, as well as behind-the-scenes analysis. The Techmeme Ride Home is the TLDR as a service.

12:25 The folks at Techmeme are online all day reading everything so they can catch you up. Search your podcast app now for ride home and subscribe to the tech meme ride home podcast or visit pythonbytes.fm/ride to subscribe. Yeah thanks for sponsoring the show and Brian, every day, like we do this once a week and it's a lot of work. These guys are on it. I could totally do this every day. If I didn't have another job I would not have any problem with catching up on Python news daily. So actually sounds quite lovely. I wonder how that podcast is doing now that not so so many people are having to commute to and from work.

13:04 That sounds like one of the things where you hope that you've given people the excuse to tell their employer on my commute between 4 and 5 p.m.

13:13 I can't do it.

13:14 I've logged off and go listen to a podcast during that time.

13:18 I listen to podcasts while I'm, I realized I was missing out.

13:20 So I started listening to podcasts while I'm doing my laundry.

13:23 I do it when I'm doing dishes.

13:26 I listen when I'm doing yard work, stuff like that.

13:29 - Yeah, I recently broke out some other older podcasts just to catch up on stuff with a big mess I had around the home.

13:35 Like it's a long story with a new puppy that's not worth going into.

13:39 (both laughing)

13:41 Maybe I'll tell you guys after the show.

13:42 It's pretty outrageous.

13:43 Anyway, yeah, and it's so enjoyable.

13:46 But what I've actually found is shows like Python Bytes and Write Home that have like a bunch of short little things that you can just drop in and out of, match a lot better now that people are not commuting so much.

13:56 you can do that 10 minutes while you're folding laundry and get a whole segment.

14:00 So I think it gets varied, but actually it's pretty interesting.

14:03 I think that they and us here are well positioned.

14:06 Like Talk Python has more of a dip than PythonBytes.

14:09 - Have you surveyed your listeners to find out what they're doing while they listen to you?

14:12 - No, not really.

14:13 - Everyone should tweet at these guys' Twitter.

14:16 What were you doing when you were hearing this podcast?

14:19 - Yeah, that's awesome.

14:20 I've got a few anecdotes, but nothing like a survey that would give me a proper answer.

14:24 Steve, I think your next item is super, super interesting.

14:27 And people speaking to Twitter, right?

14:30 Like this whole conversation started as a, "Hey, Twitter message." Like, why did this happen?

14:35 Well, let's ask Steve.

14:36 Yeah, as you know, I spend a decent amount of time on Twitter.

14:39 My handle there is Zuba, Z-O-O-B-A, which you'll probably never find in search if you're looking for my name, but that's where I am.

14:46 I really like actually searching for what people are saying about Python on Windows.

14:51 It's kind of the most honest feedback you get when they think you're not listening And so I go and listen and one of these popped up Which was oh, I tried to install python 3.9 newest release about a little bit under a month ago On my windows 7 machine and I couldn't install it And since then i've actually seen a few more posts Someone managed to bypass the installer completely and get it all the way onto their windows 7 machine and then found out it wouldn't run oh, man, yeah, and So the question was asked, like, why would you do this? Windows 7 is still a fairly big platform.

15:23 Why would you take it out? And, you know, the answer was just a bit too long for a tweet.

15:27 But someone, you know, kindly included Python bytes in the reply. And so I said, hey, I'll come on and talk about it. So let's do this topic. And the answer is, multiple legal looking documents all come together and have to be read in parallel to figure out why we dropped it.

15:45 Yeah, the small business owners know what I'm talking about.

15:48 So one of those documents is PEP 11, one of the lowest number PEPs that we have, and it's titled "Removing Support for Little-Used Platforms".

15:57 The title was not originally about Windows, but there is a section in that PEP that describes Python's policy for supporting Windows.

16:06 On release day of 3.x.0, all the supported versions of Windows covered by Microsoft Support Lifecycle will be supported by CPython.

16:17 And that's on the 3.x.0 release date.

16:20 So what that means is then you now have to go and look at Microsoft Support Lifecycle website and look up all of the different versions of Windows to see which ones are still covered by support to that date.

16:32 Windows 7 fell out of extended support in January.

16:35 There was quite a bit of noise about that because that means no more security patches except Microsoft did do a couple more security patches because some really bad stuff was found, but that's largely stopped.

16:47 Essentially, it's the point when the only people who get Windows 7 support are paying for it, and I assume they're paying large amounts of money for it.

16:54 I don't actually know how much it costs, but that's the point where, you know, you bought it, but you don't get the free support anymore.

17:01 So CPython follows that because no one is paying the core team to support Python on all of these platforms.

17:07 And so it's, it seems like The fairest point to draw that line is at some point, we have to say our volunteers can no longer keep a Windows 7 machine running. Even I can't keep a Windows 7 machine running safely because there's no security updates for it. How am I meant to develop Python on it? How am I meant to test Python on it? The burden there is too high for volunteers to handle. So we just say that's the point where it goes away. So because those two documents lined up, Windows 8 actually dropped off a couple of years ago because the support life cycle ended early for that to force everyone onto 8.1. Windows 8.1 has about three more years, so I think Python 3.12 will be the last one to support 8.1, and then it's all Windows 10 or whatever comes out in the future. Yeah, yeah, Windows 10x or whatever they call it. That's a different one. That's the Xbox. Yeah, so I, you know, I think this makes a ton of sense and two thoughts I had as you were laying out the the case here. One is if you're running a Windows 7 and you can't upgrade to even Windows 8 or more recently Windows 10 or one of the server equivalents, right?

18:14 I'm sure there's like a server equivalent like Windows 2003 server. I don't know how long I support it but whenever it goes out it probably falls under that banner as well, right? Yeah, Windows server is a bit more interesting. Their life cycles tend to last longer. But historically CPython has only kind of tied itself to the client operating systems.

18:33 Gotcha. Oh, interesting. Okay.

18:35 So to me, I feel like if you're running code, you're running systems that old, you must be running it because it's like some super legacy thing.

18:44 So do you absolutely necessarily need to have the most cutting edge Python or whatever language? It's probably something that's that way because it's calcified and you probably don't need or you probably shouldn't be putting like the newest as shiny as things on it, right? That's that one. What do you think?

19:03 Yeah, no, I totally agree with that. If that setup that you're running is so critical that you can't upgrade the operating system, how can you upgrade a language runtime?

19:12 Like, how can you upgrade anything on that?

19:15 I feel like it's in the category of, "Please just don't touch it. It's over there.

19:18 Just don't even walk by. Leave it alone. We cannot have it break. Just leave it over there." It probably still has a PS/2 keyboard plugged into it.

19:26 Oh, it might with a little blue or pink round thing. Yeah, absolutely.

19:29 The screen has probably got at least 16 colors.

19:31 Yeah, and the monitor is probably really heavy.

19:33 Windows 7 is not that old.

19:35 Some of the...

19:37 Seriously, like this stuff is old and you probably don't want to touch it, right?

19:40 Yeah, that's exactly it.

19:42 It's all the motivation that you would have for updating to Python 3.9 from 3.8. And again, we're talking about a version that's only one year old.

19:48 Like Python 3.8 is not that old.

19:50 And you desperately need to upgrade to 3.9.

19:53 You even more desperately need to upgrade Windows.

19:56 And there's just really...

19:58 There really is no question about that. The same thing applies to early versions of Ubuntu. People running Ubuntu 14 or even 16 at this point need to be facing the same thing. And we have similar discussions around OpenSSL where occasionally people will be, "Oh, I need Python 3.9 to run on OpenSSL 0.9." To which our answer is basically, "That's pretty hot-bleed." I'm gonna play the other side i totally get the reasons but i also get the questions.

20:29 Because the users and the developers or the whoever's wanting to install python they usually don't get to choose what operating system they're using but they do get to choose which version of python.

20:43 So i do get send in some cases and inside the room true i totally understand where the questions coming yeah we joke about how old these machines are and they're really not like people are setting up new machines probably with windows seven they certainly were within the last year and there's good legitimate reasons for that.

21:00 And you know when not you're making fun of some of the apparent contradictions but we're definitely not making fun of the people who have you know, often being forced into these positions.

21:09 But the reality is we can't afford as a volunteer team to maintain Python against unmaintained operating systems.

21:16 And so, you know, the advice is, stay on the previous version of Python, that the latest version of Python that works for you, it's not going to break, we're not changing it.

21:26 Anything new that comes up, security fixes will still come out.

21:30 At some point, there just has to be a line drawn.

21:32 And that's the point where we've chosen to draw it.

21:35 The other thing I want to point out that we changed in this release, which people are more excited about, is if you go to python.org to download Python for Windows, you get this real big obvious button up front that just says "Download for Windows" or "Download Now" or something.

21:48 As of Python 3.9, that's now getting the 64-bit version rather than the 32-bit version.

21:54 For a long time, it's been 32-bit.

21:55 The reason for that was compatibility.

21:57 We knew a 32-bit one would run anywhere.

22:00 When we put Python in the Windows Store, that was 64-bit only.

22:03 we kind of wanted to test the waters and see, hey, will people notice that we haven't put a 32-bit version here?

22:08 Turns out no one did.

22:10 And so when we got to 3.9, had that change, we made it 64-bit by default. So that has a flow-on effect to the ecosystem.

22:18 A lot of particularly data science packages would rather just do 64-bit only packages.

22:24 Some of them certainly get theirs done first and not the 32-bit ones.

22:28 So we expect to see some flow-on impact from that.

22:31 just broader use of 64-bit Python throughout the Windows ecosystem.

22:36 Yeah, that's super cool. And just like, the final thought I had was, you know, Django dropped Python 2, and they're like, we were able to remove so much code, and it is easier for new people to contribute because they don't have to write for two ecosystems, they write for one.

22:50 NumPy did the same thing. And I feel like this is sort of the same story.

22:53 Like, you guys can just not worry about yet another older, outdated operating system.

22:59 and stay focused on what most people care about.

23:02 One thing that someone did suggest in one discussion was why not dynamically light stuff up for the newer operating system.

23:08 And the answer is we do that.

23:10 And when we drop the older operating system, we get to delete about 100 lines of code for each point where we do that.

23:17 So it is, we get to do a cleanup.

23:19 We get to say, oh, we don't have to dynamically check for this API, this API, load this, cache this, store that, call that, call this, fallback.

23:26 we can just condense that into, oh, we know that this API is there, so we can use it and just reduce a lot of kind of effectively dead code on newer operating systems.

23:37 Is that a pre-compile like, hash if-death sort of thing, or is it a runtime thing? Does it make a performance difference?

23:44 It definitely makes a performance difference, though we try and minimize it.

23:49 But again, there's always some impact.

23:51 It tends to be in operating system calls anyway, so you expect a bit of overhead.

23:55 and so it's not going to add a significant kind of percentage overhead compared to whatever operation you're doing.

24:01 But it does certainly add a lot of cognitive burden to someone who's reading the code.

24:07 One example that we got to clean up recently, not in a previous version, was we had about, I think, 70 or 80 lines of code to concatenate two path segments.

24:17 And this is before Python's loaded, so we have to do this with the operating system.

24:20 The API call up until Windows 7, I think, so pre-Windows 7 was not secure.

24:27 And it would, you know, buffer overruns, all sorts of horrible stuff, but it was the best available function there for handling certain cases.

24:35 So we'd use it, but first we'd dynamically look for the newer, safer one, and call that.

24:40 As soon as we dropped, I think, Vista, we could delete all of that code and just unconditionally call the one safe path combined function.

24:48 And that code got a whole lot simpler.

24:51 Yeah, lovely. That's awesome.

24:53 Brian, would you say it's more robust now?

24:55 Yes. I think it would be more robust.

24:58 Actually, I thought I showed up to the Bash podcast. Is this the Python podcast?

25:03 Yeah, this is. No, that's Bash Bytes.

25:05 I love Python, of course. I still use Bash regularly. And I know a lot of people that are like sysops people and other people are using Bash daily as well. So I wanted to highlight this Cool article.

25:18 This is an article by David Pashley called Writing Robust Bash Cell Scripts.

25:23 And even though I've been writing scripts for decades, I learned a whole bunch in this, and I'm going to start changing the way I do things right away.

25:31 The first two tips right away are things I'm going to do.

25:34 First tip is to include a set-u.

25:37 And never even heard of this.

25:39 What it does is it makes your bash script exit if it encounters an uninitialized variable.

25:45 So the problem without this is like, let's say you're constructing a path name or something or a long path, and one of the directories or file names you have in a variable, if that's never set, a bash normally just silently just deletes it, and it's just not there.

26:04 And it'll still keep executing anyway, but it's not going to be what you want it to do.

26:09 So, yeah, I definitely want to turn this on so I don't use uninitialized variables.

26:14 Similarly, if any of your script statements returns a non-true value, so that's usually in scripts or shell work, non-true value means something bad happened.

26:27 If you use set-e, that will make your script exit at any point if one of the sub-statements returns an error value.

26:36 So you don't want to just keep rolling with an error condition.

26:39 So this is good.

26:41 I hopefully, I'll cautiously add this to scripts because I want to make sure they keep working.

26:46 And then a tip just to say, expect the unexpected.

26:49 There will be times where you'll have missing files or missing directories or directory that you're writing into is not created.

26:55 So there's ways to make sure it's there before you write to it.

26:59 Especially if you're running on Windows, be prepared for spaces and file names.

27:03 And so variable expansion in Bash does not really isolate spaces.

27:10 So you have to put quotes around expansion to make sure that it's a single thing.

27:14 And one of the things right away, the next one is using trap and I've never actually knew how to do this before.

27:20 So if you've got a bash script that's running and it just something's wrong and it won't exit, you can kill it or other ways to get it to stop.

27:30 But if you have the system in a state that needs some cleanup, so that this there's a way to use a trap command to exit gracefully and clean up things.

27:41 The last couple of points were be careful of race conditions and be atomic.

27:45 Those are good things to do, but at least a handful of these I'll put to use right away.

27:49 So it's good.

27:49 - Yeah, this is neat.

27:50 And a lot of the stuff I didn't really know about.

27:53 So yeah, like continue on if something went wrong, just plow on ahead.

27:58 Yeah, that's cool to know that you can make it stop.

28:00 Steve, do you ever do any bash?

28:01 You got a WSL thing going on over there?

28:04 - I've certainly done a lot more bash since I started using WSL for a lot of things.

28:08 I was aware that using an uninitialized variable would substitute nothing, but I'm very happy to know that there's a way to kind of turn that off 'cause that has certainly caught me out in the past many times.

28:21 And this looks like just a good article that I'm gonna have to go read myself now because it has everything that you learn from doing scripts in like Command Prompt or PowerShell or even Python to some extent.

28:34 I have not personally mapped those to bash equivalents.

28:37 So it sounds like this would be a good place for me to go through that and up my skills a little bit.

28:42 My favorite thing was the find command.

28:44 And once I got that, that felt as powerful as a reg X.

28:47 And I'm kind of like, "Oh, I don't need to write a whole script now.

28:49 I can just do one excessively long find command." (laughing)

28:54 - Nice.

28:55 - Yeah, you find a lot as well.

28:56 - All right, this next one, I don't want to spend too much time on because I feel like we could easily just go and spend an hour on it.

29:03 but for time's sake, we don't have a whole lot of time left for the episode because we have a bit of a hard stop.

29:08 So I'm going to go through this and get your guys' thoughts on it real quick.

29:12 There was a tweet about a GitHub repository that was a conversation on the Python mailing list.

29:21 Lots of places. So Anthony Shaw tweeted calling attention to a roadmap by Mark Shannon called Ideas for Making for 5x Faster CPython.

29:34 So he laid out a roadmap and a funding map and some interesting ideas.

29:40 And I'm going to go through them quick and then especially Steve will see what you, what your thoughts are here, how reasonable this might be.

29:46 So the idea is like, there's going to be four different stages.

29:49 And each stage thinks you can get 50% speed improvement.

29:53 You do that four times, that's, you know, compounding performance interest, you get five.

29:58 So I think it talks about three nine somewhere, but anyway, I think maybe it's got to shift its numbers a little.

30:05 Anyway, so Python 3.10, stage one was to try to improve what the adaptive specialized interpreter that will adapt types and values during execution, exploiting type stability without the need for runtime code generation.

30:19 That almost sounds a little bit like what you were talking about with the 10% increase earlier, Steve.

30:24 And then 3.11, stage two, would be improved performance for integers, a less than one machine word, faster calls and returns through better handling of frames and better object memory layout.

30:34 Stage 3, Python 3.12 requires runtime code generation and a simple JIT for small regions.

30:40 Python 13, extending the JIT to do a little bit more.

30:44 And I'm linking to a conversation, a long threaded conversation, over on Python Dev.

30:52 There's a whole bunch of stuff going on here, to read through it, but there's just like a lot of interesting implications about like, how do we pay this if we pay someone to do it? People like Steve work on CPython, and they don't get paid. Like, how is it fair to pay someone else to do it when other people are volunteering their time? There's a lot going on here. Steve, what do you think about this?

31:10 Have you been following this?

31:12 I read through the original proposal. I haven't had a chance to chat with Mark directly about it.

31:16 I will, I guess, start by saying that Mark is a very smart guy. And he has done all of this planning off on his own in secret and kind of come out and share this plan with us, which you know is it's not an ideal kind of workflow certainly when you're part of a team, but I have certainly found in the past that when you get a very smart guy or very smart girl goes off and disappears for a few weeks and comes back and says I've solved it, there's a good chance they've solved it.

31:43 So I'm very interested to see where it goes.

31:46 The part of the discussion that you didn't mention is, or that you hinted at, is this is kind of a proposal for the Python Software Foundation to fund the work.

31:54 And that part of that funding is conditional on delivery.

31:58 So the way he's proposed this would work and and the implication seems to be that the mark will do the work himself and be the one getting paid for it.

32:06 Yeah, that seemed like it wasn't clear from his GitHub repo.

32:09 But if you read the conversation was like, look, I'm pretty sure I can do these things.

32:13 This is how much would make sense for me to spend the next couple years working on it and getting paid.

32:18 how do we do a fundraiser so that I can do this for everyone?

32:21 Yeah, and you know, I think under those conditions, if the PSF is able to put the budget towards it, they are in a bit of a tight spot since PyCon is normally the big fundraiser for the year, and that didn't happen.

32:32 On the other hand, it's also the big expense, but financially, the PSF is not in their normal place where they'd be for the year because PyCon didn't happen in the same way.

32:43 But I think if they're prepared to put funding towards this, I guess if the community consensus is that this is the most important thing for us to do and there's certainly potential downsides to doing it code complexity is the big one.

32:59 And I don't actually think there's a way that you implement this or even achieve these performance gains without making the code much more complex and hence less accessible to new contributors and you know people in earlier stages of learning to code at least on the C side.

33:15 Yeah.

33:16 So there's trade-offs. I'm very interested to see what would come about.

33:20 I assume that because 3.10 is targeted for the first pass, that it's already done.

33:25 And he's already got the code.

33:26 And he's just trying to get confirmation that he can spend the next few years heavily investing in it.

33:34 Instead of having to go find a full-time job and go back to doing this in the evenings.

33:38 Which I'm fully supportive of.

33:40 Again, it's really just a big open question of, is this the most important thing for Python to be funding right now, for CPython to be getting in particular?

33:51 Someone, I forget who, raised the question of what if we put that money towards PyPy instead?

33:55 What could they do with it in that amount of time?

33:58 And ultimately, it's going to come down to someone, probably a small group, presumably the steering council will have some involvement from the technical side, the Python Software Foundation board will no doubt be involved in just deciding is this the best use of the money that we have or can go out and get for what benefits it would produce.

34:21 Yeah, when I look at it with the funding side, I see it as very fraught with challenges on the sort of community funding and the PSF funding, but I know there's so many huge companies out there that spend an insane amount of money on compute and infrastructure that make a lot of money.

34:35 and that if they could have a 5x speed up on their code, they could probably save that money right away on infrastructure.

34:44 So it seems like that they could also get funded that way. But we should probably move on just because I've got to I'm going to make sure we have time for everything else before we end up running out of time.

34:52 I just do want to call out like you should go check out that conversation. There's a very funny excerpt from Larry Hastings.

34:57 It says, speaking as the Gillectomy guy, they were talking about borrowed references being a challenge.

35:02 "Barb references are evil. The definition of a valid lifetime of a barb reference doesn't exist because they are a hack baked into the API that we mostly get away with because of the gill.

35:11 If I still had wishes left at my monkey's paw, I'd wish them away. Unfortunately, I used my last wish back in February, wishing I could spend more time at home." So bad. All right, Siva, let's get a little bit more insight from you on this last one, huh?

35:26 Because you were at the Core Developer's Friends, which recently happened.

35:30 Yeah, so I don't know exactly what day this is going to go out, but last week from recording day, we had the CPython Core Developer Sprints. So this is kind of a get-together, generally in-person event that the core development team has done for five years now. I think this is the fifth year.

35:49 In the past, we've all gone down to Facebook or Microsoft or last year we hung out at Bloomberg in London, and basically spent a week in a room together, coding, discussing, reviewing things, designing things, planning things, and otherwise just getting to actually meet our other contributors because we all work online.

36:10 We all mostly work over email and kind of bug tracker and GitHub pull requests throughout the year.

36:17 And so it's a really good opportunity to get to meet each other, get to see who we're dealing with.

36:21 It's a lot harder to be angry at someone over email when you've met them.

36:24 Yes. And so it's been a really good event. This year, because we're obviously not traveling for it, we were hosted by Python Discord, which is at pythondiscord.com. There's a server that is really well managed. It's really well organized. I was impressed. I have not been there before, but it was great. They set up, felt like thousands of channels for us, far too many, but it gave us plenty of space to kind of mingle with other core devs while we were discussing and working and planning anything.

36:53 We also did a Q&A, so there'll be a link in the notes for that, from YouTube that we live streamed, we had people submit questions ahead of time, everything from what situations should I use a mangled name in, like a double underscore led name, through to what's your least favorite part of Python, what do you most want to replace, did you ever expect Python to get so big, and we had a lot more people involved.

37:15 We normally do a panel for the host company.

37:19 So we'll get kind of their employees together and it's like part of the perk for funding the venue and typically meals and coffee and everything for the week.

37:27 This time it was public on YouTube.

37:29 It was all kind of over video.

37:31 So everyone got a bit of a turn to jump in.

37:34 So you'll get to see a lot more core developer faces than you've probably ever seen before.

37:38 You'll get to hear from a lot more of us than you have before.

37:41 And a lot of interesting things.

37:43 The big kind of ideas that came out of the week, kind of hard to say.

37:48 A lot of us did come out feeling like we didn't get as much forward momentum on stuff as we normally would in person.

37:54 But at the same time, a lot of things did move forward.

37:57 I think there were about seven or eight peps passed up to the steering council during the week, various things.

38:03 One of mine was deprecating disutils, which is an entire podcast on its own.

38:08 So I might have to call you guys another time to talk about that one.

38:12 Through to a proposal to change how we represent 3.10.

38:18 Because a lot of places we put the version numbers back to back with no separator.

38:22 And so you have, you know, 3839 with no nothing in between.

38:25 Now we're up to 310.

38:26 Or is it 310?

38:28 Yeah.

38:29 Okay.

38:30 How do we fix that?

38:31 And we had a lot of discussions about that.

38:33 There was obviously a lot of talk about JIT, about the C API, all the usual things that we talk about. But again, because it was online, it was really good to have such a range of people involved, you know, across different time zones, and people who would not normally get to travel.

38:48 Yeah, it makes it more accessible. Yeah, that's awesome.

38:50 Yeah, we have core developers in countries who can't leave.

38:52 Like they literally cannot leave their country, either because the populace is just strictly controlled, or they know they would not get back in when they tried to go home.

39:02 And so they were able to participate and that was great to you know, see and meet some of those people We had a few mentees come along to interact with the rest of the team And just overall a good week. Awesome. Yeah, cool. Yeah, people can check it out the youtube stream I definitely want to check that out. Sounds neat. All right, brian. We've got two minutes left Do you think we should do a joke? Yeah, let's do the joke. Let's just cut to the joke So we we don't miss that right? So you and I spoke about hacktoberfest going wrong and like random PRs to like, config files and changing the spelling and config file settings.

39:37 So there was a guy who posted on Twitter, said, "Hey, let me double check the name." It was Stuart McCroden.

39:48 And he posted this cool t-shirt that he got.

39:51 It says, "Hacktoberfest 2020. Any PR is a good R." And it's Lua.py.

39:56 And it has import pyenvim.

39:59 Then the pr just adds hash this imports a package That's awesome steve, did you suffer from any of these?

40:07 I did not I might have done my github notifications are a mess So yeah, I I don't even know yet Yeah, I don't I don't see poor requests until I actually go look at the repo myself for the most part Yeah, I got a bunch. I got a whole bunch. Yeah me too. Okay. I wanted to do one This should have been a topic, but the five most difficult programming languages in the world.

40:30 This was submitted to us by Troy Caudill, I think. It's not really a full topic, but I thought it was hilarious. This is an article where the author, Locate, I guess, actually took five programming languages, Malboge, Intercal, Brain, we all know that one, cow and whitespace and wrote hello world in that language.

40:55 And these are hilarious. And my favorite is whitespace because the entire language depends on space tab and line feed for writing the program.

41:03 And any non whitespace characters considered a comment. So this is great.

41:08 That's crazy. I don't know why the APL wasn't on there.

41:13 APL is just fully insane. Let me, I'll put it just in the show notes here at the bottom of this, an example of APL. That right there that I put on is I can't try to even speak this, but that is an entire thing that finds the all prime numbers from one to R in that that line. I hope you guys see at the bottom of the notes. That's insane, isn't it? That's not even intentionally bad, is it? I mean, it's meant to be a real programming language. It's as if the Egyptians who only wrote in hieroglyphics decided to write a programming language. That's how I feel Like you it's insane, but it's a legitimate language.

41:48 Like people try to use it.

41:49 They do use it anyway.

41:51 Not, not for very long.

41:52 I expect only as long as they must.

41:55 And then they immediately stop.

41:56 I just liked that.

41:58 This intercal example is so polite.

42:01 It's please, please do please do please.

42:03 Oh, please give up.

42:04 Yeah.

42:07 Apparently you have to sprinkle pleases in it or else it'll like a error because you're not polite enough, but if you do too much, it also errors because you're overly polite.

42:17 I like that. We need more passive-aggressive languages like that.

42:20 Lovely.

42:23 Cool. Well, thanks a lot, guys. It was fun.

42:26 Yeah. Thanks, Brian, Michael. Yeah, thanks. And thanks for being here, Steve. It's great to have your perspective. Thank you for listening to PythonBytes.

42:33 Follow the show on Twitter @PythonBytes. That's PythonBytes as in B-Y-T-E-S. And get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way. We're always on the lookout for sharing something cool. This is Brian Okken, and on behalf of myself and Michael Kennedy, thank you for listening and sharing this podcast with your friends and colleagues.