Transcript #263: It’s time to stop using Python 3.6

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 263, recorded December 15th, 2021. And I'm Brian Okken.

00:12 I'm Michael Kennedy.

00:13 And I'm Lais Carvalho.

00:15 Welcome Lais. Before we jump in, could you tell us who you are?

00:20 Absolutely. So first of all, thank you so much for having me. Thank you for the invite.

00:25 So my name is Laís Carvalho and I'm a developer advocate for QuantSite.

00:30 And I also do a little bit of voluntary work for Python Ireland and a few other Python things around the sphere.

00:40 Nice.

00:41 That sounds really fun. The company you work for, I'm not familiar with it.

00:45 What do you guys do there? It sounds a little bit like maybe Quant Finance, maybe some ML stuff.

00:50 Yeah, so we do everything.

00:52 So Quonsight is a consultancy company.

00:56 So basically our CEO is one of the main guys behind NumPy.

01:05 And yeah, basically we do consultancy and everything you need.

01:10 Yeah, sounds fun.

01:12 Brian, you know what else is fun?

01:13 What is fun?

01:14 Websites.

01:15 Yeah.

01:15 Yeah.

01:16 And if you think about websites, what web framework in Python is used more than Django?

01:22 Flask?

01:23 Maybe Flask.

01:24 I think they're right on the boundary, but certainly Django is at the top.

01:29 And I hear so many stories from people who say they got into Python not because they love Python, but because they wanted to do Django.

01:36 I even had a guest recently describe themselves as a Django developer who learned Python because They just wanted, they came for the Django and they stayed for Python, which is cool.

01:47 So Django has been on a bit of a terror lately in terms of its speed.

01:52 So I actually went back and looked.

01:54 If you look at the release for Django 1, that was in May 2010.

02:00 I think it came out before then, but that's as far as the release history goes back in dates.

02:04 But so 1.0 was in 2010.

02:06 Seven years later, 2.0 came out in 2017.

02:09 It seems like that just happened.

02:11 Time's flying.

02:11 >> Then in 2020, three years after two was three, and then now we have Django 4.0.

02:19 It's going seven years, three years, one years, who knows.

02:23 >> So do we get like five in March?

02:26 >> Yeah, exactly. Maybe.

02:29 But it's cool, Django 4 is out.

02:32 It's not a super huge release in terms of what's covered, but some of the highlights over there include, I think I mentioned this that it was coming, but now it is here.

02:41 the new Redis cache back in.

02:43 So there've been previously ways to plug in or use Redis as a cache back in inside of Django.

02:50 But now it's just, you just use Django's caching implementation and you tell it, by the way, here's the connection string to Redis, and that happens.

02:58 So that's pretty cool, right?

02:59 >> Yeah.

03:00 >> Another thing that Django does for you is it will render forms and stuff based on various things like models and whatnot.

03:07 And so they've changed how that works to ease customizations of forms and form sets and errorless.

03:13 They are now rendered using the template engine.

03:15 So you have more control over how forms and stuff look.

03:18 And some changes to do with the time zone as well.

03:21 It's now using Python's time zone.

03:24 So I guess it used a different one before.

03:26 I'd never really paid that much attention to which time zone it used.

03:29 But so with the release of 4, 3.2 has now reached the end of mainstream support, which is interesting.

03:37 and lies what you're gonna talk about later.

03:39 Actually, this is relevant as well, right?

03:40 I know the thing you're covering talks about Django and versions and upgrades and so on.

03:46 So that's it.

03:47 Django 3.2 is a long-term support release.

03:50 So we'll receive security fixes until April 24, but there's no more features coming to 3.2.

03:56 All the new features and improvements and whatnot are going into 4.0.

04:00 So that's pretty cool.

04:01 And let's see if you go over there, release notes.

04:04 it now supports only Python 3.8 to 3.10, which I think is pretty cool.

04:10 - That's interesting.

04:11 - Yeah, that's no messing around, right?

04:14 They're like, nope, we spent too long supporting 2.7.

04:18 We're not doing that anymore, right?

04:20 - Yeah, I think most frameworks are not supporting 2.7 anymore.

04:24 - There's a lot still supporting 3.7 though, so it's interesting that 3.7 is out as well.

04:29 - Yeah, well, they've got the 3.2.1 that still supports that if you want to kind of be on the slow burn.

04:34 Let's see what else.

04:35 There's a few things, I'm not going to go through it, I guess one more highlight maybe is you can create composite indexes really easily with the unique constraint.

04:44 So you can have a uniqueness constraint that has the first name and the last name.

04:49 So you could have the same first name or the same last name as somebody in the database, but not the same first and last name.

04:54 That's pretty interesting.

04:55 They upgraded the password hasher to script.

04:58 I have no idea what the script password hasher is, but it's more secure than the PBKDF2 hashing algorithm.

05:06 You never want those to be too easy 'cause if you can brute force guess them, that's bad news.

05:11 However, if you wanna be using this better password hasher, you have to enable it because it requires open SSL 1.1 or higher on the system and they don't assume that you have that, right?

05:23 So you've gotta basically do some work to make that happen.

05:26 And then there's a bunch of things about what was added, what was deprecated.

05:30 And again, sort of leading ahead, there's some backwards incompatible changes, things that are breaking changes, and there's also deprecation of some features that were not previously deprecated.

05:42 So you wanna be on top of that as well.

05:45 So Django 4, yeah, we're gonna get Django 5 in March or something, I'm guessing, right, Brian?

05:51 - Yeah, I think so.

05:52 - And can we take a moment to just say thank you for the amazing documentation that Django has all over?

05:58 It's so good.

05:59 Like I'm a newbie and every time I open, I don't really use Django that much, but every time I open their documentation, like sometimes I actually use their documentation to just check stuff about the web development because it's just so, so good.

06:14 So thank you very much maintainers of Django documentation 'cause it's really, really good.

06:19 - Indeed, indeed.

06:20 Sam Morley on the audience says, "I'm always impressed by how much Django managed "to cram into each release." Indeed.

06:27 Pender says, "Howdy all from Ireland." >> Hello.

06:30 >> Nice.

06:31 >> Right on. Cool. Brian, what do you got?

06:35 >> I got Python minifier.

06:37 Actually, this was a suggestion from Lance Rainsmith.

06:41 Thanks Lance for sending it up.

06:42 But he also sent some reasons.

06:44 My first reaction was, we don't need a minifier for Python. What's up?

06:49 This isn't Java or something.

06:52 But he gave a couple of reasons.

06:55 the docs for minifier have this sort of convoluted reason. Apparently AWS Cloud Formation, I don't know any of the details of this, but AWS Cloud Formation templates can have AWS Lambda functions source code embedded in them, but only if they're 4k or less. So a small amount of code. So that's That's one of the reasons why I wrote this package.

07:21 Lance, the person that sent it in, said his reason for using it is Adafruit boards, which is a cool idea.

07:30 I didn't think about that with like, you know, Adafruit boards or in CircuitPython, you've got less space to work with.

07:39 So minifying your code might be a good idea.

07:42 That's pretty cool, decent idea.

07:44 So what does this do?

07:46 It's this, like there's an example on the page of just normal Python code.

07:51 It's maybe not the best Python code, but normal-ish.

07:56 And then it replaces it with like, you know, replaces the variable names with A, B, C, things like that.

08:01 Some of your typical renaming things.

08:06 So it has like a whole bunch of different techniques that it uses.

08:10 I'm gonna go to the docs.

08:13 It combines the imports, so instead of on multiple lines, you just combine them all together, saves a little space, removes passes, which is interesting, it removes them to, just assigns it to zero in a function.

08:28 That's interesting. I didn't know you could do that.

08:30 Hoist literals, that's an interesting thing.

08:33 That's just if you've got a literal that's a bunch of different places, it just defines a variable and defines it at the top.

08:41 Kind of interesting.

08:42 >> It uses a lot of semicolons, that's wild.

08:44 >> Yeah. Remove annotations.

08:48 So type annotations, you probably don't need it runtime for something like this, so you can remove them.

08:53 Local renaming, a whole bunch of stuff, renaming globals and literals.

09:00 Sometimes if it's an API or if something else is using it, you don't want to rename the globals because those are the API.

09:08 So you can turn any or all of this stuff off.

09:11 I guess you wouldn't turn all of it off because why would you use it?

09:14 But you can turn off stuff that doesn't work.

09:16 So I would suggest if you're using Minify, test your code of course, and remove what you need.

09:23 And I actually for something like a CircuitPython and stuff like this, this might be an interesting idea.

09:29 And I'm curious--

09:31 - Let me put out another idea about where it may be useful.

09:34 - Okay. - Can you go back to the example?

09:36 So just that code sample you had?

09:38 - Yeah.

09:39 - So if you look at it, One of the things you often have to do, and I'm not suggesting this is really a good idea, but it could be applied this way.

09:47 One of the things you have to do if you wanna ship your Python code around to be used with places is to share your source code, right?

09:53 And sure, you could send a PYC file, but it's basically the same thing, right?

09:58 - Yeah. - So what about if you go in here and run this against something that you want just a little bit more obfuscation?

10:06 You don't want people poking around it.

10:07 That definitely looks less easy to read to me.

10:10 - Yeah.

10:11 - I mean, it could go farther, right?

10:12 It could rename like create certain and whatnot, but maybe those are like standard library things.

10:18 But certainly if you're trying to obfuscate your code as well, there could be some sort of use case there.

10:23 Although it's not super obfuscated, so.

10:25 - Yeah, it seems like that was a Java thing.

10:28 I don't even know if they do that anymore, obfuscators.

10:32 - Yeah, I don't know.

10:32 I haven't been in that world for a long time either.

10:35 - It might help also if you're playing golf, you know, coding golf.

10:39 You know, just write one of these.

10:42 - Yeah, indeed.

10:43 - What is coding golf?

10:45 - Sounds like a thing that black should have, maybe black.

10:49 - Yeah, that's a terrible idea.

10:51 - Yeah, I don't know what coding golf is either.

10:53 - Really?

10:54 Okay, so you have like a coding--

10:55 - I just go to the driving range.

10:56 I don't actually do any more things with golf, so I'm not very advanced with it.

11:00 - Coding golf is like the lowest scores the best, right?

11:03 In golf.

11:04 So it's coding challenges that you solve with the minimal characters.

11:10 So minimal lines of code, minimal characters.

11:15 And it's, yeah, that's what Coding Golf is.

11:18 - So you just show the Zen of Python out of the window.

11:20 - Exactly.

11:21 It's not easy to read code, but.

11:25 - Okay.

11:26 - Got it.

11:27 - All right, sounds fair.

11:29 May the best man win, or the best person win, sorry.

11:34 - Indeed.

11:34 - Yeah, also one of the things it does is that it doesn't advertise, is it throws away all your spaces and replaces them with tabs.

11:41 - Oh, interesting.

11:44 Yeah, it does look very tab filled in the--

11:46 - Well, it makes sense.

11:47 You know, four spaces versus one tab.

11:49 It's a 4x reduction right there.

11:51 So, yeah.

11:52 - Sam says, "Lias, you're in for a treat.

11:56 "CodeGolf Stack Exchange is fascinating." (both laughing)

12:01 - I can only imagine.

12:02 - Indeed.

12:05 All right, well, your topic is up next.

12:09 This is the one I was alluding to earlier, talking about Django and support for old things and so on.

12:13 - Yes, yes, indeed.

12:15 I think it's a way for the tip as well.

12:17 So then this is, it's time to stop using Python 3.6.

12:21 So in one week and one day, Python is dropping support for, well, Python 3.6.

12:29 So there's not gonna be any more releases after that.

12:33 So basically, well, you should update Python.

12:37 If you are using Ubuntu, then you can say, "Well, I don't really have to update right now because Ubuntu has long-time support." But it doesn't necessarily work like that because, well, the Python dev team is not gonna support your Python 3.6 anymore, which basically means that--

13:04 - Exactly.

13:05 - Yeah.

13:06 - That's what we were just talking about with Django, right?

13:08 Like just because you might be on an LTS version of Ubuntu that will still let you use 3.6, well, you're gonna have to give up all the new libraries like so long to Django 4.0 and probably FastAPI and all the things, right?

13:20 - Yes, exactly.

13:22 So you have your dependencies that are gonna need to be updated anyhow.

13:26 So you should update.

13:28 But then I have a question.

13:29 So Noobie here, I looked around when we were preparing the notes for the show, I looked around and I tried to, cause I'm using 3.7, so I'm still okay.

13:40 but I was looking, I wanted to try the 3.10 new stuff because I listened to a few podcasts ago and you're talking about the wonders of Python 3.10 and I was like, okay, so let's update.

13:54 But then I couldn't find the best way to update from 3.7, or let's say 3.6, to 3.10.

14:02 I saw some people saying that it would be better to just update gradually, so go from 3.6, 3.7, 3.8, 3.9, and then 3.10, 'cause it could fix the dependencies easier like that.

14:13 But then some other people just say, just go straight to the version you want, and then just deal with the problems when you get there.

14:19 - Just rip the bandaid off.

14:20 Yeah, I don't know. - Yeah.

14:21 I'm with Brian to go as far ahead as you can and then see if it works.

14:26 - All right, so just go 310 straight away, okay, cool.

14:30 - That's what I would say, but the question is still open, how do you do that?

14:35 Right?

14:36 It depends, I think, on what OS you're on.

14:38 For example, on Ubuntu, 3.9 is really about as far as you're gonna get without some jumping around.

14:45 Like that's the latest Python 3 that it'll give you.

14:48 On Mac, you only have two, so you gotta install something, but you could download the installer from python.org.

14:54 You could homebrew it or you could do other things.

14:57 You could do pyenv like Brandon Brainer out in the audience has suggested.

15:01 I got started with pyenv.

15:02 It makes this kind of thing trivial, true, except for I can't get it to work on my Mac.

15:07 Maybe it's an M1 thing or an Apple Silicon thing, but it just won't install anything.

15:12 So it's not trivial for me, but when it works, you can just say pyenv, give me three, six.

15:19 And you can be very specific, like give me PI Python 311 RC2.

15:23 I want that one.

15:24 And it'll just, it'll put them all side by side.

15:26 So I, mate, what do you think, Brian?

15:28 - I think I'm on PI ENV now, but it's because I test a lot of, a lot of, a lot of things.

15:36 For normal people that have an application and they just want to upgrade.

15:40 I say, unless you're on Ubuntu or Linux, I say just go to python.org and download the latest one and install it.

15:50 It works on Windows and Mac, usually.

15:53 On Windows, there's the Windows installer.

15:55 You can go to the Windows Store, that works.

15:58 But the Python.org one works good too.

16:02 So, usually.

16:03 - Yeah. - All right.

16:04 - Yeah, Indy.

16:05 - Cool, so I'll try, and I'll let you know how it goes on Twitter.

16:08 - Right on.

16:09 - What we can do.

16:09 - Give us a report of what you think kind of works.

16:11 - Yes, 'cause I threw the question on Twitter as well, and then I got a bunch of people saying, don't go 310 yet, it's not great for production.

16:19 - Go three nine.

16:20 If there are people just say no, just rip the bandaid as you did.

16:22 - Yeah. - So, okay.

16:24 - The Akan doctrine.

16:26 - I mean, I only support little tiny projects though.

16:29 So in Python, I support big things in C++, but I would run your tests, you know, upgrade, run the tests, make sure everything works.

16:40 - Yeah.

16:41 Oh, well, as your conversation's got a whole bunch of stuff going off in the audience here.

16:44 Alvaro says NumPy is dropping support for three seven sometime this month.

16:48 So that should push the data science community to update.

16:52 And then also on Ubuntu, you can install the dead snakes PPA and you'll get now versions of the latest version of Python.

17:01 Okay, that's interesting.

17:02 - Yeah, I saw the dead snakes PPA, yes, indeed.

17:05 And I saw that in Ubuntu that's really easy to do it, but still you can find tutorials saying go gradually and then in other tutorials, it just goes straight in.

17:17 Sam says just compile from source every time.

17:19 That's what I do.

17:20 That's what I'm doing in production, but I'm doing homebrew on my Mac.

17:23 All right, maybe that's enough.

17:24 How do I get the latest version of Python?

17:26 (laughing)

17:28 But you were gonna make a comment about Java.

17:30 Is that right?

17:31 - Oh, yes, yes, that's true.

17:33 So, 'cause I saw, I was doing a little bit of research for this and then there was a little bit of discussion of the third-party libraries and the frameworks that will drop support for Drupal 6 soon enough, as Alvaro just mentioned.

17:51 So then I saw on Twitter, someone mentioning, let me put it here, that using the Java, the new Log4j problem, let's put it like that.

18:05 - Log4j, it's so bad.

18:09 This is so bad.

18:10 - Yeah, so using it as an example for how you should upgrade your version, the version of the programming language you're using.

18:20 And then they talk about, yeah, so they say that you should upgrade as soon as you can.

18:28 - Yeah, what an interesting example they gave because this log for shell thing that I'm gonna actually talk about a little bit later.

18:35 But the problem is you have to upgrade log for J to a patched version.

18:39 The patched version only works on Java 8.

18:41 So if you've been dragging your feet in a sense and staying on an old version of Java, well now all of a sudden it's not just replace a library, it's completely upgrade the Java you're running on.

18:51 And if you were dragging your feet that hard, there's probably some kind of reason that it's a pain and not well maintained or something.

18:56 And so instead of just going, oh, we got to quick fix this problem because it's a super big deal.

19:02 It's now all of a sudden you've got to do a major upgrade when you weren't planning on it, right?

19:06 That's bad.

19:06 - Yeah.

19:08 - And then you have to have this meme.

19:11 - What a wonderful meme.

19:13 - Your next task, this is from the squid game.

19:15 Your next task is to figure out which applications in your organization is using log4j.

19:20 You have 10 minutes.

19:23 - Go.

19:26 - Oh, that's good.

19:29 I didn't catch the meme.

19:30 - Yes, so then I think my code did soon enough.

19:34 - It is, it is, awesome.

19:36 Well, yeah, thanks for highlighting that article.

19:38 That's a good one by a tomorrow.

19:40 Trowel turning, turning, trowel, trowel, turning.

19:42 Yes.

19:43 okay.

19:44 So what do I got next year for us?

19:47 Not Django, but something fun.

19:49 We just had what was probably the most contentious formula one season in 10 years, at least, and what are the most wild ones for sure?

20:01 The championship was decided on the final lap of the final race.

20:07 Like literally the two people who were contending for it, Hamilton and Verstappen, they were literally tied on points going into the final one.

20:15 So, and the final move was done in the final lap.

20:17 So I wanted to highlight this article here called "How to Visualize the Formula 1 Championship in Python Using the AirGAST API in Seaborn." So for people who are in data science, I know if you're learning it, one of the challenges I have of learning those tools is I don't have a great professional need for them.

20:38 I don't use that kind of stuff to analyze data that I work with that frequently.

20:41 So my chance to learn Seaborn, for example, it's pretty limited.

20:45 So finding some example or some fun project you can do, that turns out to be super important.

20:51 So I thought I'd pull this up as something people could do to learn to play with this kind of data.

20:56 Now, if you're not yet into this stuff, check out the "Drive to Survive" season three.

21:03 I think I got the wrong link here, but I put the trailer to it in the show notes.

21:09 So Netflix did a great series that is really good for people to just sort of get into it.

21:15 And also there's like a eight minute highlight of the last race, which was absolutely crazy.

21:20 But what I wanna talk about is some cool stuff that Jasper, just goes by Jasper, no last name, over on Medium.

21:27 So introduces this thing called the Airghast API, which if you go over there, it's looking a little old school, but not very, how do you say, not very restful, for example.

21:38 So for example, you can come over and we could get like details about the third race of the 2008 season by just going API F1 2008.3, and it's gonna come back with, well, a series of tables, which is not necessarily that useful.

21:55 And I thought the way you get the JSON, you might do like an accept application/JSON type and not HTML or something, but no, you say three.json, and that gives you the API there.

22:07 So that's pretty cool.

22:08 And you come through here and basically, they talk about how to use Pandas, NumPys, Seaborn, and that API to just build some cool graphs that actually show you the position of all the drivers across the entire season, right?

22:23 So if you wanna build this picture, you see right at the top of the article, and play around with that data from that API, then this is the way to do it.

22:33 One other thing that's pretty interesting from this article that I had no idea about is over on PyPI, we have fastf1 as a package, which is a wrapper library for F1 data and telemetry APIs with additional data processing capabilities.

22:50 That's pretty cool.

22:50 Yeah, so you go down here and like, they'll show you how to build this really cool graph of like lap by lap, the times, the various drivers, you can see in the beginning, Like there's all sorts of chaos because there's people jockeying for position, the cars are heavy, they're full of fuel, so they go slower.

23:07 You sort of see as things spread out, as the cars lighten up and stuff, 'cause they burn off 100 kilograms of fuel, then they all kind of get faster and better times.

23:16 So you can get like cool graphs and exploration of all this stuff here.

23:19 And the way you do it is really interesting is you just go over to this fast F1 and you enable the cache 'cause it downloads some stuff and you don't wanna have to download it twice.

23:28 You can say, give me this season, this Grand Prix, give me the race details, load up the laps, and then you can start just working with the data in this really interesting way.

23:34 So if I was into data science and I wanted to learn some stuff, this might be a really cool real-world data set that you can play with.

23:41 What do y'all think?

23:42 - I like it.

23:43 I think I might actually try to know more about Formula One just so that I can talk with Anthony more.

23:49 - Exactly.

23:50 Anthony Shaw's a big fan.

23:52 Yeah, indeed.

23:53 As am I these days.

23:54 It's a lot of fun.

23:55 It was really a cool season this year.

23:58 And yeah, check out the highlights from the last race.

24:00 It was absolutely off the hook.

24:02 Liza, what do you think of that in terms of a data science project here?

24:06 - A data science project, wonderful.

24:08 Like, especially if you actually like this.

24:11 I have no knowledge on F1.

24:14 I actually worked for a company that was like, I think the founders of the company were in love with F1 and that's how they got together to found the company.

24:23 And I got to talk to them and like a certain one specific circumstance.

24:29 It was like, "Oh, you like F1.

24:31 That's awesome.

24:33 I know nothing about it, so please enlighten me." (laughs)

24:38 That was it.

24:38 - If you're interested, check out the Netflix Drive to Survive series.

24:41 That's the easy way in.

24:43 It's lots of drama, not like the boring sort of 30 minutes in between the things that actually happen.

24:48 - Okay.

24:49 - What some of the people from the US don't understand is that they can turn in two directions.

24:53 That's odd.

24:54 (laughs)

24:55 - That's so odd.

24:57 However, Brian, just before we're off this, the US Grand Prix for the first time, I think was the most attended race ever, certainly for F1, but maybe, it had 400,000 attendees in Austin, Texas this year.

25:10 - Wow.

25:11 - So apparently it's becoming a thing, like soccer became a thing in the US when it didn't used to be, and now it is.

25:16 I think this is as well.

25:17 - Yeah.

25:18 - All right, Alvaro also says, interesting, about interesting libraries.

25:21 Last week I found DuckDB super interesting for data science as well.

25:25 - Yeah, definitely.

25:26 We've only touched on that a little bit, but yeah, it's a great one.

25:28 - Yeah.

25:29 - DuckDB is like a new database, like, I don't know, CockroachDB?

25:33 - A different, but I believe it has integration with like pandas and stuff, if I recall correctly.

25:41 - I think so.

25:42 - I'll have to look it up, but yeah.

25:43 - Cool.

25:44 - I'll do some real time followup, but Brian, tell us about NBDIME.

25:47 - NBDIME.

25:49 So another listener suggestion, Hendrik Vinsberg, cool last name.

25:54 He said, we recently covered JOOT or JUT, Jupiter Notebook, yeah, Jupiter Notebook Terminal Viewer.

26:05 We covered that in episode 258, he's correct.

26:08 But he said, hey, you should check out MBDIME.

26:11 So, because it's been around for a long time and it does something similar.

26:15 So I definitely did.

26:17 And I think this is a really cool set of tools.

26:20 So we've got, not only does it do an NB show, which does a--

26:24 - Right, that's the JOOT equivalent of the NB show, which is cool.

26:27 - Yeah, but this is a set of tools that is around the needs for diffing and merging notebooks.

26:34 So like if you're keeping notebooks in a Git repo, you'll have to do things like this.

26:40 And so it's got a bunch of tools.

26:44 It's got diff, it's got merge, it's got diff-web, so you can have a rich rendered diff of the notebook and NBMergeWeb, so a three-way web-based merge of notebook tools.

26:57 This just looks really cool, and I definitely think it's worth checking out if you work with notebooks and work with teams, you need to do things like this.

27:06 - Oh yeah, this looks great.

27:08 Yeah, diffing and merging notebooks is tricky, for sure, because the actual on-disk representation is basically a JSON document and then it embeds the output in there as well.

27:21 Right, so if you ran it against live data and you got some output and you run it again, of course that's gonna be different, but that's a straight get, that's a merge conflict, right?

27:28 - Yeah, I'd be interested to know how they deal with that.

27:31 I haven't looked through this, but you probably--

27:34 - It looks like it understands the cells and then just the cells or something along those lines.

27:39 - Probably.

27:40 - Yeah, they're super cool.

27:41 - Yeah, this is a great one.

27:42 - Thanks for the recommendation as well.

27:45 All right, Lais, you're up next.

27:47 - Yes, so we found out, so first of all, this is a recommendation from Fridolin.

27:54 He works, he's a software developer at Red Hat, so thank you very much for the suggestion.

28:00 And this is Project Dot.

28:04 So basically, this is an open source, cloud-based Python dependency resolve.

28:10 So we all know that PyPy just launched a new dependency resolver.

28:17 Yeah, that was a big deal.

28:20 There were some breaking changes to pip and stuff like that.

28:22 Yes, and well, show the first stone who never had dependency resolving issues with pip.

28:31 We have spent so much time like, "Okay, so it's not this library, it's this other one.

28:37 No, it's not this version, it's this other one." So this project path basically comes to solve this issue, and it uses reinforcement learning to solve these dependency issues.

28:48 So basically, from what I understood, it's a smarter PIP.

28:54 So basically, it uses a few inputs from the developer, such as the speed of the execution of the code, and if there was any errors, and your hard drive version, etc.

29:10 Yeah, it's not just what version do you want, like greater than 2.1, but you could say, I would like one that's relatively stable, or I would prefer security over recency or something along those lines. So there's a lot of different inputs to this machine learning model that then will give you the result of like pip install a thing.

29:29 Yes, and then it pre-computes this whole dependency information, it puts in a database and then just keep track of it.

29:37 And then the logs that result from actually installing the dependencies of that environment, then I use to go into as input into that reinforcement learning algorithm.

29:50 And they use that then to create future resolutions.

29:54 So basically just make the whole process of dependency resolving much faster and much easier.

30:00 - Yeah.

30:01 - It sounds like a great thing and I tried testing but well I got it to a little bit of an issue with my past versions. I was like oh I'll deal with all of this later.

30:14 Yeah exactly. Can you scroll down to the code sample where it's talking about Temos or Themos I don't remember the term exactly. It's a little further down it's like a code callout. Keep going. I've gone a little bit right there. So yeah so the way you can use it as you pip install this Thamos thing and then you configure environment and then you add dependencies with some suggested version like approximately flask one and then you ask it to advise you on your current setup and it'll go through and it actually analyzes the code and does a whole bunch of stuff and then it'll give you some information. I don't remember exactly all the details of what it would suggest but it'll give you information about the various dependencies that you have installed and so on.

31:00 - And also give you suggestions on like hardware and OS versions that you should be using.

31:05 And it's quite smart.

31:08 I thought it was quite interesting.

31:09 And it's in beta phase.

31:12 So if anyone that is listening would like to try it and then maybe just give some feedback back to Red Hat, they'll be very, very, very happy to hear from the community as well.

31:24 'Cause it's an open source project.

31:25 So please, collaborate.

31:27 - Yeah, check it out.

31:28 It looks like it might have something to do with OpenStack and their whole cloud story over there as well.

31:34 Although I don't believe it explicitly said it.

31:36 It doesn't seem to be tied to it, but it seems maybe motivated by that as well.

31:40 So yeah, cool.

31:41 That's a good one.

31:42 - Interesting. - Yeah.

31:43 - Brian, some real-time feedback here.

31:45 So we covered this article, Efficient SQL on Pandas with DuckDB.

31:51 And one of the cool things you could do is create a Pandas data frame and then do select queries on top of the data frame.

32:01 - That's pretty cool.

32:02 - Yeah, so for example, remember you have like a local variable and you say the name of local variable in the query and it like reaches into the call stack and grabs that out of the context and then works with it.

32:14 So yeah, anyway, this is one of the data sciencey things.

32:17 - Nice.

32:18 - Yep. - Fun.

32:19 - Brian, is that it for all of our items?

32:21 - I think it is, yeah.

32:22 - I think you might be right.

32:23 - Do you have any extras? - What about the extras?

32:24 You got any?

32:25 - I do, actually, nice timing for today.

32:28 So Pragmatic Bookshelf, they're the publishers of the Pytest book.

32:33 They're running a sale, actually 12 days of Christmas sale.

32:37 So every day is something different.

32:39 You don't know what it is.

32:40 I don't know what it is, but today my book is part of it.

32:44 So it's, you have to enter a promo code and you get 50% off $50 spend.

32:51 So you gotta order a couple of books 'cause they're not that expensive.

32:55 But anyway, it's pretty cool.

32:57 - Cool, yeah, hopefully that does well.

32:59 Lais, how about you, got anything extra you wanna give a quick shout out to while we're here?

33:03 - Oh yes, of course.

33:04 So I am a volunteer for Python Island, and Python Island has a speaker's coaching session happening on the 22nd of January.

33:16 So it's basically we're offering for people that would like to be a pro on giving talks, We're offering a workshop. It's the entire day and you need to Say that you need to just like give us your words that you're going to give us a talk As like a practice thing. but it's quite it's a very interesting um Workshop thing and if you would like to be part of it, just go into the meetup.com Slash python ireland and it's there. So I guess the 22nd of january - Yeah, that's a cool service, I guess you would call it, a cool option for speakers, because speaking is one of those things that really can open doors and help get you connected, but it's also super stressful the very first time you do it is in front of the audience and it's being broadcast.

34:06 - And also like the tips on how to put a talk together, 'cause there is that anxiety as well.

34:11 It's like, I have a talk, but I don't know if my, the thing that I want to talk about is a good thing, or I don't know if my slides are good, I don't know if I know how to actually convey the message.

34:24 So we would love to help you with that.

34:26 And I won't be the one giving the workshop.

34:28 There is a proper person there that has experience in doing that.

34:33 So yeah, I've done the workshop though, and it's a great workshop.

34:39 Yeah, nice.

34:39 And then I have a few other extras though.

34:42 Can I just go on?

34:44 Yeah, yeah.

34:45 Cover as many as you want.

34:46 As many as you got.

34:47 Cool.

34:47 So then I also found, I don't know how many newbies we have listening to us today, but for people that since we're talking a lot about data science as well, there's a bunch of data scientists and other people around that don't really know how to deal with Git. And even for me as a person, my personal experience with Git initially was so hard.

35:10 - Yeah. Why do I have to commit it twice? Why do I have to do that? I just want to commit it.

35:16 What's this push about?

35:18 And then, what do you mean, "Where am I?" [laughter]

35:22 I am here, in front of my computer.

35:24 [laughter]

35:26 So, if you want to learn Git, there is this website called learning-branching.js.org that gives you a step-by-step with a visual manual of how to actually get Git done.

35:40 So, you have, like, lessons.

35:42 So, you can click on the lessons and then it gives you like an explanation with some best practices and then gives you snippets of code as well.

35:51 So you have like a thing that moves.

35:54 Yeah, it's quite like straightforward and it's quite interesting.

36:00 I don't know what happened here.

36:02 My computer I think stopped working.

36:04 Yeah, anyhow, so it's very interesting and it's a very visual way of learning.

36:13 So if you're struggling with Git and you don't have time to read the book, actually, first thing, read the book.

36:20 I think chapter 10 of the Git book is the most important one.

36:24 It teaches the main things that you need to know.

36:26 But if you don't have time to do that, then if you want to train a little bit with a GUI, just this is the most straightforward thing that I found around.

36:36 - Yeah, that's cool.

36:37 I like how it's like a little fake shell.

36:40 - Yes.

36:41 And if everything was working, I could show you the like you have, so you do, you click here and it's like, it's very GUI based, but then as soon as you get out of this thing, you need to train to pass the phase 'cause it's kind of like a game.

36:54 Then you need to type it.

36:55 So you're like-- - Oh, I see.

36:57 So it has you do some kind of branch thing or checkout and then it watches to make sure that, say the branch was created or something, huh?

37:04 - Yes, yes, like you have a check.

37:06 It's quite interesting and it's quite good for learning.

37:11 - Yeah, here. - Oh, yeah, nice.

37:14 - If we can pick, no, we can't.

37:17 I don't know what's going on, I'm sorry.

37:20 But it's-- - No worries, it looks cool, it's a good suggestion.

37:22 People can check that out in the show notes.

37:24 - The, yeah, so here-- - Yeah, I like the visuals.

37:28 - This is the one, so you can type, let's say, git commit, and it's here, you can see what happens, and it tells you, so, okay, so you did a commit, and this is what happens, And then you can go back to the instructions if you don't get it.

37:43 It's quite interesting.

37:45 And then just to close the extras.

37:48 So I started coding with Python for a very short period of time.

37:54 And then I switched when I started IT, I switched to Java and then I did a few years of Java.

38:00 And then I went back into Python.

38:02 But then when I went back into Python, I was like, well, I know Java now.

38:07 What's going on?

38:08 is this language that is not verbose at all.

38:11 And you can do whatever you want with it.

38:12 How do you, how do you do objectives?

38:14 - There's no rules, what's happening?

38:15 - Exactly.

38:16 What's the story?

38:18 And I struggled a lot to find a straightforward tutorial and a straightforward manual that would tell me, so this is the comparison, this is what Python does that Java doesn't do, or this is how to do things in Python and this is how to do things in Java, this is the comparison and blah, blah, blah.

38:34 So then I found this real Python article that was just being published, I think, maybe a few weeks ago.

38:43 I wish was around when I started, but it wasn't.

38:45 - Yeah, it's only like five days old or seven days old or something, so.

38:49 - Yes, yes.

38:51 So he tells you like very straight, in a very straightforward way, the differences, the similarities between the two languages and talks about the Python data types and talks a little bit about as well, that the Zen of Python and how we do things in Python and how we do things in Java and then standard library and then comments as well as about apps and tutorials and documentation.

39:16 So it's very, very, very interesting if you're starting with Python coming from a Java background.

39:24 - Nice, yeah, that's cool.

39:25 I think using your existing skills to level up is a much better way.

39:31 Like I wouldn't know, I wouldn't know how to talk to a database.

39:33 I can do that in Java.

39:34 How do I do that in Python?

39:35 Oh, I see.

39:36 Okay.

39:36 Now I got it.

39:37 Right.

39:37 Yes.

39:38 Cool.

39:39 Cool.

39:39 All right.

39:40 I got a few extras as all everyone may imagine.

39:43 First, a quick follow-up for people who are getting started with Git and you want some more help.

39:47 Consider using OhMyZshell.

39:49 why?

39:50 Because if you go into a Git repository here and you type Git, you can hit tab.

39:57 It'll give you all the top level features of Git.

39:59 So like branch, clone, check out, commit, whatever.

40:04 So if you type get branch and hit tab though, it'll actually list, give you tab autocomplete and arrow autocomplete for all the existing branches, for example, and stuff.

40:14 So if you're having a hard time on the terminal and you want some help, you can just do Z shell and it'll actually give you like super deep autocomplete on the terminal, even into say your project structure, your repo structure.

40:27 So that's cool.

40:28 I couldn't have put it without that.

40:29 But for my actual things, I gave a talk at FlaskCon.

40:34 So that talk is up, which is really cool.

40:38 It was on HTMX and Flask and Jinja partials and things like that.

40:42 So people can check that out.

40:44 That's good.

40:45 Brian and I have talked about why Firefox is awesome and people should be using it more and not letting it fade into oblivion.

40:51 As everyone thinks Chromium is the core that we should all be building on.

40:56 And they just came out with a really cool idea called RL blocks that would be applicable to other browsers as well.

41:03 So one of the problems with web browsers is they accept all sorts of input from random strangers on the internet, which is generally not a good idea for security, right?

41:13 And it's not just the browser, it's the plugins.

41:15 So you've got things like the OG format, or Wolf is a font, parsing library, and all these different things that you might need to run inside the browser, but you really get them from somewhere else, right?

41:30 So for example, if there's a problem with say, the aug parser, you load an aug file, then the browser gets hacked into.

41:37 And there's things they've been doing to try to restrict that with like sandboxes and stuff.

41:42 So this thing, I'll just give a quick shout out to Rblox.

41:45 What they're doing is insane.

41:47 As somebody who does a little C++, tell me what you think about this.

41:50 So what they do is they use the LLVM, are no clang here, these clang, and they take these libraries like aug, they compile them to WebAssembly, and then there's another thing called WebAssembly to C, and they compile it back to C, and then they compile it into the system, as if they hadn't done anything to it, which sounds weird, right?

42:11 Why would you do that?

42:13 But what it does is it strips out a bunch of intermediate steps where viruses could live, and uses WebAssembly to like put those checks as part of the system, and then puts it back into C.

42:24 - Okay.

42:25 - Yeah, so anyway, this is a pretty wild thing that they came up with, so just worth a quick checking out.

42:31 And then, Liza, you mentioned this log4j thing.

42:35 - Yes.

42:36 - We're all gonna be suffering from this for so long.

42:39 This is so bad.

42:40 So it turned out that log4j, the problem with log4j is, let me describe this in Python terms and just see what you all think.

42:48 So log4j has this ability to parse the strings you put into the logs.

42:55 So I wanna log out a message, but instead of actually getting a user, like say instead of going to the code and getting say the machine name or the IP address that I'm on, I can put like a little symbol in the log file and log4j will go, oh, what you need is you want me to say, pull in the IP address locally and then write that in the log.

43:16 That's maybe okay.

43:17 They also have this ability to run code that they don't know.

43:21 I mean, you did some Java.

43:22 they have this ability to run code, not in the local class library, but on a remote machine somewhere else out on the internet, called, was it JNDI, J-D-N-I?

43:33 I don't always swipe the order.

43:35 Well, it turns out you can put the JDNI run code symbol into your log file, and then the log message will result in running code from anywhere on the internet in the process of your thing that's trying to log.

43:49 And it can come from anywhere.

43:50 It is so bad.

43:52 If it's in a header file, if it's in the URL, if it's in the email address where you expect an email address, you put in like $(JNDI, some path to the virus), literally the attempt to say, this user with this email address failed to log in results in running the virus.

44:07 It's anything that you may ever log, user agents, like you name it, run virus remotely in this process.

44:16 On anything you might want to log in, you're typically logging bad inputs, so you can log attempts to hack your system.

44:21 There were like 850,000 hack attempts within the first couple, like day or two of this.

44:28 It's going to be not good, which made me think of just a quick little fun comment here of an XKCD that someone put on Twitter after this, because it's all about this log4j that's not really funded.

44:40 So all modern digital infrastructure is like this Jenga tower.

44:44 And then at the very bottom, it's a project some random person in Nebraska has been thanklessly maintaining since 2003.

44:51 Pull that out, it all comes down.

44:53 So that's the world we live in, yeah?

44:55 - Yeah, especially because there was this thing, this movement on Twitter of people just blaming the Logfj developers about it and some other people just show up and, well, they haven't been getting paid for all the work that they're doing and everybody's using the application all throughout, so.

45:15 There's certainly a conversation to be had about how much energy and time should be these large companies be putting into supporting open source stuff, right?

45:26 And it's, I think there's so many little things like Log4J that nobody feels like they're using.

45:32 Sure, their web framework might use it, but they don't actually wanna use it.

45:35 It just happens to be what their web framework shows, but they are built on top of it as they're learning.

45:40 I'm sure there are many sleepless nights on this.

45:42 I don't know, not good.

45:44 As a Java person, what do you think about this?

45:46 [LAUGHTER]

45:47 Actually...

45:48 As a representative of all Java.

45:51 [LAUGHTER]

45:53 No pressure at all, right? No pressure whatsoever.

45:55 [LAUGHTER]

45:57 I can say as someone that has worked for open source for a while now that there is a big discussion to be had on that.

46:07 But there is also a big discussion to be had on how do we want open source to be sustained.

46:13 From now on because the way that it's going is not sustainable when we all know which every have big companies and we don't need to we don't even need to name the big companies but we have really big companies they have.

46:25 Have been using and i starting now to do open source project simply because it's good for business.

46:33 because it gets people hooked into what they're doing and then, well, you're going to have to use our open source packet now.

46:40 And that's us taking the market.

46:43 But we need open source to be at least a tiny bit more sustainable because the way it's going, it's not good for anyone.

46:52 - Yeah.

46:53 I guess one final thought on this.

46:54 This was not a bug that was found.

46:56 This was a feature that was used in a way that people didn't expect.

47:00 I can expand variables.

47:01 I can run remote code.

47:02 I can run expanded remote code, oops.

47:05 All right, but let's bring it back to Python with a joke.

47:08 What do you guys think?

47:09 - Yeah. - All right.

47:11 Lize, you got the first joke.

47:13 Hit us with this one. - Yes.

47:14 So we were talking about virtual environments and how much fun it is to set up Python and use all your virtual environments 'cause you have, well, fans and VPNs and condas and everything else and how much everyone always complains about having to set up their environment before I started Python. So I found this. And so apparently, this would be society if you didn't have to use virtual instance now by Python packages. And it would be wonderful.

47:44 Let me try to describe this for the world who is just listening. It's like the most idealistic futuristic view of the world. We've got like monorails and there's just shining silver spaceships zooming through the sky.

47:59 And it just looks like, oh, yeah, here's the perfect future.

48:03 We all dream what we were supposed to have.

48:05 Yes, that everyone, everyone types Python.

48:08 Everyone knows Python, but you don't have to use virtual AMPs to install their packages.

48:13 Imagine.

48:14 Are you telling me that virtual environments denied this from us?

48:17 No, this is the reason why we can't have these things is because people don't learn virtual environments.

48:22 That spaceship has its own environment.

48:24 - Oh, it's self-contained. - Oh, it's self-contained.

48:26 I got it, okay. - That's running a different version.

48:27 Yeah, because of its environment.

48:29 (laughing)

48:30 - That's a good way to look at it, Brian.

48:31 I like it.

48:32 - Yeah, me too. - Awesome.

48:34 - Brian always brings the other side of the things where it's like, oh, the data perspective.

48:38 It's wonderful.

48:39 Thank you, Brian.

48:41 - Yeah, thanks, Brian.

48:42 All right, the joke I was gonna cover is actually that Jenga tower I already showed.

48:45 So I'll just leave it at that.

48:48 One real quick bit of follow-up here, 'cause I think this will be helpful to people.

48:52 Jeremy Page says, "If you are on Bash instead of ZShell, the package bash-completion will also allow git commands, tags, branches, et cetera, to auto-complete." - And I never read a book.

49:03 I just, like, I think I read an article that's like the 10 git commands you need, and I've been good since, so.

49:11 - Yeah.

49:12 - Oh yeah? Okay.

49:14 Mr. Waste Kit.

49:16 I catch.

49:17 - Exactly.

49:18 And Brandon says, "You don't need virtual environments.

49:20 if you just set up a new virtual machine for every project.

49:22 That's true.

49:23 - Yeah, Docker will save the world.

49:25 - That's right.

49:26 - I agree.

49:27 - We could have the same picture with Docker, but they'd all just be cubes, just flowing around.

49:31 - Cubes, yeah.

49:31 Like, yeah.

49:34 Pack it like that, with those shipping containers everywhere.

49:38 - There's also the pragmatic, exactly.

49:40 The pragmatic Mario says, "I click buttons on VS Code." - Go GUI.

49:48 All right, Brian, you want to take us out of here?

49:51 Yeah, sure.

49:53 Thanks, everybody, for showing up for the show.

49:55 And it was really a lot of fun.

49:58 Thanks, Leas, for-- sorry, Lays, for showing up.

50:01 It's been great.

50:02 Bye, everyone.

50:02 Thank you.

50:03 Thank you for listening.

50:04 And thank you for the insight.

50:06 Again, it was a pleasure.