Transcript #299: Will McGugan drops by

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:04 This is episode 299, recorded August 31st, and I'm Brian Okken.

00:10 Hey, I'm Michael Kennedy.

00:11 And I'm Wilma Guggen.

00:13 Will's also known as usually the topic of Python Bytes.

00:17 Time to time you mention me.

00:21 So, it's awesome to have you here.

00:23 But you're part of Texturalize, right? Or you are Texturalize.

00:28 That's right. I guess I'm part of Texturalize.

00:31 We are a company, a very small company, but we're a tech startup.

00:34 We have three employees, and we'll have five in a few months.

00:39 Nice.

00:40 Yeah, tell us a bit about it.

00:41 You know, people know, that's amazing.

00:43 Tell people about it.

00:44 They know about Rich and Textural, perhaps, because we talk about all the things that are adopting Rich.

00:51 But you actually have a company around that, which is super fantastic, right?

00:55 So, the model is we're building Textural, which is going to be like a free and open source project distributed through PyPy.

01:02 And then, somewhere down the line, we're going to make this web service, which takes those applications and then serves them on the web.

01:08 And it'll be like a nice free tier, but we'll be able to add services on top of that, which we can, you know, charge a subscription for.

01:17 That's fantastic.

01:18 Like, like 2Es as a service.

01:20 2Es as a service, yes.

01:21 There you go.

01:22 A TAS.

01:23 A TAS platform.

01:25 All right.

01:25 Cool.

01:26 Well, it's great to see all the progress there.

01:28 It is.

01:29 Brian, am I up on the first one?

01:30 You are.

01:31 So, this one comes to us from John Hagen.

01:35 Thank you, John, for sending this in.

01:37 IPI, as the warning is gone, but they were under a pretty heavy phishing attack.

01:45 And they wanted everyone to know they had this big banner that was letting anyone who maintains a package is really where the problem lies.

01:53 If you maintained a package, like, say, rich, the goal of this phishing attempt was to get your credentials so that you could then, so that they could sign in and put malware into that package.

02:06 The more popular, the better, I presume.

02:08 So, this was a couple days ago, August 24th, so like a week or so.

02:13 So, today we received reports of, this is from PyPI, the Python Package Index.

02:18 Today, we received reports of a phishing campaign targeting PyPI users.

02:22 This is the first known phishing attack against PyPI.

02:25 We're publishing the details here to raise awareness of what is likely going to be an ongoing threat.

02:31 There was many comments and quote tweets and so on.

02:36 It said, "The background of the phishing message claims to be there's a mandatory validation process being implemented, and they invite users to follow a link to validate their package, like a so.

02:48 Otherwise, their package will be removed." So, importantly, they say, "Note, we will never remove a valid project from the index.

02:57 PyPI only removes projects which violate their terms of service or in some way determined to be harmful, for example, malware."

03:04 This takes you over to this site, if you were to click the link, to, if you look carefully here,

03:08 sites.google.com/view/ipivalidate/validate-by-pi-package, or some kind of redirect.

03:16 And it does bad things when you fill this out.

03:19 It just posts the form to somewhere else.

03:20 So I guess they were hosting it on google.com, sites.google.com, in an attempt to avoid the domain getting blocked, something like that, right?

03:29 But it posts over to linkedopports.com or something.

03:35 I don't even know how you pronounce that domain, but don't go there.

03:38 It's not good.

03:38 And it says, "The malicious releases follow a pattern, exotel." And I kind of laughed, even though it's not really funny.

03:46 One of the packages that got phished through this email is called spam.

03:52 I don't know what spam does.

03:54 But if you were like phished because your package, if you were phished by spam and your package is called spam, there's just, it's just too much meta for me.

04:04 Will, did you hear about this?

04:05 Did you get any notifications for your packages?

04:07 I didn't, but I could see how, if I got that early in the morning, I could fall for it.

04:12 Yeah, exactly.

04:13 You're just waking up.

04:14 Before my coffee.

04:15 Not another thing I got to do.

04:16 I feel like they're taking a little bit of advantage of the notifications coming from the Python Packaging Authority, where they've been sending out messages about security and about packages.

04:28 And there's the critical packages.

04:30 Will, I saw you tweet and Brian was like, "Wait, what is this?" about what is a critical package the day that stuff came out.

04:36 And so, you know, it kind of, I think is trying to hide under that, that noise and slip through the cracks there a bit.

04:44 I think I don't get very many notifications from PyPy.

04:48 I think the fact that I don't get very many, I might just take them at their word.

04:53 If you get a lot of notifications, you might learn to recognize what is a legit notification and what is some kind of phishing attempt.

05:01 So you're asking PyPI to send us more email?

05:04 No, not really.

05:06 Funny.

05:07 So you can actually look at what the code does that is the malicious versions.

05:13 They said they've taken down and they've taken down several hundred typosquatting ones that also do a pattern.

05:19 So you can just set up the code.

05:20 So you can just set up, you can just set up, you can just set up, you know, it's just hijacking the most used function or feature.

05:26 So you can just set up, you can just set up, you can just set up, you know, it's just a good thing.

05:34 So you can just set up off of this malicious place.

05:41 So you can just set up the code.

05:41 So you can just set up the code.

05:43 So you can just set up the code.

05:45 Download it with requests and then write it to a file and then just execute some executable.

05:51 And that's pretty much what it was doing to all or attempting to do all these packages.

05:56 It's kind of lazy this malware is not cross-platform when he works on Windows.

06:00 I agree.

06:03 Come on.

06:04 Well, I mean, also, they're looking for investors so that they can, like, expand to other platforms.

06:10 Maybe they'll get a macOS and a Linux one coming at some point.

06:13 Just hope our VZ is listening.

06:15 Yeah, exactly.

06:17 Like, here's an opportunity to, I don't know what it does when it runs, but it can't be good.

06:21 Whatever it does.

06:22 Can't be good.

06:24 Now, I remember I got an email from someone and I'm sorry I didn't pull the details together as part of the write-up here saying, I think it was on Twitter, a DM that said, you're probably going to talk about this.

06:39 The 2FA wouldn't necessarily help you because if they asked for the 2FA, you're going to enter it in maybe there and it might, you know, pass it through as part of the process as well, right?

06:51 It could maybe get a software-based 2FA and use it.

06:56 But you know what does help with this really, really well?

06:59 Password managers.

07:00 One password, LastPass, and so on.

07:03 If you go there and it says enter your password and you hit the hotkey to like fill out the site or you click in there like with one password, if you just access it with the latest version, it'll automatically drop down or suggest the dropdown for the site.

07:17 If you do that, it will not come up with anything because you don't have an account at sites.google.com, presumably for this login, right?

07:25 Yeah.

07:25 And so some combination thereof, I think, you know, at least I always, if I go to a site, the less sure I am, I like double check.

07:35 Does the password manager think it should fill this account into this website?

07:38 If it says no, then I triple check it.

07:41 It's not just like, oh, the password manager is not working.

07:43 Let me just copy it over, you know?

07:44 Anyway, there's some more details in here.

07:47 You can read about what they're doing and what you should do.

07:51 Some ideas on how to verify the signed certificate.

07:55 I would prefer to just use a one, just a password manager right away instead of trying to follow the chain of the cert, but you can do that as well.

08:03 Anyway, it says it's got a lot more details there if you want to check that out.

08:07 What do you all think?

08:07 Yeah, it's interesting.

08:09 And it's an ongoing threat.

08:11 I think if you're an internet user, you're getting bombarded with this.

08:16 With PyPy, if they get a hold of your, it seemed like it was more like attacking individuals.

08:22 Were they trying to get your credentials to like install malware in your Python packages?

08:28 I'm pretty sure what they were doing is they were publishing a new version of your package.

08:33 Ah, okay.

08:34 And the new version had malware in it.

08:38 Yeah.

08:38 So, yeah.

08:39 That would be a huge concern.

08:41 Yeah, yeah.

08:42 It says, yeah, we've additionally determined that some maintainers of legitimate projects have been compromised and malware published as their latest release for those projects.

08:51 Their accounts have temporarily been frozen and the bad versions removed.

08:55 But that's what they were trying to do.

08:56 Okay.

08:57 Well, good on them for catching it.

08:59 Yeah, for sure.

09:00 I love how Seth out the audience says, another win, W-I-N for Linux on the desktop.

09:05 Yeah.

09:05 How about that?

09:06 Yeah.

09:06 Or Mac.

09:07 I'm like, well, that wouldn't hit me at all.

09:09 Yeah.

09:10 However, anybody, any package that you publish, it would affect your Windows users if you were the maintainer.

09:19 Which is half the users, as we know, about, you know, half, at least half the operating systems are Windows.

09:25 Yeah.

09:26 Yeah, yeah.

09:27 Not so good.

09:28 So Seth also points out that the timing of this phishing combined with Dustin mentioned there hadn't been any phishing attacks or something like that.

09:38 It's in Dustin Ingram.

09:40 Don't give people ideas.

09:42 If, by the way, you want to hear more about this, I did interview Dustin with us like three weeks ago or so on Talk Python, talking about Python packaging and security and supply chain stuff.

09:54 But all that predated the phishing side, but not the malware side.

09:59 So people can check that out if they want.

10:00 Okay.

10:01 All right.

10:01 Anyway, make sure that you don't put your password for PyPI in the wrong place.

10:06 Yeah.

10:08 All right.

10:08 How about we talk about something nice?

10:11 Yes.

10:12 Friendly.

10:13 We can have nice things.

10:14 We can have nice things.

10:16 Like Python being on top of the IEEE spectrum languages.

10:20 So it has been for a while.

10:23 So we're just reviewing this to say, yay us, partly.

10:28 But there's some interesting information here.

10:31 So if we go through the review stuff, there's a cool chart.

10:34 So the top programming languages of 2022.

10:37 Python's at the top.

10:39 It's both the spectrum survey.

10:42 Actually, I don't know where the spectrum number comes from.

10:45 I assume it's a survey thing.

10:46 And then trending.

10:48 It's on top also.

10:49 A little different.

10:50 And cool animation graphics on there.

10:52 But one of the interesting bits is in jobs.

10:55 If you look at job listings.

10:57 I think they were looking at job listings and requirements.

11:00 SQL is at the top.

11:02 And this is a comment because it's not just SQL.

11:08 They're not going to just say, we need somebody that knows SQL.

11:11 But it's like Python and SQL.

11:13 Or Java and SQL.

11:14 Or JavaScript and SQL.

11:16 The end SQL part is coming in a lot now.

11:19 And actually more than it used to.

11:22 Which is interesting.

11:23 There's also a related article on the same on IEEE as well called The Rise of SQL.

11:30 And it's really talking about that.

11:32 It's not just because.

11:35 I mean, I do remember SQL has always been a part of programming.

11:40 Or it has been for my career.

11:42 But it's often been a larger thing.

11:45 It's not something you do in a small application.

11:47 It's something in the server or large applications.

11:50 But it's growing in using it for even little small things.

11:55 And a lot of applications, you don't have a specialist doing the database stuff.

12:00 You've got the developers doing the database everything.

12:04 So anyway, I thought that was interesting.

12:07 That just the highlighting that SQL is and always will be important.

12:12 And it's even growing in popularity.

12:14 It's interesting that ORMS haven't made a dent in that.

12:18 You know, I used to use Django ORMS back in the day.

12:20 I didn't have to touch SQL.

12:21 But clearly, people are not using ORMS that much.

12:27 If SQL is like top of the charts there.

12:30 My first thought when I looked at this was there's probably a lot of data warehouse, data lake, semi-structured data that people are exploring with SQLs and queries before they lock it down and productionize it with an ORM or something along those lines.

12:47 Right.

12:47 Like the data science side.

12:49 If a lot of that data is dropped into a database by an API or some web scraping or something, and then you have to ask it questions.

12:56 And like knowing the SQL is the asking arbitrary questions of the data before you really know what questions ask is my first thought.

13:03 But yeah, ORM is all the way for the win.

13:06 For me, I don't want to do straight SQL.

13:10 There's also a growing thing of just doing a small, like a SQLite.

13:14 Just the knowledge that SQLite is everywhere.

13:16 And I don't know if SQLite has any effect on this or if Simon Willison does.

13:22 Because he's with data set taking like CSVs and stuff and turning them into little SQLite websites.

13:31 But things like that have just even services and smaller applications collecting data.

13:39 Isn't there a JavaScript API to SQLite?

13:43 I think there's all languages have, I think, APIs into SQLite.

13:49 I think something built into the browser is local storage.

13:52 Does that not use SQLite or am I mistaken?

13:54 Don't think that that does.

13:57 But it's very similar if it's not the same.

14:00 It's like, yeah, what do they call it?

14:03 Local SQL or local DB, something like that.

14:05 I wonder if that's contributing to.

14:07 Yeah.

14:08 Dave out in the audience says, I wonder how the jobs one was measured though.

14:12 SQL can be one of those not really considered must have items that often goes into the list of requirements.

14:18 You really could just use an ORM.

14:21 Yeah.

14:22 And that might be the case.

14:24 But even with ORMs, sometimes you got to get in there and find out what's wrong.

14:30 Like do optimizations and things.

14:32 Yeah.

14:33 Kim out in the audience says, is it becoming less common for developers to know at least enough SQL to be dangerous?

14:37 I think that that's the minimum bar for what you should know to use an ORM.

14:42 You should know kind of what's happening.

14:44 You should know what a join is and that you might want to do it so it doesn't do 50 queries.

14:49 You do only one and things like that.

14:51 Yeah.

14:51 But yeah, very, very cool, Brian.

14:53 Well, where are we at next?

14:55 Well, I think Will is up.

14:59 What have you got for our first one, Will?

15:00 Oh, I'm on the wrong page.

15:02 So I came across this article.

15:03 It's by Charlie Marsh about using my pie in production at Spring.

15:09 And I looked at Spring and it's some kind of interface for researching medicine.

15:18 But he was saying that they have a big project, 300,000 lines of Python.

15:24 And they started typing it.

15:27 And that was a few years ago.

15:29 And they've since typed the entire code base.

15:32 And they were very impressed.

15:33 It's reduced bugs and made things more maintainable.

15:38 And even have all the strictness settings on to Max.

15:41 I'm a big fan of typing, but I don't have everything on Max.

15:48 I relax it just a tiny bit.

15:50 But they've got all the settings on there.

15:53 So he covers the basics of typing.

15:57 There's some interesting stuff about the history of typing.

16:00 Right.

16:01 It's interesting how typing started.

16:05 It didn't spring out of nowhere with the same goals.

16:08 It did actually evolve through various other projects.

16:12 Originally, when mypy started, I think they were building something that was

16:15 essentially more like mypyC.

16:18 It was to run Python with typing.

16:21 To use the typing to actually sort of transpile to another language almost, right?

16:25 Yeah.

16:26 Yeah.

16:26 And then it turned into a type checker, which didn't actually run your code.

16:30 It just analyzed your code.

16:31 So it's quite an interesting article.

16:34 They covered how it started and go through a bunch of their experiences.

16:39 With typing, in mypy in general, they cover improved readability.

16:46 That's a big one for me.

16:48 I love typing.

16:49 I find it makes code more readable.

16:52 Some people would disagree because you add lots of these annotations.

16:56 And some people find that clutters your code.

16:59 I find it super helpful to understand other people's code and to understand my code.

17:08 Because I don't have to remember the types of everything when I can just see them written

17:12 down.

17:12 Yeah.

17:12 I'm totally with you on that.

17:14 I think there's a fear that, oh, look, Python is becoming like C++ or something.

17:20 But it's nice and clean and it's simple until you're focused on some area, some function

17:26 or a class or something.

17:27 You're like, well, what happens here?

17:29 You can either go read the documentation, try to put it together, or you can go find all

17:34 the places it's used and try to put it together.

17:36 Or if it has types, you don't have to go do that exploration, right?

17:39 You're just like, okay, this is an integer and this is a list of users.

17:42 I know what's happening now.

17:44 I don't need to hold more information in my brain.

17:46 And I think that makes it great.

17:48 The shape of data is what really helps me.

17:50 Like you mentioned, like a number or a list or something.

17:54 But sometimes things can either conceptually, an argument to a function could be one thing

18:00 or it could be a set of things.

18:01 Is it expecting if it's just one thing, can I do that?

18:06 Or is it expect?

18:06 And types can give you that.

18:08 And one of the things you mentioned, which perfectly sums up my philosophy for typing is

18:15 I don't want to do it, but I want everybody else to put types in there.

18:20 Because it's readability counts.

18:24 Also, Mr. Wilson in the audience points out typing greatly assists with co-completion

18:31 for the editors.

18:32 Yeah, it's got a bunch of tangential benefits.

18:36 It's not just the one benefit.

18:37 I mean, it catches bugs.

18:39 But even if it didn't, even if it didn't, I think typing would be an excellent addition

18:44 to Python.

18:44 I agree.

18:45 If I can hit dot and it gives me more help, I'm already happy.

18:48 Yeah, bingo.

18:50 Yeah, I love typing.

18:52 This is a very nice article.

18:54 I'll let people read it.

18:55 Yeah.

18:56 They also talk about the pain points of how that was painful for them.

19:00 I think trying to get mypy to completely analyze everything is a different level of, I want my

19:07 public interface to say what it returns.

19:09 You know, there's like, you got to decide where do you live on this spectrum?

19:13 And what are some of the goals?

19:15 Like catching bugs, it's more important to have everything covered.

19:17 There's documentation, you know, a little bit less, I think.

19:21 Yeah.

19:21 I found it's changed my programming style.

19:24 The code I write is less dynamic.

19:28 I'm more likely to fix types quite early on.

19:32 And I don't do any, well, I don't do too much get atter and set atter.

19:37 Yeah.

19:38 And I don't use all the dynamic capabilities of Python.

19:41 I prefer to write static code that looks a bit more like C.

19:44 So I can understand why people have that.

19:46 They feel like it's taken away a bit of freedom from them.

19:50 But I do think it's given you the freedom to write solid code that doesn't have, you know,

19:56 no attributes on none type errors.

19:58 Yeah, I agree.

20:00 I think you compare it to things like TypeScript.

20:03 TypeScript has a similar idea, but TypeScript is very particular.

20:07 And if you don't get it just right, it'll give you compiler errors and it won't do the

20:12 steps it needs to do to make the JavaScript.

20:14 Whereas Python, maybe your editor will give you a warning or some tool like mypy will give

20:18 you a warning.

20:19 But it doesn't really get in the way of it still functioning, you know, which unless

20:24 you're doing something where it depends upon it, like Pydantic or FastAPI where it's actually

20:29 using that.

20:29 But most of the time it's there when you want it and you can kind of ignore it if you don't.

20:34 Yeah.

20:35 So Rich has a lot of typing.

20:38 Yeah, Rich is fully typed.

20:40 Yeah.

20:40 So it's textual.

20:43 It's not passing mypy currently.

20:47 There are some like little dynamic corners and little typing errors which we're gradually

20:50 improving.

20:51 But all new code is typed and changes are typed.

20:57 So yeah, we're really big on typing at textualize.

20:59 Right.

21:00 And how about with your pytest extensions, plugins?

21:07 You know, it doesn't really come up much, but I don't really think about it a lot, actually.

21:12 So I would like, but I have other applications that I'm working on that I definitely involve

21:18 typing.

21:18 And I started out with just the, like you were saying, trying to help with documentation.

21:24 So making sure the API is typed.

21:26 I think that's essential.

21:27 I don't think I think it's just a good idea.

21:30 Especially now with the improvements of some of the typing.

21:33 So you can do, you don't have to say union anymore.

21:36 I like the bar for or.

21:38 Yeah.

21:38 Like it's a, it's a non, you know, it's this, it's an int or a none or something like that.

21:43 That's way cleaner than it used to be.

21:45 And you don't have to import typing as much as you used to.

21:47 I don't want to import typing just so that I can type some type in something.

21:52 It seems wrong.

21:53 But, and I'm on the phase of trying to integrate it more into the rest of my code, just because

22:00 I'm, you know, even in a solo project, sometimes you're also a user because you come back to

22:05 something six months from now and try to figure out what you're doing.

22:08 And it's nice to be able to not have to look at the code.

22:11 So I like it.

22:12 Yeah, absolutely.

22:13 Absolutely.

22:14 Well, we could find Will.

22:15 Yeah.

22:16 So I want to say something nice about AI in real life and actually the podcast IRL from Mozilla.

22:24 So this episode of Python Bytes is brought to you by the IRL podcast, an original podcast

22:31 from Mozilla.

22:32 And I'm really enjoying it.

22:33 I'm listening to a whole bunch of it.

22:35 If you care about ideas behind technology, not just the tech itself, you'll enjoy IRL.

22:40 Tech has an enormous influence on our society.

22:43 Many effects are beneficial.

22:44 The influences, like for instance, the information and assistance we get through cell phones is

22:49 amazing.

22:50 I love being able to look up the closest coffee shop wherever I'm dropped on the earth or knowing

22:55 where my kids are.

22:56 But some are not so great because like, I don't want somebody else to know how often I hit the

23:01 coffee shop.

23:02 And I definitely don't want somebody else tracking my kids.

23:05 So Mozilla has always been on the lookout for possible downsides to technology and works to

23:11 mitigate negative influences of tech on the negative influences on all of us.

23:16 If ideas like that and concerns about technology resonate with you, you should definitely check out the IRL

23:22 podcast.

23:22 This season is hosted by Bridget Todd and is looking at AI in real life.

23:27 Who can AI help?

23:28 And also who can it harm?

23:30 The show features fascinating conversations with people who are working to build a more trustworthy

23:35 AI and also using AI to help us.

23:39 So I really enjoyed a few episodes so far.

23:42 There's an episode on how our world is mapped with AI.

23:45 So data and maps is being used to make decisions that affect real people, even like by districts and by

23:53 governments.

23:53 But how can people reclaim the power over their own maps and stories using AI?

23:59 This is fascinating episode.

24:01 Another episode is about gig workers who depend on apps for their livelihood.

24:05 It looks at how they're pushing back against algorithms to control how much they get paid and seeking new ways to

24:12 gain power over the data to create better working conditions.

24:16 And how about elections?

24:18 So episode four of this season addresses the role of that AI plays when it comes to both

24:24 spreading dense information around elections, but also how to combat disinformation.

24:31 This is a huge concern for democracies around the world.

24:34 And for me, especially in the US, but I know it's affects everybody.

24:38 If this sounds interesting to you, you should try it because it is interesting.

24:42 Try an episode for yourself.

24:44 Just search for IRL in your podcast player or visit pythonbytes.fm/IRL.

24:50 I think the best way is to select I go to pythonbytes.fm/IRL.

24:55 So they know you came from us.

24:57 And the link is in your show notes.

25:00 Thank you, IRL and Mozilla for supporting our show.

25:03 Yeah.

25:03 Cool podcast.

25:04 Thank you.

25:05 Thank you, Mozilla.

25:05 All right.

25:06 On to the next one.

25:08 Well, we already touched a little bit on the whole ORM thing.

25:11 And I hear some people use Django.

25:13 It's a web framework.

25:14 Yeah.

25:15 A few people use it.

25:16 I'm proud of it.

25:17 Yeah.

25:19 So really popular.

25:21 And they're picking up the speed, of course, for their releases, right?

25:24 For a long time, it was a one.

25:26 Then we had two, three, four.

25:27 Going really quickly over a couple of years there.

25:29 Well, one of the big moves with many of the web frameworks ever since Python 3.6 or so has

25:36 been how are you going to participate and facilitate using async and await, right?

25:42 If you're doing a long database query and you block with an ORM request, for example,

25:47 how do you parallelize that or scale that without much effort?

25:53 Well, the asyncio is perfect for it.

25:55 But if your APIs don't support it, you can't use it.

25:59 And Django has been making its way towards having async capabilities.

26:03 But what is the one thing that websites wait on the most?

26:08 Databases.

26:08 What is the one thing Django did not have async support for?

26:12 Databases.

26:13 So it's a little bit late here on the announcement.

26:17 So in the beginning of August, Django 4.1 came out.

26:21 And this means three, eight and above.

26:24 But the big deal is the second one actually is an asynchronous ORM interface for doing queries.

26:31 So you can do anything that's a query set.

26:34 So you can say, like if you have a class, a model class called authors, you would say author.objects.filter.

26:40 And then you do a thing and so on.

26:41 So now you can say async for and do your query.

26:45 And now it's all happening async.

26:47 And if you want to do like a join author.books, you can await getting access to that thing.

26:54 Books normally has a dot first in this example.

26:59 But they've now added also an A first.

27:02 So if you want the async version, you put the A as a prefix.

27:05 Not sure how I feel about that.

27:07 Not sure I would have gone this path.

27:10 But, you know, it doesn't really matter.

27:12 It's awesome that there's some kind of async support in the Django ORM.

27:16 So that's really, really cool.

27:17 So I think I just wanted to highlight that this has been a major blocker to like real async programming in Django.

27:23 It's like, well, you can make the web view method async, but then you can't do async stuff that you really want to do.

27:30 So, you know, where are you, right?

27:31 This is like this unlocks the final keys, right?

27:35 You could call APIs previously with, say, HTTPX asynchronously, but then block on the database.

27:40 Now just use the A version and off you go.

27:43 I guess they couldn't make the one without A.

27:46 They couldn't make first awaitable because that would break old code, I imagine.

27:51 Yes, but here's my thought, right?

27:53 So what I'm getting back.

27:55 So when I say async for author in query, you are now switching into an async mode.

28:01 So I think the thing that returns would be really great if it like now everything must be async on it.

28:07 If you just said for author in query, now it returns a synchron and everything on it must be synchronous.

28:13 This is how I would have maybe done it instead of trying to like prefix everything with A and double down on it.

28:19 But maybe it was just a bridge too far.

28:21 I don't know.

28:21 But this is what I had in mind when I said, I'm not sure what it is.

28:24 Like you can go in async mode or synchronous mode and then you're kind of there is what I had in mind.

28:29 Yeah, that makes sense.

28:30 Async for would return a special version of the object which had different first methods, the same API, but awaitable.

28:40 Right, but awaitable.

28:41 Exactly.

28:42 Exactly.

28:42 You know, that doesn't mean they can't do that in the future potentially, but yeah.

28:46 Yeah.

28:47 Okay, a few other updates just for while I'm already here.

28:50 The thing I really wanted to call it is async ORM in Django.

28:54 Good to go.

28:55 Also, you can have class-based views where you have a class and then methods like get, post, put, and so on.

29:02 Or you can have just method-based ones.

29:04 I prefer the method-based stuff with a decorator, but if you have the class-based ones, they now can also be async.

29:12 Right, so that's cool.

29:13 And there's also some validation of constraints is one of the other big changes.

29:17 So check unique and exclusion constraints defined in meta constraints.

29:21 Our options are now checked during model validation.

29:25 Apparently they weren't before.

29:26 So that seems pretty valuable too, but the ORM is the big news, I think.

29:29 Yeah.

29:31 Yep.

29:31 That must have been a big project.

29:32 All right, so yeah, it's great to see Django coming along.

29:35 It's been around for so long as a stalwart of the Python web world, and now it's much closer to the most modern features, which is great.

29:44 Yeah, very cool.

29:45 Yep.

29:46 All right, Brian, what you got next for us?

29:48 I have walrus operators.

29:53 Walruses on the brain?

29:58 I do.

29:58 I like walrus operators, the walrus operator.

30:01 But I don't think I've been using it enough.

30:03 And especially because this article is telling me all sorts of places that I should use it more.

30:07 So I've got an article from Martin Hines titled, you should be using Python's walrus operator operator.

30:16 And here's why.

30:17 And there's just some stuff that I never even really thought about before.

30:22 Like, it just starts right off the bat.

30:25 I'm talking about the basics.

30:28 In the basics section, I never would have thought about this.

30:31 So there's a list that happens to call a function to create the data in a list.

30:37 And it calls it three times.

30:38 And it calls it three times.

30:38 Now, really, I probably wouldn't have done this in code.

30:40 I probably would have called the function once and then named the variable and stuck it in there.

30:44 But you can do that.

30:46 It's still easier.

30:47 You can do the call the function in the first element and save the value and then use the value in future operations just to create a list.

30:56 So right off the bat, that's pretty cool.

30:58 I wouldn't have thought to do that.

30:59 It's nice.

31:00 I didn't actually quite follow this.

31:03 Oh, we get to save.

31:04 Here's a comprehension where a function is called twice.

31:11 So you put in the value of a function if something around the function, like if it's true or if it evaluates Boolean true.

31:22 You can do that with the walrus operator and only call it once.

31:26 So that's kind of cool.

31:28 One of the things I really liked around was I didn't think about before, but I'm definitely going to use it now, is the regular expression match function.

31:38 You often had to call match.

31:40 And then if something was found, then you do something with the match object.

31:45 You get the groups or some other thing on the match object.

31:49 It is cleaner to just go ahead and do the call.

31:53 Like, go ahead and do that query of whether or not the match returns something right with a walrus operator.

32:00 Way cleaner code.

32:01 So I like this.

32:03 And actually, it's just a fairly big article talking about a whole bunch of places.

32:08 Now, here's the place where I like wild true loops.

32:11 That always drives me nuts or having to flag something.

32:15 This is definitely a place where I started using walrus operator right away.

32:20 Instead of saying like wild true or wild flag or something, do something and then break out if necessary or set the stop bit or something.

32:32 You can do that right within the wild loop.

32:36 Actually, I don't know if it's cleaner.

32:39 It's less code.

32:40 I don't know if this is easier to read, though.

32:42 Any thoughts from you guys?

32:45 Yeah.

32:45 I think once you know it, it's not too bad.

32:50 It is quite different from Python prior to the walrus operator.

32:55 Even for me, I barely use the walrus operator because I'm working on libraries and the minimum version is 3.7.

33:02 So I haven't trained myself to read the walrus operator.

33:07 But to me, that doesn't look too bad.

33:08 That looks really clear.

33:10 Okay.

33:10 Yeah, this is nice.

33:12 Especially if you do one of those things where like the example here is getting input from the user where you might get input from the user and then say, wow, it's not exit or whatever.

33:22 Then in your loop, you get the same input with the same basic question again.

33:27 But you've got to ask it before to see if they ever enter.

33:29 You know, there's like this weird sort of do it two times and you could skip that with the walrus operator, which is very cool.

33:35 Oh, yeah.

33:36 Yeah.

33:37 I used to do that.

33:37 Like on the top example of just putting the command equals input and doing that above the loop.

33:43 Yeah.

33:43 And then do the same test.

33:44 Exactly.

33:44 Exactly.

33:45 Because I really don't like while true loops.

33:48 Only if you mean really do it forever, right?

33:52 Yeah.

33:52 Or like until.

33:53 Yeah.

33:54 Yeah, exactly.

33:54 Until it really is some case where you need to break out of it.

33:57 Yeah.

33:57 Anyway, so the rest of the article is great too.

34:01 Accumulating in place.

34:02 There's a whole bunch of cool places.

34:03 Oh, this is one I really liked.

34:05 I wanted to highlight.

34:06 Naming values inside of an F string.

34:10 So there's an example of an F string where you like take the date time and you're using, for instance, you might say use the date time value in two different formats.

34:21 It formatted in two different ways.

34:23 Once with what year, month, day and once with A, which I don't know what A is, but oh, which is the day spelled out for like Friday.

34:33 Now, it's assigning the date time today value to a today variable and then using it, using the value in a format string and then using the today variable later in the same format string.

34:49 This is a pretty cool trick.

34:51 And there's, I mean, there's multiple times where I'm using the same value in a couple places because I'm formatting a different with an F string.

34:59 So this is pretty cool.

35:00 Yeah, that is pretty cool.

35:02 The one that I really like is the list comprehension because that always drives me crazy.

35:07 If you're going to do an if section, right?

35:11 You know, if you're going to say X for X in collection, if something.

35:15 Yeah.

35:16 And that the thing you want in the list is some kind of like a database call or some other thing that has to be computed, then you need to test that computer value.

35:25 Before the walrus operator, you had to call that function twice, no matter what.

35:29 Like if that was a go get me the user from the data.

35:32 Like I want to go through all the emails and then get a list of users that correspond to them.

35:36 But maybe some of the emails don't actually exist in the database.

35:39 Every if statement has to be get me the user if it exists.

35:43 And then the list result, the value select out is also get me the user, right?

35:47 And this way, when it's really expensive like that, it's super nice.

35:50 It also is really useful in this situation when you're doing data science stuff that expects one line of a thing.

35:57 You know, you're like, I want to do, I want to pass this expression to like a pandas data frame or some other thing.

36:03 And you can kind of get a little bit more done this way.

36:06 It's really nice.

36:06 Yeah.

36:07 Yeah.

36:08 This is pretty cool.

36:08 Actually, I hadn't thought about doing it within the if clause within a comprehension or something.

36:16 Yeah.

36:17 This is the one where there was no other way.

36:20 Like there is no way in 3.7 to do this without calling the function twice in a comprehension.

36:26 You could do a totally different structure like a loop or something, right?

36:29 But in a comprehension where you have to have one line like a data science scenario, you had to call it twice until the walrus.

36:36 So I think that's fantastic.

36:37 Yeah.

36:38 I suppose you could create a list of expression of the return value of bunk and then use a zip or something.

36:48 But it's super awkward.

36:49 Right.

36:50 Exactly.

36:50 Yeah.

36:51 All right.

36:51 So very cool.

36:52 It's nice to see a bunch of different use cases because then you can see, oh, I will never do that.

36:57 That's horrible.

36:57 But this is really great.

36:59 And I didn't do that.

36:59 Right.

36:59 Like you could even see in the audience, people are reacting like this is amazing.

37:02 But this one is, I don't know about this one.

37:04 This is.

37:05 Yeah.

37:06 Yeah.

37:06 Yeah.

37:07 So cool.

37:08 Well, what do we got next from you, Will?

37:11 Yeah.

37:12 Oh, we're writing the right page.

37:14 So I'm sure we all love regular expressions.

37:18 We have a love-hate relationship with them.

37:20 Yeah.

37:21 Those, I'm not, I don't like regular expressions, but I use them a lot because they're powerful.

37:28 And there is no really, there is no alternative a lot of the time.

37:32 But they are very difficult to read.

37:34 You tend to get long strings of gibberish, which even if you're very well versed in regular expressions,

37:41 you might find it quite hard to parse and figure out what's actually doing.

37:45 You know, when you've come back to your code, you know, like in the afternoon.

37:50 But this is a library written by Al Swigert.

37:54 And I think that's how you pronounce his name.

37:56 Rhymes with Wider.

37:58 Al Swigert is an author and he's written this Python library, which gives you kind of like a nicer way of expressing regular expressions.

38:07 It compiles regular expressions from a bunch of function calls.

38:13 And these function calls are much more descriptive and they read quite well.

38:19 So you can essentially read a regular expression in the future and find it quite legible.

38:27 I like the either option.

38:30 It's either this or it's that, for example.

38:32 It's very readable.

38:33 Like here we've got one.

38:35 Exactly five digits plus optional white space plus one or more non-white space.

38:39 That's very readable.

38:40 If you read that a second time, you'd know exactly what that did.

38:43 But if you saw this, this is a very short regular expression.

38:46 Yeah, exactly.

38:50 Even if you're good at regular expressions, I've been using them for 15 years.

38:53 I'd have to like analyze that.

38:55 And it might take me several minutes to figure out what that does.

38:58 So, you know, if it's quite powerful just for such a short regular expression, but you can make much larger ones.

39:05 You know, here's something that's more complicated, but it's still quite readable.

39:09 Either non-captioning group, non-captioning group, either this or that, one or more of this plus non-captioning group.

39:16 It's readable and you can, you know, come back to it and other developers see it.

39:22 They can understand what's going on.

39:24 And in the end, it compiles it to a regular expression.

39:28 So it's just as fast and powerful.

39:31 But now it's just easier to work with.

39:35 So, yeah, it's really nice because the output of this little library is just the text pattern of the regular expression, which then you can do.

39:44 It's not like you've got to adopt this entire library for everything.

39:47 Yeah, you can, you know, just anywhere that you need to write a complex regular expression, you could use this.

39:53 I guess if you wanted to develop the regex, you could use this.

39:58 And then once you've done, you could compile it and then put the actual regex back in your code.

40:03 Or you could just leave it like that.

40:05 It's probably not, it's not slow.

40:07 You know, it just does sound strange.

40:09 I would probably leave it like this.

40:10 But I'm thinking if you're using another library where it expects a regular expression string, right, it's still totally compatible with that because you just say, give me the string and off it goes.

40:21 Yeah.

40:21 Yeah.

40:22 So it's not going to break anything.

40:23 It's not like you're switching or you don't have to port anything per se.

40:27 It's a nice drop in thing when you need it.

40:31 So, yeah, it's pretty cool.

40:32 Yeah.

40:32 Very cool.

40:33 Excellent find.

40:35 So that thing's called H-U-M-R-E.

40:39 Is that human readable regular expressions, I think?

40:42 I just like, I'm going to call it humor because then the regular expressions are humorous.

40:50 Yeah.

40:52 Yeah.

40:52 Very nice.

40:54 Okay.

40:55 I love it.

40:56 All right.

40:57 Brian, is that all of our things?

40:58 All of our main topics?

40:59 I think that is.

41:01 Do we have any extras?

41:03 I didn't, but now I do.

41:04 Okay.

41:06 Let's have it.

41:06 Out in the audience, Dean pointed out that the very first PyData Tel Aviv is happening in December, December 13th, 2022.

41:17 So if you're in Tel Aviv and you care about Python data stuff, you know, check that out.

41:21 The call for proposals, I think it's open for two weeks or something like that.

41:27 So, yeah, if you want to submit a talk or attend, then there you go.

41:32 Nice.

41:33 Cool.

41:33 Yeah.

41:33 Cool.

41:34 All right.

41:34 But that's my only extra.

41:35 Okay.

41:36 Well, you know what?

41:38 I'm not going to let Will off the hook because the reason why I wanted you on here is so that you could promote Rich CLI.

41:46 Okay.

41:46 Yeah.

41:51 So why do you want more people to use Rich CLI?

41:53 Or why do you think more people should use it?

41:55 I just think it's a cool project.

41:58 I use it.

41:59 You've got all the power of Rich, but it's on the command prompt.

42:04 So you can syntax highlight files.

42:07 You can also just generate colorful, Rich style content.

42:11 You can put those in your Bash scripts.

42:13 It's just a very useful thing.

42:16 Yeah.

42:17 You can get it from Homebrew.

42:20 If you do Homebrew install Rich, then you'll have Rich at the command prompt.

42:27 And you can use PipX.

42:29 And yeah, it's got a lot of cool stuff.

42:32 So do you have some workflow that you're using it for on a regular basis?

42:36 Or do you use all of these workflows?

42:37 It's more just a general tool.

42:40 Okay.

42:41 You know, when I'm navigating the command prompts, I want to look at a file.

42:45 If it's a large file, I can use Pager.

42:47 I can page it up and down.

42:49 Yeah.

42:50 It's just people check it out.

42:52 It can display nice tables.

42:55 You can take a CSV and turn it into a nicely formatted Rich table.

42:59 And you can generate simple things like rules.

43:01 And oh, it can display markdown as well.

43:05 So it's kind of like a general toolbox of like Rich related stuff in the command prompt.

43:11 I think I'm going to use it for...

43:13 I didn't know it did CSV so easily.

43:16 So I think I'll use it for that.

43:18 So maybe anytime you might type more or cat or something like that to see the contents of a file,

43:25 you're proposing now I could type Rich and get, you know, syntax highlighting and better.

43:29 Yeah.

43:29 Yeah.

43:30 So you do Rich, then name the file, then hyphen, hyphen, Pager.

43:34 And it'll give you a nice textual style Pager.

43:38 Okay.

43:38 So this came together in, I think, two weekends.

43:42 At some point, I'll go back and polish it a bit more because there's a few issues, people asking for new features.

43:50 Okay.

43:51 Yeah, cool.

43:51 Two weekends.

43:52 Yeah, it came in the audience and says, Rich CLI has replaced cat, JQ, and markdown tooling for me with one tool.

43:57 Cool.

43:58 Yeah, very nice, Kim.

44:00 Well, how about a joke?

44:02 Oh, yeah.

44:03 A joke.

44:03 And then I've got one more thing.

44:04 Okay.

44:05 Well, I do have a joke, as you can imagine.

44:07 So here's an example of where somebody is using open source to help keep their account secure.

44:16 And this is some kind of list of like common passwords or really reused passwords that people want to, you know, somebody has posted these, says, here's a list of passwords that people seem to use a lot and get reused a lot.

44:29 So please don't use this as a password or check and don't let people use these passwords for their accounts, right?

44:35 So someone comes along to this repo and they remove the word dolphins as a PR.

44:43 And the message is, remove my password from list so hackers won't be able to hack me.

44:49 The list is 10 million password top 1,000 lists.

44:54 I mean, this is proactive business right here.

44:58 That might actually work if the hackers are very lazy and don't look at the Git history.

45:04 It might.

45:04 Actually, there may be a very small percentage of effectiveness to this.

45:08 You're also saying my email address is this and my password is that.

45:12 So please don't put them together.

45:14 And by the way, this is my credit card number.

45:17 So don't paste that anywhere.

45:19 Yeah.

45:20 Anyway, that's my joke.

45:23 Okay.

45:24 Well, the last thing I just wanted to say is I got a new hat recently.

45:26 So I wanted to show off my new hat.

45:28 Oh, yeah.

45:28 Let's see it.

45:29 It's a top hat.

45:31 I love it.

45:31 It's a top hat with like a five inch butterfly.

45:34 Oh, it's got lots of butterflies.

45:36 And they're all leather and it's custom made from a guy in Oakland.

45:41 So it's my new hat.

45:42 It's fantastic.

45:44 And it perfectly matches both your shirt and your background.

45:46 Did that on purpose.

45:48 Yeah.

45:49 So.

45:49 Yeah.

45:49 Yeah.

45:50 Or on purpose or on dolphin.

45:52 No.

45:53 Indeed.

45:55 So.

45:56 Fantastic.

45:57 All right.

45:58 Love the hat.

45:58 Thanks, Will, for joining us today.

46:00 It's been a pleasure.

46:01 Yeah.

46:02 Thanks for having me.

46:02 Thanks, Michael.

46:03 And.

46:04 You bet.

46:04 We'll see you all later.

46:05 See you later.