Transcript #299: Will McGugan drops by

00:00 Hello and welcome to Python Bytes where we deliver Python news and headlines directly to your earbuds. This is episode 299 Recorded August 31st, and I'm Brian Okken. Hey, I'm Michael Kennedy And I'm Will Mubygin. Will's also known as usually the topic of Python Bytes Time to time you mentioned me so it's awesome to have you here, but you're You're part of Textualize, right?

00:27 Or you are Textualize?

00:28 - That's right, I guess I'm part of Textualize.

00:31 And we are a company, a very small company, but we're a tech startup.

00:35 We have three employees, and we'll have five in a few months.

00:39 - Nice.

00:40 - Yeah, tell us a bit about it.

00:41 You know, people know, that's amazing.

00:43 Tell people about it.

00:44 They know about Rich, and Textual perhaps, 'cause we talk about all the things that are adopting Rich, but you actually have a company around that, which is super fantastic, right?

00:55 So the model is we're building textual, which is going to be like a free, an open source project distributed through PyPy.

01:02 And then somewhere down the line, we're going to make this web service, which takes those applications and then serves them on the web.

01:09 And it'll be like a nice free tier, but we'll be able to add services on top of that, which we can charge a subscription for.

01:18 That's fantastic.

01:19 Like Tui's as a service.

01:20 Tui's as a service, yes.

01:21 There you go.

01:22 A TAS.

01:23 A TAS platform.

01:24 All right.

01:25 >> All right. Cool. Well, it's great to see all the progress there.

01:28 >> It is.

01:29 >> Brian, am I up on the first one?

01:30 >> You are.

01:32 >> This one comes to us from John Hagan.

01:35 Thank you, John, for sending this in.

01:37 IPI, the warning is gone, but they were under a pretty heavy phishing attack, and they wanted everyone to know they had this big banner that was letting, anyone who maintains a package is really where the problem lies.

01:53 If you maintained a package like say, Rich, the goal of this phishing attempt was to get your credentials so that you could then, so that they could sign in and put malware into that package.

02:06 The more popular, the better, I presume.

02:08 So this was a couple days ago, August 24th, so like a week or so.

02:13 It says, "Today we received reports of," this is from PyPI, the Python Package Index.

02:18 "Today we received reports of a phishing campaign targeting PyPI users. This is the first known phishing attack against PyPI. We're publishing the details here to raise awareness of what is likely going to be an ongoing threat. There was many comments and quote tweets and so on. It said the background of phishing message claims to be there's a mandatory validation process being implemented and they invite users to follow a link to validate their package, like a so, otherwise their package will be removed. Importantly, they say, note, we will never remove a valid project from the index. IPI only removes projects which violate their terms of service or in some way determined to be harmful, for example, malware. This takes you over to this site if you were to click the link to if you look carefully here, sites.google.com/view/ipivalidate/validate-ipi-package for some kind of redirect. And it does bad things when you fill this out. It just post the forum to somewhere else.

03:21 So I guess they were hosting it on google.com, you know, sites.google.com in an attempt to avoid like the domain getting blocked, something like that, right?

03:30 But it posts over to like linkedopports.com or some, I don't even know how you pronounce that domain, but don't go there, it's not good.

03:39 And it says the malicious releases follow a pattern, exotel, and I kind of laughed, even though it's not really funny.

03:46 One of the packages that got phished through this email is called spam.

03:52 I don't know what spam does, but if you were phished by spam and your package is called spam, it's just too much meta.

04:04 Will did you hear about this?

04:05 Did you get any notifications for your packages?

04:07 I didn't, but I could see how if I got that early in the morning, I could fall for it.

04:12 Yeah, exactly.

04:13 You're just waking up.

04:14 Not another thing I got to do.

04:17 I feel like they're taking a little bit advantage of the notifications coming from the Python Packaging Authority, where they've been sending out messages about security and about packages, and there's the critical packages.

04:30 Will, I saw you tweet, and Brian was like, "Wait, what is this?" About what is a critical package the day that stuff came out.

04:36 And so, you know, it kind of, I think, is trying to hide under that noise and slip through the cracks there a bit.

04:44 I think I don't get very many notifications from PyPy.

04:48 I think the fact that I don't get very many, I might just take them at their word.

04:53 If you get a lot of notifications, you might learn to recognize what is a legit notification and what is some kind of phishing attempt.

05:01 - So you're asking PyPI to send us more email?

05:04 - No, not really.

05:05 (both laughing)

05:09 - Funny, so you can actually look at what the code does that is the malicious versions.

05:14 They said they've taken down and they've taken down several hundred typo squatting ones that also do a pattern.

05:19 The code is a setup.

05:21 No, it's just hijacking the most used function or feature.

05:26 Try, get the computer name and then request to install the thing.

05:33 If the, off of this malicious place, if the platform is Win32, we're going to get this script, download this malicious exe, download it with requests, and then write it to a file and then just execute some executable.

05:51 That's pretty much what it was attempting to do to all these packages.

05:56 >> It's kind of lazy this malware is not cross-platform when he works on Windows.

06:01 >> I agree. Well, I mean, also they're looking for investors so that they can expand to other platforms.

06:10 maybe they'll get a macOS and a Linux one coming at some point.

06:13 >> Just hope that Avizi is listening.

06:16 >> Exactly. Here's an opportunity to, I don't know what it does when it runs, but it can't be good, whatever it does.

06:23 Can't be good. Now, I remember I got an e-mail from someone, and I'm sorry I didn't pull the details together as part of the write up here, saying, I think it was on Twitter, a DM that said, "You're probably going to talk about this.

06:39 The 2FA wouldn't necessarily help you because if they asked for the 2FA, you're going to enter it in maybe there and it might pass it through as part of the process as well.

06:51 It could maybe get a software-based 2FA and use it.

06:56 But you know what does help with this really, really well? Password managers.

07:00 One password, last pass, and so on.

07:03 If you go there and it says, "Enter your password," and you hit the hotkey to like fill out the site or you click in there like with one password, if you just access it with the latest version, it'll automatically drop down or suggest the drop down for the site. If you do that, it will not come up with anything because you don't have an account at sites.google.com, presumably for this login, right?

07:25 >> Yeah.

07:25 >> And so some combination thereof, I think, you know, is at least I always, If I go to a site, the less sure I am, I like double check, does the password manager think it should fill this account into this website?

07:39 If it says no, then I triple check it.

07:41 It's not just like, oh, the password manager is not working.

07:43 Let me just copy it over, you know.

07:45 Anyway, there's some more details in here.

07:48 You can read about what they're doing and what you should do.

07:51 Some ideas on how to verify the signed certificate.

07:55 I would prefer to just use a one, just a password manager right away instead of trying to follow the chain of the cert, but you can do that as well. Anyway, it says it's got a lot more details there if you want to check that out. What do you all think?

08:08 Yeah, it's interesting. And Sam, it's an ongoing threat. I think if you're an internet user, you're getting bombarded with this. With PyPy, if they get a hold of your, it seemed like it was more like attacking individuals. Were they trying to get your credentials to like, install malware in your package?

08:28 I'm pretty sure what they were doing is they were publishing a new version of your package.

08:33 Ah, okay.

08:34 And the new version had malware in it.

08:38 Yeah.

08:39 So, yeah.

08:40 That would be a huge concern.

08:41 Yeah, yeah.

08:42 It says, yeah, we've additionally determined that some maintainers of legitimate projects have been compromised and malware published as their latest release for those projects.

08:52 Their accounts have temporarily been frozen and the bad versions removed.

08:55 But that's what they were trying to do.

08:56 Okay.

08:57 >> Good on them for catching it.

08:59 >> Yeah, for sure. I love how Seth out the audience says, another win, WMIN for Linux on the desktop.

09:05 Yeah, how about that?

09:06 >> Yeah, or Mac.

09:07 I'm like, well, that wouldn't hit me at all.

09:09 >> Yeah. However, any package that you publish, it would affect your Windows users if you were the maintainer.

09:19 >> Which is half the users as we know, about at least half the operating systems are Windows.

09:25 - Yeah, yeah, yeah, not so good.

09:28 So Seth also points out that the timing of this phishing combined with Dustin mentioning there hadn't been any phishing attacks or something like that is, what's in Dustin Ingram?

09:41 Don't give people ideas.

09:43 By the way, you wanna hear more about this.

09:45 I did interview Dustin on this like three weeks ago or so on Talk Python, talking about Python packages and security and supply chain stuff.

09:55 but all that predated the phishing side, but not the malware side.

09:59 So people can check that out if they want.

10:01 - Okay.

10:02 - All right, anyway, make sure that you don't put your password for PyPI in the wrong place.

10:07 - Yeah.

10:08 All right.

10:09 How about we talk about something nice?

10:12 - Yes, friendly.

10:13 We can have nice things, not--

10:15 - If we can have nice things.

10:16 Like Python being on top of the IEEE spectrum languages.

10:21 So it has been for a while.

10:24 So this is, we're just reviewing this to say, yay us, partly.

10:29 But there's some interesting information here.

10:31 So if we go through the review stuff, there's a cool chart.

10:35 So the top programming languages of 2022.

10:37 Python's at the top, it's both the spectrum survey, actually, I don't know where the spectrum number comes from.

10:45 I assume it's a survey thing.

10:47 And then trending, it's on top also, a little different, and cool animation graphics on there.

10:53 But one of the interesting bits is in jobs.

10:56 If you look at job listings, I think they were looking at job listings and requirements.

11:00 SQL is at the top.

11:02 And this is a comment because it's not just SQL.

11:07 They're not gonna just say, we need somebody that knows SQL.

11:12 But it's like Python and SQL or Java and SQL or JavaScript and SQL.

11:16 The end SQL part is coming in a lot now and actually more than it used to, which is interesting.

11:23 There's also a related article on the same, on IEEE as well called "The Rise of SQL." And it's really talking about that, that it's not just because, I mean, I do remember SQL's always been a part of programming or it has been for my career, but it's often been a larger thing.

11:46 It's not something you do in a small application, it's something in the server or large applications, but it's growing in using it for even little small things.

11:55 And a lot of applications, you don't have a specialist doing the database stuff.

12:00 You've got the developers doing the database, everything.

12:05 So anyway, I thought that was interesting that just the highlighting that SQL is and always will be important and it's even growing in popularity.

12:14 - It's interesting that Orms haven't made a dent in that.

12:18 You know, I used to use Django Orms back in the day didn't have to touch SQL. But clearly people are not using ORMs that much if SQL is like top of the charts there.

12:31 My first thought when I looked at this was there's probably a lot of data warehouse, data lake, semi-structured data that people are exploring with SQLs and queries before they lock it down and productionize it with an ORM or something along those lines, right?

12:48 like the data science side, if a lot of that data is dropped into a database by an API or some web scraping or something, and then you have to ask it questions.

12:57 And like knowing the SQL is the asking arbitrary questions of the data before you really know what questions to ask is my first thought, but yeah, ORM's all the way for the win for me.

13:06 I don't want to do straight SQL.

13:10 - There's also a growing thing of just doing a small, like a SQLite, just the knowledge that SQLite is everywhere.

13:17 And I don't know if SQLite has any effect on this or if Simon Willison does, because he's with Dataset taking like CSVs and stuff and turning them into a little SQLite websites.

13:32 But things like that have just even, services and smaller applications collecting data.

13:40 - Isn't there a JavaScript API to SQLite?

13:44 I think all languages have, I think, APIs into SQLite.

13:49 I think something built into the browser is local storage.

13:52 Does that not use SQLite or am I mistaken?

13:55 - Don't think that that does, but it's very similar if it's not the same.

14:00 It's like, yeah, what is it called?

14:03 Local SQL or local DB, something like that.

14:05 - I wonder if that's contributing to--

14:08 - Yeah, yeah.

14:09 Dave out in the audience says, "I wonder how the jobs one was measured, though.

14:12 SQL can be one of those not really considered must-have items that often goes into the list of requirements.

14:18 You really could just use an ORM.

14:21 - Yeah, and that might be the case.

14:25 But even with ORMs, sometimes you gotta get in there and find out what's wrong, like do optimizations and things.

14:32 - Yeah, Kim out in the audience says, "Is it becoming less common for developers "to know at least enough SQL to be dangerous?" I think that that's the minimum bar for what you should know to use an ORM.

14:42 You should know kind of what's happening.

14:45 You should know what a join is and that you might wanna do it so it doesn't do 50 queries.

14:49 You do only one and things like that.

14:51 - Yeah.

14:52 - But yeah, very, very cool, Brian.

14:54 - Well, where are we at next?

14:57 - Well, I think Will is up.

14:59 What have you got for our first one, Will?

15:01 - Oh, I'm on the wrong page.

15:02 So I came across this article.

15:04 It's by Charlie Marsh, using mypy in production at Spring.

15:10 And I looked at Spring and it's some kind of interface for researching medicine.

15:19 But he was saying that they have a big project, 300,000 lines of Python.

15:25 And they started typing it and that was a few years ago.

15:29 And they've since typed the entire code base.

15:32 And they were very impressed.

15:34 It's reduced bugs and made things more maintainable.

15:38 and even have all the strictness settings on to max.

15:42 I'm a big fan of typing, but I don't have everything on max.

15:48 I relax it just a tiny bit, but they've got all the settings on there.

15:54 So he covers the basics of typing and some interesting stuff about the history of typing.

16:01 It's interesting how typing started.

16:05 it didn't spring out of nowhere with the same goals.

16:08 It did actually evolve through various other projects.

16:12 Originally when mypy started, I think they were building something that was essentially more like mypy C.

16:18 It was to run Python with typing.

16:21 - To use the typing to actually sort of transpile to another language almost, right?

16:25 - Yeah, yeah, and that turned into a type checker which didn't actually run your code, it just analyzed your code.

16:32 So it's quite an interesting article They cover how it started and go through a bunch of their experiences.

16:41 With typing and mypy in general, they cover improved readability.

16:47 That's a big one for me.

16:48 I love typing.

16:50 And I find it makes code more readable.

16:53 Some people would disagree because you add lots of these annotations and some people find that clutters your code.

17:00 I find it super helpful to understand other people's code and to understand my code because I don't have to remember the types of everything when I can just see them written down.

17:12 - I'm totally with you on that.

17:14 I think there's a fear that, oh, look, Python is becoming like C++ or something.

17:20 But it's nice and clean and it's simple until you're focused on some area, some function or a class or something.

17:27 You're like, well, what happens here?

17:29 you can either go read the documentation, try to put it together, or you can go find all the places that's used and try to put it together.

17:36 Or if it has types, you don't have to go do that exploration.

17:39 You're just like, "Okay, this is an integer and this is a list of users.

17:42 I know what's happening now.

17:44 I don't need to hold more information in my brain." I think that makes it great.

17:48 >> The shape of data is what really helps me.

17:51 You mentioned a number or a list or something.

17:54 But sometimes things can either conceptually, an argument to a function could be one thing or could be a set of things.

18:01 Is it expecting if it's just one thing, can I do that or is it expect and types can give you that.

18:09 One of the things you mentioned which perfectly sums up my philosophy for typing is I don't want to do it, but I want everybody else to put types in there.

18:21 Because it's readability counts.

18:25 >> Also, Mr. Wilson in the audience points out typing greatly assists with code completion for the editors.

18:32 >> Yeah, it's got a bunch of tangential benefits. It's not just the one benefit. I mean, it catches bugs. But even if it didn't, I think typing would be an excellent addition to Python.

18:46 >> If I can hit dot and it gives me more help, I'm already happy.

18:49 - Yeah, bingo, yeah.

18:50 I love typing.

18:52 This is a very nice article.

18:54 I'll let people read it.

18:56 - Yeah.

18:57 They also talk about the pain points of how that was painful for them.

19:00 I think using, trying to get mypy to completely analyze everything is a different level of I want my public interface to say what it returns.

19:10 You know, there's like, you gotta decide where do you live on this spectrum and what are some of the goals?

19:15 Like, catching bugs, it's more important to have everything covered.

19:18 of documentation, you know, a little bit less, I think.

19:21 Yeah, I found it's changed my programming style. The code I write is, is less dynamic, I'm more likely to fix types quite early on. And I don't do any well, I don't do too much get outer and set outer. And I don't use all the dynamic capabilities of Python, I prefer to write static code that looks a bit more like C. So I can understand why people have that they feel like it's taken away a bit of freedom from them. But I do think it's given you the freedom to write solid code that doesn't have, you know, no attributes on none type errors.

19:59 Yeah, I agree. I think you can compare it to things like TypeScript. TypeScript has a similar idea, but TypeScript is very particular. And if you don't get it just right, it'll give you compiler errors and it won't, it won't do the steps it needs to do to make JavaScript, whereas Python, maybe your editor will give you a warning or some tool like my pile give you a warning, but it doesn't really get in the way of it still functioning, you know, which unless you're doing something where it depends upon it, like pedantic or fast API where it's actually using that. But most of the time it's, it's there when you want it and you can kind of ignore it if you don't. Yeah. Yeah. So rich has a lot of, Yeah, it's fully typed. Yeah. So textual. It's not passing my pie. Currently, there are some like little dynamic corners and little typing errors, which we're gradually improving.

20:52 But all new code is typed and, and changes are typed. So yeah, we're really big on on typing and textualize.

21:00 - Brian, how about with your pytest extensions, plugins?

21:05 - You know, it doesn't really come up much, but I don't really think about it a lot, actually.

21:12 So I would like, but I have other applications that I'm working on that I definitely involve typing.

21:19 And I started out with just the, like you were saying, trying to help with documentation.

21:24 So making sure the API is typed.

21:27 I think that's essential.

21:28 I think it's just a good idea.

21:30 Especially now with the improvements of some of the typing.

21:33 So you can do, you don't have to say union anymore.

21:36 I like the bar for or, like it's a non, you know, it's this, it's an int or a non or something like that.

21:43 That's way cleaner than it used to be.

21:45 And you don't have to import typing as much as you used to.

21:48 I don't want to import typing just so that I can type some, type in something that seems wrong.

21:54 But, and I'm on the phase of trying to integrate it more in the rest of my code, just because I'm, you know, even in a solo project, sometimes you're also a user, 'cause you come back to something six months from now and try to figure out what you're doing.

22:09 And it's nice to be able to not have to look at the code.

22:11 So I like it.

22:12 - Yeah, absolutely, absolutely.

22:14 Well, we could find Will.

22:16 - Yeah, so I wanna say something nice about the AI in real life, and actually the podcast IRL from Mozilla.

22:24 So this episode of Python Bytes is brought to you by the IRL podcast, an original podcast from Mozilla.

22:31 And I'm really enjoying it.

22:33 I'm listening to a whole bunch of it.

22:35 If you care about ideas behind technology, not just the tech itself, you'll enjoy IRL.

22:40 Tech has an enormous influence on our society.

22:43 Many effects are beneficial.

22:45 The influences, like for instance, the information and assistance we get through cell phones is amazing.

22:50 I love being able to look up the closest coffee shop wherever I'm dropped on the earth or knowing where my kids are.

22:57 But some are not so great because like, I don't want somebody else to know how often I hit the coffee shop and I definitely don't want somebody else tracking my kids.

23:05 So Mozilla has always been on the lookout for possible downsides to technology and works to mitigate negative influences of tech on the negative influences on all of us.

23:16 If ideas like that and concerns about technology resonate with you, you should definitely check out the IRL podcast.

23:23 This season is hosted by Bridget Todd and is looking at AI in real life.

23:28 Who can AI help?

23:29 And also, who can it harm?

23:30 The show features fascinating conversations with people who are working to build a more trustworthy AI and also using AI to help us.

23:39 So I really enjoyed a few episodes so far.

23:42 There's an episode on how our world is mapped with AI.

23:46 So data and maps is being used to make decisions that affect real people, even like by districts and by governments.

23:54 But how can people reclaim the power over their own maps and stories using AI?

23:59 This is fascinating episode.

24:01 Another episode is about gig workers who depend on apps for their livelihood.

24:06 It looks at how they're pushing back against algorithms to control how much they get paid and seeking new ways to gain power over the data to create better working conditions.

24:16 And how about elections?

24:18 So episode four of this season addresses the role that AI plays when it comes to both spreading dense information around elections, but also how to combat disinformation.

24:31 This is a huge concern for democracies around the world and for me, especially in the US, but I know it affects everybody.

24:38 If this sounds interesting to you, you should try it because it is interesting.

24:43 Try an episode for yourself.

24:44 Just search for IRL in your podcast player or visit pythonbytes.fm/irl.

24:50 I think the best way is to select, go to pythonbytes.fm/irl so they know you came from us.

24:56 And the link is in your show notes.

25:00 Thank you IRL and Mozilla for supporting our show.

25:03 - Yeah, cool podcast.

25:04 Thank you.

25:05 Thank you, Mozilla.

25:06 All right, on to the next one.

25:08 Well, we already touched a little bit on the whole ORM thing.

25:11 And I hear some people use Django.

25:14 It's a web framework.

25:15 - Yeah, a few people use it.

25:17 Proud of it.

25:18 - Yeah.

25:20 So really popular and they're picking up the speed, of course, for their releases, right?

25:24 For a long time it was one, then we had two, three, four.

25:27 Going really quickly over a couple years there.

25:30 Well, one of the big moves with many of the web frameworks ever since Python 3.6 or so has been how are you going to participate and facilitate using async and await, right?

25:42 If you're doing a long database query and you block with an ORM request, for example, what are you, you know, how do you parallelize that or scale that without much effort?

25:53 Well, the async IO is perfect for it, but if your APIs don't support it, you can't use it.

26:00 And Django has been making its way towards having async capabilities, but what is the one thing that websites wait on the most?

26:08 Databases.

26:09 What is the one thing Django did not have async support for?

26:12 Databases.

26:13 So, it's a little bit late here on the announcement.

26:17 So, in beginning of August, Django 4.1 came out and this means 3.8 and above.

26:25 But the big deal is the second one actually is an asynchronous ORM interface for doing queries.

26:31 So, you can do anything that's a query set.

26:34 So, you can say, like if you have a class, a model class called authors, you would say author.objects.filter, and then you do a thing and so on.

26:41 So now you can say async for and do your query, and now it's all happening async.

26:48 And if you want to do like a join author.books, you can await getting access to that thing.

26:55 Books normally has a .first in this example, but they've now added also an a first.

27:02 So if you want the async version, you put the a as a prefix.

27:06 I'm not sure how I feel about that.

27:08 I'm not sure I would have gone this path, but you know, it doesn't really matter.

27:12 It's awesome that there's some kind of async support in the Django ORM, so that's really, really cool.

27:17 I think, I just wanted to highlight that, that this has been a major blocker to like real async programming in Django.

27:24 It's like, well, you can make the web view method async, but then you can't do async stuff that you really want to do, so, you know, where are you?

27:31 Right, this is like, this unlocks the final keys, right?

27:35 could call APIs previously with say HTTPS asynchronously, but then block on the database.

27:40 Now just use the A version and off you go.

27:43 I guess they couldn't make the one without A, they couldn't make first awaitable because that would break old code, I imagine.

27:52 Yes, but here's my thought, right? So what I'm getting back, so when I say async for author in query, you're now switching into an async mode. So I think the thing that returns would be really great if it like now everything must be async on it.

28:07 If you just said for author in query, now it returns a synchronous and everything on it must be synchronous.

28:13 This is how I would have maybe done instead of trying to prefix everything with A and double down on it, but maybe it was just a bridge too far.

28:21 I don't know. But this is what I had in mind.

28:23 I said I'm not sure what it is.

28:25 You can go in async mode or synchronous mode, and then you're there is what I had in mind.

28:30 >> Yeah, that makes sense. The async for would return a special version of the object, which had different first methods, the same API, but awaitable.

28:39 - Right, but awaitable, exactly, exactly. You know, that doesn't mean they can't do that in the future, potentially, but yeah.

28:46 - Yeah.

28:47 - Okay, a few other updates, just for, while I'm already here, the thing I really wanted to call it is async ORM in Django, good to go. Also, you can have class-based views, where you have a class and then methods like get, post, put, and so on.

29:02 Or you can have just method-based ones.

29:04 I prefer the method-based stuff with a decorator, but if you have the class-based ones, they now can also be async, right? So that's cool.

29:13 And there's also some validation of constraints is one of the other big changes.

29:17 So check unique and exclusion constraints defined in meta constraints are options are now checked during model validation.

29:25 Apparently they weren't before.

29:26 So that seems pretty valuable too, but the ORM is the big news, I think.

29:30 - Yeah. - Yep.

29:32 - That must mean-- - Cool, all right.

29:33 So yeah, it's great to see Django coming along.

29:35 It's been around for so long as a stalwart of the Python web world, and now it's much closer to the most modern features, which is great.

29:45 - Yeah, very cool.

29:46 - Yep, all right, Brian, what you got next for us?

29:48 - I have walrus operators. (laughs)

29:54 So I really--

29:57 Walrus is on the brain.

29:58 I do. I like Walrus operators, the Walrus operator, but I don't think I've been using it enough.

30:03 And especially because this article is telling me all sorts of places that I should use it more.

30:08 So I've got a an article from Martin Hines titled, You should be using Python's Walrus operator, operator, and here's why. And there's some just some stuff that I never even really thought about before like it just starts right off the bat.

30:25 I'm talking about the as as a basics in the basic section.

30:30 I never would have thought about that.

30:31 So there's a there's a list that happens to call a function to create the data in a list and it calls it three times.

30:38 Now really, I probably wouldn't have done this in code.

30:40 I probably would have called the function once and then named the variable and stuck it in there, but you can do that.

30:46 It's still easier.

30:47 You can do the all the function in the first element and save the value and then use the value in future operations just to create a list.

30:56 Right off the bat, that's pretty cool.

30:58 I wouldn't have thought to do that. It's nice.

31:01 I didn't actually quite follow this.

31:03 We get to save.

31:04 Here's a comprehension where a function is called twice.

31:11 You put in the value of a function if something around the function, like if it's true or if it evaluates Boolean true.

31:22 You can do that with the walrus operator and only call it once.

31:26 That's cool. One of the things I really liked around was, I didn't think about before, but I'm definitely going to use it now, is the regular expression match function.

31:38 You often had to call match, and then if something was found, then you do something with the match object.

31:45 you get the groups or some other thing on the match object.

31:49 It is cleaner to just go ahead and do the query of whether or not the match returns something, right with a walrus operator, way cleaner code. I like this.

32:04 Actually, it's just a fairly big article talking about a whole bunch of places.

32:08 Now, here's the place where I like while true loops.

32:12 That always drives me nuts or having to flag something.

32:15 This is definitely a place where I started using whileRest operator right away of instead of saying like while true or while flag or something, do something and then break out if necessary or set the stop bit or something.

32:32 You can do that right within the while loop.

32:35 Actually, I don't know if it's, it is cleaner, it's less code.

32:40 I don't know if this is easier to read though.

32:43 Any thoughts from you guys?

32:45 - Yeah.

32:46 - I think once you know it, it's not too bad.

32:50 It is quite different from Python prior to the Walrus operator.

32:54 Even for me, I barely use the Walrus operator because I'm working on libraries and the minimum version is 3.7.

33:02 - Yeah, right.

33:03 - So I haven't trained myself to read Walrus operator, but to me, that doesn't look too bad.

33:08 That looks really clear.

33:10 - Okay.

33:11 - Yeah, this is nice.

33:12 especially if you do one of those things where like, the example here is getting input from the user, where you might get input from the user and then say, "Wow, let's not exit," or whatever.

33:23 Then in your loop, you get the same input with the same basic question again, but you've got to ask it before to see if they ever enter.

33:30 You know, there's like this weird sort of do it two times, and you could skip that with the Walsh operator, which is very cool.

33:36 - Oh yeah, yeah, I used to do that, like on the top example of just putting the command equals input and doing that above.

33:42 >> Exactly. Yeah, and then do the same test. Exactly.

33:44 >> Yeah.

33:45 >> Exactly.

33:46 >> Because I really don't like while true loops.

33:49 >> Only if you mean really do it forever, or until, yeah, exactly.

33:54 Until it really is some case where you need to break out of it.

33:58 >> Anyway, so the rest of the article is great too, accumulating in place, there's a whole bunch of cool places.

34:04 Oh, this is one I really liked, I wanted to highlight.

34:07 >> String as well.

34:08 >> Naming values inside of an F-string.

34:10 So there's an example of an F-string where you take the date time, and you're using, for instance, you might use the date time value in two different formats.

34:21 It formatted in two different ways, once with what year, month, day, and once with A, which I don't know what A is, but which is the day spelled out for Friday.

34:33 Now, it's assigning the date time today value to a today variable and then using it, using the value in a format string and then using the today variable later in the same format string.

34:49 This is a pretty cool trick.

34:52 There's multiple times where I'm using the same value in a couple of places because I'm formatting it differently with an F-string.

34:59 This is pretty cool.

35:01 >> Yeah, that is pretty cool.

35:02 The one that I really like is the list comprehension, because that always drives me crazy.

35:07 if you're going to do an if section, right?

35:11 You know, if you're going to say x for x in collection, if something.

35:15 Yeah.

35:16 And that the thing you want in the list is some kind of like a database call or some other thing that has to be computed.

35:23 Then you need to test that computer value before the walrus operator, you had to call that function twice no matter what.

35:29 Like if that was a go get me the user from the data, like I want to go through all the emails and then get a list of users that correspond to them.

35:36 but maybe some of the emails don't actually exist in the database.

35:39 Every if statement has to be, get me the user if it exists.

35:43 And then the list result, the values select out is also get me the user, right?

35:47 And this way, when it's really expensive like that, it's super nice.

35:50 It also is really useful in this situation when you're doing data science stuff that expects one line of a thing.

35:57 You know, you're like, I want to do, I want to pass this expression to like a panda's data frame or some other thing.

36:03 And you can kind of get a little bit more done this way.

36:06 It's really nice.

36:07 >> Yeah. This is pretty cool.

36:08 Actually, I hadn't thought about doing it within the if clause within a comprehension or something.

36:16 >> Yeah. This is the one where there was no other way.

36:20 There is no way in 3.7 to do this without calling the function twice.

36:25 In a comprehension, you could do a totally different structure, like a loop or something, but in a comprehension, where you have to have one line like a data science scenario, you had to call it twice until the walrus came around.

36:36 So I think that's fantastic.

36:37 >> Yeah, I suppose you could create a list expression of the return value of bunk and then use a zip or something, but it's super awkward.

36:49 >> Right, exactly. Yeah.

36:51 >> All right.

36:51 >> Very cool. It's nice to see a bunch of different use cases because then you can see, "Oh, I will never do that. That's horrible." But this is really great and I didn't do that.

36:59 You could even see in the audience, people are reacting like, "This is amazing, But this one is, I don't know about this one.

37:04 This is, yeah.

37:05 Yeah, yeah.

37:06 So, cool.

37:07 What do we got next from you, Will?

37:11 Ah, yeah.

37:12 Oh, we're right on the right page.

37:15 So I'm sure we all love regular expressions.

37:19 We have a love-hate relationship with them.

37:21 Yeah, those, I don't like regular expressions, but I use them a lot because they're powerful.

37:28 There is no alternative a lot of the time.

37:33 But they are very difficult to read.

37:35 You tend to get long strings of gibberish, which even if you're very well versed in regular expressions, you might find it quite hard to parse and figure out what's actually doing when you've come back to your code in the afternoon.

37:51 But this is a library written by Al Swigert, and I think that's how you pronounce his name.

37:57 rhymes with wider.

37:58 He's an author and he's written this Python library, which gives you kind of like a nicer way of expressing regular expressions.

38:08 It compiles regular expressions from a bunch of function calls.

38:14 And these function calls are much more descriptive and they read quite well.

38:19 So you can essentially read a regular expression in the future and find it quite legible.

38:27 I like the either option. It's either this or it's that.

38:32 For example, it's very readable. Like here we've got exactly exactly five digits plus optional whitespace plus one or more non whitespace. That's very readable. If you read that a second time you'd know exactly what that did. But if you saw this, this is a very short regular expression.

38:47 Yeah, exactly.

38:48 Yeah, exactly.

38:49 Even if you're good at regular expressions, I've been using them for 15 years, I'd have to like analyze that. And it might take me several minutes to figure out what that does.

38:59 So you know, if it's quite powerful, just for such a short regular expression, but you can make much larger ones. You know, here's something that's more complicated, but it's still quite readable. Either non captioning group, non captioning group, either this or that, one or more of this, plus non-captioning group, it's readable. And you can, you know, come back to it and other developers see it. They can understand what's going on. And in the end, it compiles it to a regular expression. So it's just as fast and powerful. But now it's just easier to work with.

39:35 >> Yeah, it's really nice because the output of this little library is just the text pattern of the regular expression, which then you can do it. It's not like you've got to adopt this entire library for everything.

39:47 Yeah, yeah, you can, you know, just anywhere that you need to write a complex regular expression, you could use this. I guess if you wanted to develop the reg x, you could use this.

39:59 And then once you've done, you could compile it and then put the actual reg x back in your code. Or you could just leave it like that. It's probably not, it's not slow.

40:07 No, no, I would probably leave it like this. But I'm thinking if you're using another library where it expects a regular expression string, right? It's still totally compatible with that because you just say, "Give me the string," and off it goes.

40:21 Yeah, yeah. So it's not going to break anything. It's not like you're switching or you don't have to port anything per se. It's a nice drop-in thing where you need it. So yeah, it's pretty cool.

40:32 Yeah, very cool. Excellent find.

40:34 - So that thing's library's called H-U-M-R-E, is that humor-y?

40:40 - Human readable regular expressions, I think.

40:43 - I just like, I'm gonna call it humor, because then the regular expressions are humorous.

40:51 - Yeah.

40:52 Very nice.

40:54 - Okay.

40:56 - I love it.

40:57 All right, Brian, is that all of our things?

40:59 All of our main topics?

41:00 - I think that is.

41:01 Do we have any extras?

41:03 I didn't, but now I do.

41:05 - Okay, let's have it.

41:07 - Out in the audience, Dean pointed out that the very first PyData Tel Aviv is happening in December, December 13th, 2022.

41:17 So if you're in Tel Aviv and you care about Python data stuff, check that out.

41:22 The call for proposals, I think it's open for two weeks or something like that.

41:28 So yeah, if you wanna submit a talk or attend, then there you go.

41:32 >> Nice.

41:33 >> Cool. But that's my only extra.

41:35 >> Okay. Well, you know what?

41:38 I'm not going to let Will off the hook because the reason why I wanted you on here is so that you could promote Rich CLI.

41:46 >> Okay.

41:48 >> Why do you want more people to use Rich CLI or why do you think more people should use it?

41:55 >> I just think it's a cool project. I use it.

42:00 You've got all the power of rich, but it's on the command prompt so you can syntax highlight files You can also just generate Colorful rich style content you can put those in your bash scripts It's just a very useful thing Yeah, it's you can get it from homebrew if you do homebrew install rich and then you'll have rich at the the command prompt and you can use a peck and And yeah, it's got a lot of cool stuff.

42:32 >> So do you have some workflow that you're using it for on a regular basis or do you use all of these workflows?

42:38 >> It's more just a general tool.

42:40 >> Okay.

42:41 >> When I'm navigating the command prompts, I want to look at file.

42:45 If it's a large file, I can use pager, and I can page up and down.

42:49 Yeah, people check it out.

42:52 It can display nice tables.

42:55 You can take a CSV and turn it into a nicely formatted rich table and you can generate simple things like rules and oh, it can display markdown as well.

43:05 So it's kind of like a general toolbox of like rich related stuff in the command prompt.

43:11 I think I'm going to use it for, I didn't know it did CSV so easily.

43:16 So I think I'll use it.

43:19 So maybe anytime you might type more or cat or something like that to see the contents of a file you're proposing now I could type rich and get you know syntax highlighting and better.

43:29 Yeah yeah so you do rich and name the file then hyphen hyphen pager and it'll give you a nice textual style pager.

43:39 This came together in I think two weekends.

43:43 At some point I'll go back and polish it a bit more because there's a few issues people asking for new features.

43:51 - Okay. - Yeah, cool.

43:52 - Two weekends. - Yeah, Kim in the audience says, "Rich CLI has replaced Cat, JQ, and Markdown tooling "for me with one tool." - Cool.

43:59 - Yeah, very nice, Kim.

44:00 - Well, how about a, oh, yeah, a joke, and then I've got one more thing.

44:05 - Okay, well, I do have a joke, as you can imagine.

44:08 So here's an example of where somebody is using open source to help keep their account secure.

44:16 And this is some kind of list of like common passwords or really reused passwords that people want to, you know, somebody has posted these says, here's a list of passwords that people seem to use a lot and get reused a lot.

44:29 So please don't use this as a password or check and don't let people use these passwords for their accounts, right?

44:35 So someone comes along to this, to this repo and they remove the word dolphins as a PR.

44:43 And the message is remove my password from list so hackers won't be able to hack me.

44:49 The list of 10 million password top 1000 list.

44:55 I mean, this is proactive business right here.

44:58 - That might actually work if the hackers are very lazy and don't look at the get history.

45:04 - It might actually, there may be a very small percentage of effectiveness to this.

45:09 You're also saying my email address is this and my password is that.

45:12 so please don't put them together.

45:14 - By the way, this is my credit card number, so don't paste it anywhere.

45:19 - Anyway, that's my joke.

45:23 - Okay, well the last thing I just wanted to say is I got a new hat recently, so I wanted to show off my new hat.

45:28 - Oh yeah, let's see it.

45:30 - It's a top hat. - I love it.

45:32 It's a top hat with a five inch butterfly.

45:35 - Oh, it's got lots of butterflies.

45:37 And they're all leather, and it's custom made from a guy in Oakland.

45:41 So it's my new hat.

45:42 - It's fantastic.

45:44 And it perfectly matches both your shirt and your background.

45:47 - Did that on purpose, yeah.

45:49 Or on purpose, or on dolphin, no.

45:53 - Fantastic.

45:58 - Well, thanks Will for joining us today.

46:01 - Yeah, pleasure.

46:02 - Yeah, thanks Will.

46:03 - Thanks Michael.

46:03 - Yeah, you bet.

46:05 We'll talk to you all later.