Transcript #380: Debugging with your eyes

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 380, recorded on April 23rd, 2023.

00:13 I'm Michael Kennedy.

00:14 And I am Brian Okken.

00:15 And this episode is brought to you by us.

00:19 Support us through our courses at Talk Python Training, the Complete pytest course,

00:23 Patreon supporters, links at the top of the show notes.

00:26 So very much appreciate that.

00:28 And while you're there, you can connect with us over on Fostedon if you mastodon there.

00:33 So mastodon anywhere, but you can find us on Fostedon, at mkennedy@brianockettn@pythonbytes.

00:39 Join the show live, pythonbytes.fm/live.

00:42 Usually Tuesdays at 10am Pacific Time now.

00:46 And you see all the older versions there if you want the video as well.

00:50 And finally, Brian, a bunch of people are signing up for the newsletter that you're

00:54 sending out about things from the show every week.

00:57 So that's awesome.

00:58 People can just visit pythonbytes.fm, click on newsletter right in the middle of the top

01:03 of the screen and put in their email.

01:05 We will treat it kindly, but then we will email you stuff that we're up to, which we'd

01:09 love to do.

01:10 So we appreciate that.

01:11 And, you know, I really want to just like maybe focus on that kind of stuff.

01:15 Brian, what do you think?

01:15 Let's focus, man.

01:17 Let's focus.

01:18 Speaking of focus, we've got NumFocus.

01:21 So NumFocus is a, you know, actually, I probably should have done a little more research.

01:27 NumFocus is a collection of different resources.

01:29 And let's just take a look at the about of NumFocus.

01:34 So NumFocus has a mission of promoting open practices in research data and scientific

01:41 computing.

01:41 There's a lot of information on the NumFocus site.

01:46 You can check it out.

01:47 But if you take a look at the projects that are involved, this is crazy.

01:51 So the projects, sponsored projects, there's a lot of our favorites like NumPy, Pandas,

01:58 Jupyter, SciPy.

02:00 So many things are involved with NumFocus and collaborate with NumFocus.

02:08 And I'm not, like I said, we should have had Pamphil on to talk about it a little bit.

02:13 But Pamphil let us know something that's going on with the NumFocus group.

02:19 And it's a little, there's some changes going on.

02:23 So this was suggested by Pamphil Roy, who's in the audience right now.

02:29 So thanks for showing up.

02:30 So this was an article by Paul Ivanov called NumFocus Concerns.

02:37 And we'll link to it in the show notes, of course.

02:38 But there has been some, there's some shakeup going on in NumFocus a little bit.

02:46 There's been some problems in the past with NumFocus being able to meet the expectations

02:54 of some of the projects within the NumFocus banner.

02:58 And there was a town hall meeting in February announcing that there's a new direction.

03:03 And it caught a lot of people by surprise.

03:07 So I'm trying to highlight it here as well, so people know about it.

03:10 There have, there's really, I kind of want to point people to this article and just say

03:16 that there's some things changing.

03:17 There's apparently in the past, there was some lack of transparency of how the board

03:22 was selected.

03:23 So they're trying to make that a little bit more transparent.

03:25 There is an initiated effort to elect open board seats to try to get more people on the

03:33 board and some proposed changes to the governance structure.

03:37 And then around some of these issues, there's also some of the projects within NumFocus

03:45 are pursuing alternative venues for fiscal sponsorship.

03:50 So getting money in other ways.

03:52 So a lot of information here.

03:54 I thought it was interesting, some of the different alternatives to, there's like open

04:03 source collective or some of the ways to get money.

04:06 There's different, I mean, money is important to try to get some of the projects, some people

04:10 working on it.

04:11 So if you'd like to get more involved or just know, have more information about what's going

04:17 on with NumFocus, this is a really great write up.

04:21 So thanks for passing this along.

04:22 Excellent.

04:25 I, you know, NumFocus is interesting.

04:27 It's really one of the bigger ways that funds Python open source and outside of Python as

04:33 well, but there's not many other organizations like that.

04:38 So keeping it, keeping it healthy is definitely important.

04:42 Yeah.

04:43 I'm glad it's, it got, there's some attention drawn, being drawn to it before it, you know,

04:50 kind of implodes.

04:51 So I don't think it will.

04:52 I think we'll see NumFocus for quite a while.

04:55 So definitely.

04:57 All right.

04:57 Speaking of shining a little bit of light on something, let's talk about leaping.

05:03 Python, this, right, this, this pytest project should be one that you're focusing on, but

05:11 I beat you to it.

05:12 So here we go.

05:13 Have you heard of this leaping?

05:14 I have not.

05:15 Okay.

05:16 Well, it's because the description is so, wait, no, there's no description.

05:19 This is a small project that does, it's got 238 stars.

05:23 So it's not a huge thing, but I want to give it a bit of a shout out because I think this

05:29 is cool and I would love to hear your take, right?

05:32 So leaping is a pytest debugger, simple, fast, lightweight for Python tests, and it traces

05:40 the execution of your code and then allows you, so you run a test session, you know,

05:45 pytest dot whatever.

05:47 And then you can retroactively ask questions about how your pytest session went using natural

05:56 language.

05:57 Okay.

05:58 Okay.

06:00 So like, well, what would you possibly ask it?

06:03 So it does this by keeping track of the variable changes at variables changing over time and

06:07 other sources of non-determinism within your code.

06:11 So you would just say pytest dash dash leaping.

06:14 If you install that, then it runs.

06:16 You can ask questions like, why am I not hitting this function?

06:22 Why was this variable set to this value?

06:24 What is the value of a variable at this point?

06:29 And what changes can I make to my code to make this test pass even stuff like that?

06:33 I see this is pretty neat.

06:36 You know, I haven't, don't have any experience with it, but it sounds, sounds pretty creative.

06:43 It says it's based on both Ollama and GPT-4.

06:46 You can pick which model you would like.

06:48 And those are both pretty powerful.

06:50 So.

06:50 Why leaping?

06:52 Leaping llamas?

06:55 I don't know.

06:57 That's well, typically llamas do leap a lot.

07:00 No, I don't think they do actually.

07:02 Maybe a little bit.

07:03 Okay.

07:07 I don't know.

07:08 I can't tell you why.

07:09 Maybe I think it might come from a larger project that here, but I don't really know.

07:16 Well, I'll play with it and maybe we could get somebody on to tell us or I'll ask somebody

07:22 why leaping.

07:23 Anyway, I thought that was kind of interesting.

07:26 So I want to turn the light off.

07:28 Thanks for giving me some homework to work on.

07:30 Yes, of course.

07:31 Last one we gave, was it Mike Fiedler we gave homework this time?

07:36 I'm giving you homework.

07:37 Yeah.

07:37 Haven't heard back from Mike though.

07:39 What's up, Mike?

07:40 Yeah.

07:40 Where's that article, man?

07:41 Yeah.

07:41 Over to you.

07:45 So, okay.

07:47 So I've got an extras, extras, extra section because I kind of got it down a rabbit hole.

07:54 So on the last discussion of this NumFocus concerns, I was looking at, anyway, one of the other topics that Penfield passed over is that there's a 2024 Developer

08:11 Summit going on.

08:12 So I'll just get started.

08:13 2024 Developer Summit happening in Seattle, June 3rd to 5th.

08:19 This is a invite only thing.

08:22 So I'm just announcing it because it's cool.

08:24 Don't try to sign up because you can't, but that's okay.

08:27 It's still neat that we have one of the reasons why I wanted to bring it up is not to try

08:32 to promote it, but to say, with like some of the, what is it, XZ or something that last,

08:39 that bug that went by recently.

08:42 X-P-V-X, I can't remember.

08:43 X-Z.

08:44 Near downfall of all the internet.

08:48 Well, one of the problems was this discussion that people in a project don't talk to each

08:53 other that much.

08:54 And there's a lot of times where you can't really get away from that.

08:59 But the Scientific Python Developer Summit is one place where a lot of the people from

09:05 these Python scientific projects get together.

09:10 And it's pretty neat.

09:11 Last year was the first, and they did a whole bunch of cool things last year, including

09:18 some, yeah, a bunch of planning implemented.

09:22 They had a working group on sparse arrays.

09:25 A bunch of specs were worked on and even some pytest stuff.

09:30 So community building, lots of great resources to try to get some of these core things together.

09:37 And even some pytest plugins, which is pretty neat.

09:40 And so one of the things here was like another pytest plugin.

09:45 I'm like, cool, what's that do?

09:47 So popped over, this is pytest Regex.

09:50 And well, if you've got a large, especially parameterized, but really a large pytest code

09:58 test code base, sometimes you've got like quite a few tests coming in.

10:02 And how do you specify?

10:04 One of the ways you can pick out a subset of tests is to use the dash k option to say,

10:10 hey, I just want to use something that has tests like underscore 3D in it to try to get

10:15 those, but that might still be a long list.

10:18 And what this is, is it has the ability and there is some logic in the dash k.

10:22 So if you don't know about the logic of the dash k, definitely read my book or take my

10:26 course.

10:27 But the it isn't as powerful as a regular expression, but with this plugin, you can

10:36 use a regular expression to select the test names, which is kind of awesome.

10:41 I think it's kind of awesome.

10:43 It's also kind of scary to think of using regular expressions in test selection.

10:48 >> You're going to need to write a test for your command line.

10:51 >> Yeah.

10:53 Okay.

10:53 So, pytest Regex is one of my extra extras.

10:58 The next one on the list is this write up called by J. Carlos Roldan, I think.

11:05 My latest today I learned about Python.

11:09 And a lot of these are fun, but the thing that I wanted to highlight, I guess I always

11:15 just forget that underscores are a thing for long numbers and it's very handy for constants.

11:20 Okay.

11:21 The thing that I thought was neat was this, what was it?

11:27 There was an example of a decorator with just a class.

11:31 You don't have to import anything or decorator stuff.

11:35 If you just have a class with a dunder init and a dunder call, you can implement your

11:39 own decorator.

11:40 And I didn't realize that it was that easy.

11:42 So, kind of a cool, small example.

11:44 All right.

11:45 Next up on our extras and last is Ruff got a little faster.

11:51 So, version 0.4.0 of Ruff is supposedly greater than two times faster, which is 20 to 40%

12:01 speed up.

12:02 >> Wow.

12:03 >> So, these are pretty neat numbers.

12:06 So, it was already pretty zippy already.

12:08 So, it's pretty cool.

12:09 Anyway, those are my extras.

12:12 >> Yeah, very cool.

12:13 That was 0.4.0, yeah?

12:15 >> Yeah.

12:16 >> Okay.

12:16 I think that's not out yet, but it's going to be or something.

12:20 That's awesome.

12:20 I just did my pipx upgrade all, which is a really cool command.

12:26 Just says go find all the things that uses Python command line tools and upgrade them.

12:30 I got 1.3.0.1.37, but very cool.

12:36 All right.

12:38 Well, that's a lot of extra.

12:40 >> All right.

12:41 >> Well, yeah.

12:44 So, not the end of extra, I'm thinking, but a lot of extra.

12:48 >> Yeah.

12:48 >> So, let's talk about PyPI and packages.

12:53 Now, I've covered this a fair number of times where we've talked about, oh, there's somebody

12:58 uploading some horrible package that if you install it, bad thing happen.

13:02 Bad things happen.

13:04 But this has nothing to do with that.

13:05 Not directly, anyway.

13:06 Even though it might sound like it.

13:08 PyPI has completed its first security audit.

13:12 Okay.

13:13 So, this is an article, I believe by, no, Dustin Ingram, and says, who's part of the

13:22 Python Packaging Group Authority, says, we're proud to announce that PyPI has completed

13:29 its first ever external security audit.

13:32 The work was funded in partnership with the Open Technology Fund.

13:36 And they've done previous security stuff there.

13:39 And they selected TrailerBits, which is a very well-known security pen testing company,

13:47 to work on it.

13:49 And they spent, so if you've ever thought, like, should I have a security audit done

13:53 on my project?

13:54 Maybe.

13:55 But TrailerBits spent 10 engineering weeks of effort going, trying to break into the

14:03 systems and break them and looking at the code and making sure everything is good.

14:07 That's a lot of, I don't know what that costs, but that can't be cheap.

14:11 So, it's really cool that that was funded to make that happen.

14:18 The other important part is the scope.

14:20 So, this has to do specifically with what's called warehouse, which is when you go to

14:26 pypi.org, that thing, that website, the APIs, the stuff behind the scenes that people create

14:33 accounts at, that they upload packages to, right?

14:37 Like, that infrastructure, not PIP, not the packages stored in PIP, but like the infrastructure

14:43 that provides the website and the APIs.

14:46 As well as something called Cavitage, a custom open source container orchestration framework

14:54 that they created to deploy warehouse, which sounds interesting.

14:57 And I know nothing about this, but those are the two things which were done.

15:01 And the really nice part, everything's pretty much fine.

15:05 They decided that they didn't have any significant problems.

15:09 They found 29 different advisories.

15:11 14 were informational.

15:14 Six were low priority, eight were medium, and zero were high priority issues discovered.

15:19 So, that's pretty awesome, right?

15:20 >> That is pretty cool.

15:22 >> Yeah.

15:23 So, there's multiple articles and details published as follow-up.

15:28 So, like all of the stuff that they did there, it's all public and you can check it out if

15:32 you wish, but I feel like that's enough to give people the idea there.

15:36 So, thanks, Dustin, for writing that up.

15:38 And very good to hear that at least the infrastructure of PyPI is solid.

15:44 >> Cavitage sounds like a soup or something.

15:47 >> Mm.

15:48 >> Had a lovely Cavitage last night for dinner.

15:51 >> It does.

15:54 All right, well, that's our main items, Brian.

15:57 How are you feeling about it?

15:59 Got any more extras in there for us?

16:00 >> I have some personal extras.

16:02 >> All right, do it.

16:02 >> So, I wanted to shout out or just to highlight some personal extras.

16:07 So, on the pytest course that I have, the community was based on Slack, mostly trying

16:19 to use Slack, but Slack has this 90-day limitation thing on large communities, and it deletes

16:26 stuff.

16:26 So, I'm going to try out Podia community for the community feature of pytest courses.

16:34 So, I was just kind of hoping to reach out and say, has anybody tried Podia community

16:43 features and have a community set up on that, and how's it going?

16:46 If you have and you have some feedback for me, go ahead and contact me at onmastodon.

16:58 I'm @brianokin@fostodon.

17:00 Let me know if you have a cool community that I can check out.

17:02 That'd be neat.

17:03 And if you're interested in joining the pytest community itself, you can, of course, buy

17:08 a course, but you can also, I'm going to try to open it up to other people.

17:12 And if when I do make changes, I'll announce it both through our newsletter.

17:16 So, become a friend of the show at Python Bytes, or you can sign up for the newsletter

17:22 at Python Test Podcast also.

17:26 I'll announce it on both those things.

17:28 So, that's it.

17:29 Do you have any extras?

17:30 >> Excellent.

17:30 Yeah, let's see what we got here.

17:32 I have some extras, actually, but I got to set it up.

17:35 I don't want to spoil the joke.

17:36 It almost got the joke out there first.

17:39 So, the first thing is, recently had a lot of fun hanging out with Cecil Philip and Brian

17:45 Clark.

17:45 Those guys wrote the VS Code course at Talk Python, which is an awesome course.

17:52 Check it out at talkpython.fm.

17:54 Click on courses, right at the top.

17:56 But as sort of a follow up to that, we had a VS Code AMA.

18:02 And so, I had Brian and Cecil there, but also Luciana, who's been on the show before, and

18:07 Karthik from the Python VS Code team.

18:11 And we spent 35 minutes and 44 seconds taking questions from the audience and talking about

18:17 features and direction of Python and VS Code.

18:19 And that was a lot of fun.

18:20 So, people can check that out.

18:21 It's on YouTube.

18:22 And just, you know, go check it out if they want.

18:25 Next, do you G-Unicorn?

18:30 Not Goonicorn, because the icon is a green unicorn.

18:34 So, G-Unicorn has a CVE, which is not ideal.

18:40 CVE means there is some problem worth giving a number and a record to.

18:47 So, this is CVE-2824-1135.

18:52 And it's awaiting analysis.

18:55 It seems.

18:55 But G-Unicorn fails to properly validate transfer encoding headers, leading to HTTP request

19:03 smuggling vulnerabilities.

19:04 You don't want smugglers in your web app, do you, Brian?

19:07 No.

19:09 No.

19:09 By crafting requests with conflicting transfer encoding headers, attackers can bypass security

19:15 restrictions and access restricted endpoints.

19:17 So, I would say maybe you don't want to do that.

19:20 Hmm.

19:22 Okay.

19:24 Yeah, it doesn't sound incredibly dangerous, but it is a 7.5.

19:31 It is high in the danger level.

19:33 So, I guess it depends.

19:36 To me, it just depends on how are you actually restricting those things and what part of

19:44 G-Unicorn versus what part of your own code is actually checking whether something has

19:49 access to a thing and so on.

19:50 So, yeah.

19:54 But I want to put that out there because you might want to update your G-Unicorn.

19:59 Next up, another announcement.

20:02 You had the Sci-Fi one.

20:04 So, PyCon South Africa, PyCon ZA, is going to be a hybrid event.

20:11 And right now, the big news is that the talk submissions are open.

20:15 They prefer them in person, but they can be given remotely as well or recorded, I believe.

20:22 So, you can possibly submit a talk if you're interested.

20:26 The main conference is in October.

20:29 So, there's that.

20:30 And speaking of conferences, this one was sent in by Philip Jones.

20:34 Brian, what would happen if you had like a stealth conference that invaded some other

20:40 conference?

20:40 Like a symbiote.

20:43 [Laughter]

20:47 Yeah.

20:48 So, there's FlaskCon inside PyCon this year.

20:53 Okay.

20:54 So, on Friday, they will be having FlaskCon 2024.

20:59 And, you know, the Friday, which is May 17th, at PyCon US, and call for proposals are live.

21:07 Basically, they give you some ideas of things they might find interesting and so on.

21:15 But yeah, there's a whole series of events and introduction from David Lord, who leads

21:19 the Pallets project, which manages Flask among other things.

21:23 But yeah, there's a whole from 11am till 7pm, maybe till 6pm, depending on what you call

21:31 a conference, series just focused on Flask.

21:35 So, I think that's pretty interesting.

21:36 I'm most interested to just see how this logistically works out.

21:40 But if you're going to be there anyway, that's cool.

21:42 Yeah.

21:45 Actually, it's kind of an interesting idea.

21:46 It's on Friday, which I'm normally like, you know, going to other talks and other stuff

21:51 on Fridays.

21:52 And I'd be curious to see some other piggyback things.

21:55 Because at PyCon, there's the tutorial section before, and then there's the sprints after.

22:02 But there's also like, there's a lot less people in there.

22:05 So, there might be opportunities to do some other piggyback sub-conferences before or

22:14 after as well in the future.

22:15 Yeah.

22:16 Interesting.

22:17 Absolutely.

22:18 All right.

22:19 Are you ready to close this out with a debugging joke?

22:21 Yeah, sure.

22:23 Okay, we're going to do a little role playing here.

22:26 So, this is a conversation.

22:27 You want to be the developer or you want to be the person that's curious about how

22:32 developers work it out?

22:33 I'll be the developer.

22:36 Okay, you do the green bubble.

22:38 So, here's a text exchange between somebody who's sitting next to a software developer

22:44 on a train or something like that.

22:45 And then texting with their developer friend, go make this make sense.

22:48 Right?

22:48 Okay, so here's the non-developer.

22:51 Is it common for software engineers to take out their laptops on the train only to stare

22:56 at them without doing anything?

22:58 Well, yes.

23:00 Legally, you have to or you lose your license as a software engineer.

23:04 Seriously, he just shut his laptop, opened it back up, pressed a button and resumed staring

23:10 at it.

23:10 Oh, yeah.

23:11 And now he's browsing his phone while staring.

23:13 It's called debugging.

23:15 You stare at the code until it works again.

23:17 Why do you guys get paid so much?

23:20 Pretty good, right?

23:22 Yeah.

23:23 Well, it's yeah.

23:25 And it's further than that.

23:27 I mean, after staring at it for a while, I often bring in other people to stare at it

23:32 with me.

23:33 Can we just stare at this together for a while?

23:35 Because my staring is ineffective.

23:37 It's called cold reviews.

23:39 Exactly.

23:40 Sometimes AI will also stare at it with you.

23:43 It can also propose new ways to break it.

23:47 Yes, that's right.

23:49 Yeah.

23:49 All right.

23:52 Well, well, if I had if I had pytest leaping, I could just ask it why it's not working.

23:58 Exactly.

23:59 Come on.

24:00 Leap into action.

24:01 What's happening here?

24:02 >> All right. Well, thanks for being here, Brian.

24:05 Thank you to everyone for listening. Bye.