Transcript #245: Fire up your Python time machine (and test some code)

00:00 Hey there, thanks for listening. Before we jump into this episode, I just want to remind you

00:03 that this episode is brought to you by us over at Talk Python Training and Brian through his pytest

00:09 book. So if you want to get hands-on and learn something with Python, be sure to consider our

00:14 courses over at Talk Python Training. Visit them via pythonbytes.fm/courses. And if you're

00:21 looking to do testing and get better with pytest, check out Brian's book at pythonbytes.fm slash

00:26 pytest. Enjoy the episode. Hello and welcome to Python Bytes, where we deliver Python news and

00:32 headlines directly to your earbuds. This is episode 245, so it's not the first time. Recorded August

00:38 4th, 2021. I'm Brian Okken. I'm Michael Kennedy. I'm Juanpe. So Juanpe, thanks so much for coming on

00:46 the show. Can you introduce yourself a little bit before we get into it? Thank you very much for

00:51 having me. So my name is Juanpe. I'm from Spain and then my PhD in particle physics has been working

00:57 at CERN for four years. Then two years after my PhD finished, I decided to step away from academia and

01:04 start working in industry. And right now I'm working at FinancialForce, which is a company that develops

01:09 products for Salesforce. So I'm not developing products, I'm in the analytics team. So my job is

01:15 to analyze internal data as well as usage, product usage from a customer to allow the board to take

01:22 data different decisions and how the company should go forward. Nice. Yeah, super interesting. Give us

01:27 your thoughts real quick on one hand working for a place like CERN and then the other working on a place

01:32 that provides enhancements to Salesforce. Those sounds so different. Are they really that different or are they

01:38 similar or what's the story? Part? I mean, of course they're different, but there is a big part which is

01:44 very much the same, at least in the team that I'm working on. Because at CERN, basically what you do is

01:50 you don't know the answer to anything and you need to first know what you need to ask yourself. And this is

01:56 very similar to what happens today in my current job. Because for instance, marketing come and say,

02:01 we have this whatever campaign and we want to know if we're targeting right. And I need to know what do I need

02:09 to do to answer that question, but neither marketing knows. So it's like, let's figure things out. But yeah, I mean,

02:17 it's a pretty drastic change, but I don't know. I got a feeling that I needed to switch. I like coding a lot and I felt

02:25 at some point that I was enjoying more the coding part of being a physicist than the physics part. So I said,

02:31 I mean, you basically described why I'm not in math anymore.

02:34 Yeah.

02:38 I was working on projects and I was having so much fun and writing code on these silicon graphics,

02:42 like huge computers and stuff. And then there'd be parts where I'd be like, ah, this part's not so fun.

02:47 This part's amazing. And I realized the programming parts were amazing. And when I had to get down to

02:51 solving the math bits, I'm like, ah, darn, I gotta put it away, go work on the math again.

02:55 I mean, I remember the last year and a half, I was working on a project that was literally designing a system

03:01 that worked within GitLab CI to automate paper review publishing. So you don't need to have a lot of

03:09 people reading the paper and say, oh, this rule is not matched or do you need to fix this image?

03:15 So I built an entire pipeline in Python to check all of this that works based on pull requests and groups and so on in GitLab CI.

03:22 And I thought I've been a year and a half not doing almost any physics. So my physics work,

03:27 it was related to review paper because I was a chair of an editorial board. So I had an analysis. It was pretty cool,

03:34 but I wasn't, I wasn't doing it. I received version review the made comment, fix this, fix that, then go back to write

03:41 a pipeline to make the pipeline.

03:42 Yeah, exactly. Yeah. That sounds really cool. Yeah. Go ahead.

03:47 Will you kick us off today?

03:48 Will you want to hear about the state of the developer world? How's that sound?

03:53 I like state.

03:54 Yeah. Yeah. So here's an interesting survey results, I guess, put together by JetBrains,

03:59 the state of the developer ecosystem 2021. And I thought this would be fun to talk about because

04:04 we do cover like the PSF state of Python survey and things like that. But I thought it'd be fun to

04:11 just have a quick look at the broader landscape, what people are doing and where Python fits into

04:17 that. And you know, JetBrains has done a really good job with the PSF survey. So I thought, you know,

04:21 this, this will be similar. So let's check that out. So let me give you some stats and some rundown on

04:26 this. So basically the idea is it presents the results of the fifth annual developer ecosystem survey

04:33 conducted by JetBrains and it went out to 31,000 or had input from 31, 32,000 people. All right.

04:39 So there's a couple of interesting things they're doing in the presentation here, but say in that

04:44 world, JavaScript is still the most popular language. Not if you ask Stack Overflow, but of those 32,000

04:50 people or whatever, Python is more popular than Java overall. However, Java is used more as the main

04:58 language. So there's more people using Python for extra things or for other things and so on,

05:03 which I think that jives pretty well with my understanding of Python is that it's this really

05:07 amazing way to like bring in a little interactivity, bring in a little bit of analysis, a little Jupyter

05:13 notebook or something, even if it's not your, your main focus, right? You might be an economist,

05:17 you're not even a programmer and, but you're still Python person in a sense, whereas you probably wouldn't

05:22 be a Java person as an economist most of the time. Yeah. I'm a test. I use Python for testing.

05:28 Yeah, for sure. So let's see the top five languages that developers are planning to adopt

05:35 are Go, Kotlin, TypeScript, Python, and Rust. And the fastest growing languages are Python,

05:40 TypeScript, Kotlin, SQL, and Go. So for example, JavaScript was the most popular language. Java was

05:47 the most popular main language, but it's, they're neither the fastest growing languages. So that's

05:52 pretty much pretty interesting. Of this group, 71% people work on some sort of web backend APIs,

05:59 Flask apps, or, you know, Go apps or whatever. So they have a lot of interesting stuff here in terms

06:06 of analysis. So they have these blocks that show how popular a language is, it's been used, or how

06:12 likely people are to adopt it and pick it up. So there's a bunch of grids. If you go to the report

06:18 and you can check them out and basically the, the orange is the current state of the world in the,

06:23 there's a darker, almost black that is like the derivative. Like how quickly is that world changing

06:29 for the upswing, right? So for example, JavaScript has more orange squares, but it doesn't have as quick

06:36 of a growth or a planned growth, I guess. So Python has one of the larger ones of those. So does TypeScript

06:42 as well. And those are interesting to look into. You can compare those against different things.

06:46 You can also see the popularity of the language over time. Python's been going up and up and up,

06:50 although this year is kind of plateaued in this report. So that's, you know, maybe something worth

06:56 noting. There's obvious things going down, like objective C, you'd be insane to work on objective

07:01 C right now. And Swiss, Swift is like replaced it. Although that's going down as well. Let's see,

07:06 there's a few more things. They have these really interesting graphs that are both like grids,

07:10 but also heat maps. So you can, it'll let you answer questions like, okay, if I am currently a

07:15 Swift developer, what is the likelihood that I'm going to adopt Python? 6%. But if I'm a Kotlin developer,

07:22 that's 8% likelihood that I'm going to adopt. Oh no, I'm going the wrong way. If I'm a Kotlin developer,

07:29 I'm 10% likely to adopt, to move to Python and so on. So there's a lot of sort of like flow from one

07:35 language to another. I haven't seen any analysis like this anywhere else. Have you?

07:39 No, that's pretty interesting.

07:41 Yeah.

07:41 My head's looking at correlation or something. What's the first role? I'm curious.

07:44 I'm not planning on changing.

07:46 So they are the most likely to change.

07:48 Yeah. They're just stuck. They're just staying.

07:51 Yeah. All right. Let's see. Also interesting operating systems people use for development,

07:56 61% windows, 47% Linux, 44% macOS, which that's pretty high for macOS, given its general

08:03 popularity amongst like the computing world, I think.

08:06 Yeah. I think Linux is pretty high too.

08:08 Yeah.

08:09 It doesn't surprise me.

08:10 Yeah, exactly. And then 1% other, who knows what that is. Also questions about people using

08:15 the windows subsystem for Linux and stuff like that. There's also, if you're interested,

08:21 a similar heat map for like what type of software do you develop? So if you're trying to understand

08:26 where, like if I develop this kind of software, what is the distribution for programming languages

08:33 there? Right. Like it's interesting to say Python is popular or JavaScript is popular,

08:37 but if I'm an embedded system developer, is JavaScript still popular? I don't know. Probably not. Maybe,

08:42 but maybe not. Right. Maybe C is like really popular. So there's a really cool thing called what types

08:48 of software do you develop. There's like a grid plus heat map plus intersection of language and type.

08:53 So if I develop security software, there's a 9% chance that I would be doing Python versus a 6%

09:01 chance of Java. On the other hand, if I do blockchain, how does that break down and so on? Let's see where

09:05 is Python kind of notable? On utility little scripts, it's quite popular there. Yeah. Database backends,

09:13 pretty popular in that area. Let's see. Another one that maybe is standout would be programming tools.

09:20 Actually, that's pretty interesting and so on. Yeah. What do you guys think of this?

09:24 I think it's weird that there's 39% of the C++ developers are developing websites. What the heck?

09:29 Yeah. Yeah. What are they doing back there? Maybe the backend. Yeah. Should we both in the middle? I don't

09:36 know. But it's weird. Yeah. Yeah. Yeah. That is quite interesting. And then you get the standard

09:41 business intelligence. It makes sense. Yeah. The business intelligence one, that one,

09:45 Python is definitely crushing it there, right? It's like 30% versus 10, 15, 20% for the others.

09:51 Yeah. I guess one more, there's all these different things you all can dive into. I guess one more area

09:57 that might be worth interesting is they broke down like the type of coding and software activities you do

10:02 based on gender. So for example, if you're male, like how likely are you to do straight programming

10:09 versus testing versus user experience type stuff or versus female? And let's see. So there were some

10:15 takeaways. It says women are more likely than men to be involved in data analysis, machine learning,

10:20 UI design, and research, but less likely to be in directly doing infrastructure development or DevOps.

10:27 But I mean, I kind of had that sense as well. But just, I mean, my personal experience is completely the

10:33 opposite. So most of the DevOps people I work with are women, but I think it kind of makes sense. I mean,

10:41 in the industry for what I'm seeing. But for mine, it's completely the opposite.

10:45 Interesting. Yeah. So I'll leave this out here for people to go dive into and explore more. I feel

10:50 like I've gone probably over enough details there to give you all a sense, but there are some

10:54 interesting things to be learned, I think.

10:56 Yeah, definitely.

10:57 Very cool.

10:57 Yeah. And Matt out there in the live stream points out that that might be more than 100%. I'm not sure

11:02 which part you're talking about. I do believe a lot of these had multiple, you could check

11:06 multiple things. Like which languages am I willing to adopt? Well, I might be adopting both SQL and

11:11 Python in the next year. Something like that.

11:13 Yeah. And I think a lot of people are perpetually going to start learning Rust or Go, but never

11:20 starting.

11:20 That's true.

11:22 It's only 12 months out.

11:23 It's not much.

11:25 All right. Cornell. So this was suggested by Yale Mintz. And Michael, you know where Cornell comes

11:34 from apparently? I'm thinking Soundgarden. Some good 90s grunge. I mean.

11:40 Okay. Maybe. So Cornell is a record and replay mock server. And we're going to link to the tool,

11:50 but also there's an introduction blog post about it. And it supposedly makes it really simple to record and

11:58 replay features to perform end-to-end testing in an isolated test environment.

12:03 So the kind of the gist around it, there's a cool tool called VCRPy, which saves cassette

12:10 files for you. You send it requests and you get replies back and then you can save those request

12:17 reply sessions and stuff. And this is a bundling VCRPy with Flask to make a little server. And it's actually

12:25 really kind of a cool idea. The idea, one of the things around it is that you can do, you're not just

12:32 mocking one service. You can just mock in any external service that you're dealing with. It'll, you know,

12:37 do replays on that. And one of the benefits over rolling your own mocks or rolling your own test server or test

12:44 service is that you can, that you don't really have to think about designing the whole thing. It just kind of replays everything.

12:52 Yeah, that is cool.

12:53 It looks pretty fun. I haven't played with it yet, but definitely want to.

12:56 Hey, speaking of play with it, click on documentation. I think it is on that page right there on the bottom left.

13:01 Okay. Documentation.

13:02 And then click on the documentation of that page. There you go. And so you have this kind of like

13:08 series of animated GIFs of scene in action. And I think that that's kind of cool, right? Like you can,

13:14 you don't go along and say, oh yeah, here you're recording a bunch of API calls and then the workflow

13:18 of like how you create it. I just want to give a shout out to the animated GIFs for like, is this

13:23 interesting to me? Let me just watch the GIFs instead of actually take the time to try to adopt it.

13:28 It's simple, but it seems really effective. OnePay, what do you think?

13:31 It's a good idea. No, it's a really good idea. I mean, there are many projects that you think this

13:36 might be cool to work with. And then you start reading walls of texts and halfway through, I don't

13:41 know if it's interesting, but I mean, it's been half an hour. So having a bunch of GIFs is eye-catching.

13:47 Yeah.

13:48 Yeah, for sure. For sure.

13:49 Yeah.

13:49 Yeah. Also, I just want to quick shout out to the live stream. German points out from his experience,

13:55 the data analysis have more men than women, but it could be biased due to the tech sector

13:59 having more men in general. I do think that that's true. I think what they said is, if you're a woman,

14:02 what are you more likely to be doing? If you're a man, what are you more likely to be doing?

14:06 And it's like, of that population, what areas do you work in? Not that user experience has more men

14:13 or women. I don't think it addresses that question. I think my thoughts here, there's a lot of women who

14:19 end up in programming, not down the traditional computer science path. They go into biology and

14:25 then they're like, oh, I've actually learned a little Python and now I really like it. And I work here and

14:29 it's amazing. But they kind of get pulled in tangentially where there's a lot of guys that

14:34 like sign up for computer science and they just go through that path. And some of the areas that were

14:38 called out are more likely to take the straight computer science path people rather than the,

14:43 I got interested and I came in through like psychology or something else where there would be more women.

14:49 So I don't know. I would love to have more women in there, but I think that this is my, in broad, broadly

14:53 speaking, but I think this is my, my thoughts about why maybe those different areas seem to attract people

15:00 not so directly down the computer science path. Anyway. Yeah. All right.

15:03 one Bay, you're up. You talk to us about the next thing you got here.

15:07 Sure. so I want to talk about factory boy. I think it's a very well known library, to

15:15 basically mock objects, when you're running tests and both this and the next tool I'm going to

15:21 talk about came because I, we were working on a system that replicates and it's an entire Salesforce org.

15:28 So we have a infrastructure we've built that takes everything you have, every object you have in Salesforce

15:34 and copy it to a database. This is a way we have to have daily snapshot of the data that we can do

15:39 a time series and analysis and all the models that we have on it, instead of being furious,

15:44 let's say when you modify it, it's lost. So for this, we obviously need to communicate a lot with the API

15:50 in Salesforce. And when you get, API responses, there you need to not only treat the Jason,

15:59 plainly say there's the pain Jason object, but you will like also to have some kind of object

16:04 representation. And for this, I think it's not news for anyone. The Pydantic right now is, taking

16:10 the floor. and, the biggest issue came, when we need to start writing tests for it, because,

16:17 we get the Jason file, we stick it in the Pydantic object, it validates everything.

16:23 Everything's beautiful and works fine. But then we have a bunch of objects, a bunch of fields on the

16:27 object that cannot be nulled for instance, or they are not optional. So they need to come in the API

16:32 and we need to validate for those because if the API does not return any of those, it should break and

16:36 tell us, look, this is wrong. It's not what you expected. So when we wanted to, write tests for

16:42 those, and we wanted to create objects for those, in each test, we noticed that out of hundreds of

16:48 fields, we might need to feel, I don't know, probably 80, 90 of them because they were,

16:53 mandatory and it started to be very tedious. And I remember I opened an issue in the Pydantic. I say,

16:59 hey, have you, thought about probably allowing creating an object with random fields that validate

17:06 properly? Like this feels an integer between 10 and 20. So I just don't want to feel it. I don't want to

17:11 feel any of those because I don't care for this test. Is there a way that I can say, okay, just feel

17:16 whatever it validates. And they say, no, it's out of the scope of Pydantic, which also makes sense.

17:20 I just wanted to ask in case. and they said that probably in factory boy, they might be

17:25 interested in this. So I went to factory boy and I read the documentation. it was pretty cool

17:29 because it allows you to create, you define an inside class. So it's a meta class, not a meta

17:35 class in the terms of, Python meta classes, but it's a class called meta within the, the factory

17:42 that you want. It's weird because everything you say, yeah, this is the meta class. Why do

17:46 meta class? No, it's a class. So you inherit from factory. Then you define a class called meta meta

17:52 where you define what is your model. So this is the object I want to mock with this factory. And then

17:57 you can define many, fields with their default values. The cool thing about this is that it implements

18:04 also faker. So you can say, if I have a username, I don't want to fill it. Just give me an username.

18:08 A faker will give you. Yeah. Faker is really cool for generating stuff like that. Yeah. Yeah.

18:13 And the amount of plugins you find for faker is, outstanding. So you can fake almost anything

18:18 you think of. So the, the cool thing about this is that it's not only, you plug in the class

18:24 that you have and it will fill it, but you also can work with over M's. Like you can use

18:27 C-blocking over M's or Django over M's and it will generate an object for this over M based

18:35 on whatever you set. These are the default values. So I thought it would be great if I could do this

18:41 also for Pydantic. So I could just say, okay, these are the mandatory field that put something

18:45 faker can think about it and then we're ready to go. But reading documentation, it didn't appear

18:50 anywhere and I thought, hmm, maybe I cannot use it for this case. So I went ahead and opened an issue and

18:55 say, are you thinking about putting this also to work with Pydantic? I mean, it's now is,

18:59 booming and everyone is using it. And if you are reading JSON from an API, it's very likely that you

19:04 have hundreds of fields you don't care about. You might want to fill it with whatever. And I remember

19:08 the author said, I didn't know this. I didn't know Pydantics. Cool. You mentioned it, but

19:14 internally what FactoryGo is doing is creating a dictionary with the parameters you want to fill in

19:19 and just unpacking it in the construction of the class. have you tried it? And I was like,

19:23 no, I have not tried it. And when I tried it worked. So it works perfectly. I mean,

19:28 maybe there are some quirks of Pydantic that it cannot cover. But, if you're using Pydantic to

19:34 store your data from API calls and so on from JSON and validates and so on, FactoryGo is pretty

19:40 cool. I mean, the amount of things you can do with this, you create a, you can create many

19:44 factories for the same class. So you can create fixtures. Like, I don't know if you want to mock a

19:48 user. You can have an admin or a buyer or whatever, and, then you can just define

19:53 different factories and it will give you the usage you've defined. And, it's also pretty cool

19:57 because the faker is randomized, beneath it. So if there is, there are parts of your object that

20:05 your code does not care about, it's also a good test to have those parts being random because,

20:11 if it really doesn't care, you don't care what those fields are. And then at some point

20:15 your test fail, it happens once it means that you actually can just fix something.

20:20 Yeah, absolutely. I did see that if you need repeatable tests, but you want faker to generate

20:25 random stuff, there's a way to seed faker. Exactly.

20:28 Generate the random values, but do it in a consistent way. And one way you might want that is if you have an

20:33 edge case or some value that breaks the test, and then you want to put a break point and press debug

20:38 and go through it again. But like, you know, how are you going to get it to hit that case again in a

20:43 predictable way? Right. So if you, if you trigger, if you tell it to say, always do the same thing,

20:48 but randomly, you know, you'll make it so you can go back and look at it a second time and figure out

20:52 what's up. Yeah. You can fix that. Sometimes it's also good to have them fixed, even if you don't care.

20:58 I mean, you need to have a daytime and for some reason you need to have the daytime being whatever and

21:02 whatever, but you can validate for it. So you can just, or either set it or ensure that it's fixed.

21:08 Yeah. There are many use cases that you can exploit that thing. And it's actually really cool.

21:12 Yeah. Usually I almost always seed faker because I just, I don't, I don't, I'm not using it because

21:19 I want the randomness. So I'm using it because I don't want to come up with the data.

21:22 Yeah, exactly. So make it so that it does the same thing every time, just gives you the random

21:27 day that you want. That's right. Agreed. Very, very cool. All right. The next one up actually is

21:32 pretty sort of related to that. It's called PI instrument. Have either of you heard of PI instrument?

21:38 Not until now. When I read the notes and it sounds pretty cool.

21:41 Yeah. Right.

21:42 Yeah.

21:42 No, I haven't.

21:43 Yeah. So it's a call stack profiler for Python, which is pretty cool, right? It's just going to tell

21:49 you where your code is slow, but it's, you know, it's, it looks really clean. And when you look at

21:54 the output, it can actually give you the results in the terminal. So if you want to see, you know,

22:01 like you run this thing instead of saying Python, my exam, my Python.py file, you would just say

22:08 PI instrument, that same file and it'll run it. But then at the end, it's going to generate a whole

22:13 bunch of things about how long it took and whatnot. And then you actually get like colored output in the

22:19 terminal showing which lines of code are spending how much time in different places. And it seems like it's a

22:25 real good way to just sort of quickly dive in on where you're spending your time.

22:29 Yeah. I'm definitely going to try this. It's cool. Yeah. Yeah. One thing I like about it is the simplicity of

22:33 like pip install, PI instrument, PI instrument, your file. That, that'll give you the answer, right? Like

22:38 That, that for me, solved it. I mean, every time you want to do some profiling, you spend some time

22:44 tweaking things. So you get what you want. The fact that this is just running with PI instrument,

22:49 whatever script you want. I mean, I'm going to try for sure.

22:52 Yeah. Yeah, for sure. And when you do profiling, you end up in this sort of quantum mechanics

22:57 world of if you observe a thing, you've changed it. Yeah.

23:01 And so there might be code that is like this half is 50, 50, and this half is 50, 50 at the time.

23:07 But one is calling an external system once and the other is a tight loop. And if you profile that with

23:12 instrumentation, it's going to wreck it. It's going to make the loop look way slower because

23:18 now you've added a bunch of overhead each step where there's very little overhead to this external

23:22 service sort of thing. And this one uses sampling and the sampling doesn't really have that effect.

23:28 It just every so often, every millisecond or something that says, what are you doing now?

23:32 What are you doing now? Who called you? What are you doing now? Right.

23:35 And so it's more of a polling sort of thing rather than slowing down line by line code. So that's

23:43 probably worth doing as well. Yeah. That's pretty cool.

23:46 Yeah. It looks like you can specifically jump in and just do a section of your code that you care

23:51 about also. Exactly. If you want to say, you know, so one of the things that I hate about profiling is

23:57 it'll say 87% of your time was in the startup code and the imports. You're like, yeah, okay. That

24:03 that's not relevant to me. What I want to know is the part that I'm actually trying to test here.

24:07 How long did I spend there? And please don't pollute that with other junk about like starting up Python

24:14 or loading modules or whatever. Right. And so you can, there's an API and you can say from

24:20 PI instrument import pro profiler, and then you can do a context block in there and run it. And just

24:27 that code will tell you like how long it takes. Cool. Does anything else jump out there out at you,

24:32 Brian? And like with this example, I got on the screen here, that would be hard. It's an async example

24:37 for one. Yeah. As an async and a weight. And so they recently released PI instrument four,

24:43 which will actually give you the information about the async code as well. Right. So it'll,

24:51 let's see what it says. It has async support. PI instrument now detects when an async task hits

24:55 an await and tracks the time spent outside of the async context under the await. Whereas before it would

25:01 basically just profile the asyncio event loop or something silly like that. Right. So if you're trying

25:07 to profile async and await in asyncio, this might be your best option because it specifically supports

25:13 that. That's good. So what happened before if you use a different profile? It would say, yeah,

25:20 it says you only see the time spent in the run loop and it'll basically tell you like, here you see like

25:26 run once and the select, and then the queue control built in. It's just like, there's this asyncio event

25:32 loop that's cranking around waiting for the signal for the IO to be done. And it just says, well,

25:36 you're waiting on this. You're in the loop. You know what I mean? Yeah. Yeah. Yeah. Yeah. So,

25:41 yeah. Very cool.

25:42 And now you get a little bit better. Like it says, okay, you're awaiting on this line of your code for

25:47 a second or whatever it is. Yeah. There's also, I'll shout out a few more things here. Is it in this

25:51 stock? Yeah. So they also, there's also a bunch of simple authentication they did previously about

25:57 network calls and stuff. And there's, there's an interactive view JS app that you can get with flame graphs.

26:04 So instead of looking at the terminal, you can look at it in your web browser and explore into

26:09 those. Yeah. There's a lot of neat little things here pulled out of the show notes, but it seems like

26:13 a really nice way to do some profiling and you just high instrument your code and you have a look.

26:17 Cool. Yeah. I personally kind of like the default output. I'm, I know that a lot of people like flame

26:22 graphs. Like they, they don't really do much for me. They look like, I don't see the data, but it's cool. It has both.

26:28 Yeah. A couple of things for the live chat. Maddie says, pie instrument is a statistical or

26:33 sampling profiler, which is better prepared for profiling. I think it depends. I mean, I do the

26:39 instrument patient ones do give you more precise information, but it's also skewed with the overhead

26:46 of that information. So it's, it depends, but this is the least influential one for sure. And then

26:52 Avaro says, how would you use pie instrument with an entry point? That's a good question.

26:57 Not knowing the answer off the top of my head, maybe make another Python file that just imports

27:02 your library and calls the entry point and then profile that. But yeah, there's a real quick,

27:08 cheat, you know, just make it call it and then, high instrument that file, but there may be

27:13 some way to say like dash M and give it a module and a, thing to do. So yeah, that's it,

27:19 Brian, that's it for a pie instrument. Cool. well, I just wanted to remind everybody that,

27:23 Python 310 release candidate one came out yesterday. So Pablo announced it, just,

27:30 just on the third, I think, I think it was yesterday. not the fourth today. Yeah. Anyway. so 310 is

27:36 out. if you've got, if you've got, there, well, 310 RC one is out the timelines that we're

27:43 looking at then we're getting excited. It's coming up. So the, September 6th is the plan for RC

27:49 two and then October 4th, we're planned is the plan for the official release. So we're

27:54 just really right around the corner. It's nice. and this is definitely a time I know we've brought

27:58 this up before, but if you maintain any third party Python packages, you probably should have already

28:04 been taste testing it against a 310. But if you haven't definitely do it now to make sure that,

28:11 people that use your stuff don't, it doesn't break when they need to. And then we, in the show notes,

28:16 we put, just a list of just a reminder of some of the new changes in 310. we've,

28:22 definitely talked about some of these before structural pattern structure pattern matching

28:26 is the switch statement kind of thing. And then, yeah, lots of these other things we've covered.

28:32 I'm kind of actually, I like the union type union types. So you can like, because there's a lot of

28:38 stuff that I write that the default is none, but the normal type is something else. So you can really

28:44 easily say the type is none or it or something like that. and that's a lot cleaner than you

28:50 used than, than before. I'm, I've already started using 310 to test everything that I support.

28:56 So hope everybody else has as well.

28:58 Yeah, cool. I like the optional length checking and zip, right? Zip taking two collections and you

29:03 want to pair up the items. Like if those things don't match, that should be a problem. Also like the

29:08 or for the types, information. And I think dick and some of those types are now don't require a

29:14 from typing imports. Oh, right. Yeah. yeah, I don't think, I don't see it called out here, but

29:19 one of the problem was, maybe that's explicit type aliasis. I'm not entirely sure, but if you want

29:25 to say this type is a dictionary of strings and, integers, you would have to say from typing

29:31 import capital D dict and then dict square bracket, string comma, and whereas now you can just use

29:37 the lowercase D I C T and you don't have to have that import and you can do that sort of thing to it.

29:43 So I'm looking forward to that. Yeah. A lot of the, a lot of, with this, a lot of the common,

29:47 type type hints, you won't have to do the import anymore. And that's, that's great. So I think that's

29:53 really all I was using the import for was things like dict and set things like that.

29:58 Yeah, exactly. Didn't that, I mean, I seem to remember that 3.10 was the one that was including

30:04 these, built in types without having to import from typing. Didn't that update might break some of

30:10 the libraries that is typing. Like Pydantic and FastAPI. The, the thing that that was, was to use it in,

30:19 basically use it as a string and not actually evaluate the type. I think that like, so if you had your own

30:25 type, your own Pydantic type that was a customer, I think you could put customer, but it wouldn't be

30:29 actually evaluated until a type checker hit it or something like that. Like a forward typing.

30:35 Yeah. Yeah, exactly. So this, this ability to specify the type on like lowercase d dict

30:41 is related, but it's not the same. And I'm pretty sure that that, that fear around Pydantic is not

30:47 in 3.10. Yeah. It either got postponed or rolled back or modified. Yeah. Yeah.

30:52 I just want to talk about the one that says, what was the number six to six. Do you have it?

31:02 yeah. The precise line numbers for debugging and other tools.

31:06 It's one of it's, I think it's very underrated. It's going to be one of those things that when

31:12 people get used to, it's like, I don't know how you live without this.

31:15 Oh yeah. Yeah. There's not a good example shown right off the bat, but it is, it's pretty cool.

31:21 Yeah. Yeah. Yeah, absolutely. Very cool. And then we also have better stack trace message error messages,

31:25 right? Yeah. Yeah. Those are coming. A lot of good things to look forward to. All right.

31:29 One pay you got the last item. Great. I think it's time for it. You want to take us out, right?

31:34 Sure. Yeah. So let's talk about time machine. I said, we were building this, tool that

31:42 copies and entire self-assort, one of the things that we need to work straight everything is to

31:47 timestamp almost every action we do. this means that in many places all over the code, we have a

31:54 daytime UTC now method call. And, when we are testing the, we need to be able to mock it.

32:00 And if you've tried to patch daytime UTC now with a usual patch method, you know, it works, you can do

32:07 it, but, you need to do it with a patch and then you pass the string, but, the module where this

32:13 UTC now call is, and then you're good to go. But when you have this in many files in the same test,

32:18 you need to patch every one of those because otherwise it wouldn't work. So I tried to use

32:22 patch object and patch daytime and say, okay, I want to patch the, it's in our method, this object.

32:27 And it will of course complain and say, you cannot patch a built-in, built-in type like daytime,

32:34 daytime. So I was looking for how we could patch this. And I found, Frisgan, which is a very

32:41 well-known library thing to patch this kind of thing. But suddenly I noticed that once I started using

32:47 Frisgan, all of my tests took much longer to complete, it's not like, deal breaker.

32:55 So it went for probably five minutes to seven or seven and a half, but I was very surprised because,

33:01 our pipeline or deployment pipeline already take a long time. So every time I can reduce

33:06 it a minute, it's good for me. And when I saw it going up two minutes, I was surprised why, why is this

33:11 happening? And then I learned that what Frisgan is doing is actually scanning all your dependencies and

33:16 making patch for every call you make to the methods of data. and then, trying to see if there

33:23 was something else I found out, time machine, time machine is a very, cool, not so

33:30 well-known, I think, library that allows you to do basically the same that Frisgan allows you to do.

33:37 So you can just, patch, any, almost any method called in daytime or time, with a simple

33:43 decorator in this, in your test. It's also support pytest fixtures that you can use. the good thing about

33:48 this is that it does not scan for imports of date and daytime. And what it does is actually change the

33:55 underlying C-level calls that you make, to get the time. So every time you say, I want to patch any call,

34:04 to be on January 1st of 2019, for instance, it will just call it normally, but the C, the underlying C calls

34:11 that will be made will return this time instead of the other ones. You don't need to scan everything to patch it.

34:16 another thing that says that I thought it was pretty cool is this, you can let the time

34:21 tick after you patched it. So you say, this is for February 1st of 2018. And once you enter the mock,

34:28 either with, a decorator or with a context manager, you can also use like standard patch,

34:34 call, then time start passing, starting on that, time that you mocked, mocked it for.

34:41 so you can do a perf counters and all this thing normally, but if you need to stay in a given

34:45 day for a test, you can do it. So I thought it was pretty cool. It solved my two extra minutes running

34:51 because we have many places and many files in the project where we use it. See now. And, it was

34:56 pretty well. So this, this had, this must've had incremental. I mean, it has a little bit of time

35:01 that has to do its work, but it's, it's fast enough that you're not noticing it then.

35:05 Yeah. I'm not noticing anything. I mean, it runs more or less the same.

35:08 Okay. Well, I mean, I imagine there should be some delay, but it's not as noticeable as,

35:13 what happened with Friskin. Yeah.

35:15 Cause it took them, it took some time. I mean, I'm glad you brought this up. This is cool.

35:19 Yeah. We have a bunch of tests. Yeah, exactly. I say, Brian, you probably, this is kind of in your

35:24 world, right? Like dealing with time as a dependency. Definitely. And there's, there's, there's,

35:29 sometimes it's really, you want it fixed because you really want fixed answers. Cause like you've

35:34 timestamps and stuff are in your data. You're going to have to, I mean, it's good to compare against

35:39 known oracles, but there's all the also times where you, you, and this is where Friskin

35:45 isn't so bad is, but maybe this would be really useful too, is, is if you want to test certain

35:52 things, there's weird quirky dates. You want to make sure that your software deals with certain times,

35:56 fine. Does it work fine when it's running over like, overnight on December 31st to January

36:02 1st, things like that, when the year changes and things like that. exactly. Yeah. Yeah.

36:08 Yeah. You always want to test your boundary conditions, right. And crossing over time

36:12 or weird cases like March 29th and stuff like that. You're like, let me just try that and see if this

36:17 is going to survive. Yeah. Yeah. But then, I mean, to, to, to be fair, I think most of the time,

36:22 things, things like this are used are, like was brought up is that the time shows up in the data.

36:29 So in order to compare the, you know, or the log or something in order to compare those apples to apples,

36:34 it's nice to have the same dates there. I can't, I can't tell you how many times I've had,

36:39 had to compare two log files and strip out the times, because those are the old,

36:43 like those are the, every line's different because the timestamp's different. So yeah.

36:48 Yeah. Very cool. Nice find.

36:49 Right.

36:50 That's all for time machine. Yeah.

36:52 Yeah. Super. well that's our, that's our six items. everybody, have you got

36:58 anything extra, Michael?

36:58 Well, I have the old thing that is new again. Let's see. I have some bandits. So, the,

37:05 the drama around supply chain vulnerabilities and open source repositories goes on. So this,

37:12 this one, I think actually the other article I'm going to talk about, comes to us from Joe

37:17 Riley. Thank you, Joe, for sending that in. But basically there's some more malicious things

37:23 in by PI again, and people just remind everyone to, to be careful and be whitelist off or yeah,

37:30 this one, I don't know what this one was. If it was typo squatting this time around,

37:35 or it was just something else that got put up there. Yeah. There's one is one headline is credit

37:40 card stealing malware found in official Python repository. And the other one is the same one

37:45 about ours technical article says software downloaded 30,000 times from PI PI ransacks

37:50 developer machines, developers machines. Expect to see more of these Frankenstein type things.

37:55 Cause it's a, basically a systemic threat. Like how, how does it get dealt with? Right. I'm not

38:00 sure if they list out. Yeah. So they used, they did interesting stuff as well. Like they did simple

38:05 obfuscation of the code that was being run. So you couldn't look at it and say, look for a credit

38:10 card or look for a Bitcoin wallet, and then go do your evil deeds in Python source code. So they would

38:15 do things like, base 64 and code the Python code, and then just in memory decode it, then run it.

38:23 So they were trying to get around things like that. So anyway, people can check that out. And it's not

38:28 ideal, but just a reminder to be aware. Yeah. Yeah. Yeah. Yeah. Yeah. This is why we can't have nice

38:35 things. Come on people. Why we can't have nice things. Well, I get a couple of things I wanted

38:39 to bring up just things I've been up to. just released episode one 62 of, testing code. And

38:46 I kind of run through, all the different flavors of test-driven development that I know of.

38:52 so there are quite a few versions. So check it out if you're interested in test-driven development.

38:57 And then, I'm just, working on wrapping up the talks and continuous integration chapter for

39:02 the second edition of the pie test. It'll be coming out hopefully within a week.

39:06 Very cool. Good to see you making progress there.

39:08 do you have anything extra, Juanpe?

39:10 Nope.

39:11 Nope.

39:11 Okay.

39:12 I mean, I'm very happy to be here.

39:14 Let's go to a joke.

39:15 Yeah. It's good to have you here. All right. Let's go to a joke.

39:18 So this one's a visual. If, if you're, if you're listening, you're going to have to go to the,

39:23 scroll down to your podcast show notes at the bottom. Just click on the joke

39:27 link. One of the things you guys, I like about Python is there's a lot of stability

39:32 in the code that we write. You know, if I wrote something on Flask five years ago,

39:36 chances are it'll still run, right? If I write my Python code now, it's probably still going to run.

39:40 Yeah. There's new things. There's new shiny visualization frameworks and stuff,

39:43 but generally it's pretty stable. You know, what is the opposite of that?

39:46 JavaScript. So, so here's a little animation saying, and it says JavaScript developer bouncing

39:52 from framework to framework. And it's this incredible, like almost people are awesome type

39:57 of thing where somebody set up, you know, 50 workout balls on a running track, the whole

40:03 straight of a quarter mile running track. And somebody jumps on it and just like glides

40:09 from one to the next. What do you all think?

40:11 The fact that he's able to do this is surprising.

40:14 It's really impressive that he pulls it off. And it's on one of these like sandy, gritty running

40:21 tracks. It's going to hurt like crazy if he misses it. So maybe there's the motivation, you know?

40:25 Yeah.

40:26 So I remember the Jambrain report you said before I was thinking, I didn't say anything,

40:32 but I didn't want to mean, but you were saying, how likely are you to change languages? And it was

40:36 like, well, JavaScript, they're going to change a lot. Then I thought, oh, their language is not frameworks.

40:41 Yeah, exactly. How likely are you to change your framework? Well, that's like,

40:45 like nearly a hundred percent.

40:47 Yeah, that's true. I mean, people stick around like, you got Django developers have been doing it for years.

40:55 Yeah. 10 years. And they're more excited today than ever about it. Right. They're not like,

40:59 we're ditching this. Yeah. All right. Well, that's, does that count? Does that count as a joke?

41:04 Yeah. Oh, I laughed.

41:05 All right. Perfect. Well, that's what I brought for you all.

41:09 Well, thanks to everyone for showing up and, had a fun day today. Hope everybody else did.

41:15 Thanks a lot for me here.

41:16 Thanks, Brian. And thanks for being with us.

41:17 Bye. Bye. Bye.

41:19 Bye. Thanks for listening to Python Bytes. Follow the show on Twitter via @pythonbytes.

41:23 That's Python Bytes as in B-Y-T-E-S. Get the full show notes over at pythonbytes.fm.

41:29 If you have a news item we should cover, just visit pythonbytes.fm and click submit in the nav

41:34 bar. We're always on the lookout for sharing something cool. If you want to join us for the

41:38 live recording, just visit the website and click live stream to get notified of when our next episode

41:43 goes live. That's usually happening at noon Pacific on Wednesdays over at YouTube. On behalf of myself and

41:49 Brian Okken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your

41:54 friends and colleagues.