WEBVTT 00:00:00.001 --> 00:00:04.220 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to 00:00:04.220 --> 00:00:09.840 your earbuds. This is episode 236, recorded June 2nd, 2021. I'm Michael Kennedy. 00:00:09.840 --> 00:00:11.000 And I'm Brian Okken. 00:00:11.000 --> 00:00:12.600 And I'm Anastasia Timosuk. 00:00:12.600 --> 00:00:17.060 Hey, Anastasia. So great to have you here. Nice to have you on the show. 00:00:17.060 --> 00:00:18.120 Thank you for inviting. 00:00:18.120 --> 00:00:21.880 Yeah, absolutely. Why don't you tell people a little bit about yourself before we get into 00:00:21.880 --> 00:00:22.260 the topics? 00:00:22.260 --> 00:00:28.920 So I'm joining from Germany, Berlin remotely right now. And I have a little one, a baby 00:00:28.920 --> 00:00:35.300 doc joining as well. You might hear him on the stream. I am originally from Ukraine. I'm not 00:00:35.300 --> 00:00:41.760 German. I moved to Germany around five years ago, maybe five and a half. And my passion is Python. 00:00:41.760 --> 00:00:47.460 I used to be a C++ developer, game developer, and so many more languages. But the best one, 00:00:47.460 --> 00:00:52.840 I think, for me is Python. So I decided to stick with it for around eight years now. 00:00:52.840 --> 00:00:57.840 Oh, how cool. I started out doing my professional programming in C++. And I 00:00:57.840 --> 00:01:01.660 I know Brian still touches a little bit of C and C++ in his world. So that's cool. 00:01:01.660 --> 00:01:03.380 Yeah, it's half my life. 00:01:03.380 --> 00:01:06.000 Nice. And what kind of games? 00:01:06.000 --> 00:01:15.220 Well, they were adapted first for iPad. They were like two and a half D games. And then later on, 00:01:15.220 --> 00:01:19.080 it was mostly 3D games with Unreal Engine. 00:01:19.220 --> 00:01:25.480 Oh, cool. Yeah, that's awesome. All right. Well, once again, welcome. Welcome. So glad to have you 00:01:25.480 --> 00:01:30.320 here. Brian, do I have the first item this time around? No, you do. Go for it. Okay. What do you got 00:01:30.320 --> 00:01:35.920 for us? Well, accessibility isn't really something I probably should think about accessibility more, 00:01:36.040 --> 00:01:42.640 but I don't really. But I probably should. So I was excited to see there was a tweet recently by Matthew 00:01:42.640 --> 00:01:50.680 Feikert that said, I need to give some serious praise to a fellow scikit hep dev, Hans Deminski, 00:01:50.680 --> 00:01:57.800 on his excellent monolens tool for interactive simulations of color blindness. So I checked this 00:01:57.800 --> 00:02:04.680 out. So monolens is this is a Python package and you can pip install it. And as Matthew said, 00:02:04.680 --> 00:02:12.300 you can pip x install it. So you just always have it around, which is nice. And it just pops up this 00:02:12.300 --> 00:02:21.420 tool, this, this really cool window. And you can just, you just drag it around. And it makes the 00:02:21.420 --> 00:02:27.100 whatever the windows over all over your desktop, it just makes it black and white instead of color. 00:02:27.280 --> 00:02:34.300 So you can see what it looks like in grayscale. So I, one of the things I really liked about this is, 00:02:34.300 --> 00:02:43.040 is the example showing it with, with Matt plotlib and plots, because plots are really where 00:02:43.040 --> 00:02:50.300 you're using color to distinguish between the two different sets of data. So you really kind of want 00:02:50.300 --> 00:02:55.780 that data to look different, even if people don't see color. So that's, that's an important thing. 00:02:56.140 --> 00:03:03.800 So that was neat. And then somebody that replied to that and said, Hey, I always try to use C, 00:03:03.800 --> 00:03:11.740 C masher smasher. I'm not sure. It is a, to make sure they're colorblind friendly. So I'm like, 00:03:11.740 --> 00:03:19.280 I've never heard of this. So I went and checked out smashers. And what it, what it is, is it's a bunch of 00:03:19.280 --> 00:03:25.400 color maps. So you don't really have to think about it. So you, so there's all these great named color 00:03:25.400 --> 00:03:33.340 maps and they're, they're actually fairly attractive color changes, but the, it shows you what they look 00:03:33.340 --> 00:03:39.420 like in black and white also. So they, this isn't, it's kind of a little demo what the top 00:03:39.420 --> 00:03:44.840 that we're looking at on the stream, but the code that you have to, you just, it's just kind of built 00:03:44.840 --> 00:03:50.220 into Matt plotlib already. Like it's an extent, it's also kind of an extension to Matt plotlib and other 00:03:50.220 --> 00:03:55.000 things that use color maps. So you can just say, when you're plotting, you can just specify a color 00:03:55.000 --> 00:04:01.820 map like rainforest or something. And it, it automatically is a colorblind, friendly 00:04:01.820 --> 00:04:07.120 color map. So you can do your plots and have it still look nice everywhere. So. 00:04:07.120 --> 00:04:13.460 Oh yeah. This is really cool. And Matthew friend of the show. Thanks for sending that in. I never 00:04:13.460 --> 00:04:17.940 really thought about this and I should have, you know, I mean, I feel like maybe I should go over my 00:04:17.940 --> 00:04:24.960 websites and go, do they look terrible for people who have, you know, color vision impairments? 00:04:24.960 --> 00:04:30.300 Yeah. So really cool. And it looks like it's this independent thing that will just go over. 00:04:30.300 --> 00:04:34.140 You just move your mouse around. It works on anything. It doesn't necessarily have to do 00:04:34.140 --> 00:04:38.980 with Jupyter or Matt plotlib or something like that. Right. Right. So the model lens is just a, 00:04:38.980 --> 00:04:43.260 it's just something that works on anything. I could, I drug it over even my desktop, my background, 00:04:43.260 --> 00:04:48.700 uh, and it showed, showed the picture in black and white. So, it is cool. 00:04:48.700 --> 00:04:54.760 The other thing is, wait, there's a color maps. I can just add to, Matt plotlib. That's cool. 00:04:54.760 --> 00:04:59.440 Like rainbows and stuff. How neat. I didn't know you could just do that. So that's a, 00:04:59.440 --> 00:05:03.740 that's kind of a neat thing. And then you can like, for instance, the, the, one of the examples 00:05:03.740 --> 00:05:09.820 that they have on the CMASH or read me, is just, just sort of a simple plot. And when 00:05:09.820 --> 00:05:16.240 you're in Matt plotlib kind of just picks colors for you, unless you specify colors for different 00:05:16.240 --> 00:05:22.480 plot lines. but you can just, you can give it a color map instead of a, a specific, 00:05:22.480 --> 00:05:27.980 uh, list for each item. so, and that just kind of nice. 00:05:27.980 --> 00:05:31.640 That's super neat. Yeah. Why not do it? Anastasia, what do you think? 00:05:31.640 --> 00:05:34.900 Oh, it looks amazing really. And it's super helpful. 00:05:34.900 --> 00:05:37.740 Yeah. when you were doing the video games, 00:05:37.740 --> 00:05:40.120 but that would be great to use it as well. 00:05:40.120 --> 00:05:43.180 For sure. When you were doing games, did you have to think about this kind of stuff? 00:05:43.180 --> 00:05:49.420 No, actually we were not that far at that time. It was around seven years ago, eight. 00:05:49.780 --> 00:05:56.640 Yeah. So, yeah. The, on the, on the monolens site, one of the examples they show is using, 00:05:56.640 --> 00:06:03.060 um, having one of the plots use some sort of pattern underneath and not just color. And that's, 00:06:03.060 --> 00:06:07.260 that's, I'm not sure how to do that. So people that are great at Matt plotlib probably know how 00:06:07.260 --> 00:06:11.760 to do that really right away. But that's kind of a neat idea also to have like one of the, 00:06:11.760 --> 00:06:16.520 one of the graphs has hashes versus stars or slant lines or something like that. 00:06:16.520 --> 00:06:20.380 Oh yeah. I have it like some sort of ASCII differentiator. Yeah. 00:06:20.380 --> 00:06:20.720 Yeah. 00:06:20.720 --> 00:06:27.620 Very nice. Yeah. This is super helpful. And Matthew again, thanks for sending in and, joy. Yeah. 00:06:27.620 --> 00:06:30.440 Welcome to the live stream. Thanks for, for being here for the recording. 00:06:30.440 --> 00:06:35.920 So the next one I want to talk about is something called rapid fuzz. 00:06:35.920 --> 00:06:41.900 Yeah. So last time I talked, when we had Vincent on, I saw the fuzzy, 00:06:41.900 --> 00:06:47.420 fuzzy, fuzzy, fuzzy text matching for that, that chat bot that he was showing off. I thought, 00:06:47.420 --> 00:06:55.560 Oh, fuzzy, fuzzy is cool. So Mikel Honkala sent in a rapid fuzz and it's very much like fuzzy, 00:06:55.560 --> 00:07:01.380 fuzzy, but it turns out to be a whole lot faster and it uses some of the same ideas, but, you know, 00:07:01.380 --> 00:07:06.220 coming back to the, some of the things we were talking about, it is basically written in C++ using 00:07:06.220 --> 00:07:12.600 the Levenstein distance algorithm for words similarities, but obviously has a Python 00:07:12.600 --> 00:07:18.120 API that we all work with. And so, yeah, it's pretty neat. It's really easy to work with. You just, 00:07:18.120 --> 00:07:21.520 again, pip install it. And then you can come down here and do things like 00:07:21.520 --> 00:07:28.900 fuzz.ratio and you can give it two sentences. This is a test or this is a test exclamation mark. And it 00:07:28.900 --> 00:07:36.700 says that's 96.5% the same or, you have fuzzy, fuzzy was a bear. I guess these are, yeah. Fuzzy, 00:07:36.700 --> 00:07:41.500 fuzzy was a bear. I guess those are the same. No, was he fuzzy. Oh, was he fuzzy. Yeah. I got it. 00:07:41.500 --> 00:07:45.940 I got to read better. Was he fuzzy was a bear versus fuzzy. Was he was a bear? Oh my goodness. 00:07:45.940 --> 00:07:53.020 that's 90% the same. Given a bunch of, phrases, you can sort them by similarity. You can say, 00:07:53.020 --> 00:07:58.820 kind of use selection, like, you know, to call in sort of call center type of automation, 00:07:58.820 --> 00:08:05.380 given three choices and given some text, you can say, find which one, you know, like Atlanta Falcons, 00:08:05.380 --> 00:08:10.420 New York Jets, New York Giants, and so on. Somebody says, you know, lowercase New York Jets instead of 00:08:10.420 --> 00:08:15.280 uppercase, it'll say, well, here's the likelihood that that's a match, but here's another possible 00:08:15.280 --> 00:08:19.840 match that's, you know, and it gives you the ratios of how good of a match it is. So if you've got a 00:08:19.840 --> 00:08:24.580 select set of choices and you're asking for input on it, you can just say, well, give me the closest 00:08:24.580 --> 00:08:29.000 match. And if it's anywhere close, you can just run with that. So yeah, pretty neat, right? 00:08:29.000 --> 00:08:36.080 That is pretty cool. Yeah. And the other thing that's interesting is the performance. And before 00:08:36.080 --> 00:08:40.340 people tell me that all, all benchmarks are broken and they don't work, you know, sometimes at least 00:08:40.340 --> 00:08:45.620 they give you a sense. So here's some of the things that, they've got in terms of performance, 00:08:45.620 --> 00:08:50.740 save versus fuzzy wuzzy and the numbers are like 10 or 20 times faster. Definitely broken. 00:08:50.740 --> 00:08:56.800 It's definitely broken. I think it's because it's written in C++ instead of Python, at most of 00:08:56.800 --> 00:09:02.820 its core, you know, probably. But anyway, if you're looking for fuzzy text matching, fuzzy wuzzy is a good 00:09:02.820 --> 00:09:08.320 option. And apparently thanks to Mikko rapid fuzz is as well. So yeah, pretty neat. Yeah. We probably 00:09:08.320 --> 00:09:15.980 should do a segment on benchmarks at some point. No, no, no, no, no, we should do it, but I've 00:09:15.980 --> 00:09:21.020 written blog posts and stuff on it. And it's just an endless battle of you're doing it wrong. Your 00:09:21.020 --> 00:09:26.040 situation is not my situation and my situation. It's not as good or it's worse or it's better or 00:09:26.040 --> 00:09:30.580 you're yeah. No, I hear you. It would be interesting, but at the same time. Yeah. Okay. 00:09:30.580 --> 00:09:35.980 There we go. We just had a section on, benchmarks. Yeah. I've already just explained like the emotional 00:09:35.980 --> 00:09:40.660 trauma that I'll go through from receiving all the feedback now. And it's what do you think about 00:09:40.660 --> 00:09:46.180 this, fuzzy text matching? Well, maybe next time we can organize a battle between them. 00:09:46.180 --> 00:09:51.760 That's right. Yeah. We'll bring some in. Yeah, sure. Do you have any use for this fuzzy text 00:09:51.760 --> 00:09:57.820 matching, string matching stuff? Well, actually, yes, at work, we have, lots of, matching 00:09:57.820 --> 00:10:05.080 algorithms, but we're using, different tools and I'm not a data scientist person, but I would love to 00:10:05.080 --> 00:10:09.760 try that actually. It looks super cool. Yeah. We, we use some C++ libraries. 00:10:09.760 --> 00:10:15.920 Yeah. Yeah. Robert out there in the live stream says we would have to benchmark the episode if we had 00:10:15.920 --> 00:10:20.260 an episode about benchmarking. You see, it's like recursion. Save that thought for the end of the 00:10:20.260 --> 00:10:24.880 show, by the way. All right. And Stacia, you're up next. Structured logging. Tell us about it. 00:10:25.240 --> 00:10:32.920 well, a few years ago, I, went to meet up and I heard a talk from my Marcus Holterman about 00:10:32.920 --> 00:10:38.440 struct log. That's the first time when I heard about this and I decided to give it a try. And actually I 00:10:38.440 --> 00:10:45.760 fell in love with it. and I'm using it, since at least two and a half years, maybe two. 00:10:45.760 --> 00:10:53.180 it's awesome way to bring a bit of structure to your logs to make them more visible and more usable because 00:10:53.180 --> 00:11:01.720 usually how we log, it's like just one huge sentence, which is readable by humans, but it's not machine 00:11:01.720 --> 00:11:09.560 readable. And the idea is here, to bring more structure, to build some dashboards based on, 00:11:09.560 --> 00:11:15.060 different keys and then values and then see what's actually happening with the system without 00:11:15.060 --> 00:11:21.540 touching the logs, without scrolling through the whole log. And then just reading all bunch of 00:11:21.540 --> 00:11:31.500 things. and I already used it in production. It looks pretty well. If, you try using JSON format, 00:11:31.500 --> 00:11:37.620 just fantastic. Oh, how cool. Yeah. You can pass it all these like processors and type stuff. So you can 00:11:37.620 --> 00:11:45.000 say render out the print that, you know, stack info, the log level, all those kinds of things. That's neat. 00:11:45.000 --> 00:11:50.820 We added a bunch of, processors like custom made, which were specifically designed for our 00:11:50.820 --> 00:11:57.900 applications, which made a life of our devops parsing the logs way easier because they didn't 00:11:57.900 --> 00:12:05.860 have to write them by hand. And if you use a structured logs for all applications, not just one, but, 00:12:05.860 --> 00:12:12.500 uh, for example, for example, microservices and you pass, the key ID or like trace ID or something 00:12:12.500 --> 00:12:20.020 that will identify the path, which, the log goes through, then you might see what happened before 00:12:20.020 --> 00:12:28.660 the bug happened or maybe because, if you want to see, how the system is working, you also need to be 00:12:28.660 --> 00:12:33.220 either one of the detectives of the system or use the struct log. 00:12:33.220 --> 00:12:40.180 Yeah. It's interesting when you log out stuff, it looks like you can just do key keyword arguments and 00:12:40.180 --> 00:12:46.420 those will add to the log really nicely. So you don't have to create a message that you're going to send 00:12:46.420 --> 00:12:52.340 that embeds, you know, the value equal, you know, variable equals valuable, very equals value. You just pass 00:12:52.340 --> 00:12:56.180 them to the log message and they become part of the message like that. That's cool. 00:12:56.180 --> 00:13:01.380 Yeah. And you can also use, the initial message, which is an event like greeted here, 00:13:01.380 --> 00:13:08.660 um, as some kind of key, which would give more clues where this message is coming from and what type of 00:13:08.660 --> 00:13:15.940 event happened instead of a usual message. Yeah. Nice. Very cool. The other thing it says is if you have 00:13:15.940 --> 00:13:22.260 Colorama installed, it will automatically render in nice colors and that's very neat. I love Colorama and I love 00:13:22.260 --> 00:13:28.820 having colors in, in the code that we look at, it really makes a nice difference. So yeah, you get things 00:13:28.820 --> 00:13:35.540 like the colored, whether it's an info message or an error and whatnot. Yeah. Very neat. I like it. 00:13:35.540 --> 00:13:41.380 I keep meaning to use this more and I know I'm glad you brought it up because I definitely want to try this. 00:13:41.380 --> 00:13:46.740 Definitely try this. Yeah. Yeah. This is a really good one. This is new to me, but, quite neat. 00:13:46.740 --> 00:13:52.020 All right. Not new to me, but also quite neat is our sponsor for this episode. So this episode is 00:13:52.020 --> 00:13:57.540 brought to you by Sentry. So how would you like to remove a little stress from your life? Do you 00:13:57.540 --> 00:14:02.020 worry that users may be having difficulties and encountering errors with your app right now? 00:14:02.020 --> 00:14:05.940 Would you even know until they send that support email? I mean, yes, maybe using struck log, 00:14:05.940 --> 00:14:09.540 but are you watching the struck log now? You don't know, right? So how much would it, 00:14:09.540 --> 00:14:14.580 how much better would it be if you had that error or performance details immediately sent to you 00:14:14.580 --> 00:14:19.060 with the call stack and local variables and active user and all that stuff. And with Sentry, 00:14:19.060 --> 00:14:24.020 it's not just possible. It's easy. We use Sentry on all of our web apps, Python by set up M talk, 00:14:24.020 --> 00:14:28.340 Python training, all those kinds of things. And we know if there's some kind of problem. 00:14:28.340 --> 00:14:31.700 It's unfortunate if someone hits a problem, but it's better to know and be able to fix it right away. 00:14:31.700 --> 00:14:37.380 In fact, one time somebody ran into a problem over at Talk Python Training, getting a course and 00:14:37.860 --> 00:14:42.260 got the message. I could see who was logged in when they had the problem. And I actually fixed the bug 00:14:42.260 --> 00:14:46.100 and was about to push out the changes. And I got an email. Hey, I'm having a problem with your, 00:14:46.100 --> 00:14:51.300 your site. I'm like, yeah, I know. I just fixed it. Try again, please. And they were quite a surprise. 00:14:51.300 --> 00:14:55.700 So surprise and delight your users today. Create your Sentry account at Python by set up M slash 00:14:55.700 --> 00:15:01.460 Sentry. And please, when you're signing up, click the got a promo code redeem option and enter Python 00:15:01.460 --> 00:15:06.020 bytes. It's not automatic. And they'll make sure that you enter Python bytes as the promo code. 00:15:06.020 --> 00:15:08.820 Otherwise they won't know us from us. You'll get a bunch of cool stuff, two, 00:15:08.820 --> 00:15:12.980 three months of the team plan with many more errors and events and other features as well. 00:15:12.980 --> 00:15:16.340 So check them out at Python bytes set up M slash Sentry. That's pretty awesome. 00:15:16.340 --> 00:15:22.420 Brian, I guess you should probably also test your code maybe before you end up with errors. What do 00:15:22.420 --> 00:15:26.980 you think? Definitely. And actually, before we go on, I think I've mentioned this before, 00:15:26.980 --> 00:15:29.700 but the graphic on that is on the Sentry page is so cool. 00:15:29.700 --> 00:15:35.220 I know. I really like it too. Like, I love the upset console terminal reading a paper. 00:15:35.220 --> 00:15:42.020 Yeah. So this is, this is kind of like inside baseball, maybe, but I don't know, maybe three 00:15:42.020 --> 00:15:50.260 people might care about this. But anyway, I'm one of them. So X fail now works with by test subtests. 00:15:50.660 --> 00:15:57.940 So that's, it's neat. But I got to explain it a little bit. So subtests are kind of this weird 00:15:57.940 --> 00:16:03.220 feature of unit tests that came along in Python three, four, and it's a way it's a context manager 00:16:03.220 --> 00:16:10.260 so that you can have possibly several places where your test might fail, but continue. It doesn't stop 00:16:11.140 --> 00:16:18.340 if it fails. And that's a, that was within unit test. pytest had, well, pytest said pytest check, 00:16:18.340 --> 00:16:25.300 the plugin that I wrote that allows something similar context manager. But then pytest subtests 00:16:25.300 --> 00:16:33.620 came out, which was a plugin in about 2019 that started that, that allowed you to run the unit test 00:16:33.620 --> 00:16:40.100 subtests within from pytest. But there's also a pytest style of doing subtests also. 00:16:40.100 --> 00:16:48.500 They're a bit quirky. So I'm going to, we, I'm linking to, to two resources, an article by Paul 00:16:48.500 --> 00:16:55.540 Gansel and an episode of testing code where he and I talked about subtests. And so they're a little, 00:16:55.540 --> 00:16:58.820 before you jump in and use them right away, you should know some of the quirks about it, 00:16:58.820 --> 00:17:03.220 but they're still cool if they work for you. But one of the quirks that was around for a long time 00:17:03.220 --> 00:17:07.780 was that X fail didn't work. And X fails a way to say, I know my test is going to fail. 00:17:09.060 --> 00:17:14.100 but you know, and then you get to decide whether or not you want to make market as an 00:17:14.100 --> 00:17:22.900 X pass or market as a fail, if it, if it fails. and the, this, anyway, X fail didn't work 00:17:22.900 --> 00:17:29.700 with subtests, but it does now as of like the start of the month. So somebody named maybe Sibber on, 00:17:29.700 --> 00:17:37.620 GitHub, maybe, merged a fix or submitted a fix as a pull request and it got merged and it's 00:17:37.620 --> 00:17:44.500 now in version 0 5 0. So X fail, if you wanted to use subtests, X fail now works with them. So that's 00:17:44.500 --> 00:17:49.380 the good news. Yeah. Yeah. This looks really interesting. So the basic idea is I want to loop 00:17:49.380 --> 00:17:54.900 over a bunch of scenarios or whatever, and maybe test them all and then have the test fail if any of them 00:17:54.900 --> 00:18:01.620 did, but actually just go through them all before. Yeah. So like in, on the, on the subtests, 00:18:01.620 --> 00:18:05.780 site, there's a little example. So like, let's say you're looping through a range and you want to, 00:18:05.780 --> 00:18:09.860 you want to run all of them within, not, not a parameterized, just within the test, 00:18:09.860 --> 00:18:15.140 you're doing like several things and you can, yeah. And if something fails, you want to actually 00:18:15.140 --> 00:18:21.380 report all of the failures. and this is, this is, you know, sort of helpful with loops, but you 00:18:21.380 --> 00:18:27.700 know, why not just use parameterization? but the one part where it does really help is if 00:18:27.700 --> 00:18:33.060 you really are checking like four or five different things and you really want to know, like, let's say 00:18:33.060 --> 00:18:39.860 you're measuring something or you're checking, several dimensions of something and, and 00:18:39.860 --> 00:18:46.820 having all of the failures together would help you determine what the real problem is. So, so it's, 00:18:46.820 --> 00:18:49.940 it's, it's when you have to have all the information, this is a good idea. 00:18:49.940 --> 00:18:53.780 Very cool. Anastasia, what's the testing story in your world? 00:18:53.780 --> 00:19:00.020 Well, we use mostly parameterized testing because we don't have the subtest need. We don't need to 00:19:00.020 --> 00:19:03.700 test it multiple times, maybe in the future. Yeah. 00:19:03.700 --> 00:19:10.580 Yeah. parameterized works. I'd stick with it. So yeah, it's definitely good. All right. Another 00:19:10.580 --> 00:19:17.460 thing that I think is really neat to talk about, but I feel like it's almost down to the benchmark type 00:19:17.460 --> 00:19:23.540 of situation is what do you do with the secrets in your application? There's to get, 00:19:23.540 --> 00:19:31.220 ssh get, which is always terrifying. If you go here, you can see, oh, here's all the code that 00:19:31.220 --> 00:19:36.340 we found in this branch of this GitHub repository. For example, here's your, you know, database 00:19:36.340 --> 00:19:42.100 connection string with username and password right there. Right. So you can see all kinds of issues. 00:19:42.100 --> 00:19:46.980 If you go over here, like even a live stream, if it doesn't feel bad enough, you like watch the live 00:19:46.980 --> 00:19:51.700 stream of all the things that are coming in. Like right now, apparently there's some username and 00:19:51.700 --> 00:19:56.980 password in a URI and some kind of private key and whatnot. So you don't want that. So what do you 00:19:56.980 --> 00:20:02.740 do? Well, there's all kinds of things you can do. Do you encrypt those secrets and put them in source 00:20:02.740 --> 00:20:07.540 code? Well, then where do you store the encryption key? There's some kind of certain types of vaults 00:20:07.540 --> 00:20:12.660 you can install on your server, kind of like one password, but for servers, you could do that kind of 00:20:12.660 --> 00:20:19.300 thing. There's just leave it in there and hoping for the best. There's putting it in environment 00:20:19.300 --> 00:20:24.980 variables. That's a very, very common one. Right. But still, no matter what you pick, you kind of got to 00:20:24.980 --> 00:20:30.420 get that data back and deal with it. So I want to introduce you to Pydantic. Brian, you've heard of 00:20:30.420 --> 00:20:37.380 Pydantic, right? Yeah. In fact, I didn't know this had anything to do with secrets. Yeah. If you go to 00:20:37.380 --> 00:20:45.620 Pydantic right here at the top, I believe there might be some nice little comment here. Oh, yeah. I thought, 00:20:45.620 --> 00:20:49.300 I thought you were in here, apparently I'm in here right now. I think it toggles between us. Anyway. 00:20:49.300 --> 00:20:55.780 Yeah. So we've known, the point is we really talked about Pydantic a lot. It's a really cool way to 00:20:55.780 --> 00:21:00.180 create these classes that are kind of like data classes, point them at some data source, and then 00:21:00.180 --> 00:21:05.520 they validate it and adapt it. Right. So if I've got like a JSON document and it has a field in it, 00:21:05.520 --> 00:21:10.660 and that field is a list of something I could say in my model, this thing has a list of integers. 00:21:10.660 --> 00:21:16.260 And if it happens to be quote a string or a number that has quotes on it, it'll just, you know, 00:21:16.260 --> 00:21:21.620 automatically do the int pars type of thing to get it fixed. Or it'll tell us that it couldn't figure 00:21:21.620 --> 00:21:26.660 out what to do with the third value, something like that. It's really fantastic. But what I also didn't 00:21:26.660 --> 00:21:32.900 know was that it has a built in support for working with these user secrets. So Dennis Roy pointed this 00:21:32.900 --> 00:21:39.460 out to me. And there's all kinds of things. You can have the .env file. You can have Docker secrets. 00:21:39.460 --> 00:21:46.580 You can have environment variables. And all of these things has your secrets. And if you just derive from, 00:21:46.580 --> 00:21:51.780 instead of base model, you derive from base settings, then this will automatically determine 00:21:51.780 --> 00:21:58.340 any of the fields that are not passed to it from the environment or from .env files. What do you think? 00:21:58.340 --> 00:22:01.540 Well, that's cool. Where do the .env files go? 00:22:01.540 --> 00:22:02.900 Not in GitHub. 00:22:02.900 --> 00:22:04.100 Okay. 00:22:04.100 --> 00:22:10.500 You know, you store them somewhere else, right? You probably, what ideally I think you do is you 00:22:10.500 --> 00:22:16.260 would store like an .env template file that has, you know, put this value and then the real value here, 00:22:16.260 --> 00:22:20.180 this value and the real value there. And then you, of course, ignore, .gitignore the other one, 00:22:20.180 --> 00:22:24.100 the real one, right? So you at least have a structure. But so the idea is you come down here and say, 00:22:24.100 --> 00:22:31.700 I've got these settings and we've got like an API key and off key. We've got a Redis connection, 00:22:31.700 --> 00:22:38.420 all those kinds of things. And you can even say, I'm going to put a prefix on it. So in your environment 00:22:38.420 --> 00:22:45.140 variables is fine if you've got one app in one server, but if you've got 10 apps running or 10 APIs 00:22:45.140 --> 00:22:50.420 running on your server, what is the API key referred to? What is the database connection string with the 00:22:50.420 --> 00:22:55.300 database name in it referred to? Which one of those 10 apps, right? So you can put a prefix. So you 00:22:55.300 --> 00:23:03.540 could have like login app API key or, you know, login app API key. And you put that in there and it 00:23:03.540 --> 00:23:08.500 automatically will just let you access it as if it's API key. So you can sort of configure in the 00:23:08.500 --> 00:23:12.900 environment a little bit better. There's just lots of really neat things that you can do in here to make 00:23:12.900 --> 00:23:18.740 that work. you can say whether it's case sensitive, let's see, let me pull up, I had to take notes, 00:23:18.740 --> 00:23:24.180 some other things that were super cool. So it's a regular Pydantic model, which means it'll do all 00:23:24.180 --> 00:23:28.740 the conversions and the validation. So if something is missing that's required from your environment, 00:23:28.740 --> 00:23:34.580 it'll let you know exactly what's missing. It'll do those conversions. yeah, all sorts of stuff. 00:23:34.580 --> 00:23:40.340 It has support for raw sequence files as well, which is like a slightly different way to do it. 00:23:40.340 --> 00:23:45.380 You can have differently named ENV files, like a prod dot ENV versus 00:23:45.380 --> 00:23:52.260 U and a D dot ENV or whatever, all sorts of settings. So I've always thought Pydantic is amazing 00:23:52.260 --> 00:23:57.140 and I had no idea it had this built in support for working with this. The other thing that's really 00:23:57.140 --> 00:24:02.420 cool about this is if you go back to the top where it describes it, it says it will try to get these 00:24:02.420 --> 00:24:08.020 values from the environment if you don't pass them over. So if you're in say a testing environment, 00:24:08.020 --> 00:24:13.140 you want to actually pass values that would control it, you could just explicitly pass them along 00:24:13.140 --> 00:24:16.980 instead of, you know, having them come from the environment. So it's really easy to test, 00:24:16.980 --> 00:24:20.580 you know, set the test values instead of trying to configure a test environment. 00:24:20.580 --> 00:24:27.780 Nice. We do use it by the way, base settings, but we didn't use prefixes. Yes. Yeah. Which is a good idea. 00:24:27.780 --> 00:24:31.620 Yeah. The prefixes are cool. If you have a bunch of apps, if you just have one, 00:24:31.620 --> 00:24:35.780 yeah, it doesn't really matter. Right. Yeah. Yeah. Of course. Cool. You like this? It's working well for 00:24:35.780 --> 00:24:41.060 you? Yeah, it's working perfectly well. And we are committing on the development version with some dummy 00:24:41.060 --> 00:24:46.500 keys just to have them around. Of course. Of course. Oh, wow. How neat. Okay. Well, cool. Well, 00:24:46.500 --> 00:24:51.300 that's neat that you're using it. Brian, you got the next one. Is that right? You've already done it. 00:24:51.300 --> 00:24:59.860 No, but I just wanted to mention the, oh, wait. Nevermind. I had the wrong thing. Oh, here we go. 00:24:59.860 --> 00:25:06.260 Yeah. The quote I think you were looking for was from FastAPI. Oh, yes. Yes. Of course. Of course. 00:25:06.260 --> 00:25:12.020 Yeah. It is. I'm over the moon. Yeah. Super excited about it. Yeah. FastAPI. Thanks. 00:25:12.020 --> 00:25:17.860 Yeah. We use it. I love FastAPI as well. And to me, like Pydantic and FastAPI, 00:25:17.860 --> 00:25:21.540 they go together because I learned about them at the same time. I know there are different people 00:25:21.540 --> 00:25:26.900 and different projects, but you know. It works like magic. Yeah. Yeah. Absolutely. It really is. Yeah. 00:25:26.900 --> 00:25:31.540 And if it's not magic, maybe you should document it. Or maybe it is magic. You should document it. 00:25:31.540 --> 00:25:38.500 Definitely. Definitely. Actually, I'm the one who is usually bringing this topic 00:25:38.500 --> 00:25:44.180 to the team, how to write documentation. And first, the question is why to write documentation? 00:25:44.180 --> 00:25:50.580 Everyone knows that we need documentation, but it's hard. It's time consuming. It's annoying. And 00:25:51.700 --> 00:25:58.580 how it usually happens. Someone leaves the team. And then the last days are about handing over 00:25:58.580 --> 00:26:03.540 everything. And... Oh my gosh. I remember I've had this experience twice at least. 00:26:03.540 --> 00:26:09.460 Right? Where it's like, oh, you said, where you said you're going to, you've given me your two weeks. 00:26:09.460 --> 00:26:13.780 So your next two weeks, your two weeks notice that you're going to leave. Your next two weeks will be to 00:26:13.780 --> 00:26:18.580 start writing documentation for everything you've ever worked on and anything that people might need 00:26:18.580 --> 00:26:22.980 to do. So your next two weeks are to begin writing documentation that you should have been doing the 00:26:22.980 --> 00:26:27.540 whole time. In Germany, we have notice period of three months. So like it's three months. 00:26:27.540 --> 00:26:28.820 Oh, that's a lot of documentation writing. 00:26:28.820 --> 00:26:29.140 Yeah. 00:26:31.300 --> 00:26:36.900 Just kidding. But normally, even if you leave the team, like you, for example, move from one team to 00:26:36.900 --> 00:26:41.860 another, you don't, it doesn't mean that you have to leave the company. Still, you have to hand over 00:26:41.860 --> 00:26:49.540 everything that you worked for, let's say, in a year or even half of the year. And for example, 00:26:49.540 --> 00:26:54.340 in my experience, when I started with Python, I didn't know any Python. I had to learn it. And of course, 00:26:54.340 --> 00:26:59.780 I didn't know about Sphinx or Read the Docs or any kind of documentation for Python. And what did I do? 00:26:59.780 --> 00:27:06.180 Nothing. I didn't write it. And half a year later, I was wondering who wrote this code. So I did get 00:27:06.180 --> 00:27:14.180 blame. And of course, it was me. And I was like, what a stupid person. So yeah. And I suggest to start 00:27:14.180 --> 00:27:21.060 writing documentation now, even if you're not leaving the team. The reason why I'm bringing up the Sphinx 00:27:21.060 --> 00:27:28.500 and Read the Docs is that it will allow to have continuous documentation. And with Sphinx, 00:27:28.500 --> 00:27:35.220 you can easily write just some doc screens, which will explain what the function does, what the 00:27:35.220 --> 00:27:42.980 class is doing, add some input output parameters, and then you will automatically generate it. So 00:27:42.980 --> 00:27:50.100 there's no need to write it somewhere on Confluence or any other source. Because if there are too many 00:27:50.100 --> 00:27:55.060 sources, that's where the documentation will die, because no one will go and check it. And 00:27:55.060 --> 00:27:59.940 during the handover, usually it happens like that you write documentation somewhere where nobody knows 00:27:59.940 --> 00:28:01.780 where, and nobody reads it. 00:28:01.780 --> 00:28:06.500 Yeah, you pointed out that you've got it in Jira, and you've got it in GitHub, and you've got it in all 00:28:06.500 --> 00:28:07.060 different places. 00:28:07.060 --> 00:28:08.500 Google Docs, yes. 00:28:08.500 --> 00:28:10.500 Yeah. Especially Google Docs. 00:28:10.500 --> 00:28:11.060 Oh, yes. 00:28:12.420 --> 00:28:19.140 And then you share like 10 Google Docs with different people, and then they lose the links, and people 00:28:19.140 --> 00:28:24.820 are leaving. It's nice when people are leaving the team, but it's not nice to the people who are leaving 00:28:24.820 --> 00:28:28.980 the team to another team, because they are getting all the questions for a year. 00:28:28.980 --> 00:28:35.140 Where to find those? How can I get this function? How to get this data? 00:28:35.140 --> 00:28:42.260 Yeah. Yeah. Very good advice. You know, for a long time, Sphinx was like synonymous with restructured 00:28:42.260 --> 00:28:48.420 text, but now we've also got the Markdown with the missed parser there. So that's very cool as well. 00:28:48.420 --> 00:28:50.900 I'm a fan of Markdown instead, yeah. 00:28:50.900 --> 00:28:58.260 And also it supports the Sphinx itself. It supports different types of documentation. For example, you can 00:28:58.260 --> 00:29:04.100 write code reference, then you can go through all the code, and then you can also write extra 00:29:04.100 --> 00:29:09.940 documentation, like Markdown. Even ReadMe can be included into documentation. And you can also style it. 00:29:09.940 --> 00:29:12.100 Oh, nice. Yeah. Yeah, very cool. 00:29:12.100 --> 00:29:15.780 Yeah, there's lots of great themes to it too now. It really looks attractive. 00:29:15.780 --> 00:29:18.740 Yeah, you did recently cover that, right, Brian, the Sphinx themes? 00:29:18.740 --> 00:29:26.420 Yeah. And actually, when the Markdown, the support came on, that's when I went back and started looking 00:29:26.420 --> 00:29:34.100 at Sphinx. So some of our documentation is done in Sphinx now because it does Markdown. And you can even 00:29:34.100 --> 00:29:41.620 make it do, it's not built in, but you can make it read doc strings and interpret doc strings as Markdown. 00:29:41.620 --> 00:29:42.740 So it's cool. 00:29:42.740 --> 00:29:47.940 Yeah. Very cool. Very cool. Robert out in the live stream has an interesting addition to continuous 00:29:47.940 --> 00:29:53.140 integration and continuous delivery. So can we deploy yet? Only if the documentation is complete. 00:29:53.140 --> 00:29:53.620 Definitely. 00:29:53.620 --> 00:30:00.740 Very cool. All right. Well, that's it for our main topics. Brian, you got anything you want to share? 00:30:00.740 --> 00:30:02.180 Any extra stuff you want to throw out there? 00:30:02.180 --> 00:30:08.420 Mostly, I'm curious about pytest uses. So I'll drop a link in the show notes, 00:30:08.420 --> 00:30:15.780 but basically I've got a pinned tweet on my Twitter, and I'd like to have people tell me where they see 00:30:15.780 --> 00:30:24.020 where they're using pytest. So I've got some examples. And then I kind of went, 00:30:24.020 --> 00:30:30.100 I, my first question was people projects that have switched. But I was looking at just the, 00:30:30.100 --> 00:30:35.780 just the guts of how Python works. And there's some amazing projects that use pytest, 00:30:35.780 --> 00:30:40.660 like wheel, tip, setup tools, warehouse. Those all use pytest. That's pretty cool. 00:30:40.660 --> 00:30:45.620 Wow. How interesting. Yeah. And those are sort of almost inside of Python, which is interesting 00:30:45.620 --> 00:30:47.300 because they're not using unit tests, right? 00:30:47.300 --> 00:30:52.260 Yeah. So, and then I just learned about recently, even if it's proprietary, that'd be interesting. I just 00:30:52.260 --> 00:30:57.300 learned that Stripe and Lyft went through a pytest conversion recently. So that's kind of neat. 00:30:57.300 --> 00:31:00.580 Yeah. That's cool. Yeah. Yeah. Very cool. Anastasia, anything else you want to throw out 00:31:00.580 --> 00:31:02.100 there or let people know about while we're here? 00:31:02.100 --> 00:31:08.020 Yeah. Maybe using exceptions. Don't use space exception. 00:31:08.020 --> 00:31:10.100 Yeah. I agree. Custom exceptions. 00:31:10.100 --> 00:31:14.500 Custom ones that like a four-year app or have certain, absolutely. I definitely second that idea. 00:31:14.500 --> 00:31:19.140 All right. This, Brian, this was in danger of almost being an extra, extra, extra, 00:31:19.140 --> 00:31:25.140 extra, extra hero about it. So I'll just go quick. So Matthew Feikert's getting a couple of shout 00:31:25.140 --> 00:31:31.300 outs on the show. So he also pointed out that, whoa, super cool. PipX, which we've talked about 00:31:31.300 --> 00:31:35.700 on the show before, it lets you install Python tools, kind of like Homebrew or Apt. They're not 00:31:35.700 --> 00:31:40.100 part of a project, but you want to have them managed and installed in their own isolated environment. So 00:31:40.100 --> 00:31:44.660 you pipX instead of pip install the thing, which is great. That is now officially part of PyPA, 00:31:44.660 --> 00:31:47.060 the Python Packaging Authority. Nice. 00:31:47.060 --> 00:31:53.460 So yeah, pretty cool. So pipX is now sort of officially part of Python, not Python, the distribution, 00:31:53.460 --> 00:32:00.100 but the group, you know. Next, I will be presenting-ish. It's recorded, but then there's like a live Q&A 00:32:00.100 --> 00:32:06.420 afterwards. Manning is having a conference on developer productivity. I don't honestly remember 00:32:06.420 --> 00:32:11.940 what my top talk is going to be about. Oh yes, here it is. It's 10 tips and tools you can adopt 00:32:11.940 --> 00:32:15.940 in 15 minutes or less to level up your developer productivity. So I'm going to be speaking on that. 00:32:15.940 --> 00:32:20.740 All sorts of fun things. So if you want to check that out, it's free to register for. It's 00:32:20.740 --> 00:32:25.220 later this month, I guess. Here's just a thought I would throw out there for you. I don't expect an 00:32:25.220 --> 00:32:32.980 answer, but yikes, cloud bills can pile up. Alex Chan, who is teaching, I guess I don't, 00:32:32.980 --> 00:32:37.220 I could figure out exactly the context of this, but put out a tweet that said, 00:32:37.220 --> 00:32:43.540 I have a panicked student in my DMs who accidentally racked up an $8,000 AWS bill. 00:32:43.540 --> 00:32:48.020 My suggestion of talk to support is no good. Apparently they won't issue a billing adjustment. 00:32:48.020 --> 00:32:50.100 Anyone got ideas out there? 00:32:50.100 --> 00:32:50.980 Oh no. 00:32:50.980 --> 00:32:54.740 Could you imagine as a student, I mean, as a professional, 00:32:54.740 --> 00:32:59.380 it's still a lot of money, but as a student, $8,000 is like a ton of money. 00:32:59.380 --> 00:33:03.300 Yeah. It's like a term of bills. It depends on. 00:33:03.300 --> 00:33:09.220 Yes, exactly. Yeah. Like a semester of studies or something. So, maybe other students and 00:33:09.220 --> 00:33:14.500 basically all people out there put up billing alerts on, on whatever cloud thing you're doing, 00:33:14.500 --> 00:33:21.620 on whatever, whatever places I have, including AWS, I get periodically, I get an announcements like 00:33:21.620 --> 00:33:24.260 you, your bill is now at $50. Your bill is at a hundred dollars. 00:33:24.260 --> 00:33:28.260 Your bill is now at $500. Your bill is now at a thousand dollars. And if it goes beyond that, 00:33:28.260 --> 00:33:32.820 I'm going to have to start paying a lot of attention to what's going on with my AWS account. So just, 00:33:32.820 --> 00:33:37.860 you know, put these alerts on there. It's usually easy with whatever platform you're on. anyway, 00:33:37.860 --> 00:33:44.820 don't be that poor student. All right. What's next? Brian skin, shout it out. Hey, this might not 00:33:44.820 --> 00:33:49.940 be a total new item, but maybe we can mention it. Maybe it's interesting. Developed a flare mentioned, 00:33:49.940 --> 00:33:56.340 a flake. It didn't develop it. I don't believe a flake eight plugin for FastAPI. So if you're doing 00:33:56.340 --> 00:34:01.860 FastAPI, there's different ways to do things like routes and whatnot. And there's like the natural way, 00:34:01.860 --> 00:34:06.740 there's sort of a clumsy way. And so here's a flake eight thing to make sure you're using FastAPI. 00:34:06.740 --> 00:34:13.780 Nice. Interesting. Yep. And I think, yeah, and I think this is my last one. It is my last one here. 00:34:13.780 --> 00:34:21.380 So Sal Shannon Brook tweeted JupyterLab three will have localization. So localization means like the 00:34:21.380 --> 00:34:27.300 menus and the help text and the button hover tips and all that kind of stuff are localized for different 00:34:27.300 --> 00:34:32.980 languages. So JupyterLab three will have localization making it more approachable for 00:34:32.980 --> 00:34:40.100 people who don't want to work in an English UI and their crowdsourcing translations. So if you wanted 00:34:40.100 --> 00:34:44.980 to contribute to Jupyter and you were good at programming and in a language that's not English, 00:34:44.980 --> 00:34:48.500 but it's already done in English, you know, go check that out. That would be kind of cool. 00:34:48.500 --> 00:34:53.140 What if anybody just messes with people and like does wrong translations just for fun? 00:34:53.140 --> 00:34:55.780 I'm so afraid of that. Yeah. I think they do. 00:34:55.780 --> 00:35:01.140 I bet they do. I bet they do. And maybe not really obvious, maybe in real subtle ways. 00:35:01.140 --> 00:35:05.060 Yeah. Yeah. Yeah. Nevermind. Don't, don't, don't, don't have any ideas. 00:35:05.060 --> 00:35:08.580 Brian, don't give people ideas. This is not, that's a good one. 00:35:08.580 --> 00:35:14.180 All right. Well, that's all the extras as well. So how about a joke? 00:35:14.180 --> 00:35:20.980 Yeah. Okay. So imagine you're learning programming, you're learning Python, take one of these computer 00:35:20.980 --> 00:35:26.980 science courses where they talk about weird things like recursion. So cursion is the idea that the 00:35:26.980 --> 00:35:31.300 function calls itself with different parameters, right? Like a really common example would be 00:35:31.300 --> 00:35:36.980 factorial. So if I'm going to calculate a factorial, it's just N times N minus one times N minus two. 00:35:36.980 --> 00:35:42.020 So that's just N times factorial of the smaller number. You can just like work your way back. 00:35:42.020 --> 00:35:48.900 Right. But there should be an exit condition. Like if N equals one return, don't keep recursing. 00:35:48.900 --> 00:35:54.420 So here's a nice little graphic under the banner of only programmers would understand. And it's got 00:35:54.420 --> 00:35:59.300 the four squares. It's kind of like screen sharing. We got that infinite view. So learn to program 00:35:59.300 --> 00:36:04.100 in one corner, next corner, make recursive function, third corner, no exit condition. And then it just 00:36:04.100 --> 00:36:07.460 repeats and repeats and repeats down to smaller and smaller and smaller. I love it. 00:36:07.460 --> 00:36:12.980 This is bad. No, this is good. That's how you learn. 00:36:12.980 --> 00:36:20.420 That's right. No. Yeah, exactly. It's like when you share your screen in zoom or, or maybe Google 00:36:20.420 --> 00:36:24.180 meet, but you've still got the window up or something like that. But it's about recursion. It's beautiful. 00:36:24.180 --> 00:36:27.620 And then you silence basic exceptions and you cannot exit the program. 00:36:27.620 --> 00:36:29.140 Yeah. 00:36:29.140 --> 00:36:34.500 Do you know if, yeah, you know, if Python has a tail recursion optimization, 00:36:34.500 --> 00:36:35.140 sorry. 00:36:35.140 --> 00:36:40.420 I'm thinking, I'm thinking no, like, so the whole point is here, Brian, that we would run out of 00:36:40.420 --> 00:36:44.420 a call stack space really quickly. And that's usually the error stack overflow error. If you 00:36:44.420 --> 00:36:46.420 recurse too deep type of thing. Yeah. 00:36:46.420 --> 00:36:50.580 But with trail recursion, it basically becomes an infinite loop. So you run out of time instead 00:36:50.580 --> 00:36:51.700 of memory. Okay. 00:36:51.700 --> 00:36:56.900 So, but I don't, so that would be the advantage of tail recursion. I have no idea if it is there or not. 00:36:56.900 --> 00:37:00.660 Yeah. I mean, there's some languages that do the optimization, so they don't, they don't 00:37:00.660 --> 00:37:05.220 generate a new call stack because there's nothing to save. So yeah. 00:37:05.220 --> 00:37:05.540 Yeah. 00:37:05.540 --> 00:37:05.860 Anyway. 00:37:05.860 --> 00:37:09.140 Yeah. I don't know. I'm sure we will find out before next week. 00:37:09.140 --> 00:37:13.940 Yeah. One of the reasons why I like asking open-ended questions on the podcast. 00:37:13.940 --> 00:37:16.260 So yeah, that's awesome. Yep. 00:37:16.260 --> 00:37:19.940 Well, Brian, thank you as always. And Anastasia, thank you for being here. It was great to have 00:37:19.940 --> 00:37:20.420 you as a guest. 00:37:20.420 --> 00:37:20.980 Thanks. 00:37:20.980 --> 00:37:22.260 Thank you for inviting. Thank you. 00:37:22.260 --> 00:37:23.380 Yep. Bye.