Brought to you by Michael and Brian - take a Talk Python course or get Brian's pytest book

« Return to show page

Transcript for Episode #195:
Runtime type checking for Python type hints

Recorded on Wednesday, Aug 12, 2020.

00:00 Hello, and welcome to Python bytes where we deliver Python news and headlines directly to your earbuds. This is Episode 195. Recorded August 12 2020. I'm Michael Kennedy. And I am Brian. Okay. And this episode is brought to you by all the cool work that we're doing. tell you more about this. Right now I kind of just want to think about paying attention to stuff, Brian, maybe watching things Oh, like, Look this Yes, watch carefully. So actually, this thing sent over by praising Daniel, he sends us a bunch of good topics and links. And this one's called a watchdog. And maybe you've heard of it. It's the foundation of some web frameworks, for example, and things like that, to know if, say, a file has changed that you're editing does it need to auto restart the website. So when you refresh it, it actually reruns it, things like that. But watchdog is a cool little library that you can use, just on your own to know if something has changed. So basically, it's a simple little API, you create this thing called an observer. And you tell it to just start watching in some way and give it a path. It can recursively look, you can give it like a pattern or whatnot. And it will say, just basically start firing events back on this observer thing. When stuff happens, files, created files, deleted, files modified and so on. Nice. Yeah. Isn't that cool? It also comes with a Seelye script called watch me do. And what you can do is watch me do is your site, watch me do all one word log. And wherever your current working directory is, when you type that, it will just start watching for files that change right there, just like on the command line. Okay, cool. So if you're, you know, you don't want to write a program, but you're just happened to be somewhere and you're like, what is happening? Like, what files are being modified or being touched or being changed here? And you can just type that and boom, there goes. That's cool. Right? That actually is really cool. Yeah, we have a build process that part of is mucking up some directories and be kind of cool to use something like this. Yeah, absolutely. And so you could just pip install watchdog and then just type watch me do space log, wherever you want it to know, there's a bunch of other things that does that. You can pass commands and stuff, I suppose. Like, you could ask it to recursively watch or whatever. But yeah, I haven't had a use case for that. So. But yeah, this is a cool recommendation from prison. Yeah, nice. The example that I think I could possibly use it for is one of the things that we do that makes Python as well as all the talk Python talk Python training sites ridiculously fast as a user is we don't require you to redownload almost any of the static content, the anything JavaScript, any image in the entire site, you know, CSS, all of those things. They're cached for a year, right. And so a problem with caching stuff for a long time, is if you redeploy a new version, something's wacky, right? Something is really weird. Like, I have this problem with Twitter and Firefox, I'll go there. And periodically, it'll just say something went wrong, can't load it. And a command, our hard refresh always fixes it, because there's probably some JavaScript file that's like out of sequence, some other part of some API or some random thing like that. And so what we do to make sure that never happens is we look at every one of those static files that we have a year long cache on and we read it and we create the hash, and we put question mark hash equals the hash of the file on the end of it. So that, you know, that's a separate file that ever changes, one character of it changes, the hash changes, and it's now a totally separate thing in the web browser cache, right? Although this is perfect. The one drawback is to make things fast, you're not rehashing all the images on every request is it just says, Have I seen this file before? Do I already compute the hash, just use it right? So that's fine. But that means that little hash refresh trick only really works. If I restart the website, if like, say only a CSS file change, I could use this watchdog to watch all the static files that we're hashing. If any of them change, just instantly we compute the hash. That's it. So that would allow me to do like push deploys of just, you know, static content changes without kicking the website at all. And it would just magically, like, re detect those and start rolling. I think I might start using watchdog for that. Anyway, that's my use case that makes me excited. And I'm glad you pronounced watch me do because I looked at it and I went watch meadow. That's a weird name. That is a which I think is watching me do much better. Yeah. You know what else is weird? Weird HTTP status codes. Like I'm very familiar with 400 bad requests. That's when somebody created an API and I talked to it badly, and they've created the API correctly, or I get a 500 where there is their stuff has crashed. Because I sent them something bad and they've written their API badly, but somewhere in between their lives some odd things.

05:00 Right. Yes, have you heard of error code 418 before I'm a teapot

05:07 This is just great. So I love it. I love it. That is code 418 I am a teapot. Any attempt to brew coffee with a teapot should result in an error code 418 I'm a teapot resulting in the may be short instead

05:23 of resulting entity may be short and sad. I love it. This actually got brought up in a conversation I was having this morning. A colleague of mine Andy house suggested it he said so when is Python three nine can it come out because as an admin while And as a reminder to everybody Python three nine candidate release or the release candidate one or CCR whatever is out now. So you can play with it. It's probably I've been using it safe to use, but the the schedule or the release final releases in October I believe. And then after that bug fixes releases every almost replay, and if there are any. Andy said, you know, my favorite enhancement for Python three, nine is this 418 I'm a teapot. This is new in Python three, nine htt p library was missing the status code 418. I'm a teapot. And now it has it in there. But Wow, I've got some links for you. Because in the show notes there's this is referenced from ht CP CP, which is hypertext coffeepot Control Protocol. And this came out in in 1988 as an April Fool's joke, and then it's kind of part of it now. So it's part of HTTP is another fun error code. So HTTP the document entire document for ht cpcp says that most Erica's share the same status codes is HTTP. However, 418 is separate. But also now h B can use that also, but also for x 406. Not acceptable. So where else x is, this response code may be returned if the operator of the coffee pot cannot comply with the Accept addition request, unless the request was in the head. So it should return the list of possible coffee additions. This is bizarre that this is in the world. But this is awesome. I really love I don't realize I didn't realize it had been taken so far as this. This is great. And you know, if you go and look at the thing you link to the hypertext coffee pot Control Protocol, HTC PCP slash 1.0. The whole memo describes so much of this thing. And it even talks about things like crossing firewalls. Like modern coffee pots do not use fire However, a firewall is useful and protecting it. It also addresses the security considerations. Anyone who gets between me and my morning coffee should be in secure. unmoderated access to unprotected coffeepot from internet users might lead to several kinds of denial of coffee service attacks. The improper use of filtration devices might admit Trojan grounds. filtration is not a good virus protection method. I mean, it just goes on and on. It's beautiful. Yes, there's a there's a security layer that

08:19 I forgot about that's funny to this part of this, but I can't find it right now. Yeah, no worries, I threw in a quick link also to this place called http Which This is a slightly more serious take, although it does, of course, include the coffee pot, but it's just if you like, what the heck should I be doing? You know, in this situation, like, here's the error codes here, so and if you just click on them, it gives you a like a really nice summary. Like if you click on four, three, for example, it talks about when it's used and when it's like how it compares to four or four. And then also how that reference code is set up as an enumeration in different languages. So for example, and go it's HTTP dot status forbidden. In rails, it's colon forbidden. And then Python three, it's HTTP dot HTTP status is not forbidden, and things like that. So it tells you like the, the language version instead of typing it all in yourself and having magic strings and numbers everywhere. Yeah, so it does look like I don't know what Symphony is but, but Symphony and go do also support or what eight, status. Beautiful. We're back or on par with go in terms of the internet now. Very good. That was a good one I got one was could have been the joke, but I like it. All right. Also things that I like is that we get a chance to create amazing stuff to help people get better with Python out there. So over talk Python training, we have three new courses coming very soon. We have Getting Started data science, we have moving from Excel to Python and we have diving into Python memory management and optimizing your programs around that all three of those are coming within probably a month or so. So you want to get notified about that.

10:00 Just visit training dot talk right the front, there's little email place, you can enter your email address to get notified about you. I wanted to highlight testing code. So by test six is out. And so testing code is the other podcast I do at testing code Comm. And I had talked last week that there was going to be a pytest, six episode. And instead of doing it just by myself, I contacted Anthony satelli, and had him come on the show. And so we bs about stuff for about an hour. Some of its actual content. So I'm really trying to get, I'd really like to have the shorter episodes on there. But you know, I get talking with somebody and it's just fun.

10:43 just keeps going. Yeah, it's definitely a good one, people should check it out. What do we got? Next? We talked about pedantic, right? So pedantic is a cool library, you know, it's something I really just need to start using. Like, I haven't created new, any new projects that probably really deserve this. But I think it's definitely one of the things that I want to start using, because it's just so slick. The ability to say, I have these models, they have these types, you can validate them, you can auto convert between them, and so on. Right, so we talked about that not too long ago, actually. However, Andy Shapiro sent over a heads up about a new feature that's in the beta version of pedantic that's coming. That makes me really excited. Actually, I don't know, we'll see how people feel about this. But I'm excited about it. I'm excited because I'm a fan of type hints. Yeah, me too. I love type pins. Yeah, I love them, mostly. Because the way they make the editor help me write code, instead of going and say, Oh, how do I do this thing with this library? Right? If I just say, the type of this thing is one of the objects out of that library, and I hit dot, boom, the editor shows me all the stuff I can do. It just keeps me in flow and working on what I need to. So I love type hints, mostly for that, but also for the validation. One of the things they are like, but they do not do is they are not like static typing in other programming languages, where somehow that verifies what's being passed, right? Yeah. So like, for example, if I have a function that says it takes an integer, an age colon int, and that's what it says it takes, but then I go, and I write code, and I give it quote, seven, instead of the number seven. That's fine pythons like yeah, that's cool. You probably don't know what you're doing. But whatever. We're just gonna let it fly, right? Well, with pedantic, there's this new type annotation validator. And so what you can do is you can say, for this function, it has type annotations, and I want those to mean something. Okay? So it gives it basically, that is that runtime behavior in the static languages, it's a compile error, right? You argument string cannot be converted into whatever the compilation error happens to be. But it would give you the runtime equivalent. So all you have to do is you have a regular Python function. And it has regular types, just like it would, you know, s colon, stir count colon and that type of thing. And you can just work with it. It's a little bit smarter than the compiler, maybe of say, like c++ or C sharp though, in that, what you do is you just say at validate arguments, you give it a decorator that validates the arguments. And not only does it validate, but it will convert if it can. So for example, I'll put an example in our show notes, there's a function called repeat, it takes a string and a number. And it's just gonna echo out that string. However, many times that number exists, right? So you can say repeat quote, hello, comma, three, and it'll print hello, three times. Super simple. That passes the validation, precisely because it takes a string and an integer. But you can also say, repeat, hello, comma, quote four, and it'll still print out Hello, four times, because it can take the string four and make it an integer four. That's cool, right? But if I say, repeat, hello, comma, goodbye, boom, exception validation error, you know, the value is not a valid integer, type equals type error, integer, whatever, like some message there, right? So this is the count parameter is not and cannot be converted to the type specified by the type annotation. Oh, yeah, this is cool. How do you feel about this? I obviously don't want runtime typing for all of Python code. It's not, I don't want it. But for special places, like maybe a pure render API, exactly. The boundary of libraries like stuff going in there. Yeah, you're like, please don't send me the wrong data. Like what is your other alternative, right? If you really want to validate that you have to do is instance of an end or try to cast it to an end, you got to raise your own exception type and all that kind of, you're already going to have to do that work if you're writing a good API, or you're gonna send out weird errors like it does not have you know, type int does not have function split or some weird

15:00 thing and people have to realize like, oh, what that means is you type some kind of you send a number where string value is accepted, right. So this, if you really do depend on the types that you specify in your type annotation, this seems like a good idea. To me. The one drawback is this cannot make your code faster.

15:17 Right? If it's doing both validation and type conversion, yeah, in front of your code, it can't be faster than just not doing that. But you know, but it might make your default always about speech. So Exactly. And may make your sanity. Yeah. So one of the ways people have gotten around this in the past is instead of is like, for example, doing instead of trying to validate parameters to it to action, at an API level, I have the parameters bound bundled into like a, an address object and have adders validators written? Because adders, does a write, write put into a validated object type and then pass that thing over. Yeah, gotcha. Which is kind of what this is. Yeah. But without doing it. This is cool. Yeah, I like it. Yeah. Very cool. I like it, too. I was leaning pedantically. Previously, this makes me lean more that way. I like it. Alright. So one of the things that's really cool is, you know, we've had a lot of conversation about is Python fast? Is it slow? Is it fast for coming up with programs that work? Does it execute fast? Should you use libraries like scythe on or rewrite some stuff in rust? But Anthony Shah, he took it to another level, like further down than even something like rewrite bits in C or compile Python to C or something? Yeah, I think this is maybe an example of Anthony just having too much time on his hands. I think maybe. So Anthony wrote a, I just tried it this morning, a project called pi mult. EY mlt, you can pip install it. And it just multiplies numbers. And it only works for positive integers or negative integers. It just doesn't do. It doesn't ever display negative numbers. Anyway, regardless of that, it's an extension for Python written in assembly. Because why not? And because of Anthony as in like MV and your ad like the assembly language. That's the foundation of basically every other programming language. Yeah. In Anthony's Twitter announcement, he says, After a series of highly questionable life decisions, my Python extension, written in pure assembly is now on pi pi. It required an assembly extension for dist utils. He also wrote a GitHub action support. So it's running cicd and testing with PI test, above and beyond over the top. But there is some coolness of it. So it's a proof of concept, demonstrate how to create a Python extension in 100%. Assembly itself, and then how to write a you know how to link those up how to call a C API and create a PI object and parsed by tubal and stuff like that, basically, all the stuff you have to do to get point parameters back and are in values out of an extension, written in assembly. And, yeah, it's interesting to like it, actually. Yeah. And anyone wants to know what the code looks like. It's like, move racks, comma, x. I'm all q word of why move results to racks move Eddie to result like it's, yeah, well, but then you get a call pylon from long, which is kind of a cool thing. So there's this like, interesting mix between the just pure assembly language and the C, data types of C Python. On a serious note, though, anybody wanting to learn a little bit of assembly or something? It's not like, often you need some sort of impairment to try it out in having some way to link Python and assembly is kind of neat, actually. So I'm grateful for that, though. You write some of these commands. I mean, I wrote some assembly in college. That was a long time ago. Yeah. Well, I had you had to Anthony, because that's some impressive stuff. Yep. Would you say it's easy, though? No, no, it's hard mode. Yeah, you know, what actually is not as easy I think, as it should be, and is honestly just bizarre. Our properties in Python as in, at property, some function returns some value converged what looks like a function call over to something that looks like field access, right? Yeah. But why? Why does it have to have this like bizarre sequence of conventions where I have an add property and then a function that has a name, okay, that defines the name of the property that makes sense. But then the subsequent day, if I want to be able to set that I have to say, you know, the property was a then I have to say, decorator, a dot setter, and also have a function that is called a, because a dot center stuff is weird. And then the order of the setter, varies as well. Right? It has to be or is constrained, it has to be after the property and what I found also is that you can run into issues with inheritance. I think

20:00 If the property is defined in the base class, you try to create a setter on a drive class things don't work right as well, which is all kinds of weird, right? So there's just like, this is supposed to make life easy. Why is this so complicated? Like, surely there was just another alternative that was like easier to do to implement as well. Anyway, it's always kind of been one of the Bizarro things of Python that said, I love properties, I love that you can do like lazy calculation with them, you don't have to store them as part of the memory, if they can be recomputed. From the values, there's a lot of good reasons to have them. So rude vendor him sent over this cool project, which he created called easy property. And do you know what it's easier than regular properties? Okay, I think that's why he named it that.

20:45 So the idea is, you know, what you could do, instead of have this property and then at a dot setter, or a dot deleter, is you could just say, there's a getter at Sutter at deleter on the thing, and it doesn't matter what order they are. Or if they're all defined, like, for example, you can't have a setter without a getter. Now, conceptually, you should never really do that. But just, you know, syntactically like those things have to be defined there. And like I said, this sort of base class drive class type of thing, it gets wonky as well. So with this, you just create a function, the function name is the name of the property, and then you put at getter on it. And that makes it the kind that is, you know, the getter property, if you want to have one you can set you say, at setter, the order doesn't matter, or anything like that. Nice. And that nice. Yeah. So there's a cool little example, that he wrote, you know, you can do them separately, which is probably what I would recommend and do. But you can also have an add getter setter. So you can have one function whose job is to both do be the getter and the setter operation of the property. And then the way it works is just the value is set to be like some default value, which is what happens in the, you know, in the get version, you're gonna have an add document, or you can have an add document or so you can also have documentation for your property. And there's a little function that will come up and do that. Anyway, it's, it looks pretty nice. To me, it's not one of those things, that's going to be you're taking a heavy dependency upon to give it a try, right? It's not like a runtime thing, like, for example, that validate arguments that's in between every time you call a function all the time all the time, right. So like, this is like the thing, like if it's gonna work, it's gonna work clearly, or it's just not gonna work at all. So give it a try and see if it makes you happy. And if it sparks joy, you can have an add getter. And I like this syntax better than porphyries. Just Yeah. Yeah. And you know, I think app properties just fine. Like, okay, that totally. But, you know, at property name dot setter, like that just tries. It's just so bizarre and weird. And the ordering and the dependencies of it existing in the same class hierarchy level, I think. Yeah, don't get me started. So this is cool. I like it. Thanks for developing that and sending that. Yes. And then we talked about assembly and properties and decorators and stuff. So if anybody's left listening to this podcast,

23:15 we do more talk about recursion. No, I just kidding.

23:19 I couldn't resist highlighting the last topic. So there was an article by Ryan Howard, that was on the test project at i o blog, called non blocking assertion failures with PI test check. And I had to highlight it, because it's really the first time anybody's ever written an article about something I wrote. So that's cool. So pi test check is a library that allows you to do multiple failures within a test, mostly, because assert, normally use asserts, but asserts stop after the first failure. And sometimes you want more. And I never really thought about the different use cases. But so Ryan has the use case of using it with Selenium. Because sometimes when you're testing a page, coming back, you might have to test multiple aspects of it. Let's see, his example was, uh, what's the content of some field? And also, if the URL is correct, or something in I think with, like a Selenium or some sort of web test, that totally makes sense, because you'd have things like, you want to check the error code, and content and whether somebody's name shows up, then a whole bunch of stuff, you could check in having multiple things, right? You don't necessarily have just one assertion that captures all of the, the essence of what you're trying to determine. Right? Yeah. And so I also like to the PI test, check library and then or plugin. And then also an article that I wrote back and way back in 2015, where I started thinking about this and thinking to solve it, so Okay, yeah. Yeah, very cool. Article, right. And then I also wanted to do a public service nice, but because both Ryan and Anthony got this wrong, they're no good.

25:00 The letters in PI test. So hold on, hold on, let me try this. Let me just Yeah, okay on this one. But if I go over to word and I type pi test is a testing framework, I'll bet you that word makes it capital. Isn't that change there? Isn't that make it real?

25:15 No, I know PowerPoint. Well, so no, yeah, that's always tricky, right? When you have sort of a formal name, but the formal name is doesn't have spaces or it's lowercase, but I'm fine when we get used to it with iPhone. So there's no capital. mean, the P is capitalized and iPhone, but yeah, what? Correct here. I wonder if you type iPhone? I would doubt it. I mean, but that's Apple, and they think differently. That's what they tell me.

25:40 But there's others. I mean, so there's some I guess it's a weird thing with the tools and stuff. Some tools don't care whether you like whether you capitalize or not. The people in PI test like it not capitalized. So yeah. Anyway, yeah, let's, let's respect their lowercase peas. Yeah. All right. So I've three really quick things, though, kind of fun. So pi MC, is a cool library for Bayesian analysis and probabilistic programming and Python. So Alex, one of the core developers there, sent me a message said, Hey, we're planning the first ever pi. MC on okay, I pronounced pi MC on because it's so on. They say, it's the first Bayesian community online conference, around pi MC. So if you're into probabilistic programming and Python, check it out there a bunch of cool stuff they got going on there. Also, I think, have a call for paper. So if you want to do a prisoner presentations, if you want to do a presentation, shoot them a note. And yeah, I'll link to that in the show notes. Second, a while ago, we talked about rumps quite a while ago, rumps his ridiculously uncomplicated MAC address something as many programs or something like that. And so I ended up while I was sitting around, I have this little library that I run, because so many things in my life require taking a title, or taking some words and turning them into a filename or a URL name. Alright, so suppose I've got the title of a video, and I want to name an mp4 file that and it's got like a colon in it, or like a forward slash or some random thing that shouldn't be in there. I don't want spaces I want. So I wrote this little command line utility called URL phi that I would just run it would take whatever's in the clipboard and replace it with that canonicalized version that would work well as a file name. Oh, yeah, that was cool. But then I got tired of like, always firing up a command line terminal, maybe it's busy doing something else, I got into a new window, run that thing, and then close it down. And I just like, I just want to click something once, right, I don't want to click the terminal, and then start a new terminal. And it was like, not that much work. But you can imagine, the way I'm going on and on about this, I must be doing it more than as reasonable. So what I did is in like 45 minutes, I converted that command line app to a Mac dot app file, like a full on dot app file that I could just ship to any Mac user, they have no idea. It uses multiple Python packages. The code that I wrote in it runs as a GUI, auto starting with my Mac OS when I log in, in my menu bar. Hmm. Does that cool? Yeah. And so I linked to a little tweet that as a screenshot just says convert text, but actually, I changed it to URL phi tax, trim tax. And I could just click up there and any texts that have you know, if you copy something, and you want to like, paste an email, or like, but maybe it's got a bunch of whitespace, or like, why is this all here, right? So to put it in text editor, and just getting a little bit out, I just hit that and then I'll trim it or like a good lowercase tax. So like, have these like little cool like clipboard text transforms. And it all took like 45 minutes to turn it to a Mac App with PI to app and rumps.

28:56 So Dan, are you sharing this summer, I will happily share it. Okay. But I don't have the GitHub repo public is let me just like, put like, a screenshot and what the heck it is, and then I can make it public. But yeah, so I'll put it in the show notes. Neat. Yeah, it's super cool. And finally, I just want to let people know we're taking courses over at talk Python. When I was sitting around and I decided I want as you hover over the scrubber, I want it to show show a little thumbnail of where you would go in the video if you were to click the scrubber in that location. But subscribe. We're not like the little like time thing where you can click around and like zoom ahead, whatever. We're not like reusing Vimeo or YouTube or something that might potentially do that. I think YouTube does it. I don't know that Vimeo does. Anyway, I recently figured out how to do that. So if people are interested, they could they could check it out. But there's a cool little trick where you can take a second hidden video player and pointed at either the same video or a

30:00 smaller version of the video and then as the mouse moves, just seek it to that time and show it above where the mouse is. Isn't that crazy? Yeah, that's Yeah. So people need some like that. Yeah, a little cool trick you might consider not that often in Python. But if you happen to make your way to the JavaScript side of things, you'll do a lot of that I guess, in your your training site is have been updated. Yeah, exactly. It's not updated as we speak. Because I'm still busy transcoding 200 hours of video to 500 by 200. sighs so I'm about halfway through that I've had yesterday, I uploaded 200 gigs of data. I'm so going over my data limit this month.

30:44 That was 20% of my data limit in one day. I got more two cents, so But yeah, it'll be there. As soon as the videos are done. I'll turn it on. I'm gonna back up a little bit. This IMC? Yeah. probablistic programming. Can I use this to make a probability drive? probability drive? Yeah. Is this from science fiction? I don't know. Hitchhiker's Guide to the Galaxy. Oh, no. Yeah, I would think so. I would begin by feeding it the number 42 and see what comes out. Maybe. Yeah, that's that's probably the natural first step. Yeah. All right. Speaking of jokes, we got to you want to go first? Yes. This was submitted by Ruben Lerner inspired bunny Anthony Shah. I used to do low level programming. Then I product I bought told me no assembly required since then. I've been coding in Python. Nice. Except for Anthony Anthony's cutting in assembly. He codes in both. Yeah. His Python is assembly code or something like that. So this one, like last week, we talked about little Bobby tables, who's beautiful over at XKCD. And I've got another XKCD for us. But this one is not about databases. No, it's about source control and get Okay. All right. You want to be the the woman developer that asks the question. Alright, I'll start us off. Alright. So a couple developers speaking. First one is talking about this new source code. This is good. It tracks collaborative work on projects through a beautiful distributed graph theory tree model. Cool. How do we use it? No idea. just memorize these shell commands and type them to sync up. If you get errors, save your work elsewhere, delete the project and download a fresh copy.

32:17 It's funny because it happens too often, I think. Yeah. I think most people have like the four or five or six Git commands that they use all the time and everything else they have to look up needed. Yeah. Merge conflict. I'm deleting it. I'm gonna copy this back over.

32:35 Yep. All right. But he did. Well, thanks for being here as always, and thanks, everyone for listening. Thank you. Bye. Bye. Thank you for listening to Python bytes. Follow the show on Twitter via at Python bytes events Python bytes as in V yts. And get the full show notes at python If you have a news item you want featured just visit Python by setup M and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian knockin. This is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.

Back to show page