Brought to you by Michael and Brian - take a Talk Python course or get Brian's pytest book


« Return to show page

Transcript for Episode #252:
Jupyter is now a desktop app!

Recorded on Wednesday, Sep 29, 2021.

00:00 Hey there. Thanks for listening. Before we jump into this episode, I just want to remind you that this episode is brought to you by us over at talk Python training, and Brian through his pi test book. So if you want to get hands on and learn something with Python, be sure to consider our courses over at talk Python training, visit them via Python bytes FM slash courses. And if you're looking to do testing and get better with PI tests, check out Brian's book at Python bytes.fm slash pi test. Enjoy the episode. Hello, and welcome to Python bytes where we deliver Python news and headlines directly to your earbuds. This is Episode 252. Recorded September 29 2021. I'm Michael Kennedy.

00:40 And I'm Brian rocket.

00:41 And I'm Ethan swan. Ethan,

00:43 welcome to Python bytes. You've been over on talk Python, where you talked about some really cool data science stuff. And now you're over here. And so thanks for being here. Tell people a bit about yourself.

00:53 Yeah, I was I was on talk python 236. So it was a while ago, but that was really cool. I work for a company called at 451. It's the data science subsidiary of Kroger. And I'm a data scientist. But basically what I do is build tools mostly which are in Python for our data science department. So we have like 250 data scientists pretty large department. And I build like packages and some dashboard sort of things just like various technology helper stuff for data science. Yeah,

01:19 it sounds really fun. And you all run what we were talking about before we hit record, one of the probably one of the larger data science groups out there, right? I think of data science as being like there's a couple of folks that are embedded with like a marketing team or product team with a software development team a lot of times, but you are properly large group of data scientists.

01:38 I mean, in theory, that's what the whole company does. So it's a very cool experience. And often I think that's nice for the team I'm on because you don't usually get so many customers on internal tools. You know, we're building stuff for literally hundreds of people to use. And it's a little bit like releasing software externally. So it's Yeah, it's a lot of fun.

01:55 Yeah. Fantastic. All right. Well, we're definitely looking forward to having your insights here for the show. Now, Brian, I do want to start off here. I want to talk about some deck staining. Thanks. Yeah. So because those of us who are very attentive on Twitter saw that Brian kindly responded to somebody who sent us a message and said, Oh, I see you were talking about pallets. We should also talk about deck stain, and other DIY project resources. And maybe you could put that stupid article on your blog, you know, like, we're not a blog, we talk about pallets because it's on flask, and then pin Twitter decided, Oh, you are now classified under the Home Improvement category. So are we changing our theme or wide?

02:35 Apparently, just me that's, that's most of you. I've got a few new followers now. And most of them are people that like to make things so

02:44 well. It's fun to make things as well. But maybe we'll maybe we'll talk more about SQL and stuff like that. What do you think?

02:49 Yeah. So this, this was sent to us by Dave kutesa. Thanks, Dave. I want to talk about SQL fluff. So I had never heard of this. But it looks pretty cool. So SQL fluff is a Python, a Python package that is basically a linter for SQL. So that's how

03:09 I dressing I haven't really thought about linting and SQL code, but it makes perfect sense.

03:14 Yeah, well, I mean, there's, there's no, like, I don't really think about it too much either. But there's like things like should you capitalize all the keywords and some people just like it like that. So there's style, there's a both style guides around Steve sequel, I assume their style guides. And this lets you help helps you enforce it, not just style guides, but just you know, looking for mistakes and things. The the page looks really slick for is, I like the logo, the fluff logo, but but the one of the things that's great about it is the documentation. So the documentation looks wonderful. And there's one of the neat things about this is there's different rules or different dialects set up so that it treats different things like anzi and Postgres SQL different. And, and I'm not sure if these are, or if these are style differences or, or what they're doing different. But it's kind of interesting that they'll they'll there's a difference there.

04:15 Well, one of the things that comes to mind for me if this reports errors, and I suspect it probably does, one of the things that comes to mind for me is if you're using like Microsoft SQL Server and you're using a parameterised query, because you don't want little Bobby tables in your school, you would say at parameter name, whereas in with like MySQL or Oracle, it'd be like a question mark. Right. And I think Ron is illegal in the other syntaxes. So at least in that regard, I think I don't know for sure it's illegal, but I'm pretty sure like, it may be and it could be that you've got to say what type of parameterize specifications and other extensions that are valid.

04:49 And even I think there's some keywords right, or there's some different keywords in some cases. So it would make sense to have to know the dialect.

04:56 Yeah, yeah. And also like you were saying if if there really are big difference Or even minor differences, there might be some queries that you don't run all the time. And so you're not sure if you switch databases that they might be broken if you're trying to port. So kind of cool. There's a list of so it has rules like a lot of linters rules for failure. And I like the rules page because it talks about the rules, but also shows you the anti pattern. And in best practice, I kind of like that style. I don't know if I like the terms, anti pattern, and I really don't like the term best practice. But nonetheless, the what it's looking for, and what you should do different is, is a good thing to have in the documentation is pretty cool. I do

05:39 like the anti pattern aspect, I may be pattern, you're gonna have an anti pattern to have the pattern. I don't

05:44 know, I'm sure. One of the things that's in the documentation, I can't remember where there is that people should be aware of supposedly this even those has like an 1982. It's interesting stars. It's a it's still in alpha phase. So there's a note here that says expect significant changes. So just be aware of that.

06:06 But well, I didn't see major because you're not doing runtime behavior on it, right? It's a thing, you run against your code, and then you

06:12 look at the output. I mean, maybe it's in your ci system or something. But yeah, but

06:16 it's not in production, right? Sort of like you won't, you won't get called on a weekend, because the site went down because this thing got automatically updated or something to that effect. It could have broken your queries, but you know, whatever.

06:32 Um, and it's good to have an audience because we did have Paul from the chat, say Ethan's correct there are different keywords between different SQL dialects.

06:41 Yeah, we use? Oh, sorry, Michael, no, good, great. We use a lot of SQL. Yes, I would assume most data science shops do. But one that what this made me think of was one contentious topic. And people who write a lot of SQL, especially when you have a bunch of column names, and you're selecting regularly, you know, five to 10 columns. The comma first, I don't know if you've seen the approach where you knew line, comma, column, comma column, so it lines up really nicely, and it makes it easier to delete things. That's a very common thing that people feel strongly about. So I could imagine linters being very handy to at least enforce one style throughout the company, because, you know, we don't have that.

07:19 Yeah, nice. And then I was gonna add that panfilo ROI on the audience says it would be cool if there was a plugin for D beaver and Sam Morley asks, I wonder if it checks if inputs are sanitized? I don't know if I should. But Paul also asks, If it validates for syntactical correctness beyond just style. He does say that it catches errors and bad sequel before it hits your database. So I'm going to go with the Yes. That's pretty cool. Yeah, Ethan, I was thinking as I was watching Bryan present this that you probably do way more sequel than I do, even though I run in production websites that are backed by databases, not just because they're NO SEQUEL, but because I use rmws. And this data structure doesn't change. But for data science, you're kind of in a more exploratory mode, right?

08:04 Yeah, I think it's pretty interesting because, you know, like listening to this podcast, people talk about using ORM is alive. But in data science, you don't really think of data in that relational model as much. I mean, you can, but like thinking of as rows as objects is really not common. So I feel like my relationship with databases is totally different. My first couple years, I was mostly writing SQL, but it was it was literally just asking questions for analyses, which is such a different use case than what people use it for, for web development,

08:32 right? Yeah, absolutely. It's it's super different, super different. But if you were to explore data, wouldn't it be nice to have a desktop application instead of a web browser for doing so. So Jupiter, Jupiter lab has got to be the most popular way that people interact with data on the data science side. It's certainly an exploration stage. Anyway. So super big news. That is, old news is new again. But better. Jupiter lab desktop app is a thing. Like I can download Jupiter lab. It's an icon on my dock or on my taskbar, I click it, it runs like an app. But inside of it is duper notebook, like the whole Jupiter lab with terminal and Python consoles and kernels and all those things. That's cool. That's

09:14 That's very nice.

09:15 Yeah. Have you played with this yet? Even?

09:17 No, we so I don't know how common This is. But I think for us, at least, mostly people aren't working on their local machines. They're really connecting to obsession of Python on a remote server. So mostly what we do is we fire up Jupiter on on a remote server. And then from our laptops, we hit that URL to actually look at the notebook. So I'm not sure a desktop app would work as well for us, although maybe it's definitely interesting. And I wonder if there's some native features of desktop apps that are available that are going to be a reason to switch?

09:47 Well, what I would say right now it's it's a really nice, self contained thing. So I'll just read the description real quick. Jupiter lab app is a cross platform standalone application distribution of Jupiter lab. is a self contained desktop application which bundles the Python environment, and several popular libraries for use in scientific computing, like surely pandas and NumPy, and those kinds of things. So what you get is you get just an app that's, that's ready to go that you could just have somebody install, and you can say, here, open this notebook and run it. And long as you're using core libraries and stuff like that. You don't have to think okay, go to the terminal, you know, set up the environment and then type Jupiter lab, oh, you need to activate the kernel. And you got to do this and that, you know, it's just like, it's, it's a real simple, here's, here's the thing, no nonsense type of AB.

10:37 Yes. And you lost a whole bunch of people with just open the command line. Yeah,

10:41 that's so true. Yeah, yeah. So you don't have to hear right, you just it's on your dock, you click it just like, like you would with the word or Firefox or whatever. And it's, you're there, it starts and manages the Jupiter server in the background, there may be a whole host of command line arguments, you can give it to say, like, run, but use that server and other things along those lines, or run and use this conda environment. I didn't see any of those. And so from what I can tell is, it's kind of a local version of Jupiter. So it might be super interesting for you all in your workflow.

11:14 But one place where I think this would be really handy is teaching beginners. So I actually teach some Python, especially for data science classes at the University of Cincinnati. And one thing that regularly is really confusing to people is that you can't double click on a notebook file and have it open. Because that's such a typical experience of files on a computer, you double click, and there's an application that opens that file. Oh, interesting. And there are workarounds. If you have Anaconda navigator, it kind of works. Although it's it's a little sketchy. But I would assume that if you have a desktop app, you'd be able to register that with the operating system, whatever that process is to sound like when I click on dot i, py and B's open it. Because I find I have to teach students no startup Jupiter, open your browser, navigate to that file in

11:57 sorry, I'm sorry, where are you in the wrong folder in the terminal when you ran Jupiter which is locked out of that tree that that part of the tree of the folder and

12:06 then suddenly you're having a conversation about paths? Yeah, you go down, it really is like something I don't like to deal with. So maybe this is what I should recommend for people. When I teach,

12:14 what I would recommend is just check it out and try. So I do have a bit of a comment here from Dean out in the audience. I like the concept of Jupiter lab app, but I'm afraid it'll be a V in V and V virtual environment nightmare. So what I found interesting is it discovered You know, when you're creating kernels for Jupiter, you have to run a command, I always forget it, I always have to DuckDuckGo or search this to figure out how to do it again. But I have to get the command to say create this environment and then register that as so Jupiter finds that conda environment that vnv

12:51 right. It's i pi kernel install. I have to do this all the time. Yes, exactly.

12:55 And I know that it's basically that but the exact command I always forget. So that command, it seems like it picked up the ones that I had run previously for standalone terminal Jupiter lab. So the virtual environment story is the same as Jupiter itself. Without that I think all we're getting here is we're getting the libraries plus Python plus the server starting all bundled together. And it's basically the same as if you just run it on the command prompt.

13:24 I think as long as was a dean, as long as Dean doesn't want to be starting Jupiter from the virtual environment, it should be fine. Like what you said, Michael about the kernels, that's a much more I recommend people do it that way. Because some people do like to just install Jupiter in whatever environment they work in and launch it there. But I have a hard time imagining how that would work in this case.

13:42 Yeah, I do as well. And Dean makes the point that once you have to go and register all that kind of stuff on you like when you're down in the terminal doing this, you kind of lost those same people. And that may well be the case. But I can see this is sort of a first version of this, I can see that those are some of the desktop things that could add right? It could add a setting section where you you have a dialog for managing these things and creating new ones and so on. So could be pretty neat. Yeah, definitely something to watch. Alright, before we move on, Paul, out in the audience has a quick question for you, Ethan, tangential one. Python has some really great essay st tools like banded, but I'm not able to find good options for R. And I know that you live in a world that does both R and Python. Yeah, you haven't done this. I have no ideas.

14:28 I'm gonna come off as a fraud. But I don't know what SST is. I have to I have to admit, like what I do. I know I said, I'm, I'm a data scientist. But in some ways, that's nominal. Like really a lot of what I do is software development for the data scientists and

14:44 data scientists are your customer, in a sense, or your your target user? Yeah,

14:48 yeah. So I think a lot of what I hear from users is that there are certain measurement tools and certain statistical tools that are available and are that take longer to get to Python. So I wouldn't be surprised if that really is what's happening here, but I don't know Have Any suggestions? Yeah, okay.

15:01 Yeah, so bandit is like a tool that will scan for known security vulnerabilities like leaving debug settings on and Django. So

15:10 I was wondering if that was okay, then that I also don't know that's a little different than what I was imagining.

15:16 Yeah, awesome. All right. Well, since you got the floor, tell us about your first time.

15:22 Sure. So I found this requests cash package in a newsletter recently. And this might be a little bit of a shorter one, because unfortunately, I haven't had a reason to use it yet. But basically, what this does, scrolling down here is you can instantiate sessions, just like you would with the traditional requests library. So probably, request is one of the most commonly used Python packages, I would guess, yeah, for anybody who's not familiar, you use it to make HTTP requests, which is basically to bring anything back. And the tagline I think, is HTTP for humans, but it's just known for being easy to use, and you can access the internet. But one thing that I have found is that, especially if I'm, if I'm testing something in an interactive way, not mocking, but I really want to see if my code pulls back what I expect, sometimes I rerun the same request over and over and I say, go get this, go get this go get this, often the same data. And sometimes that data is large. And that takes a really long time. So requests cache is a way of creating a session object that looks and acts the same. But when you call a get or a POST request on the same URL, with the same data, what you get back is actually just the cached version of that data. So you're not waiting every time the first time you incur the network latency. And if the server has to do anything to like compute the data, or if it's enough data that it takes some time to get to you, you wait for that. But the second time, everything runs instantly, which is really a big advantage. So I've done some things with web scraping, where I'm building some kind of, I want to build like a function that pulls some things down and makes or pull some things out of that. But just waiting every time to run the function for it to pull from several different pages and you know, do some computation on that actually makes pretty slow. But if you were able to cache it like this, that'd be a lot faster.

17:08 Yeah, this is nice. I love the fact that it's just a standard replacement for the requests session itself.

17:13 Yeah. And if you scroll down a little more, it actually shows a way to do that with the regular requests library. And this actually scares me a little bit. This is kind of kind of magical, what's going on here, you just run a one liner with requests cash, and then suddenly, the requests library itself works differently. So I wonder if that's a monkey patching, or what's going on? I probably is, but it is, it is really slick. So I would imagine I'll have a reason to use this soon. But I haven't tested that yet. They does offer a lot of configuration options. And one thing I thought was was a good idea to look at is an expiration date. And that's sort of like when should you invalidate the cache and actually pull again, because you maybe should trust that the website is sending you all the same stuff today. But if you rerun your code in a week, make sure that it still responds the same way. So it's got some nice options like that.

17:59 I really, that's interesting. So you could use it not even just for testing, it could be for actual data, but you know, it's not getting updated very often. So

18:07 that's but yeah, for large data is what I was imagining. So yeah, like I said, there's there's been some times where I've like pull things from API's where they send back a lot of data, and you don't want to be waiting for that. Yeah.

18:17 Or even you just want to make sure that multiple calls to it are getting the same data, even if it does change. That's true. Yeah, to keep consistency. Interesting.

18:26 This reminded me a little bit of the, I don't know, if people are familiar with the app cache, or LRU cache, it used to be another new one, just called App Cache in the func tools module built into Python. And that's very, very handy. Once you know it's there. Because often you have a function that you don't want to recompute the work for. And this is almost like somebody rewrote requests with cash in it, which is pretty cool.

18:47 Yeah, it's got a lot of nice features. You know, I think a question from handful out in the audience can add cash to Redis. Because production in memory production caching, you could blow it up, right, blow up the memory. So a couple of things that stood out to me that were interesting. There was Yeah, you could throw a func tools LRU cache decorator on to an expensive thing, which is fine, but that's in memory, right. And plus, things have to be hashable and whatnot. But you could do that. But it's in memory. And a lot of times if you have scale out as you do on web apps, like in production, as in, Brian was talking about, you have web farms like five or 10 copies of micro whiskey or something running. So then there's still five times you got to do it before it really gets cached. And then also it goes to SQL Lite. So it gets stored to disk, right? So it's not even in memory, it's on disk. So like you said, there's other backends as well, but I think having just by default going to a sequel lite file with an possible expiration means you could just turn this on and leave it expire after a day. Go. Tell us about there's more than just SQL Lite.

19:53 Yeah, it does seem like you have some options. I mean, like I said, I haven't had a reason to use this. So I've been toyed around with all these but it this the way the as documented leads me to believe that it really is just a drop in replacement that you can configure what you want to use as your back end. And I do wonder so yeah, what you were saying, Michael, but having multiple instances, I do wonder how that would work? Would it check to see if any of the instances it cached this yet? Would it like proactively go reach out to the cache? or? Yeah, well, I

20:17 think if you have a memory one is going to be a hassle, right? Like one of the options is memory. But all the other ones file system grid Fs Redis, sequel lite, those are all support, you know, concurrency, they are exactly so so then it will scale across process seamlessly. Yeah.

20:33 So that could be actually really helpful for something like that. Where you have a distributed set of workers. Yeah,

20:38 yeah, for sure. Let's see some fun stuff about your monkey patching comment. Dean says monkey patching is like having a real monkey. It's very cool when other people have it. But having in my house is scary. And yeah, Sam just has a too much experience at the zoo, I think with that as well. So yeah, monkey patching little sketch. Nice. Alright, Brian, you're up next.

21:01 Okay, what do we got next? I, so I did something kind of dumb the other day. Um, so I I went ahead and I needed I pushed a new package out on pipe. Yeah. Really, I was just trying to remember how to the whole process because I wanted to just remind myself of like, if I have something new, something cool. I wanted to share. How do I get it out there to buy Pa? So I was walking through that process. And I was doing a firm's

21:31 logo and typo squatting? Apparently.

21:33 So I published pi tests slow. And then, but who was it? Brian skin said, cool, but maybe pytest, skip slow would be better. And I'm like, Man, that is a better name. Because that's what it does. it skips this load tests by default. So and this is totally lifted from the PI test documentation about, they have this example. But nobody's written a plugin for it. So I did this a little tiny thing. But so I renamed it, but how do you rename it? So I went out in search. So how do you rename something in pipe? Yeah, you can't really do it. But you can create another one. And then so this is nice. Well, who was it? Simon Wilson wrote this up. It's a pipe, I renamed cookie cutter template. And I didn't actually use the template. But I did use these steps. So the steps really are create a renamed version of the package, which I did, then publish it to pi pi under the new name, and create a final release for the old name that points to the new one, and depends on it and have dependencies so that if somebody installed the old one, they'll really get the new one. It sounds more complicated than it is it's just a few steps. But there's a cookie cutter, you can use the cookie cutter uses the setup tools. And I didn't want to do that. But so I used I did basically copied is the entire thing. And then he's got a demo. So if you look at it. So if you go to the old one old version, it'll just have a thing that says, hey, I'm going to the new one now. So I did that. And it was neat. I really appreciate the steps. And it's it's all good.

23:16 Yeah, it's cool. Because you use it for aliases, like you can install ps4 or beautifulsoup. For right, it's kind of same.

23:24 Those that doesn't really do that.

23:26 I I'm guessing I don't know. But it sounds like the same. No, but

23:31 so now, so now when I go to pipe this, if you go to the old one, it just shows it's now a new name go to the other one instead.

23:37 But if I install the old one, it it kind of just pulls in the new one. Yes, yes. Yeah. Very cool. Yeah.

23:43 Brian, you were you were refreshing on pi pi. But I actually just pushed my first ever package to pi pi a couple weeks ago. And so that was, you know, a bit of a trial. But I was amazed at how straightforward it is. The documentation is excellent. It really is, is pretty seamless, actually, for somebody who's never done it before. So who knows, hopefully I don't make any mistakes on the one package I have and need to read down. But

23:43 immutability of it, it's a little scary. But yeah,

23:43 for me, the hard part was just understanding that it really was pretty simple. And then also getting the hashes, right. So you have to like, you have to get like, you know, signatures and stuff to make sure that you can push to the API correctly.

23:43 So yeah, but even the documentation there, it's a little intimidating, but it actually turned out to be only a few minutes of work. So that was, that was pretty nice. Good for them. I guess HP IPA is the people to praise for that.

23:43 Yeah. So what was your package? Oh,

23:43 I it's called pre mark. It's a it's a spin off of a JavaScript library for making slides. And I just make a lot of slides for teaching. And I actually found an existing package by man I want to Yeah, here it is. It is not ready. That's why it's a release candidate. Okay. But I, I based it on this existing package by at Tyler Dave on GitHub, and talk to him about Little bit about it, he had already built a really lightweight tool and I just expanded on it. But I like to write my slides in markdown, which is really what this is for you write your markdown in a bunch of different files, it stitches them together and creates a what's called remark, j. s presentation. So I use this for my own teaching. Nice. I'll check it out. But it really is largely a sample project is just like, learn how to use pi pi and things like that.

23:43 Okay. Yeah, very cool. All right, up. Next, we have caching. Oh, wait, we just talked about cache, you know, I have more caching. So Django, I have two, two pieces of news on Django. This one comes from Carlton Gibson. I'm one of the Django guys, and also one of the hosts at Django chat, the podcast. So they are adding a Redis cash back end to Django. So traditionally, Django has shipped with memcached memcached D, that cash back in with multiple implementations, I think even so you can go there, like Django has an ORM. It can talk to stuff. So it has a cash back end as well. And it could talk to memcached. But it couldn't talk to Redis. And they found that the vast majority of people are using Redis. And they said, Well, why don't we have a back end for it? Well, guess what it's going to. So this was merged. And this whole conversation here around the PR and the issue is pretty interesting. So it starts out and says this PR aims to support as added support for Redis to be used as a caching back end with Django as Redis as the most popular caching back end, adding it to Django core dot, the Django core cache module would be a great addition for developers who previously had to rely on third party packages, and check out how they've got this little checklist and progress. These are the things for this PR to come along and work. So create the Redis Cache class, your pickle, serializer, etc, waiting for this other task here, some open ended documentation. So I don't think I've seen this really before, like this project tracking in the PR. looks really cool. Yeah. The other thing to note that this came in on May 23. And there's a large conversation, if you go there, there's Wow, 30 pages of conversation about it. And you can see it evolving, like okay, we've finally got the test pass, and we finally got to implement it. Now let's move on to the documentation now, etc, etc. And then finally, boom, it's September 15. It's three, three half months, something like that. It's close. So you can actually sort of track what the Jango team is doing for adding features like core important features to Django.

23:43 It's it's always so interesting to watch open source communities like this, especially on somewhat contentious issues where people disagree and how they manage these things. I think it's really impressive because a lot of teams that even meet in person regularly in our small teams still struggle with that kind of stuff. But these huge open source projects, manage it, and somehow they implemented the feature at the end. So pretty impressive.

23:43 Yeah, absolutely. It's very impressive. Also, I said this was from Carlton he participated a lot I'm not 100% sure that he was the originator this might be then Daniella. Daniella. Danielle Abbasi so sorry if I miss attributed credit there, but for whoever did this original issue, I think Carlton had put up so I'm not sure who was really sort of the initiator there. But I think it's cool. And it's also neat. How out in the open this whole thing is,

23:43 yeah, pretty open and open source.

23:43 That's right. I Brian, when she got me again,

23:43 um, I Are we done with her things? So

23:43 no, I think I got one more.

23:43 Oh, sorry. I totally I was for some reason in wrong order. Yes, Ethan Europe next. Sorry. Holy,

23:43 fine. So yeah, so I wanted to highlight Pep 612. So I happened upon this, I forget, there was some other Pep I was looking at, and they linked off to this one. But a little bit of background. A Pep is a Python enhancement proposal. It's basically like how ideas are proposed in terms of what to do with Python, as a community or as a language. And I recently have been really kind of diving in to type hinting in Python. So there's a surprising number of peps about type hinting. And what this one does is something I guess I didn't really realize I needed. It was a bit of an annoyance, but I didn't realize there was a fix coming. Basically, what it comes down to is, quite often you write functions that take in a function and return another function. So there's this example. This was the first case where they use it. Think here params. spec. I'll find a while I talked about this. But basically, you put a decorator time Yeah, yeah. What you do with decorators is you write functions that take in other functions and return a function that has the same signature, which is to say it takes in the same parameters of the same types and returns the same return type. It may have some other modifications to the function, but that's very frequent. And so sometimes what you want to say is my decorator, if I want to type the decorator say what types of things it takes in, it takes in something that is essentially a generic function type. Any kind of function is fine that takes in any parameters and returns any return type, as long as it returns the same thing. So it's like generics, which you would do with type bars. But in this case, you create something called a params. back, and then you pass that as the man I lost to where it is in here. Oh, here we go, this is what I wanted. So you pass it as the type of callable when you type the function that's taken in, and then you say, you're returning a callable with the same parameter specification, this P is a parameter specification. And you make essentially your call levels, generic on both this parameter specification and on the return value. So I know there's there's a lot to that. And I think for people who are typing everything every day, maybe this doesn't seem terribly pertinent. What I do, I said, I write a lot of Python packages for people to use. And it's important both for quality control, and so people know what the return values are, and what they should pass into functions to have a lot of typing. But really, what this got me thinking about a little bit is just that the Python typing ecosystem is still really evolving. Like for somebody who's not super close to following it, it appears that like, this is how Python works now, and maybe it's always been this way, but it really hasn't. And there's a lot of holes in how it works. There was no way to do this before. And this isn't finished yet. This is a pet, but it isn't implemented. And so right now you don't have a way to do typing for this particular feature. And then

23:43 yeah, for certain type information through different things, that is something we haven't done a lot of in Python. But yeah, as you call that generics and templates that that's like all you do. That's, that's their bread and butter, those things.

23:43 Yeah, and it's the same idea. But features that aren't there yet. So it's just kind of interesting to remember that this stuff is still being added, like keeping an eye on when this stuff comes in, it can really make things easier. And in the meantime, don't lose too much sleep, not being able to type certain things. If you can't type it perfectly. That's okay. I've actually been reading Luciana ramalho, his book flew in Python. And he makes that point really well, that it Python isn't a statically typed language, and you shouldn't get too carried away trying to type things as much as possible and helps you is worth it. But you shouldn't be religious about it.

23:43 Right. But if you are building tools, and you put this into there, eventually, yeah, it might help other people who consume your libraries. It might help the editors give better autocomplete and error checking and stuff.

23:43 And we catch bugs all the time. So as much as it's feasible, I think it's totally worth it. And actually, there's a couple other peps on that note of things still changing. There's a couple other peps that are worth looking at. There's a new more convenient way to write optional types. So right now again, I know

23:43 Oh, my gosh, this is so long. Yes.

23:43 Yeah. So you have to say optional left bracket, then the thing that is optional, and then right, optional bracket,

23:43 string or optional. user, whatever. Yeah. And you got to very comfortable Don't forget that. Yep,

23:43 that's true. You got to import it too. And so now there's, there's a pet proposing that you could just put a question mark, which I guess isn't a problem for the parser which is pretty nice. This one also is in process.

23:43 Maybe this was something that was needed the peg parser which recently went into omega three nine, right where it couldn't do it before. But yeah, that's good for now. But yeah, I you know, they have that in C sharp and they have that in Swift. And I just love like this thing, question mark, right, rather than a no check or specifying into question mark, rather than optional bracket of and

23:43 it's just, I didn't know that was in other languages that Okay, that makes a lot more sense. And

23:43 it's phonetic, right? Like, if it's, if it's an ENT, you just say ENT, if it's a into question, mark its end. Right? So exactly, just like speak it out really well. And maybe Okay, that could be not maybe not

23:43 obvious to me.

23:43 I really Oh, interesting. I feel like that's a nice a nice syntax. Maybe it isn't. Who knows, maybe that Pep won't get approved?

23:43 Yeah, I think I think it may not, but I do I do hope it does. I mean, it's the question mark, there's an end, or is there right? Like, is it there? You're not sure. Like, there's, there's some subtle symbolism there. So

23:43 I prefer the INT or none. But I like that

23:43 as well. Like, yeah, that's not that that's more convenient to write. But that's what the other

23:43 languages that support this. And I don't know, I didn't read that Pep well enough to know, there's a runtime behavior, not just the type specification behavior. So I could say x equals like user a question mark dot name, it'll either if the user is none, the name is none. Or it'll follow down that that path and say, okay, users, not none so then I'll say dot name. And all

23:43 that avoids the nun type has no attribute. Yeah, exactly. Nice. Yeah. Very cool.

23:43 So will is in the chat, and he's got Oh, it is the wrong one. I love that.

23:43 Hey, well, that's pretty good. All right. And then anything you want to tell us about one more before we wrap it up?

23:43 Oh, just another Pappy. I just another thing that is potentially changed how typing works. There's right now there's no way to specify if you've used to type dict which is to say, a dictionary with some keys having certain types. There was no way to specify what what keys were optional which ones weren't good either say they were all Optional, or they were all required, and there's nothing in between. But there's also a pep to do that. So just there's a lot of stuff on the horizon to keep an eye out for and these three peps, I think are a good reminder of that.

23:43 Yeah. Yeah. Very cool. All right now, now coming through to Brian. Yeah.

23:43 Now, so this was a, this is a suggestion by john Hagen, I just thought I'd throw it in as a, as a an extra, just one extra. So there, we've talked about the effort at Microsoft and Guido and others to make Python faster. And, like, there's a whole bunch of ideas up on a on a read in the faster see Python ideas in one of this links to a couple slide decks, talking about the making c faster, or making Python faster. And one of the things is slide deck from Guido in in it, he mentioned, various other optimizations, like maybe zero overhead exception handling. Well, that's neat, because that's already in 311. So in 311, we have Mark Shannon implementing zero cost exceptions. So if if, if you have a try statement that doesn't, doesn't catch anything, there's no cost to it. So that's

23:43 cool, I did a little playing around with this idea. And I wrote a program here that calls string upper, like 100 million times in a loop. And it does that also in a try except block with no errors. And so my understanding of this was that it will make entry in the tribal lock in the case, there's not an exception, cheaper, and I ran it 100 million times. And I got, you know, not actually the same, but it's really similar. But one of the other things, which I'm not doing in my example here, this is a just I'll put it in the show notes. The looking into this comment, Brian, is they talked about the number of the basically the size of the call stack. And some of the other things that happened in there about not pushing the exception onto the call stack or something unless it actually happens, and those kinds of things. So it's supposed to make function calls faster, as well. So even if my little example wasn't necessarily faster, and maybe, maybe something else, there's maybe other situations where it is nice. Yeah, even anything else you wanted to throw out there for people.

23:43 Well, one thing I did want to mention real fast about about the zero cost exception handling is, I think it's always tough to teach people about try except blocks and then introduce to them that they're actually pretty slow, especially if you use them in a function that gets called many times. And to be honest, I don't know the reasons for the internals being like that. So it's really nice to feel like that might not be true anymore, because they're a good practice to have to be able to say like, yeah, be careful when you write code, especially for people like data scientists who are day to day programmers to say like, Oh, it's good practice to use these, and you shouldn't have to worry about performance. So glad to see that.

23:43 Yeah, absolutely. And just following up on that real quick. If you look at the issue underlying this, was this the right one? Yeah, they, there's an issue that's linked in the show notes, and it actually shows you the disassembly into bytecode, of what it currently is, and what it is, it's going to be. And it's really, really similar. So you can see currently it does, like, the first thing he does is set up a Finally, and then stuff right at the beginning. But now just do like a no op, and then do return value in a good case. Otherwise, it will do a push exception, and then work with it, and so on. So it pushes off some of the bytecode operations that add like to the call stack, like pushing things on to it and so on at the CFL dot c level of C Python.

23:43 Yeah. Oh, that's very cool. Yeah. Well, the one thing I wanted to mention, well, I don't know if people have heard of pedalboard. I think Spotify just announced this recently. It's basically a Python package that lets you do some things you might usually do using an audio editing tool. And it's cool on its own. But I had just listened to if you're if it was last week, so the weeks before episode where Brett cannon was on, on Python bytes. And he talked about how, you know, anytime you see an issue with documentation, just put in a pull request, most of the time, it'll get accepted. And he said he's he's contributed like 200 or 300 repositories that way. So I found this last week. And then in this week, I was thinking about what I wanted to talk about on the show. So I went back to this link, and lo and behold, the last the last commit was made by Brett cannon and it's removing a stray brat backtick in the readme. So he really practices what he preaches. So he seems to be very active. He's one of only nine contributors to this and probably the rest work at Spotify. So good for him. Nice. Yeah,

23:43 that's fantastic. Nice to for that little bit of real time follow up. Alright, so I have a few extras. And again, I have my my banner for extras, extras extras. So a couple things here. Let's talk about something that Kelly Schuster Perez talked about, you know, she and Shawn doing the teaching Python podcasts and they're doing great work over there, too. One of the things that she found for teaching is this thing called irska. You probably haven't heard of this, I'm guessing. So your sketch is a project from Georgia Tech that teaches coding, but through like a DJ type of experience, she's got a cool video up there. It says five minutes and four lines of code. And I got this up there go. And so yeah, thanks, Tony for pointing that out. So here, I'll just play that what she created for everyone real quick.

23:43 Teaching, want to get folks involved through music and Python, that's a real cool project that you're sketch. And I told you good stuff about Django before, let me tell you some bad stuff. You might, you might meet little Bobby tables in the Django ORM. If you're running query set order by and passing some use of user input into what you might be ordering by, you might be ordering by backtick, semicolon drop table, dash dash or something like that, which you wouldn't want to do. Basically, there's a SQL injection vulnerability in Django, what is it 320 up to three to five and 33300, up to 313. but less than that, right? Less than three to five and less than 313 13. So if you have those, you definitely want to patch it straight away. That's a critical vulnerability. So

23:43 that's on untrusted input. So

23:43 that yes, that is untrusted input, don't freak, if you're not taking, what would you like to sort by please type here. But still, you know, it's easy enough to just do a GitHub update, just an update to the requirements. Now, if you're on your code is on GitHub, and this is the requirement, you pinned your version, you probably have already gotten this as a security announcement and an email sent to you. It's a jeans feature. It is such a nice feature. But if you don't pin your version, they're like, well, you're on the latest version. You're good, right? You won't know. So it still may slip through. Alright. Yeah. And Chris may on the live stream has some philosophical thoughts for us. He says sometimes he doesn't even trust his own input. Yes, we all we've all been

23:43 there. They don't inject yourself.

23:43 Alright, shall we wrap this up with some laughs? Yes, Brian, it's gonna take some roleplay. And again, a nice little cartoon for us. This is Qa 101. Speaking of the CVE I just spoke about and you know, if you fix a minor bug, you might get credit like whatever. We fixed a little tiny bug, right? format it in a log file, you fix a critical bug, like, wow, that seems super important. You've been doing good work this week, right? So here's, here's two developers in an open office sort of space, bring up the guy. I'll be the woman developer.

23:43 Okay. Which priority? Should I give this bug?

23:43 Is it easy to fix? Yep,

23:43 I'll fix it immediately.

23:43 critical, critical. finding the correct bug priority is key. They say so very nice. I'll link to that little cartoon in the show notes. Get it? Because you're going to get more credit for fixing critical bugs. And if you can fix it right away. That looks way more work. Much more work. Brian, I was like a fixed like medium bugs. Even and I took out the critical Exactly.

23:43 You do your T shirt sizing after you finish after you after you take all the work. You've seen. Everything you took was a large.

23:43 Yeah. So yeah, exactly. I keep asking people. So what are the points equal in hours? No, we can't talk about that.

23:43 I use powers of two. What do I know? Yeah.

23:43 Cool. Well, thanks, Ethan for coming on the show. It's fun. Yeah, this

23:43 is great. Thanks for having me. Yeah, it's

23:43 been fantastic to have you here. Thanks for being here, Brian. Thanks. As always. Thanks. Bye, everyone. Thanks for listening to Python bytes. Follow the show on Twitter via at Python bytes. That's Python bytes as in BYT. s. Get the full show notes over at Python by sarafem. If you have a news item we should cover just visit by them by sarafem and click Submit in the nav bar, we're always on the lookout for sharing something cool. If you want to join us for the live recording. Just visit the website and click live stream to get notified of when our next episode goes live. That's usually happening at noon Pacific on Wednesdays over at YouTube. On behalf of myself and Brian rockin This is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.

Back to show page