Transcript #252: Jupyter is now a desktop app!

00:00 Hey there, thanks for listening.

00:01 Before we jump into this episode, I just want to remind you that this episode is brought to you by us over at TalkBython Training and Brian through his pytest book.

00:10 So if you want to get hands on and learn something with Python, be sure to consider our courses over at TalkBython Training.

00:17 Visit them via pythonbytes.fm/courses.

00:21 And if you're looking to do testing and get better with pytest, check out Brian's book at pythonbytes.fm/pytest.

00:28 Enjoy the episode.

00:29 - Hello and welcome to Python Bytes where we deliver Python news and headlines directly to your earbuds.

00:33 This is episode 252, recorded September 29th, 2021.

00:38 I'm Michael Kennedy.

00:40 - And I'm Brian Okken.

00:41 - And I'm Ethan Swan.

00:42 - Ethan, welcome to Python Bytes.

00:45 You've been over on Talk Python where you talked about some really cool data science stuff and now you're over here.

00:50 So thanks for being here.

00:52 Tell people a bit about yourself.

00:53 - Yeah, I was on Talk Python 236.

00:56 So it was a while ago, but that was really cool.

00:57 I work for a company called 8451.

00:59 It's the data science subsidiary of Kroger.

01:02 And I'm a data scientist, but basically what I do is build tools, mostly which are in Python for our data science department.

01:08 So we have like 250 data scientists, pretty large department.

01:12 And I build like packages and some dashboard sort of things, just like various technology helper stuff for data science.

01:18 - Yeah, it sounds really fun.

01:20 And you all run, what we were talking about before we hit record, one of the, probably one of the larger data science groups out there, right?

01:27 I think of data science as being like, there's a couple of folks that are embedded with like a marketing team or a product team or the software development team a lot of times, but you are a properly large group of data scientists.

01:38 - I mean, in theory, that's what the whole company does.

01:40 So it's a very cool experience.

01:42 And often I think that's nice for the team I'm on because you don't usually get so many customers on internal tools.

01:48 We're building stuff for literally hundreds of people to use and it's a little bit like releasing software externally.

01:53 So it's, yeah, it's a lot of fun.

01:55 - Yeah. - Interesting.

01:56 Fantastic.

01:56 All right.

01:57 Well, we're definitely looking forward to having your insights here for the show.

02:01 Now, Brian, I do want to start off here.

02:03 I want to talk about some deck staining.

02:05 Thanks.

02:06 Yeah.

02:06 So, because those of us who are very attentive on Twitter saw that Brian kindly responded to somebody who sent us a message and said, Oh, I see you were talking about pallets.

02:17 We should also talk about deck stain and other DIY project resources.

02:22 And maybe you could put that stupid article on your blog.

02:25 You're like, we're not a blog.

02:26 We talk about palettes because it's on Flask.

02:28 And then in Twitter decided, oh, you are now classified under the home improvement category.

02:33 So are we changing our theme or what?

02:35 >> Apparently just me.

02:37 That's most people.

02:38 I've got a few new followers now.

02:40 And most of them are people that like to make things.

02:43 So.

02:44 >> Well, it's fun to make things as well.

02:46 But maybe we'll talk more about SQL and stuff like that.

02:49 What do you think?

02:50 >> Yeah.

02:50 So this was sent to us by Dave Cochessa.

02:54 Thanks, Dave.

02:54 I want to talk about SQL Fluff.

02:57 I had never heard of this, but it looks pretty cool.

03:01 SQL Fluff is a Python package that is basically a linter for SQL.

03:09 >> How interesting. I haven't really thought about linting SQL code, but it makes perfect sense.

03:14 >> Yeah. I don't really think about it too much either, but there's things like should you capitalize all the keywords and some people just like it like that.

03:24 There's both style guides around SQL, I assume there's style guides, and this helps you enforce it.

03:34 Not just style guides, but just looking for mistakes and things.

03:39 The page looks really slick.

03:42 I like the logo, the fluff logo.

03:44 One of the things that's great about it is the documentation.

03:48 The documentation looks wonderful.

03:51 One of the neat things about this is there's different rules or different dialects set up so that it treats different things like ANSI and Postgres and MySQL different.

04:03 I'm not sure if these are style differences or what they're doing different, but it's interesting that there is a difference there.

04:16 >> Well, one of the things that comes to mind for me, if this reports errors, and I suspect it probably does, One of the things that comes to mind for me is if using like Microsoft SQL Server and you're using a parameterized query because you don't want little Bobby tables in your school.

04:29 You would say at parameter name.

04:31 Whereas in with like MySQL or Oracle it'd be like a question mark right and I think one is illegal in the other syntax.

04:38 So at least in that regard I think I don't know for sure it's illegal but I'm pretty sure like it may be and it could be that you've got to say what type of parameterized specifications and other extensions are valid.

04:49 >> I think there's some keywords.

04:51 Are there some different keywords in some cases?

04:54 It would make sense to have to know the dialect.

04:56 >> Yeah. Also, like you were saying, if there really are big differences or even minor differences, there might be some queries that you don't run all the time, and so you're not sure if you switch databases, that they might be broken if you're trying to port.

05:10 Kind of cool. It has rules like a lot of linters, rules for failure, and I like the rules page because it talks about the rules, but also shows you the anti-pattern and best practice.

05:23 I like that style.

05:24 I don't know if I like the terms anti-pattern, and I really don't like the term best practice.

05:30 But nonetheless, what it's looking for and what you should do different is a good thing to have in the documentation. It's pretty cool.

05:38 >> I do like the anti-pattern aspect.

05:41 Maybe pattern? You can have an anti-pattern, you should have the pattern. I don't know.

05:44 >> I'm not sure. One of the things that's in the documentation, I can't remember where there is that people should be aware of.

05:51 Supposedly, this even though it has 1982, that's interesting stars, it's still in alpha phase.

06:00 There's a note here that says expect significant changes.

06:04 Just be aware of that.

06:06 >> Cool. It doesn't seem major because you're not doing a runtime behavior on it.

06:10 It's a thing you run against your code and then you look at the output.

06:13 >> I mean, maybe it's in your CI system or something.

06:16 Yeah, but it's not in production, right?

06:19 So like you want, you won't get called on a weekend because the site went down because this thing got automatically updated or something to that effect.

06:28 I guess it could have broken your queries, but you know, whatever.

06:32 And it's good to have an audience because we did have Paul from the chat say Ethan's correct.

06:39 There are different keywords between different SQL dialects.

06:41 Yeah, we use a, oh, sorry, Michael.

06:44 No, go ahead.

06:44 Great. We use a lot of SQL as I would assume most data science shops do.

06:48 But one what this made me think of was one contentious topic in people who write a lot of SQL, especially when you have a bunch of column names and you're selecting regularly, you know, five to ten columns.

07:00 The comma first.

07:01 I don't know if you've seen the approach where you do a new line, comma, column, comma, column.

07:06 So it lines up really nicely and it makes it easier to delete things.

07:09 That's a very common thing that people feel strongly about.

07:13 So I could imagine linter is being very handy to at least enforce one style throughout the company because you know, we don't have that Yeah, nice and then I was going to add that pantheroy on the Audience says it would be cool if there was a plugin for d beaver and sam morley Asks, I wonder if it checks if inputs are sanitized I don't know if it should but paul also asks if it validates for syntactical correctness beyond just style He does say that it catches errors in bad sql before it hits your database. So i'm going to go with yes That's pretty cool.

07:44 Yeah.

07:44 Ethan, I was thinking as I was watching Brian present this, that you probably do way more SQL than I do, even though I run and production websites that are backed by databases, not just because there's no SQL, but because I use ORMs and the data structure doesn't change.

08:00 But for data science, you're kind of in a more exploratory mode, right?

08:04 Yeah, I think it's pretty interesting because, you know, like listening to this podcast, people talk about using ORMs a lot, But in data science, you don't really think of data in that relational model as much.

08:15 I mean, you can, but like thinking of as of rows as objects is really not common.

08:20 So I feel like my relationship with databases is totally different.

08:22 My first couple years, I was mostly writing SQL, but it was it was literally just asking questions for analyses, which is such a different use case than what people use it for for web development.

08:32 Right?

08:33 Yeah, absolutely.

08:34 It's super different, super different.

08:35 But if you were to explore data, wouldn't it be nice to have a desktop application instead of a web browser for doing so?

08:43 So Jupyter, JupyterLab have got to be the most popular way that people interact with data on the data science side.

08:49 It's certainly an exploration stage anyway.

08:51 So super big news that is old news is new again, but better.

08:55 JupyterLab desktop app is a thing.

08:58 Like I can download JupyterLab.

09:00 It's an icon on my dock or on my taskbar.

09:03 I click it, it runs like an app, but inside of it is Jupyter Notebook, like the whole JupyterLab with terminal and Python consoles and kernels and all those things.

09:13 - That's cool.

09:14 - That's very nice.

09:15 - Yeah, have you played with this yet, Ethan?

09:17 - No, so I don't know how common this is, but I think for us at least, mostly people aren't working on their local machines.

09:25 They're really connecting to a session of Python on a remote server.

09:29 So mostly what we do is we fire up Jupyter on a remote server and then from our laptops, we hit that URL to actually look at the notebook.

09:36 So I'm not sure a desktop app would work as well for us, although maybe, it's definitely interesting.

09:41 And I wonder if there's some native features of desktop apps that are available that are gonna be a reason to switch.

09:47 - Well, what I would say right now is it's a really nice self-contained thing.

09:52 So I'll just read the description real quick.

09:54 JupyterLab app is a cross-platform, standalone application distribution of JupyterLab.

10:00 is a self-contained desktop application which bundles the Python environment and several popular libraries to use in scientific computing like surely pandas and NumPy and those kinds of things.

10:11 So what you get is you get just an app that's ready to go that you could just have somebody install and you can say here, open this notebook and run it.

10:19 And long as you're using core libraries and stuff like that, you don't have to think, okay, go to the terminal, you know, set up the environment and then type JupyterLab.

10:28 you need to activate the kernel and you got to do this and that.

10:31 It's just like it's a real simple, here's the thing, no nonsense type of app.

10:37 >> Yes, and you lost a whole bunch of people with just open the command line.

10:40 >> Yeah, that's so true.

10:42 >> Yeah. So you don't have to hear it.

10:43 It's on your dock, you click it just like you would with Word or Firefox or whatever and you're there.

10:49 It starts and manages the Jupyter server in the background.

10:53 There may be a whole host of command line arguments you can give it to say, like run, but use that server and other things along those lines or run and use this conda environment.

11:04 I didn't see any of those.

11:06 And so from what I can tell is it's kind of a local version of Jupyter.

11:11 So it might be super interesting for you all in your workflow.

11:14 One place where I think this would be really handy is, teaching beginners.

11:18 So I actually teach some Python, especially for data science classes at the university of Cincinnati.

11:23 And one thing that regularly is really confusing to people is that you can't double-click on a notebook file and have it open?

11:28 Because that's such a typical experience of files on a computer.

11:31 You double-click and there's an application that opens that file.

11:34 >> Interesting.

11:35 >> There are workarounds.

11:37 If you have Anaconda Navigator, it works, although it's a little hitchy.

11:42 But I would assume that if you have a desktop app, you'd be able to register that with the operating system, whatever that process is to say, like when I click on.ipynb, open it.

11:51 Because I find I have to teach students, no, start up Jupyter, Open your browser, navigate to that file in the browser.

11:58 - Were you in the wrong folder in the terminal when you ran JupyterLab?

12:01 Well, sorry, you're now locked out of that tree, that part of the tree of the folders.

12:06 - And then suddenly you're having a conversation about paths.

12:08 Yeah, you go down.

12:09 It really is like something I don't like to deal with.

12:11 So maybe this is what I should recommend for people when I teach.

12:14 - What I would recommend is just check it out and try.

12:17 So I do have a bit of a comment here from Dean out in the audience.

12:21 I like the concept of JupyterLab app, but I'm afraid it will be a VENV, virtual environment nightmare.

12:27 So what I found interesting is it's discovered, you know, when you're creating kernels for Jupyter, you have to run a command, I always forget it, and always have to duck, duck, go or search this to figure out how to do it again.

12:42 But I have to get the command to say, create this environment and then register that as so Jupyter finds that content environment, that VENV, right?

12:51 It's ipykernel install.

12:53 I have to do this all the time.

12:55 - Yes, exactly.

12:56 And I know that it's basically that, but the exact command, I always forget.

12:59 So that command, it seems like it picked up the ones that I had run previously for standalone terminal JupyterLab.

13:07 So the virtual environment story is the same as Jupyter itself without that.

13:13 I think all we're getting here is we're getting the libraries plus Python, plus the server starting all bundled together.

13:20 And it's basically the same as if you just run it on the command prompt.

13:24 - I think as long as, was it Dean?

13:26 As long as Dean doesn't wanna be starting Jupyter from the virtual environment, it should be fine.

13:31 Like when you said, Michael, about the kernels, that's the much more, I recommend people do it that way.

13:35 'Cause some people do like to just install Jupyter in whatever environment they work in and launch it there.

13:40 But I have a hard time imagining how that would work in this case.

13:43 - Yeah, I do as well.

13:44 And Dean makes the point that once you have to go and register all that kind of stuff, like when you're down in the terminal doing this, you've kind of lost those same people and that may well be the case.

13:53 But I can see, you know, this is sort of a first version of this.

13:57 I can see that those are some of the desktop things it could add, right?

14:00 It could add a setting section where you have a dialogue for managing these things and creating new ones and so on.

14:06 So, could be pretty neat.

14:08 - Yeah. - Definitely something to watch.

14:09 - All right, before we move on, Paul out in the audience has a quick question for you, Ethan, a tangential one.

14:15 Python has some really great SAST tools, like Bandit, but I'm not able to find good options for R.

14:22 And I know that you live in a world that does both R and Python.

14:26 - Yeah.

14:27 - Do you have any thoughts on this?

14:28 I have no ideas.

14:29 - I'm gonna come off as a fraud, but I don't know what SAST is.

14:33 I have to admit, like what I do, I know I said I'm a data scientist, but in some ways that's nominal.

14:40 Like really a lot of what I do is software development for the data scientists.

14:44 - Data scientists are your customer in a sense, or your target user, yeah.

14:48 - Yeah, so I think a lot of what I hear from users is that there are certain measurement tools and certain statistical tools that are available in R that take longer to get to Python.

14:56 So I wouldn't be surprised if that really is what's happening here, but I don't personally have any suggestions.

15:01 - Yeah, okay.

15:02 Yeah, so Bandit is like a tool that will scan for known security vulnerabilities, like leaving debug settings on in Django.

15:10 - Oh, see, I was wondering if that was, okay.

15:12 Then that, I also don't know.

15:13 That's a little different than what I was imagining.

15:16 - Yeah, awesome.

15:18 All right, well, since you got the floor, tell us about your first item.

15:22 - Sure, so I found this requests cache package in a newsletter recently.

15:29 And this might be a little bit of a shorter one 'cause unfortunately I haven't had a reason to use it yet.

15:33 But basically what this does, scrolling down here, is you can instantiate sessions just like you would with the traditional requests library.

15:41 So probably request is one of the most commonly used Python packages, I would guess.

15:47 For anybody who's not familiar, you use it to make HTTP requests, which is basically to bring anything back.

15:52 The tagline, I think, is HTTP for humans.

15:55 But it's just known for being easy to use and you can access the Internet.

15:58 But one thing that I have found is that, especially if I'm testing something in an interactive way, not mocking, but I really want to see if my code pulls back what I expect.

16:08 Sometimes I rerun the same request over and over, and I say, "Go get this, go get this, go get this." Often the same data, and sometimes that data is large.

16:17 And that takes a really long time.

16:18 So requests cache is a way of creating a session object that looks and acts the same.

16:23 But when you call a GET or a POST request on the same URL, with the same data, what you get back is actually just the cached version of that data.

16:34 So you're not waiting every time.

16:36 The first time you incur the network latency, and if the server has to do anything to compute the data, or if it's enough data that it takes some time to get to you, you wait for that.

16:45 but the second time everything runs instantly, which is really a big advantage.

16:48 I've done some things with web scraping where I'm building some kind of, I want to build a function that pulls some things down and makes or pull some things out of that.

16:57 But just waiting every time to run the function for it to pull from several different pages and do some computation on that, actually makes it pretty slow.

17:05 But if you were able to cache it like this, that'd be a lot faster.

17:08 >> Yeah, this is nice. I love the fact that it's just a stand-in replacement for the request session itself.

17:13 >> Yeah. If you scroll down a little more, it actually shows a way to do that with the regular requests library.

17:19 This actually scares me a little bit.

17:20 This is magical. What's going on here?

17:23 You just run a one-liner with requests cache, and then suddenly the requests library itself works differently.

17:28 I wonder if that's a monkey patching or what's going on there.

17:31 >> It probably is.

17:32 >> But it is really slick.

17:34 I would imagine I'll have a reason to use this soon, but I haven't tested that yet.

17:39 It does offer a lot of configuration options.

17:41 One thing I thought was a good idea to look at is an expiration date.

17:46 That's like when should you invalidate the cache and actually pull again, because you maybe should trust that the website is sending you all the same stuff today.

17:53 But if you rerun your code in a week, make sure that it still responds the same way.

17:57 It's got some nice options like that.

17:59 >> I really like this.

17:59 >> That's interesting. You could use it not even just for testing, it could be for actual data, but you know it's not getting updated very often.

18:07 >> Yeah, for large data is what I was imagining.

18:10 Yeah, like I said, there's been some times where I've pulled things from APIs where they send back a lot of data and you don't want to be waiting for that.

18:17 Or even you just want to make sure that multiple calls to it are getting the same data, even if it does change.

18:22 That's true.

18:23 Yeah.

18:24 So keep consistency.

18:25 Interesting.

18:26 This reminded me a little bit of the, I don't know if people are familiar with the @cache or LRU cache.

18:32 It used to be another new one just called @cache in the FuncTools module built in Python.

18:37 that's very, very handy once you know it's there.

18:40 'Cause often you have a function that you don't wanna recompute the work for.

18:43 And this is almost like somebody rewrote requests with cache in it, which is pretty cool.

18:47 - Yeah, it's got a lot of nice features.

18:49 You know, I think a question from a handful out in the audience, can it cache to Redis?

18:54 Because production in memory production caching, you could blow it up, right?

18:59 Blow up the memory.

18:59 So a couple of things that stood out to me that were interesting there was, yeah, you could throw a Functools LRU cache decorator onto an expensive thing, which is fine, but that's in memory, right?

19:11 And plus things have to be hashable and whatnot, but you could do that, but it's in memory.

19:16 And a lot of times if you have scale out as you do on web apps, like in production, as in Brian was talking about, you have web farms, like five or 10 copies of Microwizky or something running.

19:28 So then there's still five times you got to do it before it really gets cached.

19:32 And then also it goes to SQLite.

19:35 So it gets stored to disk, right?

19:37 So it's not even in memory, it's on disk.

19:39 So like you said, there's other backends as well, but I think having just by default, going to a SQLite file with an possible expiration means you could just turn this on and leave it.

19:49 Expire after a day, go.

19:50 Tell us about the backends.

19:51 There's more than just SQLite.

19:53 - Yeah, it does seem like you have some options.

19:54 I mean, like I said, I haven't had a reason to use this, so I haven't toyed around with all these, but the way this is documented leads me to believe that it really is just a drop-in replacement that you can configure what you want to use as your backend.

20:05 And I do wonder, so yeah, what you were saying, Michael, about having multiple instances, I do wonder how that would work.

20:10 Would it check to see if any of the instances had cached this yet?

20:14 Would it like proactively go reach out to the cache?

20:16 Or would it assume--

20:17 - Yeah, well, I think if you have the memory one, it's gonna be a hassle, right?

20:19 Like one of the options is memory, but all the other ones, file system, GridFS, Redis, SQLite, those are all support, you know, concurrency, they support.

20:29 - Yeah, exactly.

20:30 - So then it will scale across process seamlessly.

20:33 Yeah, so that could be actually really helpful for something like that, where you have a distributed set of workers, yeah.

20:38 - Yeah, for sure.

20:39 Let's see some fun stuff about your monkey patching comment.

20:43 Dean says, "Monkey patching is like having a real monkey.

20:45 "It's very cool when other people have it, "but having it in my house is scary." And yeah, Sam just has too much experience at the zoo, I think, with that as well.

20:55 So yeah, monkey patching's a little sketch.

20:58 - Nice.

20:59 - All right, Brian, you're up next.

21:01 >> What do we got next? I did something dumb the other day.

21:09 I went ahead and I pushed a new package out on PyPI.

21:15 Really, I was just trying to remember the whole process because I wanted to just remind myself of if I have something new, something cool I wanted to share, how do I get it out there to PyPI?

21:28 I was walking through that process and I was doing it for a plugin.

21:31 - Your own typo squatting.

21:33 - Apparently.

21:34 So I published pytest Slow, and then, who was it, Brian Skin said, "Cool, but maybe pytest Skip Slow would be better." And I'm like, oh man, that is a better name.

21:48 'Cause that's what it does.

21:50 It skips the slow tests by default.

21:52 So, and this is totally lifted from the pytest documentation about, they have this example, but nobody's written a plugin for it.

21:59 I did this, it's a little tiny thing.

22:02 I renamed it, but how do you rename it?

22:05 I went out and searched, so how do you rename something in PyPI? You can't really do it.

22:09 But you can create another one.

22:12 This is nice. Well, who was it?

22:15 Simon Willison wrote this up.

22:18 It's a PyPI renamed cookie cutter template.

22:21 I didn't actually use the template, but I did use these steps.

22:25 The steps really are create a renamed version of the package, which I did, then publish it to PyPI under the new name and create a final release for the old name that points to the new one and depends on it and have dependencies.

22:40 There's somebody installed the old one, they'll really get the new one.

22:43 It sounds more complicated than it is.

22:45 It's just a few steps, but there's a cookie cutter you can use.

22:49 The cookie cutter uses setup tools and I didn't want to do that.

22:54 I just basically copied the entire thing.

22:58 Then he's got a demo, so if you look at it, so if you go to the old version, it'll just have a thing that says, "Hey, I'm going to the new one now." I did that and it was neat.

23:11 I really appreciate the steps and it's all good.

23:16 >> Yeah, that's cool. You can also use it for aliases.

23:19 You can install BS4 or BeautifulSoup4, Right. And it's kind of the same.

23:24 Oh, is that, is that how they do that?

23:26 I I'm guessing, I don't know, but it sounds like the same.

23:29 I didn't know that.

23:30 So now, so now when I go to Python, if you go to the old one, it just shows it's now a new name.

23:36 Go to the other one instead.

23:37 So, but if I install the old one, it, it kind of just pulls in the new one.

23:41 Yes.

23:41 Yeah.

23:42 Yeah.

23:43 Very cool.

23:43 Yeah.

23:43 Brian, you were, you were refreshing on PyPI, but I actually just pushed my first ever package to PyPI a couple of weeks ago.

23:49 and so that was a bit of a trial, but I was amazed at how straightforward it is.

23:53 The documentation is excellent.

23:55 It really is pretty seamless actually for somebody who's never done it before.

23:58 Who knows? Hopefully, I don't make any mistakes on the one package I have and need to read it.

24:02 >> The immutability of it is a little scary, but yeah.

24:06 >> For me, the hard part was just understanding that it really was pretty simple.

24:11 Then also getting the hashes right.

24:14 You have to get signatures and stuff to make sure that you can push to the PyPI correctly.

24:21 - Yeah, but even the documentation there, it's a little intimidating, but it actually turned out to be only a few minutes of work.

24:27 So that was pretty nice, good for them.

24:29 I guess PyPI is the people to praise for that.

24:32 - Yeah, so what was your package?

24:33 - Oh, it's called Premark.

24:36 It's a spinoff of a JavaScript library for making slides.

24:41 And I just make a lot of slides for teaching.

24:43 And I actually found an existing package by, Man, I want to.

24:49 Yeah, here it is.

24:50 It is not ready.

24:51 That's why it's a release candidate.

24:53 OK, but I I yeah, I based it on this existing package by at Tyler Dave on GitHub and talk to him a little bit about it.

25:00 He had already built a really lightweight tool and I just expanded on it.

25:03 But I like to write my slides in markdown, which is really what this is for.

25:08 You write your slides and markdown in a bunch of different files.

25:11 It stitches them together and creates a what's called remark JS presentation.

25:14 So I use this for my own teaching.

25:16 But it really is largely a sample project to just like learn how to use PyPI and things like that.

25:22 Okay.

25:22 Yeah, very cool. All right. Up next, we have caching. Oh, wait, we just talked about caching.

25:28 No, I have more caching. So Django, I have two pieces of news on Django. This one comes from Carlton Gibson, one of the Django guys, and also one of the hosts at Django Chat, the podcast.

25:42 So they are adding a Redis cache backend to Django.

25:47 So traditionally, Django has shipped with memcache, memcache D, that cache backend with multiple implementations, I think even.

25:57 So you can go there like Django has an ORM, it can talk to stuff.

26:00 So it has a cache backend as well.

26:02 And it could talk to memcache, but it couldn't talk to Redis.

26:06 And they found that the vast majority of people are using Redis.

26:09 And they said, well, why don't we have a backend for it?

26:11 Well, guess what?

26:12 it's going to.

26:13 So this was merged and this whole conversation here around the PR and the issue is pretty interesting.

26:18 So it starts out and says, this PR aims to add support for Redis to be used as a caching backend with Django as Redis is the most popular caching backend, adding it to Django.core.cache module would be a great addition for developers who previously had to rely on third-party packages.

26:38 And check out how they've got this little checklist and progress.

26:41 These are the things for this PR to come along and work.

26:44 So create the Redis cache class, do a pickle serializer, et cetera, et cetera.

26:48 Waiting for this other task, here's some open-ended documentation.

26:52 So I don't think I've seen this really before, like this project tracking in the PR.

26:58 - I liked it. - It's really cool.

26:59 - Yeah, I do too.

27:00 The other thing to note that this came in on May 23rd and there's a large conversation.

27:04 If you go there, there's 30 pages of conversation about it and you can see it evolving.

27:10 Like, okay, we finally got the test pass and we finally got it implemented.

27:13 Now let's move on to the documentation now, et cetera, et cetera.

27:16 And then finally, boom, September 15th, that's three, three and a half months, something like that.

27:21 It's closed.

27:22 So you can actually sort of track what the Django team is doing for adding features, like core important features to Django.

27:29 It's, it's always so interesting to watch open source communities like this, especially on somewhat contentious issues where people disagree and how they manage these things.

27:37 I think it's really impressive because a lot of teams that even meet in person regularly in our small teams still struggle with that kind of stuff, but these huge open source projects manage it.

27:47 Somehow they implemented the feature at the end.

27:48 So pretty impressive.

27:50 - Yeah, absolutely.

27:51 It's very impressive.

27:51 Also, I said this was from Carlton.

27:53 He participated a lot.

27:54 I'm not a hundred percent sure that he was the originator.

27:57 This might be Daniel Abassi.

28:02 So sorry if I misattributed credit there, But for whoever did this, the original issue, I think Carlton had put up.

28:09 So I'm not sure who was really sort of the initiator there.

28:12 But I think it's cool.

28:13 And it's also neat how out in the open this whole thing is.

28:17 - Yeah.

28:18 - Putting the open in open source.

28:20 - That's right.

28:20 Hi, Brian, what you got?

28:22 - It's me again.

28:22 Are we done with our things?

28:26 So we were gonna do-

28:27 - Oh wait, no, I think I got one more.

28:29 - Oh, no, sorry.

28:29 I totally, I was for some reason in wrong order.

28:31 Yes, Ethan, you're up next, sorry.

28:33 - Totally fine.

28:34 So yeah, so I wanted to highlight PEP 612.

28:38 So I happened upon this, I forget, there was some other PEP I was looking at and they'd linked off to this one.

28:44 But a little bit of background, a PEP is a Python Enhancement Proposal.

28:48 It's basically like how ideas are proposed in terms of what to do with Python as a community or as a language.

28:54 And I recently have been really kind of diving into type hinting Python.

28:59 So there's a surprising number of PEPs about type hinting.

29:01 And what this one does is something, I guess I didn't really realize I needed.

29:05 It was a bit of an annoyance, but I didn't realize there was a fix coming.

29:08 Basically, what it comes down to is quite often you write functions that take in a function and return another function.

29:16 There's this example.

29:19 Where's the first case where they use it?

29:22 Think here, param spec.

29:24 I'll find it while I talk about this.

29:27 >> A lot of decorator time.

29:29 >> Yeah. What you do with decorator is you write functions that take in other functions and return a function that has the same signature, which is to say it takes in the same parameters of the same types and returns the same return type.

29:42 It may have some other modifications of the function, but that's very frequent.

29:45 Sometimes what you want to say is my decorator, if I want to type the decorator, say what types of things it takes in, it takes in something that is essentially a generic function type.

29:55 Any kind of function is fine that takes in any parameters and returns any return type, as long as it returns the same thing. So it's like generics, which you would do with type vars.

30:04 But in this case, you create something called a param spec, and then you pass that as the...

30:13 Oh man, I lost it where it is in here. Oh, here we go. This is what I wanted. So you pass it as the type of callable when you type the function that's taken in, and then you say you're returning a callable with the same parameter specification. This p is a parameter specification. And you may make essentially your callables generic on both this parameter specification and on the return value.

30:38 So I know there's a lot to that.

30:40 And I think for people who are typing everything every day, maybe this doesn't seem terribly pertinent.

30:46 What I do, I said I write a lot of Python packages for people to use.

30:49 And it's important both for quality control and so people know what the return values are and what they should pass into functions to have a lot of typing.

30:57 But really, what this got me thinking about a little bit is just that the Python typing ecosystem is still really evolving.

31:04 Like for somebody who's not super close to following it, it appears that like this is how Python works now and maybe it's always been this way, but it really hasn't.

31:12 And there's a lot of holes in how it works.

31:14 There was no way to do this before and this isn't finished yet.

31:16 This is a pep, but it isn't implemented.

31:19 And so right now you don't have a way to do typing for this particular feature.

31:23 And that-- - Yeah, flowing type information through different things.

31:27 That is something we haven't done a lot of in Python.

31:29 - But as you called out generics and templates, that's like all you do.

31:33 That's the bread and butter of those things.

31:35 - Yeah, and it's the same idea, but features that aren't there yet.

31:37 So it's just kind of interesting to remember that this stuff is still being added.

31:41 Like keeping an eye on when this stuff comes in, it can really make things easier.

31:45 And in the meantime, don't lose too much sleep not being able to type certain things.

31:49 If you can't type it perfectly, that's okay.

31:52 I've actually been reading Luciano Romalho's book, "Fluent Python," and he makes that point really well, that Python isn't a statically typed language and you shouldn't get too carried away trying to type things.

32:02 As much as is possible and helps you is worth it, but you shouldn't be religious about it.

32:07 - Right, but if you are building tools and you put this into there eventually, it might help other people who consume your libraries, it might help the editors give better auto-complete and error checking and stuff.

32:18 - And we catch bugs all the time.

32:20 So as much as is feasible, I think it's totally worth it.

32:23 And actually there's a couple other peps on that note of things still changing.

32:26 There's a couple other peps that are worth looking at.

32:28 There's a new, more convenient way to write optional types.

32:32 So right now you can say, I know.

32:34 - Oh my gosh, I've wanted this for so long, yes.

32:37 - Yeah, so you have to say optional left bracket then the thing that is optional and then right bracket.

32:42 - Like optional bracket string or optional bracket user or whatever, yeah.

32:45 And you got to import optional, don't forget that, yeah.

32:47 - That's true, you got to import it too.

32:49 And so now there's a PEP proposing that you could just put a question mark, which I guess isn't a problem for the parser, which is pretty nice.

32:55 This one also is in process.

32:58 - Maybe this was something that was needed, the peg parser, which recently went into, - Oh, maybe. - Was that 3.9, right?

33:03 Where it couldn't do it before, but maybe it can now. - Yeah, that's a good point.

33:06 - But yeah, you know, they have that in C# and they have that in Swift, and I just love like this thing question mark, right?

33:12 Rather than a null check or specifying, into question mark rather than optional bracket of int.

33:18 It's just clear. - I didn't know that was in other languages.

33:20 That, okay, that makes a lot more sense.

33:21 - And it's phonetic, right?

33:23 Like if it's an int, you just say int.

33:25 if it's an int question mark, it's int.

33:27 - Right, so you can even just like speak it out really well.

33:29 Like int, maybe, okay, that could be null.

33:31 - Maybe. - Could be none.

33:32 - Yeah, it's not obvious to me.

33:35 - Really, oh, interesting.

33:35 I feel like that's a nice syntax, but maybe it isn't.

33:38 Who knows, maybe that won't get approved.

33:40 - Yeah, I think it may not, but I do hope it does.

33:43 I mean, it's the question mark.

33:44 There's an int or is there, right?

33:45 It's like, is it there?

33:46 You're not sure.

33:47 There's some subtle symbolism there.

33:50 - See, I prefer the int or none.

33:54 I like that as well.

33:55 I agree.

33:56 - Yeah, that's not bad.

33:57 - Now that that's more convenient to write, but that's what that's all about.

33:58 - And the other languages that support this, and I don't know, I didn't read that PEP well enough to know there's a runtime behavior, not just a type specification behavior.

34:06 So I could say x equals like user question mark dot name.

34:11 It'll either, if the user is none, the name is none, or it'll follow down that path and say, okay, user is not none, so then I'll say dot name.

34:20 - Oh, that avoids the none type has no attribute.

34:22 - Yeah, yeah, exactly.

34:23 - That's really nice.

34:23 - Yeah.

34:24 - Wow, very cool.

34:25 - So Will is in the chat and he's got, oh, I did the wrong one.

34:32 I love that.

34:32 (laughing)

34:35 - Hey, Will.

34:36 - That was pretty good.

34:38 - All right, and then Ethan, you wanna tell us about one more before we wrap it up?

34:41 - Oh, just another pep, yeah.

34:42 Just another thing that is potentially changed how typing works.

34:46 Right now there's no way to specify if you've used a type dict, which is to say a dictionary with some keys having certain types, there was no way to specify what keys were optional and which ones weren't.

34:59 You could either say they were all optional or they were all required and there's nothing in between.

35:02 But there's also a PEP to do that.

35:04 So just there's a lot of stuff on the horizon to keep an eye out for.

35:07 And these three peps I think are a good reminder of that.

35:10 - Yeah. - Yeah.

35:11 Yeah, very cool.

35:12 All right, now, can we throw it to you, Brian?

35:14 - Yeah, now, so this was a suggestion by John Hagan I just thought I'd throw it in as an extra, just one extra.

35:24 We've talked about the effort at Microsoft and Guido and others to make Python faster.

35:30 There's a whole bunch of ideas up in the faster CPython ideas.

35:38 This links to a couple of slide decks talking about making Python faster.

35:45 One of the things is a slide deck from Guido.

35:49 In it, he mentions various other optimizations like maybe zero overhead exception handling.

35:56 Well, that's neat because that's already in 3.11.

36:00 In 3.11, we have Mark Shannon implementing zero-cost exceptions.

36:07 If you have a try statement that doesn't catch anything, there's no cost to it.

36:13 >> That is very cool. I did a little playing around with this idea, and I wrote a program here that calls string upper, like a 100 million times in a loop, and it does that also in a try-except block with no errors.

36:27 So my understanding of this was that it will make entering the try block in the case there's not an exception cheaper, and I ran it a 100 million times, and I got not exactly the same, but it's really similar.

36:40 But one of the other things which I'm not doing in my example here, this is a gist, I'll put it in the show notes.

36:46 The looking into this comment, Brian, is they talk about the number of basically the size of the call stack and some of the other things that happen in there about not pushing the exception onto the call stack or something unless it actually happens and those kinds of things. So it's supposed to make function calls faster as well. So even if my little example wasn't necessarily faster, maybe something else.

37:09 There's maybe other situations where it is nice.

37:12 - Cool. - Yeah.

37:14 Ethan, anything else you wanna just throw out there for people?

37:17 - Well, one thing I did wanna mention real fast about the zero cost exception handling is, I think it's always tough to teach people about try except blocks and then introduce to them that they're actually pretty slow, especially if you use them in a function that gets called many times.

37:29 And to be honest, I don't know the reasons for the internals being like that.

37:33 So it's really nice to feel like that might not be true anymore, because they're a good practice to have to be able to say like, be careful when you write code, especially for people like data scientists who aren't day-to-day programmers to say like, "Oh, it's good practice to use these and you shouldn't have to worry about performance." So glad to see that.

37:47 - Yeah, absolutely.

37:48 And just following up on that real quick.

37:51 If you look at the issue underlying this, where was it?

37:56 Is this the right one?

37:57 Yeah, there's an issue that's linked in the show notes and it actually shows you the disassembly into bytecode of what it currently is and what it's going to be.

38:06 And it's really, really similar.

38:09 So you can see currently it does like, The first thing he does is set up a finally and then stuff right at the beginning.

38:15 But now just do like a no op and then do a return value in the good case.

38:19 Otherwise it'll do a push exception and then work with it and so on.

38:23 So it pushes off some of the bytecode operations that add like to the call stack, like pushing things onto it and so on at the c of al.c level of CPython.

38:34 - Yeah, oh, that's very cool.

38:36 - Yeah.

38:37 - Well, the one thing I wanted to mention, - Well, I don't know if people have heard of pedal board.

38:41 I think Spotify just announced this recently.

38:43 It's basically a Python package that lets you do some things you might usually do using an audio editing tool.

38:48 And it's cool on its own, but I had just listened to, I forget if it was last week or the weeks before episode where Brett Cannon was on, on Python Bytes.

38:58 And he talked about how, you know, anytime you see an issue with documentation, just put in a pull request.

39:03 Most of the time it'll get accepted.

39:04 And he said he's contributed like 200 or 300 repositories that way.

39:09 So I found this last week, and then in this week, I was thinking about what I wanted to talk about on the show, so I went back to this link, and lo and behold, the last commit was made by Brett Cannon, and it's removing a stray back tick in the readme.

39:21 So he really practices what he preaches.

39:23 So he seems to be very active.

39:25 He's one of only nine contributors to this, and probably the rest work at Spotify.

39:29 So, good for him.

39:31 - Nice. - Yeah, that's fantastic.

39:32 Nice for that little bit of real-time follow-up.

39:35 - Fantastic. - Yeah.

39:36 - All right, so I have a few extras, and again, I have my banner for extras, extras, extras.

39:42 So a couple of things here.

39:43 Let's talk about something that Kelly Schuster-Perez talked about, you know, she and Sean doing the Teaching Python podcast, and they're doing great work over there.

39:54 So one of the things that she found for teaching is this thing called Ear Sketch.

40:00 You probably haven't heard of this, I'm guessing.

40:02 So Ear Sketch is a project from Georgia Tech that teaches coding, but through like a DJ type of experience.

40:11 She's got a cool video up there that says five minutes and four lines of code.

40:14 And I got this up there going.

40:16 So yeah, thanks, Tony, for pointing that out.

40:18 So here, I'll just play what she created for everyone real quick.

40:21 [MUSIC - EAR SKETCH, "FIVE MINUTES AND FOUR LINES OF CODE"]

40:23 I can see the look that's written on your face.

40:25 Wonder how I still fit this crown.

40:27 I can see the look that's written on your face.

40:29 Wonder how I still fit this crown.

40:31 I can see the look that's written on your face.

40:33 - People are teaching, wanna get folks involved through music and Python.

40:37 That's a real cool project that Ear Sketch.

40:39 And I told you good stuff about Django before.

40:41 Let me tell you some bad stuff.

40:43 - Oh no.

40:44 - You might meet little Bobby Tables in the Django ORM.

40:49 If you're running query set, order by, and passing some piece of user input into what you might be ordering by, you might be ordering by backtick, semicolon, drop table, dash dash, or something like that, which you wouldn't want to.

41:05 So basically there's a SQL injection vulnerability in Django.

41:09 What is it?

41:10 320 up to 325, and 300 up to 313, but yeah, less than that, right?

41:20 Less than 325 and less than 313.

41:23 So if you have those, you definitely want to patch it straight away.

41:27 that's a critical vulnerability.

41:29 So check that out.

41:30 - That's on untrusted input.

41:32 So.

41:33 - Yes, that is untrusted input.

41:35 Don't freak if you're not taking, what would you like to sort by?

41:38 Please type here.

41:39 But still, you know, it's easy enough to just do a GitHub update, just an update to the requirements.

41:47 Now, if you're on, your code is on GitHub and this is the requirement, you pinned your version, you probably have already gotten this as a security announcement and an email sent to you.

41:58 - Yeah, that's such a nice feature.

42:00 - It is such a nice feature.

42:01 But if you don't pin your version, they're like, well, you're on the latest version.

42:03 You're good, right?

42:04 You won't know.

42:05 So it still may slip through.

42:06 All right.

42:07 Yeah, and Chris May on the live stream has some philosophical thoughts for us.

42:11 He says sometimes he doesn't even trust his own input.

42:14 Yes, we've all been there.

42:16 - Chris, don't inject yourself.

42:17 - All right, shall we wrap this up with some laughs?

42:22 - Yes.

42:23 Brian, this is gonna take some role playing again, a nice little cartoon for us.

42:26 This is QA 101.

42:28 Speaking of the CVE I just spoke about, and you know, if you fix a minor bug, you might get credit, like whatever.

42:35 We fixed a little tiny bug, right?

42:36 Formatted in a log file.

42:38 You fix a critical bug, like wow, that seems super important.

42:41 You've been doing good work this week, right?

42:42 So here's two developers in a open office sort of space.

42:46 Brian, you be the guy, I'll be the woman developer.

42:49 - Okay, which priority should I give this bug?

42:51 Is it easy to fix?

42:53 - Yep, I'll fix it immediately.

42:56 - Critical, critical.

42:58 Finding the correct bug priority is key, they say.

43:01 So very nice, I'll link to that little cartoon in the show notes.

43:04 - Oh, get it.

43:05 - Because you're gonna get more credit for fixing critical bugs.

43:08 And if you can fix it right away.

43:09 - Yeah, it looks like you did way more work.

43:11 - You did so much more work.

43:12 Brian over there only fixed like medium bugs.

43:16 Ethan and I took out the critical ones.

43:17 - Exactly, you do your T-shirt sizing after you finish.

43:20 After you take all the work, you assume everything you took was a large.

43:23 Yeah.

43:25 Exactly.

43:27 I keep asking people, "So what are the points equal in hours?" No, we can't talk about that.

43:32 Okay.

43:34 Do I use powers of two?

43:36 What do I do?

43:38 Cool. Well, thanks, Ethan, for coming on the show.

43:40 It was fun.

43:41 Yeah, this was great. Thanks for having me.

43:42 Yeah, it's been fantastic to have you here.

43:44 Thanks for being here.

43:45 Brian, thanks as always.

43:46 Thanks.

43:47 Bye, everyone.

43:48 to Python bytes, follow the show on Twitter via at Python bytes, that's Python bytes as in b y t s, get the full show notes over at Python bytes.fm. If you have a news item we should cover, just visit Python bytes.fm and click Submit in the navbar. We're always on the lookout for sharing something cool. If you want to join us for the live recording, just visit the website and click live stream to get notified of when our next episode goes live. That's usually happening at noon Pacific on Wednesdays over at YouTube.

44:17 On behalf of myself and Brian Okken, this is Michael Kennedy.

44:21 Thank you for listening and sharing this podcast with your friends and colleagues.