Transcript #237: Separate your SQL and Python, asynchronously with aiosql

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 237, recorded June 9, 2021.

00:09 I'm Brian Okken.

00:10 I'm Michael Kennedy.

00:11 And I'm Mike Groves.

00:12 Hey Mike, thanks for showing up today. Tell us a little bit about yourself.

00:15 Yeah, well thank you guys for having me. This is actually really fun.

00:18 My name is Mike Groves. I am leading the engineering at a company called Assembly AI.

00:23 We are mainly a Python shop, but we do a lot.

00:28 We do speech-to-text for developers. We're an API company.

00:33 So you can really think of us as we are trying to make speech-to-text and all of the ways that you can take that text and do things with it really easy for developers

00:44 so that they can integrate this value into their products and make their customers happier.

00:49 Yeah, that's really cool.

00:50 I had somebody who, you know, I had spoken to you guys at Assembly AI for some Talk Python stuff,

00:56 but someone else that I'm working with reached out to me and said, hey, I'm doing this stuff with this Assembly AI API and I don't know how to save the file that well.

01:04 Can you help me? I'm like, oh, this looks really simple. This is really nice.

01:07 So yeah, you guys are doing good work.

01:08 Yeah, we try to make it really simple.

01:09 Yeah.

01:10 Yeah, fantastic.

01:11 Brian, shall we kick it off?

01:13 I think we should.

01:14 With a TUI.

01:15 Oh, yeah.

01:16 We've heard of a GUI.

01:16 A GUI, a graphical user interface.

01:18 And this new trend of taking stuff that happens in the terminal and making it nicer.

01:25 We've covered things like Rich and stuff before, right?

01:27 Yeah, a lot.

01:28 I love Rich.

01:29 Yeah, I do too.

01:30 And I'm a super, super big fan.

01:32 So I want to cover something that's like a derivative of Rich from Will McGann.

01:36 See, here we are messing up names already.

01:40 Sorry, Will.

01:41 And the idea is if I wanted to create like Emacs or a richer UI with something like Rich,

01:48 I can sort of control stuff on the screen and how it prints, but it's not full on curses,

01:53 right?

01:53 It's not, I want to write in this section over here and I want to dock this other bit to the

01:57 right and have Rich automatically put stuff into it and so on.

02:00 So I introduce you to Textual.

02:03 It was called rich.tui, T-U-I, as in text, terminal user interface, but it's now called Textual.

02:10 And the idea is we can have these cool things.

02:12 Like I would like a thing to be a header of my terminal app and the footer of my terminal app.

02:18 And I want to dock a 30 column wide section to the left and then just fill out the rest

02:23 in a little area that just takes the rest of the space.

02:25 How cool is that?

02:26 Yeah, this is great.

02:27 Yeah.

02:28 And then each one of these areas is written to and controlled by Rich.

02:32 So all the nice stuff that we've already talked about with Rich and the really nice

02:37 things you get from there.

02:38 So, you know, if people remember like Rich lets you have spectrums and like centered text

02:45 or right aligned text and tables and just all sorts of good stuff.

02:49 So think of doing that.

02:51 But on top of this, like it's almost like a layout for terminals.

02:55 Yeah, he's kind of rethinking how to do terminal stuff.

02:58 So this, I know you can do all of this stuff with Curses libraries and things like that,

03:03 but Curses isn't always easy on all platforms.

03:07 So he's rethinking all of it.

03:10 And I think it's really kind of fun.

03:12 Yeah.

03:12 I do want to point out something here really quick.

03:16 There's a comment in the readmeant says, this project is currently a work in progress and

03:20 may not be usable for a while.

03:22 So, you know, just kind of maybe watch the repo and get notifications and stuff and see,

03:27 you know, keep your eye out for it.

03:29 On the other hand, this kind of thing with these early stages, a lot of people contact us and

03:34 say, I really want to get into open source.

03:35 I want to contribute to some project.

03:38 What should I work on?

03:39 Should I work on Django?

03:40 It's like, well, that's going to be a little bit complicated.

03:42 It's highly polished and there's a ton of dependencies like projects like this that

03:46 are in the early stages are really good for jumping in and getting involved if it sparks

03:50 your interest.

03:51 So, yeah.

03:52 And there's a lot of room for people to do things like out of the testing, do documentation,

03:57 test on different platforms and really help it so that Will can concentrate on doing more

04:01 features.

04:02 Yeah.

04:03 Yeah, exactly.

04:03 Even documentation, like a little tutorial or whatever.

04:06 And Kim out in the live stream, Kim says, I would argue Curses isn't easy anywhere.

04:11 I agree.

04:12 It always feels like, oh boy, we're going down this path.

04:15 Mike, what do you think?

04:16 Well, it looks like a really cool library.

04:19 And I think for me, I try to think of what would I use it for and certainly a lot of internal

04:24 tooling that we could use this for so that we can make better ways of monitoring our machine

04:31 learning systems and how that all works.

04:33 And so, yeah, I think that would be something I would like to look at.

04:37 And also just a lot of times, too, is just trying something out, even if you have a good day or two to just try it out and hit the bug so you can really communicate that with the community.

04:45 And what is broken and what could be fixed?

04:49 Yeah, absolutely.

04:50 I'm excited to see where this goes.

04:52 Yeah.

04:53 So next up, we want to talk about pip-tools.

04:57 actually, one of the listeners, John Hagen, contacted us a while ago and said, hey, have you covered this?

05:03 And we thought we had, but I don't think we have.

05:05 So I'm going to cover it now.

05:07 pip-tools.

05:09 So we've talked about other things around Pip, like poetry and there's other ways to, you know, some virtual environment controllers and stuff.

05:18 This isn't that.

05:19 This isn't an all-in-one thing.

05:21 But one of the things that, like, for instance, poetry gives you is, and a couple other packaging things, is lock files and stuff.

05:30 And kind of a lock file really isn't, I mean, it isn't really what it does.

05:34 I don't know if it's a lock file or a pinned requirements.

05:37 But the idea around, pip-tools does a whole bunch of stuff.

05:42 But right now, I'm just going to concentrate on the compile part.

05:44 So the problem it solves is, so let's say I've got, like, I'm going to give an example.

05:49 I've got, like, a requirement.

05:51 My dependency is rich and typer is, like, two libraries I'm depending on.

05:56 I could just have a simple requirements.in file that has this, just those two labels in it.

06:03 And then with pip-tools, I can take that and take that requirements.in file and compile it.

06:10 I run pip compile.

06:11 And then I get, like, a pinned list.

06:15 And it actually just goes ahead and writes the requirements.txt file.

06:18 So just from an easy, I require these things, and it'll pop out the actual pinned dependencies.

06:27 And now I can deploy this, and all the other developers and stuff can use it and be okay with it.

06:33 Of course, before you do the deployment, please test this.

06:39 So that's the idea, as you update your requirements and then test it before you deploy it.

06:46 But this little extra step of taking, like, I've got a loose list of requirements, and I want to create a pinned list.

06:53 You can also, there's pip-tools also has a way to just say, let me get the syntax right, upgrade.

07:00 So you say --upgrade, and it updates your requirements.

07:03 One of the things I love about it also is it doesn't just do one file.

07:07 So if you've got a requirements.txt and a dev requirements.txt, for instance, with your testing tools, pytest and such, you can throw that in a different file, and it'll update those too.

07:19 So you can have as many, actually, as many requirements files as you want, and it handles it just fine.

07:24 So this is a really cool thing.

07:26 Jonathan also gave us an example project, this Python blueprint that has example code in it, and it uses this workflow, and it has it in the readme.

07:38 So if you want to check out a project that uses this, go ahead and do that.

07:41 I don't really, actually, to be honest, I don't use pip-tools for anything other than this.

07:47 This is what I use pip-tools for, is just this.

07:49 I know it does other things too, but this is what I use it for.

07:52 I'm psyched about this.

07:53 This looks fantastic.

07:55 I have requirements.txt and a dev requirements-dev.txt, and in there, I've been putting, like, at the top, here's the stuff that I really wanted to install.

08:06 And below it, these are the things I want to pin, not because I actually care about them, or you would actually need to mention them, but I want things like to pin the bot to see that there's a security problem in the dependency I'm using, not the thing itself.

08:19 And so it'll bump the version and fix it, right?

08:22 And that means, I just noticed a couple days ago that there's these other libraries.

08:26 I'm like, where did this come from?

08:27 This is in my virtual environment, and I didn't, it's not in either of those scenarios, right?

08:32 Well, some new dependency was added to the main dependency that I didn't pay attention to.

08:37 So now I'm, like, babysitting the bottom half of my requirements.txt file, which seems like a thing I don't want to do with life.

08:42 Yeah.

08:43 This is beautiful.

08:44 I'm all about this.

08:45 I'm very much thinking that this is going to be something I'm using out there.

08:49 My guys seem to agree.

08:50 Yeah, I absolutely agree.

08:51 And I actually have, we do not use this.

08:53 I do not use this, but I will be looking into this right after this podcast.

08:57 So, you know, like, why am I not doing what?

09:00 This is, this will make my life a lot easier.

09:02 And I'm definitely going to look into it.

09:04 Yeah.

09:04 Now, Juergen brings up, oh, you, the pip-tools is awesome.

09:10 Yes, it is.

09:12 And he also brings up that it's all about the app versus library story.

09:16 So pip-tools is definitely on the app side.

09:18 So you, you wouldn't want completely pinned dependencies in a library because it might conflict with the rest of the application.

09:27 Right.

09:28 Or pin you to say, I have to have the one that has the vulnerability in it rather than the newer one that fixes it, right?

09:33 You don't want that.

09:34 Yeah.

09:35 Right.

09:35 Yeah.

09:35 Awesome.

09:36 Yeah.

09:37 So I think.

09:38 Yeah, I think you're up.

09:39 I think I'm up.

09:40 All right.

09:40 Now you.

09:40 Okay.

09:41 So here, here's a hint on the pronunciation, but you're going to have to give this a shot on your own.

09:45 Yeah.

09:46 So I'm going to say it's, it's penguin, but I guess you could call it penguin.

09:49 I don't know.

09:50 I don't know what the proper pronunciation there is, but so penguin is a library for automating the generation of, of tests.

10:00 So basically what you do is it's CLI.

10:03 You set this up.

10:04 You point it at your code.

10:06 You can sometimes give it hints.

10:09 And then it goes off and it processes the, the, your, your files and produces tests.

10:17 Just disclaimer here.

10:18 They mentioned this several times, and this is very important.

10:20 It will run your code.

10:22 So, and it, it will try many different inputs.

10:25 So you have to be very careful with what that code is doing.

10:29 So they suggest wrapping it in a Docker container to prevent it from affecting or poisoning your, your file system.

10:37 Yeah.

10:37 Maybe don't give it access to the production database.

10:39 Something like that.

10:40 Yes, exactly.

10:41 Exactly.

10:41 Exactly.

10:42 Exactly.

10:42 So, so yes, this, this, this, I had, I didn't go deep into it, but I know that they're using some interesting search techniques.

10:51 This is actually developed by a college at a university and there's a white paper behind it.

10:55 I don't think it's here, but if you, if you look it up, I'm sure you could find the white paper that's associated, but it's supposed to be a very interesting technique around searching for, you know, the inputs that would be, you know, that would find good edge cases.

11:12 So it does take a while to run, they say, so that you can give it some hints and they talk about that within the documentation.

11:19 But I really think this is something for me, you know, I look at tools, I'm like, well, what would I use it for?

11:24 And, and I think, you know, a lot of times we're busy and we don't get the coverage that we really want to get because, you know, deadlines and, you know, we want to land this customer.

11:32 And, and so we have to get, we have to, you know, work really quick to get this one feature done.

11:37 And we don't, we only get half coverage instead of getting, you know, a 90% or plus.

11:41 So, so I think this tool could help maybe bridge that gap in those cases and maybe give you a jump, a jumpstart to, to getting the coverage that you're looking for on your projects.

11:53 So that's what I'm looking at it for.

11:55 And yeah, I think it's a cool project.

11:58 It looks like one of those systems, a little bit like hypothesis.

12:01 Are you familiar with hypothesis?

12:02 Only by name.

12:04 I haven't actually looked into it.

12:05 Where it kind of tries to determine, oh, what are some edge cases and some, like you give it bounds of data and it'll test different scenarios for you.

12:12 It looks, it looks quite interesting.

12:14 Brian, are you familiar with this?

12:15 I'm not, I am actually often leery of test generators.

12:19 But I don't know if people find it useful.

12:23 Maybe it's worth checking out.

12:24 Yeah.

12:24 Yeah.

12:25 Yeah.

12:25 It looks neat.

12:26 Yeah.

12:26 I'm the same.

12:27 I would say I'm the same, Brian.

12:28 I did, you know, I was a little bit leery, I guess about, about the idea of it.

12:34 For me, as I said, I think it's just really that gap that, you know, if you have a gap in coverage and you're really just trying to get a quick, a quick jumpstart on, on your, your coverage.

12:43 This might be a good tool to start with.

12:45 That's, that's my thought.

12:46 Yeah.

12:46 Yeah.

12:47 Very cool.

12:47 That's good.

12:48 Although, Brian, I don't know if you really want to encourage it too much.

12:51 You're getting out there.

12:52 If I'd known about Penguin earlier, I might not have bought Brian's testing book.

12:55 You didn't need to highlight that.

12:57 Indeed.

13:00 Indeed.

13:01 All right.

13:02 You got the next one?

13:03 Well, I wanted to, I think we're at the sponsor bit.

13:07 I believe we are.

13:08 Yeah.

13:08 So let's, yeah.

13:10 So I don't have our sponsor up.

13:12 I got you.

13:14 There we go.

13:14 This episode of Python Bytes is brought to you by Sentry.

13:17 How would you like to remove a little stress from your life?

13:20 Well, do you worry about users that, do you worry that users may be having difficulties or encountering errors in your app?

13:26 Right now, would you even know until they sent you a support email?

13:31 How much better would it be to have the error and performance details immediately sent to you, including the call stack and values of local variables and the active user recorded in the report?

13:41 That'd be awesome.

13:42 With Sentry, it's not only possible, but simple.

13:44 In fact, Michael uses Sentry on all of his web properties, including Python Bytes here.

13:50 He actually fixed a bug triggered by a user and had the upgrade ready to roll out as he got the support email.

13:56 That's also Sentry, but also because Michael's awesome.

13:59 But anyway, surprise and delight your users today.

14:02 Create your Sentry account at pythonbytes.fm/sentry.

14:06 And please, when signing up, click the got a promo code and redeem and enter pythonbytes, all one word.

14:14 It's good for two free months of Sentry's team plan, which will give you up to 20 times as many monthly events as well as other features.

14:22 That's pythonbytes dash slash Sentry and promo code pythonbytes.

14:27 Yep.

14:28 Thanks, Sentry.

14:28 All right.

14:29 I guess I got the next one.

14:30 This one is sent in by a friend of the show, Brian Skin.

14:34 Thank you, Brian.

14:35 And he let us know that there is something out there for all of us who love these external packages and all of these amazing Lego building blocks that are PyPI.

14:45 And yet that's running other people's code with dependencies on them getting things right.

14:51 And this is under the PyPA.

14:54 So this is sort of pretty neat in terms of being officially the Python Packaging Authority group, right?

15:01 I believe.

15:03 And so it's called the advisory dash DB.

15:05 It's a security advisory database for Python packages published on PyPI.org.

15:11 And the idea is if somebody finds some major problem with the package or maybe even worse, maybe it's like a typo squatting scenario, but more like, you know, that part where they were supposed to check the input in that form.

15:24 And then they did this direct SQL query.

15:26 They didn't do that.

15:27 Now they are.

15:28 You really, really should change that so that they're using like parameterized queries now.

15:33 And so there's vulnerabilities directory that is just a bunch of YAML files.

15:38 You come over here, like let's take one here that maybe is security conscious bleach.

15:43 I love the name of this package.

15:45 The idea is you take user input and you sanitize it by putting bleach on it, which is fantastic.

15:50 But even the sanitizer is going to have problems, right?

15:52 So we checked this out back apparently in 2020.

15:56 There's a problem with bleach in PyPI.

15:59 It said in Mozilla bleach before 3.12, a mutation XSS cross site scripted in bleach.clean win RC data.

16:08 And either the SVG or math tags are whitelisted and such and such as said like, oh my gosh, but here's all the version of effects.

16:15 And here's the fixed version.

16:16 And then there's some more info about like where that was spoken about, like where the problem was discovered and so on.

16:22 This is really cool.

16:23 So if you depend on these packages and we already spoke at the beginning about how having a tracking your dependencies, not just the things you directly install, let you let you be more aware of this.

16:32 Right.

16:33 You could look at that pip-tools generated requirements.txt file and see this problem.

16:37 And then very likely GitHub through dependent bot would even be proposing a fix.

16:42 What do you guys think?

16:43 Yeah, that's great.

16:44 I think this is cool.

16:45 So how do you use it?

16:47 Do you just look stuff up or?

16:49 You let it help you.

16:50 I mean, you can come over here and obviously look it up.

16:52 It depends on what's going on.

16:54 I think more if you're maintaining a package, you can do a PR to let people know.

17:00 There's a triage service, which goes through the NIST.

17:04 It pulls a lot of data from the NIST, what's that called?

17:07 The National Vulnerability Database Data Feed.

17:09 Say that again.

17:11 So if it gets submitted there, a lot of the data gets pulled back in.

17:15 They have a tool to perform some heuristics to match that back up.

17:19 And then you can do, most importantly, is there's an API that you can use.

17:23 So these vulnerabilities, when submitted to this GitHub repository, are then submitted outwardly to this place

17:30 called the Open Source Vulnerability or the Database for Open Source Vulnerabilities.

17:34 And then this has an API that people can call to learn about these problems.

17:39 So if you ran, I don't know, if you were an API company and you wanted to make sure your API wasn't getting hacked by having a bad dependency,

17:46 maybe you make this part of your CI, CD or something, Mike.

17:50 Yeah, absolutely.

17:50 I'm going to have to tell our DevOps guru, Mitch, to look into this.

17:55 Exactly.

17:57 So, I mean, for the most part, having stuff pinned in GitHub will trigger an immediate security notification to you when GitHub finds out about it,

18:06 when it makes it out.

18:07 But if you want to be a little more proactive or you're a security researcher or something like that, this looks like a good one.

18:13 In the longer term, we're looking, we're working with the PyPI team to build a pipeline to automatically get these vulnerabilities into PyPI.

18:21 So maybe like you could see it or something like that.

18:23 I'm not exactly sure what the story is there.

18:26 But it's a start for basically storing that data and sharing that data.

18:30 And then hopefully like it, it makes its way over to PyPI and such.

18:34 Sam Morley, welcome.

18:35 Glad you made the live show, man.

18:37 Said, is there a tool that can run like a pre-commit hook?

18:40 I feel like maybe with just a tiny bit of wrapper, you could call that API over at the open source vulnerabilities project and get some information back.

18:48 Yeah, that'd be cool.

18:49 Yeah, that would be a cool thing.

18:50 Like, don't check that in.

18:51 It's got a problem.

18:52 Don't push it.

18:54 Yeah, exactly.

18:56 Exactly.

18:56 So anyway, I think this is a pretty cool one.

18:59 Thanks, Brian Skin for sending it in.

19:01 Yeah.

19:01 So I don't know.

19:02 Sorry, but I was going to say, yeah, you're a C++ developer, right?

19:06 Yeah.

19:06 And C++ is all about overriding functions by type.

19:10 Like, oh, it takes an int.

19:11 It takes a float.

19:11 This one takes a string and it's totally different.

19:13 Python doesn't have that.

19:14 Yeah.

19:15 What's funny is I don't actually use it that much in C++.

19:17 I mean, you learn about it a lot, but I don't, I mean, I normally don't write a whole bunch of functions that have the same name, but take different

19:25 parameters.

19:26 But you can do, yes, you can do that.

19:28 Definitely do it in C++ and C.

19:30 And there's probably other languages you can do that in, but those are the two main languages that I work with.

19:35 Yeah.

19:36 C#, for example, does it.

19:37 Okay.

19:37 As well.

19:38 The Python, you can do that.

19:39 The Python, you cannot.

19:40 So Python, if you try to do that, the second one wins and you just have another name that is attached to the function.

19:49 I was, and I never really thought that I had a problem with this, but there are definitely times where it would be simpler to not try to put a switch in place or something and just have several functions that dealt with related things, but in different functions.

20:05 So there's an article called The Correct Way to Overload Functions in Python.

20:11 And maybe it is, but I'm not somebody that likes to use the phrase the correct way because somebody will tell me I'm wrong.

20:20 But anyway, it's an interesting article.

20:21 You might, for example, hear like you shouldn't overload functions in Python.

20:25 You're doing it wrong if you do it at all, for example, right?

20:27 Yeah, exactly.

20:28 Although this is a cool solution.

20:30 I totally like it.

20:30 Definitely.

20:31 It's a cool solution.

20:32 So there's this, there's apparently a built-in way with Functools.

20:37 So there's a, Functools has a decorator called Single Dispatch.

20:44 And what it does is you can decorate a, basically a function signature with a single function.

20:50 You say, I'm going to single dispatch this function.

20:53 And then you register all, you do like the name of the function dot register as decorators around other functions.

21:02 And the example shown in the articles just uses underscore.

21:07 I'm guessing that's just because you don't care what the name is.

21:10 I don't know, but that seems like a good way to do it.

21:13 And then you've got, then after that, you just have, that works.

21:17 It works like you've done function overloading.

21:19 Yeah, that's awesome.

21:20 So in the example it takes, you can call this format function with a string, with a date, with a date time, and a time.

21:28 And you actually get, there's like three different functions that get called.

21:31 One for a string, one for a date, one that just falls back or something, right?

21:35 Yeah.

21:35 I mean, it's a pretty simple example.

21:38 And if you, and it's pretty clean code.

21:40 So I would definitely try to keep all these things together in one, you know, in one module, of course.

21:47 You would be mean to put them in different places.

21:49 That would be so wrong.

21:51 Or even like spread out in the same function would be terrible.

21:54 But keeping them together, this is great.

21:59 One of the things that the article mentions is that this, you can't really, it's called single dispatch because it's based on the type of the first, I think it's based on the type of the first parameter.

22:10 So if you want to do multi-parameter ones, there is a third-party plugin called multiple dispatch.

22:18 You can just pip install.

22:21 And then it kind of works the same.

22:23 Although I kind of like this one better, honestly.

22:25 The syntax is a little different.

22:27 And it probably does single dispatch too, I'm guessing.

22:31 Yeah, just one parameter.

22:33 But in this one, yeah, similar sort of thing.

22:37 It just kind of works on multiple parameters.

22:40 I'm digging the multiple dispatch style where you just say dispatch.

22:44 This is arguments are list stir.

22:45 You know, we're going to maybe get to where Mike is going at the end of the show.

22:49 But it would be even cooler if you could just say at dispatch and then put a type like a colon list, b colon stir.

22:57 So I want to dispatch on types and stuff like that.

23:00 And have the dispatch decorator look at the function that's passed in and look at the type parameters and then just do that.

23:07 So you don't have to say the types more than once.

23:10 Anyway, we come back to that as well.

23:12 Yeah.

23:13 So in a past life, I was a Java engineer.

23:17 So we call it dynamic dispatch in Java.

23:22 And actually, there's a lot of patterns, object-oriented patterns that kind of are derived from that feature.

23:30 From, again, from the C++.

23:32 Well, from the old days.

23:35 And, yeah, no, to me, this actually, you know, when I saw it, I was like, okay, I could definitely use this for a lot of cases.

23:42 I know that, like, when you might want to build, when you have, like, a piece of code that has to work with a variety of different types, but they're very similar types, maybe.

23:51 And you want to do something with that family of types.

23:55 You know, I think that that would be something I would, you know, I would think to look at, to solve with something like this.

24:02 Yeah.

24:02 Yeah.

24:03 I kind of had the same thought.

24:05 And I do not think I would make this any sort of normal workflow use case for my code.

24:10 But there are times where you have some code that says, if, you know, argument is, the type of this is a list, do this thing.

24:17 If it's not a list, then make an empty list, put it in, and then do this other thing.

24:21 You know, like, if you're doing that type switching already, this dispatch thing might actually make it more clear, right?

24:27 It's not common to do it in Python.

24:28 But there are times you're like, I kind of want to be able to take a string or a date time and then just parse the string of it.

24:33 You know, those kinds of things.

24:35 So as an API company, this is actually kind of interesting, fits into what we do as an API company.

24:40 We have to, a lot of times, you know, we're not very, we're not overly strict on our input types.

24:45 We allow like truthy and falsy kind of types to come in for like Booleans.

24:49 And I could see us using this for that, right?

24:53 So that way, instead of using all the helper code we've already written for that, have it more along the lines of, okay, if it's this type of input, you know, then you can convert it this way.

25:01 If it's this type of input, you can convert it that way.

25:04 And I think we could probably write some code to handle our inputs differently using this pattern.

25:09 Yeah.

25:09 Yeah.

25:10 Very neat insight there.

25:11 Brian, you blew up our chat with us.

25:12 Well, you want to pull out some highlights?

25:16 Yeah.

25:16 All right.

25:17 So John Sheehan says, function overloading was one of those things like switch statements that I missed when I first started Python, but then I found I rarely needed it.

25:24 Yeah.

25:24 I'm kind of with you on that.

25:26 Like I said, there's a few cases for me that I'm like, where I'm doing a type check or is instance of that I think is the one time.

25:32 But that's, you know, a couple of times in a whole program.

25:34 What do you think, Brian?

25:35 Yeah.

25:36 I definitely missed it at first.

25:38 And now I just don't even really notice it.

25:40 Yeah.

25:42 Then you've got, you're going to get it says Luciano Romano.

25:46 Romano also explains this in Fluent Python.

25:48 I just read the pre-version of the second edition.

25:51 Getting some polish.

25:53 Yeah.

25:53 Very nice.

25:54 That's a really good book.

25:55 Fluent Python as well.

25:56 Came out there says, that's the kind of thing I was thinking of.

25:59 Yeah.

26:00 Oh, sorry.

26:01 That's a different comment.

26:02 We'll come back to that in a minute.

26:02 And then Sam Morley.

26:05 I don't tend to find this kind of dispatching that useful.

26:08 I generally just write a master function that takes star args, star kwrgs and dispatch to other functions.

26:13 Yeah.

26:13 Yep.

26:14 But if you're switching on his instance, it could be, it could be a time.

26:17 Yeah.

26:18 And then stepping back one quick topic.

26:20 Kim Vanewick says, would be handy if some pip tooling could automatically check the PyPA advisory before downloading a package.

26:26 That would be slick.

26:27 Yeah.

26:28 I just did a typo squatting and PyPI security episode over on Talk Python.

26:32 And we had a lot of interesting ideas, like almost like a, have I been pwned?

26:36 Like, remember what I installed?

26:38 And if you ever see, I installed something out of vulnerability.

26:40 Quick, shoot me a note.

26:41 Let me know.

26:41 Hey, a few weeks ago, you installed this thing.

26:43 You might want to get rid of it now.

26:45 Stuff like that.

26:45 But I think this is another interesting thing along those lines.

26:48 All right.

26:48 Like kind of cache that data and then just say, you know, I know you want to install it.

26:53 And maybe this is what they're talking about with integrating that into pypi.org itself.

26:56 Who knows?

26:57 Very cool.

26:57 All right, Mike, you got the last one.

26:59 Yeah.

27:00 So, AIO SQL is a, as it says, simple SQL in Python.

27:05 it's a SQL templating framework or, or library, that really you, what you do is you give it some

27:13 SQL files with some, it has some conventions around how you define queries.

27:17 And then the essentially gives you a query mapping, that you can then use in your code.

27:22 and this is kind of a, it's a minimalistic way of, of access to your, of accessing your database.

27:27 it is, you know, with the A in front of it, it's, it's, natively asynchronous.

27:32 it works really well with asyncpg.

27:35 I know that, we use, Postgres.

27:38 So, you know, that, that was a good, that was a highlight for us to, to look at it.

27:43 and, you know, I, I think we all have worked with ORMs.

27:47 you know, I've come back and forth on ORMs over the years.

27:50 you know, I think this is a nice fit for when you have, when you, when you don't have to do,

27:56 a lot of touch, a lot of different tables or do a lot of joins.

28:00 Maybe you're, have a small microservice that just needs to do a couple, you know, reads and writes updates.

28:06 and, and also I think this really helps because ORMs, especially when you're dealing at scale,

28:12 you have to really know the ORM at a very deep level to understand what it's doing.

28:16 And this kind of exposes all of, you know, it really just brings you down to the SQL level.

28:20 So now, you know, exactly what you're doing and you, you can be a little bit more direct on what you want to,

28:24 what you want to do with your data.

28:26 Yeah.

28:26 Yeah.

28:27 Sometimes you're like, well, I want to do the join and then the filter, not the filter, then the, I don't know.

28:32 Like if you're really good at SQL, you can, you can put these little tricks and like, oh, it's actually

28:37 likes it better if we do it this way rather than that way.

28:40 And, oh, I'm probably not going to let you do that.

28:42 So let me just describe this to people out there because it's breaking some paradigms for me.

28:46 So this is a library that allows you to write queries against a database,

28:51 but you do it almost with data access, data access layer style of things.

28:57 So you would, you don't do a quote, select star from such and such.

29:01 You, you would say like query dot get all greetings or get user by username.

29:06 And you pass a like username equals such and such in an asynchronous way, which is pretty interesting.

29:12 But the way that that has meaning is you create a dot SQL file that has like a doc string like thing.

29:18 And you write a bunch of SQL, almost like stored procedures, but just in a text file, this thing parses it.

29:24 And then it becomes like a smart query data access layer.

29:27 Correct.

29:28 Yeah.

29:28 So it basically builds out other data access layer or data access object kind of for you,

29:33 where you can basically map in these, these, these queries.

29:37 And it's not just queries.

29:39 You can do mutations and you can do, you can do a DDL and all of that.

29:42 So, yeah.

29:43 Yeah.

29:44 It gives you a lot.

29:44 Create tables or whatever.

29:45 Yeah.

29:46 It could be a function you call.

29:47 Yeah.

29:47 Absolutely.

29:48 So.

29:48 Brian, what do you think?

29:49 Do you touch SQL much these days?

29:50 I'm touching it more and more.

29:53 And I love this actually.

29:56 Cause I always assume if there's a bug, it's probably in my SQL statement.

29:59 But this looks pretty cool.

30:01 So get, get your, get your selects and stuff figured out what kind of queries you want to do ahead of time.

30:06 And, and then use them in your code.

30:09 This is cool.

30:10 Yeah.

30:10 Yeah.

30:10 I mean, I'm very repelled by writing raw SQL statements in my code.

30:15 Very much like Kim Van Wijk.

30:18 Others like this looks exactly what I like, like not writing SQL, but not using an ORM.

30:23 No, it's no secret to the world out there that I'm a MongoDB fan.

30:26 So I, I'm not, I don't have like direct use for this a lot, but I do find this super valuable,

30:31 especially if people on the team are really good at SQL and they're like, oh, this, this library is inhibiting me from like using my superpower on the database.

30:39 Right, Mike?

30:40 Exactly.

30:40 Yeah, that's exactly.

30:41 And I was just about to say that, you know, if, if you have a DBA or you have someone who is, you know, basically they are focused or concentrated on the database and, and optimizing those queries.

30:51 You know, I think what, what this allows you to do, then you can go define your queries and you can write them yourself, how they are as developer.

30:59 And then maybe you can take that to your, to the experts and have him refine them for you without actually changing the abstraction.

31:05 Right.

31:05 Like without, you still get that.

31:07 Yeah.

31:07 Yeah.

31:07 There's also times where like, let's say I've got a internal tool that I'm going to put together and it needs a database.

31:12 And it's really not that big of a deal to tell like your, your IT department and your supervisor and stuff.

31:18 They're going to throw up a MySQL database and, or something and, and interact with it or Postgres.

31:23 But to say, hey, I'm going to throw Mongo in there.

31:26 That might bring up a, just a, just a discussion that you're just not ready to have.

31:31 So.

31:31 Yeah, for sure.

31:32 Let me suggest a pairing, you know, kind of like as you pair wines, a pairing with this library.

31:38 I think Pydantic might go fantastic with this, right?

31:41 So you get these results back as tuples and then you could just, you know, star args that into your Pydantic models and you get your validation and all sorts of like stuff.

31:53 And then you get a list of Pydantic, like you're one list comprehension away from something beautiful here.

31:58 Yeah.

31:58 This is, that's exactly what we're doing.

32:00 You know, we're using, you know, I think right now we're building out a new stack and it's basically, it's, it's, it's FastAPI, Pydantic, you know, AIO SQL.

32:08 Oh, interesting.

32:09 And you're finding it works well.

32:11 Oh, it's, yeah.

32:12 And it's very performant.

32:13 So it's, you know, because it's asynchronous, right?

32:15 We're using asyncpg all the way down to the database too.

32:18 So it's pretty much asynchronous from end to end.

32:20 Fantastic.

32:21 This is a really good recommendation.

32:23 Yeah.

32:23 You'd like it too, right, Brian?

32:24 It's pretty sweet.

32:25 Yeah, this is great.

32:26 Yeah, I do.

32:27 I really like the separation of concerns of the SQL statements and the Python code, but the flexibility of having both.

32:34 It's pretty cool.

32:34 Awesome.

32:35 Brian, is that it for us?

32:36 Yeah.

32:37 Anybody have any other topics they want to bring up?

32:41 Oh, you know, I do.

32:42 Kind of, kind of crazy.

32:44 So it's not quite an extra, extra, extra, extra here all about it.

32:46 But a couple of things sent in from some listeners.

32:49 So let me make sure I credit these.

32:52 So Daniel Mulkey sent this one in and said there is now, has been for a little while from Science Direct, there's a journal for academics called SoftwareX.

33:04 And this library is pretty cool.

33:06 So it aims to acknowledge the impact of software in today's research.

33:11 So if you're in academics, one of the big challenges is that whole publish or perish sort of thing.

33:16 And it would be great to take this cool library and make it a thing that's out on PyPI or Conda that people can use.

33:22 Oh, but that's going to take a week and a half.

33:24 And you really only get credit for your citations in articles, not in code, right?

33:30 So this kind of addresses that to try to give people a place to publish their projects in a meaningful way.

33:35 And I just want to point out that there's a special issue on the software that contributed to the LIGO experiment, the gravitational wave discovery, which I'm pretty sure got the Nobel Prize.

33:44 I don't totally remember.

33:46 But if you just start poking around here, you know, you might find the word Python periodically in this thing here.

33:54 So over this one, we got some Python.

33:57 I'm just randomly clicking articles and they're all coming up Python.

34:00 So I think that that's pretty sweet.

34:01 If people are in science and they do this computational side of things, check this out.

34:06 Very cool.

34:07 Yeah, that's number one.

34:08 Number two, HiCon has been finished for a little while.

34:11 And we were able to go through some like meeting platform type thing where we could watch the replays.

34:17 But it's a little bit wonky in the way that the playback worked.

34:20 Like I had to do some weird stuff to actually get logged back in after a while.

34:24 Anyway, it was fine for live stuff, but it was weird for playback.

34:27 YouTube is pretty awesome for playback.

34:29 And here's the I put in the show notes, the playlist for all of HiCon 2021 US.

34:35 So that's pretty cool.

34:36 I don't know how many videos are in here.

34:38 It looks like 86 videos.

34:39 So if you want to catch up on the HiCon talks, there's a really good way.

34:42 Nice.

34:42 It looks like Juergen had a lightning talk there.

34:46 Yeah.

34:47 It's cool.

34:48 Yeah.

34:48 Very cool.

34:49 Well done.

34:49 And also want to make sure I get the credit for this one right.

34:54 Madison Swain Bowden, who we both met, I believe, at PyCascades, shared this with us.

35:00 That AICS, ImageIO, which is used all over biological imaging and microscopy and that kind of stuff from the Allen Institute.

35:12 Allen Brain Institute, I'm guessing.

35:15 It was just released.

35:15 So if you're doing anything with graphics and pictures and file formats and stuff for that, check that out.

35:21 And yeah, that's it for me.

35:23 I think, Mike, you got one as well, right?

35:24 Elise?

35:26 Yeah.

35:26 Yeah.

35:26 So yeah, PEP563 was, I think it's a postponed evaluation of annotations.

35:34 And basically, there was some, as I think I mentioned before, I'm a big fan of FastAPI and Pydantic.

35:43 And there was some worry a couple months ago that this change would effectively break these projects in 3.10 if it was introduced.

35:53 And so I just wanted to give a shout out to the community for jumping on this and just being really open-minded about this and basically postponing this a little further so that better decisions can be made on how to navigate this change.

36:08 So I don't want to go too deep into it, but I read some of the threads on this and it was just really positive exchange within the community.

36:19 And I think it turned out really well.

36:20 There was like a unanimous decision, I believe, to postpone it.

36:23 And it saved these projects from having to, well, basically.

36:27 At a minimum scramble.

36:28 Yeah.

36:28 Worst case, not work.

36:31 Yeah.

36:31 Yeah.

36:32 Let me see if I can give the elevator pitch.

36:33 There was a proposal in Python 3.10, I guess, 5.3.8.

36:37 I know, sorry.

36:38 5.6.3.

36:40 I don't know where that came from.

36:41 5.6.3.

36:42 PEP563.

36:43 There was a possibility where there's an attempt to simplify and speed up type annotations.

36:47 And by, because right now, if you want to do a type annotation, if I'm going to say I've created a Pydantic model, it's called user.

36:53 And I'm going to say this thing returns a user.

36:54 I have to import user at the top.

36:56 And it was like, well, you're not going to get those evaluated or really dealt with unless you're doing type annotation analysis.

37:03 So we're going to treat those as strings, basically.

37:06 Yeah.

37:07 Right.

37:07 And things like FastAPI and Pydantic and others were going, they would actually get the type information, standard Python style, and then use that to modify behavior.

37:18 Right.

37:18 Like to do, oh, it says it's a list of ends.

37:20 So we're going to convert from this list of those, or maybe a list of user.

37:23 We're going to convert these JSON things to users.

37:26 But with this change, it would have said, well, it's a string.

37:28 And what does user mean?

37:29 Right.

37:30 If you don't know the actual type where it's coming from, there could be multiple users.

37:33 It could be a user.

37:34 It couldn't track down.

37:35 Has it been imported yet?

37:36 All sorts of weirdness around it.

37:38 So the runtime behavior of these type annotated things like Typer, Pydantic, FastAPI, we're not sure how they were going to keep working with this being adopted.

37:48 And it sounds like it's postponed at least until 3.11.

37:51 Yeah.

37:51 Exactly.

37:52 Yeah.

37:52 Thanks for bringing this up.

37:53 That's a good one.

37:54 Yeah.

37:54 All right.

37:55 Well, kind of end of the show.

37:57 Do we have any jokes?

37:58 I got something for you.

38:00 I'll tell you.

38:02 You're loving of it.

38:03 So this is from Zach, RMRF on Twitter.

38:06 It says, did you hear about the four Pythonistas that robbed a bank?

38:10 Three got caught.

38:11 And when questioned, they all stated that the fourth would never be caught.

38:15 Couldn't be caught because he knew RegEx and was something of an escape artist.

38:20 That's funny.

38:23 Yeah.

38:24 I think that's pretty good.

38:25 Well done, Zach.

38:25 Thanks for sending that in.

38:26 Brian, you got anything as well?

38:28 Yeah.

38:28 Let me pull it up.

38:30 So Rich, not Rich, Will McCoogan asked about Rich and said, hey, if I wrote a book about

38:41 Rich, how much money would you pay for it?

38:44 But also what should be in the book?

38:46 Of course, I said it should cover how fast it is to install it.

38:51 Kind of a get rich quick guide.

38:52 Love it.

38:58 Love it.

38:59 And I believe the response was something like, get out.

39:01 Get out.

39:01 Yeah.

39:02 Definitely.

39:03 Oh, nice.

39:04 Will's getting a lot of coverage this week.

39:06 That's beautiful.

39:06 He's doing good work.

39:07 All right.

39:08 Speed of coverage.

39:09 Yeah.

39:09 Thanks for being here, Mike.

39:10 Well, thank you guys for having me.

39:11 This was really fun.

39:12 Yeah.

39:12 Thank you.

39:13 Thanks for everybody for joining on the live stream as well.

39:15 Wow.

39:16 Thank you for listening to Python Bytes.

39:18 Follow the show on Twitter via at Python Bytes.

39:20 That's Python Bytes as in B-Y-T-E-S.

39:23 And get the full show notes at Pythonbytes.fm.

39:27 If you have a news item you want featured, just visit Pythonbytes.fm and send it our way.

39:31 We're always on the lookout for sharing something cool.

39:34 On behalf of myself and Brian Okken, this is Michael Kennedy.

39:37 Thank you for listening and sharing this podcast with your friends and colleagues.