Transcript #385: RESTing on Postgres
Return to episode page view on github00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.
00:05 This is episode 385, recorded May 27th, 2024. Did I get that right?
00:11 Yeah.
00:11 I am Brian Okken.
00:12 And I am Michael Kennedy.
00:14 This week's episode is sponsored by Mailtrap. More about them later.
00:17 If you want to connect with us, we're all on Fostadon, the show, and Michael and I.
00:23 We're Fosstodonians.
00:24 Fosstodonians, yeah.
00:25 So those links are on the show notes.
00:28 Also, in the show notes or at pythonbytes.fm, you can go to live if you want to join us.
00:34 Join the audience.
00:35 Usually, it's Tuesdays at 10 a.m. Pacific time.
00:39 You can also use that link to get older videos, too, if you want to watch what happens live.
00:45 And finally, if you'd like to receive an email a day or two after the show to find out all of the links and to have a link to the show.
00:57 But make sure if you listen to this and you want to make sure that you don't miss any links and be able to go check those out later, you can become a friend of our show.
01:06 And we'll send you those emails.
01:07 So you had an announcement before we get started.
01:10 I do.
01:11 Well, I just wanted to say, Brian, the reason that we're not just doing this at a normal time, we're just kind of late.
01:16 And I got some messages from folks in places like New Zealand.
01:18 Like, awesome.
01:19 This is the perfect time.
01:20 So there's always a perfect time somewhere in the world.
01:22 But you've got this thing going on tomorrow.
01:24 And then for the rest of the week, I'm going to this thing called the Giant Loop Ride.
01:30 This off-road rally that's at a hot springs and maybe 300 other off-road riders out in the southeast desert of Oregon.
01:37 And riding the mountains and the desert and open bar, live music, all sorts of shenanigans going on.
01:44 Anyway, if somebody...
01:45 Like a swimming pool?
01:45 Yeah, that swimming pool is a hot springs, by the way.
01:48 How crazy is that?
01:48 If anyone is listening out there and is also going, just come say hi.
01:51 That's all I'm saying.
01:52 But also why we're doing this in the middle of the night for some places.
01:58 So I would say if you have a podcast that kind of runs late, sometimes it'll make you tired, wouldn't you think?
02:03 Would you have to rest a little?
02:04 Yeah, yeah.
02:06 Let's talk about Postgres, PostG, rest, as in Postgres, rest API.
02:13 So this one comes to us from Mark Little.
02:15 So thanks, Mark, for sending us over.
02:17 It's a good one.
02:17 This is basically a web server, web front end for your database.
02:23 So you've got Postgres.
02:25 And you might think, all right, well, I would just like to expose the tables as endpoints with, you know, ability to do queries over HTTP endpoints.
02:34 You somehow pass the query over.
02:36 You want to have security, things like that.
02:39 So this is an app that you can run and it automatically, with zero API work for you, turns your Postgres instance into a somewhat restricted API, basically.
02:52 Cool, right?
02:52 Yeah.
02:53 Yeah.
02:54 Now, obviously, these things, I don't know, I always have mixed feelings when I talk about these things.
02:58 On one hand, if you just say, I would like, say, my view front end or something, or even my Python code to just have access to the database as an API, just read, you know, the crud things of all the tables, you're good to go.
03:11 This is what you want, right?
03:12 On the other hand, sometimes I want to have more specific things.
03:16 Instead of just, I want to go through the user, the create new user API endpoint that uses, I don't know, argon2 for memory hard, password hashing and other checks.
03:29 And maybe, you know, like every time I come up against these things, I'm like, oh, they're really cool.
03:33 But at the same time, I kind of want a little more specific control over how it works.
03:39 So if you're just like, I just have a database, I just want to be able to expose it real quick, let's go.
03:42 Postgres rest sort of thing.
03:44 Cool, right?
03:46 So the way it works is it's written in Haskell, which you may, and it's 4% Python, so I guess there's a little Python in there.
03:53 But, you know, it doesn't really matter what it's written in.
03:55 It's just a thing you run, and then you talk to it from your Python or from your JavaScript or whatever it is, okay?
04:02 So there's a couple of things you care about.
04:05 One, you probably care about performance.
04:06 I'm not super familiar with the performance of Haskell, but I believe it runs on top of the JVM.
04:11 Sorry if I get that wrong, but I think so, which makes it pretty quick.
04:14 It says it'll do 2,000 requests per second on the Heroku free tier, which is pretty fast, actually, right?
04:21 That's a pretty wimpy server.
04:22 And 2,000 requests per second is pretty high end.
04:25 Three reasons for that.
04:26 Haskell, and it's written used in the Warp HTTP server, a compiled language with lightweight threads.
04:32 I also learned about Warp, so I'm learning about all sorts of things.
04:35 It delegates as much as it can to the database, so it's like a really thin layer over the database, as I was describing.
04:41 So serializing JSON responds directly in SQL.
04:45 It does the data validation, the authorization, combining row count and retrieval, data post in a single command.
04:53 All those different things are happening in the database, not in this thing, right?
04:57 And finally, it uses the Haskell, instead of SQL, it's the Haskell library, which manages database connections, a binary protocol for talking to it, and is stateless for horizontal scaling.
05:14 Also, very important for security, you don't want to just go, and the whole thing is read-write on the internet.
05:20 How could you get, you know, a little Bobby Tables doesn't find it?
05:23 So it uses JSON web tokens, and basically what you do is you create a user in the database, and then you set authentication, like, this user can only read, and they can only read these tables.
05:34 But they can write to that table, I don't know.
05:35 And then that's what you can do when you authenticate over this, right?
05:40 So it basically leverages database users and their implicit permissions to control what you can do with the API.
05:46 There's ways to version the API, which is kind of interesting.
05:49 As the database changes, you could keep it stable, it seems like, and it documents itself through OpenAPI.
05:55 Well, this thing's pretty neat.
05:57 So if you have a Postgres database, and you're like, look, we really just need read, write, update, you know, maybe even an internal app, and you just want to have it real simple, check this out.
06:08 It's kind of cool.
06:08 Cool.
06:09 I didn't know that Heroku had a free tier anymore.
06:12 But it is.
06:13 Let's check out.
06:15 Let's go back here and do a little.
06:16 It may have been from previous times.
06:19 Yeah, that statement, it must be from previous times, because this was updated a month ago, but it didn't.
06:25 Yeah, I think that's just whatever the wimpy free tier was, that's giving you a sense of what you can do.
06:31 Okay, cool.
06:32 Neat.
06:33 I want to talk about AsyncIO a little bit.
06:37 So there's an article called from Jacob Padilla, how Python AsyncIO works, recreating it from scratch.
06:48 And I kind of like this idea, because I mean, I love AsyncIO, but sometimes it just feels magical.
06:56 And to sort of walk through some of the, how it might be working in the background is kind of cool.
07:03 So it goes through generators review, talks about the event loop, talks about sleeping, then yielding to await and await with AsyncIO.
07:13 And I just kind of like the format that this is written in to just sort of get your, get a good mental model of how AsyncIO works.
07:21 So this is, I guess I highly recommend this article.
07:25 Looked really fun.
07:27 The colors are nice too.
07:29 I, you know, I got a sucker for a good webpage that's easy to read, but, but this is good.
07:34 I skimmed it briefly already.
07:37 And like, yeah, this kind of goes through how awake Async works.
07:42 Async just sort of feels mad too magical sometimes, but.
07:45 It is pretty magical.
07:46 And at the heart of it is generators, right?
07:49 So generators themselves feel super magical.
07:52 And once you kind of understand how those work, you understand, well, okay, well, that's how you kind of partition the work.
07:58 And then you just run it in, in steps, each, each fragment of the generator and off you go.
08:03 But yeah, it seems pretty wild.
08:04 Yeah.
08:04 So it kind of walks through all the little steps and then it shows what the, so it has like the, the final bit.
08:09 And then it rewrites the whole thing in actual AsyncIO so that we can see what it looks like.
08:14 so that's great.
08:15 along with that, when I noticed the link that some, there was, camera where I got it from, but there was a comment also about if you want to like learn AsyncIO from trying to recreate it from scratch.
08:27 There was also a, David Beasley talk from the, from, I think it's from PyCon.
08:32 Yeah.
08:32 PyCon India from 2019.
08:34 I hadn't watched this.
08:36 I still haven't watched this, but it's on my list now.
08:38 build your own Async, from David Beasley.
08:42 Awesome instructor.
08:44 So I'm sure this is a good video as well.
08:45 So we'll link, we'll link to that.
08:47 so some, some learning AsyncIO.
08:49 Awesome.
08:50 Good stuff.
08:51 Indeed.
08:52 What, what else is good is our sponsor Mailtrap.
08:56 All right.
08:56 Let's talk about them.
08:58 This episode is sponsored by Mailtrap, an email delivery platform that developers love.
09:02 An email sending solution with industry best analytics, SMTP and email API, as well as SDKs for major programming languages and 24 seven human support.
09:14 Try for free at Mailtrap.io.
09:17 Thank you.
09:17 Thank you.
09:17 Thank you.
09:17 Thank you.
09:18 Thank you.
09:18 Thank you.
09:18 Thank you.
09:18 Thank you.
09:18 Thank you.
09:18 Mailtrap.
09:19 Let's move on to something in a little higher order.
09:21 Shall we, Brian?
09:21 Sure.
09:22 Perfect.
09:23 Follow on from what you were talking about.
09:25 AsyncIO, parallel programming.
09:27 It's basically scales the weighting.
09:29 Now this, this is going to be a dated statement, I believe, but at the moment, Python is not super good at doing computational parallelism.
09:38 Right.
09:38 But you know what is GPUs.
09:41 GPUs are awesome at doing parallel programming.
09:45 So, you know, your typical Python program has one thread, or if you don't do anything in most languages, they have one thread.
09:52 If you scale out your CPUs, maybe you got 16.
09:54 My Alienware gaming PC has that.
09:56 My M2 MacBook Pro, I think, has 10 threads, you know, but there's way more.
10:01 It's get a lot more work done if you scale those out, right?
10:04 But if you scale their GPUs, they have 16,384 threads, things like that, right?
10:08 Incredible.
10:09 So I ran across this thing called Bend, B-E-N-D, and it comes from a higher order company, which has a couple of, it's like a runtime for Bend and then has Bend the language.
10:21 And it's all about taking things and running them in parallel and making them basically be Python.
10:27 So it says, with Bend, you can write parallel code for multi-core CPUs and GPUs without being a C or CUDA, respectively, expert with 10 years experience.
10:37 And it feels just like Python.
10:39 So they give us this example.
10:40 Here's a function called sum.
10:42 Not the best example because it shadows a built-in, but whatever.
10:45 Let's get rolling with it, right?
10:47 And so it says, def sum, take a depth.
10:49 And there's some value that it starts with and then adds them all up, right?
10:54 So it says sum all the numbers from 0 till 2 to the end and 2 to the depth, right?
11:00 It has a switch statement that once it gets to the end, it stops and then sort of recurses down as it would.
11:07 And there's nothing that looks parallel about this whatsoever.
11:10 But what happens is this code actually runs in parallel potentially on those thousands of GPU cores.
11:16 Oh, wow.
11:17 Wild, right?
11:18 Yeah.
11:19 So the way it does that is, there's a picture here, is it breaks these things up into what are called computational graphs instead of work.
11:31 And then it just, it goes and processes them to see if that part of the graph can be parallelized, which is kind of nuts.
11:39 So you don't even write the parallel code.
11:42 It just looks at your code and goes, oh, that loop that we just saw or that recursion type of thing we just saw, that actually could be done.
11:48 Recursively by just changing the parameters or something, right?
11:52 So pretty wild.
11:54 And so, yeah, they have this HVM, highly massively parallel runtime that achieves near ideal speedups as a function across the available cores.
12:02 And then Ben is the programming language that runs in parallel, powered by this thing.
12:06 Looks like Python.
12:08 I don't know how compatible, say, its standard library is or things like that.
12:13 Yeah.
12:13 And then they have this thing called Kind, which is a parallel proof checker.
12:16 Have no idea about that.
12:18 I just said it's coming soon.
12:19 So this is available on GitHub.
12:21 And you come over and you can find their language.
12:24 And guess what?
12:25 It has 15,000 stars.
12:26 I'd never even heard of it until recently.
12:27 Yeah.
12:28 It's pretty popular.
12:29 Written in Rust, not that that's super relevant, but it's like powers a language that looks like Python.
12:34 That's pretty cool.
12:35 I'm cautiously optimistic about that.
12:39 It could be good.
12:41 So it looks like the real implementation for it is here, although it looks pretty short.
12:47 I don't know.
12:48 Updated a couple of days ago.
12:49 But here it says, yeah.
12:51 I don't know.
12:52 We'll have to see how large of a thing is it.
12:55 I guess that's just some pretty big scroll bars.
12:57 So maybe the whole runtime is here.
12:59 We'll see.
12:59 Maybe the number of files might be lower, but could have tons of stuff in there.
13:03 Yeah, that's true.
13:04 Anyway, is it going to be a thing?
13:07 I'm not sure.
13:07 Maybe.
13:08 Maybe not.
13:09 But it's cool to see people being creative for this kind of stuff.
13:12 You know?
13:12 Yeah.
13:13 Actually, it sounds pretty cool.
13:14 Yeah, it does.
13:16 To, I guess, simplify the way you get your code ready for massive parallel running.
13:22 If it can just look at your code and go, yeah, I got it.
13:25 Yeah, if it can look and say, this is what's often referred to as embarrassingly parallel,
13:30 right?
13:30 There's like no state shared.
13:33 You could just take the pieces and break them up and run them on cores.
13:35 Yeah.
13:36 If it just finds those automatically and does that, which is what it claims to do.
13:40 Yeah, that's great.
13:41 That's really great.
13:42 Yeah.
13:42 That's really great.
13:43 All right.
13:45 Next up, I've got regular expressions.
13:47 So we're getting all sorts of deep topics today.
13:51 We are.
13:51 We sure are.
13:52 So I ran across, I was looking at Reddit.
13:56 And so this book is an older book, about a year, I think it's a year old.
14:01 It's called The Smartest Way to Learn Python Regular Expressions.
14:06 And I don't normally pitch other people's books or anything on the show, but I thought it was
14:12 kind of nice that it's a LeanPub thing.
14:14 So they're doing their own thing.
14:15 And I guess I'll get into why I think this is kind of neat.
14:18 But the minimum price is free.
14:20 So if, and since they're allowing that, I encourage everybody to throw a few bucks their way if they
14:26 can, because why not?
14:28 It's good to have people teaching things.
14:31 Anyway, ran across it on Reddit.
14:33 And the topics sound neat.
14:36 So they go through an introduction of regular expressions.
14:40 They do an application.
14:41 I talk about puzzle learning and stuff.
14:43 And then I skimmed the rest.
14:44 But it looks like great stuff.
14:46 And some Python related thing to teach regular expressions would be good.
14:51 And then I went and watched their video.
14:53 And now I am even more intrigued.
14:55 So the idea around this book is that modern ways to learn are different than possibly people
15:01 my age might be used to, just reading a book and trying things out.
15:04 But this is set up.
15:06 There's a little video introduction to the book.
15:08 They're using the book itself as a long form teaching people.
15:13 But it's also, there's a video, there's a video course attached to it.
15:17 So if you, like, you get the book and you're reading through it, there's links to the different
15:22 course chapters within the book, which is kind of a neat idea.
15:26 And then there's puzzle solvers for each thing you're learning.
15:31 So you can go apply it in a puzzle environment.
15:34 And they're using a thing called Finfinkster.
15:39 It's an awful name, actually, I think, but F-I-N-X-T-E-R.
15:44 But it looks like there's a whole bunch of other people using it, too.
15:48 And I've never tried it.
15:49 But it's this kind of an online trying out some code platform, I think.
15:53 So I'm going to try this out.
15:57 I'm curious about the whole idea of reading a book and clicking on it and doing that as
16:03 well.
16:03 Mileage vary for me because I'm probably going to download this onto a Kindle.
16:09 And I can't click on anything with the Kindle, but at least the one I'm using.
16:13 But anyway, it might be fun to go through.
16:15 Yeah.
16:16 Looks excellent.
16:17 Give us a report when you get back.
16:20 Sure.
16:21 Yeah.
16:22 See how it went.
16:23 Yeah.
16:23 And, you know, not enough regular expression material out there for just deep, diving deep
16:29 and really getting your head around it.
16:31 Yeah.
16:32 They're very powerful if you're good at them.
16:33 Yeah.
16:34 Big if.
16:34 Those are our items.
16:38 Do we have any extras?
16:39 I have one.
16:40 That's quite interesting.
16:41 And I don't.
16:43 Somebody gave me a bit of a tip towards this.
16:46 I don't remember who it was.
16:47 So I think it was on Mastodon.
16:49 But thank you.
16:50 And sorry about forgetting who to credit here.
16:53 On Wikipedia, we have the Python Conference, aka PyCon.
16:57 And what does it say about it?
16:58 It says, the Python Conference, also called PyCon, is the largest annual convention for the discussion
17:04 and promotion of the Python language.
17:06 Scroll down a little bit, Brian.
17:07 You see things like, where was it?
17:09 In 2003, it was in D.C.
17:10 It had 200 attendees.
17:12 2012 was in Santa Clara.
17:14 2,300.
17:15 Oh, 2016.
17:17 Yeah, baby.
17:18 Portland.
17:19 3,391.
17:22 Almost.
17:23 Almost.
17:24 The record.
17:25 Almost.
17:27 3,393.
17:30 Two more people beat that number, came over that number from Cleveland, Ohio in the before
17:37 days.
17:37 And then COVID hit, knocked it down, and it's slowly building back.
17:43 Like in Pittsburgh, it was up to 2,500 again.
17:46 So what is that?
17:47 It's still got about 50% more to add or a third looking backwards.
17:53 It's down.
17:54 That's not what I want to talk about.
17:55 I want to show you what is in this Wikipedia table right below it.
17:59 Where we're going next.
18:01 The Long Beach, baby.
18:02 Going to California.
18:03 Let's go.
18:03 Going to the beach.
18:05 Going to some warm weather.
18:06 Going to a vacation spot.
18:07 I will bet you that this significantly beats those numbers because it's just a destination,
18:14 right?
18:14 People want to just go there to be there, right?
18:17 Well, it's kind of nice to see it on the West Coast.
18:19 I mean, nothing against Pittsburgh, but it's only two hours away from, where was it?
18:24 Cleveland.
18:24 Yeah.
18:25 Yeah.
18:26 Yeah.
18:26 So we did have Salt Lake City in the intermission times, but Salt Lake City was still under a
18:32 lot of funk.
18:33 It was the first one that came back in person after COVID.
18:38 So it was kind of funky.
18:39 I went to the 2023 one.
18:41 I know that, and I guess I'm going to go on the record of saying, I think that there's
18:46 a lot more cool destination things, destination places on the coasts.
18:51 Yeah.
18:51 And Midwest, obviously.
18:54 But I think doing it kind of in the middle of the country makes sense so that people can
18:59 get easy flights from anywhere in the country.
19:01 Just saying.
19:02 Yeah.
19:04 And what are you optimizing for, right?
19:06 Are you trying to move it around?
19:07 Are you trying to make it as centrally located?
19:10 Are you trying to make it as cheap as possible?
19:12 Are you trying to make it a destination?
19:14 All these different things play into this.
19:16 And this is the biggest funding for the PSF.
19:18 So it's not just like a curiosity, right?
19:21 This is the funding for the PSF to a significant degree.
19:24 It's interesting, though, that Portland was way back in 2017.
19:27 And this in 2026 is going to be the next time that it's going to be on the West Coast.
19:33 I don't think that I know that everybody on the East Coast thinks that Salt Lake is in
19:37 the West, but it's not really.
19:39 Not even the same time zone, Brian.
19:41 Not even the same time zone.
19:44 Mountain time, Pacific time.
19:45 Let's go.
19:46 Anyway, this is my extra.
19:47 Planned tentatively, the PyCon after Pittsburgh.
19:52 So next year is still Pittsburgh.
19:54 It's round two.
19:55 But then Long Beach, California.
19:56 And I'm here for it, Brian.
19:58 Also, I've always been curious about it.
20:00 Sometimes I'm curious about the numbers of how many people showed up in Montreal.
20:04 And it's cool to see.
20:06 You can just look it up on Wikipedia.
20:07 Yeah, yeah.
20:08 It's all right here.
20:09 I've got something completely random.
20:13 Prince of Persia was a video game from 89.
20:18 I remember it.
20:19 Oh, not only was it a video game.
20:22 It was one of the best video games.
20:23 It was so good.
20:24 Yeah.
20:25 Well, it was like this.
20:26 One of these cool, like, you know, racer through all sorts of stuff.
20:30 We had like the little runner thing and had to jump over all sorts of stuff.
20:33 It was a fun game.
20:35 And then there was a bunch of different permutations of it.
20:39 And it's still going strong.
20:41 I think still stuff going on.
20:43 There's a anyway.
20:44 Cool game.
20:46 But I ran across this.
20:48 There's a book about it.
20:50 So Jordan Mechner is a person that wrote it and originally wrote the Prince of Persia game.
20:58 And he wrote a book called Replay.
21:02 It's a memoir of an uprooted family.
21:05 And apparently it goes back to 1938 when somebody in his family, his parents fled Austria to escape the Nazi persecution.
21:17 And then, you know, they traveled to the U.S.
21:21 Eventually they went to France.
21:22 Then they went to eventually, I think, actually, I'm just making this up.
21:26 I don't know if we ever made it to the U.S.
21:29 I think so.
21:30 But then he goes back eventually.
21:34 Yeah, because he goes back eventually and works in France for a little while.
21:39 But then did this book about his life, his family and his life story and about the Prince of Persia.
21:47 So I'm kind of excited to read this.
21:49 I really like these looks inside the storytelling of the history of some of these famous games and famous platforms.
21:56 Yeah, they're very interesting.
21:58 Yeah.
21:59 And apparently his dad wrote the music for the original video game.
22:04 Okay.
22:04 It's kind of cool.
22:05 On a, I think, anyway, one of the older computers in the 80s.
22:11 So that's it.
22:12 That's my extra.
22:13 Awesome.
22:14 Let's close it out with a work hack joke.
22:18 I don't know what you call this, Brian.
22:19 So we have this woman developer sitting here, pretty happy.
22:23 And this boss comes in, manager says, I don't like this bash script you created.
22:29 Write it in Python instead, commands the boss.
22:32 Import subprocess.
22:33 Subprocess call work.sh.
22:36 Well, that's a day's work done.
22:37 Nice.
22:39 How about that?
22:41 Just if you want me to write in Python, I'll just call the script from Python.
22:44 Let's go.
22:45 Yeah.
22:45 Well, one of the times where I'm like, where early on, I guess not early on.
22:51 Anyway, 20 years ago or so, I find myself in a new job.
22:56 And they've got all these builds, make scripts for C++ code.
23:00 And I can't remember.
23:01 They were in batch files and bash and all sorts of stuff.
23:05 And I said, can you rewrite this?
23:08 Because it's kind of a mess.
23:09 So I rewrote it in a combination of, I had a combination of bash, Perl, Python, said, a whole
23:17 bunch of things, whatever I wanted to, to what I thought was an elegant solution.
23:21 And the team was horrified.
23:23 They're like, we're not learning that many languages.
23:26 I'm like, okay.
23:27 So I rewrote it all in Python.
23:29 And that was the first time I really grabbed Python for do whatever I want as a shell script
23:36 language.
23:36 Yeah, that's very cool.
23:38 Yeah.
23:38 I love it.
23:39 It's a subprocess call.
23:40 Yeah.
23:40 Subprocess.call.
23:42 That's right.
23:42 Yeah.
23:43 Nice.
23:43 Yeah.
23:44 Don't wait for it to complete.
23:45 Just let it go.
23:46 Well, yeah.
23:49 Works.
23:49 No error handling needed.
23:51 Let's just let it run.
23:52 No, it's a good joke.
23:53 And that comes from your credit where credit's due.
23:54 That comes from Linux handbook.
23:56 Linux handbook.
23:57 Nice.
23:57 Cool.
23:58 Well, thanks, Michael, for another wonderful episode.
24:00 As always, thanks for everyone for listening.