Transcript #148: The ASGI revolution is upon us!

00:00 Hello and welcome to Python Bytes where we deliver Python news and headlines directly to your earbuds.

00:04 This is episode 148 recorded September 11th, 2019. I'm Michael Kennedy.

00:11 And I am Brian Okken.

00:12 And this episode is brought to you by DigitalOcean. Check them out at pythonbytes.fm/digitalocean.

00:17 More on what they have to offer later.

00:18 Brian, I feel like Python doesn't ship often enough. It's like an 18-month release cycle.

00:27 that's a really long time to wait for these features to come out, you know, especially in the world of agile and scrum and Yeah, two weeks prints like a year and a half is a long time to see an idea and have it happen What do you think that's like eons in internet time, right? So right for sure. I'm actually excited about this There's an announcement. It's it's not been announced or decided yet, but there's a PEP PEP 602, which is under discussion We've got links to both the discussion and the and the pep, but it's to bring a annual release cadence so every year every 12 months instead of every every 18 months and There's some specifics around it which are kind of neat. It does not mean that people will develop each feature each release in 12 months. It's actually a 17th month cycle, which is weird. It's interesting It totally makes sense when you read it.

01:19 My first thought was, "How do they fit 17 months into 12 months?" You go really, really fast, and then you change a little bit of space time.

01:25 There's five months of unversioned overlap time where some of the new features can get worked on and they're not doing releases.

01:31 Yeah, that's really interesting because they have a cool graph in here that shows them overlaid and actually all the releases laid out.

01:39 I don't know, it's about a third, a little more than that down the page that we're linking to.

01:43 And it shows, okay, we're going to work on the alpha features, which is creating the feature, and then beta is kind of where it stabilizes and gets cleaned up.

01:51 But as it switches to beta for one, it kicks off for development, maybe a little bit later.

01:56 But there's this overlap where you stop working on one and you also concurrently start adding new features.

02:02 That's what happens in software projects anyway.

02:05 Like in a commercial world, you're wrapping up one release while people are starting to work on features for the next release anyway.

02:13 It's just this reality anyway.

02:15 I think it's good.

02:16 some rationale and goals, smaller releases, less stuff within each release.

02:22 One of the big ones is there's a often a rush to get stuff into the first beta because if it didn't get in it's not going in this one and it has to wait 18 months and waiting a year it's only six months difference but it that's a big difference. So there are also some weirdness with how I mean there's annual sprints and PyCon, which is about the same time every year, but having the release cycle be sometimes offset by six months every time just was weird.

02:53 Yeah, it means like the sprints can more carefully align with say the alpha stage or something of the next version, right? They time it right. There's also mentioned in the non-goals section about switching to an annual release calendar is the last for more natural calendar version.

03:10 So for example, Python 320, 3.20 might be the one released in October 2020.

03:19 The one Python 3.23 would be the one in October 2023 and things like that, that might be nice.

03:26 - Yeah, but we're only up to like three eight now.

03:28 How do we get from three eight to three 20?

03:31 Yeah, switching.

03:31 - It goes three nine and then three 20.

03:33 That's like, that's how you count.

03:34 - Yeah. - No, it's pretty good.

03:36 - Yeah, so this is proposed by Lucas Lenga, and he's currently the release manager for Python, so he's the one with all the skin in the game, and yeah, I think this is nice.

03:46 I've always thought it was weird to have such a long release cycle, so I'm happy to see this.

03:50 - I mean, there's risks and downsides, of course, that would be on the other side.

03:55 Also, with that same graph, you notice that there's five versions supported at a time instead of four, so there's the test matrix of people doing distributions They gotta test five versions instead of four if they're testing everything.

04:10 It is what it is.

04:12 - Yeah, it's interesting.

04:13 There's some rejected ideas as well.

04:15 Keep the current 18 month release cadence.

04:18 This is rejected.

04:19 We don't like it.

04:20 Double the release cadence to achieve nine months between major versions.

04:24 But it's too irregular and too short.

04:26 And just slow down releases and don't freeze them at beta one.

04:30 Yeah, I like it.

04:30 This is pretty good.

04:31 Glad to see it happen.

04:32 - It wasn't even considered to have weekly release cycles.

04:35 I'm not sure why.

04:36 - Yeah, we just ship whatever's on master, on the master branch.

04:40 Yeah, daily, nightly builds.

04:42 - Nightly builds.

04:42 - Might hold off on that.

04:44 So this next thing I wanna talk about, it's pretty awesome, Brian.

04:47 - Is it?

04:48 - Yeah, it's called Awesome ASGI, or ASGI.

04:52 I don't know, I'm not sure what the phonetics of ASGI, but we have WSGI, which are the WSGI servers.

05:00 This is like the traditional web service in a gateway interface where basically all the different web frameworks like Django, Flask, Pyramid, and so on, implement that, and then all the servers like MicroWZ and gUnicorn and whatnot support that.

05:16 That's how they click together. That's their docking mechanism.

05:21 WZ is broken, fundamentally irreparable broken, for building asynchronous web apps.

05:29 That's unfortunate because async and await and asyncio is most important on the web server.

05:35 What do web servers do?

05:36 They wait on databases, they wait on the network, they wait on third party APIs like Stripe.

05:41 They just wait.

05:42 That's mostly what they do.

05:44 And so if you could let your server do other stuff while that's waiting, you could massively amplify the concurrent number of users for the same amount of hardware and things like this.

05:53 So enter ASGI, I'm gonna go with ASGI for now until somebody shows me the right way to say it.

05:59 So ASGI is a new interface for this successor to WSGI.

06:04 And it came out of Django channels, which is a way to add some asynchronous behavior to Django.

06:13 And it's also got boosted by things like Starlette and Uvicorn and so on.

06:18 So what this is, is it's an awesome list for all sorts of things around asynchronous web Python things.

06:26 So we have async web apps, async web frameworks, async web servers like uveicorn and so on.

06:34 So if you are wanting to build some sort of high load, highly scalable website and you want to figure out what should I be using, it's super confusing right now because things come and go so fast.

06:48 Like, Gepronto was really this big thing until it wasn't.

06:51 Stopped getting worked on, right?

06:53 Like, the shelf life is no longer 10 years.

06:56 it's maybe a couple months.

06:58 So knowing what's happening in this space is pretty hard.

07:01 There's some good frameworks talked about here.

07:04 Bocadillo, channels, FastAPI here is really good.

07:07 Core is a solid one for converting Flask to Async, Responder, Starlet.

07:12 Starlette seems to be the foundation of almost everything these days.

07:15 So it's really probably the most important here.

07:18 But a nice grouping of all these various things that you might want to work with.

07:22 - Yeah, and they also want to try to do more than just applications, but also libraries and middleware items as well.

07:29 - Yeah, so this is not as popular as say, like just a straight awesome Python list or Mahmoud Hashemi's awesome apps list or something like that.

07:37 But I do think it's important and you know, hopefully people go out there, if they have some ideas, they can contribute some projects to the list.

07:44 It's not too hard.

07:44 - Yeah, definitely.

07:45 - All right, what's next?

07:46 - Well, there's another awesome, a couple other awesome projects, but they're not awesome lists.

07:51 Two awesome projects I want to talk about, sort of, are Jupyter and Pangeo.

07:57 This comes up because there was an announcement titled "Jupyter meets the Earth," which is kind of fun.

08:05 Plus, they're both circular logos.

08:07 The Earth has a logo.

08:09 Looks like the Earth, yeah.

08:10 This is a write-up about a really cool announcement that the National Science Foundation is giving two million dollars of funding over three years as part of an EarthCube grant program to go towards Jupyter infrastructure and Jupyter impangio and it's hopefully will be pretty cool. It's a actually a really neat write-up talking about how the adoption of open languages like Python and on top of that Jupyter and others have really been taken up by a lot of different science backgrounds and we really need some more earth-based science to utilize this and so this will help it out quite a bit. It's kind of neat.

08:52 This is super cool and it's really great to see them getting so much funding.

08:56 Jupyter is definitely doing well in the funding space and I didn't know where Pangea was before but it sounds like this puts them really solidly in the stable and growing, getting lots of support space. One of the things I love about Jupyter and Python being the center of around and some of the data science is the openness. So people are really pushing for open data as well so that anything that gets published can be seen by other people. I think that pushes science forward really far because just even people in their bedroom might have a cool idea with the data and find some things that other people can't and help contribute. It's really kind of incredible to be part of this. So. - Yeah, absolutely.

09:38 And one of the use cases is for climate data, climate change studies, and things like that.

09:43 So definitely need some attention on that.

09:48 I'm sure there's plenty already, but more is good here.

09:52 So this write-up is by Lindsay Hagee and Fernanda Perez.

09:55 And just want to mention, Lindsay was on Talk Python quite a while ago talking about Pangeo and her earth science research she's doing, so geoscience, things like that.

10:06 So if you want to dive deeper, check that out.

10:07 Yeah, very cool.

10:08 Yeah, for sure.

10:09 Now, before I spring some pretty big news on people, let me tell you real quick about DigitalOcean.

10:15 So, all of our infrastructure runs on DigitalOcean.

10:18 They've been big supporters of the show.

10:20 Their newest thing is that they just launched two new things as a service.

10:24 So, you can instead of have servers to manage, you just check a box and now you have services at your disposal.

10:31 Those are MySQL and Redis.

10:33 So if you want to use MySQL or Redis, you can now start doing that over at DigitalOcean.

10:39 Previously, they had Postgres as a service, and Postgres is very popular in the Python space, but MySQL and Redis also are as well, and Redis kind of plays an alternate role anyway.

10:49 It's a traditional relational database, caching, things like that.

10:54 So check that out, along with a bunch of other good things, pythonbytes.fm/digitalocean.

10:59 Now, I told you about the cool awesome list, right Brian?

11:02 The awesome ASGI.

11:03 Yes, asynchronous is here to stay.

11:07 It is here to stay.

11:08 Now, there is a very notable absent framework in there.

11:13 Talks about court.

11:15 That's kind of the async version of Flask, if you will.

11:18 There's some talk that maybe those two projects will work together, just make Flask itself asynchronous.

11:23 But that's a different story.

11:24 But there was nothing in there really for Django.

11:26 I mean, there's channels, but that's a separate project, right?

11:29 - Yeah.

11:29 - Until now.

11:30 - Yeah, so tell me, I'm excited, what's going on?

11:33 - Yeah, so I don't know exactly how recently this was announced because there's no dates on the announcement, which makes it tough to know.

11:42 But Django 3.0, all the details for its roadmap are now out, so the release notes for Django 3.0 are here.

11:50 It's expected that Django 3.0 is going to be released December 2019, so that's like a Django Christmas present or whatever, something for the holidays, right?

12:00 Coming soon, and there's tons of changes, right, about, for example, Django 3 only supports Python 3.6 and above, they are dropping support even for 3.5.

12:12 And if you want 3.5 support, you have to go back to 2.2, and of course, 2.2 or 2.0 already dropped support for legacy Python, so you gotta even go farther back to get way back there.

12:21 You don't wanna go back there.

12:22 >>No. >>But that's all good.

12:24 There's a bunch of other changes, area and DB support and various things.

12:28 You can check that out.

12:29 But the big news I think here is ASGI support for Django 3.

12:35 They say, "Django 3.0 begins our journey to making Django fully async capable by providing support for running it as an ASGI app." And that doesn't mean they're dropping WSGI support.

12:50 They expect for these to sort of pair together, as you might expect.

12:54 A lot of these can be run in either way, but you have to run it on an ASGI server to get the async features, as we kind of hinted at before.

13:02 Pretty cool, huh?

13:03 Yeah, that's very cool.

13:04 Yeah, so you don't need to change anything about your code, but you'll be able to start writing asynchronous methods.

13:08 So when you're waiting on that database or that web service, you can now wait on a thousand of them instead of one in the same process with about the same amount of work, which sounds like a pretty awesome change for almost no effort.

13:18 Well, you said you don't have to change anything.

13:20 Does Django have an upgrade migration path for people to upgrade?

13:24 Well, I think you just have to start rewriting your view methods.

13:27 You got to add the async keyword, async_def instead of def, and you got to start using async libraries.

13:33 It's going to be interesting, the database story.

13:35 Django has a built-in ORM, and it says things like there are some side effects to be aware of.

13:40 Django is now aware of the asynchronous event loops, and will block you calling code marked as async_unsafe, such as ORM operations.

13:50 So maybe I don't know what the story is around the ORM and async.

13:55 Maybe it's not totally clear, but it seems like a really good place to actually have async because the thing you wait on most is databases, so you should probably be able to wait on them.

14:02 Yeah, and I actually am also interested in MariaDB.

14:06 I hadn't really got into that before, so that's kind of an interesting side story as well.

14:10 Yeah, absolutely.

14:12 So there's all sorts of stuff we're linking to in Django 3.0, but I think the big news is front and center async support right in the main project.

14:21 - And I wonder if the drop of 3.5 is because people like f-strings.

14:25 - Yeah, yeah, I think so.

14:27 Yeah, it's like, we can't not use f-strings.

14:31 I want to run Flint on this project, but I can't because it might run in 3.5.

14:34 Let's make that a possible.

14:35 - Yeah, yeah, okay.

14:37 - I've heard about this possibility of different software engineers having different levels of effectiveness.

14:43 And oftentimes, this is referred to as the 10x engineer, This special person that makes all this money and actually makes the startup do its thing or whatever.

14:52 John Carmack and Doom.

14:55 So you found another kind of engineer.

14:57 Yeah, I really like this list.

14:59 I found the 1x engineer.

15:01 The 1x engineer, awesome.

15:02 What constitutes a 1x engineer?

15:04 Well, there's a whole list.

15:06 And you can contribute it because it's based on a GitHub repo.

15:10 I think it's definitely a fun take on the 10x thing.

15:13 But if you look at these lists, I I think some of these are really good and I think actually should have been like something like maybe 3x or 4x or 8x or something.

15:22 I'm not sure how high this scale goes anyway, but here's a few of them.

15:27 1x engineer has a life outside of engineering, writes code that others can read, doesn't act surprised when someone doesn't know something, asks for help when they need it, is able to say things like "I don't know", constructively participates in code reviews, shares knowledge, never stops learning. So I like that because obviously they listen to Python Bytes, talk Python and testing code. Absolutely. Yeah. Has unproductive and productive days. Fails from time to time. Doesn't hate on framework Y when working with framework X. Anyway, we pulled out a handful of these to put in the show notes but the list is pretty long and I like it. It's kind of a neat fun take on just basically this is what we expect from everybody. Very nice. It says, I've never heard of that in lieu of nodding and pretending and also is willing to have a brightly colored desktop background.

16:18 Very nice. Yeah I think the new 10x engineer has pink not black. That's right.

16:23 Now this is actually this is really great I like it it's fun and it's written in very fun fonts and style right? There's a button there that says you can go away from the fancy style. What is that font? It's like I think it's Atari 2600 font or something.

16:41 Maybe Commodore 64 font is beautiful.

16:43 - Yeah, at least it's not Comic Sans.

16:45 - Nice, yeah for sure, thank goodness, or Blinky.

16:48 Okay, now there's a topic I think is important, Brian, that we should touch on.

16:53 Will you just tell me if we've spoken about this before?

16:56 It has to do with Python 2 going out of the world here, sunsetting Python 2.

17:01 Have we spoken about Python and Legacy Python?

17:04 - Maybe we haven't, so if we haven't, People should get on to the Python 3 thing.

17:08 - That's right.

17:09 Now, what we're linking to is an official document that formalizes the sunsetting of Python 2.

17:16 So this is over on python.org/docs, just sunset Python 2 with dashes.

17:21 I'll link to it.

17:22 And it's sort of a Q&A, like, what if I didn't know Python 2 was going away?

17:27 What should I do?

17:28 And so on.

17:28 So let me just talk to you real quick about this.

17:31 This is something good you can share with people who are like, wait, why is our Python old?

17:35 What are you telling me?

17:36 So it says officially January 1st, 2020 will be the last day or the day that Python 2 is sunsetted.

17:44 I thought it would be really great.

17:45 New Year's resolution, write no more Python 2 code, I guess is the way to interpret this.

17:50 And it says, Q&A, like why are we doing this?

17:54 We need to sunset Python 2 so that we can help Python users.

17:57 We want to add new features to Python 3 and stop supporting Python 2, even though we're not adding features to it.

18:05 And it says, you know what, back in 2006, long ago, we started working on Python 3.

18:12 And in 2008, we said 2015 was the end of Python 2.

18:16 That didn't work out so well, so now we're really meaning it 2020.

18:20 And then it says, how long is it until the sunset date?

18:24 The Python clock will tell you, it says.

18:25 Which is three months, 20 days, 10 hours, 31 minutes, and 37 seconds.

18:30 That's less than four months.

18:31 That's coming up.

18:32 for real. Yeah, it's even faster if we hold off releasing this episode. That's right, it's already gonna be out of date. It says, "What will happen if I do not upgrade by January 1st?" Oh, I'm gonna come to your house and ridicule you. No, that's right, yeah, you'll be mocked by all your peers. No, it says, "Look, if people find catastrophic security fixes in Python 2 or in software written in Python 2, most volunteers will not help you. You know, over time you'll find it harder to hire people, you'll find it harder to use modern libraries and so on. So probably not the world is going to end straight away, obviously, but you know, you're going to be in quicksand at some point, right, to some degree. It says, okay, fine, I wrote some code in Python 2, how should I port it to Python 3? It says, please read the official porting guide here.

19:20 And finally, I didn't hear anything about this till just now. Where was it announced?

19:26 We talked about it at software conferences, the Python announcement mailing list, Python blogs, textbooks, technical articles, social medias, and companies that sell Python support.

19:35 Oh, and podcasts.

19:37 Does it say podcasts?

19:38 No, I've amended it.

19:40 Yeah.

19:40 Yeah, I believe we have talked about it.

19:42 Now that I've gone through, I do remember speaking about Legacy Python before.

19:45 Yeah, we have.

19:46 And then the last question of how can I make sure that I don't miss stuff like this again?

19:50 Listen to Python Bytes.

19:52 That's how.

19:52 That's right.

19:53 - That's the official recommendation from the PSF, I'm sure.

19:56 - No, it's--

19:57 - Or mailing lists, I guess.

19:58 But I do think, I mean, we cover, we try to cover this kind of stuff, for sure.

20:02 - Yeah.

20:03 Duh.

20:04 - Yeah, so there is the official sunsetting details from the PSF and the core developers.

20:10 It looks good to me.

20:11 - Yeah. - Cool.

20:11 All right, well, that's it for our maid items.

20:13 What else you got you wanna throw out there?

20:14 - Well, there's two big news, and yours is definitely gonna happen, and mine is hopefully gonna happen.

20:19 >> Mine is in the past, so it is more likely to have happened.

20:23 >> Okay, so tell me about yours.

20:24 >> All right. We have been working, we, me, and a bunch of other folks have been working with the Humble Bundle folks to release another Python Humble Bundle.

20:34 Not just a bunch of books like an O'Reilly bundle thing, but a bunch of cool tools and library courses, everything.

20:41 So we just launched the Humble Level Up Your Python Bundle.

20:47 And yeah, there's tons of good stuff, right?

20:49 There's three courses from Talk Python Training.

20:52 There's the Code Challenges from Python Bytes.

20:54 There's some PyCharm stuff.

20:55 There's books from Dan Bader, from Matt Harrison.

20:59 There's stuff from Trey Hunter.

21:02 All sorts of good things in there.

21:03 So be sure to check that out.

21:05 You can get it much, much cheaper for a very short time.

21:10 So probably, I guess a week, a week to two left if you listen to this the moment that we release it.

21:15 So, there'll be some time but not a lot.

21:18 And it's like $1,000 worth of stuff for 15 bucks or something like that.

21:22 Seems like a good deal.

21:23 - Yeah, yeah, it seems great.

21:25 - I see this next thing that you're talking about.

21:26 You're trying to work on my social life, aren't you?

21:28 Get me out of the house.

21:29 - Yes, but I want to make it really easy on me.

21:32 This next one is all about me.

21:34 Well, it's about you too and everybody else in Portland, but mostly about me.

21:39 I thought it'd be cool, there's a Python meetup every month downtown Portland.

21:43 and I don't live downtown Portland, I don't work in downtown Portland, so whammy.

21:48 But I'm gonna put my, I guess, effort where my mouth is and start a Westside Portland Python meetup.

21:57 - Yay, that's awesome.

21:58 Yeah, 'cause to get down to the east side of Portland, if you're out somewhere by Intel or Nike or one of these, a lot of the tech companies here in town, it's like an hour, just one way.

22:09 It's a tremendous amount of traffic you gotta go through.

22:11 So this will definitely open up a whole 'nother angle for meetups.

22:15 - Yeah, and I'm not trying to like, you know, steal their thunder or something.

22:18 I've talked with the Python meetup dude from Portland and we're working together to make sure we don't overlap it.

22:25 So I think we're shooting for like the second week in the month because they have things on the third and fourth.

22:32 There's nothing there yet, but if by the time this comes out, there will be.

22:36 I registered pythonpdx.com.

22:39 That's where we're gonna announce stuff.

22:41 Hopefully for October, October, but if not, then we'll get it done for November, but I'm really shooting for October.

22:48 This is made possible because Roden Schwartz, the company I work for, has this awesome room that would work great for the meetup and they've agreed to let us use it.

22:57 So that's great.

22:59 - Yeah, sounds great.

22:59 - Yeah, that's all I got, but I don't have any jokes.

23:03 - We don't have any jokes, hold on.

23:05 - I mean, we could just stare at each other for a few minutes, but it's not fun.

23:08 Ooh, neat.

23:10 How did the angry function exceed its call stack size?

23:12 I don't know, why?

23:13 It got into an argument with itself.

23:15 [laughter]

23:16 What's the object-oriented way of becoming wealthy?

23:19 I'm not sure.

23:21 Inheritance.

23:22 Oh yeah, definitely.

23:23 Of course.

23:24 All right, well, not terribly funny, but these are some good programming dad jokes.

23:29 Yeah, yeah.

23:30 Nice.

23:31 All right, well, thanks for being here as always, Brian.

23:35 All right, thank you.

23:36 Bye.

23:37 Yeah, good to chat with you.

23:38 Bye.

23:39 to Python Bytes. Follow the show on Twitter via @pythonbytes. That's Python Bytes as in B-Y-T-E-S.

23:46 And get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Auchin, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.