Transcript #7: Python 3.6 is out, Sanic is a blazing web framework, and are failing our open source infrastructure?

00:00 This is Python Bytes, Python headlines and news delivered directly to your earbuds.

00:05 It's episode 7, recorded Wednesday, January 4th, 2017.

00:10 Hey Brian.

00:11 Hey, happy new year.

00:13 Yeah, happy new year.

00:14 It's great to be back together.

00:16 We've gone three weeks without discussing cool things that came out in the Python community and I think it's time to get back together.

00:23 The Python news must flow.

00:25 It must, and plus I missed this.

00:27 This is fun.

00:28 Yeah, it's definitely, definitely fun.

00:29 So like many of our episodes, this one is sponsored by Rollbar.

00:33 They help you take the pain out of errors.

00:35 So thank you to Rollbar.

00:36 We'll have more to say about them throughout the show.

00:39 So, Brian, you had a little warning.

00:41 You wanted to just get it out of the way?

00:43 Yeah, I've got – there's a couple – I want to try to list some of the names of the articles.

00:48 The people that wrote the articles that we are reading and talking about in this episode, and some of them have names that I am most definitely going to mispronounce.

00:57 I hope you don't take offense at it. I have a last name that everybody mispronounces. So I understand you're used to it. Yeah, I get Oaken and mostly Oaken is what I get. But but you're not a tree. So, hey, so before we get to the first episode, the first news item, I actually want to do something open up my terminal, I'm going to type types of that. Okay, I'm gonna type Python three dash capital V. Do you know what it says? It not Python three five two. It says Python 3.6. That's right. Python 360. Yeah, so Python three six was announced on December 23. And it's now Python 360 is our official latest release. And this is really exciting. I've already started using some of the new features in my writing. I'm definitely taking advantage of the f-strings. I don't know how you would call those.

01:57 Yeah, I think the other languages call it string interpolation, but it's the F character that indicates it. Swift and C# call it string interpolation.

02:06 Oh, I'm going to call them the F-in strings.

02:08 I love it.

02:10 But in celebration of this, I know we've already talked about 3.6, but I've got links to the in our show notes for the announcement and the what's new article from docspython.org.

02:21 But there's also a couple other articles that are cool. There's a Python 3.6 is packed with goodness article from Sirdar Yigalap. No, I got that one wrong. That's on InfoWorld. But it's a a list of things that he thinks is cool in three six, including a sink await and use the use the use of that more the the improvements of memory and speed. And I I'm really excited about that. And some something that I didn't know about is some of the improved API support so that tracers and debuggers and stuff can hook into Python easier.

02:57 Yeah, it's definitely packed with goodness. I think that's that's great. Another article that you pointed out was adopt Python 3. I really like that one as well and I'll let you go to some of the stats and say the name as well, the author. Yeah so this is on Medium and I'm gonna try this one because it's a cool name Dibya Chakravarti. I don't know, cool name but yeah so there this other adopt Python 3 article talked I thought it was interesting so there's a I have heard of a couple of sites called the wall of superpowers and Python 3 readiness that are two pages that list most popular Python packages and how much of them are Python 3 ready and the numbers are higher than I expected actually 187 out of 200 on one of them and 341 out of 360 on the other and I think this is important because a lot of people tried to switch to Python 3 a long time ago and one of their favorite packages wasn't supported but it probably is now so check those out and then this person also wrote up an IPython notebook and had some statistics to pull out not just the most popular but everything he pulled everything on PyPy that was listed as stable and had at least one release in 2016. That was 6000 packages of those. So there must be a lot of like packages on PyPy that are not our PyPI that are not updated very often. But I really like it when the statistics do some basic filtering, like, nobody cares whether that package that almost no one's using hasn't been updated for 10 years supports Python two or Python three.

04:49 Yeah, but of the what he calls stable and active, it was interesting, there's 14% of those don't even cover Python 2, they're Python 3 only packages.

05:02 And 28%, which is higher than I would hope, but 28% are Python 2 only.

05:10 But the author estimates that a lot of those could easily be switched if they're really important to you.

05:16 But he does list out, he has a list of all the ones that he calls sticky, the 25% of 28, so I don't know, that's not very many, 14, 7% total.

05:30 Yeah math on the fly.

05:32 Anyway, of those sticky packages that may be a problem for you.

05:36 But you know, I was in one of those camps a year ago, I tried to switch our testing to Python three, and there was a package that I needed that wasn't switched.

05:46 And it is now so I don't have that excuse anymore.

05:49 Yeah, that's really great.

05:50 I love the wall of superpowers.

05:52 This article is really interesting.

05:53 And I think it makes it's a good argument and it's data driven, which is cool.

05:58 In fact, like you said, you can get the IPython notebook for it.

06:02 My final thought on Python moving from legacy Python, Python three, six, the new Python is you get a lot of benefit in this release just by installing and running on it if you don't touch your code at all, because there's a lot of memory and performance speed up.

06:15 So that's really cool.

06:16 Yeah.

06:17 And it's the instead of walls of and the wall of superpowers.

06:23 Maybe we could look at maybe roads and bridges.

06:25 Yeah, there's walls, there's roads and there's bridges.

06:29 And, you know, I don't know how it is in the rest of the world.

06:31 I know in Germany, it's not quite this way.

06:33 In the US, we tend to neglect our physical infrastructure.

06:38 I had to swerve the other day because there was a huge pothole in the road and so on.

06:43 It's a problem.

06:44 But I think the real problem for us these days is actual digital infrastructure.

06:50 So there's a really great article, not an article, it's more like a report or even a book, it's like 150 pages, called "Roads and Bridges, the Unseen Labor Behind Our Digital Infrastructure." It's written by Nadia Egbal, and she did a really good job.

07:04 It's a new, relatively new report, it came out in July.

07:07 It basically talks about open source and mostly open source infrastructure, not open source projects.

07:14 So not, not necessarily things like Django, but more things like PyPI and pip, right, the underlying foundation, things like OpenSSL that everybody builds on top of.

07:28 And it turns out there's actually a really big problem across the board.

07:32 Like if you think physical infrastructure is neglected, you should see the digital infrastructure.

07:36 All right, so I actually did a whole session with this, some of the folks from the Python community, Eric Kulsher from Read the Docs, Donald Stuff from PyPI on Talk Python on episode 84.

07:48 And we talked about this a little bit.

07:50 Basically, this report goes through and outlines all the challenges that they see for supporting things like PyPI, supporting things like OpenSSL.

08:01 And they said that until Heartbleed, you know, Heartbleed was that really serious bug that you could sort of send a bunch of requests to a server running OpenSSL that was vulnerable and you could actually get it to leak information, unencrypted information back out.

08:17 So until then, it was like that thing was just barely coming along.

08:21 Like there's almost nobody that worked on it.

08:23 like one guy or a couple of guys that very very short term were able to work on it and two-thirds of all the web servers use that for their communication.

08:33 And think about that. Another example they talked about RubyGems, the website done by volunteers. Somebody had found some kind of vulnerability in it and the the people who had supported it they're like all right we'll fix this on the weekend because we have jobs we can't work on this infrastructure and like I mean that's like pip to Ruby right it's really important and it turned out it got hacked and taken down before the weekend.

08:56 And so people had to like schedule vacation days so they could get it back running.

09:01 One more example is PyPI itself.

09:04 And Donald Stuft works on that.

09:07 He was employed by HP.

09:10 HP went through a bunch of layoffs and not even knowingly, I think, took out his position.

09:15 Now there's Donald's doing that in his spare time.

09:17 Who else supports PyPI?

09:19 They do 400 terabytes of traffic a month and it costs almost $40,000 to keep pip running.

09:26 So you can say pip install a thing, that's like $40,000 worth of infrastructure.

09:31 And there's only a few baskets that are supporting that thing.

09:34 It's quite precarious.

09:35 So I wanted to bring up roads and bridges, the unseen labor behind our digital infrastructure.

09:40 This report, I think people who care about this will find this very well-written, very interesting and lots of neat stories.

09:45 Well, even if they're depressing.

09:47 (laughing)

09:48 Yeah, it is interesting. Thanks for bringing that up.

09:51 Yeah, absolutely. I threw a bunch of little excerpts that I got from the article.

09:56 People can flip through and see if they're interested in that.

09:59 Speaking of things that we take for granted, I know that a lot of...

10:03 There's some libraries out there that people use all the time, and if a library is used by a lot of people, making changes to it can be a little scary.

10:11 RC2 of Matplotlib 2.0.0 came out in December and Thomas Caswell is, I don't know if he's the only person supporting it, but he's one of the main developers and he was telling me that there's, it's been, one of the changes in this is some of the changes in the default style. So the default, if you just, if you're not styling your plots specifically and you're just going with whatever Matt Plotlib comes up with, those are going to change. And that took 10 years in the making, apparently, to change that. So this this library has been around for a while. Anyway, so he would love to have feedback. So we've got a list of in our show notes, there's a list of changes, there's a list of releases, list of changes, and 2.0.0 is coming out soon-ish. He's not sure when, But he'd like to have people try it out and see if any of the default styles or what they think of him He wants feedback. So well, I have a little feedback for him that one of the posts you link to in the show notes It is basically a set of pictures old version new version with default styling Almost universally they look better to me. They look more modern They don't look like here's a cheap unstyled sort of thing. I'm just gonna drop in They actually look like something that you'd be really excited to share, right?

11:34 So thumbs up for me.

11:36 Yeah, I thought so too.

11:37 And I thought I didn't even think about how hard it would be to change something under the, you know, change the rug out of a lot of people.

11:46 And you know, a lot of people don't follow him or follow Python.

11:49 So I'm glad we're bringing this up to try to get people to go check it out.

11:52 Yeah, I totally agree.

11:53 So it's easy to look through it.

11:55 If you care about Matplotlib, get out there and give them some feedback while you can.

11:59 Hey, before we move on, let me tell you about Rollbar.

12:01 Oh, please tell me about Rollbar.

12:03 I recently sent out an email, like a newsletter email, to a bunch of people taking some of my courses, and it turned out that I had typed in a URL incorrectly.

12:13 So it was like, "Click here to launch this video," and it was like, "Player/Lecture/1234." Well some non-numerical number or a letter got in there.

12:23 My web app was trying to parse that as an integer.

12:27 So I hit send on this email to thousands of people, and then all of a sudden my phone And my phone starts going beep beep beep beep beep, crash crash crash, error in your website.

12:36 And I'm like, what the heck is this?

12:37 I pull it up and like, I immediately see what's going on.

12:39 I see the stack trace.

12:40 I open up my, open up PyCharm, quickly change it, do a push to deploy.

12:45 And within five minutes of sending the email, nobody ever sent me a message.

12:49 I found the error and fixed it.

12:51 And the rest of the people that opened the email didn't crash.

12:53 And that was from Rollbar.

12:54 And if you guys want that same type of behavior, that same type of safety net, you can get it at rollbar.com/pythonbytes.

13:01 - That's awesome.

13:02 I wonder how long you would have figured out on their own without that.

13:06 I would have never known until somebody complained.

13:09 Like so, it was really good.

13:10 I felt bad about it, but it was good that at least I got it fixed right away.

13:14 Yeah, awesome.

13:15 One of the things I'm a huge fan of is MongoDB and document databases.

13:20 I really think document databases are like this perfect sweet spot between performance, maintainability, ease of use, and relational integrity.

13:29 There's a really great article I want to point out if you're interested in MongoDB or document databases in general and Python and you want to get started.

13:36 The guys over at RealPython, Michael Herman, wrote an article called "Introduction to MongoDB and Python." They talk about what's the difference between SQL and NoSQL.

13:45 They talk about the primary way to talk to the database through PyMongo, like inserting and querying data.

13:51 And they also talk about this thing called MongoEngine.

13:54 MongoEngine's like the SQL alchemy.

13:55 You create classes, you put in constraints and defaults, and then you map those classes to documents in your database.

14:02 Yeah, I'm really excited about this.

14:04 I'm trying to use PyMongo on one of my projects, so this is good.

14:07 Yeah, it's definitely good.

14:09 PyMongo is pretty sweet.

14:10 MongoDB is easy to set up and use.

14:12 It's open source, so that's all good.

14:14 I actually did a one and a half hour presentation at this conference in London called Software Architect, and they recorded it and put it up, and it was basically the same stuff.

14:21 So I put a link in the show notes as well to this thing I called applied NoSQL with MongoDB and Python.

14:27 So if people want to like a video version of mostly the same topics, they can check that out as well.

14:32 And actually from you, I learned about TinyDB, I think on one of your podcasts.

14:36 Oh, yeah, TinyDB is awesome.

14:38 Yeah, so that was neat too.

14:39 Yeah, also document database.

14:42 You know, I love things that are for humans.

14:44 Yeah, next up, we've got an article introducing Maya date times for humans.

14:50 And most of the For Humans stuff comes from Kenneth Reitz.

14:55 Do you know how to pronounce his name?

14:56 - I've been saying Reitz, I think that's right.

14:58 It felt like a German name, but if it's Reitz, I'm sorry, Kenneth.

15:01 - So Kenneth, if you're listening, please let us know.

15:05 Or I could, I guess, look at any talk that he's given and try to get that, but anyway.

15:11 But the date times seem like something that's just easy until you actually try to do something that's not.

15:20 It quickly goes from easy to deal with to hard to deal with very fast.

15:26 And so I'm excited that Kenneth has taken this on and tried to come up with his own library for dealing with date times and date time math and parsing dates and human readable stuff.

15:39 It just sounds exciting and I'm gonna give it a try.

15:42 - Yeah, he has Request, which is the most popular Python package ever, downloaded 7 million times a month, which is insane.

15:50 He has records, which is like just right SQL for human sort of thing, and now dates.

15:54 A very, very cool stuff.

15:55 Nice work, Kenneth.

15:56 - Yeah, definitely.

15:57 And another package that I'm interested in that's coming up is an alternative to Flask or something Flask-like.

16:06 - There are more and more reasons to move away from legacy Python to Python, especially Python 3.5 and above because of the cool concurrency stuff.

16:14 So what are the things that people talk about when they talk about Node.js, which I'm not a huge fan of Node.js, but you know, it does have its benefits is it's sort of asynchronous from the start programming model.

16:25 Right?

16:26 Okay.

16:27 Now that's been traditionally hard in Python because the WSGI API doesn't really support it super well.

16:33 Frameworks like Flask and Pyramid, they don't, you can't just write async methods that all of a sudden become async, right?

16:41 like too much it's too much in the built-in stack that doesn't understand async. This guy I don't know what his real name is I can just know him as channel cat created this thing called Sanic which is called a Python 3.5 or higher web server written to go fast and it's based on the API flask it leverages the async and await functionality of Python 3.5 which I think is really really cool it also uses the UV loop which is a really fast asynchronous IO loop for Python 3. It's really cool. So if I want to create some kind of method like a view action method sort of thing, just like Flask, I'll say app.route and give it a like a URL so like slash. But instead of saying def some method, you say async def some method. And you can await throughout your method and all the the cool benefits of letting the threads go on to do other work while you're waiting on databases, web services, files, things like that just happen magically.

17:33 So you can get really good performance numbers.

17:36 Wow, neat.

17:37 Yeah, so it's pretty cool.

17:38 And the fact that it's a Flask, more or less like the Flask API, means you don't have to like start from scratch and learn a totally new thing.

17:44 They actually have some performance numbers compared to other things.

17:47 And they don't have Pyramid in here, which is too bad, because Pyramid flies on Python 3.

17:51 But when you think of asynchronous programming in Python, a Python web framework, typically you would probably come up first with Tornado.

18:00 And they have some tests they did.

18:02 test they did and they said, okay, Tornado did like 2000 requests per second with a 44 second millisecond latency response time basically.

18:11 I said, okay, well, what about Flask?

18:13 Well, Flask did twice as good.

18:14 So like 5000 requests a second, 20 millisecond response time, almost double good in both directions.

18:20 The Sanic thing using Python three in the UV loop is 33,000 requests per second with a three millisecond response time.

18:28 Wow, that's incredible.

18:30 Yeah.

18:31 See, Python's better than legacy Python.

18:33 Yeah, Python definitely.

18:35 That's really cool. So Python 3.5, the async/await is a cool programming model and have this nice web framework to plug in there. This thing's going to go places.

18:43 We're going to be talking about it for a while, I bet.

18:45 Yeah, definitely. At some point I need to talk with you about trying to figure out if I were going to start up a new application, would I care about asynchronous stuff right away?

18:57 Yeah, that's a good question, right? Like how much do you really care?

18:59 really care because I mean, on like, let's say on the talk by fun website, I think I have 10 processes farmed out to handle the request.

19:07 So like, you know, let's just go with flask if I were running flask 5000 requests a second, that's 50,000 requests a second, that's like a pretty insane amount that I'm gonna have to hit before I care, right.

19:18 But still, there are people who care and having that built in.

19:21 That's pretty cool.

19:22 Yeah, definitely.

19:23 I can't believe we're wrapped it up.

19:24 We've got all these done already.

19:26 I know, I know.

19:27 I think we've got a bunch of cool topics.

19:29 I love Sanic.

19:30 I love that Python 3.6 is out.

19:34 And I'm happy to be back and sharing it with everybody.

19:36 It was a nice break, but it's even nicer to be back.

19:38 - Yeah, do you have any extra news you wanna share with us?

19:41 - No, I just stepped away from the computer.

19:44 I went to California, put my toes in the sand, and just kind of forgot about things for a while, and came back refreshed and excited to keep doing what I'm doing.

19:54 How about you?

19:55 You're still working on your book, right?

19:55 Definitely, and I've ramped up the schedule. So I spent the Christmas break at the keyboard and I've got I'm I think about halfway done of with the rough draft approximately But going through the first level editor I'm still I'm still shooting for a beta at least a beta release by Picon the the my goal of getting a physical copy by Picon seems actually insane right now, so but that's where I'm going to do it. We're all rooting for you Brian, you can do it.

20:28 And a whole bunch of test and code episodes coming up. I've got one interview recorded and about four interviews scheduled so these will keep going. I have not quit. All right, awesome. Glad to hear those are coming. Yeah, all right. Well thanks a lot for talking to me today. Yeah, it's been great. Thanks for sharing the news with me and we'll see you later. Thanks.

20:47 Thank you for listening to Python Bytes. Follow the show on Twitter via @PythonBytes bytes that's Python bytes as in b y t e s and get the full show notes at Python bytes dot fm. If you have a news item you want featured just visit Python bytes dot fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Okken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and