Transcript #16: Postmodern Python and Open-source Financial Awards

00:00 This is Python Bytes, Python headlines and news delivered directly to your earbuds.

00:04 It's episode 16, recorded March 6th, 2017.

00:08 And this is your host, Michael Kennedy, here with my co-host, Brian Okken.

00:12 Hey, Brian.

00:13 Hello.

00:14 Hello.

00:14 Well, I'm excited to talk about another week's worth of Python news.

00:18 But before I do, I want to say thank you.

00:20 Thanks to Rollbar for sponsoring this episode.

00:21 Yes, thank you.

00:22 Yeah.

00:23 So you've got the first item, and it's kind of an artistic postmodern type of thing, But with error handling, huh?

00:30 Yeah, so there's an article on Journal Panic, and I got to say, that's a cool name for a website.

00:36 But it was called Postmodern Error Handling in Python 3.6.

00:41 And it's actually-- it's not really, I guess you could consider it error handling.

00:46 It's mostly like how to structure your code so that you don't make mistakes and let the language help you out.

00:54 The examples are--

00:56 I highly recommend the read, because the examples are hilarious. And it's, it goes through some error prevention, looking at using enum classes, and the named tuple class, which I have to admit, I just like learned about I've used named tuples, but I've only learned about the class recently, and using type hints in my pi to to help avoid errors in your code.

01:20 So yeah, I think it's really nice. Some of these are three, four only, right, like enum support was added in Python 3.4, which is pretty cool.

01:29 Yeah, there's somebody who commented, basically the idea was they were talking on some message board like core or something about like, how would you handle this kind of error?

01:39 Let's say, it's not true or false, it's true or false or maybe, right?

01:45 And they propose like this three state type of thing.

01:47 And somebody said, well, if you were using Rust, you could have enums.

01:50 And he says, well, I'm using Python 3, so I have enums.

01:53 Right?

01:54 That was pretty funny.

01:55 - Definitely, okay.

01:56 Oh, and yeah, the tail, it's nice.

01:58 I only noticed this this morning is the tail end of this article references your talk Python, so that's cool.

02:05 - Yeah, oh, that's awesome.

02:06 Yeah, thanks for referencing that.

02:07 That's really cool.

02:08 Yeah, so I think name tuples are so excellent.

02:11 I've really started using name tuples a lot.

02:13 I think they're great.

02:15 Type hints?

02:16 - Yeah, name tuples I've been using a lot because they're very convenient, but the syntax is a little ugly, And the name tuple class actually makes the syntax a little bit cleaner and I think that's cool.

02:30 - Yeah, very cool.

02:31 And of course, mypy, which is great for static type, well, type hinting, I guess, and type discovery.

02:37 - Yeah, and in the show notes, I'm also including a link to another article on mypy and type hints called Python Type Annotations, and that's from the Cactus Group, and it's another pretty good tutorial for type hints in mypy.

02:52 - Yeah, excellent, excellent.

02:54 So the next one we're gonna talk about, and this is sort of a sequence that I'm gonna go through here.

02:59 One leads to another in a very cool way.

03:01 So mid last year, mid 2016, Mozilla awarded a little over half a million dollars to nine open source projects just in Q2 2016.

03:13 That's really cool, right?

03:14 - That's very cool.

03:15 - Yeah, so they have these things they call tracks, foundational technology tracks, and mission partner tracks.

03:21 And so what they do is they go out and they find open source projects that somehow the success of that open source project furthers the goals of Mozilla, you know, an open source free web that Firefox and other things run well on, right?

03:38 And so why is this on the Python thing?

03:40 Well, they chose a really cool project.

03:43 So they said that they've already funded eight projects for $385,000.

03:49 and they're still considering more, and they have actually applications available for what's called a mission partner.

03:56 So if you have an open source project and you think the mission of your project lines up well with Mozilla, maybe that could be your job, right?

04:03 Like maybe Mozilla would pay you to work on your project.

04:05 So first of all, just check this out.

04:07 That's awesome.

04:08 - Yeah, definitely.

04:08 That's cool.

04:09 - Yeah, they also have a track on secure open source for improving the security of open source software.

04:17 And then the one that's really interesting to me, I think to the audience, is the foundational technology track.

04:23 And one of the things that was awarded there, one of the projects was PyPy, the alternate implementation that's a JIT for Python.

04:32 And it's been working with Python 2.7, but not really great with Python 3.

04:37 It's kind of sort of iffy.

04:39 Some of the features are supported and so on.

04:41 So they received $200,000 in donations to make that happen.

04:45 That's really great.

04:46 And that'll make PyPy a lot more compelling if we can get that updated to 3.

04:52 Yeah, absolutely.

04:53 I mean, it's one thing to say we're on 2.7 and then we'll move in a little bit, but it's another to say, and we're going to build on a technology that has no Python 3 story.

05:00 Like that's a pretty serious blockade.

05:02 But now they have an alpha version of PyPy 3.5 out and it's looking really good.

05:08 So more on that in another section.

05:10 Okay, cool.

05:12 Speaking of announcements, I've just been sort of following what Intel's been doing.

05:17 I think Talk Python had an Intel episode.

05:20 Yeah, we had a great conversation about how Intel's actually thinking about the architecture of their chips so that it lines up with the way that Python executes code, CPython.

05:31 Yeah, and they're continuing working on this.

05:35 The announcement that we're linking to is an announcement from Intel that says, "Intel distribution for Python 2017 update 2 accelerates five key areas for impressive performance gains and so it looks like they're continuing on and this this this version that they've got is both something that it's so it's a special release of Python that's compiled most for knowing that it's going to run on an Intel architecture and it supports Windows, Linux, OS X and It's got two seven and three five oddly three six missing. What's going on guys, but the improvements are pretty cool. There was a comment about widespread optimum optimizations for numpy and scipy FFTs with stated sometimes in could possibly improve 60 X over update one and so there really are hammering into trying to make this fast for some some Intensive stuff. Yeah, that's really cool. You know if you're doing any sort of computational stuff involving numpy and scipy that's That's pretty amazing performance increase and to make it basically as fast as native C and the Intel High performance C libraries. That's really something they also touched on some improvements in memory management What I thought was cool was arithmetic and transcendental expressions from numpy are able to use multi-cores now.

07:01 And I just like that because I don't know what a transcendental expression is.

07:05 I think that's like what your face looks like when you're meditating maybe.

07:08 - Yeah, that sounds awesome.

07:11 It already sounds intriguing.

07:12 What good naming the nomenclature.

07:14 - Yeah, but anyway, I think it's neat that they're both, that they're working on it and also that there is a, it's not just for paid people.

07:22 There is a free standalone version.

07:23 That's cool.

07:25 - Yeah, yeah, that's very cool.

07:27 Okay, the next one I want to talk about that's also about performance.

07:30 But before we do, I wanna talk about errors.

07:32 We don't like errors in our web apps, do we?

07:34 - Definitely not.

07:35 - No.

07:36 So our sponsor this week, Rollbar, will totally help you take the pain out of errors and solve that problem.

07:42 So I use them on my own sites.

07:43 I know many people out there do as well.

07:45 So what you do is you basically install with just a few lines of code, Rollbar into your Flask, Django, Pyramid web apps.

07:54 And if there's ever an error, you'll get a notification, you'll get a report containing all the details that you need.

08:00 So here's some kind of crash, here is the trace back, here's the browser and the platform and everything that was possibly passed in the whole web request right there for you.

08:11 So you'll get notified straight away of any errors and you won't even probably have to debug it.

08:15 You'll just have to look and go, "Oh dear, we have to go fix this." So definitely check them out at rollbar.com and I use them.

08:22 I think they're great.

08:23 Wonderful.

08:24 Yeah, absolutely.

08:25 And thanks for sponsoring the show, guys.

08:26 All right, so let's talk about performance.

08:29 And this is follow on from the Mozilla one.

08:33 So the guys at PyPy released some performance graphs and stuff from their work on implementing Python 3.5.

08:42 And they said, look, the core new thing that we need to work on is this whole async IO story, right?

08:50 That's sort of the kernel of the new stuff in Python 3.5.

08:53 And so what they did is they said, well, let's, let's put out some numbers on how we're doing in that area.

08:58 I want to say thanks to Guy Fickel, who actually sent this over to us to say, hey, you should talk about this.

09:04 They said, look, we're going to take things like Tornado, AsyncIO, Curio, GeoEvent, and Twisted, and we're going to run them on PyPy 3.5 and see how they do.

09:16 And they did pretty well, actually, they have a bunch of graphs, and they're, they're basically five to 10 times faster than CPython for all of those workloads.

09:24 Wow.

09:25 Yeah.

09:26 So, if you could run five times fewer servers because you don't need them with pretty much the same code, that would be pretty handy, right?

09:35 Definitely.

09:36 Yeah.

09:37 That's cool.

09:38 Yeah.

09:39 So, there's a lot of interesting things.

09:40 I feel like this whole let's leverage async I/O plus something else is really blowing up these days.

09:47 a lot of the stuff happening in the web frameworks with Jopranto, with async, AIO HTTP, with Sanic, with this. There's just there's a lot of stuff going on right now trying to do something with async and async and await and async I/O to make things faster. So I think we'll continue to hear good stories around this. Yeah it's cool that it keeps progressing. Yeah absolutely.

10:09 Definitely. Well what's unclear is like which one of these is going to be the path right? Like is Gepronto gonna be the way to go? Is AIOHCP the way to go?

10:17 Like Sand Neck, there's so many sort of flowers blooming it's kind of tough to pick the right one because they all seem so promising in slightly different ways. Yeah and from somebody that's trying to maybe to set up a project and trying to pick, that needs to pick one right now, I could see that that might be a little confusing but I don't think these are terribly different things.

10:40 I don't know, maybe it is. I have no idea what it would be like if you had to like, if you went with, I don't know, Sanic or something and then Sanic disappeared for some reason you needed to switch, how difficult that would be. But at the very least, having a lot of people look at it and try to make things faster is a good thing. Yeah, absolutely. Ned Batchelder is the guy that supports coverage.py and to measure your, that's most people use for coverage of looking at coverage of their code. And he wrote a couple articles called "A Tale of Two Exceptions" and he's got two parts. And there's a... what was going on was he was trying to get all of his test suite to run on Jython. And there was some... I don't know the details of the problem, but there was a an issue with the Jython that made it so that, like, the reporting mechanism doesn't quite work or doesn't work. So it's not a crucial part of the system of coverage, but they didn't work so he wanted to skip the tests that depended on that while running on Jytha.

11:44 And wow, it's an interesting tale. He kind of walks through the entire thought process of why he chose different attempts and maybe inheriting from the exception class and picking another base for the exceptions that the coverage exceptions use and ends up kind of of leaving it not quite wrapped up at the end of the first post and then with some feedback from one of the readers on the first post came up with a way to use decorators and meta classes to apply the decorators to be able to skip those more effectively and it's a couple pretty cool articles so yeah nice work Ned that's a nice write-up so what do you have to do if you want to do coverage on Jython, you basically run it and collect the report. And then you have to like actually somehow process it with CPython.

12:38 Well, I'm not exactly sure what the I'm sure that there's there's enough support to understand your coverage. But the the coverage package has like a whole bunch of cool stuff like generating HTML reports and, and a lot of other type of reports. And it's, it's possible that those are the parts that are that are missing.

12:57 Yeah, yeah, this is definitely an interesting use of meta classes.

13:01 There's some multiple inheritance thrown in there.

13:03 There's a lot of stuff in this actually.

13:05 Yeah, and one of the things that I wanted to point out was, it's refreshing to, I've seen a lot of articles with people saying, "Look, I've got this cool solution I came up with for this particular problem." And it's very refreshing to see somebody say, "I've got a sticky problem with a solution that I'm still not quite happy with it, and here it is." And it's nice.

13:26 And another thing, a good takeaway from it is he didn't present all of the code that he could have.

13:33 He boiled the code that he puts in the articles, he boiled those down so that you could understand the problem, but they're not huge.

13:42 That's extra work on his part, but it makes it for a nice quick read.

13:46 Yeah, a lot of people ask me, "How did you come up with this type of problem?" Or, "Could you explain the thinking that got you here?

13:52 Because I don't see how you got from A to B." And like, this is a good example of like laying that out.

13:56 I think it's nice.

13:57 Yeah.

13:58 And also to kind of prevent people from saying, well, why didn't you try X?

14:02 Because he already did and he's showing it.

14:04 So it's nice.

14:05 Yeah.

14:06 Nice.

14:07 All right.

14:08 Check that one out.

14:09 That's very cool.

14:10 The last one is also about async and OA and it's also about performance.

14:11 So, but this is a totally different story.

14:14 So all this stuff that we've been talking about, the Mozilla thing, the PyPi, the PyPi tests that they ran, the majority of those were testing web frameworks.

14:22 So I want to write a web server.

14:23 It's going to process some requests.

14:24 Let's do it faster.

14:26 This is totally different.

14:27 This is about I want to consume services really quickly.

14:32 And the AIO HTTP library actually has some really cool stuff to do this that I just learned about.

14:38 So I thought I'd share it with you guys.

14:39 Wow.

14:40 That's great.

14:41 Yeah.

14:42 So you've heard of requests, right?

14:43 Like the most downloaded package ever?

14:44 Well, definitely.

14:45 Yeah.

14:46 Yeah.

14:47 So we all know about requests, right?

14:48 It's downloaded like 7 million times a month or something insane.

14:50 Well, AIO HTTP has a similar library as request.

14:55 It's actually very, very similar in the way that you use it and its features and so on.

15:00 However, request itself is not...you can't await it in async and await, right?

15:07 So it doesn't like use some sort of deferred IO callback in order to complete requests.

15:13 It's just blocks.

15:14 So the big difference with AIO HTTP is you can await the responses at different levels.

15:21 You can do it on the network calls, you can do it on the parsing, you can actually do it, even has like a file based thing so you can await writing the files.

15:28 And so this person wrote up a cool little example, putting those both side by side.

15:33 And the code is quite similar.

15:35 If you didn't have async and await, it would be not so similar, it'd be really nasty looking.

15:39 But because you do, it becomes real, real similar.

15:42 So what we're going to do is I want to get a bunch of stats about basketball players in the NBA.

15:46 There's an API for that apparently.

15:48 So it's going to run some code and it's going to go collect all the stats and it took 12 minutes on requests.

15:54 Using AIOHCP and AIO files, it took 22 seconds.

15:59 Wow, that's really awesome.

16:01 Right?

16:02 That's 33 times faster.

16:03 Definitely.

16:04 And the code is virtually identical.

16:06 That's pretty cool.

16:07 So the difference is like basically, you begin a request to the API.

16:11 normally you're just waiting on the network, right?

16:13 You're waiting for a response, right?

16:15 But you should be able to kick off a whole bunch more of those requests until one of them comes back and you have to process them.

16:22 So it doesn't even use threads to pull this off.

16:24 It just uses IO callback type things.

16:27 - Really? Okay.

16:28 - Yeah.

16:29 Yeah, pretty awesome.

16:30 So this is definitely one of those things that shows the power of Python 3.5 and up.

16:34 - Yeah, it's nice to have it on the client side too.

16:38 We've got a lot of examples recently of async and await on the server side.

16:43 - Yeah, absolutely, absolutely.

16:45 So I wanna squeeze one more piece of news in here.

16:47 - Okay.

16:48 - Yeah, before we wrap it up.

16:50 So I talk about PyPI a lot on Talk Python.

16:54 We grab a lot of packages out of there and talk about them here on Python Bytes.

16:59 There's quite a milestone that just passed there two days ago, three days ago, something like that.

17:02 - Very cool milestone.

17:03 - A very cool milestone.

17:04 So there are now over 100,000 packages on PyPI.

17:09 How cool is that?

17:10 - It's very cool.

17:10 I wonder if the guy that, or the, I shouldn't say guy, the person that got the 100,000th one in there, if they know about it.

17:18 - If they know that they are the one that put it over the top?

17:21 Yeah, we need to find out.

17:22 We need to contact like Donald Stuft and see if he knows, if he can find out.

17:26 I bet there's some sort of query that can be done that'll answer that question.

17:28 - Yeah, that'd be cool.

17:29 - That'd be very cool.

17:30 Okay, all right, that's it for me.

17:33 Got any news to share with anyone?

17:34 - Just that I've been in the throes of trying to switch over.

17:38 I switched over the Testing Code podcast to a new domain name at testingcode.com.

17:45 And hopefully that should be seamless to anybody that's already subscribed.

17:50 But yeah, just if you see anything, that's going on.

17:55 We should let you know.

17:56 Okay, awesome.

17:57 Testing Code, very cool.

17:58 So congrats.

17:59 You have a whole new system driving it and everything, right?

18:01 Like a whole new website.

18:02 It's basically hands off for me.

18:04 I'm letting Software as a Service do most of the work for me.

18:08 - Sounds very relaxing.

18:09 (laughing)

18:10 - Now, I was gonna switch the, I have Test Podcast as the Twitter thing, with a bunch of followers, and I also have Test & Code as a Twitter, but there's only like four people following, so I have no idea what to do with that.

18:25 How about you, got anything you wanna announce?

18:27 - No, not too much, I'm just continuing to working on classes, creating more online classes.

18:32 So I actually have a surprise one coming that nobody is, I'm sure will be unexpected, I think it'll be fun and maybe next week I can talk about it.

18:39 >> I'll practice my surprised voice.

18:41 >> Yeah.

18:42 All right.

18:43 Well, thank you everyone for listening.

18:46 Brian, great to talk to you as always.

18:48 >> As always.

18:49 See you.

18:50 >> Thank you for listening to Python Bytes.

18:52 Follow the show on Twitter via @PythonBytes.

18:54 That's Python Bytes as in B-Y-T-E-S.

18:58 And get the full show notes at PythonBytes.fm.

19:01 If you have a news item you want featured, just visit PythonBytes.fm and send it our way.

19:06 the lookout for sharing something cool.

19:08 On behalf of myself and Brian Okken, this is Michael Kennedy.

19:12 Thank you for listening and sharing this podcast with your friends and colleagues.