Transcript #16: Postmodern Python and Open-source Financial Awards

00:00 This is Python Bytes, Python headlines and news delivered directly to your earbuds.

00:04 It's episode 16, recorded March 6, 2017.

00:07 And this is your host, Michael Kennedy, here with my co-host, Brian Okken.

00:13 Hey, Brian.

00:13 Hello.

00:13 Hello, hello.

00:14 Well, I'm excited to talk about another week's worth of Python news.

00:18 But before I do, I want to say thank you.

00:20 Thanks to Rollbar for sponsoring this episode.

00:21 Yes, thank you.

00:22 Yeah.

00:22 So you've got the first item, and it's kind of an artistic, postmodern type of thing,

00:28 but with error handling, huh?

00:30 Yeah.

00:30 So there's an article on Journal Panic, and I've got to say that's a cool name for a website.

00:36 But it was called Postmodern Error Handling in Python 3.6.

00:40 And it's actually, it's not really, I guess you could consider it error handling.

00:46 It's mostly like how to structure your code so that you don't make mistakes and let the language help you out.

00:53 The examples are, I highly recommend the read because the examples are hilarious.

00:59 And it goes through some error prevention, looking at using enum classes and the named tuple class,

01:09 which I have to admit, I just like learned about, I've used named tuples, but I've only learned about the class recently.

01:15 And using type hints in mypy to help avoid errors in your code.

01:21 Yeah, I think it's really nice.

01:22 Some of these are 3.4 only, right?

01:25 Like enum support was added in Python 3.4, which is pretty cool.

01:29 Yeah, there's somebody who commented, basically the idea was they were talking on some message board, like Cora or something,

01:36 about like how would you handle this kind of error.

01:39 Let's say, it's not true or false.

01:42 It's true or false or maybe, right?

01:43 And they proposed like this three-state type of thing.

01:47 And somebody said, well, if you were using Rust, you could have enums.

01:50 And he says, well, I'm using Python 3, so I have enums, right?

01:54 That was pretty funny.

01:54 Definitely.

01:55 Okay.

01:56 Oh, and yeah, the tail, it's nice.

01:58 I only noticed this morning is the tail end of this article references your talk Python, so that's cool.

02:05 Yeah, oh, that's awesome.

02:05 Yeah, thanks for referencing that.

02:07 That's really cool.

02:08 Yeah, so I think named tuples are so excellent.

02:11 I've really started using named tuples a lot.

02:13 I think they're great.

02:14 Type hints?

02:16 Yeah, named tuples I've been using a lot because they're very convenient.

02:20 But the syntax is a little ugly, and the named tuple class actually makes the syntax a little bit cleaner, and I think that's cool.

02:29 Yeah, very cool.

02:30 And, of course, mypy, which is great for static type, well, type hinting, I guess, and type discovery.

02:37 Yeah, and in the show notes, I'm also including a link to another article on mypy on type hints called Python type annotations,

02:45 and that's from the cactus group, and it's another pretty good tutorial for type hints in mypy.

02:52 Yeah, excellent, excellent.

02:53 So the next one we're going to talk about, and this is sort of a sequence that I'm going to go through here.

02:59 One leads to another in a very cool way.

03:01 So mid-last year, mid-2016, Mozilla awarded a little over half a million dollars to nine open source projects just in Q2 2016.

03:12 That's really cool, right?

03:14 That's very cool.

03:14 Yeah, so they have these things they call tracks, foundational technology tracks, and mission partner tracks.

03:21 And so what they do is they go out and they find open source projects that somehow the success of that open source project furthers the goals of Mozilla,

03:31 you know, an open source free web that Firefox and other things run well on, right?

03:37 And so why is this on the Python thing?

03:40 Well, they chose a really cool project.

03:42 So they said that they've already funded eight projects for $385,000, and they're still considering more.

03:51 And they have actually applications available for what's called a mission partner.

03:56 So if you have an open source project and you think the mission of your project lines up well with Mozilla, maybe that could be your job, right?

04:03 Like maybe Mozilla would pay you to work on your project.

04:05 So first of all, just check this out.

04:07 That's awesome.

04:07 Yeah, definitely.

04:08 That's cool.

04:08 Yeah.

04:09 They also have a track on secure open source for improving the security of open source software.

04:17 And then the one that's really interesting to me and I think to the audience is the foundational technology track.

04:22 And one of the things that was awarded there, one of the projects was PyPy, the alternate implementation that's a JIT for Python.

04:32 And it's been working with Python 2.7, but not really great with Python 3.

04:36 It's kind of sort of iffy.

04:38 Some of the features are supported and so on.

04:40 So they received $200,000 in donations to make that happen.

04:45 That's really great.

04:46 And that'll make PyPy a lot more compelling if we can get that updated to 3.

04:52 Yeah, absolutely.

04:53 I mean, it's one thing to say we're on 2.7 and then we'll move in a little bit.

04:56 But it's another to say, and we're going to build on a technology that has no Python 3 story.

05:00 Like that's a pretty serious blockade.

05:02 But now they have an alpha version of PyPy 3.5 out and it's looking really good.

05:08 So more on that in another section.

05:10 Okay, cool.

05:11 Speaking of announcements, I've just been sort of following what Intel's been doing.

05:16 I think Talk Python had an Intel episode.

05:20 Yeah, we had a great conversation about like how Intel's actually thinking about the architecture of their chips

05:26 so that it lines up with the way that Python executes code.

05:31 CPython.

05:31 Yeah, and they're continuing working on this.

05:34 The announcement that we're linking to is an announcement from Intel that says,

05:39 Intel distribution for Python 2017 update 2 accelerates five key areas for impressive performance gains.

05:47 And so it looks like they're continuing on.

05:50 And this version that they've got is both something that it's a special release of Python that's compiled for knowing that it's going to run on an Intel architecture.

06:02 And it's a good thing.

06:04 And it's got 2.7 and 3.5.

06:07 Oddly 3.6 missing.

06:09 What's going on, guys?

06:10 But the improvements are pretty cool.

06:12 There was a comment about widespread optimizations for NumPy and SciPy FFTs with stated sometimes could possibly improve 60x over update 1.

06:26 And so they really are hammering into trying to make this fast for some intensive stuff.

06:34 Yeah, that's really cool.

06:35 You know, if you're doing any sort of computational stuff involving NumPy and SciPy, that's pretty amazing performance increase.

06:43 And to make it basically as fast as Native C and the Intel high-performance C libraries, that's really something.

06:50 They also touched on some improvements in memory management.

06:54 What I thought was cool was arithmetic and transcendental expressions from NumPy are able to use multicores now.

07:00 And I just like that because I don't know what a transcendental expression is.

07:05 I think that's like what your face looks like when you're meditating, maybe.

07:08 Yeah, that sounds awesome.

07:10 It already sounds intriguing.

07:12 What good naming in nomenclature.

07:14 Yeah, but anyway, I think it's neat that they're both working on it and also that there is a – it's not just for paid people.

07:22 There is a free standalone version.

07:23 That's cool.

07:24 Yeah, yeah.

07:25 That's very cool.

07:26 Okay.

07:27 The next one I want to talk about is also about performance.

07:30 But before we do, I want to talk about errors.

07:32 We don't like errors in our web apps, do we?

07:34 Definitely not.

07:35 No.

07:35 So our sponsor this week, Rollbar, will totally help you take the pain out of errors and solve that problem.

07:41 So I use them on my own sites.

07:43 I know many people out there do as well.

07:45 So what you do is you basically install with just a few lines of code, Rollbar into your Flask, Django, Pyramid web apps.

07:54 And if there's ever an error, you'll get a notification.

07:56 You'll get a report containing all the details that you need.

08:00 So here's some kind of crash.

08:03 Here is the traceback.

08:04 Here's the browser and the platform and everything that was possibly passed in the whole web request right there for you.

08:11 So you'll get notified straight away of any errors and you won't even probably have to debug it.

08:15 You'll just have to look and go, oh, dear, we have to go fix this.

08:17 So definitely check them out at Rollbar.com.

08:20 And I use them.

08:22 I think they're great.

08:23 Wonderful.

08:23 Yeah, absolutely.

08:25 And thanks for sponsoring the show, guys.

08:26 All right.

08:27 So let's talk about performance.

08:28 And this is follow on from the Mozilla one.

08:32 So the guys at PyPy released some performance graphs and stuff from their work on implementing Python 3.5.

08:41 And they said, look, the core new thing that we need to work on is this whole asyncio story.

08:49 Right.

08:50 That's sort of the kernel of the new stuff in Python 3.5.

08:53 And so what they did is they said, well, let's put out some numbers on how we're doing in that area.

08:58 I want to say thanks to Guy Fichel, who actually sent this over to us to say, hey, you should talk about this.

09:03 They said, look, we're going to take things like Tornado, Async IO, Curio, G-Event, and Twisted,

09:11 and we're going to run them on PyPy 3.5 and see how they do.

09:15 And they did pretty well, actually.

09:17 They have a bunch of graphs, and they're basically five to ten times faster than CPython for all of those workloads.

09:24 Wow.

09:25 Yeah.

09:26 So if you could run five times fewer servers because you don't need them with pretty much the same code, that would be pretty handy, right?

09:35 Definitely.

09:35 Yeah.

09:36 That's cool.

09:36 Yeah.

09:36 So there's a lot of interesting things.

09:39 I feel like this whole let's leverage Async IO plus something else is really blowing up these days.

09:47 We've had a lot of stuff happening in the web frameworks with Jepronto, with Async, AIo HTTP, with Sanic, with this.

09:56 There's a lot of stuff going on right now trying to do something with Async and Async and Away and Async IO to make things faster.

10:03 So I think we'll continue to hear good stories around this.

10:06 Yeah.

10:06 It's cool that it keeps progressing.

10:08 Yeah, absolutely.

10:09 Definitely.

10:10 What's unclear is which one of these is going to be the path, right?

10:14 Is Jepronto going to be the way to go?

10:16 Is AIo HTTP the way to go?

10:17 Sanic, there's so many sort of flowers blooming.

10:20 It's kind of tough to pick the right one because they all seem so promising in slightly different ways.

10:26 Yeah.

10:26 And from somebody that's trying to maybe set up a project and trying to pick, that needs to pick one right now, I can see that that might be a little confusing.

10:36 But I don't think these are terribly different things.

10:40 I don't know.

10:41 Maybe it is.

10:42 I have no idea what it would be like if you went with, I don't know, Sanic or something and then Sanic disappeared for some reason.

10:49 You needed to switch.

10:50 How difficult that would be.

10:52 At the very least, having a lot of people look at it and try to make things faster is a good thing.

10:57 Yeah, absolutely.

10:58 Ned Batchelder is the guy that supports coverage.py and to measure that most people use for coverage of looking at coverage of their code.

11:11 And he wrote a couple articles called A Tale of Two Exceptions.

11:15 And he's got two parts.

11:17 And there's a, what was going on was he was trying to get all of his test suite to run on Jython.

11:23 And there was some, I don't know the details of the problem, but there was an issue with the Jython that made it so that, like, the reporting mechanism doesn't quite work or doesn't work.

11:34 So it's not a crucial part of the system of coverage.

11:38 But they didn't work.

11:39 So he wanted to skip the tests that depended on that while running on Jython.

11:44 And, wow, it's an interesting tale.

11:46 He kind of walks through the entire thought process of why he chose different attempts and maybe inheriting from the exception class and picking another base for the exceptions that the coverage exceptions use.

12:02 And ends up kind of leaving it not quite wrapped up at the end of the first post.

12:07 And then with some feedback from one of the readers on the first post, came up with a way to use decorators and meta classes to apply the decorators to be able to skip those more effectively.

12:21 And it's a couple pretty cool articles.

12:24 Yeah, nice work, Ned.

12:26 That's a nice write-up.

12:27 So what do you have to do if you want to do coverage on Jython?

12:30 You basically run it and collect the report.

12:33 And then you have to, like, actually somehow process it with CPython?

12:39 Well, I'm not exactly sure what the – I'm sure that there's enough support to understand your coverage.

12:44 But the coverage package has, like, a whole bunch of cool stuff like generating HTML reports and a lot of other type of reports.

12:53 And it's possible that those are the parts that are missing.

12:57 Yeah, yeah.

12:58 Yeah, these are – this is definitely an interesting use of meta classes.

13:00 There's some multiple inheritance thrown in there.

13:03 There's a lot of stuff in this, actually.

13:05 Yeah.

13:05 And one of the things that I wanted to point out was I've – it's refreshing to – I've seen a lot of articles with people saying, look, I've got this cool solution I came up with for this particular problem.

13:16 And it's very refreshing to see somebody say, I've got a sticky problem with a solution that I'm still not quite happy with.

13:22 And here it is.

13:23 And it's nice.

13:26 And another thing, a good takeaway from it is he didn't present all of the code that he could have.

13:33 He boiled the code that he puts in the articles.

13:37 He boiled those down so that you could understand the problem, but they're not huge.

13:42 That's extra work on his part, but it makes it for a nice quick read.

13:46 Yeah.

13:46 It's – a lot of people ask me, like, how did you come up with this type of problem?

13:50 Or, like, could you explain the thinking that got you here?

13:52 Because I don't see how you got from A to B.

13:54 And, like, this is a good example of, like, laying that out.

13:56 I think it's nice.

13:57 Yeah.

13:57 And also to kind of prevent people from saying, well, why didn't you try X?

14:01 Because he already did and he's showing it.

14:04 So it's nice.

14:05 Yeah, nice.

14:05 All right.

14:06 Check that one out.

14:06 That's very cool.

14:07 The last one is also about async and OA and is also about performance.

14:11 So – but this is a totally different story.

14:14 So all the stuff that we've been talking about, the Mozilla thing, the PyPy, the PyPy tests that they ran, the majority of those were testing web frameworks.

14:22 So I want to write a web server.

14:23 It's going to process some requests.

14:24 Let's do it faster.

14:25 This is totally different.

14:27 This is about I want to consume services really quickly.

14:31 And the AIO HTTP library actually has some really cool stuff to do this that I just learned about.

14:38 So I thought I'd share it with you guys.

14:39 Wow.

14:39 That's great.

14:40 Yeah.

14:40 So you've heard of requests, right?

14:42 Like, the most downloaded package ever.

14:45 Well, definitely.

14:45 Yeah.

14:46 Yeah.

14:46 So we all know about requests, right?

14:48 It's downloaded, like, 7 million times a month or something insane.

14:51 Well, AIO HTTP has a similar library as requests.

14:55 It's actually very, very similar in the way that you use it and its features and so on.

14:59 However, requests itself is not – you can't await it in async and await, right?

15:06 So it doesn't, like, use some sort of deferred I.O. callback in order to complete your request.

15:13 It just blocks.

15:13 So the big difference with AIO HTTP is you can await the responses at different levels.

15:20 You can do it on the network calls.

15:22 You can do it on the parsing.

15:23 You can actually do it.

15:24 It even has, like, a file-based thing.

15:26 So you can await writing to files.

15:28 And so this person wrote up a cool little example, putting those both side by side.

15:33 And the code is quite similar.

15:34 If you didn't have async and await, it would be not so similar.

15:37 It would be really nasty looking.

15:39 But because you do, it becomes real similar.

15:41 So what we're going to do is I want to get a bunch of stats about basketball players in the NBA.

15:45 There's an API for that, apparently.

15:48 So it's going to run some code, and it's going to go collect all the stats.

15:51 And it took 12 minutes on requests.

15:53 Using AIO HTTP and AIO files, it took 22 seconds.

15:59 Wow.

16:00 That's really awesome, right?

16:01 That's 33 times faster.

16:03 Definitely.

16:03 Wow.

16:04 And the code is virtually identical.

16:06 That's pretty cool.

16:07 So the difference is, like, basically, you begin a request to the API, and normally you're just waiting on the network, right?

16:13 You're waiting for a response, right?

16:15 But you should be able to kick off a whole bunch more of those requests until one of them comes back and you have to process them.

16:21 So it doesn't even use threads to pull this off.

16:24 It just uses IOCallback type things.

16:27 Really?

16:27 Okay.

16:28 Yeah.

16:28 Yeah, pretty awesome.

16:30 So this is definitely one of those things that shows the power of Python 3.5.

16:34 Yeah, and it's nice to have it on the client side, too.

16:38 We've got a lot of examples recently of async and await on the server side.

16:43 Yeah, absolutely, absolutely.

16:45 So I want to squeeze one more piece of news in here.

16:47 Okay.

16:47 Yeah, before we wrap it up.

16:50 So I talk about PyPI a lot on Talk Python.

16:53 We grab a lot of packages out of there and talk about them here on Python Bytes.

16:58 There's quite a milestone that just passed there two days ago, three days ago, something like that.

17:02 Very cool milestone.

17:03 A very cool milestone.

17:04 So there are now over 100,000 packages on PyPI.

17:08 How cool is that?

17:09 It's very cool.

17:10 I wonder if the guy that, or the, I shouldn't say guy, the person that got the 100,000th one in there, if they know about it.

17:18 If they know that they are the one that put it over the top?

17:20 Yeah, we need to find out.

17:22 We need to contact like Donald Stuffed and see if he knows, if he can find out.

17:26 I bet there's some sort of query that can be done that will answer that question.

17:28 Yeah, that'd be cool.

17:29 That'd be very cool.

17:30 Okay.

17:30 All right.

17:31 That's it for me.

17:32 Got any news to share with anyone?

17:34 Just that I've been in the throes of trying to switch over.

17:37 I switched over the Test and Code podcast to a new domain name at testandcode.com.

17:44 And that all, hopefully that should be seamless to anybody that's already subscribed.

17:50 But there, yeah, just if you see anything, that'll be, that's going on.

17:55 We should let you know.

17:55 Okay, awesome.

17:56 Test and Code.

17:57 Very cool.

17:57 So congrats.

17:59 You have a whole new system driving and everything, right?

18:01 Like a whole new website.

18:02 It's basically hands-off for me.

18:04 I'm letting software as a service do most of the work for me.

18:07 That sounds very relaxing.

18:09 Now, I was going to switch the, I have Test Podcast as the Twitter thing with a bunch of

18:17 followers.

18:17 And I also have Test and Code as a Twitter, but there's only like four people following.

18:23 So I have no idea what to do with that.

18:24 How about you?

18:25 Got anything you want to announce?

18:26 No, not too much.

18:28 I'm just continuing to work on classes, creating more online classes.

18:32 So I actually have a surprise one coming that nobody is, I'm sure will be unexpected, but

18:37 I think it'll be fun.

18:37 And maybe next week I can talk about it.

18:39 I'll practice my surprised voice.

18:41 Ah!

18:42 Yeah.

18:43 All right.

18:45 Well, thank you everyone for listening.

18:46 Brian, great to talk to you as always.

18:48 As always.

18:48 See you.

18:50 Thank you for listening to Python Bytes.

18:52 Follow the show on Twitter via at Python Bytes.

18:54 That's Python Bytes as in B-Y-T-E-S.

18:57 And get the full show notes at Pythonbytes.fm.

19:00 If you have a news item you want featured, just visit Pythonbytes.fm and send it our way.

19:05 We're always on the lookout for sharing something cool.

19:07 On behalf of myself and Brian Okken, this is Michael Kennedy.

19:11 Thank you for listening and sharing this podcast with your friends and colleagues.