Brought to you by Michael and Brian - take a Talk Python course or get Brian's pytest book


« Return to show page

Transcript for Episode #21:
Python has a new star framework for RESTful APIs

Recorded on Wednesday, Apr 12, 2017.

00:00 KENNEDY: Hello and welcome to Python Bytes, Python news and headlines delivered directly to your ear buds. Today is Wednesday April 12th, 2017 and this is Episode #21. I'm Michael Kennedy.

00:00 Okken: And I’m Brian Okken.

00:00 We're going to tell you about all the cool things that we found this week in the Python eco-space.

00:00 before we get to that, Brian, let's say, ‘Thanks.’

00:00 Well. thank you, Michael.

00:00 Yes, you’re welcome. Thanks to Rollbar, actually.

00:00 Oh, yeah, yeah, definitely. Thank you, Rollbar.

00:00 Yeah, thanks. Thank you, Rollbar. They’re sponsoring this episode again and if you want to check out Rollbar, get some special deals, it's Rollbar.com/pythonbytes. We'll talk more about them later.

00:00 let's talk about some profiling.

00:00 Doug Helman, he's the guy that did the Python Module of the Week – I don’t even know when that started – for Python 2, and I can’t remember when he started the third one. I don't know if we've mentioned it on the show yet but it's a really good resource. It’s PyMOTW.com (Python Module of the Week). He goes through a lot of the standard library and just does a little page on how to use different bits of the standard library and this recently put up a profile and pstats. It talks about Profile and C Profile for basically just for if you have a piece of your code that you think might be slow and how to Profile that and figure it out.

00:00 Yeah, definitely. I think this is awesome. You know, Doug, way to go. Very, very cool and keep going with that. We’re going to be checking out, of course, as you roll out new ones.

00:00 definitely want to encourage people to think about profiling. I've been doing a lot of work recently on certain parts of my web app where performance is somewhat important and things have changed. For example, I switched out the database back end and depending on how that was working, it used to be fast but became slow in some situations. Maybe I needed to restructure the way my data was stored maybe I needed new indexes or whatever. Even just profiling the web application itself actually can tell you a lot about not just where your Python is slow but where the web services you're connected to are slow, or the databases you're talking to are slow, or you can see something like you made 600 requests to the database during this page load. You can really bet there is probably some sort of lazy loading problem going on right. So, sometimes just a cursory glance at profiling stats will tell you like, ‘Oh, I didn't realize I was doing that wrong’ and you quickly fix it. So, definitely check that out.

00:00 The built-in profile stuff in C Profile works great for a lot of this stuff I work on, but I'm not often working on web stuff. Are those the right tools for something like web applications or are there other things that you search for?

00:00 One of the things that can be challenging about profiling web apps is there's a lot of startup code that runs and that can be like computational. So, if you try to profile your apposite starts up it could be swamped with actual, just-getting-the-thing-going traffic. Then you hit a request for a page and it's small enough it doesn't show up, so maybe you got to hit it 100 times, or you know something to get it to happen. So, what I do – I don’t know about the other frameworks, pretty sure they must have something like this – but in pyramid, by default there's a little red box on the right side in the development mode and you can click on it. It will take you to – it’s called the Pyramid Debug Toolbar – it takes you to the back there and one of the options is to turn on profiling per page request. You just check a box and then request a page. It'll tell you how many milliseconds it took and if you click on it, it will give you the output from this just for that page.

00:00 That’s pretty cool.

00:00 You don’t have to do anything. Yeah, it’s super cool.

00:00 Alright, well, thanks.

00:00 Yeah, sure.

00:00 of the web, we have a new project coming out that I wanted to bring everyone's attention to. I don't know how old it is, it's like weeks, maybe months, I'm not sure; It's not very old.

00:00 by Tom Christie. You may know Tom Christie from the Django REST framework. He came up with something called API Star, and this is an alternate REST framework. So, Django REST framework has been around for like, five years. It's very popular and well-known, it’s highly used, but it's also very much wedded with the Python 2 prior-web world. It's from 5 years ago, and it's based on stuff that was sort of put in motion before then. If you want to explore some new and interesting ideas, you can't really do it and that's what Tom was saying. So, he actually came up with this thing called API Star and it has got some slick features that are really worth looking at. It’s really new, so I don't know if it's quite ready for primetime; I just honestly don't know, I'm not saying it's not. But the idea is, he calls it a smart web API framework designed specifically for Python 3. It’s super easy to get started. Just pip install apistar, apistar new – template minimal. You can either run it or run the test straight away.

00:00 I thought you'd like that little API star space test. That's pretty cool.

00:00 Yeah, and a nice interface, too. It looks really clean.

00:00 Yes. Here's where the magic comes in. It's all about the type annotations. So, in Python 3, you can say like, I could define a function called, ‘Show query parameters’ and I could have a thing called ‘query parameters’. If I wanted it to be of type, http.QueryParams, I would say :http type QueryParams.

00:00 tells Python, not at run time but at design time or development time, ‘Hey, this thing supposed to be a query params instance’, right?

00:00 Yeah. Okay, that’s cool.

00:00 Okay, so check this out. If you put a parameter on that API method and you say it is one of those and it's available as part of the request, it will automatically come into your system regardless of what you call it and where you put it. It looks at the type, hence the type parameters; ‘So, the thing you're saying, this thing called query params? I have one of those in the request.’ Similarly, if you return something from there by default, your return dictionaries like all these web frameworks in Python do, but you can also use type annotation as to say, ‘No, this return is a response.’ And then return something else, actually using the type annotation to control how the processing of their quest is done. Similarly, you can put things in like your url, so like, /users/seven. Normally that would all just come in as a string, but in this API, you can say, like as a function on your rest method, you can say, ‘user_id:int’ and it will take that thing and convert it to an int, out of the url, and pass it in. So, there's all this really cool stuff around type annotations that I really love. And, not only that, it's faster than Sanic. So, very cool. Good job, Tom.

00:00 Yeah. I know you said that already and I kind of went past me without me recognizing it, but the built-in testing with PyTest, yay!

00:00 It's awesome, right?

00:00 Yeah, definitely cool.

00:00 Yes, it's a really new framework. Check it out. It's already got quite a few stars on GitHub. API Star. There's a link in the notes and some little example code as well.

00:00 Alright.

00:00 But that's about fast web frameworks. Sometimes we don't care if things are fast, right?

00:00 Yeah, so actually we we're kicking around the idea of talking about this last week and decided not to but we're putting it in now. It's an article called, “Yes, Python is Slow, and I Don't Care.”

00:00 I never really noticed it being slow for what I have it for, but basically that's kind of the point of the article. Fast is great. Having Sanic and API Star be super-fast, of course that's wonderful. But there's a lot of Python uses for Python and for other programming, that the most expensive resource is not your CPU. Actually, it's very seldom your CPU anymore, so therefore you usually don't really have to care about the speed of Python and I'd have to agree. A big chunk of this article is, ‘You should be optimizing for your most expensive resource and that's your people.’ Your people are way more expensive than your computer time.

00:00 And so are bugs.

00:00 Yes, and definitely, so are bugs. I know that myself, we have, at work, we use Python for our testing and it's because we can write smaller test code. The time our tests run, is all in iO time. It's communicating with instruments, it's not sitting in our computer. So, I can have small, elegant tests that I can read well is way more important than how fast Python is.

00:00 Yeah, you’re talking about testing devices and hardware. The same thing applies to the web. Usually waiting on either a database, a web service or a socket somewhere, and the timing usually doesn't matter, right?

00:00 Yeah. Well, of course you wanted to be fast but the slow part isn't the Python.

00:00 I guess that ties in with the other articles we've had so far. Developing an API quickly and elegantly is going to eliminate mistakes better. If you can do that with less code, great.

00:00 Absolutely. And if things are slow, there's always options, right? You could write it in C. You could write in Rust. You can write in Cython. You could take that little bit and do something. Or just change, if you start at the beginning, profile it and change your algorithm. It very well may be that, that makes a huge difference.

00:00 Yeah, it might be you think it's Python, but like you said, it might be your database or your accessing some resource or something. Figure that out.

00:00 if it really is Python, which I think there are rare cases where it really is. But if it is there's options, with Cython, taking a look at Intel's offering, or Pypi or something.

00:00 Exactly, I just saw some stuff that Pypi is supposed to be getting quite a bit faster still. And of course, they got that big grant from Mozilla to make Python 3 work much better so, all sorts of goodness there.

00:00 Definitely.

00:00 Now before we get to the next item, let me tell you about Rollbar.

00:00 guys probably heard me talk about it before, but I've been using Rollbar for a long time. The idea is, you basically put a few lines of code into your web application. If ever there is an error, it will automatically capture all the information that you need about it and ship that over to Rollbar and then send you a notification all sorts of different ways you can get notifications and it comes with all almost all the context you need to actually solve the problem. For example, it might have the stack trace and the local variables in the url, and all those sorts of things that you might not even have to debug your code. You can just look it and go, ‘Oh, dear. I need to check for this’ or make sure that's not done or whatever.

00:00 if you want more reliable web apps just be sure to check out Rollbar.com/pythonbytes.

00:00 Wonderful. Thank you, Rollbar. We have Rollbar to thank for why Python Bytes website is always up.

00:00 That's right, absolutely. It's integrated in all the sites. It's beautiful.

00:00 one of the things that, I kind of want to take it back to the basics for a minute, okay? One of the things that I think people many people know about but a lot of people don't, is hashing. When to use hashing, how to use hashing and things like that.

00:00 there's a nice article called, “A Quick Introduction to Hashing” by Gerald Nash and this is specifically for Python developers. It has some nice examples of like, if I want to hash a string using say, sha256 or something like that for the hash, how do I do that? What does that look like?

00:00 Can we back up a second? What is a hash?

00:00 Sure. So, a hash is a little bit like encryption, in that you can look at the hash and basically, it's obfuscated, right? You don't really know what it is. But unlike an encryption, it can't be reversed. It can be un-hashed. You might take a 10-megabyte set of data and hash that into 256 characters or something like that. The idea is, if you have the same input and you run the hash over and over again, you should get the same output. Even if it varies slightly, you'll get a totally different output not a little bit different but dramatically, like unrelated, sort of-type of thing.

00:00 it’s used for verifying messages, verifying download files are not tampered with, but most importantly, it’s used for passwords. Storing raw passwords in a database, not the best. So, we typically hash them, but there's all sorts of other interesting uses you can use for hashing. In my websites, I have the static files, JavaScript, CSS images etc. basically, cached like, super-hard; they’re cached for a year, at least. I think maybe possibly longer. So, if you go to the site and you come back later, it's not going to go back and look again. What I do, the technique to make sure that nothing ever shows up stale on the site – you’d have to wait a year for it to come back together – is I put a little hash of the raw files at the end of the url of every bit of static content that I have. So, it caches it, but it caches it with the fingerprint of it embedded. If the file ever changes, it instantly generates a new file and it pulls it out. So, hashing is used for all sorts of good stuff.

00:00 great little article and shows you guys how to do that. If you're not taking advantage of hashes, they’re super easy. Here's a nice way to see how to do it.

00:00 Wonderful.

00:00 is a very fun article – I forgot to write this person's name – but there is an article called, “Wedding at Scale: How I used Twillio, Python and Google to Automate My Wedding.”

00:00 I feel like I missed this opportunity when I got married.

00:00 Yeah and this is just fun. Actually, the modern person's problem with like, how to invite people and I love this article. The nerd in me likes it. There's no way I would have had time to figure this out when I was trying to plan my wedding, but if you’ve got the time, it's great. He threw all the potential guests and their phone numbers into Google Spreadsheet and then used Python and a Python package called G Spread to access the Google Spreadsheets and pulled that out. With Python, he used Twilio and SMSd people an invitation to the wedding. And then not only that, left urls for people to get back to him; there's urls and there's also replies. He set up a Flask app to collect everything, of all the replies to that. And then he even goes on to sending out reminder texts and having people select which food option they wanted; it's pretty incredible this story.

00:00 Yeah, a real mashup of a of bunch of cool technologies. That's awesome.

00:00 Yeah, it’s cool.

00:00 Nice. I have another fun one for us, for our last item here today. I have to be careful about how I speak about this device, because my understanding is speaking of it causes it to do things. So, let's call it the Amazon virtual assistant thing that sits on your desk, alright?

00:00 Okay.

00:00 There's a guy who created a framework for creating what are called skills on this assistant. You can teach it to do new things. The guy who wrote this article is Neil Stewart. What I thought was really interesting was, he was visiting a friend or something like that, and they had one of these assistants. He thought it was really cool, he’s like, ‘I'm ordering one.’ But he decided like, ‘I'm going to challenge myself to build a new skill for it before it even arrives.’

00:00 as developing for hardware that's a pretty interesting idea, to develop for it before it exists in your hands, right?

00:00 Yeah, we do it all the time.

00:00 The thing that he came up with is something called VoiceOps. He has a bunch of AWS (Amazon Web Services) accounts and whatnot, and he can ask it things like, how’s status of AWS servers and what not. It will tell him how many running and stopped instances there are, what RDS (Amazon Relational Database Service) databases are on his account and things like that.

00:00 that's a pretty fun skill to add to this assistant.

00:00 Yeah, I didn't know you could do that, add skills to it. That's pretty cool.

00:00 Yeah, and apparently, you can do it in Python, which is even cooler. He looked around and saw that there's a few things, but he wasn't really happy with it so he decided to create a new framework called Python, the name of the thing (Alexa) and it's also on GitHub. The article here that we have linked, this one actually has a bunch of cool examples on how to affectionate, he has a hello world type of thing. He also links to something called Echo Shim.

00:00 Shim allows you to create one of these skills, put it into AWS and on their lambda system on Python and then test it in the browser without even having hardware.

00:00 Cool.

00:00 Yeah. So, pretty fun projects if people are into that assistant thing and Python. Check it out.

00:00 I will check it out. That’s neat.

00:00 Do you have one of those things? Any of those, like a Google Home or any of those types of things?

00:00 No. Actually, that whole concept of like letting some other company just listen to my house I don't like the idea.

00:00 Yeah, I'm not there yet either. I'm not super crazy about privacy and what not. The location’s on my smart phone stuff, but that somehow like crosses a little bit of a line to me. I don't know. I'm sure I'll be over it someday.

00:00 Plus, I don't trust my kids to not just order a bunch of stuff all the time just by talking to it.

00:00 Yeah, after what happened the first time I got a phone that had Siri on it, I know what the kids did to that thing. This is even more accessible to them, so I'm not sure I want that anywhere near my kids. (Laughs)

00:00 that's it for the week. Brian, what you got going on with you personally?

00:00 I had an interview with Casey Rosenthal from Netflix on chaos engineering that I finally got edited and out last week on testing code, so that's out there.

00:00 Like Chaos Monkey, Chaos Gorilla, all that kind of stuff they do at Netflix?

00:00 Yeah. Basically, we tried to talk about not just what they're doing there but the concept of taking steady state measurements and doing experiments that might change the study state on a system, as opposed to traditional testing methods. It's pretty interesting.

00:00 Nice. I’m definitely going to listen to that. That’s great.

00:00 How about you?

00:00 So, I spent the weekend reworking a bunch of my web stuff around the training courses, training site and so on. I rebuilt a new player because I wanted to surface the closed captioning a little bit better for people, things like that. I did a bunch of JavaScript and whatnot. The other thing I did is, I added full text search to the whole site. So, you go there and you can search into the videos even, into the text of the videos, so it's beautiful. This is what I was actually thinking about when we were talking about profiling. When I first wrote this, I wrote what you might think of as naively, I wrote it and it's all backed by like a bunch of text data and MongoDB. It’s not using full text in Mongo, it’s using something else that I've done. But basically, the back end is Mongo and there's a bunch of data in there, like 10-megs/20-megs, I'm not sure exactly how much of text that I wanted to search. I ran and thought, ‘This would be nice and fast.’ It was like 600 milliseconds a page, which like is acceptable maybe for other people. For me this is not acceptable. I'm like, ‘No way. This is not going to stay. What is wrong with this?’ So, I spent a lot of time in the profiler and I’ve gotten it down to 47 milliseconds.

00:00 Oh, okay, cool.

00:00 It's more than 10 times faster. That's like, 10.5 times faster by using the profile. It was just, you know, ‘This part is slow. How can I do this differently?’ If you want to see the fruits of that profiling, you can click the link at the bottom.

00:00 Cool. Can we get that text search on Python Bytes?

00:00 Yes, absolutely. We're getting full text search on Python Bytes soon.

00:00 think we need it, though. We have a bunch of transcripts that we have interesting stuff in, we've got a pretty solid show notes for this podcast; I think we need a full text search. So, yeah, we'll be sure to announce it shortly.

00:00 Okay, great.

00:00 I need one more weekend, I guess.

00:00 I’m adding work to you, on air. Sorry about that.

00:00 That's all right. It's on the road map.

00:00 thanks so much for finding these things and sharing with everyone.

00:00 Thank you.

00:00 You bet. Catch you next time.

00:00 you for listening to Python Bytes. Follow the show on Twitter via @pythonbytes and get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbyes.fm and send it our way. We’re always on the lookout for sharing something cool. On behalf of myself and Brian Okken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.

Back to show page