« Return to show page
Transcript for Episode #29:
Responsive Bar Charts with Bokeh, Flask, and Python 3
00:00 Michael KENNEDY: Hello and welcome to Python Bytes, Python news and headlines delivered directly to your earbuds. This time it’s Episode #29 and we recorded on June 6th, 2017. I’m Michael Kennedy.
00:00 Brian OKKEN: I’m Brian Okken.
00:00 KENNEDY: We're here to share some really cool stuff that we found with you, but before we do, Brian, let's say thanks.
00:00 OKKEN: Yes, let's.
00:00 KENNEDY: Alright, thank you to Rollbar. So, Rollbar is sponsoring the show, as they have many other ones. Rollbar is a great friend and supporter of the show. Be sure to check out what they’re offering at Rollbar.com/pythonbytes.
00:00 We'll talk about that more later. Right now, I want to talk about charts and stuff that are responsive.
00:00 OKKEN: There's an article on the Full Stack Python website by Matt McKay called, “Responsive Bar Charts with Bokeh, Flask and Python 3.” I thought that was a great excuse to play with it because I've used matplotlib. I've used matplotlib a lot. I haven't used Bokeh and I've wanted to and this was a really great walk through. I went ahead and went through it and did the little tutorial and it walks you through writing a Flask app from the start and explains kind of what you're doing. It goes through a little bit of the Bokeh, you’re just creating a bar chart, and it fills it with random data. But it also shows you how to do tool tips and how to fill those in with a function call back. Now I've got a little Bokeh Flask app running on my laptop.
00:00 KENNEDY: Yeah, that's really cool. Yeah. Matt is really good at writing these articles that are focused. They're not too short so they're all fluffy, but they don't drag on, right? You get down to it pretty quickly.
00:00 OKKEN: Yeah and there's a lot of detail missing, I'm sure to really get into it, but having a top to bottom, full example running very quickly and then you can dive into playing with different bits and pieces of it, it was really helpful. I liked it.
00:00 KENNEDY: Yeah, which makes it nice and interactive, as opposed to say matplotlib which you can generate like a P&G, but it's good luck for hovering over it and interacting with it and stuff. That's what's cool about Bokeh.
00:00 OKKEN: We've got an application at work that's actually similar to this, you know the bugs found over so many days; not exactly this, but similar enough to work. We need to have a little, small application at work to generate some data around art, around the test data, that I like to be able to have an interactive chart with.
00:00 KENNEDY: You should definitely check this out, that'd be really cool. Flask is super simple; it sounds all nice and easy.
00:00 OKKEN: Yeah.
00:00 KENNEDY: Speaking of nice and easy, one of the things people ask for a lot… It's easy to get started with a Pyramid, Flask or something like that. You get it running and playing around and you’re like, ‘Alright, I want to put this on a production server and I want to set up load balancing and scaling and redundancy and SSL… And like, Oh, my God, this is a skill I didn't realize I needed but don't have.’ It’s huge amounts of work.
00:00 So, there's a project that I want to highlight, and I don't think we've covered it on the show. Let me know. It's called Zappa and the idea is to take AWS Lambda… Do you know AWS Lambda?
00:00 OKKEN: Yes.
00:00 KENNEDY: So, the idea is AWS Lambda is basically, ‘Here is a function – a Python function – run it please.’ And then you can set that function to be run on AWS events like, ‘Hey this file and S3 changed.’ But there's also something called the API Gateway, which lets you map your domain, SSL certificates, all that, a URL, into a particular function very much like Flask routes or things like this. So, you can kind of set up web endpoints in AWS in Lambda. So, what this is, these guys built this thing called Zappa and the subtitle is “Serverless Python Web Services,” which I love the clash within the short name. ‘Serverless services.’
00:00 Anyway, what it is is basically. it is a Python WSGI application which could be Flask, could be Pyramid, could be Django, whatever, could be its own thing. You write this normal application that looks like a normal web app, but then you can deploy it to AWS Lambda. So, every request comes in, runs in its own little container on Lambda, that's pretty wild, right?
00:00 OKKEN: Yeah.
00:00 KENNEDY: So, what's the deal? Why is this a good thing? There’s drawbacks as well which I’ll maybe touch on, but Lambda, the way you pay for it is, you pay in terms of CPU used, as opposed to if you get some virtual private server, like Ubuntu, or something like this that you have to pay for because it’s turned on. Whereas here, you only pay for per request. If you have something that only takes say, 10,000 requests a day. It could be a really small amount of time that you're actually paying for. How long does it take to serve those requests? Assuming it's pretty quick app.
00:00 OKKEN: I have no idea. You're the web guy. (Laughs)
00:00 KENNEDY: Exactly. But it's pretty cool. You can handle a ton of traffic for just paying for a few milliseconds of server time. It’s pretty cool. You can even do async stuff, which is pretty cool. They've got an example in there where you can call an API and the API can actually kick off another AWS Lambda function, but not in its own execution but in another one. And if you need like a hundred, little doctor container-type things to run this, that's all transparent at the Lamba side, you don't even care. Just say, ‘Here go run this.’ It's a really interesting way to look at building web applications, mostly deploying web applications. The example there deploying is a Flask app, by the way.
00:00 OKKEN: So, every endpoint, like in my path, is going to end up calling a different function and be a different Lambda server?
00:00 KENNEDY: Yes, and actually probably be in its own container that only exists for 40 milliseconds and I think goes away. So, there’s sort of a new instance of the thing created for every single request. Different, right? Very, very different.
00:00 OKKEN: It became neat to see an application built traditionally and then with this, and to compare them if you can even tell the difference.
00:00 KENNEDY: Yeah, it sure would. So, they actually have, not a ton of really popular ones, but they do have at the bottom a couple of apps and some of them are like, ‘The Small Business Registration for Virginia’ website. There’s some sort of a governmental app. The thing that I've noticed about these is, there’s just a little bit more latency than makes me happy, because every request kind of like starts up the whole web server process. There's a little bit of latency in API Gateway and AWS Lambda because it starts up a new container, I think. So, you’re sort of guaranteed a baseline of a couple hundred milliseconds response time, if I understand this right. Maybe it's not that low, I'm not sure, but not that high. There's some extra latency that you pay for, the way Lambda works here, but it's still quite interesting.
00:00 OKKEN: Actually, I was looking down at some of the bottom and of the examples was like a MailChimp sign up utility little micro-service, and actually that totally makes sense. To have a static website but there's something that's a little bit dynamic and has to run aside, something like that would make sense.
00:00 KENNEDY: Yeah, if you have mostly a static site you just want a little interactivity, you can take that little interactivity and make an API or make it a separate thing that runs. It's really cool.
00:00 OKKEN: Okay, cool.
00:00 KENNEDY: Speaking of server-less server stuff, what if PyPI wasn't accessible?
00:00 OKKEN: Yeah, exactly. One of the things we covered in Episode #24 was the notion of creating your own local package store and kind of doing that with the built-in pip. I was contacted by and I'm going to forget his name right now (Dominic Rodger), sorry, somebody that wrote a little blog post, and actually this is an older one, about the same topic called, “Using a Local Cache for Pip Packages.” The reason why I’m highlighting it today is that I like that it's a couple aliases that didn't occur to me before. So, it is pretty much the same thing using pip install--download to download packages into a specific place. And then how to install locally, but aliasing them to a couple names, one called pip cash and one called pip install; of course, you can call them whatever you want. Actually, that’s pretty clever and I probably just take his names because I’ll remember it.
00:00 KENNEDY: Exactly. He's already written the documentation. It is a cool idea. You can basically say, ‘pip install—download, some location’ and then when you want to actually install from it, you say, ‘pip install--no index--fine,’ you give it this link and it's a complete pain remember that stuff, but if you just type, ‘pip cache’ as if you had typed ‘pip install’ and ‘pipinstall’ as if you had typed ‘pip (space) install’ and instead it goes out of this local directory, that's pretty awesome.
00:00 OKKEN: Yeah, and like you said when we were talking before the show, if you had a whole bunch that you knew of, like you're teaching a class or you’re going to get on a plane or something and you wanted to grab a bunch of stuff to work with, just having something like that. And he does highlight also, this works fine for a requirements.text file, so you could just have a a big requirement.text file.
00:00 KENNEDY: Yeah, that's cool. Pip cache-R requirements.txt. Boom, you've got them all.
00:00 I think that's really cool. My idea was it would be great if you could say, somehow tie this together with the data sources that says, ‘Here's the top five hundred packages you might ask for,’ so you kind of pre-load your cache with a bunch of local stuff. So, very likely, if you want requests, if you want Beautiful Soup, whatever, you just got it. I think that would be pretty sweet.
00:00 OKKEN: Yeah, definitely.
00:00 KENNEDY: Very cool. I love this one. It's nice and simple, but you could start using it today without much effort.
00:00 OKKEN: Yes.
00:00 KENNEDY: You know what else is not much effort? Tracking down errors with Rollbar.
00:00 So, Rollbar has been helping me out with my website a lot. I've run into it an issue or two here. The idea is you basically add just a couple of lines and if you’re doing Pyramid, you put it in your config file. Similarly, with Flask or Django, you can add it in there. It even works with other stuff like Node.js, but I don't do Node.js because I don't like it.
00:00 Anyway, for my Python web apps, you can just put it in there, super easy, and it will actually capture all the errors and send them back with details like, ‘What was the call stack? What were the local variables?’ And they added this other cool thing that I'm wanting to play with, I haven't got it working yet, is people tracking. So, like right now everyone who listens to the podcast knows if they find a way to crash my site, I know about it. But I have no idea who did that. I have no idea how to tell them, Hey, sorry the thing that broke, here you could try it, I fixed it.’ So, periodically I'll get emails from people that say, ‘Hey, sorry this didn't work.’ I’m like, ‘I know. I just didn't know how to tell you, but I know. I’m working on it.’ So, there's a new thing called people tracking that you can actually add, if you have a logged-in user, who caused that error and it'll check that for you as well, which is pretty cool. So, check them out at Rollbar.com/pythonbytes.
00:00 OKKEN: The logged-in user, that would be very effective for your courses then.
00:00 KENNEDY: Exactly, because they already have to have an account, there already logged in, so if something crashed for a registered user, then I can actually contact them back and say, ‘Sorry, here's what happened. Here’s how I fixed it.’ Things like that.
00:00 OKKEN: Especially for a service like that, where people are paying for it, being able to tell them, ‘I saw that you had trouble and I fixed it,’ that's cool. That's great. Thanks, Rollbar.
00:00 KENNEDY: Yeah, I think that takes the sting out of it, yeah. Thanks, Rollbar. Very much appreciate the support for the show.
00:00 So, PyCon was fun, right?
00:00 OKKEN: Oh, it was really fun, yeah.
00:00 KENNEDY: Yeah, and one of the things that was cool – there were all these booths that people had set up, so many things, the talks were fun, the people I met was were fun. I felt a little bit bad that I didn't actually get to escape from my booth very far, our booth, because people were excited to come talk to us and that was great.
00:00 But one of the things I did get to go see was the ActivePython folks. They talked about you a little bit as well, so I think that you went and saw them, you must have. What they had built is like a simple little game in Python using PyGame and was like a scroller. Like imagine like 1980s, 1990s, Space Invader-type of standard 2D scroller game. But the thing that was cool was they actually used TensorFlow, Keras, Intel, the Math Kernel Library, and ActivePython to actually derive the AI of the game.
00:00 OKKEN: Wow.
00:00 KENNEDY: Isn’t that cool? You could go over there and they had two screens. It was pretty sweet. On one was the game the PyGame, game scroller thing, you could sit there and play.
00:00 But on the other was actually the neural network like visually doing its thinking while it was fighting you.
00:00 OKKEN: That's pretty cool.
00:00 KENNEDY: I thought it was pretty neat trick. Basically, the AI would figure out when to shoot you, whether or not to shoot you, where to aim, things like that. It was all a neural network. So, the guys over there, Peter… sorry, Peter, I don't remember your last name, Peter wrote this up about some of the lessons they learned, how they basically use PyGame. It's going to be open source they tell me, but it's not yet available. They're still getting the code ready to be put up on GitHub. Maybe, if we're lucky, we can link to the GitHub repo, but if it's really interesting to you, just send the guy a message on Twitter and I'm sure he'll hook you up.
00:00 OKKEN: Well, you have a link to that blog post. I'm sure they'll link it up there, hopefully.
00:00 KENNEDY: Yeah, I'm sure they’ll update it.
00:00 They had a few lessons learned of writing a game that is driven by TensorFlow. They said, ‘Choosing the right data to train your network is super important. Prepping your data is key. Just experiment with the network topology.’ And actually, the last thing I talked about, looking at the neural network think do its magic, they said, ‘Visualization is really important for understanding what it is you’ve built.’ This is a great thing, check it out. They've got a pretty detailed write up on it.
00:00 But on a side note, Brian, do you think we're going to get to a place with all this machine learning and AI and stuff who were the world is powered by programs that nobody understands how they work?
00:00 OKKEN: Do you think we're not already there?
00:00 KENNEDY: (Laughs) That’s a great response. I think were there from a complexity, and just like there’s too much crap and too many layers, but theoretically applying effort, you'll be able to go through and like stack trace it, right? But at some point, when it's just a bunch of neural networks working in orchestration, like why did it decide that? I don't know. It decided that. It's crazy.
00:00 OKKEN: Yeah, I think possibly. I just don't know enough about AI in neuron networks and so it looks like magic to me. Somebody that knows a lot about it, possibly it doesn't look so much like magic and its more obvious how answers could be derived, but I don't know.
00:00 KENNEDY: The craziest example I can think of is where Google got two of their AIs to invent an encryption language and a third one to try to break the encryption language. The AIs actually invented their own encryption that nobody had ever seen, and then the other AI would break it, then the encryption would get harder and just went on and on like that, and nobody knows what happened.
00:00 OKKEN: That's pretty cool. I think some of those things are definitely possible and those are, like you said, complexity building on complexity. But to me, I don't think there's a fear of a singularity thing. I'm kind of a skeptic on ‘The computer is going to take over the world’ scenario.
00:00 KENNEDY: I'm with you, I'm certainly with you. I'm not a skeptic on these types of things disrupting careers and jobs in society.
00:00 OKKEN: Yeah, definitely.
00:00 KENNEDY: I don't think that we're gonna see Skynet right away.
00:00 OKKEN: Hopefully not. Or we won't see it coming, right?
00:00 KENNEDY: Exactly. (Laughs)
00:00 Speaking of things that you want to try to understand with the debugger…
00:00 OKKEN: Yeah. This is an older article from Raphael Pierzina, “Debug Test Failures with Pdb.” Raphael does a lot of stuff with Pytest and that's how I know him. He also works with Cookie Cutter.
00:00 KENNEDY: Yeah, he's a busy guy.
00:00 OKKEN: I'm linking this up because it's pretty clever. I had legitimate error in the the demo code I used on the book and I left it in there because I wanted to show a legitimate error in some code. I also show the fixed version, of course. I used the debugger to understand what was going on with my code. One of the cool things about Pytest is, Pdb is not like, the best. I grew up on things like Visual Studio and Pdb is definitely not Visual Studio-like.
00:00 KENNEDY: It is not exactly the same.
00:00 OKKEN: No, but you can do quite a bit of powerful things with it, like show different variables and show your listing and exactly where it is. Often you have to stick a breakpoint in your code in order to use it, but Pytest has a --pdb option, which just sticks you right in the debugger right at the assert. So, right where your test failed, you'll be right there and you can examine all the variables and whatever, even set up an interactive terminal with all of the local variables there so you can play with them. That's listed in this page, but there's also some of the other features that are great for debugging with Pytest, like being able to run just the last fails, last failures and things like that, are listed in this article. So, I wanted to highlight that from Raphael.
00:00 KENNEDY: Yeah, nice work, Raphael.
00:00 So, if you are at work and you wanted to know if your phone – your voice over IP Cisco phone – was not working at your house, how would you do that? Suppose you have one.
00:00 OKKEN: Amazon pod or whatever the latest thing is?
00:00 KENNEDY: Your home pod from Apple, yeah. You would say something like, ‘Hey, home pod. Is my phone working?’ and it would say, ‘Here's what I found on the internet for you.’ Oh, my gosh, hopefully that thing works someday. (Laughs)
00:00 Anyway, there's a really cool article called, “Monitoring my VOIP provider with Home Assistant.” I want to bring this up, not because the programming is super intense or anything, but it's just a little fun thing that you can do without a lot of work. So, this guy has a Cisco ATA, sort of home voice over IP phone with two lines. His home is set up with home automation in automated bunch of things using something called Home Assistant, home-assistant.io. Have you seen this thing? I think we talked about it before.
00:00 OKKEN: Yeah.
00:00 KENNEDY: So, this is like a full-on Python central hub for your home, which is really cool. What he did was, he plugged in some new code, some new modules for Home Assistant that would look at his Cisco phone and then watch it, and let him know if it ever went down for whatever reason. He's got a new open source package, pyciscopa, I guess. I'm not what the pa stands for. You can use that now if you want to, but it's also does cool push notifications to your mobile phone, which is pretty cool.
00:00 It's not so much that I think people necessarily need to monitor their home phone because, I don't know about you, but I don't even have a home phone. I have a lot of phones in my home, but they're not home phones. They go when people take them. But this sort of integration story, ‘Here's a thing, it wasn't integrated with Home Assistant. Here's a little bit of Python code and magic happens.’ And that's really what I wanted to cover here.
00:00 OKKEN: That's pretty nice and actually, I'm about to get a home phone so I might be able to use this.
00:00 KENNEDY: You will totally know if it goes offline.
00:00 OKKEN: Yeah. So, it looks like the SPA is based on, because it's an SPA ATA device, so whatever that means.
00:00 KENNEDY: Yes, Cisco SBA. Yeah, whatever the hardware is. Okay, cool. That's a fun little thing. It shows you how to basically extend Home Assistant, which if I had a smart home I would be doing all sorts of stuff. Or I'll just get a home pod and just ask it questions, and it can tell me to go look on the internet.
00:00 OKKEN: Yeah, but I wouldn't be home to ask it though, so you need to have two, I guess. Like, ‘Home pod, ask my home pod at home…’
00:00 KENNEDY: (Laughs) ‘Mobile home pod, ask home home pod..’
00:00 Alright, man. How’s the book? Are you still working on it? Is it beta or is it published?
00:00 OKKEN: It's been up. We've had some a lot of good sales so far and a lot of feedback. I'm really glad that people are reading it and nobody's finding things too bad with it, so far. A couple typos here and there. The chapter on configuration is coming out next week. The beta’s out; it was out right before PyCon and we're doing a chapter every couple of weeks but there's only a couple chapters left and it'll be done.
00:00 KENNEDY: That’s cool. It must feel great to be wrapping it up.
00:00 OKKEN: Yeah, definitely. Having some early sales is good. It's a good motivator to finish it.
00:00 KENNEDY: Very good. I'm just on a podcast recording and course recording bender. I've courted almost two months-worth of Talk Python episodes in the last week and a half and I'm getting the next month the rest of this week. So, things are going to be all laid out for the summer. And there's a ton of exciting things coming, including some stuff on server-less AWS Lambda.
00:00 OKKEN: Oh, really? Neat. You're going to probably try to get me to record like, 18 episodes ahead of time on this or something. I don't know how that's going to work, being that it’s news.
00:00 KENNEDY: Well, we're going to use our AI and our machine learning to predict the future and they will totally knock it out. It’s going to be awesome.
00:00 OKKEN: Can we get AIs to mimic our voices and then we don't even have to do the podcast anymore?
00:00 KENNEDY: Yeah, it can just like give us the Cliff Notes later. It will be great.
00:00 OKKEN: Probably not.
00:00 KENNEDY: Until then, everybody, thank you for listening. Brian, thanks for sharing the news.
00:00 OKKEN: Thank you.
00:00 KENNEDY: Thank you for listening to Python Bytes. Follow the show on Twitter via @pythonbytes and get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbyes.fm and send it our way. We’re always on the lookout for sharing something cool. On behalf of myself and Brian Okken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.