Transcript #29: Responsive Bar Charts with Bokeh, Flask, and Python 3

00:00 Hello and welcome to Python Bytes, Python news and headlines delivered directly to your earbuds.

00:06 This time it's episode 29 and we recorded on June 6th, 2017. I'm Michael Kennedy.

00:12 And I'm Brian Okken.

00:13 And we're here to share some really cool stuff that we found with you. But before we do,

00:16 Brian, let's say thanks.

00:17 Yes, let's.

00:18 All right. Thank you to Rollbar. So Rollbar sponsoring this show as they have many other

00:22 ones. Rollbar is a great friend and supporter of the show. So be sure to check out what they're

00:27 offering at rollbar.com slash Python Bytes. We'll talk about that more later. Right now,

00:31 I want to talk about charts and stuff that are responsive.

00:33 There's an article on the Full Stack Python website by Matt McKay called Responsive Bar Charts with

00:40 Boca, Flask and Python 3. And I thought that was a great excuse to play with it because I've,

00:47 I've used, what's that other one? The other chart thing, Matt Plotlib. I've used Matt Plotlib a lot.

00:53 I haven't used Boca and I've wanted to. And this was a really great walkthrough. So I went ahead and

01:00 went through it and it walks, I did the little tutorial and it walks you through writing a Flask

01:06 app from the start and explains kind of what you're doing. Goes through a little bit of the Boca,

01:12 just there's, you're just creating a bar chart and it fills it with random data. But it also shows you

01:19 how to do like tool tips and how to fill those in with a, with a function callback. And now I've got

01:25 a little Boca Flask app running on my laptop. Yeah. That's, that's really cool. Yeah. Matt is really

01:32 good at writing these articles that are focused, you know, they're, they're not too short, so they're

01:37 all fluffy, but they're also not like, they don't drag on, right? You can get, get down to it pretty

01:40 quickly. Yeah. And there's a lot of detail missing, I'm sure to really get into it, but having

01:45 like a top to bottom full example running very quickly, and then you can dive into playing with

01:50 different bits and pieces of it. That was very helpful. I liked it. Yeah. It's pretty cool. So

01:55 the way Boca works is it kind of integrates the Python backend with the, the visualization JavaScript

02:03 front end, unless you really wire data all the way from like your Python app down to JavaScript,

02:08 you don't have to like become a JavaScript developer and in an extreme way, which I think is cool.

02:13 So it's using, I guess it's sort of like a template thing to fill out the data that ends

02:19 up going in that JavaScript is drawing. So the drawing happens on the JavaScript side. Is that

02:23 right? Yeah. Which makes it nice and interactive, right? As opposed to say matplotlib, which

02:28 you can generate like a PNG, but good luck for hovering over it and interacting with it and stuff,

02:33 right? That's what's cool about Boca. Yeah. We've got an application at work where I definitely

02:37 want to, it's actually similar to this, the, you know, the bugs found over so many days, not

02:42 exactly this, but similar enough to where we need to have a little small application at work to

02:49 generate some data around our, you know, around the test data that I'd like to be able to have an

02:54 interactive chart with. So yeah, you should definitely check this out. Yeah. That'd be really cool. And

02:59 flask is super simple and it sounds all, all nice and easy. Yeah. Speaking of nice and easy,

03:03 you know, one of the things people ask for a lot, like it's easy to get started with say pyramid or flask or

03:09 something like that. And you get it running and playing around, then you're like, all right,

03:13 I want to put this on a production server and I want to set up load balancing and scaling and

03:18 redundancy and like SSL. I'm like, Oh my God, this is a skill. I didn't realize I need it, but don't have,

03:23 right? It's like huge amounts of work. So there's a project that I want to highlight and I don't think

03:29 we've covered it on the show. Let me know. It's called Zappa and the idea is to take AWS Lambda.

03:35 Do you know AWS Lambda? Yes. Yeah. So the idea is AWS Lambda is basically here's a function,

03:41 a Python function, run it please. And then you can set that function to be run on AWS events like,

03:48 Hey, this file in S3 changed, but there's also something called the API gateway, which lets you map

03:55 your domain, SSL certificates, all that URL into a particular function, very much like flask routes

04:02 or things like this. So you can kind of set up web endpoints in AWS, right? In Lambda. So what this is,

04:11 is these guys built this thing called Zappa and the subtitle is serverless Python web services,

04:17 which I love the, the clash within the short name, their serverless services. Anyway,

04:25 what it is basically, it is a Python whiskey application, which could be flask, could be pyramid,

04:33 could be Django, whatever, could be its own thing. And you write this normal application that looks like

04:40 a normal web app, but then you can deploy it to AWS Lambda. So every request comes in, runs in its own

04:48 little container on Lambda. That's pretty wild, right? Yeah. So what's the deal? Like, why is this a good

04:54 thing? There's drawbox as well, which I'll maybe touch on, but Lambda, the way you pay for it is you pay

05:01 in terms of CPU used, right? As opposed to if you get some virtual private server like, Ubuntu or

05:09 something like this that you have to pay for because it's turned on. Whereas here you only pay for per

05:15 request. So if you have something that only takes, you know, say 10,000 requests a day, like it could be a

05:22 really small amount of time you're actually paying for. Cause how long does it take to serve those

05:26 requests? Assuming, you know, it's pretty quick app. I have no idea. You're the web guy.

05:30 Exactly. But it's, it's pretty cool. So it, you know, you can handle a ton of traffic for, you know,

05:36 just paying for a few milliseconds of server time. It's pretty cool. And you can even do async stuff,

05:41 which is pretty cool. So you can say, they've got an example in there where you can call an API and

05:46 the API can actually kick off another AWS Lambda function, but not in its own execution, but in

05:53 another one. And if you need like a hundred little Docker container type things to run this, like that's all

05:58 transparent at the Lambda side. You don't even care. Just say here, go run this and so on. So

06:03 it's a really interesting way to look at building web applications, mostly to deploying

06:09 web applications. Yeah. The example they're deploying is a flask app, by the way.

06:13 So every end point, like in my path is going to end up calling a different function and be a

06:19 different Lambda server.

06:21 Yes. And actually probably be in its own container that only exists for 40 milliseconds and then goes

06:26 away. So like there's sort of a new instance of the thing created for every single request

06:31 different, right? Very, very different.

06:33 It'd be kind of neat to see an application built like traditionally and then like with this and to

06:38 compare them and see if you can even tell the difference.

06:41 Yeah. And sure would. So they actually have not a ton of like really popular ones, but they do have

06:46 at the bottom, like a couple of apps and some of them are like the small business registration for

06:51 Virginia website or something like this, like some sort of governmental app. And the thing that I've

06:57 noticed about these is there's just a little bit more latency than makes me happy because every

07:02 request kind of like starts up the whole web server process. And there's a little bit of latency and

07:09 API gateway and AWS Lambda because it starts up a new container, I think.

07:13 And so like you're sort of guaranteed a baseline of, you know, a couple hundred milliseconds response

07:19 time. If I, if I understand this right, maybe it's not that low. I'm not sure, but yeah, not that high,

07:25 but there's some extra latency that you pay for the way Lambda works here, but it's still quite

07:32 interesting.

07:32 Yeah. Actually I was looking down at some of the bottom and one of the examples was like a

07:37 MailChimp signup utility, a little microservice. And that actually, that totally makes sense to have,

07:43 like you have a static website, but there's something that's a little bit dynamic and has to run

07:48 aside, something like that would, would make sense.

07:51 Yeah. If you have mostly a static site and you just want a little interactivity, you can take that

07:55 little interactivity and make it an API or make it a separate thing that runs. Yeah. It's really cool.

08:00 Yeah. Okay. Cool.

08:01 All right. Speaking of serverless server stuff, what if PyPI wasn't accessible?

08:05 Yeah, exactly. One of the things we covered in episode 24 was the notion that of creating your own

08:13 local package store and kind of doing that with the built-in pip. And I was contacted by,

08:22 and I'm going to forget his name right now. I'm sorry. Somebody that wrote a little blog post,

08:27 and actually this is an older one about the same topic called using a local cache for pip packages.

08:33 And the reason why I'm highlighting it today is that it, I like that it's a couple aliases that

08:39 didn't occur to me before. So it's, it's pretty much the same thing using pip install dash dash

08:44 download to download packages into a specific place and then how to install locally, but aliasing them to

08:53 a couple names, one called pip cache and one called pip install. Of course you can call them whatever you want.

08:58 Actually, that's a pretty clever. And I probably will just take his names because I'll remember it.

09:04 And anyway.

09:05 Exactly. He's already written the documentation. Yeah. It is a cool idea. Like you can basically say pip install --download some location.

09:13 And then when you want to actually install from it, you say pip install --no index --find.

09:19 And you like give it this link and it's a complete pain to remember that stuff. But if you just type pip cache as if you had typed pip install and pip install as if you had typed pip space install.

09:29 And instead it goes out of this local directory. That's pretty awesome.

09:33 Yeah. And like you said, or we were talking about before the show, it need to just take if you had a whole bunch that you knew of, like you're teaching a class or you're just you're going to get on a plane or something.

09:44 You wanted to grab a bunch of stuff to work with just having something like that. And he does highlight also this works fine for a requirements.txt file.

09:53 So you could just have a big requirements.txt file and download them all.

09:57 Yeah, that's cool. pip cache dash our requirements.txt. Boom. You've got them all.

10:01 Yeah.

10:01 I think that's really cool. And like my idea was it would be great if you could say somehow tie this together with a data source that says here's the top 500 packages you might ask for.

10:10 So you kind of preload your cache with a bunch of local stuff. So very likely if you want requests, if you want beautiful soup, whatever, you just got it.

10:19 All right. I think that would be pretty, pretty sweet.

10:21 Yeah, definitely.

10:22 Yeah. Yeah. Very cool. I love this one. Thanks. It's nice and simple, but you know, you could start using it like today without much effort.

10:29 Yes.

10:29 All right. You know what else is not much effort? Tracking down errors with Rollbar.

10:33 So Rollbar has been helping me out with my website a lot. And actually it's, you know, I've run into it like an issue or two here.

10:41 The idea is you basically add just a couple of lines. And if you're doing pyramid, you put it in your config file. Similarly with Flask or Django, you can add it in there. It even works with other stuff like Node.js, but I don't do Node.js because I don't like it.

10:55 Anyway, for my Python web apps, you can just put it in there super easy and it will actually capture all the errors and send them back with details. Like what was the call stack? What were the local variables? And they added this other cool thing that I'm wanting to play with. I haven't got it working yet is people tracking.

11:11 So like right now, everyone who listens to the podcast knows if they find a way to crash my site, I know about it, but I have no idea who did that. I have no idea how to like tell them, hey, sorry, the thing that broke you here, you could try it. I fixed it.

11:25 So periodically I'll get emails from people and I say, hey, sorry, this isn't working. Like, yeah, I know. I just didn't know how to tell you that I know I'm working on it.

11:32 So there's a new thing called people tracking that you can actually add. Like if you have a logged in user, like who caused that error and it'll check that for you as well, which is pretty cool.

11:40 So check them out at rollbar.com slash Python bytes.

11:43 The logged in user, that would be very effective for like your courses then, then since somebody's work.

11:48 Exactly. Cause they already have to have an account. They're already logged in. So if something crashed for like a registered user, then I can actually contact them back and, you know, say, sorry, here's what happened. Here's how I fixed it. Things like that.

12:00 Especially for something, a service like that, where people are paying for it, being able to tell them, I saw that you had trouble and I fixed it. That's cool. That's great. Anyway, thanks rollbar.

12:09 Yeah. I think that takes the sting out of it. Yeah. Thanks rollbar. Very much appreciate the support for the show. So PyCon was fun, right?

12:15 Oh, it was really fun. Yeah.

12:16 Yeah. And one of the things that was cool that I mean, all these, there were all these booths and people had set up so many things. The talks were fun. The people I met was fun.

12:24 I felt a little bit bad that I didn't actually get to escape from my booth very far, our booth, because people were excited to come talk to us. And that was great. But one of the things I did get to go see was the active Python folks.

12:39 And they talked about you a little bit as well. So I think that you went over there and saw them. You must have. And what they had built is like a simple little game in Python using PyGame.

12:48 And it was like a scroller. Like imagine like 1980s, 1990s, like Space Invader type of standard 2D scroller game. But the thing that was cool was they actually used TensorFlow, Carreras, Intel, the math kernel library, and active Python to actually derive the AI of the game.

13:11 Wow.

13:11 Isn't that cool? So you could go over there and they had two screens. It was pretty sweet. And on one was the game, the PyGame game, scroller thing. You'd sit there and play.

13:21 But on the other was actually the neural network doing like visually doing its thinking while it was fighting you.

13:27 That's pretty cool.

13:28 I thought that was, that was pretty trick. And basically the AI would figure out when to shoot you, whether or not to shoot you, where to aim, things like that. And that was all a neural network.

13:37 So the guys over there, Peter, sorry, Peter, I don't remember your last name. Peter wrote this up with some of the lessons they learned, how they basically use PyGame.

13:46 It's going to be open source, they tell me, but it's not yet available. They're still getting the code ready to be put up on GitHub.

13:53 So maybe if we're lucky, we can link to the GitHub repo. But if it's really interesting to you, just send the guy a message on Twitter and I'm sure he'll hook you up.

14:01 Well, if you have a link to that blog post, I'm sure they'll link it up there, hopefully.

14:06 Yeah, I'm sure they'll update it. So they had a few lessons learned of writing a game that is driven by TensorFlow.

14:15 So they said, choosing the right data to train your network is super important. Prepping your data is key. And just experiment with the network topology. And actually, the last thing I talked about the looking at the neural network, think, do its magic.

14:31 So visualization is really important for understanding what it is you built. On a side note, yeah, so this is a great thing. Check it out. Like they've got a pretty detailed write up on it.

14:40 But on a side note, Brian, do you think we're going to get to a place with all this machine learning and AI and stuff where the world is powered by programs that nobody understands how they work?

14:51 Do you think we're not already there?

14:53 I think we're... That's a great response. I think we're there from a complexity and just like there's too much crap and too many layers.

15:02 But theoretically, applying effort, you'll be able to go through and stack trace it, right? But at some point, when it's just like a bunch of neural networks working in orchestration, like, why did it decide that? I don't know. It decided that. You know, it's crazy.

15:19 Yeah, I think possibly. But at that point, I mean, there's also, I just don't know enough about AI and neural networks. And so it looks like magic to me.

15:27 Yeah.

15:27 Somebody that knows a lot about it possibly doesn't look so much like magic. And it's more obvious how answers could be derived. But I don't know.

15:36 Yeah, the craziest example I think of is where Google got two of their AIs to invent an encryption language and a third one to try to break the encryption language.

15:48 And the AIs actually invented their own encryption that nobody had ever seen. And then the other AI would break it, then their encryption would get harder. And it just went on and on like that. And nobody knows what happened.

15:57 Yeah, that's pretty cool.

15:58 Pretty wild.

15:59 Yeah, and I think some of those things are definitely possible. And those are, like you said, complexity, building on complexity. But I think this, I just want to like, to me, I don't think there's a fear of a singularity thing. I'm kind of a skeptic on the computer's going to take over the world scenario.

16:17 I'm with you. I'm certainly with you. I'm not a skeptic on these types of things disrupting careers and jobs and society.

16:25 Yeah, definitely.

16:25 I don't think like, we're going to see Skynet right away.

16:28 Yeah, hopefully not. Or we won't see it coming, right?

16:32 Exactly. What's that noise? Oh, H.

16:35 Yeah, so speaking of things that you want to try to understand with a debugger.

16:40 Yeah, this is an older article. It's from Rafael Pirzina. It's debug test failures with PDB. And Rafael's works. He does a lot of stuff with pytest. And that's how I know him. He also works with Cookie Cutter.

16:56 Yeah, he's a busy guy.

16:56 I'm linking this up because it's pretty clever. I had legitimate error in the demo code I use on the book. And I left it in there because I wanted to be able to use, I wanted to show a legitimate error in some code.

17:11 I also show the fixed version, of course. I used the debugger to understand what was going on with my code. And one of the cool things about pytest is PDB is not like the best. I mean, I grew up on like things like Visual Studio and PDB is definitely not Visual Studio like.

17:28 It's not exactly the same, no.

17:30 No, but you can do quite a bit of powerful things with it. Like show different variables and show your listing and exactly where it is. But often you have to like stick a breakpoint in your code in order to use it. But pytest has a --PDB option, which just sticks you right in the debugger right at the assert.

17:52 Oh, that's cool.

17:53 So right where your test failed, you'll be right there and you can examine all the variables and whatever. Even set up an interactive terminal with all of the local variables there so you can play with them. And that's listed in this page. But there's also some of the other features that are great for debugging with pytest, like being able to run just the last fails, last failures and things like that are listed in this article. So I wanted to highlight that from Raphael.

18:20 Yeah, nice work Raphael. So if you were sitting at home, Brian, let me put this in a way. If you were at work and you wanted to know if your phone, your voice over IP, Cisco phone was not working at your house, how would you do that? Suppose you have one.

18:37 Amazon pod or whatever the latest thing is.

18:40 Your home pod. Your home pod from Apple.

18:44 Yeah.

18:45 You would say something like, Hey, home pod is my phone working? And it would say, here's what I found on the internet for you. Oh my gosh. Hopefully that thing works someday. Anyway, there's a really cool article called monitoring my VoIP provider with home assistant.

19:01 So I wanted to bring this up, not because the programming is super intense or anything, but it's just like, here's a little fun thing that you can do without a lot of work. So this guy has a Cisco ATA sort of home voice over IP phone with two lines.

19:16 And his home is set up to automate a bunch of things using something called home assistant, home-assistant.io. Have you seen this thing? I think we talked about it before.

19:26 Yeah.

19:26 Yeah. So this is like a full on Python central hub for your home, which is really cool. And so what he did was he plugged in some new code, wrote some new modules for home assistant that would look at his Cisco phone and then watch it and let him know if it ever went down for whatever reason.

19:47 And he's got a new open source package, PI Cisco PA, I guess. I'm not sure what the PA stands for. And you can use that now if you want to, but it's also does cool, like push notifications to your mobile phone, right? Which is pretty cool.

20:02 So it's not so much that I think people necessarily need to monitor their home phone because I don't know about you, but I don't even have a home phone.

20:09 I have a lot of phones in my home, but they're not home phones. They all go when people take them. But this sort of integration story of here's a thing. It wasn't integrated with home assistant. Here's a little bit of Python code and magic happens. And that's really what I wanted to cover here.

20:22 That's pretty nice. And actually I'm about to get a home phone, so I might be able to use this.

20:29 You will totally know if it goes offline.

20:31 Yeah. So it looks like the SPA is based on because it's an SPA ATA device. So whatever that means.

20:38 Yeah. Cisco SPA. Yeah. It's a, whatever the hardware is. Okay, cool. So yeah, that's, that's a fun little thing. It shows you how to basically extend home assistant, which if I had a smart home, I would be doing all sorts of stuff.

20:50 Or I'll just get a HomePod and just ask it questions. And it can tell me to go look on the internet.

20:54 Yeah, but I wouldn't be home to ask it though. So you need to have two, I guess. Like HomePod, ask my HomePod at home.

21:00 Mobile HomePod, ask home HomePod.

21:07 All right, man. How's the book? Are you still working on it?

21:11 Yeah.

21:11 Is it beta or is it, is it like published?

21:13 It's been up. We've had some good, a lot of good, a lot of sales so far. And I'm really, and a lot of feedback. I'm, I'm really glad that people are reading it and nobody's finding things too bad with it so far. A couple of typos here and there.

21:25 The chapter on configuration is coming out next week. And so we're doing the betas out. It was out right before PyCon and we're doing a chapter every couple of weeks, but there's only a couple of chapters left and it'll be done.

21:40 So that's cool. That must feel great to be wrapping that up.

21:43 Yeah, definitely. And having some early sales is good. It's good motivator to finish it.

21:47 Very, very good. Well, I'm just on a podcast recording and course recording bender. Like I've recorded almost two months worth of Talk Python episodes in the last week and a half. And I'm getting the next month, the rest of this week.

22:00 So things are going to be all laid out for the summer and there's a ton of exciting things coming, including some stuff on serverless AWS Lambda.

22:08 Oh, really? Neat.

22:09 Yeah. That'd be cool.

22:09 You're going to probably try to get me to record like 18 episodes ahead of time on this or something. And I don't know how that's going to work being that it's news.

22:18 Well, we're going to use our AI and our machine learning to predict the future and then we'll totally knock it out. It's going to be awesome.

22:25 Hey, can we get like AIs to mimic our voices and then we don't even have to do the podcast anymore?

22:29 Yeah. It can just like give us the cleftos later. That'd be great.

22:32 Yeah. No, probably not.

22:33 All right. Until then, everybody, thank you for listening. Brian, thanks for sharing all the news, man.

22:38 Thank you.

22:38 Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S.

22:46 And get the full show notes at Python Bytes.FM. If you have a news item you want featured, just visit Python Bytes.FM and send it our way.

22:54 We're always on the lookout for sharing something cool.

22:57 On behalf of myself and Brian Okken, this is Michael Kennedy.

23:00 Thank you for listening and sharing this podcast with your friends and colleagues.