Transcript #29: Responsive Bar Charts with Bokeh, Flask, and Python 3

00:00 Hello and welcome to Python Bytes, Python news and headlines delivered directly to your earbuds.

00:06 This time it's episode 29 and we recorded on June 6 2017. I'm Michael Kennedy and I'm Brian Okken and we're here to share some really cool stuff that we found with you. But before we do, Brian, let's say thanks. Yes, let's. All right. Thank you to Rollbar. So Rollbar sponsoring this show as they have many other ones. Rollbar is a great friend and supporter of the show. So be sure to to check out what they're offering at rollbar.com/pythonbytes.

00:29 We'll talk about that more later.

00:31 Right now I want to talk about charts and stuff that are responsive.

00:33 There's an article on the full stack Python website by Matt McKay called responsive bar charts with Boca, Flask, and Python 3.

00:43 And I thought that was a great excuse to play with it because I've used, what's that other one, the other chart thing, Matplotlib.

00:52 I've used Matplotlib a lot.

00:53 I haven't used Boca and I've wanted to, and this was a really great walkthrough.

00:58 So I went ahead and went through it, and it walks, I did the little tutorial, and it walks you through writing a Flask app from the start and explains kind of what you're doing.

01:10 Goes through a little bit of the Boca, just you're just creating a bar chart, and it fills it with random data, but it also shows you how to do like tool tips and how to fill those in with a function callback.

01:24 And now I've got a little Boca flask app running on my laptop.

01:28 - Yeah, that's really cool.

01:30 Yeah, Matt is really good at writing these articles that are focused, you know, they're not too short, so they're all fluffy, but they're also not like, they don't drag on, right?

01:39 You can get down to it pretty quickly.

01:41 - Yeah, and there's a lot of detail missing, I'm sure, to really get into it, but having like a top to bottom full example running very quickly, and then you can dive into playing with different bits and pieces of it.

01:52 It was very helpful.

01:53 I liked it.

01:54 - Yeah, it's pretty cool.

01:55 So the way Bokeh works is it kind of integrates the Python backend with the visualization JavaScript front end and lets you really wire data all the way from like your Python app down to JavaScript.

02:08 You don't have to like become a JavaScript developer and in an extreme way, which I think is cool.

02:13 - So it's using, I guess it's sort of like a template thing to fill out the data that ends up going in that JavaScript is drawing.

02:21 So the drawing happens on the JavaScript side, is that right?

02:24 - Yeah, which makes it nice and interactive, right?

02:26 As opposed to say, matplotlib, which you can generate like a PNG, but good luck for hovering over it and interacting with it and stuff, right?

02:33 That's what's cool about Bokeh.

02:34 - Yeah.

02:35 We've got an application at work where we definitely wanna, it's actually similar to this, the bugs found over so many days.

02:42 Not exactly this, but similar enough to where we need to have a little small application at work to generate some data around the test data that I'd like to be able to have an interactive chart with.

02:56 - You should definitely check this out.

02:58 Yeah, that'd be really cool.

02:58 And Flask is super simple and it sounds all nice and easy.

03:03 Speaking of nice and easy, one of the things people ask for a lot, it's easy to get started with say Pyramid or Flask or something like that.

03:11 And you get it running and playing around, then you're like, all right, this on a production server and I want to set up load balancing and scaling and redundancy and like SSL and like, oh my god, this is a skill I didn't realize I needed but don't have right, it's like huge amounts of work.

03:25 So there's a project that I want to highlight and I don't think we've covered it on the show let me know.

03:31 It's called Zappa and the idea is to take AWS Lambda.

03:35 Do you know AWS Lambda?

03:36 Yes.

03:37 Yeah.

03:38 So the idea is AWS Lambda is basically here's a function, a Python function, run it please.

03:43 And then you can set that function to be run on AWS events like hey, this file on s3 changed.

03:50 But there's also something called the API gateway, which lets you map your domain SSL certificates, all that URL into a particular function, very much like flask routes or things like this.

04:03 So you can kind of set up web endpoints in AWS, right in Lambda.

04:09 So what this is, is these guys built this thing called Zappa, and the subtitle is serverless Python web services, which I love the clash within the short name, they're serverless services.

04:24 Anyway, what it is, is basically it is a Python WSGI application, which could be Flask, could be Pyramid, could be Django, whatever, could be its own thing.

04:36 And you write this normal application that looks like a normal web app, but then you can deploy it to AWS Lambda.

04:45 So every request comes in runs in its own little container on Lambda.

04:51 That's pretty wild, right?

04:52 Yeah.

04:53 So what's the deal?

04:54 Like, why is this a good thing?

04:55 There's drawbacks as well, which I'll maybe touch on.

04:57 But Lambda, the way you pay for it is you pay in terms of CPU used, right?

05:04 As opposed to if you get some virtual private server like Ubuntu or something like this, that you have to pay for because it's turned on.

05:13 Whereas here you only pay for per request.

05:15 So if you have something that only takes, you know, say 10,000 requests a day, like it could be a really small amount of time you're actually paying for.

05:24 Because how long does it take to serve those requests, assuming you know, it's pretty quick app.

05:29 I have no idea.

05:30 You're the web guy.

05:31 Exactly.

05:32 But it's pretty cool.

05:33 You know, you can handle a ton of traffic for, you know, just paying for a few milliseconds of server time.

05:39 It's pretty cool.

05:40 And you can even do async stuff, which is pretty cool.

05:42 So you can say, they've got an example in there where you can call an API and the API can actually kick off another AWS lambda function, but not in its own execution, but in another one.

05:54 And if you need like 100 little Docker container type things to run this, like, that's all transparent at the lambda side, you don't even care.

06:01 Just say here, go run this.

06:03 so on. So it's a really interesting way to look at building web applications, mostly to deploying web applications. Yeah, the example they're deploying is a Flask app, by the way. So every endpoint, like in my path, is going to end up calling a different function and be a different Lambda server? Yes, and actually probably be in its own container that only exists for 40 milliseconds and then goes away. So like there's sort of a new instance of the thing created for every single request.

06:31 Different, right?

06:32 Very, very different.

06:33 It'd be kind of neat to see an application built like traditionally and then like with this and to compare them and see if you can even tell the difference.

06:41 Yeah, it sure would.

06:42 So they actually have not a ton of like really popular ones, but they do have at the bottom like a couple of apps.

06:48 And some of them are like the small business registration for Virginia website or something like this, like some sort of governmental app.

06:56 And the thing that I've noticed about these is there's just a little bit more latency than makes me happy.

07:02 Because every request kind of like starts up the whole web server process.

07:08 And there's a little bit of latency and API gateway and AWS Lambda because it starts up a new container, I think.

07:14 And so like, you're sort of guaranteed a baseline of, you know, a couple hundred milliseconds response time.

07:20 If I understand this right, maybe it's not that low.

07:23 I'm not sure.

07:24 But yeah, not that high.

07:26 But there's some extra latency that you pay for the way Lambda works here, but it's still quite interesting.

07:33 Yeah, actually, I was looking down at some of the bottom and one of the examples was like a MailChimp signup utility, a little microservice and that actually that totally makes sense to have, like you have a static website, but there's something that's a little bit dynamic and has to run.

07:48 Yes.

07:49 Aside, something like that would would make sense.

07:51 Yeah, if you have mostly a static site, you just want a little interactivity, you can take that little interactivity and make it an API or make it a separate thing that runs. Yeah, it's really cool.

08:00 Yeah, okay, cool.

08:01 All right, speaking of serverless server stuff, what if PyPI wasn't accessible?

08:05 Yeah, exactly. One of the things we covered in episode 24 was the notion that of creating your own local package store and kind of doing that with the built-in pip. And I was contacted by, and I'm going to forget his name right now, sorry, somebody that wrote a little blog post, and actually this is an older one about the same topic, called "Using a Local Cache for pip Packages" and the reason why I'm highlighting it today is that it, I like that it's a couple aliases that didn't occur to me before. So it's it's pretty much the same thing using pip install --download to download packages into a specific place, and then how to install locally, but aliasing them to a couple names, one called pipcache and one called pip install. Of course you can call them whatever you want. Actually that's a pretty clever and I probably will just take his names because I'll remember it. And anyway... Exactly, he's already written the documentation. Yeah. It is a cool idea like you can basically say pip install --download some location and then when you want to actually install from you can say pip install --noindex --find, and you give it this link, and it's a complete pain to remember that stuff.

09:22 But if you just type pip cache, as if you had typed pip install, and pip install as if you had typed pip install, and instead it goes out of this local directory, that's pretty awesome.

09:33 Yeah, and like you said, or we were talking about before the show, it'd need to just take, if you had a whole bunch that you knew of, like you're teaching a class, or you're going to get on a plane or something, You wanted to grab a bunch of stuff to work with just having something like that.

09:49 And he does highlight also this works fine for a requirements.txt file.

09:53 So you could just have a big requirements.txt file and write them all.

09:57 Yeah, that's cool.

09:58 pip cache -r requirements.txt.

09:59 Boom, you've got them all.

10:01 Yeah, I think that's really cool.

10:02 And like my idea was, it would be great if you could say there's somehow tie this together with a data source that says here's the top 500 packages you might ask for.

10:11 So you kind of preload your cache with a bunch of local stuff.

10:15 So very likely if you want requests, if you want beautiful soup, whatever, you just got it right.

10:20 I think that would be pretty, pretty sweet.

10:21 Yeah, definitely.

10:22 Yeah, very cool.

10:23 I love this one.

10:24 Thanks.

10:25 It's nice and simple, but you know, you could start using it like today without much effort.

10:29 Yes.

10:30 Right.

10:31 You know what else is not much effort?

10:32 Tracking down errors with Rollbar.

10:34 So Rollbar has been helping me out with my website a lot.

10:37 And actually, it's you know, I've run into like an issue or two here.

10:41 The idea is you basically add just a couple of lines.

10:45 And if you're doing pyramid, you put it in your config file.

10:48 Similarly, with flask or Django, you can add in there and even works with other stuff like node j s.

10:53 But I don't do node j s because I don't like it.

10:55 Anyway, for my Python web apps, you can just put in there super easy, and it will actually capture all the errors and send them back with details like what was the call stack?

11:05 What were the local variables?

11:06 And they add to this other cool thing that I'm wanting to play with, I haven't got it working yet is people tracking.

11:12 So like right now, everyone who listens to the podcast knows if they find a way to crash my site, I know about it.

11:18 But I have no idea who did that.

11:20 I have no idea how to like tell them, "Hey, sorry, the thing that broke, you here, you could try it, I fixed it." So periodically, I'll get emails from people and they say, "Hey, sorry, this didn't work." Like, "Yeah, I know.

11:30 I just didn't know how to tell you that I know, I'm working on it." So there's a new thing called people tracking that you can actually add, like if you have a logged in user, like who caused that error and it'll check that for you as well.

11:39 It's pretty cool.

11:40 So check them out at rollbar.com/pythonbytes.

11:41 The logged in user, that would be very effective for like your courses then, since somebody's were...

11:48 Exactly, because they already have to have an account, they're already logged in.

11:51 So if something crashed for like a registered user, then I can actually contact them back and say, "Sorry, here's what happened.

11:58 Here's how I fixed it." Things like that.

12:00 Especially for something, a service like that, where people are paying for it, being able to tell them I saw that you had trouble and I fixed it.

12:06 That's cool.

12:07 That's great.

12:08 Anyway, thanks, Robar.

12:09 Yeah, I think that takes the sting out of it.

12:10 Yeah, thanks, Robar.

12:12 Very much appreciate the support for the show.

12:14 So PyCon was fun, right?

12:15 Oh, it was really fun.

12:16 Yeah.

12:17 Yeah.

12:18 And one of the things that was cool that I mean, all these, there are all these booths and people had set up so many things.

12:22 The talks were fun.

12:23 The people I met was fun.

12:24 We're fun.

12:25 I felt a little bit bad that I didn't actually get to escape from my booth very far, our booth, because people were excited to come talk to us.

12:33 And that was great, but one of the things I did get to go see was the active Python folks.

12:39 And they talked about you a little bit as well.

12:40 So I think that you went over there and saw them, you must have.

12:43 And what they had built is like a simple little game in Python using Pygame.

12:48 And it was like a scroller, like imagine like 1980s, 1990s, like Space Invader type of standard 2D scroller game.

12:59 But the thing that was cool was they actually used TensorFlow, Kerares, Intel, the math kernel library, and ActivePython to actually derive the AI of the game.

13:11 Wow.

13:12 Isn't that cool?

13:13 So you could go over there and they had two screens.

13:14 It was pretty sweet.

13:15 And on one was the game, the Pygame game, scroller thing.

13:20 It's out there in play.

13:22 But on the other was actually the neural network doing like visually doing its thinking while it was fighting you.

13:28 pretty cool.

13:29 I thought that was that was pretty trick and basically the AI would figure out when to shoot you whether or not to shoot you where to aim things like that and that was all a neural network.

13:38 So the guys over there, Peter, sorry, Peter, I don't remember your last name.

13:41 Peter wrote this up with some of the lessons they learned how they basically use Pygame.

13:47 It's going to be open source, they tell me but it's not yet available.

13:51 They're still getting the code ready to be put up on GitHub.

13:53 So maybe if we're lucky, we can link to the GitHub repo.

13:56 But if it's really interesting to you, just send the guy a message on Twitter and I'm sure he'll hook you up.

14:01 >> Well, if you have a link to their blog post, I'm sure they'll link it up there, hopefully.

14:06 >> Yeah, I'm sure they'll update it.

14:08 So they had a few lessons learned writing a game that is driven by TensorFlow.

14:14 So they said, choosing the right data to train your network is super important.

14:20 Prepping your data is key and just experiment with the network topology.

14:25 And actually the last thing I talked about, the looking at the neural network think, do its magic.

14:31 So visualization is really important for like understanding what it is you built.

14:36 On a side note, yeah, so this is a great thing.

14:37 Check it out.

14:38 Like they've got a pretty detailed write up on it.

14:40 But on a side note, Brian, do you think we're going to get to a place with all this machine learning and AI and stuff where the world is powered by programs that nobody understands how they work.

14:51 Do you think we're not already there?

14:54 I think we're, that's a great response.

14:56 I think we're there from a complexity and just like there's too much crap and too many layers, but theoretically applying effort, you'll be able to go through and like stack trace it.

15:07 Right.

15:08 when it's just like a bunch of neural networks working in like orchestration.

15:13 Like why did it decide that?

15:15 I don't know.

15:16 It decided that, you know?

15:18 It's crazy.

15:19 - Yeah, I think possibly, but at that point, I mean, there's also, I just don't know enough about AI and neural networks.

15:25 And so it looks like magic to me.

15:28 Somebody that knows a lot about it possibly doesn't look so much like magic and it's more obvious how answers could be derived, but I don't know.

15:36 - Yeah, the craziest example I can think of is where Google got two of their AIs to invent an encryption language and a third one to try to break the encryption language.

15:48 And the AIs actually invented their own encryption that nobody had ever seen.

15:52 And then the other AI would break it, then their encryption would get harder and it just went on and on like that and nobody knows what happened.

15:58 - Yeah, that's pretty cool.

15:59 Yeah, and I think some of those things are definitely possible.

16:02 And those are, like you said, they're complexity building on complexity.

16:06 But I think this, I just wanna like, to me, I don't think there's a fear of a singularity thing.

16:12 I'm kind of a skeptic on the computer's gonna take over the world scenario.

16:17 - I'm with you, I'm certainly with you.

16:19 I'm not a skeptic on these types of things disrupting careers and jobs in society.

16:25 - Yeah, definitely.

16:26 - I don't think like, we're gonna see Skynet right away.

16:28 - Yeah, hopefully not.

16:30 Or we won't see it coming, right?

16:32 - Exactly.

16:33 What's that noise?

16:34 Oh, wait.

16:35 - Yeah, so speaking of things that you want to try to understand with a debugger.

16:40 - Yeah, this is an older article.

16:42 It's from Rafael Pierzina.

16:45 It's Debug Test Failures with PDB.

16:48 And Rafael's works, he does a lot of stuff with pytest and that's how I know him.

16:54 He also works with Cookie Cutter.

16:56 - Yeah, he's a busy guy.

16:56 - I'm linking this up because it's pretty clever.

16:59 I had legitimate error in the demo code I use on the book And I left it in there because I wanted to be able to use--

17:08 I wanted to show a legitimate error in some code.

17:11 I also show the fixed version, of course.

17:14 I used the debugger to understand what was going on with my code.

17:17 And one of the cool things about pytest is PDB is not like the best-- I mean, I grew up on things like Visual Studio, and PDB is definitely not Visual Studio-like.

17:29 But it's--

17:29 It's not exactly the same, no.

17:31 No.

17:31 But you can do quite a bit of powerful things with it.

17:34 and like show different variables and show your listing and exactly where it is.

17:39 But often you have to like stick a break point in your code in order to use it.

17:44 But pytest has a --pdb option, which just sticks you right in the debugger, right at the assert.

17:52 - Oh, that's cool.

17:53 - So right where your test failed, you'll be right there and you can examine all the variables and whatever, even set up an interactive terminal with all of the local variables there.

18:03 so you can play with them.

18:05 And that's listed in this page, but there's also some of the other features that are great for debugging with pytest, like being able to run just the last fails, last failures, and things like that are listed in this article.

18:18 So I wanted to highlight that from Rafael.

18:21 - Yeah, nice work, Rafael.

18:22 So if you were sitting at home, Brian, let me put this in a different way.

18:25 If you were at work and you wanted to know if your phone, your Voice over IP, Cisco phone, was not working at your house.

18:35 How would you do that?

18:36 Suppose you have one.

18:37 - Amazon Pod or whatever the latest thing is.

18:41 - Your HomePod, your HomePod from Apple.

18:45 You would say something like, hey HomePod, is my phone working?

18:48 And it would say, here's what I found on the internet for you.

18:51 Oh my gosh, hopefully that thing works someday.

18:53 Anyway, there's a really cool article called Monitoring My VoIP Provider with Home Assistant.

19:01 So I wanted to bring this up not because the programming is super intense or anything, but it's just like, here's a little fun thing that you can do without a lot of work.

19:08 So this guy has a Cisco ATA sort of home voice over IP phone with two lines.

19:17 And his home is set up with home automation in a automated bunch of things using something called home assistant, home dash assistant.io.

19:24 Have you seen this thing?

19:25 I think we talked about it before.

19:26 Yeah.

19:27 a full on Python central hub for your home, which is really cool.

19:34 And so what he did was he plugged in some new code wrote some new modules for home assistant that would look at his Cisco phone, and then watch it and let him know if it ever went down for whatever reason.

19:48 And he's got a new open source package, pi Cisco, PA, I guess I'm not sure the PA stands for and you can use that now if you want to.

19:57 But it also does cool like push notifications to your mobile phone, right, which is pretty cool.

20:03 So it's not so much that I think people necessarily need to monitor their home phone because I don't know about you, but I don't even have a home phone.

20:09 I have a lot of phones in my home, but they're not home phone.

20:11 They all go when people take them.

20:14 But this sort of integration story of here's a thing, it wasn't integrated with Home Assistant, here's a little bit of Python code and magic happens.

20:21 And that's really what I wanted to cover here.

20:22 That's pretty nice.

20:24 And actually I'm about to get a home phone so I might be able to use this.

20:29 You will totally know if it goes offline.

20:31 Yeah, so it looks like the SPA is based on because it's an SPA ATA device.

20:36 So whatever that means.

20:38 Yeah, Cisco SPA.

20:39 Yeah, it's a whatever the hardware is.

20:41 Okay, cool.

20:42 So yeah, that's a fun little thing.

20:45 It shows you how to basically extend Home Assistant, which if I had a smart home, I would be doing all sorts of stuff.

20:50 Or I'll just get a HomePod and just ask it questions and it can tell me to go look on the internet.

20:54 Yeah, but I wouldn't be home to ask it though.

20:56 So you need to have two, I guess.

20:58 Like HomePod, ask my HomePod at home.

21:03 Mobile HomePod, ask home HomePod.

21:05 All right, man, how's the book?

21:10 Are you still working on it?

21:11 Yeah.

21:12 Is it beta or is it like published?

21:13 It's been up.

21:14 We've had a lot of sales so far and a lot of feedback.

21:18 I'm really glad that people are reading it and nobody's finding things too bad with it so far.

21:24 A couple of typos here and there.

21:25 The chapter on configuration is coming out next week.

21:29 And so we're doing the beta is out.

21:31 It was out right before PyCon and we're doing a chapter every couple weeks, but there's only a couple chapters left and it'll be done.

21:40 So that's cool.

21:41 It must feel great to be wrapping that up.

21:42 Yeah, definitely.

21:43 And having some early sales is good.

21:46 It's good motivator to finish it.

21:47 - Very, very good.

21:48 Well, I'm just on a podcast recording and course recording bender.

21:53 Like I've recorded almost two months worth of Talk Python episodes in the last week and a half, and I'm getting the next month the rest of this week.

22:00 So things are gonna be all laid out for the summer and there's a ton of exciting things coming, including some stuff on serverless AWS Lambda.

22:08 - Oh really, neat.

22:09 - Yeah, yeah, it'll be cool.

22:10 - You're gonna probably try to get me to record like 18 episodes ahead of time on this or something, and I don't know how that's gonna work being that it's news.

22:18 - Well, we're gonna use our AI and our machine learning to predict the future, and then we'll totally knock it out.

22:24 It's gonna be awesome.

22:25 - Hey, can we get like AIs to mimic our voices and then we don't even have to do the podcast anymore?

22:29 - Yeah, it can just like give us the cleft nose later.

22:31 That'd be great.

22:32 - Yeah, no, probably not.

22:33 - All right, until then, everybody, thank you for listening.

22:36 Brian, thanks for sharing all the news, man.

22:38 - Thank you.

22:39 - Thank you for listening to Python Bytes.

22:41 Follow the show on Twitter via @pythonbytes.

22:43 That's Python Bytes as in B-Y-T-E-S.

22:47 and get the full show notes at pythonbytes.fm.

22:50 If you have a news item you want featured, just visit pythonbytes.fm and send it our way.

22:54 We're always on the lookout for sharing something cool.

22:57 On behalf of myself and Brian Okken, this is Michael Kennedy.

23:01 Thank you for listening and sharing this podcast with your friends and colleagues.