WEBVTT 00:00:00.001 --> 00:00:04.800 Hello and welcome to Python Bytes, where we deliver Python news directly to your earbuds. 00:00:04.800 --> 00:00:08.080 This is episode 90, recorded August 2nd, 2018. 00:00:08.080 --> 00:00:09.120 I'm Michael Kennedy. 00:00:09.120 --> 00:00:10.140 And I'm Brian Okken. 00:00:10.140 --> 00:00:10.520 Hey, Brian. 00:00:10.520 --> 00:00:11.340 Good to be with you again. 00:00:11.340 --> 00:00:12.760 It's good to talk to you again. 00:00:12.760 --> 00:00:13.000 Yeah. 00:00:13.000 --> 00:00:16.940 And it's also good to have DigitalOcean sponsoring this episode. 00:00:16.940 --> 00:00:18.500 So thank you to DigitalOcean. 00:00:18.500 --> 00:00:21.480 We're both customers and their sponsors of ours. 00:00:21.480 --> 00:00:24.440 So it's kind of this weird mix of everybody loving it. 00:00:24.440 --> 00:00:29.080 So pythonbytes.fm/DigitalOcean will get you $100 credit for your servers. 00:00:29.220 --> 00:00:30.100 If you're a new customer. 00:00:30.100 --> 00:00:31.080 So check that out. 00:00:31.080 --> 00:00:34.300 In the meantime, we should probably talk about some data analysis. 00:00:34.300 --> 00:00:35.060 We should. 00:00:35.060 --> 00:00:39.080 And who better to talk about data analysis than Jake VanderPlas? 00:00:39.080 --> 00:00:40.060 I honestly don't know. 00:00:40.060 --> 00:00:43.740 He does awesome work over there at the eScience Institute. 00:00:43.740 --> 00:00:45.660 And I love listening to his talks. 00:00:45.660 --> 00:00:49.020 So there's a set of videos that he has on. 00:00:49.020 --> 00:00:52.960 This is actually from last year, but I didn't count. 00:00:52.960 --> 00:00:56.860 There's 11 videos and it's called Reproducible Data Analysis in Jupyter. 00:00:57.600 --> 00:01:00.860 But they're, I mean, each of the videos is like five or six minutes. 00:01:00.860 --> 00:01:02.100 So they go pretty fast. 00:01:02.100 --> 00:01:03.780 This is a really cool thing. 00:01:03.780 --> 00:01:08.440 I think everybody should check these out anyway is because they're, he goes through a problem 00:01:08.440 --> 00:01:14.840 of, or just a data set that's, that the bikes cross, the bike crossings at a particular 00:01:14.840 --> 00:01:17.200 bridge in Seattle, I think. 00:01:17.540 --> 00:01:20.920 But it really doesn't matter the data, how the data gets in there. 00:01:20.920 --> 00:01:23.480 But he's using, doing all this stuff live. 00:01:23.480 --> 00:01:27.880 He's doing a Jupyter notebook and pulling data in. 00:01:27.880 --> 00:01:32.020 And sometimes the tables like the table that he ends up with doesn't look quite right. 00:01:32.020 --> 00:01:38.040 So he uses a different, different column to, as the index or the, for instance, I'm one 00:01:38.040 --> 00:01:43.860 of the, in the first video, he puts a graph up and all the data is sort of packed together. 00:01:43.860 --> 00:01:47.260 So he changes the sample rate into just weekly data. 00:01:47.420 --> 00:01:50.880 And all that stuff, I didn't even know you could do those things. 00:01:50.880 --> 00:01:57.380 So it's not necessarily a complete, it's kind of a full pass through all of, using all the 00:01:57.380 --> 00:02:04.100 tools you can use to do exploratory data analysis with, Jupyter and doing it live and watching 00:02:04.100 --> 00:02:04.940 a pro do it. 00:02:04.940 --> 00:02:06.160 It's a thing of beauty. 00:02:06.160 --> 00:02:11.300 And he's, like I said, each particular tool that he use, uses, it's not an in-depth 00:02:11.300 --> 00:02:16.980 study on exactly how to use that to its completeness, but you just get a glimpse at all the 00:02:16.980 --> 00:02:18.500 power that you can do with all these things. 00:02:18.500 --> 00:02:18.740 Yeah. 00:02:18.740 --> 00:02:19.440 I really like it. 00:02:19.440 --> 00:02:22.920 And I think this sort of view into exploring data is super interesting. 00:02:22.920 --> 00:02:25.120 It really shows the power of Jupyter Notebooks. 00:02:25.120 --> 00:02:28.800 Like when I first saw them, I thought, oh, well, there's like a simplified programming environment 00:02:28.800 --> 00:02:32.560 for people that aren't like real programmers and don't want to work with different files 00:02:32.560 --> 00:02:33.780 and stuff like that. 00:02:33.780 --> 00:02:38.740 But the more I saw people using it and interacted with it, I realized it's just for people solving 00:02:38.740 --> 00:02:41.940 problems entirely differently than the type of problems I solve. 00:02:41.940 --> 00:02:43.260 And it's really great for that. 00:02:43.260 --> 00:02:43.560 Yeah. 00:02:43.560 --> 00:02:50.360 And, and, working with data, when you're throwing up a graph or, or a plot of the data, 00:02:50.360 --> 00:02:53.080 sometimes you might be plotting it wrong. 00:02:53.080 --> 00:02:55.880 You're like, well, maybe I could see something if I plot it this way. 00:02:56.280 --> 00:02:58.320 And the, it's not interesting. 00:02:58.320 --> 00:02:59.800 It's just a bunch of points everywhere. 00:02:59.800 --> 00:03:04.920 But if you plot it a little different with a different axis, maybe, or, or a different 00:03:04.920 --> 00:03:07.680 type of plot, it might show you interesting information. 00:03:07.680 --> 00:03:13.500 And this is actually fascinating to me, this, this notion of people using large data sets 00:03:13.500 --> 00:03:18.580 and then trying to figure out like in real time how to use it well. 00:03:18.580 --> 00:03:22.500 And then once you've, once you've already figured that out, if you want to put some program together 00:03:22.500 --> 00:03:26.520 using some of those tools to monitor those things, that's a great idea. 00:03:26.520 --> 00:03:32.680 But that, the ability to just use a notebook to, to just explore stuff is pretty fascinating. 00:03:32.680 --> 00:03:32.960 Yeah. 00:03:32.960 --> 00:03:34.840 And, you know, there's some really interesting use cases. 00:03:34.840 --> 00:03:39.480 Like suppose you want to go hit a bunch of APIs and download some data and then graph it. 00:03:39.480 --> 00:03:44.400 If you did that in like a script, every time you want to view it slightly differently, you're 00:03:44.400 --> 00:03:45.600 rerunning that down. 00:03:45.600 --> 00:03:47.900 You're getting or computing the data, however you do that. 00:03:47.900 --> 00:03:48.180 Right. 00:03:48.180 --> 00:03:52.600 Whereas in notebooks, you can just rerun one cell at a time, change that cell, run it again, 00:03:52.600 --> 00:03:54.640 and you don't have to recompute or reacquire the data. 00:03:54.640 --> 00:03:57.020 So it's really, there's a lot of interesting aspects to it. 00:03:57.020 --> 00:03:57.220 Yeah. 00:03:57.220 --> 00:04:01.080 And when, well, that's another thing he starts out with showing how to, within the notebook, 00:04:01.080 --> 00:04:07.240 um, grab a URL of data and, put it in a CSV file or some local file. 00:04:07.240 --> 00:04:10.680 So you don't have to, so you don't have to do that network all the time. 00:04:10.680 --> 00:04:10.920 Yeah. 00:04:10.920 --> 00:04:11.500 Nice. 00:04:11.500 --> 00:04:12.240 But anyway. 00:04:12.240 --> 00:04:12.960 Yeah, that's a good one. 00:04:12.960 --> 00:04:15.960 I really liked the series and I'm glad you, you, you brought it up. 00:04:15.960 --> 00:04:19.560 Another thing I want to bring up was something we haven't covered very much. 00:04:19.560 --> 00:04:21.560 Have we talked about GUIs and Python yet? 00:04:21.700 --> 00:04:22.680 You know, I think you're important. 00:04:22.680 --> 00:04:24.360 We probably should start talking about it. 00:04:24.360 --> 00:04:24.740 Yes. 00:04:24.740 --> 00:04:31.180 So because we were on our kick, of course, people said, you know, said, Oh, have you heard about 00:04:31.180 --> 00:04:31.440 this? 00:04:31.440 --> 00:04:32.180 Have you tried that? 00:04:32.180 --> 00:04:34.000 And here's another, here's another one. 00:04:34.000 --> 00:04:35.460 This one comes from Mike Barnett. 00:04:35.460 --> 00:04:40.760 he sent me a couple emails, sort of charting the progress of this project that he 00:04:40.760 --> 00:04:41.060 built. 00:04:41.060 --> 00:04:42.480 the name pretty much says it all. 00:04:42.480 --> 00:04:44.140 Py simple GUI. 00:04:44.140 --> 00:04:46.460 So it's for simple Python GUIs. 00:04:47.260 --> 00:04:51.580 And it's just another really simple way to take what would have been like a command line 00:04:51.580 --> 00:04:54.260 program and make it a lot better. 00:04:54.260 --> 00:05:00.800 So, you know, bonus points to Mike, because this not only has a screenshot, this has many 00:05:00.800 --> 00:05:02.380 screenshots and many examples. 00:05:02.420 --> 00:05:08.180 So as all the Python GUI libraries out there should have, if you want people to use them 00:05:08.180 --> 00:05:09.040 screenshots. 00:05:09.040 --> 00:05:10.680 So check it out. 00:05:10.680 --> 00:05:11.480 It's pretty cool. 00:05:11.480 --> 00:05:17.660 What you do is you work more or less in a hundred percent Python language API, and you 00:05:17.660 --> 00:05:21.060 don't work down at the GUI toolkit layer, which is cool. 00:05:21.060 --> 00:05:21.540 I think. 00:05:21.540 --> 00:05:26.420 So you define the UI and it has this sort of auto layout mechanism and it has things like 00:05:26.420 --> 00:05:30.760 slide bars and text boxes and buttons and it works in all the things. 00:05:30.760 --> 00:05:31.860 So that's pretty nice. 00:05:31.860 --> 00:05:36.440 It works on like a, it says one of the examples is, do you have a raspberry pie with a touchscreen? 00:05:36.440 --> 00:05:42.300 Well, you know, why don't you just write this one screen full of GUI code and you've got 00:05:42.300 --> 00:05:43.640 your touchscreen GUI working. 00:05:43.640 --> 00:05:45.240 Isn't that cool? 00:05:45.240 --> 00:05:45.960 How neat. 00:05:46.040 --> 00:05:46.160 Yeah. 00:05:46.160 --> 00:05:52.180 So for better or worse, it's based on Tkinter, which is good because it comes with Python. 00:05:52.180 --> 00:05:53.060 There's no dependencies. 00:05:53.060 --> 00:05:57.600 So literally you just pip install py simple GUI, or if you want, there's a single Python 00:05:57.600 --> 00:05:59.200 file you can include with your code. 00:05:59.200 --> 00:06:02.600 So there's not even the pip install package stuff. 00:06:02.600 --> 00:06:08.200 You can literally just go, here's the file, py simple GUI.py put next to my, my program. 00:06:08.200 --> 00:06:09.480 And that's all it needs. 00:06:09.480 --> 00:06:10.660 So that's cool. 00:06:10.660 --> 00:06:15.440 But I honestly, I think Tkinter looks a little dated, right? 00:06:15.460 --> 00:06:19.820 Like it looks like it belongs a little more in like war games than it does in 2018. 00:06:19.820 --> 00:06:24.820 But anyway, it's still pretty nice as it is. 00:06:24.820 --> 00:06:30.860 And one of the upcoming things that he lists on sort of the upcoming goals, or if anybody 00:06:30.860 --> 00:06:35.560 wants to help out is port this to other graphics engines, because you don't work in the toolkit 00:06:35.560 --> 00:06:38.760 API directly, but this is like a translation layer. 00:06:38.960 --> 00:06:45.460 You could hook this up to say wxPython or Python for QT, QT for Python, things like that. 00:06:45.460 --> 00:06:50.140 So you could get one of these modern looking UIs just by, if somebody's willing to do that 00:06:50.140 --> 00:06:50.600 translation. 00:06:50.600 --> 00:06:51.680 That's a neat idea. 00:06:51.680 --> 00:06:53.280 I'd like to see people working on that. 00:06:53.280 --> 00:06:53.460 Yeah. 00:06:53.460 --> 00:06:54.180 Wouldn't that be sweet? 00:06:54.180 --> 00:06:57.800 Then it could actually detect if you even have them and it wouldn't even, oh, do you have 00:06:57.800 --> 00:06:58.200 WX? 00:06:58.200 --> 00:06:58.640 Okay, great. 00:06:58.640 --> 00:06:59.220 We'll go with that. 00:06:59.220 --> 00:07:00.220 Oh, you have QT. 00:07:00.360 --> 00:07:01.700 Let's just run that version. 00:07:01.700 --> 00:07:03.640 Don't have to get a dependency installed. 00:07:03.640 --> 00:07:04.260 That'd be best. 00:07:04.260 --> 00:07:04.640 Yeah. 00:07:04.640 --> 00:07:11.100 And yeah, you're right that the TK stuff looks dated, but there's a lot of use cases where 00:07:11.100 --> 00:07:14.140 you can use a GUI that doesn't have to be pretty. 00:07:14.140 --> 00:07:14.440 Yeah. 00:07:14.440 --> 00:07:17.500 Well, you know what else can look dated is the command line to a lot of people. 00:07:17.500 --> 00:07:23.500 So maybe this is a lot better interaction for non-technical people, even if it does have 00:07:23.500 --> 00:07:25.520 a little bit funky shading on the buttons or something. 00:07:25.520 --> 00:07:25.960 Yeah. 00:07:25.960 --> 00:07:30.140 I mean, I don't get it, but I do have, I fight that battle every once in a while. 00:07:30.140 --> 00:07:32.520 I'll tell people this, just run this command line thing. 00:07:32.520 --> 00:07:34.680 What do you mean run the command line thing? 00:07:34.680 --> 00:07:35.520 Oh dear. 00:07:35.520 --> 00:07:36.840 They're like, I studied accounting. 00:07:36.840 --> 00:07:40.840 I don't know where the terminal is on my Mac. 00:07:40.840 --> 00:07:41.260 Okay. 00:07:41.260 --> 00:07:41.980 Okay. 00:07:41.980 --> 00:07:43.580 Then click this. 00:07:43.580 --> 00:07:44.420 Yeah. 00:07:44.420 --> 00:07:46.820 One other thing to call out for, I think that's interesting. 00:07:46.820 --> 00:07:49.000 This is Python three only. 00:07:49.000 --> 00:07:52.360 So no Python two for PySimple GUI. 00:07:52.360 --> 00:07:57.080 And more and more, again, we're seeing things where it used to be, well, I can't switch to 00:07:57.080 --> 00:07:58.480 Python three because this isn't supported. 00:07:58.480 --> 00:08:02.000 Now it's like, if you don't switch to Python three, you don't get these cool libraries. 00:08:02.000 --> 00:08:03.260 And here's just one more example. 00:08:03.260 --> 00:08:03.780 Yeah. 00:08:03.780 --> 00:08:04.700 Great. 00:08:07.700 --> 00:08:12.840 Well, I, we've got a, got a team that is migrating to get. 00:08:12.840 --> 00:08:18.640 And so this isn't directly Python related, but I ran across this article called useful tricks. 00:08:18.640 --> 00:08:22.260 You might not know about get stash and get stashes. 00:08:22.260 --> 00:08:25.980 The stash command is something that I actually did. 00:08:25.980 --> 00:08:31.660 It took me a while to run across because it's not, it's not something you have to use, but a 00:08:31.660 --> 00:08:37.560 stash is a way to sit to, so let's say you've got a, a, a repository where you've, you've 00:08:37.560 --> 00:08:38.080 cloned it. 00:08:38.080 --> 00:08:42.120 Now you've made some changes on it and you're not ready to do anything with your changes, 00:08:42.120 --> 00:08:47.960 but you need to like maybe pull down a new version or you branched at the wrong point or 00:08:47.960 --> 00:08:48.320 whatever. 00:08:48.800 --> 00:08:54.320 stash is a way to save off all of your changes, all the dirty stuff in your directory, save 00:08:54.320 --> 00:08:56.700 it away and just like hide it somewhere. 00:08:56.700 --> 00:09:01.820 And then you can reapply those changes later after you, you know, update or pull or something. 00:09:01.820 --> 00:09:08.760 And I'm still working through how to integrate stash into a good workflow, but I wanted to 00:09:08.760 --> 00:09:11.880 highlight this article useful tricks, useful tricks. 00:09:11.880 --> 00:09:16.620 You might not know about get stash because I'm, I'm learning it and, I wanted other people 00:09:16.620 --> 00:09:18.120 to know about it also. 00:09:18.300 --> 00:09:18.700 Yeah, that's cool. 00:09:18.700 --> 00:09:23.580 One of the ideas that seems like it might be relevant here is suppose you've got some 00:09:23.580 --> 00:09:26.060 branch checked out and you're doing some work, you know, like halfway through it. 00:09:26.060 --> 00:09:30.620 And somebody comes along and says, Hey, I'm on the same branch as you and we have this bug. 00:09:30.620 --> 00:09:35.340 Could you just like fix this really quick or make this quick change so that I can carry on? 00:09:35.340 --> 00:09:39.000 And you're like, Oh, but this work I have here is like half done. 00:09:39.000 --> 00:09:40.360 It won't be done until tomorrow. 00:09:40.360 --> 00:09:46.720 So you could like stash that away, get the latest, do some work, push that in and then reapply 00:09:46.720 --> 00:09:51.380 that stash to like get your work back without actually committing it and messing up that 00:09:51.380 --> 00:09:51.820 whole branch. 00:09:51.820 --> 00:09:52.500 Yeah, definitely. 00:09:52.500 --> 00:09:58.020 And the use case and then the use case we often use is, like, like 00:09:58.020 --> 00:10:01.560 the test team is working on different, different tests around. 00:10:01.560 --> 00:10:05.860 but we're sharing in utility libraries and fixtures and stuff. 00:10:06.100 --> 00:10:12.280 And, somebody updates, a crucial fixture or a utility communication utility. 00:10:12.280 --> 00:10:17.740 And, so you want to use that, but you're, you're in the middle of writing your test or 00:10:17.740 --> 00:10:19.140 writing, changing something new. 00:10:19.140 --> 00:10:26.040 These aren't merge conflicts at all, but get doesn't let you like pull in the new stuff on 00:10:26.040 --> 00:10:27.340 top of your old stuff. 00:10:27.340 --> 00:10:28.620 I mean, you can do a merge, right? 00:10:28.620 --> 00:10:32.860 But you still have to commit it to your local repo before you can do a pull, which you might 00:10:32.860 --> 00:10:33.680 not want to do yet. 00:10:33.760 --> 00:10:36.300 And if you're just starting out or whatever, you might not want to do that. 00:10:36.300 --> 00:10:37.460 Or if you just want to look at stuff. 00:10:37.460 --> 00:10:42.820 So that's the case where a lot of, we're playing with this workflow is to just stash 00:10:42.820 --> 00:10:44.780 away your changes to a pull. 00:10:44.780 --> 00:10:49.540 And again, people can correct me if I'm, if I'm using the term pull wrong, because I'm 00:10:49.540 --> 00:10:55.360 still learning at the right times to do pulls and fetches and merges and all that 00:10:55.360 --> 00:10:55.640 stuff. 00:10:55.640 --> 00:10:56.380 So yeah, nice. 00:10:56.380 --> 00:10:56.640 Yeah. 00:10:56.640 --> 00:10:57.480 This is really cool. 00:10:57.480 --> 00:11:00.180 Two things that you call it here that I thought were pretty cool. 00:11:00.180 --> 00:11:01.020 Well, I guess three. 00:11:01.020 --> 00:11:02.640 One is you can label your stashes. 00:11:02.640 --> 00:11:03.880 So like, you know what they mean? 00:11:03.880 --> 00:11:05.100 They're not just hashes. 00:11:05.100 --> 00:11:06.580 So that's good. 00:11:06.580 --> 00:11:11.260 Also, I didn't know you can do a dash you to include untracked files. 00:11:11.260 --> 00:11:12.580 That's pretty cool. 00:11:12.580 --> 00:11:13.360 That's pretty cool. 00:11:13.360 --> 00:11:15.320 I didn't know that either before I read the article. 00:11:15.500 --> 00:11:19.780 And the other one that the last one that I didn't, I totally didn't know you could do 00:11:19.780 --> 00:11:26.160 is once you have your stash saved, you could say, well, I probably shouldn't have done it 00:11:26.160 --> 00:11:26.800 as a stash. 00:11:26.800 --> 00:11:28.940 I probably should have just put that on a branch. 00:11:28.940 --> 00:11:35.420 There's a way to say, get stash branch and then a name, and then you can specify which stash. 00:11:35.740 --> 00:11:40.760 And it just takes all those files that you've, all those changes and creates a branch. 00:11:40.760 --> 00:11:42.000 That's cool. 00:11:42.000 --> 00:11:42.240 Yeah. 00:11:42.240 --> 00:11:43.100 I really like that one. 00:11:43.100 --> 00:11:44.160 Like, oh, I stash it. 00:11:44.160 --> 00:11:48.040 And actually, what I want to do is more work and sort of parallel, like break it off, split 00:11:48.040 --> 00:11:51.580 off my work without committing it to sort of convert the stash to a branch. 00:11:51.580 --> 00:11:52.020 That's cool. 00:11:52.020 --> 00:11:52.440 Yeah. 00:11:52.440 --> 00:11:52.640 Yeah. 00:11:52.640 --> 00:11:53.020 Very nice. 00:11:53.020 --> 00:11:53.600 I like it. 00:11:53.600 --> 00:11:57.300 So let me tell you about this new thing that DigitalOcean has. 00:11:57.300 --> 00:12:03.060 So they have virtual machines and floating IPs, and they have spaces and load balances and 00:12:03.060 --> 00:12:06.000 all these sorts of things, even domains and DNSs. 00:12:06.000 --> 00:12:11.420 And if you have a lot of stuff going on at DigitalOcean, well, you might have like 20 virtual 00:12:11.420 --> 00:12:13.920 machines, and some of them are for some project. 00:12:13.920 --> 00:12:15.100 Another one is for another project. 00:12:15.100 --> 00:12:17.680 Like, how do you know which one is for which? 00:12:17.680 --> 00:12:19.060 And is that one safe to delete? 00:12:19.060 --> 00:12:20.620 I think we're done with it, but I'm not sure. 00:12:20.620 --> 00:12:22.280 I actually don't really know what it belongs to. 00:12:22.680 --> 00:12:27.180 So they came up with this new feature called Projects, where you can group droplets, load 00:12:27.180 --> 00:12:32.520 balancers, domains, IP addresses, all that kind of stuff into one, two different projects. 00:12:32.520 --> 00:12:34.920 So you can say, this one is, say, for the training site. 00:12:34.920 --> 00:12:36.760 These three parts all fit together there. 00:12:36.760 --> 00:12:38.800 This one is for the Python Bytes podcast. 00:12:38.800 --> 00:12:42.560 And these two servers and spaces all fit together over there. 00:12:42.560 --> 00:12:43.640 So pretty cool. 00:12:43.640 --> 00:12:44.320 Check that out. 00:12:44.320 --> 00:12:47.560 It's just one more way to make your hosting life easier. 00:12:47.560 --> 00:12:48.340 Yeah. 00:12:48.340 --> 00:12:51.240 And be sure to visit pythonbytes.fm/DigitalOcean. 00:12:51.400 --> 00:12:53.020 If you're a new user, you get $100 credit. 00:12:53.020 --> 00:12:54.880 So that makes it even nicer. 00:12:54.880 --> 00:12:58.240 So one of the things I'd like to see, Brian, is more async stuff. 00:12:58.240 --> 00:13:02.520 And I think the place where it's most beneficial is around the web, actually. 00:13:02.520 --> 00:13:03.320 Well, yes. 00:13:03.320 --> 00:13:08.980 Last week, I said, because we can have, like, a bunch of different worker processes, it's not really necessary. 00:13:08.980 --> 00:13:09.780 Right? 00:13:09.860 --> 00:13:14.840 You can get, like, if you've got an eight-core server, you could have, say, 16 little uWSGI worker processes. 00:13:14.840 --> 00:13:21.980 And each one can sort of computationally chew up its stuff, like one core, and it sort of gets shared by the OS. 00:13:21.980 --> 00:13:26.800 But really, there's some limit where you don't want to create more because you run out of memory. 00:13:26.800 --> 00:13:27.480 Right? 00:13:27.540 --> 00:13:33.980 Like, I think the 16 on the training site probably take, like, two gigs of RAM. 00:13:33.980 --> 00:13:38.340 So you can't have many of them, or you'll run out of RAM unless you have a lot of room there. 00:13:38.900 --> 00:13:42.300 At some point, you maybe are waiting on a database call. 00:13:42.300 --> 00:13:42.560 Right? 00:13:42.560 --> 00:13:43.340 You do a request. 00:13:43.340 --> 00:13:48.040 The request says, well, in order to process this request, I need to have, like, hit this database. 00:13:48.040 --> 00:13:51.320 And actually, this is a query that takes 500 milliseconds to return. 00:13:51.320 --> 00:13:58.820 That thread is really doing nothing but just waiting on a socket to return something from, say, Postgres or MongoDB. 00:13:58.820 --> 00:14:04.220 And just as well could be doing other work if it could let go, but, you know, maybe it doesn't. 00:14:04.220 --> 00:14:04.500 Right? 00:14:04.620 --> 00:14:14.260 So if we can build this so that we could build it with async and await, any time our code is waiting, it immediately gives up its thread, and it will go on to do more processing. 00:14:14.260 --> 00:14:23.400 And so, like, for example, this is one of the ways, this is basically the fundamental process concept of how, like, Node.js can do hundreds of thousands of requests on one server. 00:14:23.400 --> 00:14:24.420 All right? 00:14:24.420 --> 00:14:24.860 Concurrent. 00:14:24.860 --> 00:14:28.100 Because most of those are waiting on a database or something else. 00:14:28.100 --> 00:14:28.660 Right? 00:14:29.520 --> 00:14:36.800 However, the problem is many of the Python, popular Python frameworks don't support this concept of async. 00:14:36.800 --> 00:14:44.320 We have new frameworks, Sanic, Jepranto, others that do support it, but those are not the old frameworks. 00:14:44.320 --> 00:14:44.540 Right? 00:14:44.540 --> 00:14:50.840 So there's, like, these new ones that are exciting and fast, and there's the old ones that everybody knows how to work with and have deployed. 00:14:50.840 --> 00:14:52.640 But bridging that gap is a challenge. 00:14:53.080 --> 00:15:01.560 So Andrew Godwin, the guy who worked on Django channels, works on Django channels, he came up with a Django async roadmap. 00:15:01.560 --> 00:15:04.100 And it's pretty interesting and pretty thorough. 00:15:04.100 --> 00:15:13.060 And it talks about, like, the time frame and how they might make Django support this sort of world where you can have async methods. 00:15:13.060 --> 00:15:24.400 Yeah, that's actually really cool because, I mean, if Django and Flask have to get there, or especially Django, or it's going to, something else will take over. 00:15:24.400 --> 00:15:25.060 Right, exactly. 00:15:25.060 --> 00:15:29.500 I mean, this is one of the times you hear people say, I'm switching to Go because it does better concurrency than Python. 00:15:29.500 --> 00:15:36.640 Well, if Django and the other frameworks just had it baked in, like, that whole argument would largely go away. 00:15:37.180 --> 00:15:40.920 So anyway, this is really cool, and I really like how he's put it together. 00:15:40.920 --> 00:15:46.640 He said he thinks it's time, the time has come to start seriously talking about bringing async functionality to Django. 00:15:46.640 --> 00:15:53.680 And he's shared it previously with some people internally, but this is him kind of coming out and saying, I'm opening it up for public feedback. 00:15:53.680 --> 00:15:56.660 So he has some interesting goals. 00:15:56.660 --> 00:16:02.620 He says, the goal is to make Django a world-class example of what async can enable for HTTP requests. 00:16:02.620 --> 00:16:06.160 And that means various things at different parts of the stack. 00:16:06.260 --> 00:16:10.080 So doing ORM requests in parallel, right, just waiting on the database. 00:16:10.080 --> 00:16:16.880 Instead of waiting the 500 milliseconds, you just continue doing processing and then get back to it when the ORM responds. 00:16:16.880 --> 00:16:20.380 Allowing views to query external APIs without blocking. 00:16:20.380 --> 00:16:22.540 So, you know, we talked about the retry stuff. 00:16:22.540 --> 00:16:29.620 If you're calling, like, a credit card or other sort of external API, right, then, you know, that would go away faster. 00:16:29.620 --> 00:16:33.500 You could do, like, slow response, long polling, super easy. 00:16:34.140 --> 00:16:35.920 It's like a sort of WebSocket stand-in. 00:16:35.920 --> 00:16:38.060 All sorts of performance improvements. 00:16:38.060 --> 00:16:41.840 So it's imperative that they keep Django backwards compatible. 00:16:41.840 --> 00:16:49.140 And to make sure that when people come to the project, this is an option they can turn on, not something they have to learn. 00:16:49.280 --> 00:16:53.020 So, like, part of the beauty of these frameworks is they've been really easy to get started with. 00:16:53.020 --> 00:16:56.780 Let's not throw this at people at the very first thing they ever do. 00:16:56.780 --> 00:16:57.200 Yeah. 00:16:57.200 --> 00:16:57.780 Yeah. 00:16:57.780 --> 00:17:02.080 There's a place for it and sometimes not a place for it. 00:17:02.080 --> 00:17:02.960 Yeah, yeah, exactly. 00:17:02.960 --> 00:17:04.200 All right. 00:17:04.260 --> 00:17:05.340 So I said, why now? 00:17:05.340 --> 00:17:13.320 Well, Django 2.1 will be the first release to support Python 3 and above and not the previous ones. 00:17:13.320 --> 00:17:18.040 And Python 3.5 and above, this is where async and await the language syntax. 00:17:18.040 --> 00:17:20.460 And, like, it truly has become properly supported. 00:17:20.460 --> 00:17:23.740 So that's why I think now is the time to start working on this. 00:17:24.020 --> 00:17:32.920 Yeah, and then sort of the timeline is broken out into different Django releases and which ones, what sort of goals might be for each one. 00:17:32.920 --> 00:17:33.700 That's nice. 00:17:33.700 --> 00:17:34.420 Yeah, it's pretty cool. 00:17:34.420 --> 00:17:39.900 Like, it doesn't make any sense to parallelize the web methods necessarily without parallelizing, say, the data access. 00:17:39.900 --> 00:17:41.900 So maybe start with the ORM, actually. 00:17:41.900 --> 00:17:42.660 Things like that. 00:17:42.660 --> 00:17:43.020 Yeah. 00:17:43.020 --> 00:17:43.960 I love this. 00:17:43.960 --> 00:17:47.240 I think that it's also good also talking about people. 00:17:47.240 --> 00:17:51.760 Like, it's not just individual developers, but companies that have applications. 00:17:52.480 --> 00:18:01.920 They might want to think about concurrency, but they don't necessarily – they don't need to do it right now, but they know that they're going to eventually do it to see this roadmap. 00:18:01.920 --> 00:18:04.600 And maybe that will help people. 00:18:04.600 --> 00:18:07.460 In the article also, it talks about funding. 00:18:07.460 --> 00:18:16.500 So if a lot of companies are relying on this or looking forward to it in the future, maybe kicking in some dollars to help it go faster is a good thing. 00:18:16.500 --> 00:18:16.760 Yeah. 00:18:16.760 --> 00:18:21.760 It blows my mind how many huge companies are basically built on Python infrastructure. 00:18:22.120 --> 00:18:23.500 But contribute zero to it. 00:18:23.500 --> 00:18:23.880 Yeah. 00:18:23.880 --> 00:18:32.540 And that's something that we're, as a society, and not just the Python community, but the web-using community, we've got to tackle that. 00:18:32.540 --> 00:18:37.960 But part of this is, I guess, just also things like this to say, hey, this is where we're going. 00:18:37.960 --> 00:18:42.060 And some of these problems need people focused on it, not just volunteer time. 00:18:42.660 --> 00:18:47.060 So some direct money to hire somebody for six months or a year would be a good idea. 00:18:47.060 --> 00:18:47.340 Yeah. 00:18:47.340 --> 00:18:50.100 A lot can get done with just like a few months of focus time. 00:18:50.100 --> 00:18:50.960 Yeah, definitely. 00:18:50.960 --> 00:18:51.360 Yeah. 00:18:51.360 --> 00:18:53.720 I mean, that's how the new PipeEye got launched. 00:18:53.720 --> 00:19:00.220 So this is all the way up to, it looks like, a mostly async Django by Django 3.2. 00:19:00.220 --> 00:19:00.940 Yeah, that's awesome. 00:19:00.940 --> 00:19:02.460 That's a way – it's pretty conservative. 00:19:02.460 --> 00:19:03.760 It's not too wild. 00:19:04.100 --> 00:19:06.240 And I really think it's a well-thought-out plan. 00:19:06.240 --> 00:19:08.700 So I'm happy to see Andrew put it out there. 00:19:08.700 --> 00:19:09.060 Nice. 00:19:09.060 --> 00:19:09.540 Yeah, nice. 00:19:09.540 --> 00:19:13.060 So you got some music you're going to play for us? 00:19:13.060 --> 00:19:17.420 We both actually end up processing a ton of audio files, don't we? 00:19:17.520 --> 00:19:20.000 We do, and I usually don't do it in Python. 00:19:20.000 --> 00:19:24.560 But this PyDub, plus what a great name, PyDub. 00:19:24.560 --> 00:19:30.160 But it makes me think that maybe a little bit of audio processing within Python might make sense. 00:19:30.160 --> 00:19:35.560 PyDub, the tagline is manipulate audio with a simple and easy high-level interface. 00:19:35.560 --> 00:19:40.600 But it's really actually pretty cool with just a single line. 00:19:40.600 --> 00:19:46.260 Like, for instance, from MP3, you can pull in an MP3 file into a variable. 00:19:46.260 --> 00:19:52.720 And then once you've got it there, you can do things like use the bracket operators to get the first 10 seconds. 00:19:52.720 --> 00:19:53.340 It's crazy. 00:19:53.340 --> 00:19:56.960 You can use the slice on it, and you can use indexing operators. 00:19:56.960 --> 00:19:57.940 It's crazy. 00:19:57.940 --> 00:20:03.620 And then adding or subtracting integers changes the volume by that number of decibels. 00:20:03.620 --> 00:20:06.720 The use of operators is pretty cool. 00:20:06.720 --> 00:20:12.700 Plus, you know, so slicing and chopping, but you can do crossfade and repeat and fade. 00:20:12.700 --> 00:20:15.780 And I'm not quite sure what the difference between crossfade and fade are. 00:20:15.780 --> 00:20:21.440 But anyway, changing formats from, like, say, WAV to MP3 or something. 00:20:21.440 --> 00:20:22.860 Adding meta tags. 00:20:22.860 --> 00:20:23.620 That's pretty cool. 00:20:23.620 --> 00:20:25.080 That's the one that got my attention. 00:20:25.080 --> 00:20:26.640 I'm like, oh, oh. 00:20:26.640 --> 00:20:30.180 This might help on some production stuff I'm doing. 00:20:30.180 --> 00:20:35.300 Making sure of a specific bit rates or MP3 has a quality level. 00:20:35.520 --> 00:20:37.880 You can pass all that stuff in for saving. 00:20:37.880 --> 00:20:43.060 Anyway, it's just like a really, we'll include a code snippet of a few things you can do. 00:20:43.060 --> 00:20:46.980 But it's pretty easy to maintain code once you've got it in place, I think. 00:20:46.980 --> 00:20:47.980 Yeah, this looks really interesting. 00:20:47.980 --> 00:20:50.280 If you do anything with audio, people should check this out. 00:20:50.700 --> 00:20:58.480 I did talk about, like, this tradeoff and how Django solving its async problem within itself would be great. 00:20:58.480 --> 00:21:02.440 But I still think there's room for exploration on the web in the Python world. 00:21:02.440 --> 00:21:04.340 And so this next one is pretty much that. 00:21:04.340 --> 00:21:07.160 It actually describes itself as an experimental framework. 00:21:07.160 --> 00:21:10.240 But it's called Molten, a modern API framework. 00:21:10.420 --> 00:21:11.060 Have you heard of this, Brian? 00:21:11.060 --> 00:21:11.640 No. 00:21:11.640 --> 00:21:19.280 So it's a minimal, fast web framework specifically for building APIs with Python. 00:21:19.280 --> 00:21:22.860 So I don't even know if it has, like, a template language for HTML. 00:21:22.860 --> 00:21:25.500 It's all about just building APIs. 00:21:25.500 --> 00:21:27.680 But it looks pretty awesome, actually. 00:21:27.680 --> 00:21:30.260 Yeah, and pretty terse and small. 00:21:30.260 --> 00:21:37.880 Yeah, one of the things that I like that it does is it uses type annotations for a whole bunch of cool things. 00:21:38.600 --> 00:21:44.560 So the other framework I saw do this was API star, but I don't think it quite used it as much. 00:21:44.560 --> 00:21:52.100 So, for example, you can have an API function that has a name, which is a string, and an age, which is an integer. 00:21:52.100 --> 00:21:58.120 And it will automatically pass that data over to you, as you call it, which is pretty awesome. 00:21:58.120 --> 00:22:00.140 It also does request validation. 00:22:00.140 --> 00:22:07.320 So you can create a class, which looks very much like a data annotation or looks like a data annotation class. 00:22:07.880 --> 00:22:10.040 And you give it a decorator and say, this is a schema. 00:22:10.040 --> 00:22:16.440 And what happens is if you say my API function takes this class as an argument. 00:22:16.440 --> 00:22:19.680 So their example has a to-do class. 00:22:19.680 --> 00:22:24.560 So if you say the input is colon to-do, right, you annotate it as a to-do. 00:22:25.260 --> 00:22:37.620 Then it will actually parse all the things like the ID and the description and all the various pieces out of the input and verify that, you know, the ID is a string. 00:22:37.620 --> 00:22:38.740 The ID is an integer. 00:22:38.740 --> 00:22:40.080 The description is a string. 00:22:40.080 --> 00:22:42.440 All that kind of stuff just by using type annotations. 00:22:44.400 --> 00:22:45.460 That's pretty cool. 00:22:45.460 --> 00:22:46.060 Yeah, yeah. 00:22:46.060 --> 00:22:47.080 It's pretty sweet. 00:22:47.080 --> 00:22:50.320 Here, I'll throw out the next one and see what you think about this. 00:22:50.320 --> 00:22:58.100 They also support dependency injection for allowing you to pass, like, different data access layers and stuff like that. 00:22:58.160 --> 00:23:02.780 So if you want to test it, you could pass in, like, a mocked-out data layer. 00:23:02.780 --> 00:23:10.680 Whereas by default, you just sort of register at AppStartup and it'll create all the different pieces of infrastructure and pass them to the methods automatically. 00:23:10.680 --> 00:23:11.260 Okay. 00:23:11.380 --> 00:23:12.200 Some people like that. 00:23:12.200 --> 00:23:14.640 Yeah, you know, I don't see that very often in the Python space. 00:23:14.640 --> 00:23:15.760 I have mixed emotions. 00:23:15.760 --> 00:23:16.860 Sometimes it's nice. 00:23:16.860 --> 00:23:17.520 Sometimes it's not. 00:23:17.520 --> 00:23:18.740 But anyway, it supports that. 00:23:18.740 --> 00:23:19.860 You don't have to use it, right? 00:23:19.860 --> 00:23:31.340 But I do think the validation and the schema and the auto-mapping of your sort of JSON documents to and from just strong classes with Python-based declarative requirements and stuff is really cool. 00:23:31.340 --> 00:23:38.680 Yeah, I think the extra thing that they're adding, this idea of using annotations as a schema, it's pretty cool. 00:23:38.680 --> 00:23:39.160 That's neat. 00:23:39.160 --> 00:23:40.400 Yeah, I really like it, too. 00:23:40.560 --> 00:23:47.280 And the other one that I looked at, sorry if I get this a little bit wrong, but there's some other framework that also used annotations that I thought was really cool. 00:23:47.280 --> 00:23:53.200 But it used them in a way that Python itself didn't make a lot of sense of. 00:23:53.200 --> 00:23:57.620 So, like, you could say, like, I'm getting an API key passed to me. 00:23:57.620 --> 00:24:02.860 And you would say colon header to say this API key is coming out of the header. 00:24:02.860 --> 00:24:05.360 But when you actually work with it, it's not actually a header. 00:24:05.360 --> 00:24:05.960 It's a string. 00:24:05.960 --> 00:24:07.480 It just came from the header. 00:24:07.800 --> 00:24:11.360 And so things like PyCharm and stuff would freak and go, that doesn't have this method. 00:24:11.360 --> 00:24:14.480 You're like, no, I know it's a string, even though I just actually said it's a header. 00:24:14.720 --> 00:24:18.380 Like, this is cool because the thing you say it is actually is what it is. 00:24:18.380 --> 00:24:21.580 The framework is consistent sort of with the programming model. 00:24:21.580 --> 00:24:22.220 I like that a lot. 00:24:22.220 --> 00:24:23.060 Yep. 00:24:23.400 --> 00:24:24.900 Anyway, pretty cool. 00:24:24.900 --> 00:24:28.040 And people can check that out if they're building APIs. 00:24:28.040 --> 00:24:29.640 Remember, it's in the experimental stage. 00:24:29.640 --> 00:24:33.900 But, you know, you can play with it, see if it fits your needs or make it better. 00:24:33.900 --> 00:24:34.500 Definitely. 00:24:34.500 --> 00:24:35.180 Nice. 00:24:35.180 --> 00:24:35.780 Yeah, pretty cool. 00:24:35.780 --> 00:24:36.300 All right. 00:24:36.300 --> 00:24:37.460 Anything else you want to share with us, Brian? 00:24:37.460 --> 00:24:38.860 No, I can't believe we're already done. 00:24:38.860 --> 00:24:39.220 I know. 00:24:39.220 --> 00:24:40.320 Same for me. 00:24:40.320 --> 00:24:41.440 I covered it all last week. 00:24:41.440 --> 00:24:44.420 So just always fun to share this stuff with you. 00:24:44.420 --> 00:24:45.120 Thanks for being here. 00:24:45.120 --> 00:24:45.840 Definitely fun. 00:24:45.840 --> 00:24:48.580 And everybody keep on sending us things that we should check out. 00:24:48.580 --> 00:24:49.960 I love getting tips from people. 00:24:49.960 --> 00:24:50.340 Absolutely. 00:24:50.340 --> 00:24:51.040 Same here. 00:24:51.040 --> 00:24:51.780 See you later. 00:24:51.780 --> 00:24:52.220 Bye. 00:24:52.220 --> 00:24:54.840 Thank you for listening to Python Bytes. 00:24:54.840 --> 00:24:57.400 Follow the show on Twitter via at Python Bytes. 00:24:57.400 --> 00:25:00.300 That's Python Bytes as in B-Y-T-E-S. 00:25:00.300 --> 00:25:03.740 And get the full show notes at Pythonbytes.fm. 00:25:03.740 --> 00:25:08.060 If you have a news item you want featured, just visit Pythonbytes.fm and send it our way. 00:25:08.060 --> 00:25:10.760 We're always on the lookout for sharing something cool. 00:25:10.760 --> 00:25:14.160 On behalf of myself and Brian Okken, this is Michael Kennedy. 00:25:14.160 --> 00:25:17.780 Thank you for listening and sharing this podcast with your friends and colleagues.