Hello and welcome to Python Bytes, where we deliver Python news directly to your earbuds.
This is episode 90, recorded August 2nd, 2018.
I'm Michael Kennedy.
And I'm Brian Okken.
Hey, Brian.
Good to be with you again.
It's good to talk to you again.
Yeah.
And it's also good to have DigitalOcean sponsoring this episode.
So thank you to DigitalOcean.
We're both customers and their sponsors of ours.
So it's kind of this weird mix of everybody loving it.
So pythonbytes.fm/DigitalOcean will get you $100 credit for your servers.
If you're a new customer.
So check that out.
In the meantime, we should probably talk about some data analysis.
We should.
And who better to talk about data analysis than Jake Vanderplass?
I honestly don't know.
He does awesome work over there at the eScience Institute.
And I love listening to his talks.
So there's a set of videos that he has on.
This is actually from last year, but I didn't count.
There's 11 videos and it's called Reproducible Data Analysis in Jupyter.
But they're, I mean, each of the videos is like five or six minutes.
So they go pretty fast.
This is a really cool thing.
I think everybody should check these out anyway is because they're, he goes through a problem
of, or just a data set that's, that the bikes cross, the bike crossings at a particular
bridge in Seattle, I think.
But it really doesn't matter the data, how the data gets in there.
But he's using, doing all this stuff live.
He's doing a Jupyter notebook and pulling data in.
And sometimes the tables like the table that he ends up with doesn't look quite right.
So he uses a different, different column to, as the index or the, for instance, I'm one
of the, in the first video, he puts a graph up and all the data is sort of packed together.
So he changes the sample rate into just weekly data.
And all that stuff, I didn't even know you could do those things.
So it's not necessarily a complete, it's kind of a full pass through all of, using all the
tools you can use to do exploratory data analysis with, Jupyter and doing it live and watching
a pro do it.
It's a thing of beauty.
And he's, like I said, each particular tool that he use, uses, it's not an in-depth
study on exactly how to use that to its completeness, but you just get a glimpse at all the
power that you can do with all these things.
Yeah.
I really like it.
And I think this sort of view into exploring data is super interesting.
It really shows the power of Jupyter Notebooks.
Like when I first saw them, I thought, oh, well, there's like a simplified programming environment
for people that aren't like real programmers and don't want to work with different files
and stuff like that.
But the more I saw people using it and interacted with it, I realized it's just for people solving
problems entirely differently than the type of problems I solve.
And it's really great for that.
Yeah.
And, and, working with data, when you're throwing up a graph or, or a plot of the data,
sometimes you might be plotting it wrong.
You're like, well, maybe I could see something if I plot it this way.
And the, it's not interesting.
It's just a bunch of points everywhere.
But if you plot it a little different with a different axis, maybe, or, or a different
type of plot, it might show you interesting information.
And this is actually fascinating to me, this, this notion of people using large data sets
and then trying to figure out like in real time how to use it well.
And then once you've, once you've already figured that out, if you want to put some program together
using some of those tools to monitor those things, that's a great idea.
But that, the ability to just use a notebook to, to just explore stuff is pretty fascinating.
Yeah.
And, you know, there's some really interesting use cases.
Like suppose you want to go hit a bunch of APIs and download some data and then graph it.
If you did that in like a script, every time you want to view it slightly differently, you're
rerunning that down.
You're getting or computing the data, however you do that.
Right.
Whereas in notebooks, you can just rerun one cell at a time, change that cell, run it again,
and you don't have to recompute or reacquire the data.
So it's really, there's a lot of interesting aspects to it.
Yeah.
And when, well, that's another thing he starts out with showing how to, within the notebook,
um, grab a URL of data and, put it in a CSV file or some local file.
So you don't have to, so you don't have to do that network all the time.
Yeah.
Nice.
But anyway.
Yeah, that's a good one.
I really liked the series and I'm glad you, you, you brought it up.
Another thing I want to bring up was something we haven't covered very much.
Have we talked about GUIs and Python yet?
You know, I think you're important.
We probably should start talking about it.
Yes.
So because we were on our kick, of course, people said, you know, said, Oh, have you heard about
this?
Have you tried that?
And here's another, here's another one.
This one comes from Mike Barnett.
Uh, he sent me a couple emails, sort of charting the progress of this project that he
built.
Uh, the name pretty much says it all.
Py simple GUI.
So it's for simple Python GUIs.
And it's just another really simple way to take what would have been like a command line
program and make it a lot better.
So, you know, bonus points to Mike, because this not only has a screenshot, this has many
screenshots and many examples.
So as all the Python GUI libraries out there should have, if you want people to use them
screenshots.
So check it out.
It's pretty cool.
What you do is you work more or less in a hundred percent Python language API, and you
don't work down at the GUI toolkit layer, which is cool.
I think.
So you define the UI and it has this sort of auto layout mechanism and it has things like
slide bars and text boxes and buttons and it works in all the things.
So that's pretty nice.
It works on like a, it says one of the examples is, do you have a raspberry pie with a touchscreen?
Well, you know, why don't you just write this one screen full of GUI code and you've got
your touchscreen GUI working.
Isn't that cool?
How neat.
Yeah.
So for better or worse, it's based on Tkinter, which is good because it comes with Python.
There's no dependencies.
So literally you just pip install py simple GUI, or if you want, there's a single Python
file you can include with your code.
So there's not even the, the pip install package stuff.
You can literally just go, here's the file, py simple GUI.py put next to my, my program.
And that's all it needs.
So that's cool.
But I honestly, I think Tkinter looks a little dated, right?
Like it looks like it belongs a little more in like war games than it does in 2018.
But anyway, it's still pretty nice as it is.
And one of the upcoming things that he lists on sort of the upcoming goals, or if anybody
wants to help out is port this to other graphics engines, because you don't work in the toolkit
API directly, but this is like a translation layer.
You could hook this up to say WX Python or Python for QT, QT for Python, things like that.
So you could get one of these modern looking UIs just by, if somebody's willing to do that
translation.
That's a neat idea.
I'd like to see people working on that.
Yeah.
Wouldn't that be sweet?
Then it could actually detect if you even have them and it wouldn't even, oh, do you have
WX?
Okay, great.
We'll go with that.
Oh, you have QT.
Let's just run that version.
Don't have to get a dependency installed.
That'd be best.
Yeah.
And yeah, you're right that the TK stuff looks dated, but there's a lot of use cases where
you can use a GUI that doesn't have to be pretty.
Yeah.
Well, you know what else can look dated is the command line to a lot of people.
So maybe this is a lot better interaction for non-technical people, even if it does have
a little bit funky shading on the buttons or something.
Yeah.
I mean, I don't get it, but I do have, I fight that battle every once in a while.
I'll tell people this, just run this command line thing.
What do you mean run the command line thing?
Oh dear.
They're like, I studied accounting.
I don't know where the terminal is on my Mac.
Okay.
Then click this.
Yeah.
One other thing to call out for, I think that's interesting.
This is Python three only.
So no Python two for PySimple GUI.
And more and more, again, we're seeing things where it used to be, well, I can't switch to
Python three because this isn't supported.
Now it's like, if you don't switch to Python three, you don't get these cool libraries.
And here's just one more example.
Yeah.
Great.
Well, I, we've got a, got a team that is migrating to get.
And so this isn't directly Python related, but I ran across this article called useful tricks.
You might not know about get stash and get stashes.
The stash command is something that I actually did.
It took me a while to run across because it's not, it's not something you have to use, but a
stash is a way to sit to, so let's say you've got a, a, a repository where you've, you've
cloned it.
Now you've made some changes on it and you're not ready to do anything with your changes,
but you need to like maybe pull down a new version or you branched at the wrong point or
whatever.
stash is a way to save off all of your changes, all the dirty stuff in your directory, save
it away and just like hide it somewhere.
And then you can reapply those changes later after you, you know, update or pull or something.
And I'm still working through how to integrate stash into a good workflow, but I wanted to
highlight this article useful tricks, useful tricks.
You might not know about get stash because I'm, I'm learning it and, I wanted other people
to know about it also.
Yeah, that's cool.
One of the ideas that seems like it might be relevant here is suppose you've got some
branch checked out and you're doing some work, you know, like halfway through it.
And somebody comes along and says, Hey, I'm on the same branch as you and we have this bug.
Could you just like fix this really quick or make this quick change so that I can carry on?
And you're like, Oh, but this work I have here is like half done.
It won't be done until tomorrow.
So you could like stash that away, get the latest, do some work, push that in and then reapply
that stash to like get your work back without actually committing it and messing up that
whole branch.
Yeah, definitely.
And the, the, the, the use case and then the use case we often use is, like, like
the test team is working on different, different tests around.
Um, but we're sharing in utility libraries and fixtures and stuff.
And, somebody updates, a crucial fixture or a utility communication utility.
And, so you want to use that, but you're, you're in the middle of writing your test or
writing, changing something new.
These aren't merge conflicts at all, but get doesn't let you like pull in the new stuff on
top of your old stuff.
I mean, you can do a merge, right?
But you still have to commit it to your local repo before you can do a pull, which you might
not want to do yet.
And if you're just starting out or whatever, you might not want to do that.
Or if you just want to look at stuff.
So that's the case where a lot of, we're playing with this workflow is to just stash
away your changes to a pull.
And again, people can correct me if I'm, if I'm using the term pull wrong, because I'm
still learning at the, the right times to do pulls and fetches and merges and all that
stuff.
So yeah, nice.
Yeah.
This is really cool.
Two things that you call it here that I thought were pretty cool.
Well, I guess three.
One is you can label your stashes.
So like, you know what they mean?
They're not just hashes.
So that's good.
Also, I didn't know you can do a dash you to include untracked files.
That's pretty cool.
I didn't know that either before I read the article.
And the other one that the last one that I didn't, I totally didn't know you could do
is once you have your stash saved, you could say, well, I probably shouldn't have done it
as a stash.
I probably should have just put that on a branch.
There's a way to say, get stash branch and then a name, and then you can specify which stash.
And it just takes all those files that you've, all those changes and creates a branch.
That's cool.
Yeah.
I really like that one.
Like, oh, I stash it.
And actually, what I want to do is more work and sort of parallel, like break it off, split
off my work without committing it to sort of convert the stash to a branch.
That's cool.
Yeah.
Very nice.
I like it.
So let me tell you about this new thing that DigitalOcean has.
So they have virtual machines and floating IPs, and they have spaces and load balances and
all these sorts of things, even domains and DNSs.
And if you have a lot of stuff going on at DigitalOcean, well, you might have like 20 virtual
machines, and some of them are for some project.
Another one is for another project.
Like, how do you know which one is for which?
And is that one safe to delete?
I think we're done with it, but I'm not sure.
I actually don't really know what it belongs to.
So they came up with this new feature called Projects, where you can group droplets, load
balancers, domains, IP addresses, all that kind of stuff into one, two different projects.
So you can say, this one is, say, for the training site.
These three parts all fit together there.
This one is for the Python Bytes podcast.
And these two servers and spaces all fit together over there.
So pretty cool.
Check that out.
It's just one more way to make your hosting life easier.
Yeah.
And be sure to visit pythonbytes.fm/DigitalOcean.
If you're a new user, you get $100 credit.
So that makes it even nicer.
So one of the things I'd like to see, Brian, is more async stuff.
And I think the place where it's most beneficial is around the web, actually.
Well, yes.
Last week, I said, because we can have, like, a bunch of different worker processes, it's not really necessary.
Right?
You can get, like, if you've got an eight-core server, you could have, say, 16 little micro WSGI worker processes.
And each one can sort of computationally chew up its stuff, like one core, and it sort of gets shared by the OS.
But really, there's some limit where you don't want to create more because you run out of memory.
Right?
Like, I think the 16 on the training site probably take, like, two gigs of RAM.
So you can't have many of them, or you'll run out of RAM unless you have a lot of room there.
At some point, you maybe are waiting on a database call.
Right?
You do a request.
The request says, well, in order to process this request, I need to have, like, hit this database.
And actually, this is a query that takes 500 milliseconds to return.
That thread is really doing nothing but just waiting on a socket to return something from, say, Postgres or MongoDB.
And just as well could be doing other work if it could let go, but, you know, maybe it doesn't.
Right?
So if we can build this so that we could build it with async and await, any time our code is waiting, it immediately gives up its thread, and it will go on to do more processing.
And so, like, for example, this is one of the ways, this is basically the fundamental process concept of how, like, Node.js can do hundreds of thousands of requests on one server.
All right?
Concurrent.
Because most of those are waiting on a database or something else.
Right?
However, the problem is many of the Python, popular Python frameworks don't support this concept of async.
We have new frameworks, Sanic, Jepranto, others that do support it, but those are not the old frameworks.
Right?
So there's, like, these new ones that are exciting and fast, and there's the old ones that everybody knows how to work with and have deployed.
But bridging that gap is a challenge.
So Andrew Godwin, the guy who worked on Django channels, works on Django channels, he came up with a Django async roadmap.
And it's pretty interesting and pretty thorough.
And it talks about, like, the time frame and how they might make Django support this sort of world where you can have async methods.
Yeah, that's actually really cool because, I mean, if Django and Flask have to get there, or especially Django, or it's going to, something else will take over.
Right, exactly.
I mean, this is one of the times you hear people say, I'm switching to Go because it does better concurrency than Python.
Well, if Django and the other frameworks just had it baked in, like, that whole argument would largely go away.
So anyway, this is really cool, and I really like how he's put it together.
He said he thinks it's time, the time has come to start seriously talking about bringing async functionality to Django.
And he's shared it previously with some people internally, but this is him kind of coming out and saying, I'm opening it up for public feedback.
So he has some interesting goals.
He says, the goal is to make Django a world-class example of what async can enable for HTTP requests.
And that means various things at different parts of the stack.
So doing ORM requests in parallel, right, just waiting on the database.
Instead of waiting the 500 milliseconds, you just continue doing processing and then get back to it when the ORM responds.
Allowing views to query external APIs without blocking.
So, you know, we talked about the retry stuff.
If you're calling, like, a credit card or other sort of external API, right, then, you know, that would go away faster.
You could do, like, slow response, long polling, super easy.
It's like a sort of WebSocket stand-in.
All sorts of performance improvements.
So it's imperative that they keep Django backwards compatible.
And to make sure that when people come to the project, this is an option they can turn on, not something they have to learn.
So, like, part of the beauty of these frameworks is they've been really easy to get started with.
Let's not throw this at people at the very first thing they ever do.
Yeah.
There's a place for it and sometimes not a place for it.
Yeah, yeah, exactly.
All right.
So I said, why now?
Well, Django 2.1 will be the first release to support Python 3 and above and not the previous ones.
And Python 3.5 and above, this is where async and await the language syntax.
And, like, it truly has become properly supported.
So that's why I think now is the time to start working on this.
Yeah, and then sort of the timeline is broken out into different Django releases and which ones, what sort of goals might be for each one.
That's nice.
Yeah, it's pretty cool.
Like, it doesn't make any sense to parallelize the web methods necessarily without parallelizing, say, the data access.
So maybe start with the ORM, actually.
Things like that.
Yeah.
I love this.
I think that it's also good also talking about people.
Like, it's not just individual developers, but companies that have applications.
They might want to think about concurrency, but they don't necessarily – they don't need to do it right now, but they know that they're going to eventually do it to see this roadmap.
And maybe that will help people.
In the article also, it talks about funding.
So if a lot of companies are relying on this or looking forward to it in the future, maybe kicking in some dollars to help it go faster is a good thing.
Yeah.
It blows my mind how many huge companies are basically built on Python infrastructure.
But contribute zero to it.
Yeah.
And that's something that we're, as a society, and not just the Python community, but the web-using community, we've got to tackle that.
But part of this is, I guess, just also things like this to say, hey, this is where we're going.
And some of these problems need people focused on it, not just volunteer time.
So some direct money to hire somebody for six months or a year would be a good idea.
Yeah.
A lot can get done with just like a few months of focus time.
Yeah, definitely.
Yeah.
I mean, that's how the new PipeEye got launched.
So this is all the way up to, it looks like, a mostly async Django by Django 3.2.
Yeah, that's awesome.
That's a way – it's pretty conservative.
It's not too wild.
And I really think it's a well-thought-out plan.
So I'm happy to see Andrew put it out there.
Nice.
Yeah, nice.
So you got some music you're going to play for us?
We both actually end up processing a ton of audio files, don't we?
We do, and I usually don't do it in Python.
But this PyDub, plus what a great name, PyDub.
But it makes me think that maybe a little bit of audio processing within Python might make sense.
PyDub, the tagline is manipulate audio with a simple and easy high-level interface.
But it's really actually pretty cool with just a single line.
Like, for instance, from MP3, you can pull in an MP3 file into a variable.
And then once you've got it there, you can do things like use the bracket operators to get the first 10 seconds.
It's crazy.
You can use the slice on it, and you can use indexing operators.
It's crazy.
And then adding or subtracting integers changes the volume by that number of decibels.
The use of operators is pretty cool.
Plus, you know, so slicing and chopping, but you can do crossfade and repeat and fade.
And I'm not quite sure what the difference between crossfade and fade are.
But anyway, changing formats from, like, say, WAV to MP3 or something.
Adding meta tags.
That's pretty cool.
That's the one that got my attention.
I'm like, oh, oh.
This might help on some production stuff I'm doing.
Making sure of a specific bit rates or MP3 has a quality level.
You can pass all that stuff in for saving.
Anyway, it's just like a really, we'll include a code snippet of a few things you can do.
But it's pretty easy to maintain code once you've got it in place, I think.
Yeah, this looks really interesting.
If you do anything with audio, people should check this out.
I did talk about, like, this tradeoff and how Django solving its async problem within itself would be great.
But I still think there's room for exploration on the web in the Python world.
And so this next one is pretty much that.
It actually describes itself as an experimental framework.
But it's called Molten, a modern API framework.
Have you heard of this, Brian?
No.
So it's a minimal, fast web framework specifically for building APIs with Python.
So I don't even know if it has, like, a template language for HTML.
It's all about just building APIs.
But it looks pretty awesome, actually.
Yeah, and pretty terse and small.
Yeah, one of the things that I like that it does is it uses type annotations for a whole bunch of cool things.
So the other framework I saw do this was API star, but I don't think it quite used it as much.
So, for example, you can have an API function that has a name, which is a string, and an age, which is an integer.
And it will automatically pass that data over to you, as you call it, which is pretty awesome.
It also does request validation.
So you can create a class, which looks very much like a data annotation or looks like a data annotation class.
And you give it a decorator and say, this is a schema.
And what happens is if you say my API function takes this class as an argument.
So their example has a to-do class.
So if you say the input is colon to-do, right, you annotate it as a to-do.
Then it will actually parse all the things like the ID and the description and all the various pieces out of the input and verify that, you know, the ID is a string.
The ID is an integer.
The description is a string.
All that kind of stuff just by using type annotations.
That's pretty cool.
Yeah, yeah.
It's pretty sweet.
Here, I'll throw out the next one and see what you think about this.
They also support dependency injection for allowing you to pass, like, different data access layers and stuff like that.
So if you want to test it, you could pass in, like, a mocked-out data layer.
Whereas by default, you just sort of register at AppStartup and it'll create all the different pieces of infrastructure and pass them to the methods automatically.
Okay.
Some people like that.
Yeah, you know, I don't see that very often in the Python space.
I have mixed emotions.
Sometimes it's nice.
Sometimes it's not.
But anyway, it supports that.
You don't have to use it, right?
But I do think the validation and the schema and the auto-mapping of your sort of JSON documents to and from just strong classes with Python-based declarative requirements and stuff is really cool.
Yeah, I think the extra thing that they're adding, this idea of using annotations as a schema, it's pretty cool.
That's neat.
Yeah, I really like it, too.
And the other one that I looked at, sorry if I get this a little bit wrong, but there's some other framework that also used annotations that I thought was really cool.
But it used them in a way that Python itself didn't make a lot of sense of.
So, like, you could say, like, I'm getting an API key passed to me.
And you would say colon header to say this API key is coming out of the header.
But when you actually work with it, it's not actually a header.
It's a string.
It just came from the header.
And so things like PyTarm and stuff would freak and go, that doesn't have this method.
You're like, no, I know it's a string, even though I just actually said it's a header.
Like, this is cool because the thing you say it is actually is what it is.
The framework is consistent sort of with the programming model.
I like that a lot.
Yep.
Anyway, pretty cool.
And people can check that out if they're building APIs.
Remember, it's in the experimental stage.
But, you know, you can play with it, see if it fits your needs or make it better.
Definitely.
Nice.
Yeah, pretty cool.
All right.
Anything else you want to share with us, Brian?
No, I can't believe we're already done.
I know.
Same for me.
I covered it all last week.
So just always fun to share this stuff with you.
Thanks for being here.
Definitely fun.
And everybody keep on sending us things that we should check out.
I love getting tips from people.
Absolutely.
Same here.
See you later.
Bye.
Thank you for listening to Python Bytes.
Follow the show on Twitter via at Python Bytes.
That's Python Bytes as in B-Y-T-E-S.
And get the full show notes at Pythonbytes.fm.
If you have a news item you want featured, just visit Pythonbytes.fm and send it our way.
We're always on the lookout for sharing something cool.
On behalf of myself and Brian Okken, this is Michael Kennedy.
Thank you for listening and sharing this podcast with your friends and colleagues.