Hello and welcome to Python Bytes, where we deliver news and headlines directly to your earbuds.
This is episode 144, recorded August 20th, 2019. I'm Brian Ogden, and Michael is on vacation,
but I have two awesome guest hosts. Welcome, Chris May.
Hey there.
And also welcome, Mahmoud Hashemi.
Hello, everybody. It's good to be back.
So both of you have been on Test and Code, but have either of you been guests on Python Bytes before?
I think I actually guest hosted once in a similar situation where Michael was on vacation or you were on vacation.
I think it was me and Michael that time, so it's good to get the reverse.
Cool. And I definitely never have been on Python Bytes, but it's been something I would have been excited about ever since I thought I had the opportunity.
Dreams do come true.
Indeed.
I really appreciate both of you coming on.
As a reminder to everybody, this episode is brought to you by DigitalOcean.
Check them out at pythonbytes.fm/DigitalOcean.
Get a $50 credit for new users.
Lots of good stuff.
More about that later.
Let's just jump in.
I'm going to put Chris on the spot and have him go first since he's new here, and we can heckle.
Yeah, bring it on.
No.
Yeah, my first topic I selected was Why Your Mock Doesn't Work by Ned Batchelor.
And it's an article that just came out like perfect timing for my team and I because there are members of my team who are building an application that's syncing or bringing data from one database server to another across cloud platforms.
And essentially, they're trying to do a test-driven development.
And they came to me and they're like, how are we supposed to do this?
Like, there are all these third-party clients that we're connecting to and trying to get authorization from and all these different things.
And they quickly found out that mocking these third-party tools was hard.
I was just so thankful that Ned wrote this article to show us that one problem is that sometimes mocking, the way that they're setting up their mocks, the item that they were mocking never actually mocked the object that they were trying to mock.
It's a little hard to explain in video, or in audio, rather, but Ned actually has some great illustrations to show exactly how mocking works and when it works and how it can be messed up.
I saw those illustrations.
I don't know if he's using GraphViz or what, but we're definitely getting the point across.
Yeah.
I was wondering, because I thought he did such a great job with those illustrations.
Like, how did he take the time to design those?
It's good to know that there may be a tool behind that.
Well, I mean, and he's got, yeah.
I mean, he used them in his Python name and values talk in the blog post.
And I always, like, especially for beginner to intermediate programmers, I always send them to that Python names and values talk.
It's a really good use of a half hour.
Nice.
Hey, look, we're getting bonus content already.
Yeah, and that actually, that names and values talk actually makes you a better mock user as well, because it helps you understand how that all works.
Plus, Ned's just really good at explaining stuff.
Absolutely.
The other thing I love about this article is he also links to two other resources, another article and a video where people gave examples of when you don't want to use mocking and also good practices when you do.
And one of the takeaways I took away from my team is that, especially if you're trying to mock third party code, you're going to need to create tests that, of course, test your own application against the mock, but also test the mock against the third party tool.
And so this is a bunch of stuff that you're going to invest in if you want to test with mocks.
I mean, if I can jump in, what I'll say, whenever you're integrating, right, and there's an SDK or a library for integrating, chances are you're not going to use 100% of that library or that service.
You're probably going to use a little sliver of it.
I know it seems like, oh, great, they wrote all the code for me.
Let me jump right in and integrate directly with their library.
But it really makes sense a lot of the time to use a gateway or some other pattern to kind of like make explicit which parts of that library you use.
Because eventually when you have to integrate with another service or you want to throw it away, it's just going to be a lot easier to sort of like maintain that side of the integration.
So, yeah, great post.
Yeah, I also like to, I think it was in this post here where you mentioned kind of what you were saying, there's only a, usually there's only a thin sliver of the functionality you need.
And to use, you know, to create a mock that only does that, but also to make sure that across your organization or across your team to only have one mock object that handles like that one third party tool.
That's a good idea.
Well, Mahmoud, what do you got for us?
So what I've got for the group here is a little library slash application that I ran into while doing some research for a talk that I'll get into later.
But basically it's called Vermin.
And the name's not really explained, but the only reason, I mean, the only reason I can see behind it is that Python has snakes and snakes are kind of vermin and vermin kind of sounds like version.
And so what vermin is, that's like, you know, V-E-R-M-I-N.
Oh, wait, I just got it.
It's like version minimum.
Okay.
So what it does is it's a rules-based Python version compatibility detector with a pretty clever name.
So what it does, maybe you heard the news, but basically now all 360 top downloaded PyPI libraries now support Python 3.
There's like a website for it.
And the way that that's determined is I think Brett Cannon has a library or application which basically looks at the setup py and then looks at the like index classifiers.
And so if the person who published the package says it supports Python 3, it takes it at its word and says that this library says it supports Python 3.
Now the top 360 libraries have done so, which is a pretty good milestone for us, especially as we approach 2020.
But the thing is that it's based on what people say.
So if you, when it comes to your own code, right, like if you're like a team at a company or a maintainer of an application that doesn't necessarily have a setup.py,
there's, you know, can I use Python 3 is not really going to help you because you don't have those classifiers.
So basically vermin, what it does is it scans over every single py file, every single module, and it tells you the minimum Python version necessary to import that module.
And it even gets down to the function level.
So basically, you can use this to scan your code and say like, look, there's a Python 2.7 like feature being used by this function over here in this module.
And then you can also certify that, hey, this module over here uses like Python 3.5 syntax.
Or, hey, that one actually is using a 3.7 thing, but we're only on 3.6.
So it's rules-based.
It's got over a thousand rules.
It's pip install.
You pip install vermin.
And it has a little command line entry point.
And, yeah, when I found it, it had like less than 80 stars on GitHub.
And I'm hoping, you know, I saw how much work went into this.
I'm hoping that maybe it blows up a little bit because it's a very useful tool.
And I'll explain why in my next segment.
But basically, yeah, definitely check it out.
Vermin.
That's really cool.
I mean, I could definitely see.
Yeah, it sounds really cool.
It sounds like a lot of work went into that.
Exactly.
A lot of work went into it.
And it's still a little bit quirky, I'll tell you.
Like, I ran it with just plain vermin as described.
And I got one result.
And then I ran it with dash V, which made it kind of like, it's supposed to make it more verbose, I thought.
But it actually made it like look harder.
And it came back with a more accurate result.
So maybe when you run it, run it with dash V.
Or maybe like, you know, help the guy out.
I think his name is Morten Christensen.
Maybe help him out and give him a PR.
Make the verbose act the same way as the non-verbose.
But no, great tool regardless.
Ah, okay.
Verbose might be more picky.
Interesting.
Well, I think vermin looks cool.
And I'm excited about going from, not just from 2 to 3, but for instance, we now are excited about Python 3.8 features or 3.7 features.
And I want to make sure that, you know, it's possible to move things around.
Exactly.
There's a lot of use for that and forever, I think.
So anyway, I don't want to talk about a new feature.
I actually want to talk about something that's been around for a while, but I don't see it a lot in code.
And I just saw this recently, an article called the non-local statement in Python.
And I'm going to attempt this name, Abolash Raj.
I think that's it.
But the idea is if you use a local variable, like if you assign to a local variable, it creates a new thing in the namespace, a local name.
And, you know, the normal thing that everybody knows is unless you add the global.
But global doesn't just make it so that you can assign a variable that's not declared in this function.
It also makes it so that it's a global scope variable.
And sometimes that's not really what you want.
You just want it to reference.
Like, for instance, if you've got a local function defined within another function,
and you want to reference or assign to an outside variable,
it's really easy to see when you look at the code.
But I forgot this was around.
I don't use it very much, but it's pretty cool.
I think I run into this occasionally, and I usually just make things global.
So, Mahmoud, you probably have more experience than me.
Do you run into this ever?
Oh, my goodness.
So, back when I was, like, really learning the ropes in Python,
and I remember it so clearly back in, like, 2011,
I was playing with the concept of, like, dynamic scope,
which is in contrast to lexical scope.
So, that's, like, lexical scope, you sort of look within your block or your function, right?
And dynamic scope is this older concept where it's, like,
you would look up the stack for something.
And I think Emacs Lisp still uses this.
Kind of glad that Python doesn't.
But, you know, in, like, the mad science, like, you know,
mode of learning all of Python's cool features,
I wanted to, you know, do something like that.
Yeah, somewhere in there, I was, like, messing with generators and dynamic scope,
and I found myself just hitting right into a wall.
And I talked to Raymond Hedinger, and he's, like, oh, yeah, what you want is non-local,
and that's going to be sometime in the far future.
Yeah, anyways, I mean, here we are, and it's there.
And, frankly, if I use it, I'm over that phase.
So, if I use it, I feel like I'm doing something wrong,
and I should probably, like, factor this in a clearer way.
But that said, I'm sure it still comes up.
And, you know, I think everyone's probably going to have a little mad science phase of their own.
And so, let them use non-local and deal with the consequences.
Well, right.
I mean, it's, like, one of these features that it's used so little
that you're just adding complexity to the code because most people won't know what it means.
Right.
So, you got to be careful.
Yeah.
You know, it's still, like, it has its own little place,
and I'm sure some, like, someone has a legit use for it beyond, like, just messing around.
So, I'm glad it's there still.
Yeah, definitely.
I'm going to take a little bit of a sponsor break and thank DigitalOcean for sponsoring this show.
DigitalOcean has been a longtime supporter of Python Bytes, and we really appreciate it.
And not only that, all of the infrastructure for pythonbytes.fm runs on DigitalOcean.
We've got our website, our database, and even the audio file hosting is done through DigitalOcean.
It's been wonderful, super affordable, and solid.
There's enough features to get us going and to let us grow, but not so many that it's too complicated.
So, I think it's a great platform, easy to start, easy to grow, and highly recommended.
So, if you'd like to give it a shot, visit pythonbytes.fm/DigitalOcean.
It'll give you $50 free credit for new accounts, so you can give it a try.
I think it's a good place to be.
Get yourself a droplet.
Yeah, a droplet.
So, let's start this all over again, and not back from the beginning, but it's Chris's turn again.
So, Chris, what you got?
Well, this week, Microsoft announced that they've improved Python support in their Azure platform.
I think, in particular, they've announced general availability for a bunch of new Python things.
Some of which I've...
So, the company I work for, we've primarily used Microsoft Azure, and I've used some of these tools in beta.
But so now it's, like, available to everybody.
And I'm excited because reading through the announcement from Microsoft, I wasn't 100% sure what was new.
What I was really thankful for is Brett Cannon tweeted a link to it and said, here are the specific points that he was excited about.
The first off is that it's debuting with Python 3.6.
But 3.7 support is currently being worked on.
And actually, I can already see it in preview, which is nice.
And especially good news for me is that 3.8 support won't take nearly as long.
So, I'm hoping that very shortly after they release Python 3.8, it'll be available in Azure.
And then the second thing is that there's native async and await support in there.
And that's pretty exciting.
But it really...
The other thing that I wanted to pull from that, that Brett didn't mention in his tweet, is that it's actually pretty impressive, like, the amount of stuff that Microsoft does for us.
In particular, the Azure Functions, it's just amazing to me because they have a way of deploying to the web where you essentially just create a new Git remote.
And you just push your code up there.
And when it goes...
When your code arrives there, it essentially sticks into a Docker image and runs it for you.
And it's a pretty fast startup.
And all these things that's, like, so complicated, I can't imagine what's all going on.
And you don't have to worry about it at all.
You just have something that runs in your computer.
You push it up, and it's done.
And it's really, really impressive.
I've never played with Azure Functions.
So are those kind of like Lambda Functions on AWS?
Yeah, they sure are.
And I hear they're probably about as easy.
I've never actually...
I've intended to play with Azure Lambdas.
In fact, at the local Python meetup, we've had a couple people talk about how easy it is to do.
And I was just thankful that here at work, I've been able to try the Azure Functions.
So it's really nice.
Nice.
I'll have to give that a shot sometime.
Yeah.
Actually, I think Microsoft offers, like, a free, like, month or two-month thing if you want to do it.
And on top of that, the Azure Functions are incredibly cheap.
I just...
They have a calculator up, and you can check to see how much it'll cost to do certain things.
And I maxed it out.
Like, I have a function that calls a bunch of API endpoints and saves them to the database.
And we realized if we did it 48 times a day against every single website that the company owns, we still would be in the free tier.
So it's really impressive what you can do.
That's cool.
Well, I'm kind of excited about this next topic.
Like, super excited.
Because we talked about...
I'll have to dig up the episode, but we did bring up awesome Python applications before.
Right.
But we have an update.
Well, yeah.
No, I don't remember what it was.
If it was test and code Python bytes.
But I remember accidentally, like, sort of dropping this idea on the podcast, and it kind of blew up a little bit ahead of when I expected to launch it.
But, yeah.
So I'm here to bring, like, an awesome Python applications update.
And so, basically, awesome Python applications, for those who haven't heard of it, is sort of like your standard awesome list, where it's like you're on GitHub, and there's a readme, and there's a ton of links, you know.
And all these links point to applications that are free and open source Python.
And so, at first, it looks like a normal awesome list.
One, it's actually generated from some structured YAML.
And, you know, when I started out, it was like 25 applications.
Like, Zulip and Sentry sort of came to mind.
But then, like, the more I looked, the more I found, and it grew to something like 180 by the time, like, you know, it sort of went a little viral.
And then, these days, it's like 255.
Frankly, it'll hit 260 by the end of the week.
And so, the list keeps growing almost faster than I can keep up with it.
Thank goodness it's somewhat automated.
And so, this last weekend, I was at sort of the local Python conference, PyBay, you know, maybe CU PyBay 2020.
It was a pretty big hit there.
I basically, you know, I'll put in the show notes sort of a link to my slides and where the video will be and whatnot.
But what I did for my talk there was I took, like I said, structured YAML.
So, I have all of the links, all of the repos.
Technically, any awesome list could do this, and I'm surprised that more haven't.
But I went and I cloned all of the repos.
And this isn't just GitHub.
This is things that are on GitLab, things that are, like, sort of, like, off any particular software forge.
There are things that are on Bitbucket, things that are in Bazaar.
You know, it's a whole mix of things.
And I cloned them all.
And it was something like 30 gigabytes of code.
Yeah, and these packages, they date, or these applications, rather, date all the way back to, like, 1998, like Mailman and G-Edit.
You know, if you're a GNOME user, G-Edit uses Python.
And 95% of them have had a commit this year, 2019.
So, these are all very active.
And the reason I've compiled this isn't so much that we can go use them.
Like, of course, I actually use several of them without even realizing that they're Python.
But the thing is that, as application developers, we need exemplars we can go look to for patterns in testing and documentation and architecture and packaging.
Things that people do without necessarily writing a blog post about them or, you know, going on a podcast or giving a conference talk.
So, basically, I went and I cloned it, and I did all this analysis.
And this is how I found Vermin.
And so, Vermin, the good news here, like, let me just, I should have led with this, right?
But the good news is that, I was surprised by this, fully two-thirds of these Python applications, which are on median, like, you know, 10 years old, two-thirds of them support and use Python 3.
Wow.
So, I was expecting to find that a lot of these had sort of been, you know, it's tough to maintain open source.
And, you know, I thought that they would sort of have maybe, like, sort of fallen behind a little bit, right?
Goodness knows that most corporations I run into, well, I don't know about most, but, you know, a lot of them are still using Python 2 for their Python applications.
You know, even, like, you know, Google, everyone's favorite, right?
Even Google hasn't, like, fully ported over to Python 3 yet.
So, yeah, but 66% support Python 3.
And I've had a bunch of other interesting findings.
If you go to my slides, you can check them out.
And I'll also be giving the talk at PyGotham 2019.
So, if you're in New York, early October, I think tickets are available.
So, there's so many cool things in here.
And there's stuff that I knew about once and forgot about, like Eric, one of the Python IDEs.
So, that's fun.
Yeah.
Yeah, I mean, the newest one I found, someone at the conference came up and told me about it, is actually, I think, nia.is or .si or something like that.
It's a torrent tracker for anime.
And so, this particular anime fan informed me that this is one of the largest torrent trackers out.
And, you know, it's written in Python.
It's right there on GitHub.
And, you know, it's not on PyPI.
It's not really discoverable.
You need this word of mouth to kind of find these applications and all the lessons contained therein.
Like, I calculated it out.
Over 2,000 years of project maintenance, right, like in these repos that was just waiting to be learned from.
And, you know, I think the list should grow even further.
So, if any of the listeners have applications that they happen to know are written in Python, you know, hit me up on GitHub.
I'll definitely, like, you know, consider and curate.
Nice.
Well, this is cool.
Thanks.
Absolutely.
It's amazing to see how much that's grown.
I remember when you were on whichever podcast you were on and I was just inspired.
And now I'm even more so, again, to see, like, how many things have been built with Python.
And I still haven't reached the bottom yet.
That's amazing.
My dream is really to, like, you know, make a static site for this so that you can filter.
Sure, because basically, like, I can, I now have the data and I'll look at publishing this data set somewhere.
But basically, it's like, you know, show me the projects that use Docker.
Show me the projects that use PyInstaller.
Show me the projects that use Twisted versus whatever.
You mentioned AsyncIO right earlier.
And I was kind of surprised.
There are quite a few older projects that have actually adopted Async, like AsyncIO rather, you know.
Like, and it's pretty even split between concurrent futures, Twisted, Tornado, G-Event, and AsyncIO.
They're basically all around 20% on this list.
How about that?
Nice.
Yeah.
So, figure it out.
Well, I'm going to wrap this up with a really easy one.
We've talked about pre-commit several times, I think, on the show.
But if you, and pre-commit is a tool to allow you to run some code before you check in your code to make sure things that are whatever.
You've got a style guide or some checks or whatever you want to do.
It really is general purpose.
But how do you get started?
And actually, this is something that I needed myself.
The pre-commit documents now, they just recently added a quick start guide.
And I love it.
It just kind of walks you through installing pre-commit, configuring it for the first time, taking a look at what the hooks are, installing the hooks,
because you have to install the hooks also, which is something I wasn't aware of.
And then the initial attempt to just run your hooks against your project.
That's awesome.
But I wanted to add, and one of the reasons why I wanted to bring it up on the show wasn't just because, hey, go check this out.
But I'd like to add that doing this slowly is a good idea.
So I tried this with like three or four or five different hooks, like flake 8, black, and some other things, and then let it loose on my code base.
And it changed like everything.
And I thought, I don't really know what's going on here.
So I'd like to modify it a little bit to say, if you're adding this to a project, for each new hook, add them one at a time to your YAML file.
Install it and run it against your files first.
And then commit that.
If you're happy, review the changes.
If you're happy with it, your test pass and all that, check them in, and then move on to the next one.
I think it'll be easier for the rest of your team to swallow the fact that you just changed all the files.
That's great news.
This is one thing when I saw it in the notes I was super excited to see because we want to start doing this here at work.
And now that I know that there is a quick start guide to do it, it makes me so much more excited to do it.
Because I tried messing with the GitHub hooks or the GitHub hooks themselves.
I'm like, I really need to learn Bash.
Well, not just that.
They're just basically unusable in a team environment without something like this to manage them.
So I think Anthony Sotow kicked this off, and he's really just kept running with it right out of the park.
He's done a great job.
And the community as a whole, actually, too, because anyone can contribute sort of useful pre-commit hooks there, too.
That's awesome.
All right.
So that's done with our six.
Does anybody have any extra information they'd like to shout out?
I have a couple of things I'd like to share.
First off, the Humble Bundle, Humble Bundle, easy for me to say, by the No Starch Press, is throwing a special with the Python Software Foundation, where if you buy, you can get a collection of the books for a certain price.
And the PSF gets money, which is wonderful.
They can use as much money as they can get, I think.
And I'm all about supporting them.
I know friends of the show, PyBytes, have released their newbie bytes, which I know they have put in a ton of hard work into.
And I'm excited for them to have got it up and running.
I'm excited to, like, as somebody who runs a local Python meetup group, I'm excited that there's another resource out there to help people who are very new to Python to learn how to use it.
Nice.
That's cool.
I didn't have anything prepared, but, I mean, I guess I'll just shout out again.
Like, you know, I'll be doing an East Coast tour, like PyGotham and the Maintainers Conference in Washington, D.C.
You know, if you want to get a ticket to one of these, I'd be happy to meet anyone out there, chat about Python and versioning and all this stuff.
I should also, like, you know, shout out my wife.
She helped me make all the graphs for this awesome Python applications presentation.
And, yeah, she was using, like, real Python, I think, to learn a lot of the pandas to do so.
She really, again, just did an amazing job.
So, thanks, Maya.
Sweet.
Well, I was going to plug the Py3 readiness, but you already brought it up, so we'll drop a link to the other one.
The top 360 packages are now Python 3, which is great.
And I was hoping that one of you had a joke.
So, I only have one, and it's not very good.
So, either of you got a joke?
Joke.
Oh, my goodness.
You go first.
I'll see if it's an act I want to follow.
Okay.
Well, just, I was actually looking for some.
So, I was looking for programming one-liners, looked at a Reddit thread, read a great deal.
And I had a great answer, which was, any joke can be a one-liner with enough semicolons.
I think that joke is obfuscated enough that I'm not going to try to follow it.
Oh, I've got a good, bad one.
So, a SQL statement walks into a bar up to a couple of tables and says, can I join you?
Oh.
Yeah, yeah.
Just dropping that one on us.
Okay.
All right.
Well, let's leave it at that.
So, thank you, everyone, for sharing all these awesome items with us.
And we'll catch up with you later.
All right.
Sounds great.
Thanks, Brian.
Bye-bye.
Thank you for listening to Python Bytes.
Follow the show on Twitter via at Python Bytes.
That's Python Bytes as in B-Y-T-E-S.
And get the full show notes at pythonbytes.fm.
If you have a news item you want featured, just visit pythonbytes.fm and send it our way.
We're always on the lookout for sharing something cool.
On behalf of myself and Brian Okken, this is Michael Kennedy.
Thank you for listening and sharing this podcast with your friends and colleagues.