Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to
your earbuds. This is episode 199, almost 200, Brian. Almost. Almost. Recorded September 9th,
2020. I'm Michael Kennedy. And I'm Brian Okken. And this episode is brought to you by us. We'll
tell you more about the things that we're doing and what we have to offer later. But for now,
I want to talk about not any huge announcement, but a very, very small announcement that it's
also kind of huge. Very cool. Yeah. So micro, you might say. You might say it's microscopic,
but it's a big deal. So MicroPython has just had a new release. And this is significant because the
last release from MicroPython was December 19th of last year. So it's been, you know, what is that?
Almost 10 months since the last release. So if you care about MicroPython, this is like a pretty big
deal. So version 113 is out. And one of the major things is it has improved asyncio,
async and await support. And it has cool Bluetooth improvements. And it even comes with the PEP 526,
that walrus operator. Oh, nice. Yeah. So there's a lot of stuff going on here. They basically,
MicroPython, as you can imagine, comes with a micro asyncio module. I'm guessing the U is really a MU.
U asyncio module that's supposed to be more compatible with CPython's asyncio. So the main idea is that the
task object, you can use a task object for each coroutine, which allows you to have like unbounded
queues of work and stuff like that. Whereas previously, I think you had to pre-allocate it for some reason.
I never tried to do asyncio on MicroPython, but apparently it was a little bit wonky. And so now it's more
consistent. Another thing that I thought was interesting about this release, Brian, was they went through and
auto-formatted all the code in the entire repository. And if you look at their repo, there is Python and MicroPython,
but a lot like CPython, it's a lot of C as well. Right? I think it's like 80% C or something.
Okay.
And so they auto-formatted the Python code with black, and that probably won't surprise too many people,
but they auto-formatted C with that community's equivalent of black, and get ready for the name,
Uncrustify. They uncrustified the C code.
Okay.
Do you know about this?
No, actually, I'm writing it down because I'd like to check this out.
Isn't that great? So you can basically configure how it formats your C code. And it's like black. It's
like, you know, it just runs automatically and fixes your C code. And boy, if you thought Python
code needed some fixing, I'll bet you the C code. And there's more variety in there, right?
Yeah. I've seen some ugly C code in my day.
Yeah, for sure. For sure. Because with the curly braces and semicolons, you have a lot more flexibility
in how you format stuff, whereas Python forces some of that structure upon you.
All right. They also added Blue Kitchen. Apparently Blue Kitchen is a Bluetooth stack. And so they added a
micro Mew Bluetooth module as an alternative to the Nimble stack as well. And there's a Unix port,
and you can build the Unix port with the Bluetooth support for either of these bindings. That's pretty
cool. That comes with new events. So if you're doing like Bluetooth discovery and things like that,
you can like scan for Bluetooth stuff more easily. It also came with a big memory leak fix for ARM
chips, which is big in the small space, ironically. So apparently, there was certain memory that was
not reclaimed by the garbage collector and other types of things. So if you've, you know, and at the
same time on these very small devices, you care a lot about not wasting memory, right? It's not like
you got 32 gigs like my laptop does. So like whatever, right? It's going to matter if you lose memory on
like a little micro Python chip. And also they set it up so you can run parallel tests against multiple
micro Python targets. So I'm guessing like different chips and different devices, you can just say go test
them, run it synchronized across two or more micro Python targets. I don't know if that's to speed it
up or to like, test the whole thing on different platforms in parallel or whatever. But it sounds like a
good improvement. And it does come with a few breaking changes. So people can check that out.
Anyway, if you care about micro Python, and I'm guessing this may make its way over to CircuitPython
as well at some point, knowing that those projects are really close. So anyway, this looks really good
if you're doing big stuff on small things.
Yeah, it's good to see this going forward.
And thanks to Matt Trentini for sharing this and bringing it to our attention.
I wish I was better at remembering names, but somebody I had on testing code brought up
respx and or respix. I should stop. R-E-S-P-X. It's a utility for mocking out the Python HTTPX library.
So that somebody brought that up and I had never heard of it before. So I figured we'd bring it up
here. So with requests, if you want to mock it, there's a whole library called responses and cleverly
named a bit hard to search for. And then with HTTPX, if you want to mock it, there is respx.
I'm going to name it response X, but we just the other part, you just don't write it.
It's like the opposite of a silent E. It's it's not written.
Maybe.
Sorry, keep going. It sounds like a great idea.
Well, interestingly enough, when you write the two imports out, they're the same number of characters.
Maybe that's part of the reason why they did that. Anyway, that quick start is like shows it's really quick.
You just it's a really just a few lines of code to mock out a request response call or response call.
You can do things like things that you would expect, like an assert that something was actually called
and what the status code was and things like that.
But you can also have custom content and you can return JSON content.
of course, there's some examples. What I like is the documentation is pretty nice.
There's some examples on how to use it with both pytest and UnitTest,
including how to set up like a mocked API fixture for pytest.
You can have multiple tests using the same mocked endpoint.
And there's a bunch of nice things about it.
Like, for instance, if you want to mock out a whole bunch of different URLs,
you can use some regular expressions to set what URLs you want to catch with that.
and then also the content that you return from your mocked object, your mocked request is you can have a callback that can generate that on the fly based on what you pass in it.
If you have a HTTPX based application, you definitely need to know about this for testing.
Yeah, absolutely.
And one of the things that's challenging about testing the HTTPX is the async stuff, right?
And so this is a really cool way to get in there and just make it work, right?
Yeah.
Just put a decorator there and it just mocks it out for you like you would expect.
Looks very clean.
Yeah, pretty easy.
And there's a lot of magic going on under the hood to make that a clean interface, but I think they did a good job.
Yeah.
So HTTPX comes from sort of an async derivative of requests, which I think is a really fantastic library.
And I've done some cool stuff with HTTPX.
So yeah, I'm a fan of this.
It's great.
Now, before we move on, let's talk about a couple of things that we got going on that people can check out that will definitely help support the show.
And they might find interesting.
I know that you've got some updates for your book.
Is that right?
Yeah.
The Python testing with pytestBook, just a minor, the book itself has a minor update.
So there's, it wasn't a big enough change to change the hardback version, but the ebook will be updated.
I don't think the hardback is going to be updated, but it's a couple lines of code in a couple of code examples in chapter five.
So it's a very minor thing.
But the big difference is the code download.
So we've updated the code download.
So even if you're working with the hardback book or using the old version and don't update the ebook, please redownload the code.
It'll make your learning experience better.
And the main change is that I've pinned the dependencies in the target project or the example project so that everything works better.
Good.
There was a tiny DB is a database that I use in the project.
And it had some incompatible changes.
And instead of trying to update everything to use the new, the new tiny DB, I just pinned it to an old version.
Yeah, that makes sense.
I mean, it's not like you're trying to teach people tiny DB.
You're like, here's a dependency.
We're trying to get around to something.
Yeah.
So that's not important to the content of teaching people how to write tests.
Yeah, that's the problem of creating content.
The world moves on and they're not always compatible with what you did.
How about that?
Yeah.
Over at Talk Python, we've got a whole bunch of stuff coming.
And so what I'd like to encourage people to do is just to go to talkpython.fm or training.talkpython.fm.
And right there you can, at the front of the training site, you can just enter your email.
So if you get a bunch of announcements, because I believe we are working on five or six courses right now are under active development.
We've got all sorts of great stuff.
And rather going through it, just sign up to get notified when those come out.
Very cool.
Indeed, indeed.
So a while ago, I went on this Twitter journey, let's say.
I don't know how to really explain it.
It was like I posted a quick question that led to a ton of feedback.
And then, wow, so many pieces of information and ideas and variations were sent around.
What I was trying to say, like, I'd like a dictionary that contains objects that I can access, say, with different keys.
Like, I'd like to put a bunch of users and access them by ID, but maybe also by email or by city.
And that would come with a bunch.
So it was like this sort of exploration there.
And one of the recommendations that came over sort of around that is like, hey, you should check out this thing called GetPy.
Okay.
I don't know where the name comes from, but it doesn't tell a lot about what it is.
But it's a vectorized Python dictionary and set implementation.
And vectorized as in it matches up perfectly with NumPy and so pandas and all the things that are built upon NumPy to plug straight into them.
So imagine I want to have a dictionary that has data in NumPy, but lets me treat it like a regular Python dictionary or a set and things like that.
So that's what this is.
It basically brings a super high performance Python dictionary and set implementation that automatically integrates into the Python scientific ecosystem, which I think is pretty cool.
Yeah.
And it's built upon this thing called parallel hash map.
So parallel hash map is apparently the current state of the art unordered map set with minimal memory overhead and super fast runtime.
So like a C binding.
And so this is just a Python wrapper on top of the C library.
That's a super fast dictionary and set.
So that's pretty awesome.
And here's another one for you, Brian, for your C++ adventures.
The integration between getpy and parallel hash map is this thing called pybind11.
So pybind11 is a compatibility layer between C++11 and Python.
So if you want to write like modern C++ and then plug it in easily to Python, here you go.
Nice.
Yeah.
So that's pretty cool.
Okay.
There's two classes, GP, so getpy, gp.dict and gp.set.
And they're designed to be basically similar to the standard dictionaries and sets from Python.
But there's a few differences.
So check out the docs.
And then I threw in a quick little example here that has two numpy arrays.
And you can say, here are the keys, here are the values, put them together.
And then you can just access different values.
And it's also typed, which I think is kind of interesting, right?
Like it has like an unsigned 8-bit integer or something like that.
Much like numpy is.
But it's more like a Python array where you specify the numerical type than it is just an unbounded list.
Right.
So there's some interesting stuff going on here.
Does it solve your problem that you were looking for?
No, not at all.
But it's very interesting.
But it's still neat.
Yeah.
Yeah, it's still neat.
Speaking of neat, we already talked about black once, but it's pretty neat.
Let's talk about it again.
Yeah, this was sent in by John Hagen.
And he mentioned that, I mean, I'm pretty sure we've mentioned ISORT before.
I know we've mentioned black.
But a lot of projects use both.
So ISORT will sort your imports so you don't have to.
And then just so they're consistent and they're alphabetically sorted.
And then there's some other.
Right.
And there's PEP 8 recommendations about that, I believe, about like grouping stuff that comes from the library, from standard library versus externals and whatnot.
Yeah.
And so you don't have to think about it.
You can just use ISORT.
Black also is becoming more and more popular, I think.
Or it's just constantly very popular.
Reform, it's all of your code, but it includes the import statements as well.
So there was this issue that some projects wanted to use both ISORT and black.
And there is, and they kind of fight with each other right out of the box.
If you run ISORT, you'll have one answer and black will do something different in some cases.
So what do you do?
Well, there's black had a configuration page, and we'll link to this in the show notes, that has documents what settings you have to set for ISORT so that it's compatible with black.
They also do that for FLAKE 8 and PILINT, which is nice.
But FLAKE 8 and PILINT are not that complicated.
It's the ISORT that's, there's like six settings you have to change.
But ISORT 5 just came out.
And ISORT 5 has what they call profiles.
And so if you run ISORT with Profile Black, black is one of the built-in profiles, it will sort the imports such that they're compatible with black.
Nice.
So now if you use ISORT with the black profile and black, also they won't fight with each other anymore.
That's very cool.
Black also came out with some new changes.
So if you're interested in black, check those out.
They're not huge changes, but some minor fixes.
The profile feature is pretty cool.
And even if you don't care about black, I think you should check it out.
They do, apparently, Django, PyCharm, Google, OpenStack, Clone, Adders, and Hug are other profiles that are included.
And also, they're just good examples so that you can look at how different projects are configuring their ISORT.
You can configure yours if you want.
Yeah, I really like it.
That's super cool.
And it is nice to have the top of your files nice and clean and organized in some certain way.
It's interesting.
There's a lot of people do different linters or code reformatters.
But this combination of ISORT plus black is something that's becoming more and more common in a lot of projects.
Yeah.
Very cool.
So I have something interesting for you.
If you were to think of the influence of Microsoft Excel relative to the influence of, like, all of human genetics and the genome science of the world, and they got into a fight, who would win?
Like a superhero.
You seem like apples and oranges.
I have no idea.
Well, they did get into a fight, and Excel won.
So here's the story.
There's an article that was sent over by Chris Moffitt.
He's the guy who wrote the Moving from Excel to Python and Pandas course over at Talk Python.
So we did a bunch of research into, like, all these funny things and weird things around Excel.
So he sent this one over, and I thought I'd cover it because it's fun.
So on The Verge, there's an article that talks about how scientists have renamed human genes to stop Excel from misreading them as dates.
Okay.
Is that crazy?
Yeah.
So you think it was just, like, formatting and stuff like that.
So there's, like, tens of thousands of genes in the human genome, and each gene is given a name and a numerical code, and then you used to talk about it.
So, like, this one controls, like, what color of hair you have or whatever, right?
So over the past year or so, 27 human genes have been renamed all because Excel kept rereading them as symbols.
They're symbols as dates.
Okay.
Well, the examples are important.
Yeah.
So, like, March 1.
No spaces.
March 1 is one of the genes, right?
But it gets converted to 03 slash 01 slash the year, or if you're in Europe, 01 slash 03 slash the year, right?
Either way, that's not what you wanted.
So March 1 actually stands for membrane-associated ring CH-type finger 1, which, of course, is the first of March.
Now, it sounds funny, and, like, we're making fun of, like, ah, whatever.
There was a study in 2016 that examined the genetic data shared alongside 3,600 published papers and found that one-fifth of them, which, what is that?
That's, like, 700 papers were affected by Excel errors.
Oh, man, yeah.
Isn't that messed up?
So, there's a scientific body in charge of naming genes called Hugo Gene Nomenclature Committee.
Who knew?
There's a committee for naming genes, but apparently.
So, they published new guidelines for naming genes, and they said, you have to consider what happens if you type into Excel.
If it gets reformatted, that's not okay.
Got to pick a new name.
So, they put, like, weird suffixes and stuff on these things to make them work, like, March 1 is now, like, March 1 N1 or something like that.
Anyway, so, why do I bring this up on the Python show?
You know, so much of this work that people are doing there can totally be solved by Pandas and Jupyter Notebook and stuff.
And so, you know, the guy who was quoted in there was like, hey, I made these mistakes when I was in grad school because all I knew how to use was Excel.
So, here's a bit of an example along with a plea to help folks who are overusing Excel to take a step forward and use, you know, something like Pandas and Jupyter.
And, you know, you're going to be able to do a lot more cool processing anyway, which I think is great.
Yeah.
Also, one other thing, if you think that geneticists have an exclusive right to these mistakes, there's a really cool article, blog post over on Oracle's blog, actually, called The 10 Costliest Spreadsheet Boo-Boos in History.
And they're really hilarious.
And, like, thank God this didn't happen to me.
Well, and just, like, awesome stock photo that they have for this, too.
Yeah, it's so bad.
Yeah, it's, like, perfect.
It could just come out of Excel.
Yeah.
So, some of them are fairly mundane, but others, like MI5, the British Intelligence Agency, bugged over 1,000 wrong phone numbers because the 000 and the last three digits of the number got, you know, misstated.
Oh, no.
Yeah.
Or Eastman Kodak was forced to reinstate financial results for two quarters by from 2 million and 13 million respectively due to a spreadsheet mistake.
Yeah, there's just all these crazy examples of stuff just going wrong.
Like, oh, sorry, that was a billion dollar mistake because we were off by some random thing.
Anyway, there's a bunch of errors like this.
And it's really interesting to think how you might use the Python tools to not have these errors.
Yeah.
I think Excel has too much influence over the world.
It does.
But if you were going to use Python and you had a Jupyter Notebook, would you run it in your downloads folder?
No.
No, definitely not.
Yeah, so this is another just kind of an interesting and shocking, wherever you think about it before.
But Glyph has written an article called Never Run Python in Your Downloads Folder.
It's not just about that.
Okay, so there's your advice.
Don't run Python in your downloads folder.
But I think it's a good article to spread around and read because it's a nice short tutorial on how syspath works and how it's populated.
So Python has this thing called syspath that is, that's where it looks up.
So if you say import something or mostly that, where it's importing things, also the dash M or something.
If you say dash M and then some module, where does it find that?
And it finds it in lots of places.
One is the normal system include directory or the system packages.
But there's other places too.
And one of the places where Python uses is the location you're in when you run Python.
So a little example, he talks about a lot of people are now using, instead of running pip directly, they run python-m pip, which I also am including that in my advice usually, because I'm tired of trying to fix people's errors when their Python and their pip are pointing to two different places.
That's frustrating.
So what happens if you run python-m pip install something, some wheel in your downloads directory?
Well, normally, everything's fine.
Except if there's a malicious fake pip.py in your downloads directory that happens to have gotten there because somebody wrote some malicious JavaScript code, stick it there.
It's possible.
I don't know if it would happen, but it's possible.
It's more likely now.
Yeah, especially now.
Thanks, Cliff.
But then it's going to run that instead of the pip that you expect it to run, which is bothersome.
So this is hidden.
There's some extra advice in here.
I encourage everybody to read the whole article.
Understanding how the Python path variable works as well, because sometimes other applications will, and if the installer of an application can change things and change your Python path.
It shouldn't, but sometimes they'll do that out of convenience.
So occasionally look at your Python path and make sure that there's nothing weird in there, and maybe contact application developers or if they're doing something odd.
If you are mucking with Python path, the recommendation is put absolute paths.
Don't put anything relative in there.
You want to be able to have complete control over that.
The problem with the downloads folder is not that the downloads folder has weird permissions or higher permissions or anything of that nature.
It's just that web virus maliciousness might, it's most likely to drop the payload there, and the Python path plus that operation or that aspect is what is likely to lead to trouble.
Yeah.
Right.
Like you wouldn't say don't run it in your documents folder unless you copy virus Python files in there.
Yeah.
And the example was real of like sometimes, like let's say you're behind a firewall or something like that, and pip install just doesn't work, and you haven't figured out how to do proxies yet, and you really just need some package.
You might just download the wheel somewhere.
Okay.
You might have that, and the wheel might be fine, but put it somewhere else.
Don't leave it in your downloads directory.
Put it someplace else.
And please use virtual environments.
That'll help things as well.
Yeah.
Absolutely.
Good advice, and definitely another thing to be aware of.
That internet, it's a scary place.
Yes, but we love it.
It's also a good place.
It goes both ways, because we do love it.
That's it for our main topics, but I do want to share two quick things, and they both have to do with the internet.
Number one is we just passed 5 million downloads, Brian.
Wow.
That's awesome.
Yeah, and we're in the top 150 or so of all tech podcasts in the world.
So I just want to say thank you, everyone, that's helped to make that happen, because that's kind of a milestone.
So that's really cool.
Yes, thank you.
Indeed.
Also, I recently finally broke down and got a Wi-Fi mesh router.
Routers?
Can you say singular router?
It's got to be plural, right?
So one of those routers where it has a bunch of different stations, and they all work together.
But because of the way it works, you don't actually have to connect to different ones at different places.
It's just all one Wi-Fi.
Okay.
And man, I got this Linksys Wi-Fi 6 mesh router, which is quite pricey, but man, it is so, so awesome.
So if people are suffering from being home all the time, they're like, gosh, I got to get better internet.
My internet was actually pretty good here, but we started getting a ton of radio interference from different sources, and it would degrade it and stuff.
So I have my recording area office above my garage.
It's like a little studio apartment thing we built above the garage, which is a separate building across from the house.
And over here, if I go speed test, I get 400 megabit measured off my Wi-Fi now.
That is solid.
Okay, so how far away are the mesh nodes?
It's probably 50 feet, but it goes through several outside walls, through a couple inside walls.
There's layers it's got to go through.
But now with the mesh, I could put one of the nodes much closer to the wall that I'm close to that spans that gap between houses.
Okay.
And how are they talking to each other?
They have their own channel, and they do some kind of back channel Wi-Fi 6 thing.
So anyway, super good recommendation.
Yes.
How about you?
Any extra stuff you want to share with the world?
No.
Just that I'm sort of sick of it and fires and all that sort of stuff.
Yeah, man.
It's looking a little smoky outside.
And yeah, the whole West Coast.
And my sympathies go out to the folks in Northern California.
I just had a meeting with someone, and oh my gosh, is it smoky there.
It looked like night in the daytime.
In the background, I could see their window, and it was dark, as if they were in a different time zone.
But they weren't.
Not good.
So hopefully that'll end soon.
But before we end this episode, I got a question, Brian, for you.
Are you a real programmer?
Like a real one?
Like a hardcore?
I mean, you do C++, so that puts you a little closer.
Oh my God.
Imposter syndrome flares every time I get that question.
So I don't think so.
Let's go to XKCD and answer it.
Okay?
Okay.
So XKCD has this cartoon called Real Programmers.
And it starts with two people debating about what kind of editor they're using, right?
It says there's somebody working.
They're using Nano, the editor.
And the person, come on, Nano?
For real?
Real programmers use Emacs.
And then Brian walks in.
Hey, real programmers use Vim.
Come on.
Oh yeah, well, real programmers use Ed.
No, no, real programmers use Cat and just stick it on the end.
Real programmers use a magnetized needle in a steady hand.
Excuse me, but real programmers use butterflies.
Everyone turns and looks at the person.
They open their hands and they let the delicate wings flap once.
The disturbance ripples outward, changing the flow of the eddy currents in the upper atmosphere,
which acts as a lens that deflects incoming cosmic rays, focus them to strike the derived platter
and flip the desired bit.
Nice.
Of course, there's an Emacs command to do that.
Oh yeah?
Yeah.
Command X, Command M, Command Butterfly.
Nice.
Yeah.
Yeah, yeah.
CXCMCM Butterfly.
Indeed.
Damn it, Emacs, they say.
Does anybody use Emacs?
Yeah.
Well, I don't know, but I'm not a real programmer because I don't use butterflies.
I'm sorry.
No, I don't use butterflies.
Okay.
Well, we're in the joke section.
I got to share a terrible joke that my wife came up with.
You wait.
Okay.
99 bottles of beer on the wall.
99 bottles of beer.
Take one down.
Pass it around.
Now everybody's infected.
Morons.
Hey, that sounds like college here in the US.
I think that's what they're doing.
I'm going to tell from all the news I'm watching.
Oh my gosh.
Jeez.
Sorry.
Yeah.
These jokes are the 2020 jokes.
Oh yeah.
So this won't make sense in the future.
Yeah.
So in the future, when you speak to a historian and you say, oh, you're a historian.
What do you study?
Oh, I actually study the year 2020.
Like that won't be enough.
You have to say, well, which part do you study?
Like the beginning?
Do you study the financial crash?
Do you study this crazy election?
Do you study the pandemic?
Like you got to be like 20, that year is not enough to say what your specialty is.
You got to pick one.
Pick a quarter.
Yeah.
It's a crazy time, right?
I study the flyers.
Oh boy.
Anyway.
Well, good talking with you as always.
You as well.
Catch you later.
Thank you for listening to Python Bytes.
Follow the show on Twitter via at Python Bytes.
That's Python Bytes as in B-Y-T-E-S.
And get the full show notes at pythonbytes.fm.
If you have a news item you want featured, just visit pythonbytes.fm and send it our way.
We're always on the lookout for sharing something cool.
On behalf of myself and Brian Okken, this is Michael Kennedy.
Thank you for listening and sharing this podcast with your friends and colleagues.