Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.
This is episode 391, recorded July 9, 2024.
And I am Brian Ock.
And I am Michael Kennedy.
This episode is sponsored by Code Comments, an original podcast from Red Hat.
Listen to their segment later in the show.
You can connect with us on Mastodon, of course.
The links are in the show notes.
We're all on Fosstodon, but you can get to us from any Mastodon.
You can also join us if you're listening to it later.
You can join us live by going to pythonbytes.fm/live, be a part of the audience.
Or if you just want to see the show later, it's all there.
It's usually 10 a.m. Pacific time on Tuesdays, but sometimes it changes.
But if you go to that live thing, it'll tell you when the next one is.
And finally, I'd really like to thank a lot of the people that have subscribed to the mailing list, the newsletter.
If you go to our pythonbytes.fm, there's a newsletter link.
And we'll send you all of the links of the stuff we cover every week.
And we'll just send it to your inbox.
So even if you miss an episode, you won't miss any of the goodness.
So speaking of goodness, what do you got for us, Michael?
Well, you know how we all love PyPI and the ability to go and just pip install a thing and make it, make our apps be so much more.
Right.
Programming in Python becomes like Lego block clicking together, not algorithm class.
Yeah.
You know what I mean?
Which is amazing.
It's amazing.
However, there are issues that you run into if you use third party packages.
Not the least of which is you now probably should have a virtual environment.
You definitely require some pip install commands.
Just stuff all along those lines, right?
Just the management of setup before you can even run your app.
Plus then any potential changes.
If people don't pin their versions, then you're at the whims of what potentially can happen there.
So what some people do is something called vendoring or vendorizing dependencies.
So for example, if I depend on some functionality from requests, I don't know if that's a super good example, but let's say it is.
I could just download the source code of requests, stick it into my code and use it.
It's not probably not a great example because it itself has a bunch of dependencies, but stuff that's kind of like pure Python, no other dependencies.
You could have it as a third party package, or you could just stick the code of that somewhere to your app and refer to it as a relative import within your app.
Right.
Yeah.
Then people just pip install your package, pip x install your package, or even just get a script and run it or a set of scripts.
So there's this project from M Williamson called Python vendorize that I want to talk about.
So it'll vendorize packages from PyPI, as I just described, right?
So it allows you pure Python dependencies to be vendorized.
That is, it'll copy that into your project best use for small pure Python dependencies to avoid version conflicts with other packages, require different versions and so on.
So what you do is you set up a vendorize.toml file and in there, you basically set up, what is your module where, what sub section of your module do you want it to go into what sub module and then what.
PyPI packages you want.
So example here is like a hello package.
So they create a dunder, just not an underscore vendor folder.
And then they say the packages are six.
And once you run it, it'll create that underscore vendor folder and it'll put the six dist info, the dunder and nit pie, the six up high, all the stuff it needs to basically have that there.
So then in your app, you can say from dot underscore vendor import six rather than having an external dependency.
What do you think?
Um, kind of neat.
How do you keep up?
Yeah, I've got questions.
Like how do you keep up with, updates and things like that?
Well, I believe that you just run a command line.
Um, right.
Just run Python vendor eyes in the directory and I don't know if it'll read download it, but it will create it.
So, you know, worst case, delete the folder, run it again.
Yeah.
And then that'll update it.
So the whole point of this is like, you want less change than normal.
Yeah.
You want to like kind of freeze it in a place and things like six that don't really change.
I mean, or have gotten kicked out of the.
standard library.
Yeah, exactly.
A lot of stuff that's really super stable.
Um, and pretty small, right?
You don't want to, because if, if you do something that's got a bunch of dependencies, you've then got to start doing their dependencies and then it gets really wonky.
Right?
Yeah.
For small things.
Yeah, actually.
Um, it's pretty cool.
Um, yeah.
Neat.
Well, that's what I got for you.
The first one, I got a bunch of other stuff, as you can see.
No, I, I like it.
Um, there's, there's projects that I've, open source that it really wasn't intended for somebody to actually use as a dependency.
It's like an exam, some example code that happens to be pip installable, but, but somebody would probably take it and just copy it and run from there and using something like vendorize would work.
Yeah, for sure.
Cool.
Um, I would like to talk about something not as strong as this, but weaker.
Um, some weak, so weak, so some weak references.
Uh, so this is a, article from Martin Heinz, a guide to Python's weak references using weak ref modules.
So weak ref is a, built in standard library module.
And, I actually have never played with it.
And I kind of knew that Python must have weak references, but I just didn't really explore it before now.
And this is a great introduction to just talk about what they are.
So if, I mean, we, the, the term weak reference might be new to somebody that's, that's like, I don't know, maybe new to you.
Um, it's a, it's a term that we talk about in, in like C++ and other things a lot, because of the, using, and I use it a lot in C++, but, using strong references and weak references.
Python also has strong references and weak references.
A strong reference is just sort of a copy of something.
Um, but a weak references is a way to point to something else, but not, not muck up the garbage collection.
So this is a great article.
It talks about, starts out with, talking about, sorry, starts out talking about garbage, dark garbage collection and how, how weak references are used with garbage, how garbage collection and weak references and strong references affect that.
So why do we care?
Well, it is used in things like log, the logging module for instance.
Um, so you have named, this is a cool example cause you have, you have named, logging modules.
Uh, oh, where was it?
This is an example.
Anyway, some named modules that, you, you, you named a logging module or a named logger.
Sorry.
Uh, it's been a rough weekend.
And, and then like, if you want another one of the same name, it might be there.
It might not.
So it's a caching sort of thing that how logging uses it, but there's also ways to use it as a, like for trees.
If you're building a data structure, where you might want, bi-directional, links between objects.
Um, one of the objects, it shouldn't be really hard links in both directions.
So one of those links should be, a weak reference, like the, the, the link between a parent and a child and a tree, structure would be good like that.
Or other things like he talks about, he talked about using an observer, building an observer pattern from the design pattern book, using weak references, just some really cool stuff.
I don't know.
I don't build a lot of data structures.
Like there's enough data structures in Python already.
But if, especially if you're in a, in a CS class or you have, some special needs for data structure, weak references are built in and they might help you a lot.
So.
Yeah.
And they're pretty interesting.
The only chance, only time I've really played with them is for the Python memory course that I created at talk Python to like understand.
Cause you want to look at stuff and see, we did this, it's alive.
We did this.
Now it's garbage collected or now it's reference count deleted.
Um, but if you have a, a pointer to it, then obviously it's never going away.
So weak references allow you to ask questions like that.
Like, I think you can do interesting stuff with caching too.
For example, like if you've got a cache and you've handed out an instance of the object and it's still alive and people are still using it, the parts of the app are still using it.
You can have a weak reference to it.
And if someone else asks for you, like you can upgrade the weak reference to a strong reference, right.
And hand that out again without recreating it.
But if no one's using it, it'll get cleaned up because a weak reference won't keep it around.
So it's like sort of a self-managing cache type of structure could be fun to make too.
Yeah.
But that said, I was thinking just like you, I don't usually make data structures these days.
Python's pretty much got something for you.
Right.
But, you know, people are building, well, there's some third-party library data structures I use too.
And they probably use weak references.
And I just haven't poked into there to find out.
Yeah, exactly.
Let someone else do cool stuff with it for us.
But I love the idea of the, like the logging module that uses named items, doing something like a cache named item thing.
Yeah.
Very cool.
Do you know what else is cool?
Code comments from Red Hat.
Yeah.
This episode is brought to you by Code Comments, an original podcast from Red Hat.
You know, when you're working on a project and you leave behind a small comment in the code, maybe you're hoping to help others learn what isn't clear at first.
Sometimes that code comment tells a story of a challenging journey to the current state of the project.
Code Comments, the podcast, features technologists who've been through tough tech transitions and they share how their teams survived that journey.
The host, Jamie Parker, is a Red Hatter and an experienced engineer.
In each episode, Jamie recounts the stories of technologists from across the industry who've been on a journey implementing new technologies.
I recently listened to an episode about DevOps from the folks at Worldwide Technology.
The hardest challenge turned out to be getting buy-in on the new tech stack rather than using that tech stack directly.
It's a message that we can all relate to.
And I'm sure you can take some hard-won lessons back to your own team.
Give Code Comments a lesson.
Search for Code Comments in your podcast player or just use our link, pythonbytes.fm/code dash comments.
The link is in your podcast player's show notes.
Thank you to Code Comments for supporting the show.
This one is Make Time Speak from Preycin.
Preycin's been on the show before, a friend of the show and also a former co-guest, co-host.
And the idea is it's a little bit of a human-friendly way to refer to time.
You might know about things like, I think it's Arrow, that has a humanized thing that says, you know, five minutes from now or in ten minutes or just now.
Those kinds of things.
But the way this one works is it talks in sort of colloquial way of saying the time.
So you create a clock object and you give it a language to use, like English, German, Swahili, I think, all those things, Dutch.
And then you can ask it, you know, what is 11 colon 15?
It'll say quarter past 11 or a bunch of different times.
What is 729?
And well, it says that in Swahili, which I can't get that.
I'm not going to get that right.
But it'll convert time into spoken expressions in multiple languages.
Super easy to use.
Pure Python.
So you could vendorize it, I guess.
And so on.
Even as plugins.
So super easy to use if people want to check that out and play with it.
This is pretty fun.
Yeah.
I like it.
Very simple.
But if you've got a use case like that, you have a date time and you wanted to say it in a more human version, well, here you go.
All right.
Nice.
Yeah.
I am going to cover a topic that I get asked all the time.
So I talk about testing a lot and machine learning and AI is kind of a big thing now.
So I get questions like, how do I test machine learning projects?
And I got an answer.
I have no idea.
So I'm excited that somebody made an attempt at this.
Here is a article called How Should You Test Your Machine Learning Project?
The Beginner's Guide by Francois Porcher.
So and it's published in the Towards Data Science blog.
Anyway, kind of a cool intro.
talked about some of the simple stuff.
I mean, there is like, how do you test machine learning?
It's complex.
But there are there's a lot of pieces that are pretty straightforward to test.
So cool introduction.
Had a project.
This article also includes a repository that you can play with directly, which is nice.
You just follow along with the code.
So this is doing what is it doing?
It's essentials of testing with a machine learning pipeline focusing on fine tuning BERT for text classification on an IMDB data set.
So that's just what he's using.
He's using pytest and pytest Cove, which are awesome things to start with.
And so it kind of goes through some of the easy stuff right away is starting with some of the simple things like has a clean text function.
So a function within within the source that takes a string and makes it all lowercase and strips it.
But it might do other things, too.
But these are these are great examples.
You've in a lot of machine learning stuff, you've got a lot of little helper functions along the way.
May as well go test those and it'll get in the habit of writing tests, too.
And this in this case is just giving giving some examples of of some random text input and what the clean output should look like.
And these are your expectations of like if I pop pop this data into this function, what should the output look like?
So it's a great way to get started.
I personally would have put this in a parameterized, but I guess we're trying to teach people slowly.
These are really three test cases.
They could be three test functions, but it works.
And so I'm referring to a test function that does a test for capital letter stripping and removing extra spaces.
And what how how it should handle the empty string.
And this is actually a good point.
One of the things they test with interviews a lot is the edge cases for testing.
So like with what test examples are like derivative small cases that you wouldn't possibly think about.
And it's important to test those, too.
Like what is it?
What is an empty string get cleaned as or good?
A good thing.
Like if I already had the word spaces in lowercase, how would that end up showing up in the output?
Things like that.
So good start.
And then jumps up to higher level things.
He talks about a larger chunk of the script.
So he's got a tokenized text, tokenized text function that uses a lot of sub pieces, uses the tokenizer with certain input.
And how you test that?
Well, this is a great example of just figuring out really some examples, some example input and what how you would expect it to be tokenized on the output.
Looking at the length and the shape of the result.
And then, you know, making sure that that not all values or I don't know what this would be.
Oh, he's making sure that all values are torched by torched tensors.
I don't even really know what that means.
But, you know, thinking about what the output should be.
If even if you don't know the specifics, you can have some way to describe how it should sort of look.
And these are good enough tests or they possibly are good tests to have anyway.
So I think this is a good, good starting point to to start a discussion on your team for how to how to add testing to a machine learning project.
Yeah, it's interesting.
I really would have no idea how to test machine learning.
It seems like black box type stuff.
So, yeah, this is a lot more to work with than I would have come up with, I think.
Yeah, just getting started, taking a chunk out of it and then where to go from there.
So after you kind of have a sense of some of the easy stuff, some of the middle level stuff of testing examples and shapes and whatnot.
What's left?
Well, that's where a quick introduction to how code coverage works and looking at what other what what the rest of your code is doing and that maybe you want to add tests to.
Or maybe those are the things that you manually test or something.
So anyway.
Yeah.
Excellent.
Sounds good.
Well, those are our items.
Do you have any extras for us this week?
I am out of extras.
Clean out.
Well, I'll tell where I'll make up for it for you.
Okay.
So wonderful news from Authy.
You know, Authy, the 2FA password thing that you can get for multi-factor authentication.
Super nice because, you know, so many of the devices are locked to or some of the apps are locked to one platform like Google, a Google authenticator.
You lose your phone or you have to reformat it or something.
Sorry.
Good luck.
Now.
You know, there's no sinking things like that.
But with Authy, you have an account.
It syncs it across your different devices.
One device can authenticate another.
If you want to add a new one.
It's really nice except now Authy is urging users to stay alert after 33 million phone numbers were leaked.
How?
Well, there's an authenticated API endpoint, but apparently it would return an error that would indicate whether the phone number that you passed in to try to authenticate with was valid or invalid.
Like, sorry, that phone number doesn't exist or sorry, wrong password.
Something like that.
I think is the deal.
Okay.
And so somebody just hammered it with, you know, every phone number combination they could think of and recorded the results when it said that phone number exists.
And we know that Authy has it.
And then we know that you have 2FA and all of these things.
And so from what I could tell, no real information about people was stolen.
But given that they know you have 2FA and they know that you have, that this is your phone number, they can start sending you all sorts of spoof things, social engineering type things.
Right.
Yeah.
Well, and Authy recently canceled their desktop apps.
You know, Authy being a Twilio, the parent company, canceled their desktop apps.
It just seems like it's really in a kind of a state of disrepair and lack of love and a lack of confidence in Michael at this point.
So I went through the super fun experience of resetting about 30 different 2FA logins.
And boy, oh boy.
I learned some things, Brian.
I've learned that some companies make it super easy to reset.
Because my thought was, look, if this is, you know, what else potentially has happened?
I'm going to revoke all of my 2FA logins and set new secret keys that will generate new passwords.
So even if they were able to get a hold of everything in my account, that stuff doesn't work anymore effectively.
Right.
That was my plan.
And it took like six hours or something, five hours.
You go to different places and you'll see some of them will let you, some have an awesome button.
Reset 2FA here's a QR code you scan.
Boom.
You're good to go.
Others say, your Google authenticators is enabled.
Like what?
I don't have a Google authenticator.
There's like 50 apps that are 2FA apps, T-Mobile and like 10 of the other ones say, use your Google authenticator here.
Like, no, it is not.
It's like, use your internet explorer six here.
Like, no, there are other browsers.
Please don't just say use your Google authenticator.
Right.
But you could just go, yep, this is my Google authenticator.
It's called something else and it doesn't come from Google, but sure enough, I'm going to set this up.
Right.
And like Christopher out in the, in the audience here, that is my next recommendation is well, if not Authy, what?
Because Google authenticators garbage.
Like I said, you, if you, your phone gets messed up, you've lost all logins forever.
There's not a sink.
At least last time I use it, there's no way to sink it or export it or any of that stuff.
Right.
That's bad news.
So Bitwarden.
Bitwarden is awesome.
This is a premium feature.
So you have to have the premium version of Bitwarden, which is $10 a year or 80 cents a month or something.
I'm like, yeah, fine.
That's, that seems fair.
But Bitwarden is cool.
It's open source, multi-platform.
Um, you just scan, scan stuff or enter the code and that they give you for the 2FA and off it goes.
And because it has a browser plugin, you can just click on your name when it says type in your 2FA code.
You don't have to go pull it up.
You just click the button and boom, it auto fills it, which is great.
I don't put it in my one password because I'm just not ready to say my 2FA logins and my passwords are all stored behind one single platform.
Because then your 2FA is kind of toast if somebody breaks into that.
So Bitwarden for 2FA, one password for logins for me at the moment.
What do you think?
Well, I'm using, maybe I shouldn't tell people, but yeah, I'm using Authy.
So are they, they're still supporting it on like, aren't they?
Uh, they're not supporting it on, they used to have a desktop app.
They don't have that anymore.
They have a iPad and an iOS and Android app.
Since you have an Apple Silicon one, you can run the iPad version on your Mac, just like a desktop app.
So it's kind of feels the same, except for it doesn't have like, like the keyboards behave weirdly and stuff.
Right.
Cause it doesn't expect you to have a keyboard.
Maybe you're using it a lot more than I am, but it doesn't bother me to run, run it on my phone.
But yeah.
Well, I mean, it doesn't bother me either, but I've got, there's like a bunch of different apps that I have.
For example, the credit card front end system for talk Python courses.
It has a remember me button.
It never remembers me.
Never.
It has a 2FA.
It never remembers the 2FA.
So even if I say, remember me 20 minutes later, I'm putting in the password and the 2FA.
And then 20 minutes later, I'm putting the password in the job.
I'm like, ah, so there's like a few places like that.
They're just constantly asked for the 2FA.
Um, digital issues a little bit like that.
Like if, you, every single time you're putting in the 2FA, there's not like a trust this device sort of thing.
And so I ended up, I'm probably in used to be in offing now in bit warden.
I'm probably not like five times a day, at least every day.
So anyway, so one final thing before I move off of this, after all of this, they said, I don't know how this helps.
It doesn't seem like it should help, but somehow they said, as part of our recommendation to users, it's very important that you upgrade to the latest version of Authy.
Why that because the endpoint hands, I don't understand, but, anyway, it says you must.
And then if you go and look at the upgrade, all that says for the, it says you must get version 26.1.0.
What does it say here?
Bug fixes, not, this is an important security update and you need to update because we're trying to protect your privacy.
They're hiding behind bug fixes and it's disgraceful, right?
This is bad.
So all these things taken together, I'm like, you know what?
It may be safe.
It may be not, but I'm out.
Like this is not where my, my important things live.
So.
Okay.
Yeah.
And OFAC also out there says, Hey, if you're okay with recommending paid services, which I am one password is what I migrated away to from Authy.
Yeah.
One password is awesome.
But like I said, I have my logins at one password.
So I put my two FA and bit word.
All right.
Woo.
That was a long extra.
That should just been a thing, right?
Extra.
Remember a while ago, I did this article unsolicited advice for Mozilla and Firefox, like your AI, your, your good citizen AI project.
Probably won't save Mozilla.
It probably needs more than that.
And it won't really probably help Firefox either.
So let's do some things that help them because I'm in principle, a fan of them.
Well, I said, one of the things that my main recommendation was like a privacy focused Google docs.
Well, they didn't do it, but, proton, the Proton Mail people just came up with a, like a Google docs equivalent, but with end to end encryption.
And no AI training and no AI training and, you know, all the kinds of things you would like about your data without the negatives.
So if you have proton, there's, there's now collaborative docs with it, which is kind of cool.
And it looks pretty, I think it looks pretty nice.
So just want to give that a shout out.
Does it, do you know if, I probably don't want to ask, does our Google docs open to scanning for AI?
Do you know, I believe that the, the freeware versions are the free versions, but the business ones, maybe not, you know, I think your business workspace stuff is not open to that, but like your, your personal Gmail is open to like scanning for ads and stuff.
Whereas the business one is that's, so that's the price you pay.
Last thing, is the code in a castle thing I'm doing in October 5th.
The early bird discount closes tomorrow.
If you listen to this, right.
When it comes out, so if you're interested, please check it out.
Uh, it'd be super awesome to spend a week hanging out in Tuscany, doing all sorts of things together.
And yeah, that's, that's, it's for my extras, Brian.
That's a lot of extras, Tony.
Uh, I mean, Michael.
Yeah.
All right.
Shall we have a joke?
Yeah.
This joke I called, I lied.
And it's a, it's like a cartoon.
It's a woman behind a esoteric programming sort of thing.
She's got a gun.
It says, I lied.
I don't have Netflix, take off your shoes.
We're going to learn rust.
And I just thought about like this, all this like rust energy.
Like we're converting that to the rest.
We're rewriting that in rust.
It's just like, you're doing rust.
That's what the world's doing.
Sit down.
So she invited someone over to like watch Netflix.
Like I lied.
I don't know Netflix.
Take off your shoes.
We're going to learn rust.
I thought it would catch the zeitgeist.
Well, right.
It's bizarre, man.
It's amazing.
Okay.
And I'll, I'll, I'll maybe I'll put that is the, the chapter art for this one.
Cause the picture, the eyes are amazing.
The desperation.
That's pretty good.
Yeah.
I've resisted.
I've resisted rust so far.
I mean, I'm, I'm, I'm happy that things are getting faster and whatnot, but I haven't
learned it yet.
We'll see.
Same.
All right.
Well, cool.
Nice episode.
Thanks for joining today.
You bet.
Fun as always.