Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.
This is episode 327, recorded March 13th, 2023, and I am Brian Okken.
And I am Michael Kennedy.
This week's episode is sponsored by Compiler Podcast from Red Hat.
Listen to their spot later in the show.
And connect with the show on Fosstodon at pythonbytes at fostadon.org.
And both Brian and Michael are there also, Brian Okken and M. Kennedy.
You can also join us on YouTube or join us live by going to pythonbytes.fm/live to be a part of the audience.
It's really kind of fun.
Usually it's Tuesdays at 11.
This week it's Monday, but usually it's Tuesdays at 11.
And you can watch older videos on the YouTubes as well.
So thanks, Michael, for showing up again this week.
We've got quite a few episodes under our belt.
So are you excited to get started?
We do.
I am.
You know, technology can be a tangled mess sometimes.
And not long ago we spoke about Untangle.
And then, I believe it was Ian, sent in and said, you know, that was really cool.
Yeah, it was Ian.
Thank you.
Said, I know you're a huge fan of Pydantic.
It's true.
And maybe you want to check out something that is similar to Untangle, which would let you talk to XML through Python in an object-oriented style way.
A little more dynamic.
So he sent in Pydantic, the Pydantic-XML extension.
Have you heard of this, Brian?
No.
No, I hadn't either.
It's totally news to me.
But the idea is basically, you know, the way Pydantic traditionally works is you point it at a JSON file or a Python dictionary.
And it can create an object graph hierarchy of all the pieces that it knows.
So you can say it has a name and a number, but then also has a list of locations and the locations model of these Pydantic objects and so on.
And that's how Pydantic has worked from day one, more or less.
It's based on dictionaries because that's the way that you speak APIs, right?
And so it was very closely tied to APIs and JSON exchange there.
So this one does basically the same thing, but for XML.
And it's glorious, right?
It's glorious with the data validation, the required versus optional, the type conversion, all of those things.
It supports dictionaries, listsets, tuples, unions.
It has LXML parser support for high-speed parser processing.
You can pass in an element tree as well, which is the XML-etree-element tree class, which allows you to do parsing traditionally.
So how do you get going?
Well, you create a class here with pure Pydantic.
You derive from base model.
Here you derive from base XML model.
So it's slightly different, but, you know, it's fine.
And check this out.
In the XML document that they're talking about here, there's a top-level node has a thing called status, an attribute called status in XML.
Okay.
And, yeah, or in the product it does, part of it anyway.
It has two possible valid values.
It can either be running or in development.
It can't be ran or prod or any.
It has to be those two words.
So because it's Pydantic, you can just say the type of this is a literal running comma development.
Isn't that awesome?
And that's it.
That is pretty good.
You're done validating that that is correct.
And you set that equal to an adder, which means it's not coming from the body of the XML node, but it's coming from this attribute name status down here.
So cool, huh?
Yeah.
Then you can have launched.
You could have launched, which is a numerical date.
So the running ones have 2023 and 2019 as launched, but the one that's in development, well, it doesn't have a launch date, so it's missing.
So the optional aspect of Pydantic is at play here.
And then there's a title for that element.
And that just comes, you say it's string, it just comes straight out of the body of the node because it's not set to an attribute, but it's just the base one.
I guess, presumably, you can only have one of those per node.
Okay.
So is title special or can you name it whatever you want then?
You can name it whatever you want, I'm pretty sure.
Okay.
Yeah.
Oh, yeah.
It says extracted from the element text.
Nice.
Okay.
Yeah, yeah, exactly.
And so then the overall XML document, I had it reversed when I first started talking about this.
There's a company and the company has products, right?
So there's a company class.
It has a trade name from its attribute, SpaceX in this case.
And then it has a node, which has a website as its text value.
But the text value is HTTPSSpaceX.com, right?
And so you can say the type is a URL and it'll actually parse it out as a URL, not just a string, which is really cool.
Okay.
And then in standard Pydantic style, it has a list of products and you give it the tag name that it's product.
The node name is product and it just loops through that list.
Isn't that a clever way to parse that with validation and data conversion and all that?
Not only that, I'm really glad you walked me through it because the first time I looked at this, I was a little bit lost on how to think about this and how it's building it up from different components and attributes and elements.
It's pretty neat.
Yeah.
If I've got to do XML again, I'm all over this.
So there's a bunch of stuff about how you talk about heterogeneous collections, aliases, union types, model.
You can go through it if you want.
But I think this little quick getting started bit they have right at the top of the website that I'm linking to.
That's pretty good.
Yeah.
Nice.
Cool.
Very good.
Yeah.
Anyway, that one's a great one.
Thanks, Ian, for sending that in.
I'm psyched to know about it.
Well, next, I kind of want to talk about virtual environments.
So I use the virtual environment, VENV, built into Python.
I, in the past, have used the virtualenv extra package that you can install.
But since, I don't know, it's been quite a few versions of Python, the built-in one's pretty darn good.
So I'm happy with it.
Anyway, there's a lot of people that kind of don't really get how they work.
Trying to get people on board with that they should use them is great, but trying to use them effectively.
Like, one of the mistakes I've seen a lot of people make with virtual environments is using them, but then when they go to test in CI, actually trying to activate the virtual environment.
And you don't really have to.
You can just use the binaries directly.
And so I'm really happy this article is around.
So Brett Cannon wrote an article called How Virtual Environments Work.
And this is excellent.
And it's a short read.
So one of the things that starts with a little history, not a lot of history, just a little to remind people that back in the day,
we had global and the working directory or your current directory, and that's it.
It wasn't anything else.
And I kind of remember this of trying to find if I'm sharing some code, trying to find some on the web,
and then just downloading it and sticking in my directory and see if it works.
It's just part of your code now.
That's not what we have today.
And partly in thanks to virtual environments.
So it's better now.
You can still complain about them.
That's fine.
But it's better now.
And then he goes on to talk about the structure.
And there's really not much there.
I mean, when you're building a virtual environment, it's kind of a lightweight throwaway thing.
Don't think of it as this huge thing.
It's just a little directory.
And it's got a bin and an include and a site packages directory for the Python that you're using.
And on Windows, it's a little different.
But we'll just hand wave around that.
In the Unix environment, it's mostly symbolic links to, I mean, you do have stuff installed there.
But as far as replicating the Python environment, it's your Python interpreter isn't copied in there.
It's symbolic linked.
So you don't have to worry about that too much.
It's the site packages and the bin and everything and how that's there.
So how does Python deal with that?
Well, it deals with it through a pyvenv.cfg.
It's a config file that tells Python when you run Python from this virtual environment where the home directory should be, where the system, whether or not to include system packages in the site packages.
And then the version and the executable and some other stuff like the command, if you wanted to recreate it.
I don't know why that's there.
But in general, this is enough to tell Python if you just run it from that environment that you just get all the right stuff.
And so if you're putting it in a script, just use those.
But if you're using it from the shell, then, of course, you're going to activate the shell.
But the activation, he's stressing, and this is important to understand, it's optional.
You don't have to hit activate as long as you're calling stuff within the environment.
And it kind of goes on to talk about really what it's doing.
What does the activation do, though, if you're curious?
It doesn't do much.
It sticks some stuff in your path.
And it's like a virtual environment, sets a virtual environment variable.
And it registers a deactivate shell function.
And that's about it.
It changes your prompt, too, to let you know that you've activated it, which is cool.
And then he goes on to talk about how partly why he's dug into this lately is because for VS Code, they're creating a little tiny, but you can use it anywhere you want, a extra extension called microvenv.
Microvenv.
I don't know.
So this is a single file, less than 100 lines to kind of emulate all of that.
And the reason is because Debian doesn't include virtual environments by default.
So they kind of have to want to work around that.
So anyway, really great summary of virtual environments.
Yeah.
I'm peeling away a little bit of the magic to let you know what's happening in there, right?
Yeah.
Well, and also because it's sort of magical to some people, a lot of people are concerned about like trying to copy it or something.
And you shouldn't have anything kept that's important within your virtual environment.
You should be able to recreate it whenever you want.
So there should be lightweight things.
Oh, the one thing I really wanted to highlight, the reason why I really wanted to talk about this was because of a flag.
So where's that flag?
There's a flag, dash, dash.
I've got to find it.
Do you know what I'm talking about?
Anyway.
Okay.
I'll help you search.
There's like no, there's like, like, yeah, without pip.
Dash, dash, without dash pip.
Okay.
So without pip, excellent thing to know about because, oh, here it is.
BNV without pip.
That will get it.
So that doesn't ask you, ask you to upgrade pip.
So especially in CI and other places, you don't, you don't care about upgrading it right now.
I mean, I get it if I'm in the development mode.
I do want to upgrade it.
I want to use the latest one.
But in a CI environment or a lot of automated places, I don't need to do that.
I can just use whatever's there.
It's going to be fine.
So turning that off is awesome.
And it saves some time.
It's not just, it doesn't say not, it isn't really not install pip or upgrade.
It just doesn't try.
So it assumes pip's already there is all.
It uses the system pip.
So that's it.
Cool.
That's cool.
Yeah, it just falls back to the global one, but runs it for that environment.
Yeah.
And apparently it saves a lot of time of that.
So that's great.
Very cool.
Well, before we move on, our sponsor.
Oh, yeah.
Let's cover our sponsor.
And I really, really appreciate Red Hat and the compiler podcast for sponsoring this episode.
So just like you, both Michael and I are big fans of podcasts and really happy to share a new one from a highly respected open source company.
Compiler is an original podcast from Red Hat.
Compiler brings together a curious team of Red Haters to simplify tech topics and provide insight for a new generation of IT professionals.
The show covers topics like what are the components of a software stack?
Are big mistakes that big of a deal?
And do you have to know how to code to get started in open source?
Compiler closes the gap between those who are new to technology and those behind the inventions and services shaping our world.
They bring together stories and perspectives from the industry and simplifies its language, culture, and movements in a way that's fun, informative, and guilt-free.
I recently listened to an episode titled Testing PDFs and Donkeys.
It was great.
It's part of a Stack Unstuck series.
It's a great series, and it talks about the entire tech stack, software tech stack, especially around web stuff, starting with the great stack debate.
There's episodes on front-end frameworks, fundamentals, databases, and OS.
Even OS is in system calls, and then it even talks about testing.
Even though testing really isn't part of what you think of as the tech stack, it's kind of part of all of it.
So I'm glad they covered it, especially for people either jumping into software or software old hats like me trying on new hats like embedded systems or control systems, people learning how to do web applications.
These are great overview episodes, and they're going to be a great episode.
They're timed well.
They're timed how they need to be.
Some of them are 45 minutes.
Some of them are 25, and I like that flexibility.
Learn more about the compiler at pythonbytes.fm.compiler.
The link is in your podcast player show notes, and thank you to Compiler for keeping this podcast going.
Thank you, Red Hat.
Thank you, Compiler.
Good show.
Check it out.
All right.
On to the next one, Brian.
Okay.
This one is a project by Raid.
And if you've worked with databases in Python, especially if you're using an ORM like SQLAlchemy, SQL Model, Kiwi, any of these things, what's really nice about those is you create classes in Python.
And then through some sort of magic, somehow there's a startup thing that makes sure the database exists, that the database has tables that map over the classes.
So, for example, if I create a class and say it's these three columns, and here's an index, and this one must be unique, it'll talk to the database and make that happen.
But for the rest of database management, you've got to go and write stuff in SQL or DDL, data definition language, or whatever that is, right?
The stuff where you create the tables and create those types of scripts, create users.
So, Raid created DB declare, a declarative layer for your database that adds on top of those types of things, like SQLAlchemy is what I described, for that kind of work.
So, it's a pretty new project.
People can check it out.
The idea is, let me find a quick example here.
So, what you can do is you can come and say, I want to create a database, and it's got this name, and I want to create a role.
And here they have the name of the role as a hungry user.
They have to log in.
Here's their password.
They get privileges on this certain database.
And you can model out those types of things.
And then on top of that, you can just use SQLAlchemy itself as part of this process.
You create a SQLAlchemy engine, and you call run on that.
And you can also, it'll create the SQLAlchemy models.
There's an example a little bit further, I'll also link to this, that shows how to do basically the standard SQLAlchemy stuff that will create the tables with the primary keys and so on.
So, this one's just a short one.
But if you like the way that SQLAlchemy works, also with SQLAlchemy, you get migrations.
Or with SQL model through Alembic.
The idea is that this is going to be extended in the future as well to have some of those type of transformational behaviors.
But for now, it's really the extra stuff like table creation, database creation, user roles, and management.
So, pretty cool.
People can check that out if they find that useful.
Want to stay more in Python and less in SQL scripts.
Well, especially the roles and permissions, having that covered by that, that's pretty cool.
That's a piece that always trips me up.
You don't just have the root user, just have read access, full access, just run as admin.
Just kidding.
Yeah, exactly.
It always seems like it's covered as an advanced topic, but it's almost the first thing you need to figure out.
Yeah, exactly.
How to separate user roles.
Right.
Are we going to put this on the internet and just let people have at it, or are we going to put a little data protection in there?
Anyway, people can check that out.
It's a good one.
Next, let's talk about Nox.
So, I use both Tox and Nox on various projects.
A lot of my open source stuff is Tox-based for testing, just because I'm used to it.
But I'm starting to use Nox more and more.
And I want to cover this article by Seth Larson called Testing Multiple Python Versions with Nox and PyEnv, P-Y-E-N-V.
Now, I personally don't use PyEnv, but I have before.
And one of the things that's tripped me up before is how to use it with Tox and Nox.
So, basically, even – so, check this out if you want to – check this article out if you want to learn more about Nox.
But also, the trick about – even if you're a Tox user, the trick about how to use PyEnv with it, with the global.
There's an example here.
It's awesome.
So, let's go over this a little bit.
So, if you want – one of the first things I want to try to do with Nox, when it wasn't obvious to me from the documentation,
is just how do I set it up like I would Tox to just test multiple – my stuff with multiple Python versions?
And that's the example that it shows right off the bat.
You have a Nox file.
It's NoxFile.py.
And it's Python code.
So, you import Nox, and you can set up a session for multiple Python versions.
And then, within this defining test, this can be anything.
So, the function names around a session are – they're just – they're what you'll use later.
So, we'll cover that in a little bit.
But then, within your session, you do stuff.
You either install or run.
There's probably other stuff, too.
But this is what I use, is install and run.
So, installing . means installing the current project that you're working on.
And then, there's an example here for requirements files.
So, a dev requirements.
But if you're using pyproject.toml, it could be also part of your .install, if you want.
And then, run test.
And, of course, run pytest.
So, good job, Seth.
And then, it goes through how to run it.
So, you can either just type Nox, and it'll run everything.
Or you can say Nox-S for –session to run test.
And if you want to run a specific one, like just 3.11, Python 3.11, you can say test-3.11.
I kind of like that there's a dot in there.
It's pretty easy to understand.
So, I just really like how simple this is to get the basics down.
The basics of, I want to run tests on my project on over multiple Python versions.
And this is pretty clean.
This is already a decent argument to switch to Nox if you're on the fence between Tox and Nox.
I agree.
It's so nice because not only is it clean, you get autocomplete support from your editor.
You get whatever editor you're using.
It'll tell you if you've done something wrong.
There's more support than just, well, here's an arbitrary text file I'm typing stuff into.
I hope it works.
Yeah.
And then, you can – I mean, I've used it also to, just like I do, is Tox with doing something
like adding linting and coverage checks and all sorts of stuff.
I do want to – actually, that's one of the things.
I'm glad you wrote this because it's a reminder.
I did want to write a – like an example of the workflow differences between using Tox and Nox
and showing a side-by-side comparison of those two.
So hopefully in the future I can get that written.
But one of the things that gets me is that with the run command, you have to separate every little piece of your command by –
they have to be quoted as quoted strings.
Like pytest tests has to be two different parameters to the run argument.
And if you have a bunch of flags, each of the flag needs to be different things.
Now, this – most – some people might not care about this.
I kind of care, and it bugs me because I don't have to do that with Tox.
So what I do is I just – since it's Python, I just write a string with all of the things that I want in it,
and then I use split to create a –
Split on space, something like that.
Yeah.
I just use split on space to create an array with all of the elements.
And then when I run – I pass it to run and do the star thing so that it explodes it and passes it in altogether.
Yeah.
Nice.
And then here's the trick – the magic trick about PyEnv at the bottom is that if you say PyEnv global
and list all of the environments that you want to have available, it makes it available.
If you're using PyEnv, it isn't by default.
So you have to run this for each session or shell invocation to get it to work for PyEnv people.
Cool.
But that trick works with Tox also.
And the other thing I wanted to mention was one of the things I really like about Nox is that if you –
like this example has PyPy3 and 3.8, 9, 10, 11, 12 all there.
By default, Nox will not fail if you don't have one of these around.
So if you only have like 3.11 installed, it'll just run that and it'll skip the others.
You can make it fail if it doesn't have it, but you don't.
By default, it just skips them, which is kind of cool.
The Tox is the reverse.
Tox is if by default, it'll fail if it's not there, but you can tell it to skip if it's not there.
Yeah, that's cool.
One really quick thing if people are copy and paste from that example, I'm pretty sure the –r dev requirements needs a –r space dev requirements in there.
Just people are copy-pasting, right?
Because that's the command is install, pip install –r space filename.
This always surprised me.
I've seen it in multiple tutorials.
I don't know if that's true.
I think you might be able to do my way without the space.
I don't know.
Okay, well, you may be able to.
We could try it.
Yeah, we could try it later.
Yeah.
All right.
Quick question for you, Brian, because I don't know the answer.
Damian asked, does someone know if, how, Nox or Tox work with poetry?
Do you know?
Nothing works with poetry.
I don't know, actually.
So, poetry, I'm sure there's – I don't know.
Probably, you talked about the PyProject Toml integration.
So, I mean, it probably is more or less –
It probably works.
Probably.
Yeah, it probably works.
I'm sure many people listening know.
Sorry, Damian.
I don't know.
Either.
All right.
Hold that thought.
Yeah, that's all of our things, isn't it, Brian?
Do the search.
Yeah.
We were quick.
Do you have any extras for us?
Oh, I always got extras.
So, let's go through here.
Remember when we talked about how much drama there was around – how was it?
Google, maybe.
Someone was giving away, like, two or 4,000 YubiKeys to the top 100 or top 1,000 maintainers on PyPI – maintainers of the top projects on PyPI.
And that was because there was going to be a requirement for PyPI that the very top 1% or some small percent was required to have 2FA.
Yeah.
Well, guess what?
If that caused drama, wait until you hear about this.
GitHub makes 2FA mandatory next week for anyone who is an active developer.
So, basically, like, if you're making contributions to projects, public projects, I believe, something like that.
So, yeah.
Okay.
Security accounts of more than 100 million users.
I'm not sure exactly what the definition of an active developer versus an active contributor because I might contribute to the code without writing any actual software.
But whatever.
It's splitting hairs.
The only reason I really bring this up is not to, like, go into depth.
That's why this is an extra.
But if it was a big deal that, you know, 1,000 Python developers had to do 2FA and it sounded like it was, what about 100 million?
It's going to cost some drama.
And then how many of those people who are contributing to PyPI are doing so in some way or another through GitHub?
I would say the majority, probably.
Yeah.
Actually, so I don't think it's going to be drama.
Hopefully, people are just cool with it.
I think the mess up with PyPI was the dongle thing.
I think people thought they had to have the hardware thing.
And they don't.
I mean, I use a software 2FA system.
And it's not just them.
I don't know about you.
But I just looked.
I got, like, half a dozen, dozen different things I got to log into with Authy.
Yeah.
I think I have about 30 accounts or so that are 2FA.
Yeah.
And I'm happy that I do.
That is not a complaint.
I mean, that's not me whining.
That's me, like, going, yes.
Occasionally, I'm annoyed by it.
Sometimes I'm lying.
Yes.
Like, right now, I went to look at a GitHub thing, and I have to log in, so I don't have
time to do that right now.
So, occasionally, it's annoying.
Well, here, let me tell you why this is annoying so often.
I'm going to take this and make it a whole episode, aren't I?
So, the reason it is annoying is there's so many places.
Like, what is the point of the 2FA?
The 2FA is if somebody steals your account login information through some kind of data breach
or through password reuse or whatever, that someone else can't go and use those credentials
to log in as you.
They have to have the second factor.
Well, here's why it's annoying.
Every time I log into my credit card processor, I think almost every time I log into DigitalOcean,
it's like, hey, how are you doing?
What's your 2FA factor?
It's like, you know, I've given that to you about 100 times in the same browser, right?
It should, at some point, go, you know what?
They've given us the 2FA.
We trust them.
It's not, I'm not concerned someone is on my computer logging into my thing.
I'm concerned about the seven other billion people who might want to log in from somewhere
else, right?
So, I think there should be a little bit of like, hey, if you've already logged in on this
device, maybe you don't need the 2FA every time.
You could even refresh it monthly, but not four times this morning, right?
That's when I'm like, ah, 2FA.
It's driving me nuts.
So, that's my right.
Yeah, but I mean, to be fair, GitHub doesn't do that, or PyPI.
No, GitHub is great.
GitHub, I have no complaints.
Okay, because I don't have to do it every time for GitHub.
No, no, GitHub is really good.
Yeah, and I've been using 2FA for GitHub for quite a long time.
It's been optional for a long time.
Yeah.
Or a while, at least.
I have a short, very short memory.
I'm really good with open source because I have the same memory span as the general technology
memory of open source.
Nice.
All right.
I have one other really quick thing.
You know, Brian, we always have good luck reaching out to our listeners about things.
And this one is a little bit different.
So, I recently got a brand new adventure motorcycle as of last week, which is awesome.
And I found some fun places to take it and ride.
Like, I rode up into the snow around here and the coastal range and stuff like that.
I'm looking for somewhere fun in the Northwest to go riding that's like not intense motocross
off-road.
But, you know, it would be a lot of fun for these types of things.
Just a nice view.
Yeah.
Just get out and get out in the woods and cruise around this summer, this spring.
And so, listeners out there who know where to ride around here, that's not one of the
couple huge off-road vehicle like state-sponsored areas around Portland.
People got it.
Shoot it in.
And if you want to know why I kind of got this bike, how much fun it was, there's a cool
video I linked to with Ben Townley and another guy, something Raymond.
I can't remember his first name.
Anyway, you can check that out.
And, yeah, that's all I got for my extras.
How about you?
I mean, while we're asking for, like, contributors, I just, we just passed.
We were driving around this weekend and saw, like, a group of motorcy, like, 10, 20 people
riding motorcycles.
And since we've got a couple of Harley places around here, so there are, like, Harley groups
around.
But when I was a kid, I was, like, scared of these people.
And these people.
Just people with motorcycles.
Mostly wearing black leather.
But now they're, like, I mean, it's mostly people my age or older.
You know, it's 50 to 70-year-olds riding motorbikes just to hang out with your friends.
Well, that's cool.
I mean, at least that's what I see.
But I think it'd be cool if I could see, like,
are there, like, e-bike gangs?
Or, like, are there e-bike groups of people, like, just a bunch of e-bikes riding together?
Anyway.
I bet there are.
E-bikes are awesome.
Like, electric bicycles.
They're so cool.
And I'm sure there are, actually.
But how are you going to find them, right?
That'd be cool to see a picture of, like, a bunch of them.
Yeah.
Anyway.
Do you have to get one of those club patches for it?
Can we get it?
Probably.
Do you have any extras before?
No, I don't have any extras.
I was just BSing.
So let's do a joke.
Yeah, awesome.
All right.
Let's do a joke.
And, boy, I didn't do this.
I didn't plan this.
But, boy, did it line up good.
So this one comes to us from ProgrammingHumor on Reddit.
And just check out this picture, Brian, here really quick.
Describe the picture to folks.
You see this?
I don't know what I'm saying.
So it's some sort of logging into GitHub.
Okay.
And somebody's got to do a code review here on the end of the morning.
Oh, there's over a million lines changed and 20 deletions.
1,094,000 lines changed, 20 removed.
So not too bad there.
But 2,945 files to review.
And it's zero of those.
It's like, let's get started.
So the title is, anyone else have this kind of colleague?
What a way to start a Monday.
So any guesses what they did?
I'm guessing they, like, applied black to their project.
And it just changed everything.
Yeah, maybe.
The comments section is pretty good, too.
Someone else, like, suggested that maybe the git commit message is fixed typo.
Fixed typo.
Or something like that.
Added stuff.
Small update.
Yeah.
That's funny.
Reformatted every line of gun.
Exactly.
So it's a case of the Mondays.
One of the best shows ever.
Replace all spaces with tabs.
Exactly.
Nice.
So anyway, cool.
All right.
Well, that's what I got for you.
For this wonderful episode.
I had a lot of fun.
Hope everybody else did, too.
Yeah, I did, too.
And thanks, everyone, for listening.
See y'all later.