Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.
This is episode 339, recorded June 6th, 2023.
Is it the 6th? Yeah.
I am Brian Okken.
I'm Michael Kennedy.
Today's episode is sponsored by InfluxDB from Influx Data.
Thank you, and we'll talk about them more later in the show.
If you want to reach any of us or the show, we have a contact form, of course.
And then also M. Kennedy at Fostadon and Brian Okken and Python Bytes, all Fostadon locations.
If you're listening to us on a recording or on YouTube or on a podcast player,
please join us on YouTube at pythonbytes.fm/live, at least occasionally,
because it's fun to have people hanging around while we're recording.
It's usually Tuesdays at 11, and you can watch older versions there, too.
So let's kick it off with something stacky.
Something stacky.
You feeling like some pancakes?
A stack of pancakes?
Yeah.
How about a PyStack?
So the reason I was late to this recording, Brian, was I was just talking with Pablo and Matt,
maintainers and creators of PyStack.
Have you heard of PyStack?
I have not.
So PyStack is a tool that uses forbidden magic to let people inspect the stack frames of a running Python process,
or even a core dump that was captured from a Python process that crashed,
helping you quickly and easily learn what it's doing.
How cool is that?
Pretty good.
Yeah.
So here's the deal.
Yeah.
So here's the deal.
I've got a Python app.
This is especially important if you have mixed code.
So if you're talking with C, C++, Rust, those kinds of things, because it will cross those boundaries as well.
But let's just say pure Python even.
I've got a Python web app, and I go to the server, and I try to connect to it.
It won't really respond.
It connects, but it just hangs.
Go to the server.
It's not 100% CPU.
In fact, it's 0% CPU usage, so it's not spinning and busy.
What the heck is it doing?
Is it a deadlock?
Is it waiting on the database?
What is going on?
So what you can do, even in production, you can go up to that process, and you can say,
give me a snapshot of exactly what this process is doing.
And what you see is you see a call stack.
Let me find an example here of what it looks like.
It looks like this.
Silly zoom.
So what you'll see is like, hey, on this particular thread, we're seeing on this file,
on this line, this function was called.
And check it out.
It even has the arguments passed to the function.
Oh, wow.
That's nice.
Yeah.
And then you can see what function that's calling with the arguments passed to it, and
what function that's calling with the arguments passed to it.
You can do this on a running function without altering its behavior, basically.
It doesn't inject any code or anything.
The only behavior it alters is that it freezes it for a second, potentially, which could,
I guess, make something time out.
But other than that, you could do this in production even to see what's happening.
And what's extra cool is if even if the process crashes, you can grab the core dump and it will
go back and analyze that as if it was a running process.
Oh, that's pretty cool.
Yeah.
So there are a ton of features.
If you can get to the section where it says, what can PyStack do?
So it works with both running processes and core dump files.
It'll tell you if a thread is currently holding onto the gill, if it's waiting to acquire it or
it's trying to drop it.
So you can, you know, one of the examples that Matt and Pablo spoke about was they were calling
into custom C code.
That was a Python extension.
That call that was coming in, that was, that was acquiring the GIL.
But then there in the destructor for some object that it was waiting for it to go away, it
was like waiting on a background thread to do some cleanup.
That background thread also was trying to do a callback to let Python know what's happening
and was trying to acquire the gill, but it couldn't.
So because the one that was waiting on it was already holding the GIL and wasn't going to
give it up because that's how the GIL works.
Right.
So you can use it for like these deadlock situations and see if it's running a GC, you
can see both the call stack in intertwined for both Python and C or C++ or Rust altogether.
And it'll even do things like go out and find the debugging symbols for say your Python runtime.
Even if you don't have it, it can potentially go and get those and bring that extra information
in.
What else should we see here?
Safe to use on running processes.
You can run it on a process in memory running process without pausing it at all, which will
minimize the impact it might have.
But it'll also potentially have like not 100% precise information.
It could be out of sync.
So yeah, it even works on like corrupted process core dumps because the process died because it
got corrupted memory or something.
So if you've thought about GDB or some of these other types of things because you're like,
oh my gosh, I've got to figure out why this crashed.
Here's a core dump.
Let me start looking at it.
Well, PyStack may be the thing you want.
That's pretty cool.
And one final bonus for you, Brian.
Suppose you have a pytest test.
This one.
Suppose you have a pytest test and that test while it's running deadlocks or is very slow or
something like that, you can have PyStack as a pytest plugin.
And then when you run your code, how do you do it?
I think it's you just, where is it?
You know, but anyway, when you run it, you can say basically analyze my test.
And here's a certain threshold to consider a failure and take a snapshot of that and so on.
Yeah.
I like the threshold notion of just like, if it gets this bad, tell me why.
Yeah.
Nice.
Cool.
So if people want the full details, I suppose they could go check out the YouTube live stream
channel for Talk Python now or in three weeks, they could listen to the podcast.
But super, super cool tool.
If you've got a process that is crashing, that is hanging, maybe it's doing this in production
and it only gets deadlocked after, you know, 12 hours of being hammered on.
You can't easily just debug it locally and get this to happen.
Or if it's completely crashed and you have a core dump, this, these guys are doing lots
of magic to make it possible.
Nice.
Yep.
Cool.
Well, next, I kind of want to give everybody some news.
So last year, actually, it was in July last year.
So we talked about in episode 293, we talked about some giveaway, some PSF, the PSF saying
that there's like the top 1% of the critical packages are going to have to use two-factor
authentication.
And it was big drama at the time, right?
Yeah, well, because like there was some confusion over the keys and stuff like that,
or hardware keys and, and, and, and yeah, some pushback against that or just some confusion
around it, I think.
But we've seen some, some even more attacks against Python projects in the last year.
I mean, it's only, it's only, it's been less than a year since that.
And so the change is this year, PyPI is going to require everybody to use two-factor authentication,
not the top 1%, the top 100%.
So, and it's, you got till the end of the year, I think.
And it's, let's see, we were linking to an article from the Python package index saying,
securing PyPI accounts via two-factor authentication.
And as of today, they're going to require, they're announcing that every account that maintains,
every account that maintains a project or organization on PyPI will be required to enable two-factor
authentication on their account by the end of 2023.
So that's the news, really.
There's some discussion as to why in this article, but there's some information on how to, how to prepare.
But it's not, I mean, it's not that bad.
I did it last year.
If you've got, especially if you're already using a smartphone, I think that using something like Authy
or something like that on a smartphone would work just fine.
So, yeah.
What else?
There's, it's kind of, I guess there's not much really more to say.
about it is that this is happening and you got to kind of do it.
You got to do it by the end of the year.
But why not?
Why wait?
Just go ahead and do it.
And it's really everybody.
So it's, so let's say you've got an open source project and there's like, you know, 20 people
contributing.
That would be cool.
Maybe there's like five, but if only one of you is ever pushing to PyPI, then it's just
one of, I think it's just one of you, unless you're doing an organization thing.
I think it's just a person pushing.
So if other people are on Git and not using two-factor for Git, but they're,
just pushing to your repo.
I think that's still fine.
That doesn't matter.
It's the people actually actively interacting with PyPI that need to be authenticated.
Yeah.
That's what it sounds like to me as well.
It's kind of, if, if you're actually have an active account on PyPI, right?
It's not necessarily GitHub.
Although I think GitHub itself also has a 2FA requirement now.
And there is some discussion here about like people that don't interact with their, with
a project, but still have a PyPI account.
And I'm not exactly sure why.
Apparently there's some people that need it that, I don't know.
Why would you have a PyPI account if you're not pushing stuff to?
Yeah.
That's a good point.
It is a good point.
Well, so there was so much drama about it and there was that person that deleted their,
all their packages because they were frustrated as like a thing of protests and it caused some
issues.
And well, I'm fine with this.
This is, this is great.
I think it's supply chain issues are really, really serious.
So it's, it's okay with me.
Yeah.
Me too.
So shall we thank our sponsor?
We shall, but first I just want to point out.
Yeah.
I think Authy is a fantastic option for the 2FA stuff that you were pointing out, right?
As you mentioned, like one of the things that is a huge hassle for, for a lot of the systems
is guess what?
You can install this 2FA tool onto your phone and it's completely safe.
And all that local, that 2FA, it'll never go anywhere until you want to get a new phone.
And then you're completely out of luck and you've got to somehow reset it or worse, you lose
your phone, but it's not, there's no way to recover the 2FA code.
So what I really like about Authy is it will, you can install it in multiple locations.
Like you can install it on your desktop and your mobile device.
And they're just in sync.
If you add one somewhere, it appears elsewhere.
So if people feel like TFA is a huge pain, I think Authy is one of the choices that's
pretty good for that.
I didn't know.
You can also do like 1Password and so on.
But to me, having the passwords there and the 2FA thing in the same place seems to violate
some aspect of security.
I mean, I know 1Password is pretty safe, but 2FA should be about having the password and
the thing separated in my mind.
And so I don't use my password managers 2FA thing.
Yeah.
I just thought, I thought I had like just a couple accounts with Authy and I just looked
and I've got like, I got a scroll.
I've got a whole bunch of things on Authy right now.
Yeah.
I think I have 40 or so myself.
All right.
Now let's tell people about our sponsor.
All right.
This episode of Python Bytes is brought to you by Influx Data, the makers of Influx
DB.
Influx DB is a database purpose built for handling time series data at massive scale for real-time
analytics.
Developers can ingest, store, and analyze all types of time series data, metrics, events,
traces on a single platform.
So dear listener, let me ask the question, how would boundless cardinality and lightning-fast
SQL queries impact the way you develop real-time applications?
Influx DB processes large time series data sets and provides low latency SQL queries,
making it a go-to choice for developers building real-time applications and seeking crucial insights.
For developer efficiency, Influx DB helps you create IoT, analytics, and cloud applications
using timestamp data rapidly and at scale.
It's designed to ingest billions of data points in real-time with unlimited cardinality.
Influx DB streamlines building once and deploying across various products and environments from
the edge, on-premise, and to the cloud.
Try it for free at pythonbytes.fm/influx DB.
The links are also in your show notes on the podcast.
Thanks, Influx DB, for supporting the show.
Yep.
Thank you.
Everyone check them out to help support the show.
All right.
Let's talk about queues, Brian.
Okay.
Yeah.
So I want to talk about Propan.
Now, Propan is a project that's not, you know, tens of thousands of GitHub stars.
I think it looks pretty compelling.
It's put together by Lance Nick, Lance Nick, Lance Nick, Lance Nick, I'm going to go with
over here on GitHub.
And it is a powerful and easy to use Python framework for building asynchronous web services
that interact with any message broker.
So what are some of the options of the message brokers here?
We've got RabbitMQ, Redis, Nats, Kafka, SQS, some of the other ones like Redis streams.
If you're using these and you want a cool declarative way to interact with them, then Propan might
be your thing.
So right now what they have is async APIs for you, and they're working on synchronous ones,
but they don't have them yet.
So let me just give you an example, Brian.
Over here, it says, first, let's take the quick start from AIO Pika, which is a way to
talk, way to listen for events.
This is the important part.
Listen for a certain set of events coming into a message queue.
Okay.
So what you do is you say, I'm going to connect to the message queue server, and I'm going to
listen to a particular queue.
Then you await creating the connection.
You await creating a channel.
You await connecting to the queue.
And once you do it, then you use the iterator.
You loop over the iterator as messages come in, and then you get them.
And then you, of course, run that code that does that right.
That's the imperative way where you do all the steps yourself.
So this other way is what you do is you go to, you basically create this thing called a
broker using propan, and you point it at one of these queues, like Redis or something.
And then you just, kind of like you would in FastAPI or Flask, you say, you put a decorator
on a function, and you say, at broker.handle, and you give it the name of the queue.
So if a message comes into that named queue, call this function.
Oh, I like that better.
Isn't this nice?
It's kind of like, I'm listening for this URL, like if, you know, slash courses, slash ID
of a course.
I want to get you details about that course, right?
You would put that in Flask or Pyramid or FastAPI.
This is the same thing, but for message queue.
So you say, this function receives stuff that goes to that queue.
Oh, I like it.
Yeah.
That's what those interfaces should be like.
Yeah, absolutely.
It totally should.
So this is pretty interesting already, but it gets a little bit cooler.
You can go and create one of these apps and just run a server directly, right?
So you can say, I want to run this as a system daemon on Linux, let's say, and it's just going
to, you know, use the Propan server to run.
That's fine, but there's tons of infrastructure around running these types of things as web applications.
And if you already have a web app that receives like JSON requests, you know, it's got some
kind of API endpoint, but you also want to have it handle stuff that might be put into the
message queue.
Then it has integration with the virtual down into, you can do it manually into any web framework,
or it's got things like a FastAPI plugin, which is pretty cool.
Oh, cool.
Yeah.
So let's see.
Actually, if I go to the examples, I'll pull up a Flask one.
That's probably the best, which you got to use Quart because it's only async.
That's the Flask async variant.
So what you can do is in your, let's see, I'll just say in your Quart app, you create this
broker to sort of listen as well.
In addition to create your, your Flask or Quart app.
And then you might have, you know, a function that says app.route, listen for forward slash,
and that's a JSON endpoint.
Or you might have broker.handle some queue message, and that's the queue coming in.
So it's kind of like, well, here's the messages coming in over the web, and here are the ones
coming over, message queuing.
But, you know, it's just, it runs in Microwisgee or G Unicorn or whatever.
Yeah, that's nice.
Yeah.
Last thing.
This is inspired by Pydantic and FastAPI.
And so let me see about a good example here.
You can do things like declaring that the body of the message is, is a dictionary, or you can
have Pydantic base models that are coming in.
So you can say, here's a Pydantic.
When a message comes to the message queue, it's going to be represented by, let's say,
JSON.
And that JSON, I want to parse into a Pydantic model.
You can just say, much like FastAPI, in your handler, you say body colon, you know, the name
of your custom Pydantic class.
Boom.
Now it's automatically parsing that based on the type.
Oh, based on, that's neat.
And the last thing, they also have this concept of modeling pytest fixtures.
So you can create functions that will do things like, you know, process requests or give you
extra information or whatever, you know, what you would do with pytest fixture type things.
And you can have those as well in here, which is pretty cool.
So there's a lot of cool, it's like a fusion of interesting Python frameworks for message
queuing.
I like it.
So ask your doctor if propane is right for you.
Ask your doctor.
That's right.
Yeah.
I, you know, it's, it's interesting because this message queuing type of architecture is
super powerful at unlocking tons of interesting asynchrony.
Like, well, if I've got a request come in and I got a, you know, place an order and we
got to check the warehouse, whether we have them.
And that's a janky old API call that's slow.
Like, well, how do I scale that would be one option with threads and async and await.
The other one would be just like, well, throw that into a queue to say, check that out.
And then, you know, let it run completely disassociated.
Right?
Yeah.
Scroll to the bottom, the key features.
So one of the things I want right down there, testability.
Propane allows you to test your app with without external dependencies.
You do not have to set up a message broker to test.
You can have a virtual one.
That's pretty cool.
Yeah.
This is cool.
So it's, it's not super popular.
Like I said, however, it does look pretty neat.
Sure does.
All right.
Over to you.
So that, that was a little bit of a new thing.
I want to talk about a little bit of an old thing, which is make files.
We haven't talked about it for a while, but make files are still fairly popular for Python
projects.
I think I I've got them on several internal projects at least.
And they, they come in handy.
You got to be careful that a lot of people, sometimes people on your team won't be familiar
with them, but if it's a common thing for your team to use make files or for you, why not
use them on a Python project?
So this, what I'm going to cover is a, an article for getting the author name right now.
Let's see.
Ricardo and drag called make file tricks for Python projects.
And I'm going to hop down to the actual template.
What it is, it's a, it's a little, it's a small template as a starter template for a Python
project, but it has some pretty cool features.
And the, the actual templates at the bottom of the article, but we kind of go through some
of the different things that you might want to put in there.
And so to start off some, a little bit, I always forget to do this.
These are things I want to, I always want to do, but I forget in my make files, things
like making sure that it fails.
If anything throws a incorrect error code.
And also warning, if you did something wrong, like undefined variables or you're using, you
can turn off this built-in rules.
And I don't really know what the built-in rules thing does.
It's just, I find my make files more pleasant if I disabled them.
So this is good.
The virtual environment thing.
So there's a little snippet that he includes that you can use the PY variable to select which
Python to run.
So if you already have a virtual environment, it uses that, which is cool.
That's pretty clever.
Yeah.
And if you don't, it uses the global one.
And then also with pip.
So use the, it uses that PY variable to pick pip if it's there or not.
And it uses the global one.
So that's pretty cool.
Actually, it'd probably be better to just blow up if you didn't have a virtual environment.
Anyway, some of the, some stuff like PWD and current working directory and work root.
These are good things to add in because sometimes you'll call a make script from a different directory.
So your actual current directory is different and it mucks things up.
So there's some good correction there.
I do like this.
There's some little magic stuff about default goal and help message.
And I had to read this a little bit to understand what's going on.
But what happens is it, the default goal being help means that if you just type make with
nothing, no arguments, what should it do?
And a cool thing to have make do is to print out all the things that you can do with the
make file, like all the, all the targets and what they do.
And so that's what this does by having these, having this little greps thing.
Is it grep?
I don't know if it's anyway, it's searching through your file.
And, using awk and saying, Hey, if you've got a comment against, at the site of a target,
that means that's the help message.
So it'll print that stuff out.
Um, Oh, that's cool.
Yeah.
Uh, some, I don't really muck with my Python path too much, but if you have to muck with
your Python path for a make file to, to find libraries or something like that, or find
the code that you're running, there's examples on how to do that, which is nice.
Um, I guess that's really kind of what I wanted to talk about.
Um, and I was surprised it's doing all this stuff and it's really, and some examples on how
you can use the path thing.
Oh, having, adding a little, create virtual environment within a make file.
This is nice.
Just so that people working on the project.
Yeah.
Make dot V and V.
You could have V and V also.
And it just makes your virtual environment.
Why do you need a target for that?
And it's because, and you've, you've discovered this, but sometimes, new Python developers
kind of forget is that it's, it's kind of annoying to just create a virtual environment.
It's good to, after you've created it, update the update setup tools and wheel and build.
And then also if you have a requirements file, why not just install it right away instead
of having, having that as another command.
So kind of a fun template for starting make files with Python project.
Yeah.
That's, what is that?
A modern take on an old idea.
Yeah.
Um, the, and if you are new to make files, one of the things to be careful about that some
people don't quite sometimes real remember is spaces matter within make files kind of like
they do in Python, but spaces and tabs matter.
So in, in make files, you're using tabs.
It has to be a tab.
It cannot be space, unless something's changed that I don't know about, but that,
that, that has messed me up before.
So use tabs within make files.
All right.
Yep.
Sounds good.
Excellent one.
Um, I guess that's everything.
Yeah.
Any extras?
Uh, no, not really.
You.
I got a couple here, just a couple of conference ones.
So high con Portugal has their call for participation.
So.
Um, got, got a little bit of time left on that.
What is that till the 30th, June.
And when will it be, it will be September seven to nine, which is cool.
So if you're in and around or want to go to Portugal, there you go.
Cool.
On the other hand, if you happen to be interested in Django and are in, Europe that just got
announced as well.
So people can check that out.
I want to go soon.
I'm not going, but I want to go.
Yeah, indeed.
So that's also announced link to both of those in the show notes.
Are you ready for a joke?
Yeah.
Well, this becomes because apple.com.
Did you see that they announced this crazy vision thing, Brian?
Yeah, but it doesn't come with the snorkel.
It's just the snorkel mask.
Yeah.
It's just the snorkel mask.
It's, it doesn't even come with that little, handheld sub submarine thing that you can
drag yourself around either.
So, yeah.
So they announced if you haven't noticed yet, Apple announced vision pro, which is a $3,500
ski goggle looking thing.
That is both augmented reality and virtual reality kind of turn the dial.
I'm highly suspicious of this.
I think it's going to not do great, but it does look pretty awesome for certain use cases.
Like for example, you could sit on the sidelines of a football game and, and get like a 3d view.
So you could look to the right and see down the sideline and then look ahead and watch
the game.
I got that.
That's pretty epic.
Is it worth $3,500?
I don't know.
We'll see.
Uh, but okay.
So that's setting the stage for the joke.
So here's the joke.
The average pseudo technical person has got like an Oculus Rift in there.
They got their handheld controllers that they're doing.
Right.
And then we have the pseudo, the rich pseudo technical people wearing the Apple one sitting
there watching TV.
And then Brian, you want to describe the actual technical people, advanced high tech setup they
got here.
That's just a dude at a desk with like, you know, with using a computer, but it's, oh,
there's important stuff to it though.
It's dual monitor.
We will note the dual monitor.
Yes.
And the mechanical keyboard.
These, this is not your average desk worker.
Yeah.
Anyway, there's my follow on to WWDC.
I know I would.
And that's, that's even, okay.
I've got the, the big curved monitor.
So I don't have two right now, but you have two monitors.
I have one big monitor for my, my working desk and I have a big curved monitor for my gaming
PC, but just one for both as well.
I used to have dual monitors and I would, I was always trying to like juggle them.
I'm like, you know, I just want big monitor.
That's better.
Yeah.
I actually, when we did the pandemic thing, I went to one big monitor at home and then
at work, I still had the two split ones, but then I was just tired of doing this all day
long.
Yeah.
So I'm like, oh, we got to just, so I went to a big one, but that's kind of privilege
speak.
So I don't know.
It is a little bad.
Just for people who are interested, I, if I do need a second monitor, like sometimes when
I'm recording a course, I want to be able to see what the recording is doing.
So I want to see my video overlaid with maybe what's on the screen with whatever settings,
like scale, like exactly what's being recorded as the person's going to see it in case something
goes weird with that.
So I'll take my iPad, plug it into my mini and then use Duet.
Duet is a really cool software that I think works on Mac and Windows and basically turns
that into a second monitor just periodically when you want it.
You know?
So that's, that's what I do if I really feel like I need extra, extra space.
So, so go back to the Apple vision thing or the eye dork.
What's it called?
Apple vision.
Yeah.
I think it's, I, I, I dork pro.
Um, there's part of one of the, the things on there is somebody doing a, like a meeting
where you can supposedly see other people in the meeting, like as if you were still there,
they were with you or something and I thought, well, that one go up a little bit.
Yeah.
This is like the, the group FaceTime is what that is.
Yeah.
Except for, oh no.
Yeah.
I see it.
Okay.
Wouldn't, wouldn't they see you with the goggles on?
So if everybody's doing it, wouldn't everybody just, you just be able to see people with goggles.
Um, that's interesting.
I think it might scan you and put an avatar of you up there.
Oh yeah.
It's an AI you.
It's not really, I think it's, I think it's an AI you actually.
Yeah.
I think so.
I haven't tried this out.
And where's your camera?
Where do you put your camera?
Like for, so anyway.
Yeah.
There's a lot of interesting and stuff.
I'm actually interesting things and stuff.
I'm actually excited about announced at, at WWDC vision pro.
Uh, yeah, there's, there's like, for example, large language model dictation for iOS and Mac.
So I don't know how many people know who have tried this, but I, for multiple reasons have tried to do,
um, dictation on the Mac.
Partly because I have like mild grade at this point, RSI issues.
And so if I can limit typing, that's good.
And maybe I've got a lot of stuff I need to blaze through, like a bunch of email or something.
I'd love to dictate to it.
But the dictation system on Mac is like 10 years old or something.
It's really bad.
You can't even say new paragraph, for example.
Like, nope.
They don't just write out new paragraph.
Whereas on iPhone, you can say new paragraph or do this, or like you can navigate around way better.
They're not the same systems.
So both of those are being replaced with like ChatGPT level of AIs.
And so dictation to your computer or your device is going to get way better.
And so that means less typing, less RSI, just different input modalities.
If you need a break, like those kinds of things, I'm really psyched about.
Vision Pro, we'll see.
Yeah.
There's potential there, but there's also way more potential for jokes.
Yes.
It's going to be good.
Speaking of jokes, we'll wrap it up with one from Kim in the audience.
If an avatar is an option, T-Rexes will be meeting with the elves and talking frogs in no time.
That would be great.
I'm here for that.
Let's do it.
Yeah.
I'm here for it.
All right.
Bye, Brian.
Bye.
Bye, everyone.