Hey there, thanks for listening. Before we jump into this episode, I just want to remind you
that this episode is brought to you by us over at Talk Python Training and Brian through his pytest
book. So if you want to get hands-on and learn something with Python, be sure to consider our
courses over at Talk Python Training. Visit them via pythonbytes.fm/courses. And if you're
looking to do testing and get better with pytest, check out Brian's book at pythonbytes.fm slash
pytest. Enjoy the episode. Hello and welcome to Python Bytes, where we deliver Python news and
headlines directly to your earbuds. This is episode 271. Really? Wow. Recorded February 16th,
2022. I'm Brian Okken. Hi, I'm Michael Kennedy. And I'm Steve Dower. Welcome, Steve. So who's
Steve Dower? Who's Steve Dower? Yeah. Number of things. Probably most interesting to this audience
is I'm a core developer on CPython, one of our Windows experts. So I spend a lot of my time
focusing on making Python run better on Windows. I also work at Microsoft, where I also spend a lot
of my time making Python run better on Windows. So I'm kind of a bit of a one-trick pony, I guess.
But I feel like it's good work and it helps a lot of people. So if I have a problem with Python on
Windows, it's your fault. If there's solutions for Python on Windows, then it's my fault. I'll let
other people own the problems. So if I go to the Windows store, I can now install Python from there.
And you were part of that, right? Oh, I should have had that up on screen, shouldn't I?
Yeah, that was actually... The request came from people within Microsoft were like,
hey, why can't we get Python up on the store? And my response to all of these is like, well,
if the community is willing to do it, which is half me and is half the people who would have to take over
if I stopped doing it, then yeah, we'll go ahead and do it. And so I got actual work time for that.
That was a contribution from Microsoft for that one. But yeah, the community was on board and
it's going really well. That's also the one that we tied into the default Python.exe that's on every
Windows machine now. So if you go to a brand new machine and just type in Python, you'll get straight
to the PSF's Python. Microsoft is not doing it anymore. We just contributed the change. And now I
switch hat and do it with the other hat on. So it's real Python, right? It's exactly the same as what you
get from python.org. It's just delivered, you know, easily fast install automatic updates.
And a couple of edge issues that we're working on bringing down. So yeah, fantastic.
Automatic update. I know this wasn't one of the topics, but now I think I might have to rethink
how I'm installing Python on my desktop at work. So that's a cool idea.
I have only had store installs on my own machines since 3.8. I think I haven't,
apart from testing, I haven't actually used the regular installer on my own.
But I mean, you, of course that makes sense, but.
I mean, you know, it's always testing, right? Every time I'm using Python, I'm testing.
So.
Chris May out there says, thank you so much for making my work life in Windows easier.
Anytime.
Yeah. Well, Michael, why don't you kick us off with a story or topic?
I have got a good one. So I'm a big fan of FastAPI and FastAPI being built on Starlet.
So by the transitive property, I'm also a fan of Starlet. And there's this thing I want to cover
called FastAPI events. So when a request comes in to a particular API endpoint, or if you convert it
over to a web app, to a web page sort of request or something, you might want to dispatch that out to
say like WebSocket listeners or something along those lines. So there's this cool project called FastAPI
events. It's pretty small and new. So I'm going to try to give it some visibility. It's only got 36 stars.
It's pretty new, but the idea is that you can go through and basically create this middleware
handler that will let you say when a request comes in, here's the way when an event is raised,
here's the thing that's going to handle it. And then in some API endpoint, you can say dispatch,
give the event a name and some dictionary data to be passed along. I suppose it doesn't have to be
a dictionary. It could be whatever. And then in other parts of your code and say, I want to just hear
about this event that happens no matter what API endpoint received it, no matter where in like how
deep down in the code it was received and so on. So then way down here, you just put a role handler
decorator on there. You say, I want to capture all the events that start with some substring like cat
star for like category, whatever, or this one is actually literally about cats. And then you can just go
through and write these functions that will then handle that. And you can also pass them off to
queues like you can use the SQS, the simple queuing service from AWS, I believe that is, as the endpoint
instead of it just being your app. So if you've got like lots of scale out and stuff like that.
Wow. Of course. So just like a neat way to do logging or even distributed logging, I guess,
if you've got forwarding handlers in there, you can just...
Yeah. Yeah. It seems like it, right? Like, or if you want to sort of build up, like, here's the
request transaction and here we're at this stage, or like you could maybe do like visibility into
long running workflows with this kind of thing or something along those lines, I would think.
So yeah, there's also an echo handler for debugging. I kind of like that. Like if I just need to see what
is happening, it'll just print whatever's happening. It'll just start printing out all the behaviors that
you're logging. So...
And then when you want to stop doing that, you just take away the handler and you don't have to
search the entire code base for print and find everywhere that you added it in for debugging.
Exactly. Alvaro out there says this looks similar to Django events. Yeah, I suspect. I suspect it is
similar. Yeah. Anyway, pretty short and simple, but if you're looking for a way to sort of
put notifications in a structured way into a FastAPI app, well, here you go.
Oh, I'm thinking of a whole bunch of more abusive ways to use this.
Yeah, you can write some really impressive spaghetti code with this.
Yeah, I'm sure that you can. Get the cloud involved in everything.
Yeah. So let's switch gears a little bit and talk about testing. Imagine that. I've got a testing topic.
So this is... I'm pretty excited. This is... I've been asked a lot about testing pipelines,
testing data science stuff, and I'm not... That's not something I do day to day. So I'm really glad
to find people talking about it. So this... We've got an article from Peter Baumgartner.
Ways to use testing... Ways I use testing as a data scientist. And I actually just really love this
article. It's great. To start with, he starts off with what he uses testing for. As a data scientist,
he uses testing to make sure things work, to document his understanding, and to prevent future
errors. Well, that seems straightforward. But the reason why he wrote this up is apparently because
there's a lot of software... There's a lot of testing stuff out on the web, but it's not... It's like geared
towards test engineers or software developers. And he's like, I'm not a software developer. I'm a...
And, you know, I'm doing something else. I'm doing analysis. I'm not a software person, even though... Yeah,
you are. But to write this up in a context where data people might understand it better. For instance,
he doesn't even start off with writing... Having written tests. His analysis is like, if you're doing
notebooks or other code, just use a cert a lot. So he's using a cert all over the place, including... He says,
"Where do you have? Use it for as many intermediate calculations and processes as you can, as it makes
sense." Because... And doing things like checking obvious stuff. Like he's got an example of a table
count where he's counting up all the yeses. Well, you can do a little bit of math just to make sure
the math works. So like all the yeses and nos and missings should all add up to the same count. Go
ahead and throw an assert in there because sometimes it doesn't. And in this example, he said that he
actually caught an error because he was looking at two different data frames. So they really weren't,
they didn't add up to the same. So you can catch things like that. So just double checking yourself on
things as you go wrong away, go as you're developing. One of the cool quotes he has in here is like,
as he has a habit of when he's using notebooks to whenever he's visually inspecting the output,
if you're visually looking at the data that comes out, maybe write an assert statement to do that
analysis so that it's always checked. And this is a cool use of putting asserts in notebooks. I like this
idea. The article goes on, it's pretty extensive, talking about checking the data, using hypothesis
to, well, not the data, not the data at this part, but your assumptions around the data. So using hypothesis
to check your assumptions and hypothesis will think, show you things that maybe you didn't consider like
NANDs, are you handling those correctly? Empty series or empty data structures that are going into your
into your code? Are you handling those? If I mean, hypothesis does have take some handholding,
but it does make you think about really what is the shape of the data going in? And do you, can you,
do you need to limit it? What hypothesis is looking at, or do you need to change your code to handle more things?
Hypothesis is great. I've used that for a couple of, you know, parsing projects or combining projects.
I spent way too long adding all the strategies to be able to test a URL parser that I was calling into.
But it's fantastic for finding kind of things that you would not have thought of.
Yeah. I mean, it's finding things, but it's also, and it does make, yeah, that aspect of it seems like
the point of it. But the real value I get on a hypothesis is thinking, making sure I really
understand the data that's going to come in and thinking through those. It goes on to talk about
actually testing your data using things like Pandera, which I wasn't familiar with, and another
package called great expectations to look at like putting schemas around the data coming in and making
sure that the data always matches a schema. Going on to talk about a range act assert and using
pytest. pytest comes in with, he's only really writing formal tests when he's writing libraries
for other people. But all these other packages to be able to test with data science, I think this is a
great addition to the data science community.
Yeah. Alvaro talks about how this is often referred to as defensive programming. And then,
you know, I feel for him a little bit, says that work, we use this with our Fortran code. So there's that.
But I do think this is a really interesting way of thinking about defensive code. You know, I just,
I think of writing defensive code as like, oh, I'm gonna have a bunch of is statements to verify this
thing's not none or verify that this is the right type and that it has like a reasonable value and raise
exceptions. I haven't really thought so much of it for like notebooks. So that's pretty interesting.
And one of the neat things about like, if you're actually putting a search in your code, you can actually, you can write
tests against your code that don't even have any certs in them. And because the search will happen
within your code and the test will still fail and catch it. So it's kind of cool.
Yeah. Yeah. Very cool. Good stuff.
Yeah.
Steve, I am super excited to hear about what you're,
you got coming up because this is brand new being a core developer. I feel it is appropriate that you
break this.
I mean, I'm not going to lie when it came to, you know, what am I going to talk about? Okay.
What's the most recently accepted PEP that was somewhat controversial? And I think just as you
kind of look down to the section on rejected ideas, which is considerably longer than the accepted ideas,
you can probably get a bit of a sense for just what went on with, with exception groups. And I know,
Michael, you've just had a conversation, you've learned all about them. So you can, you can take
over when I run out here, but I'll share my thoughts with it, but yeah, go ahead. I'd love
to hear about it. This is sort of inspired by trio, right?
The, the end goal kind of is, so this is an interesting PEP and we've got a few of these on
the go at the moment. It's kind of like a stepping stone towards a better programming model or a stepping
stone towards better libraries. So it's, it's something that I think in, in my opinion, very few kind of
application developers, kind of the last developer in the chain is like, I'm often not
going to use them and they're not going to need them. But as you go further in towards the lower
levels of libraries, especially people writing async schedulers are going to find incredible value out
of them. Essentially what the idea is, is that when you're running multiple tasks in parallel,
if some of them fail, we don't currently have a, have a neat way to capture the exceptions from all
of the ones that failed. There, there's some approaches that would be like, wait for all of
them to complete and wrap it in a list. And then you get some exception that contains a list of exceptions,
but that's lost a whole lot of context. you can get just whichever exception happens first,
but then you lose all the other exceptions and there's just been no real way to handle it. So an
exception group essentially does bundle up all the exceptions, internally in some way. But the
really interesting thing is the except star syntax, which I'm going to have to scroll a long way down
to find where that comes up. but, but this is really clever because if you're in that situation where
say you're running 10 parallel processes, so here's kind of the first example of it, then
exceptions are no longer control flow at this level. Cause if you're, if you've run 10 things and you're
waiting for 10 things to complete, you're not actually doing control flow with the exceptions anymore.
What you're doing is handling the exception, but then the control flow is going to go back to where it
was anyway, because you, because you, you're going to, you know, be doing something different. So for
example, if a file doesn't open, then you, you would want to, you know, do something different,
right? You're going to stop going on and trying to read from the file. but if you've tried to open
10 files and three of them failed, you at the outside level. So at the end, at the inner level
for each file that may have failed, you'll do something different at the outer level.
All you're really going to do is say, Hey, this task failed because a file couldn't be opened.
And maybe you do something else, but it's at the outside level. So except star takes that exception
group and it's going to give you a chance to handle each exception, essentially on its own.
It will group them together. So in this example, if you know, five tasks report spam error,
then you'll get into this except spam error block with all five of them at once. which is just
what is that a list of spam exception, spam error exceptions, something like, or a tuple, something
like that. I think it's a tuple. I think with the stars in time, I believe, something
iterable basically. Yeah. Yeah. so something you can iterate over to see the exceptions, but
it's really just, you know, this happened at some point and you process it. And if the group actually
contains multiple types of exceptions, then each handler that matches is going to be called for all the
exceptions that match that. So you could have this tri block raise an exception group that has some spam
errors. It has some full errors. It has some bar errors and all three except, except star blocks are
going to get called with the exceptions that match those, which is a bit, it's, it's definitely going
to confuse a lot of people. It confuses me, which is, you know, why I was keen to actually spend
a bit more time digging into it, and trying to figure out what's really valuable about this.
and I do think the most valuable one is really where the error is canceled error, because if for
whatever reason, five of your tasks have been canceled, then you need to capture that and do
something with that outside of it. But it doesn't necessarily mean you want to throw away the five
successful results. And so you do kind of want to, to keep a bit of everything going on. it's,
and like I say, it's a building block on its own. This isn't enough to do anything new and useful.
The next thing that comes along is task groups and that's, you know, being worked on by,
uh, you know, I expect a lot of the same people who worked on exception groups because with task
groups, now you can actually start, there we go. Guido has just merged task groups. Excellent.
then now you can actually, like run the task group. And if the group raises any errors,
then you'll catch them through an exception group. And so that enables a whole lot of, new uses
and new ways to use asyncio or just async generally, no matter the library, as you say,
trios already had something like this for a while. and from their nursery thing.
Yeah. And so that is now being standardized. So libraries can kind of share their implementations
and work together on it. So one of the reasons you need this is if I start two web requests and
three database queries, and then I go to wait on them, you know, then the, if, if several of them
fail, the error state captured in totality is a, a tree of errors that represent, well, this,
this, this, this task started this other task, which then had this error, this other one.
Right. So you need some way to deal with a, a group of errors that could happen
kind of all at once. Right. And one of these task groups that gets kicked off.
Yeah.
Yeah. So the new task group thing is super cool. So you say async with task group as TG,
and there's two things that are neat about it. One is right now, if you fire off a bunch of tasks
and async and await style, they're basically unrelated. Like if one fails, that means nothing
for the other, right? They're just like, well, here's a bunch of stuff that happened. And this
creates a relationship between them. Right. So that if one fails, I think it might not schedule new ones,
something to that. Like it's brand new. I'm just seeing the tweets. So I think that that's the
story. I believe that was the story of Rio. The other thing that's interesting here that in this
example, which I all linked to from Yuri that he posted, he tweeted about the news was notice that
the first one says task group, create task for some task, and then it awaits something that creates
another task. There's nowhere where you say, store all those values into like some lists of tasks,
then go to the task and iterate them and wait for them or gather them or whatever the heck it was
you had to do before. This now makes tasks fire and forget. I can say, run this, run this. And within
that, I could do more of those types of things. And then you just block at the with context manager
level to wait for all the tasks to finish, which I think is a real big improvement. Because right now
you've got to like constantly juggle, well, I've got to return a task from this so I can go wait on it
later and all those sort of oddities. And this cleans up a lot of that.
Right. And of course, being Python, I don't know exactly how the syntax works, but being Python,
that TG object, the task group doesn't actually disappear at the end of the with block. So if
that's got results stored into it, then you still have access to those and all of the information
about the task group, even after you've waited for it to complete running.
Oh yeah, that's cool. Yeah. So I think this is a nice addition to async
I/O and Python. This is cool. And apparently 3.11 is coming.
Yeah, coming in 3.11.
Yeah.
I do see a question from Sam Morley in the chat there. Is there a way to short circuit
so that you don't re-catch certain exceptions? My understanding, and Michael, if you've got a
better one, correct me, is that the exception, the accept blocks work in the same way as regular ones.
The first one that matches a particular exception will handle it, and the later ones don't, even if they
would also match. So if you have, so if the spam error is a subclass of foo error, but there's another
subclass of foo error, then spam errors will get handled by the spam error handler. The foo
and foo error handler will handle all the other ones apart from the spam error subclass.
Nice. Yeah. I don't know much about the accept star other than it was basically a requirement for the
task group stuff to be implemented properly. So once one came in, then the other could come in. Yeah.
It's the only feasible way to actually do something as a result of an exception group. Otherwise,
you do end up with a very generic exception, and then you write a for loop over all the exceptions that
are handled and try and figure it out yourself. So you'd end up rewriting the code, and it was just
not going to be feasible. It needed to be syntax. And so it is. Yeah. Right on. Very, very exciting and
very timely. Thanks, Steve. I'm kind of glad that I put off learning how to do async code until 3.11.
This looks easier. It's a good band and a good time for async.io. Well, cool. All right. I guess I'm
up with the next one, huh, Brian? Yeah. Let's see what you got. I have got some other interesting things.
I'm here about showing off the underappreciated projects or the new projects. Just a couple of stars here.
And we talked about overloading before, but I thought this was a clean way to do it that people
can think about. And Steve, I would definitely love to hear your thoughts on this. So Felix the cat
created this library called pyoverload. And the idea is basically once you have type information,
then you can have method or function overloading. The idea of being like, okay, I have a function called
foo or whatever. And you can say, if it takes an integer, I want this implementation to run. If it takes a
string, I want some other implementation to run. Right. That's sort of the traditional C++,
C# definition of it. Right. But in Python, we don't have that really because the language started
without type. So how are you going to figure out the type to overload it? You know, right. That just
like, doesn't make any sense. So with this one, you could sort of use like traditionally, you could use
this instance. We're going to do one thing or another. Is it a single thing? Or is it a list of those
things? What are we going to do? But with this one, you can put just at overload and then whatever the
signature is, if you can say it has no functions or has no parameters, or it has like two integers,
or it has three integers, or it has like a list of them, whatever. And there's even a way to sort of
say somewhere down here, there's a way to say like, if none of them match call this particular one.
So basically it's, it's just straight function overloading in Python. If that's the thing you want,
Steve, does this make you cringe or do you like it? Well, you know, I'm, I'm, I'm not going to lie.
I'm not the most into static typing in my Python code as a lot of other people. And, and there's a
lot of, you know, there's a lot of complicated reasons, but I think for a situation like this,
I mean, if I know if I was writing a function that took a string or an int, the very first line would be
converted to whichever type I actually want. And then the rest of the function is going to look identical.
Yeah. And that's sure. And that, in that case where like, there might be a, an unparse type of thing,
for sure. I think you wouldn't really do an overload. That would be insane.
And my, my kind of gut feel, and you know, I'm always open to, to examples proving me wrong.
in which case I, you know, I would write the is instance code that's in those examples.
You know, my, my kind of gut feeling is that if you're doing two drastically different things in
the function based on the type, you need two functions. and once you've got two separate
functions, you know, if the people calling don't know what they're passing you, then that's, you know,
they've got a problem and it's not, you know, so much my responsibility to fix it with overloading.
That said, overloading is really cool. and you know, I am the exact opposite person when it comes
to like C and C++. I will do all the craziest possible stuff with overloading in those languages,
because I think it fits the language and it's a lot of fun. and, and there's definitely occasions
and value for having it in Python. we do have the single dispatch decorator has been part of Python
for a while, which will do this on the very first parameter. this, you know, very trivially extending
it to the whole function signature is, is really cool. So, you know, it's, if, if I needed to do this,
I would probably want to use this, a library like this. would I, would I, you know,
I'd probably, I would probably reconsider my API design choices up to that point.
but, but I can understand the attraction of, of getting to, you know, to, to reuse,
reuse the name and not make the person calling it think too hard about how, you know, what's actually
going to run. Yeah. The place where this sort of seems interesting to me is, you know, there's some,
a lot of like tricks and juggling people do with like star, star, star, star, KWRs where like,
okay, depending on how you pass it stuff, we'll do a bunch of things. Yeah. And I'm always looking
for a way to like, not do that. Yeah. How, how can I not, how can I remove that? Like,
it's completely opaque. I have to do a Google search and read the docs to figure out what is
at all possible here. Well, one of these days I I'm going, probably going to take all of the kind
of patents for that kind of thing that I've collected and turn it into a book, but writing
book just feels like way too much work. So anytime soon. Sorry. my, my colleagues at work can
ping me at any time and I'll, I'll give them a patent for what they're trying to do. But that's,
um, I do have quite a set of, oh, you, you're trying to make stuff weirdly work in this way.
Here's, here's a nice way that you can enable that without having to resort to, you know, type checks and
everything. Yeah. Yeah. I've been using, I mean, I've been using Python for a long time and I do
remember one of the first things that I noticed is I couldn't do overloading. And at the time,
so this was, you know, many years ago, I was using a lot of overloading in my C and C++ code.
and, and I was like, oh, I can't do overloading. But one of the things I've noticed
is actually the, instead of keeping wishing that I had overloading in Python, I've noticed that I
don't really use it in C and C++ anymore. I've, it's gone the other way.
Yeah. I really, I'd rather be more explicit about the, and just have a function that two functions
that, that are some, maybe they're similarly named, but they have an appendix that's,
that's different so that if you have different data, you pass it. And I'm with you, Michael,
I'd rather have people go, well, which one do I need? I'll look it up. Then just, passing the
wrong data type and having me, so because, you know, sometimes if they've, if they haven't
converted the data, like the string string versus number is a scary one for me because I'm often
getting, getting my numbers from an API or something and they come in as a string. If you forgot to
convert it and you passed it to the wrong thing and you're really doing something completely different,
um, that's, that's not a good thing, but I got bit by that one just yesterday,
updating one of my, one of my CI builds to use Python 3.1, I mean, 3.10. but you know,
is it, is it a string or is it a number? Interesting. Yes. But yeah, certainly that conversion would be,
you know, would be worrying. The other one is, is it a string or is it a list of strings? And that's
the one that bites us in Python all the time. And I don't even know how you resolve an overloaded function
based on, is it a string or can I iterate it? Well, like in that case, actually, I would rather
just have that part be part of the function at the, at the top of it, if it can handle both to,
to check the type and, and iterate or not, but you know. Yeah. Well, all right, let me close this out
with two quick thoughts. first, I think this is interesting because it's one of the things that's
possible with modern Python. Like once we've added typing, now you could consider this as a thing,
whereas previously it really was highly impractical, I think as a way to do, do it. So I think that's
kind of cool. And then two, I think it might be an, an entryway for people who are not where Brian
and I'll put myself in there as well yet of going like, actually these things I thought I need,
I don't need those, right? There's a lot of stuff I thought I needed and I haven't used it for three
years. So maybe I actually don't need it, but that's not how you maybe first approach,
approach solving your first problem in Python that you're coming from C++ or whatever,
C#, whatever. This might be a gateway. So anyway, those are my two thoughts.
One more thought from Dean after Python 3.11, do we get Python 95?
I, there, there was, you know, there was a windows 3.12. So I think Python gets to do a 3.12 as well.
I think it was only available in China. Interesting. And I believe I like to follow
on with that Dean. Very funny. I believe that Windows 10 was named, you let me know if you
know different, Steve. Windows 10 was named Windows 10 because there used to be the check Windows 9 as
the, the starting string for 95 and 98. So you can't be nine because then you're going to be 95. So we got a
kick on past it. There was some embarrassingly big, language run times out there still doing that
check. that, that really struggled with Windows 9. and, and showed up in enough places that,
yeah, I think it just made sense for everyone to just skip it.
Not skipping 13. We're skipping nine. It's too unlucky. All right. Awesome. Brian, what you got for us?
Oh, what do I have next? I have, the next generation, Seaborn interface. So Seaborn is a really awesome,
uh, plotting library built on matplotlib. and I, you know, actually I don't use it that much, but I've always been
intrigued by it and what kind of watching what plotting libraries do and stuff. And one of the things I was curious about,
which I'm really grateful for this article, is some of the history behind it. So the,
the article starts off a next generation Seaborn interface talks about the background and goals.
but, some of the, some of the great things in here, let me grab, grab some notes.
this work grew out of a long running effort to refactor Seaborn internals. so that, functions,
you know, anyway, where I wanted to get at was he was developing a refactor of the internals.
And he's like, wait, wait a second. If I want to refactor it, maybe I should expose more stuff.
And some of the background was originally Seaborn was originally conceived of as a toolbox to do,
uh, of domain specific statistical graphics to be used alongside matplotlib. So the intent was people
would use both Seaborn and matplotlib together. However, things that people have doing, are doing
things differently. A lot of people just grab Seaborn by itself. Some people even just learn Seaborn before they
even learn matplotlib, which is an interesting thing. And that's how I thought you were supposed
to be doing this, but the concept was, and, and then at over time, there's a whole bunch of features
that have been added to Seaborn to where it's like really slick looking, but to do the same thing by
hand in matplotlib is a lot of work. So there's some things that like, if you, if you Seaborn's almost
there, but you need to tweak it a little bit and you have to do things manually. Well, then you have to
just do everything by yourself and it's a lot of work. So the idea around this, this, a rewrite
of the API is let's rework some of the internals so that a lot of the little sub components that go
inside of a plot are exposed. that way people can get access to it to do a more fine tune
configuration, within the, so they don't really have to just do everything by hand. It's either all
or nothing Seaborn or matplotlib, you can kind of do both more easily, which is a kind of a cool idea.
there's a whole bunch of great details in here that talk about some of the API changes,
basically exposing the internal. If you create a plot, there's nothing there and it won't show up.
You have to create layers on the plot. And then within the layers, you've got marks and,
and different components that go into it. I kind of like this idea of building things up,
what I really like is the public aspect of this. So you've got a, you've got a library that's out
in the open. it's being used by a lot of people already. And somebody saying, maybe we should tweak
the API and do something different and just go ahead and doing that in the open saying, Hey, we're going
to do this. There's a note at the top, or I'm thinking about doing this note at the top saying
it's a work in progress. Don't depend on these examples because things might change,
but this is the direction we're trying to go. trying to get feedback from people. And,
I think this is a lot of thing, things that a lot of people struggle with when they're maintaining
packages that have been around for a long time is, I want to do things a little different,
but am I going to break everybody? and talking through it. So anyway, this is a great read,
especially if you're a data plotting kind of person.
Yeah. Very nice. I always want to do more with visualization and I'm sure that I have some good
data I could pull up. Yeah. I ended up basically just writing APIs and websites these days,
but, but I really should be pulling this up and doing some of these graphs and I'm really happy
these are, these are around. Steve, how about you?
See, Seaborn's great. It's always, like back when I first discovered it, one of its major
selling points was simply importing Seaborn would magically make your default map plot.lib charts look
nicer. which, which map.lib is.
I love it. It's like the bootstrap of map plot.lib.
It really was. It's like they, they just apply their style by default and every map plot.lib chart
suddenly looked nicer, which, you know, map plot.lib's done their own styling work now. So it's
less valuable for that. I do like this API. It looks good. And, and as Dean's pointing out in
chat, it's like map plot.lib has an object oriented plotting API similar to this, possibly identical,
just like everyone else. I've never learned the object oriented API. but, but it is there.
And it's, it's, you know, it's, that's the modern one. It's like, I know a lot of people say map
plot.lib is impenetrable, and kind of hard to build things up, but it does have a really nice API
there. It's just not the pipe plot one that kind of imitates mat labs old API. and, and so,
you know, having it there is, is really nice. And Seaborn, you know, having their own is also great.
another, nice, thing that to read about in this is, he's does a hat tip to a
ggplot or ggplot two or whatever it's called. saying that, yes, it's going to look, a lot of
this is similar to ggplot, but, it isn't that I'm trying to copy it or maybe that's, that's definitely
influence, but, it is, that Seaborn is, is important because we think about things
differently in Python than we do in R and, and, and just having, it would be, but also a hat tip to
another library that is a, a wrapper around ggplot. If you just want that, you can do that in Python too.
That's available. So, I, it is interesting to, these are, we think of these as competing libraries,
but they're really not competing with each other. They're working together to push the, push
plotting forward. So, yeah. Nice. Dean out there points out you can do plot.style.useSeaborn or ggplot,
which is another one. Let me throw out. Oh yeah, go ahead, Steve.
ggplot's certainly the one to copy from. I mean, that's the, there's a reason that one is, you know,
as universally popular as, you know, any plotting library can possibly be. it's probably even,
you know, it's probably competing with Excel for popularity of plotting data. Realistically, it's,
it's, it's a really nice API and it looks good and everyone's familiar with it. And so, you know,
there's nothing wrong with copying from ggplot. Nice. I got one more shout out to throw,
into this conversation, the XKCD plotting style for, matplotlib. So you've got, I mean, this is
fantastic. It looks like the, it really does look like XKCD would, you know, the comic would do
for, for these. So, this is fantastic. I love it. What I love is I actually see this. I see this
in papers and stuff like that. People just go ahead and use the XKCD style and for serious stuff. And it
just is, it's awesome. I love it. I think there's actually some value to having like cartoony looking
graphics, like UI sketches and graphs to say like, look, this is speculative. This is just like,
don't read too much into it. I'm trying to give you an idea rather than an exact thing. And I think
sort of a UI like cartoony looking sketches. And this also plays into that. Yeah. Right. Steve,
you got the last one. I got the last one. Yeah. So this is another kind of recent, delivery from,
from the CPython core team. we can now compile CPython to WebAssembly. so, and to a lot of
people that probably means very little, but I guess the brief, brief summary is,
is WebAssembly is, kind of what the JavaScript in your browser compiles to before it runs.
So it's skipped that initial step of being JavaScript and it's now ready to run in the browser.
so it's, it's a lower level, there are tool chains out there that can compile all sorts of languages
directly to WebAssembly. and so in this case we've taken, I believe we use one of the,
I don't know the exact tool chain that's used and it may not matter, but it basically takes the C code
and compiles that to WebAssembly gives you a package that can be brought into an electron app
or a node JS app or a web browser, modern web browser and be run in the browser. there is,
so this page is a little bit dated. there's been a bit more work since then, but I found this is the
best overview of where things are kind of at long list of C extensions that don't work. Probably unsurprising,
like the browser doesn't have a lot of this stuff in it. Yeah. You don't have,
what no TK, all the different APIs, the win 32 API underneath or whatever it was delegating to you.
Yeah. Don't no Tkinter. What? No. Yeah. No, no Tkinter, no sub process, C types. Apparently
you can do, I've, I've heard there is a lib FFI port to, to the M script and kind of platform.
so how, how, how this kind of works is kind of when you take WebAssembly in, into a browser,
it has access to nothing. Like it starts off in a really enclosed kind of box of things that it can
do. and, and that doesn't actually work for Python at all because the very early thing that
tries to do is search the file system for what files it should be loading. so, so you, we actually
build it as part of another platform. And script is one platform that kind of poly fills a whole lot of
native looking APIs so that code that's compiled on top of them script and is able to use it. And that
this little demo, which I just hit start REPL, this is running on that. So this is a build of
Python 3 11 alpha four built with clang running on em script. And I can, I believe I can do this
and be like OS Lister. And, and it thinks there's a file system there. Now that's not my file system.
That's in memory. it can be changed to browser storage. but this is entirely in the
browser. Like there's nothing downloaded. There's nothing running on my machine here.
There's nothing running in the cloud. it's literally in the browser. I can probably freeze
my browser with this. Like I can do an infinite loop and do it, do it. Let's see if this cuts me off.
I'll just let that run. What happens if you hit clear?
Started again. We're going to start again. No, no, no, no. It's done. I'll just refresh. And yeah.
so yeah. And, and the, there's a second one that the actual builders, it was committed supports,
which is WASI W-A-S-I. that's a slightly different approach to adding all the functionality
around a WebAssembly module. it's, so it's a little bit more flexible, a little bit more controlled.
And scripted is really like, give me POSIX system inside my browser, all in memory. and so, and so
we have two options, and these are available in the, in the main branch. I don't, at the moment,
we're not shipping prebuilt modules for WebAssembly. That might be a possibility. if that's something
that you'd like to see, then I guess go to discuss.python.org and post about it. It's probably
a post there. I should have looked for a post there. but we're not currently doing prebuilt releases,
but, but I think we could, I think this is one of these options where the WebAssembly build is
totally portable. And so if we build it, we can distribute it. And then, you know, websites that
want to do something like this could just download it from our servers and, and run it. So I think
there's, there's a lot of potential here. what I'm, you know, and, and, and it's at the potential
stage, right? This is another stepping stone to bigger and better things. our kind of responsibility
as the core team is to enable it. And now we really want people to come in and pick this up and do
awesome things with it. Firstly, so we can figure out what gaps still need to be filled. but also just,
just to, just to expand, you know, the growth and the reach of Python, to bring it into places
that currently doesn't exist or can't work, and you know, give it new life and new places,
open it up to new people. This is fantastic. Congratulations. and so the work for this
primarily done by Katie Bell, Christian Himes, and Ethan Smith. So I think Christian got to do all the
merge commits, but it's definitely been number of people working on this for a while. those are the,
the, the primary three. I'm really excited for the possibility for this. I think one of the
things that could be amazing, obviously running it in the front end is a thing that could be done.
I saw the documentation said it was about 10 megs to download it. I'm sure you can put that on like a
CDN. So you kind of hit it once somewhere for a particular version of Python. That's pretty good.
You know, it's, we all have pretty fast things these days. Yeah. It's still bigger than doing it.
Yeah. What gets me really excited though, is, putting that into an Electron JS app.
Yeah, absolutely. Right. Because Electron JS is a really interesting way to bring web technologies
cross platform as much as I like, oh, I said an Electron app. Still it's, it's really opened up the
possibility for a lot of things, but it really has meant, okay, you're doing TypeScript, you're doing
JavaScript and you just have to go full on in that world. So here you could still do like your front
end and whatever, but having the core logic of that desktop app being in Python running in this,
that's exciting. If that's, that can be put together.
I should also add two things. Pyodide is a project that people have probably heard of before,
which has been working on this for considerably longer than the core team has. And so I think a
lot of the patches that needed to happen have come from them and they now get to spend more time focusing
on the data science stack, which, cause they've got ports of NumPy and pandas and other libraries,
uh, to actually do data science in the browser. and the other interesting thing that I saw was
someone from, from CondorForge suggesting that they could elevate was wasn't builds to
their kind of automated level. And so all of CondorForge may suddenly become available to use
in the browser on top of a build of Python like this.
Wow. So that, that would unlock so much. That would be incredible.
Yeah. Interesting.
I imagine initially it would unlock a lot of bug reports, but, but we need to work through those first.
Yeah. I was just thinking of, you know, take the top 1000 most popular packages,
you know, could you get 90% of those compiled to like other WebAssembly things that then could be
included and then imported here somehow.
Exactly. And the top 1000 with native code, cause it's only the native code, right? The Python code
still compiles in the browser, just like it would in the CPython interpreter.
it's only the native code that has to be ported and built. And so once that's done, then,
you know, grow up and running. So the top 1000 is probably more than you need.
Yeah, absolutely. All right. Awesome. I'm looking forward to seeing where this goes.
So many neat options. There's, there's just cool ways to say, like ship the Python runtime
to places where maybe it would have been hard to get. Now you drop this WASM file plus something that
can run WASM. And then now you've got a deployable shipable. Yeah.
See Python runtime without Tkinter and a few things, but still you might not miss it.
It depends what you're doing. I mean, most apps are not Tkinter apps is all I'm saying.
I'm not trying to bang on it.
No, no, but I just haven't. Every time it comes up that it's still there. I'm like,
really? We still have that. Okay.
Don't ask me what I've been spending my week working on, Brian. It's not going to make you happy.
Are you, are you creating a Tkinter base killer for against textual?
No, no, unfortunately not.
Awesome. All right. Well, Brian, are we at extras?
We are at extras. Do you want to kick us off?
I will kick us off. So I've got a couple of things that I think are interesting. Let's
start with this one. So we've talked about, oh, oh, my Z shell, right? Yeah.
A lot. Love it. I just came across, realizing that actually this is a Portland company that puts
together the sort of core maintainers of, that. So I just thought it was funny to give a quick
shout out to, planet Argonne. They're not really in the Python space, but they're in Portland,
which I thought was kind of fun. and then what is this? This, next one comes to us, I think
via PyCoders. That's where I got this. Django just reformatted all of Django with black. And I know
I know I was just having a discussion with somebody like, oh, your code doesn't have,
doesn't follow PEP 8. Oh, like, oh, I don't want it to follow PEP 8. Yeah. But if people are
going to use your code, you know, like literally you got to import it, then it probably should follow,
like it should not come up with all sorts of warnings. And so I thought it was interesting that
Django just said everything, make it black. Steve, what do you think about that?
I'm totally on board with, with just using black on everything. I don't agree a hundred percent with
the style, but I agree a hundred percent with not arguing about it. So yeah, it's close enough.
Yeah. Plus, there's enough tweaks like that make it good. Like I really, I'm really grateful
that that has, you can tweak the line length for instance. Yes. Because I mean, here's example.
What if I want it really short? So, for no, for seriously, for formatting the code for the,
the pie test book, I wanted them all quite a bit shorter so that they fit better in a book format.
And I could use black to cover with that and convert everything with, with black to make them like that.
So it was great. Nice. Awesome. All right. And the final one is I have been doing some stuff with,
more fun things on YouTube, trying to put these little short videos together. So here's
a, how long are the six minutes, 44 second video on using time Delta to get like,
how many weeks are in some time span that, you know, the cool tricks you can do there.
So people should check that out. That's my latest Python short thing. And yeah, that's,
that's it for my extras. okay. So I've got a quick one. Just, I've got,
I don't have a graph, something to throw up, but I just, I was looking at looking at the get history of,
um, of, a repo and trying to figure out whether I included one of my coworkers,
uh, branches in it, if I merged it yet or not, things like that. And I was on the command line
and I just learned, I'm like, can I just do this with the command line? Apparently I didn't know this
exists. So apparently a get log --graph just shows you the get graph that your branch
history or the branch graph on the command line. And I didn't know it was there until today.
uh, I started using it, tweeted about it. And then a whole bunch of people said,
uh, Oh, you should use these flags too. That makes it even nicer. So, so it's fun to,
to learn something old as a new thing. And then somebody else told me, how about get K? So,
get K is a, is a graphical browser of your repository that just comes with most
get installs that I didn't know was there. I'm like, do I need to install it? I'll just type it and see what
happens. And it popped up this graphical interface. I'm like, this is great. This is exactly what I
wanted. So get K is pretty cool. I didn't know about that one. I I've seen the command. I've never
actually run it to see what happens. So I was not feeling quite brave enough.
Did it mean get kill or was it something else?
I was just most get commands scare me until I've run them the first hundred times or so.
How about you? Yeah, I got, I got a couple of extras. Okay. Can I get my screen back up there?
there's, I was feeling a little bad about, you know, being a bit self-serving here,
but then Michael just promoted his video series. So I don't feel too bad anymore.
Get it. Get it on, man.
This is the Python 3.11 alpha five download page. And we have a new addition this time around,
which is this windows installer for arm 64. So arm 64 is not a massive, massive platform for windows yet,
but it's growing and, we want to have Python support on it. So the builds have been running in
the background for a while, but we've never actually released it. We're hoping to get it out with 3.11.
That is going to depend largely on, do people use it? Do they love it? Do they hate it?
My experience so far with it has been that it is noticeably faster, on at least on the arm 64
devices I've had access to compared to the Intel devices, which is really, really cool. and
there's, there's like the test suite is, kind of 30 to 50% faster, which is huge, huge really. So,
so I think there's a lot of potential here. I may just have had awesome hardware. I'm not sure it was
a virtual machine, so it's kind of hard to tell. Yeah. but this is fantastic. This is new.
if you have an arm 64 device, like a surface pro X, or there's a couple out there from other
manufacturers, I'm running windows 11 arm on my MacBook pro and through parallels.
I then use please in download and install it and let, let me know how it goes. If you get it
through the windows store, which is currently still not public, you need to get the link from basically
from one of my tweets, to the windows store, you'll automatically get the arm 64 version on arm 64
as well. So this installer is the traditional one. otherwise you get it through the store.
The other thing, which I wasn't going to do, and then I spent a bit of time working on this,
uh, a couple, couple of years back at the, at the core dev sprints, I forget who I was chatting
with. I was chatting with one of the other core devs about everyone typing from collections, import deck,
uh, and misspelling it. And it's like, you tell, you know, so a deck D Q U E is double ended Q,
very useful data type for certain purposes, but people would type it deck, like deck as in D E C K.
Because it's phonetically what it sounds like.
Exactly. And so as, as a bit of a joke, I made a package that when you, when you installed it,
it would give you from collections, import deck. and obviously the thing that that collection
should be is a, a deck, a double ended Q of 52 cards representing the cards in a normal deck.
and over time for various reasons, it's just kind of grown. And, and I recently, you know,
added support for calculating poker hands to it. And so now you could build a game with this.
it does, it uses enums. It's got shuffling, dealing, jokers are optional. and you can
calculate a poker hand and yeah, my, my little compare them poker hand one greater than poker hand two.
I, I spent a lot of time. most of my work on this over the last week was writing the tests
that proved how incorrect that function was until I wrote the tests for it. but now at this point,
yeah, it's, yeah, you can look at the values it gives back. It's actually a tuple with an enumeration
saying what the hand is and then a selection of the card values in a way that makes the tuples
comparable. So you can actually look and see, you know, it's a pair of, of aces that will have the
number 14 there for the ace. And the next highest card was a 10. So if someone else has a pair of aces
and their next highest card was a nine, then you're still going to compare higher. So I'm pretty proud
of that function. Yeah, that's clever. But, but yeah, this is, and, and,
and it's code style black. Nice. Oh, very nice. So yeah, it's, it's one short file. and it does
still override deck in the collections module for you. So I love it. It doesn't over. So, so that deck
isn't there, right? It's like DQ is, is untouched, but if you try and import DCK from collections,
then you'll get it. Nice. Hey, one of the quick things to shout out, we're hiring contractors
to help develop features for pypi.org. It says at the top of pypi.org. That's, do you know anything
about this? I guess if people want to work on pypi.org, that's pretty neat. Yeah, no, they, they have
funding and, there, there is a post that describes the surveys. I believe this is the organizational
accounts project they're looking at. Yeah. Organization accounts. So, if, if like me,
you are kind of the, one of the primary Python people at your company, then you'll spend a lot
of time helping people publish packages to pypi. If that's the business you're in certainly is for,
for us, there's a lot of packages from Microsoft up on pypi. and the kind of corporate
account for that is, but it does exist. We have a user account with 483 projects. This is all manually
curated right now. cause pypi just doesn't have the functionality to kind of hand out permissions
to it safely. The teams and all that kind of stuff. Yeah. Yeah. So, so I believe the idea of this
is to add that functionality to pypi. So I would love it if someone comes along and does this. I believe
we've contributed some of the funding towards this. So, you know, yeah, it looks like it.
Okay. Oh, so Steve, I've got a 3.11 question for you. So, 3.11 is an alpha. So what does,
what does that mean really? does that mean I can like start using 3.11 or should I wait?
it, it means you can, it means we still may change stuff that will break you and we won't
apologize. Okay. But if my code runs, can I trust it or? Yeah. If, if you're, if all of your tests pass,
then you should be, you should be out of trust it fairly well, certainly existing code.
there will be new features available in the alpha that have not been as thoroughly tested yet or may
change again. But again, if you're running existing code, you won't be using those. So, so that won't
matter, but yeah, it's totally viable to use. You can specify 3.11 dev on GitHub actions. I believe it
compiles from source when you do that. Now, they don't have a build there. They should for beta.
beta is when we really want people to start doing stuff at this point. alpha is so that
people can test the new features kind of targeted testing on anything new that we've put in beta is
when we really want people to start, porting libraries, especially kind of the core libraries to
be able to work with it. and, and just test it because if existing code doesn't work on the beta,
we want to hear about that so we can fix it in the runtime and not force you to fix it in your code.
Okay. But if I'm like a package maintainer, I can start, if it's got GitHub actions for it,
I can start testing, having my CI test against 3.11 then two.
Absolutely. Okay. You will likely want to market as it's okay if it fails.
Okay. Yeah. Awesome. Okay. Thanks. should we do a joke?
Let's do a joke. Let's, let's do a joke. All right. So this one coming from the programming humor
one and it's a, like you talked about the visualization stuff, right? And this one,
it says there's a search that says how to get labels on MATLAB bar charts to be horizontal.
Well, look what the results came back from Google was says you're not alone. Help is available. If
you're experiencing difficult thoughts, please call one one six dash one two three, or if you,
this is an emergency called nine nine nine. And the underlying bit here is it isn't that drastic
Google, but thanks. And I believe it might also work on being, I'm not sure.
There's a few scrolling on. I think there's a being equivalent down here. Yeah. Not just Google
being thinks you're an emergency as well. yeah, that's awesome. Yeah. So it's not that,
that, that much of an emergency. I'll go to stack over for it.
Nice to know that the big search engines are looking out for our mental health.
That's right. People become very upset after failing to get those bar charts.
This is not the emergency, but it's coming next when you realize what the answer is.
It's something that I don't know.
Nice. All right.
Anyway, that's the joke I found for us guys.
Well, thanks everybody. Thanks, Steve for coming and thanks Michael again.
Yeah. Thanks for having me.
Thanks all. Bye. Bye.
Bye.
Bye, everyone.
Thanks for listening to Python bytes.
Follow the show on Twitter via at Python bytes.
That's Python bytes as in B Y T E S.
Get the full show notes over at Python bytes.fm.
If you have a news item we should cover, just visit Python bytes.fm and click submit in the nav bar.
We're always on the lookout for sharing something cool.
If you want to join us for the live recording, just visit the website and click live stream to
get notified of when our next episode goes live. That's usually happening at noon Pacific on Wednesdays
over at YouTube on behalf of myself and Brian Okken. This is Michael Kennedy.
Thank you for listening and sharing this podcast with your friends and colleagues.