Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to
your earbuds. This is episode 192, recorded July 22nd, 2020. Had to look that one up.
I am Brian Okken. And I'm Michael Kennedy.
And I can't believe we're heading close to 200. This is crazy.
Oh yeah. Been at this for a while. That's going to be like four years almost.
Yeah. Again, this episode is sponsored by us and we'll tell you a little bit more about
other things that we're doing a little later in the show. But first, some of the ways that people
tell each other what they're up to is their personal GitHub readme on their GitHub profile,
right? Yeah, that's right.
So I was impressed by something that I saw recently. Simon Willison, he's the co-creator of Django.
He posted something called a blog post saying how to do a building a self-updating profile readme
for GitHub. So at the top of it, I'm going to quote this. It says,
GitHub quietly released a new feature at some point in the past few days.
Profile readmes. This is news to me.
Yeah. So if you create a repository with the same name as your GitHub account.
So in Simon's case, it was Simon W. So github.com/Simon W slash Simon W.
So two, you go too deep and then add a readme.md or readme markdown file to it. GitHub will render
the contents at the top of your personal profile page. So that's neat. In that case, it's just one
up. So if you go to github.com/Simon W, you see his, but his looks really awesome. It's got a whole
bunch of cool stuff in it because he took it one step further. It's not a static markdown file.
He's got another article that talks about it, but this article here walks through exactly what he
does. And also it's all open source. So you can see his code. He uses a GitHub actions. There's
both a button that he can push to make it happen, but there's also any post to his own Simon W repo
will cause this to happen. But the GitHub actions run, he contributes to a lot of open source
projects. So he takes a certain set of repos that he has and pulls the latest releases and have like
latest release notes using the GitHub GraphQL API. So there's an example of that. There's an example
of using feed parser to pull blog entries off of his blog and an example of using a SQL query to grab,
guess he's got a site called TAIL for today. I learned grabs a few links off of that. So he's
got a three column setup for a read me that is kept up to date using GitHub actions. How cool is that?
That is awesome. Yeah. So normally you go to your GitHub repo and you have your picture
on how many followers you have and whatnot. Some other cool stuff we'll talk about later,
but then has your pinned repositories and that green ish heat map of how frequently you contribute to
various projects or just to get up in general. But now you can have right at the top, you know,
whatever you want to write, which that's pretty awesome. I think I might have to do this.
Yeah. I mean, you still get all that other stuff, but it's just that other stuff is below
this read me info. That's pretty neat.
Yeah. Very cool. And it's super simple, right? Anyone can write a readme.md file.
Yeah. And one of the reasons why I brought this up is I think there's a lot of people trying to
utilize, I mean, this day of COVID and quarantine and stuff, I'm glad I'm not looking for a job.
And I think that if you are looking for a job, making your GitHub profile look professional
and show the content that you want to show off and having things like, you know, blog posts on
your GitHub profile, that's pretty cool. It is really cool. And just, you know, you know that people,
employers say they check people's GitHub profile accounts, right? So how many people have, are going
to have these unique special ones that show they care, right? Not too many.
Well, the people that listen to our podcast.
Exactly. Yeah.
All the awesome people. Okay. So that's really cool. I definitely didn't know about that. Thanks for
sharing that. It looks neat. So we got this next one from Connor Furster and he works in engineering,
but also does data science-y things. And he sent over this project that he works on that is incredibly
cool. So a lot of times what people want to do is they want to take symbolic math or math that you
might write out by hand, turn it into Python code through pandas and numpy and whatnot, scikit learn
or scikit in general, and then run it through Jupyter and get an answer. But he says he works in design
engineering and you have to do a lot of calculations and those have to be kept as part of legal records
to show the project design history. And one, yeah, one thing you do is do them by hand. That's kind of
crazy. A lot of people use Excel. That's a nightmare. Like Excel is like unbounded go-to's you can't see,
which is always tricky. So you could do it with Jupyter, but then you just got this pile of code
and here's the answer and so on. Right. So you want to like the theoretical view to verify the formula
you're using. Right. So he created this thing called hand calcs and C-A-L-C-S like calculus or
calculations. Anyway, hand calcs. And the idea is you type in Python code into a Jupyter cell,
and then you can do a percent percent render from the hand calcs project and it will turn it into
symbolic math. This is beautiful. Yeah. As if you had written it out by hand. Yeah. With like,
as an example in the little video demo, we've said before we like those and everybody does,
but it has, has like square root symbols with a bunch of symbols underneath it and
all sorts of symbols that, yeah. Yeah. Looks like, like what you would have had to show if you were
in math class, right? Yep. Exactly. And it will show steps like symbolic steps from step A, step B,
step C. And you can say, show it shorthand or expand it out longhand and show me all the steps you
use to like solve these problems and all kinds of cool stuff. Wow. Yeah. The reason it looks so good
is it basically converts symbolic Python math over to latex and latex is like the de facto math
representation language for academic papers. So, you know, you want to have like integral signs,
you want to have infinite summations, all that kind of stuff. No problem. Oh, this is really cool.
Isn't it cool? And then you can also use the symbolic tag to get it to do other, like show more
symbolic stuff. You can do longhand, shorthand, you can have it do units. They'll put units like
millimeters cubed or whatever, and it'll carry the units through the calculation in symbolically.
Yeah. But looking at all these formulas, it's giving me nightmares. Don't look anymore. Okay.
Well, I guess the thing you would, you would want to think about the trade-off is would you rather
look at them in their proper mathematical form or in like programming meaning, not, you know,
where you turn it into like star star pow instead of, you know, proper exponents and stuff.
No, no, I was just kidding. This is beautiful stuff. But when we got into integrals, that's
where basically that's where my brain left and I never really caught integrals that way.
Yeah. Yeah. Cool. All right. So if people have to take programming math, but they want to represent
it more nicely, check out handcaps. Looks awesome.
Yeah. Nice. Oh, I'm next.
Actually, I'm not. I'd like to talk to all of us about our sponsor and our sponsor is a
Talk Python Training and a testing code today.
Tell me about Talk Python Training.
I'll tell you about what I'm working on this week. I started writing a new course.
We have a couple of new courses that are fun that are coming. And the one that I started working on
is called Python memory management and tips.
Tell me more.
Yeah. So if you ever wondered, like, what happens, like, how does it free up memory?
What algorithms make, like, work better with Python memory and what algorithms can make it
more expensive or slow? What are some of the tips and tricks you can do to, like, dramatically
decrease the memory consumption, like, two or three times with almost exactly the same code
type of thing? Well, I'm writing a course on that.
Oh, that's neat.
Yeah. Thanks.
Especially for people, like, talking about doing some more, we can get Python on smaller
operating system.
Yeah.
Architectures like CircuitPython and stuff. That's important.
Yeah. That's a really good point that on the small memory constraint pieces, you might care a lot
for sure.
Yeah.
How about testing code?
Well, I was interviewing somebody recently, David Lord. His actual interview will come out sometime
in August. But he said, I was looking at testing code and a lot of the recent episodes really haven't
been about testing. What's up with that? And I said, yeah, it's and code, test and code. But yeah,
so there is a lot of testing focus because primarily because I think that software engineering doesn't
talk about testing enough. But I do cover a lot of stuff. I'm going to highlight a few of the last
episodes. I talked to Sebastian Ramirez on episode 120 about Fest API and Typer. Talked with Brett Cannon
on episode 119 about packaging and PyProject.TOML and what's going on there. 121 is a diversion. It's a
completely different sort of talk. I talked to somebody about 3D printing and finite state machines
and stuff. And it's just sort of a fun people doing Python and cool things.
Very cool.
And then again, talking, thinking about people possibly looking for jobs in episode 122, we talk about
better resumes for software engineers. So there's, yeah, there is a lot of stuff for everybody. Even
if you cringe when you think about testing, please check out testing code. We are still putting out
episodes. If you want to hear more, I'd love to hear what people want to hear about.
Yeah. It makes our job so much easier when we get suggestions.
Yeah. Suggestions and questions and things that can flow into things.
Yeah. Like a suggestion to return the print statement so you don't have to put the parentheses.
Yeah. So this is crazy and I don't really have much of a comment here, but I saw the thing by Guido
and then I saw this article by Jake Edge on LWN.net. I don't know what LWN stands for, but it doesn't matter.
Anyway, the non-return of the Python print statement. So this is odd, I thought. We have talked about the new
peg parser in Python that's going on. But one of the things that happened with that is, I guess, one of the reasons
why Python 2 to 3, they went from a print statement to a print function was it made the parsing easier.
But with the peg parser, you can do all sorts of crazy things and you can have functions that syntactically look like statements
and have it work just work, sort of. So as an example, we could use a print statement instead, instead of having to be, put the
parentheses in, you could avoid the parentheses. Anyway, he just put it out there as an idea and essentially people said, no, yuck. What do you think about this?
It's interesting. It would be one fewer things that has to happen to move to the next stage from a 2 to 3 conversion.
On the other hand, this looks like one of the easiest conversions for that step. To me, I'm not a fan of having statements and functions in the language
because it looks to me like functions basically solve the same problem with a little more
clarity.
They're a little more functional. You can span the multi-line if the arguments are super long
that you need to.
Like with the print statement, you'd have to use a continuation backslash and other weirdness like that.
So, you know, just because you can doesn't mean you should.
I guess that's probably how I feel about it, but
I wouldn't use it if it were in the language. Let me put it that way.
Yeah, I'm for the no yuck camp. I think that print statement shouldn't have been a statement in the
first place. And then I think Python 3 fixed it. Having it be a function is the right thing to do.
I wish there were more statements that were functions instead. Also, I think I wish assert was a function
instead of a statement. Because people doing, thinking that it's a function and putting a parentheses
around assert causes problems. But that's not what this is about. It's interesting. I brought it up
because people should know about this weird, wacky discussion.
Yeah, that's funny.
I'm glad that it got thumbed down. And I don't think it's going to happen.
You're willing to make a statement about it?
Yes.
All right. Well, I'm going to make a statement about Flask. I think Flask, you just had David
Lord on the show, right? That's not out yet, but pretty cool. Yeah. And he's lead maintainer of
Flask these days. So Flask is, at least at the API level, got to be the most popular web framework
there is. Because it's slightly more popular than Django if you look at some of the recent surveys.
But if you look at the other frameworks, many of them are Flask-esque, if you will, right?
Yeah.
Things that are like Responder or Cynic or whatever. They have this idea of like sort of the same style,
right? So there's an article called FastAPI for Flask users. And I'm actually a big fan of
FastAPI. I'm hoping to have some opportunity to use it soon. Like the APIs that I've worked on,
they've been around for a while, they predate FastAPI. And I don't really want to go create a whole
new site just so I could use a different framework. That sounds like maintenance to me. So I haven't
got a chance to use it in production yet, but FastAPI looks awesome. So there's an article called Fast
FastAPI for Flask users. And it says, look, you probably know the Flask API. Here is the equivalent
for FastAPI.
Okay.
Yeah. And so there's talk about some of the advantages and they're pretty awesome. So automatic data
validation in FastAPI doesn't exist in Flask, generally speaking. Automatic documentation generation,
built in best practices like type annotations and Pydantic scheme schemas and whatnot.
It comes ships or recommend, I guess, as terms of like a requirement, you have to have a ASGI server.
So it comes with UVicorn, which is one of the, it's like Gicorn plus uv loop for async stuff.
And in a lot of ways, it's super similar. So if you want to create a view method instead of
app.route, you would say app.get. And so FastAPI, would you imagine the name indicates it's
mostly for building APIs?
Yes.
All right. So when you talk about functions and what they're going to do, you say not just
here's a URL, but here's a URL and an HTTP verb. So app.get forward slash or app.put slash account
or something like that, which is pretty cool. In the route, you can specify variables. So in Flask,
you would, you could have a user ID and in the string route, you would say int colon user ID. If
you want Flask to convert that to an integer, right? It's fine. It works. Okay. But that the rest of the
tooling doesn't help you know it's an integer just because Flask knows it's going to be an integer,
right? So in FastAPI, you put the variable up there as well. But then in the function,
you put the variable name as a type, and then it will actually convert that to an integer using the
Python language tools or specification rather than the string API thing. So that's handy. If you want a
query string in Flask, you just have a URL, you can go to request.args, and you can get the value out of the
query string. In FastAPI, you just put the query string values, or sorry, the keys as arguments, and they just get
passed in. That's pretty cool. Yeah. If you have a API that takes a JSON post, like, you know, it's
accepting a JSON document, you can just say it takes a dictionary, and that gets posted in. But you can go
way, way further, which is awesome. You can define a Pydantic model, which is a class that has types
and validation on the class, right? Yeah. And then you can say my view method or my API method takes,
like in the example they have is a sentence that has got like various components, nouns, verbs, and
whatever. You can say, here's a function, and it has an argument called sentence, and it'll take that JSON
document, parse it into the Pydantic model, and pass it to you pre-validated.
That's definitely one of the benefits of FastAPI is this data validation.
Yeah, this data, this is like built-in data validation, because how much, how many times
do you spend like effort, oh, I got a string, but I got to convert it to an integer. I got to make
sure that this value is here. I got to make sure that this one is, you know, like, whatever, like it
matches some, some set of sub strings or whatever. Just let, let the framework handle it. It also has
the equivalent of blueprints, which it calls routers, and this automatic validation I talked about. So
anyway, there's a nice article that says, you know Flask, let's teach you FastAPIs real quick by just doing a
this equals that.
Yeah, I love this, because there's a lot of people that have been writing APIs in Flask for a long time. And so it's
just second nature to them. And having something to say, hey, I want to try FastAPI, but is the learning curves going to be
a problem? Well, with something like this is decoder ring. And it is set up for, you can just sort of skim
through and go, well, how do I do URLs? Oh, this is, this is how you do it. And URL variables and
different things. It's set up really nice.
Yeah. Yep. Definitely fun. Definitely useful.
So do you use Twitter?
I do use Twitter. Sometimes happily. Sometimes I get dragged into stuff. Sometimes I use it in write only
mode where I want to make a statement, but I don't really want to go read it. But yeah, definitely.
Yeah, well, I have a, this is probably common, sort of a love-hate relationship with Twitter. I use it a lot
and like keeping track of other people. But sometimes I don't really like that it's a pain to delete old
stuff. Because I think of it as a current conversation. I don't really look at what somebody wrote a year
ago. And I don't really care what I wrote a year ago. So I have used Twitter deletion tools before.
They seem kind of weird that I have to go out and give my credentials to some other website or
something though. So, but I know.
I'm sure that'll be fine. It'll be fine. Don't worry about it.
Yeah. Anyway, but there's APIs. So you could use the Twitter API, but how? And so I thought it was
really cool that Chris Albin is somebody that tweets about data science a lot. And he posted a little
snippet that he said he uses. At least he did it first at one point. But it's a cool little example
of using a library called Tweepy to interact with Twitter and to delete old tweets for your account.
So it's just this really short little Python script. But it deletes tweets. There has defaults, but you
can change those. Obviously, it's just a Python script. So you can change whatever you want.
But it's set up to delete tweets that are older than 62 days and that have likes less than 100 people
and that you haven't liked yourself. So the idea being, if you go through some of your old tweets
and the ones that you're like, oh yeah, that was cool. I want to keep that around. Just like it,
like your own tweets and then run this script and it'll delete some old stuff. I would definitely
have to change that 100 count to something else because I don't think I've ever had a tweet liked
by 100 people.
That's a big number. You know, Twitter used to show how many tweets you actually had.
And I don't think it shows it anymore on my profile. At least I don't see it immediately.
How many tweets I had just followers and following and likes and stuff like that. But yeah, pretty cool.
It's like keep the highlights, right? Just keep my highlights. I don't need every random thing of,
oh, I went on, had a hamburger today. People don't need that as a piece of history.
Yeah. So when, and you don't know what's going to stick and what's going to not. And I was actually
reading an article recently about, about Twitter, about how that, what that says to you. If somebody
like, for instance, you're trying to get a job and somebody looks at your Twitter account, having the
junk in there that like nobody really related to having that automatically called out and just having
the highlight reel, that's not a bad idea for some of the old stuff.
Yeah. And you could turn it way down. You could say, look, if there's no likes or no retweets,
just drop it. Yeah. Yeah. It might even be good for me to like, just to go back a couple of days,
but if nobody's liked it in a couple of days, maybe just take it away.
That didn't happen.
Yeah. That didn't happen.
So people could end up clinging to their old tweets, but they probably shouldn't.
Right.
Yeah. So I want to talk about an article by Itamar Turner-Trowing. Now we spoke about him,
sort of, not by name, I don't think, but we talked about Phil, the data science memory
profiler a little while ago.
Okay. Yeah. Right.
So he's the guy who wrote that. I actually had him on Talk Python on episode 274 as well,
talking about that. So that was pretty cool. But he, independent of that, he wrote this article
that I came across that I liked called Clinging to Memory, How Python Function Calls Can Increase
Your Memory Usage. And this is part of my research for working on that course that I was talking about,
that memory, Python memory management stuff.
Okay.
So he talks about like, hey, we're going to have this thing. It's going to load up some NumPy data
and then it's going to pass it to a function. The function is going to make some changes,
take the return value of that, pass it to another function. It's going to make some more changes.
So basically three steps and said, look, we'd expect that we've loaded two gigs of memory.
And yet when you run Phil against it, you end up with three gigs of maximum memory usage,
which is a little bit weird. And the reason is those initial like intermediate values that you're
working with on step one and step two, the way Python decides a variable goes out of scope
is in this case, the function returns, not like it's never used again, but it's just the function
returns. In which case it's going to hang on to all the intermediate copies all the way to the end.
Interesting.
Right? Like some languages, they determine that and they get rid of it. Like in C#,
the JIT compiler will notice like, okay, a variable is not used after half the way. So we're going to
make it eligible for GC basically, unless it's in debug mode, then keep it around in case somebody
sets a break point. They want to see it. So there's a lot, a lot of the tricks that things can do.
Python doesn't do them. So it's going to stick around for this
length of the function. So what can you do to make it not stick around as long? Because maybe
you only have two gigs and you don't want to use three gigs or whatever. Right? So he talks about
three different solutions. One is to don't hold onto the intermediate variables and just chain into
one massive function call, like pass the results of one to two to step two, pass the results of step
two to three. And there's no variables holding on. So it'll be gone. Right? That's an option.
Another one is to iteratively change the variable, say like data equals load data from first step.
Data equals step two of processing of data. Data equals step three of processing of data. And that
way you're dropping the reference count to the first, to the intermediate steps along the way.
Right? So that's an option. And then there's a third one that's more complicated about creating
like a sort of a ownership management type of thing that people can check out as well. But I just thought
it was interesting to think about, you know, when, how long do these things stick around and what
techniques might you use that are incredibly simple? Like just reuse the variable name, problem solved
in terms of having too much memory usage.
Interesting. Yeah. When I look at these, they all look kind of like the same, but having,
having the answer be that they, they use different amounts of memory is not obvious.
Right. It's not obvious at all, but it's, you could easily look at this one where you're iteratively
changing the variable and say, Oh, you shouldn't do that. You should name it more clearly. Cause
maybe the type is changing along the way and it would be weird, but you could say, yeah, but
this one works cause it will fit into Ram and the other one won't. So we're willing to accept this
like slightly imperfect code because it works better. Anyway, there's a lot of interesting trade-offs you
can make here, but I just, it's, it's only the tip of the iceberg for things like this you could do,
I think, but it's interesting to just put it on your radar.
Yeah. Yeah, it is interesting. Yeah. And actually, and like we said, I think that more and more as we
start using Python for other applications or non desktop kind of things, like when we're
a non-server things, if we're using it for, there's a couple ends of it. If you're using
small devices, like in circuit Python or something, you're going to care about this stuff. But also if
you're using very large data sets, then we care about it again. And, it doesn't matter how much
memory your computer has having multiple copies of gigabytes of data when you don't have to,
will slow things down. Yeah, for sure. Or even if it's like an API and you just happen to be doing,
it's not that extreme, but you happen to be doing a thousand of them at a time. Same story.
Yeah, exactly. And as we use Python more and more and more applications, we're going to start caring
about that again. Yeah, absolutely. That's the end of our six. I actually have been just so swamped
with stuff. I don't have anything extra to talk about. Do you have any extra items?
I do. And this is just a follow-up email we got from a listener named Adam. Thank you, Adam.
And you had talked about pickling things. Apparently you're a fan of dill pickles and
no wait, pickled strings, pickled lists, pickled dictionaries. No, we were talking about a pickling
and how it didn't make sense most of the time, but there might be some use cases and you're like,
what might be a use case that we really need? Right?
So Adam said, Hey, I got a use case that worked for us. I worked on an API that spoke to a third
party service that was wonky and it was over raw sockets. So you'd have to create these byte arrays
and send them along. And the thing was also not available 24 seven. It would sometimes crash,
things like that. So what they would do is they could set a flag in their app and it would pickle
all the messages that it would, would have sent. And if the site comes back, it can like rehydrate
those things and then ship them along, or you could pull them up for debugging and look at their
details and whatnot. So it was like, Oh, we got to save this exactly as we would have sent it.
Let's just pickle it.
That's pretty cool.
Yeah. That seems like a pretty good one.
Yeah.
And there was a feature flag they could turn on and off, which was kind of cool. Yeah.
They could also do that for the messages they got from the service. Pretty cool.
real quick. Python three, eight, four is out. I've already brew upgraded mine. So that's all good.
And big news, big news. I can't believe it. I've been selected.
For what?
I, if I go to my GitHub repo, I don't have the cool read me thing that you're talking about,
but under my picture, it says I have a pro account because I had to pay for some stuff,
but I noticed that I'm an Arctic code vault contributor.
Wow.
So remember we spoke about the Arctic code vault where GitHub is taking a bunch of the popular projects
and then like sticking them over in some vault in Norway or somewhere like that, Greenland,
to preserve it. And if the code that you've contributed to GitHub was selected, then now
you get this cool little highlight that's like a snowflake that says Arctic code vault contributor.
And you can hover over it and it'll say why. So yeah, I've contributed apparently to a couple
of things and you might be as well.
Well, yeah, you, the listener might, but I just checked mine and I am too. So that's neat.
Yeah. Awesome.
Neato.
Yeah. So.
I think we covered that once.
Yeah.
Yeah. We definitely covered the code vault, but yeah, I think this, you are a contributor
thing. The little badge is new and I don't know. It makes me happier than it probably should.
Yes. It's so cool.
It's cool.
Yeah.
Yeah. It's super neat.
Testing is cool.
I love testing and having good code coverage is cool.
Yeah.
So I've got a joke for you, a cartoon, if you will.
Okay.
From this place called geek and poke.
They have all sorts of good stuff there.
And, you can click on the picture and it'll take you to the actual comic.
So there's a two people, a woman developer and a man developer staring at each other.
And the woman is the more senior one.
They're looking at each other and it says QA best practices.
She's looking, looking at the guy and says, never just remove a failing test.
The guy stares back blankly for a second.
Says, only remove the assert statements.
Yup.
So how to sustain a decent code coverage.
Uh, yeah, you can fix a test.
You can make a test, not fail, remove the assert statements.
It's good.
Yeah.
That's funny.
You said you actually like test for failure though on yours that they potentially could
fail.
Yeah.
Well, I think that's one of the reasons why we do code coverage on all or not code
coverage.
We do code coverage, but we also do a review.
What's the word again?
Review.
Yes.
Code reviews.
Yes.
We do code reviews on tests because we have had a test show up before that exercise.
Um, we, you know, with a test equipment, we do a lot of complicated things.
You set up everything, run some stuff, and then we've often have people forget to check
anything at the end.
And, and the, so it is important to look at the end to see, is there any way this can
actually fail?
Is it, or is it just exercising things?
I mean, actually exercising things isn't a bad thing because you can get a search in
your code or, except, or an exception.
Yeah.
You still test something, but you're not testing very much.
Yeah.
You're testing it runs basically.
Yeah.
So awesome.
Well, yeah, just never remove and fill in a test.
Only the search statements.
It's terrible.
We should not give that idea to people.
No, we should totally delete this joke.
It didn't happen.
It wasn't funny.
Yeah, it wasn't funny.
Thanks a lot, Michael.
Yeah, you bet.
Great to see you as always.
Thank you for listening to Python Bytes.
Follow the show on Twitter at Python Bytes.
That's Python Bytes as in B-Y-T-E-S.
And get the full show notes at Pythonbytes.fm.
If you have a news item you want featured, just visit Pythonbytes.fm and send it our way,
where I was on the lookout for sharing something cool.
This is Brian Okken, and on behalf of myself and Michael Kennedy, thank you for listening
and sharing this podcast with your friends and colleagues.