Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to
your earbuds. This is episode 163, recorded live on location here at the Portland West Python
Meetup. Hello, everyone. And it was recorded January 7th, 2020. Brian, it's 2020. Yeah,
was that difficult to remember? It's, you know, I'm really not used to it. I just got used to
2019. So we're having problems, but we'll get there. Yeah, well, it's really great to be here
with everyone. This is the first time we've done a live recording in a while. We do this at PyCon a
lot, but PyCon's not that frequent. So here we are in Portland. It's great. Yeah, thanks. And thanks
for everybody for coming out. This is wonderful. Yeah, absolutely. I've got the first one, right?
You've got the first item. So some Zen. I think it's a new year. It's a new decade. Let's kick it
off with a little Zen. Yeah, a little Zen. So we're going to take 20 minutes and just meditate. So the
first one I want to talk about, there's probably going to get this name wrong. Why did I pick this?
Zadka. It's a pretty cool name, though. I wrote an article called Meditations on the Zen of Python.
And if you don't know about the Zen of Python, hopefully you do. We're going to put a link in
the show notes. But there's also in any REPL, you can say import this and it will show you a whole bunch
of little cones, a little snippets of the Zen of Python. One of the cooler things you can do in Python,
second only to import anti-gravity. Yeah, that's good. So Moshe says that this is a,
the Zen of Python is not the rules of Python or the guidelines of Python. It is full of contradictions
and illusions. It is not intended to be followed. It is intended to be meditated upon. So you can read
these and think about your code. And actually, I kind of like it, like that idea, because when I've read
through them, there are some that contradict each other. I want to bring out a few of them that he
points out. The first is beautiful is better than ugly. And one of his comments is that consistency
helps. So things like Black and Flake 8 and PyLint are very useful in making consistent looking code.
Right. But those are only table stakes, right? It's just not ugly. It's not beautiful because of that.
Right. So even more important, only humans can judge what humans find beautiful. So code reviews and
collaboration and a collaborative approach to writing code are the only realistic way to build
beautiful code. So listening to other people is an important skill in software development. And also
just looking at code and seeing if you think it looks nice. I think it's good. So a couple more.
Complex is better than complicated. And that one always like threw me. I wasn't sure why that was in
there. Moshe says, when solving a hard problem, it's often a case that no simple solution will do. In that
case, the most Pythonic strategy is to go from the bottom up, build simple tools and combine them to solve
the problem. That's kind of a Unix style. So is that how you view the complex is better than
complicated?
I don't know if I've ever thought of in these terms. And I do like this article because it did make me
think like that. But I certainly think about software that way. I feel like so many people get caught
trying to overthink it. And they're like, oh, I can't get started. I'm trying to like get the right,
the exact right thing before I can write my first line of code. And it's like, no, you're never going to
know until you work on it for two days. Then you're like, oh, I should have done this. But well,
you didn't know what you knew now. So what are you going to do? Just get started. Yeah. It's my
philosophy. And the last one I want to talk about is readability counts. So in the face of immense
pressure to throw readability to the side and just solve a problem, the Xenopython reminds us that
readability counts. Writing code so it can be read is a form of compassion for yourself and others.
And one of the reasons why I highlighted this is we're going to talk about optimization and speed later
on. And speed is great, but making sure that your program is readable and maintainable is very
important. I really liked this article. Well done because we've all heard about the Xenopython.
We've probably all typed import this, but it's a little vague. And this is not, here's what it
means. This is, here's, here's me trying to think about it sort of philosophically. And, you know,
I've never seen anyone write that way about it before. And I thought it was really cool.
I'd love to other people to like come up with their ideas about it. So that'd be cool.
Yeah, absolutely. So Python and the web doesn't usually have like a James Bond sort of
places getting raided by the police, secret service, international angle to it. But this
next item does. Really? Yeah, it does. So you know what the most popular web server is in the
world? Used to be Apache. Now it's Nginx. Our stuff runs on Nginx, for example. However, there
was some news a few weeks ago. The Nginx offices in Russia were raided by the Russian police and
some of the core developers were detained. And the reason is, this is not as interesting as I made it
out to be, I don't really think. But the reason was, the guy who created it, I have his name here,
Sisyov, he created it. And he, at the time, he was working for Rambler.ru. And Rambler.ru is like a
Google Yandex type of company search engine in Russia. And he worked on this in his spare time. And he
open-sourced Nginx and then later turned it into a company. Well, Rambler came along and said, hey,
you know what? You worked on that while we were employing you. Nginx is ours. We're taking it over.
Meanwhile, Nginx has been bought by F5, an American company, and they own it. And so there's all this
intrigue around it. And yeah, so that happened.
So a bunch of my friends from Spokane that, when Agilent closed down there, went to work for F5,
and I'd never even heard of them before. And then they show up in this news article.
Yeah.
It's pretty interesting.
Yeah, they're all about the networks, but apparently servers. So I received an update,
an email from Nginx a few days later. And I'll just read it so I get it right. They said,
promptly following this event that I just described, we took measures to ensure the security of our
master software builds for Nginx, Nginx Plus, Nginx WAF, and Nginx Unit, all of which are stored on
servers outside of Russia. No other products are developed in Russia. F5 remains committed to
innovating with Nginx, Nginx Plus, et cetera, et cetera, all the products. And we continue to provide
the best-in-class support. You can expect that to come. So who knows where this is going to go?
That's pretty interesting.
If you use Nginx, should you worry about this?
That was why I brought it up. Because if it's the most popular web server in the world,
some folks, their ears are going to perk up and say, wait a minute, what? I mean,
I don't think this is like Russia trying to impose its will on the world. I think this is just a
dispute between a Russian company and some Russian folks who created Nginx, but it could have knock-on
effects quite far. So yeah, it's pretty interesting. I think just keep an eye on it, really.
This episode is brought to you by, well, us.
Really?
Yeah, normally it's brought to you by other companies, but this time, you know, we both
have interesting things to talk about and we have a gap. So I just want to tell you about
some of our stuff. So if you want to check out the courses that I'm creating or, you know,
set up the business stuff, just visit pythonbytes.fm/biz. And here there's something about
testing over at pythonbytes.fm/pytest. And people can check that out there as well.
Cool. Did you set up a URL redirect from there?
I don't want to say the whole name of the whole domain name and URL. So yeah, that's yours.
It points to Python testing with pytest, which I was published in 2017, but I still think is the
very best way to get up to speed very quickly on pytest.
Yeah, absolutely. And you also, we also have a Patreon that you set up at pythonbytes,
at patreon.com slash pythonbytes. So I have some thoughts on this next one, but why don't you
kick it off? This one's from the creator of Flask, but not the current maintainer of Flask.
Oh, that's true. Right. So I brought this up because I was curious what your thoughts were.
So the next one is from Armin...
Armin...
Armin Ronecker.
Armin Ronecker. Thank you. He wrote an article called,
I'm Not Feeling the Async Pressure. The idea is like, async is all the rage. We've actually talked
about async quite a bit on the podcast. And I think a lot of people are concerned about it. And
it's one of the reasons why it's going in place is because I think there's some pressure of people
to leave Python to go to Go or other things.
Node.js was an early example of that, yeah.
Yeah. But before you go towards async, Armin is warning people to make sure you understand flow
control and backpressure. And I had never heard of backpressure, but backpressure is the resistance
that opposes the flow of data through a system. Backpressure sounds quite negative, but is here to
save your day. If parts of your system are async, you have to make sure the entire flow through the
system doesn't have overflow points. And then Armin goes through an example with a reader and writer,
and it seemed like simple code, but it really got hairy really kind of fast. And the example,
yeah, got hairier than I expected. And he says, async brings you new foot guns. Async and await are
great ways to encourage writing stuff that will behave catastrophically when overloaded.
For you developers of async libraries, here's a new year's resolution for you. Give backpressure and
flow control the importance they deserve in documentation and API. And there's just some
hidden things within buffers and things like that. So yeah, well, this is a pretty nuanced article
and it's pretty interesting. It comes from someone who knows a thing or two about the network programming,
Armin being the original creator of Flask. That said, my reaction to reading it,
my reaction was there were a lot of examples of, and here's if you overpressure the system when you
write an async system, it will fall down, right? Imagine you only allow 50 database connections and
suddenly you get 10,000. My sort of reaction to this was, well, what is the response of the system
going to be when it's single threaded and you give it 10,000 connections requests? They're all going to
time out except for a call. It's just going to crash.
Yeah.
So does it crash in an async world or does it crash in a non-async world? With enough traffic,
yes. But at the same time, if you can write basically the same code, put async and await in
front of a few bits and you can get 10 or a hundred times the amount of performance out of a given piece
of hardware before it crashes, that seems better to me. So I mean, I'm sympathetic to the problem,
but at the same time, it's always like, well, if we give it way too much pressure,
it's going to crash. Well, if it wasn't parallel, it would crash before then.
My thoughts were like, can you, are there ways to throttle? Because I don't know enough about all
the way to do network stuff. So if I'm setting up a web service, for instance, can I throttle the input
to say to not allow 10,000 connections and just allow 5,000 or something?
Right, right. Potentially. So maybe with something like UVicorn or something, you could set that up.
I honestly don't know. It seems to me the danger that he's addressing is when the system itself is
generating the input. Like we had this example of a guy who sent us a message and said, Hey,
I had this web scraping thing. It was really slow. We turned async loose on it and it crashed the server
because it ran out of memory because it processed it too quickly, right? Like there, you need to find
a way to slow it down. But when you're running a web server, you don't control how many people want
to get to it. There's just people want to get to it and they either can or they can't.
And with async, more of them would be able to than otherwise. That's my thoughts.
Okay. And I guess that my thoughts would be if you're going to throw async at a problem,
make sure that you do capacity testing on it as well.
Yeah. Well, it's going to fail somewhere else, right? And so you're going to,
maybe your database isn't set up for it. Maybe your microservices can't handle all the traffic.
Like there's going to be something, right? It's just going to show up somewhere else. But
in general, I think you're going to get better scalability with it than without it. So
if you're not generating that pressure, if you're not generating the traffic, then I don't know what
choice you have. Okay. But that's my thought. It was an interesting read.
Yeah, it definitely was. It definitely was. Let's go for something a lot simpler than like a deep
thing on streaming and buffers and async. How about a new way to time your code?
Yeah, that sounds good.
Yeah. So this one comes to us from Doug Farrell, who works on the RealPython team. And as part of their
work, they've got to time all sorts of things for their articles. You know, Dan Bader and crew over there.
And so they came up with this thing. Either they came out of RealPython,
I think, or possibly they were just using it a lot. But this thing called code timing.
So if you've got some code and you want to know how long it takes to run, you could say, you know,
create a time, like capture the daytime, do some work, capture dot now again and do a delta.
Or you could even use performance counters and other things. But you can use this library.
You just, there's a bunch of different things you can do. You create a timer class and you can call
start, do some work, stop. And out comes the time. Or you can put it in a context manager,
like a width block. Soon as it goes to the width block, when it's done, that bit is timed.
Or you can use it as a decorator. And you can also wire up a logger, which is kind of cool.
So you can see like, it'll just output standard Python logging with time information of when it's
doing bits of its thing. Give it a name and it'll tell you how long it took.
Cool. Well, they should add statistics too, so I can get min, max, and average, and standard deviation.
Yeah, that would actually be cool. Well, it's, you know, open source, I bet they accept PRs.
Actually, there's a bunch of features I want to add to it. And I started messing around with it.
And I'm like, put it down. I have other things to do. I'm already too busy. I don't need this.
I'm going to leave it alone. But yeah, it's a pretty cool little timer class. And I'm going to
probably use that. I like it.
Yeah. I thought this, a nice follow-on for this, the timer would be an article called
Making Python Programs Blazingly Fast.
That sounds good. We all want that.
You need to time stuff. You should never, I mean, hopefully we've all learned that
premature optimization is one of the most horrible things you can do as a programmer.
What I've learned is it's incredibly hard to guess where something is slow. Even if you know,
this takes a second, you look at the code, I'm like, oh, I bet it's there. Like, no, no, no. That's
like 5%. It's over here.
Yeah. Because it's super hard to know.
You're throwing the first version of the rough draft of your code down and you write something down and
you go like, I know I can do this better, but I'm going to just make it work.
here. And you know, you're going to have to optimize that. And it turns out to not be the
slow part.
Right. Exactly.
Yeah. So you need to find out where the slow part is. So this article called Making Python
Programs Blazingly Fast by Martin Hines goes through a few things. He doesn't cover this timer,
but there's a few other ways you can time it. There's a, you can use the command line time
function to just time how long something runs.
That might work. You might just go, I made a change. It was five seconds. Is it more or less?
Yeah, exactly. That'll tell you.
Python-mcprofile can tell you a whole bunch about your program.
Do you use cprofile much?
I don't really.
Yeah. I've used it. So it's, it's pretty awesome actually.
Yeah. And then he goes through an example of writing a wrapper function for a timer,
which is similar to what this last article was.
It's one facet of GoTimer.
One of the things that he doesn't cover is the time it function within that's built into Python,
which allows you to just run a single function a whole bunch of times and then tells you how,
how long that takes.
Yeah. Then you get your average and your deviation, all that.
But then the article goes through how to make things faster. So once you've found the slow parts,
how do you make it faster? And these are some good suggestions. Use built-in types over custom types.
Use caching and memoization through LRU cache, which is a built-in thing into Python.
Local variables and local aliases while, when looping. This is something I didn't expect. This
is something like if you've got a multiple dot, dot, dot, dot, something into a function call,
creating a local copy of that makes things run faster.
Every traversal of that dot is expensive in Python, whereas like C++, not so much.
Yeah. Especially if it's in a loop. So I use functions. I don't understand why this was there.
Kind of duh, but you know.
Well, apparently the variable lookup in a function scope is faster than a global variable lookup or
something like that that he was talking about. So by forcing all the variables into the function
scope, they actually come out faster. So there's all these like little weird edge cases.
Yeah. I don't have any code that's not in a function. So don't repeatedly access attributes
in loops. Okay. There's a similar sort of thing. One of the things I didn't realize is the F
strings have been tuned to be very fast. So if you're doing string formatting, use f-strings over
other things.
How many of you out there are using f-strings these days?
Right on. Like that's five, 10 times faster. I don't know. There was a thing by Raymond Hedinger
that's mentioned in that article.
Yeah. So it's way, way faster than the other ways that it was awesome.
And then use generators because I added at least experiment with generators because generators
are really about saving memory. But if you're really dealing with some large data, memory saving
could result in speed up. So I would say throw those in and see if it's faster.
As soon as you start hitting that page file, you're done.
I love generators. I throw them everywhere.
I do too.
Anyway, I think this was an interesting article on speeding up Python. And I warned people against
a premature optimization though. So, but it's fun.
Perfect. Yeah. This is a really good one. I like it. And it's a good follow on to the
other ones we have. Brian, yeah, you're here. So you spoke about CDK, the cloud development
kit from AWS. One of the big gives I have with working with the cloud is I work from home.
I want to go work in a coffee shop. Maybe I'm traveling. I want to work from the hotel and
the internet's bad. I still want to be able to work on my code and know the internet is not
available. Whoops. I guess my app won't run anymore. Right? Well, that is a problem, which
I mostly solve by avoiding the cloud directly. But there's another cool project called local
stack. Talked about Moto before, which is a way to like mock out AWS. And this is actually
built on Moto, but it actually does quite a bit more. This comes to us from Graham Williamson
and Jan Gazda. So thank you both for sending this in. And basically what it is, is it's
like a local AWS, not just a little bit, like a lot of local AWS. It has S3. It has SES for
simple email. It has EC2. It has DynamoDB as Lambda has elastic search, all of that stuff locally.
He showed us like tons of a huge list though. They have all that stuff.
It has a bunch. I don't know how many it has, but I would say it's somewhere on the order of,
I don't know, 2025 different services.
Probably the most common ones.
Yeah. Yeah. Probably the most common ones. And then apparently there's also like some kind of pro
thing I've not used, but then you get more services if you buy the pro version, but the lesser one,
I guess is open source, which is pretty cool.
That's neat. That's great. Like if you're on an airplane or something.
Yeah. Or you just, you want to have like a little local dev environment.
You don't have to pay for that, even though it's less than pennies, but yeah.
It depends what you're doing, I guess. But yeah. So yeah, basically it brings together some of these
tools. It brings together Modo. It brings together this thing called Dynalite and puts sort of a
unifying layer on top of it. So it's pretty cool. A lot of it runs in Docker. It helps to kind of
get a repeatable experience there.
That sounds neat.
Yeah, absolutely. All right. Well, that's it for our main items, everyone. Got any extras you want to
share with folks?
Well, I just, I don't know if we've covered this before. I saw an advert for the Python job board
on python.org.
Yeah. Yeah. I saw that. I hadn't seen it before, but yeah, apparently there's the, there's now,
yeah, we're joking around. We're laughing because the internet's not quite cooperating. That's fine.
We don't need it. Who needs the internet? What did I say about the cloud? What did I just say,
Ryan? You better hope you're not trying to test something right now. Carry on. Yeah. So the job board is
cool. Right. I hadn't noticed it either, but it's on python.org.
I don't know if you have to pay for stuff, but you can just list jobs. So that's cool.
Yeah. And Python's in demand. People want to have jobs writing code in Python, right? Like,
you know.
Do you have any extras for us?
I do have a couple. Let me pull these up here for the audience as well. So I have pictures
because some of these are very, very graphical. So there's this really cool one. You never asked him
probably heard of the guy who created Python. So he was interviewed, he's Dutch. He was interviewed by this
Dutch newspaper about programming. And the title, my Dutch is a little off, but it's like Python is
half my life. Right. So I worked on a Python for half my life or something like this. And they said,
we're going to, this is like a developer thing. Let's put some code and show some Python.
Yeah.
What code do we see there?
I don't know. Is it JavaScript?
Document.getElementByID. Yeah. Not so much. Not so much. That was a pretty interesting little
thing that actually happened. The next one that's pretty interesting. I don't have a picture for it.
It would just be like a bar that's rusting or something, but no, it's pretty cool.
Though Microsoft, you know, they're all about C and C++, right? Like Windows is based on C and C++.
They are actually been doing experimentations with Rust and they're coming out with a Rust
based programming language for like rewriting things like Windows and Rust. That's a pretty big
jump for Rust. And the reason is Rust is especially good at memory management and memory ownership. So
things like buffer overflow vulnerabilities and stuff just go away in Rust, which is like,
you know, every first Tuesday, here's the seven buffer overflows. They're going to like lose all
your data if you don't patch by the next two days that you get. And like, they're trying to avoid that,
I'm guessing. So that sounds interesting.
Do you know Rust yet?
I've looked at Rust. It looked like C. I said, I'm going to go back to Python.
I mean, not exactly, but it looked like C-ish.
Maybe I should take a look at it.
Yeah, it's pretty interesting. Two more quick things. So I'm doing a webcast of Python for
the .NET developer, kind of interactive one hour thing at Crowdcast on the Crowdcast platform. I
think that'll be fun. So links in there. It's free to sign up. People can check that out. And then
Reuven Lerner was talking to him today and he has a new free course that he released called
Ace Python Interviews. So people are out there looking for a Python job. Here's like 50
little exercises and questions answered live and like live coding responses to 50 interview questions
that are explained.
Reuven's a really cool guy. So I think that'd be cool to look at.
Yeah, yeah, absolutely. It looks really good. And that was also free. So no harm, no foul there.
People can check that out.
And I've got a job opening. So if anybody's looking, I'm mostly, last time I interviewed was for Python
person. So I'm probably just going to take some of these things and convert them to C++.
So, you know. So if you want to pass Brian's interview, it may be a good idea to take this
course. Don't tell my boss.
Yeah, yeah. All right. So are we ready for a joke? We always end up podcasts with a joke.
Yeah.
This one's very visual. So I'm going to put this up on the screen for you as well.
And this is really like a sort of infographic. I'm a fan of infographics. And this one helps you
understand like the different types of jobs in software development, which can be very tricky,
right? Like what is the difference between a lead developer and a full stack developer and a coder?
Well, here we go. So it says, there's this person and pretty much it's the same looking
person for every job description. And it says there's a coder. There's a little caption that
says he writes. Software engineer, he writes code. Lead developer, he writes code. DevOps,
well, he writes code. Infrastructure is code, right? Data engineer, actually, what does he do?
He writes code.
He writes code. Full stack developer, he writes code alone.
Computer programmer, he writes code too. Sysadmin, he writes, this is actually a guy eating donuts
with a big beard and looking very disheveled. It says he writes, in fact, we don't really know
what he does. All right. Well, that's the joke for Dave. I guess that's the podcast for today as
well. So thanks a lot. Yeah. Thanks a bunch. And thank you everyone.
Bye.
Thank you for listening to Python Bytes. Follow the show on Twitter via
at Python Bytes. That's Python Bytes as in B-Y-T-E-S. And get the full show notes at
Pythonbytes.fm. If you have a news item you want featured, just visit Pythonbytes.fm and send it our
way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Okken,
This is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.