Transcript #163: Meditations on the Zen of Python

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to

00:04 your earbuds. This is episode 163, recorded live on location here at the Portland West Python

00:11 Meetup. Hello, everyone. And it was recorded January 7th, 2020. Brian, it's 2020. Yeah,

00:20 was that difficult to remember? It's, you know, I'm really not used to it. I just got used to

00:24 2019. So we're having problems, but we'll get there. Yeah, well, it's really great to be here

00:28 with everyone. This is the first time we've done a live recording in a while. We do this at PyCon a

00:32 lot, but PyCon's not that frequent. So here we are in Portland. It's great. Yeah, thanks. And thanks

00:36 for everybody for coming out. This is wonderful. Yeah, absolutely. I've got the first one, right?

00:40 You've got the first item. So some Zen. I think it's a new year. It's a new decade. Let's kick it

00:46 off with a little Zen. Yeah, a little Zen. So we're going to take 20 minutes and just meditate. So the

00:51 first one I want to talk about, there's probably going to get this name wrong. Why did I pick this?

00:55 Zadka. It's a pretty cool name, though. I wrote an article called Meditations on the Zen of Python.

01:01 And if you don't know about the Zen of Python, hopefully you do. We're going to put a link in

01:05 the show notes. But there's also in any REPL, you can say import this and it will show you a whole bunch

01:12 of little cones, a little snippets of the Zen of Python. One of the cooler things you can do in Python,

01:17 second only to import anti-gravity. Yeah, that's good. So Moshe says that this is a,

01:23 the Zen of Python is not the rules of Python or the guidelines of Python. It is full of contradictions

01:28 and illusions. It is not intended to be followed. It is intended to be meditated upon. So you can read

01:36 these and think about your code. And actually, I kind of like it, like that idea, because when I've read

01:41 through them, there are some that contradict each other. I want to bring out a few of them that he

01:45 points out. The first is beautiful is better than ugly. And one of his comments is that consistency

01:51 helps. So things like Black and Flake 8 and PyLint are very useful in making consistent looking code.

01:57 Right. But those are only table stakes, right? It's just not ugly. It's not beautiful because of that.

02:01 Right. So even more important, only humans can judge what humans find beautiful. So code reviews and

02:06 collaboration and a collaborative approach to writing code are the only realistic way to build

02:12 beautiful code. So listening to other people is an important skill in software development. And also

02:17 just looking at code and seeing if you think it looks nice. I think it's good. So a couple more.

02:23 Complex is better than complicated. And that one always like threw me. I wasn't sure why that was in

02:30 there. Moshe says, when solving a hard problem, it's often a case that no simple solution will do. In that

02:36 case, the most Pythonic strategy is to go from the bottom up, build simple tools and combine them to solve

02:43 the problem. That's kind of a Unix style. So is that how you view the complex is better than

02:48 complicated?

02:48 I don't know if I've ever thought of in these terms. And I do like this article because it did make me

02:52 think like that. But I certainly think about software that way. I feel like so many people get caught

02:56 trying to overthink it. And they're like, oh, I can't get started. I'm trying to like get the right,

03:00 the exact right thing before I can write my first line of code. And it's like, no, you're never going to

03:05 know until you work on it for two days. Then you're like, oh, I should have done this. But well,

03:09 you didn't know what you knew now. So what are you going to do? Just get started. Yeah. It's my

03:13 philosophy. And the last one I want to talk about is readability counts. So in the face of immense

03:17 pressure to throw readability to the side and just solve a problem, the Xenopython reminds us that

03:23 readability counts. Writing code so it can be read is a form of compassion for yourself and others.

03:28 And one of the reasons why I highlighted this is we're going to talk about optimization and speed later

03:33 on. And speed is great, but making sure that your program is readable and maintainable is very

03:39 important. I really liked this article. Well done because we've all heard about the Xenopython.

03:43 We've probably all typed import this, but it's a little vague. And this is not, here's what it

03:49 means. This is, here's, here's me trying to think about it sort of philosophically. And, you know,

03:54 I've never seen anyone write that way about it before. And I thought it was really cool.

03:57 I'd love to other people to like come up with their ideas about it. So that'd be cool.

04:01 Yeah, absolutely. So Python and the web doesn't usually have like a James Bond sort of

04:07 places getting raided by the police, secret service, international angle to it. But this

04:16 next item does. Really? Yeah, it does. So you know what the most popular web server is in the

04:21 world? Used to be Apache. Now it's Nginx. Our stuff runs on Nginx, for example. However, there

04:27 was some news a few weeks ago. The Nginx offices in Russia were raided by the Russian police and

04:34 some of the core developers were detained. And the reason is, this is not as interesting as I made it

04:40 out to be, I don't really think. But the reason was, the guy who created it, I have his name here,

04:45 Sisyov, he created it. And he, at the time, he was working for Rambler.ru. And Rambler.ru is like a

04:52 Google Yandex type of company search engine in Russia. And he worked on this in his spare time. And he

04:58 open-sourced Nginx and then later turned it into a company. Well, Rambler came along and said, hey,

05:03 you know what? You worked on that while we were employing you. Nginx is ours. We're taking it over.

05:10 Meanwhile, Nginx has been bought by F5, an American company, and they own it. And so there's all this

05:16 intrigue around it. And yeah, so that happened.

05:20 So a bunch of my friends from Spokane that, when Agilent closed down there, went to work for F5,

05:25 and I'd never even heard of them before. And then they show up in this news article.

05:29 Yeah.

05:29 It's pretty interesting.

05:30 Yeah, they're all about the networks, but apparently servers. So I received an update,

05:34 an email from Nginx a few days later. And I'll just read it so I get it right. They said,

05:39 promptly following this event that I just described, we took measures to ensure the security of our

05:44 master software builds for Nginx, Nginx Plus, Nginx WAF, and Nginx Unit, all of which are stored on

05:52 servers outside of Russia. No other products are developed in Russia. F5 remains committed to

05:58 innovating with Nginx, Nginx Plus, et cetera, et cetera, all the products. And we continue to provide

06:03 the best-in-class support. You can expect that to come. So who knows where this is going to go?

06:09 That's pretty interesting.

06:09 If you use Nginx, should you worry about this?

06:13 That was why I brought it up. Because if it's the most popular web server in the world,

06:17 some folks, their ears are going to perk up and say, wait a minute, what? I mean,

06:21 I don't think this is like Russia trying to impose its will on the world. I think this is just a

06:26 dispute between a Russian company and some Russian folks who created Nginx, but it could have knock-on

06:32 effects quite far. So yeah, it's pretty interesting. I think just keep an eye on it, really.

06:37 This episode is brought to you by, well, us.

06:41 Really?

06:41 Yeah, normally it's brought to you by other companies, but this time, you know, we both

06:44 have interesting things to talk about and we have a gap. So I just want to tell you about

06:48 some of our stuff. So if you want to check out the courses that I'm creating or, you know,

06:53 set up the business stuff, just visit pythonbytes.fm/biz. And here there's something about

06:59 testing over at pythonbytes.fm/pytest. And people can check that out there as well.

07:04 Cool. Did you set up a URL redirect from there?

07:07 I don't want to say the whole name of the whole domain name and URL. So yeah, that's yours.

07:11 It points to Python testing with pytest, which I was published in 2017, but I still think is the

07:18 very best way to get up to speed very quickly on pytest.

07:20 Yeah, absolutely. And you also, we also have a Patreon that you set up at pythonbytes,

07:23 at patreon.com slash pythonbytes. So I have some thoughts on this next one, but why don't you

07:28 kick it off? This one's from the creator of Flask, but not the current maintainer of Flask.

07:33 Oh, that's true. Right. So I brought this up because I was curious what your thoughts were.

07:38 So the next one is from Armin...

07:40 Armin...

07:41 Armin Ronecker.

07:42 Armin Ronecker. Thank you. He wrote an article called, I'm Not Feeling the Async Pressure. The idea is like, async is all the rage. We've actually talked

07:50 about async quite a bit on the podcast. And I think a lot of people are concerned about it. And

07:55 it's one of the reasons why it's going in place is because I think there's some pressure of people

08:00 to leave Python to go to Go or other things.

08:02 Node.js was an early example of that, yeah.

08:04 Yeah. But before you go towards async, Armin is warning people to make sure you understand flow

08:10 control and backpressure. And I had never heard of backpressure, but backpressure is the resistance

08:16 that opposes the flow of data through a system. Backpressure sounds quite negative, but is here to

08:23 save your day. If parts of your system are async, you have to make sure the entire flow through the

08:29 system doesn't have overflow points. And then Armin goes through an example with a reader and writer,

08:35 and it seemed like simple code, but it really got hairy really kind of fast. And the example,

08:41 yeah, got hairier than I expected. And he says, async brings you new foot guns. Async and await are

08:48 great ways to encourage writing stuff that will behave catastrophically when overloaded.

08:53 For you developers of async libraries, here's a new year's resolution for you. Give backpressure and

08:58 flow control the importance they deserve in documentation and API. And there's just some

09:05 hidden things within buffers and things like that. So yeah, well, this is a pretty nuanced article

09:10 and it's pretty interesting. It comes from someone who knows a thing or two about the network programming,

09:15 Armin being the original creator of Flask. That said, my reaction to reading it,

09:21 my reaction was there were a lot of examples of, and here's if you overpressure the system when you

09:27 write an async system, it will fall down, right? Imagine you only allow 50 database connections and

09:33 suddenly you get 10,000. My sort of reaction to this was, well, what is the response of the system

09:39 going to be when it's single threaded and you give it 10,000 connections requests? They're all going to

09:44 time out except for a call. It's just going to crash.

09:46 Yeah.

09:46 So does it crash in an async world or does it crash in a non-async world? With enough traffic,

09:52 yes. But at the same time, if you can write basically the same code, put async and await in

09:57 front of a few bits and you can get 10 or a hundred times the amount of performance out of a given piece

10:02 of hardware before it crashes, that seems better to me. So I mean, I'm sympathetic to the problem,

10:08 but at the same time, it's always like, well, if we give it way too much pressure,

10:13 it's going to crash. Well, if it wasn't parallel, it would crash before then.

10:16 My thoughts were like, can you, are there ways to throttle? Because I don't know enough about all

10:21 the way to do network stuff. So if I'm setting up a web service, for instance, can I throttle the input

10:27 to say to not allow 10,000 connections and just allow 5,000 or something?

10:32 Right, right. Potentially. So maybe with something like UVicorn or something, you could set that up.

10:36 I honestly don't know. It seems to me the danger that he's addressing is when the system itself is

10:43 generating the input. Like we had this example of a guy who sent us a message and said, Hey,

10:47 I had this web scraping thing. It was really slow. We turned async loose on it and it crashed the server

10:53 because it ran out of memory because it processed it too quickly, right? Like there, you need to find

10:57 a way to slow it down. But when you're running a web server, you don't control how many people want

11:01 to get to it. There's just people want to get to it and they either can or they can't.

11:04 And with async, more of them would be able to than otherwise. That's my thoughts.

11:09 Okay. And I guess that my thoughts would be if you're going to throw async at a problem,

11:14 make sure that you do capacity testing on it as well.

11:18 Yeah. Well, it's going to fail somewhere else, right? And so you're going to,

11:22 maybe your database isn't set up for it. Maybe your microservices can't handle all the traffic.

11:25 Like there's going to be something, right? It's just going to show up somewhere else. But

11:29 in general, I think you're going to get better scalability with it than without it. So

11:33 if you're not generating that pressure, if you're not generating the traffic, then I don't know what

11:37 choice you have. Okay. But that's my thought. It was an interesting read.

11:39 Yeah, it definitely was. It definitely was. Let's go for something a lot simpler than like a deep

11:44 thing on streaming and buffers and async. How about a new way to time your code?

11:49 Yeah, that sounds good.

11:50 Yeah. So this one comes to us from Doug Farrell, who works on the RealPython team. And as part of their

11:56 work, they've got to time all sorts of things for their articles. You know, Dan Bader and crew over there.

12:01 And so they came up with this thing. Either they came out of RealPython,

12:05 I think, or possibly they were just using it a lot. But this thing called code timing.

12:11 So if you've got some code and you want to know how long it takes to run, you could say, you know,

12:16 create a time, like capture the daytime, do some work, capture dot now again and do a delta.

12:23 Or you could even use performance counters and other things. But you can use this library.

12:27 You just, there's a bunch of different things you can do. You create a timer class and you can call

12:31 start, do some work, stop. And out comes the time. Or you can put it in a context manager,

12:36 like a width block. Soon as it goes to the width block, when it's done, that bit is timed.

12:40 Or you can use it as a decorator. And you can also wire up a logger, which is kind of cool.

12:45 So you can see like, it'll just output standard Python logging with time information of when it's

12:50 doing bits of its thing. Give it a name and it'll tell you how long it took.

12:53 Cool. Well, they should add statistics too, so I can get min, max, and average, and standard deviation.

12:58 Yeah, that would actually be cool. Well, it's, you know, open source, I bet they accept PRs.

13:02 Actually, there's a bunch of features I want to add to it. And I started messing around with it.

13:05 And I'm like, put it down. I have other things to do. I'm already too busy. I don't need this.

13:11 I'm going to leave it alone. But yeah, it's a pretty cool little timer class. And I'm going to

13:15 probably use that. I like it.

13:15 Yeah. I thought this, a nice follow-on for this, the timer would be an article called

13:21 Making Python Programs Blazingly Fast.

13:24 That sounds good. We all want that.

13:25 You need to time stuff. You should never, I mean, hopefully we've all learned that

13:29 premature optimization is one of the most horrible things you can do as a programmer.

13:34 What I've learned is it's incredibly hard to guess where something is slow. Even if you know,

13:38 this takes a second, you look at the code, I'm like, oh, I bet it's there. Like, no, no, no. That's

13:42 like 5%. It's over here.

13:44 Yeah. Because it's super hard to know.

13:45 You're throwing the first version of the rough draft of your code down and you write something down and

13:52 you go like, I know I can do this better, but I'm going to just make it work.

13:55 here. And you know, you're going to have to optimize that. And it turns out to not be the

13:59 slow part.

13:59 Right. Exactly.

14:00 Yeah. So you need to find out where the slow part is. So this article called Making Python

14:04 Programs Blazingly Fast by Martin Hines goes through a few things. He doesn't cover this timer,

14:10 but there's a few other ways you can time it. There's a, you can use the command line time

14:14 function to just time how long something runs.

14:17 That might work. You might just go, I made a change. It was five seconds. Is it more or less?

14:22 Yeah, exactly. That'll tell you.

14:23 Python-mcprofile can tell you a whole bunch about your program.

14:27 Do you use cprofile much?

14:29 I don't really.

14:30 Yeah. I've used it. So it's, it's pretty awesome actually.

14:32 Yeah. And then he goes through an example of writing a wrapper function for a timer,

14:36 which is similar to what this last article was.

14:40 It's one facet of GoTimer.

14:42 One of the things that he doesn't cover is the time it function within that's built into Python,

14:47 which allows you to just run a single function a whole bunch of times and then tells you how,

14:52 how long that takes.

14:53 Yeah. Then you get your average and your deviation, all that.

14:56 But then the article goes through how to make things faster. So once you've found the slow parts,

15:02 how do you make it faster? And these are some good suggestions. Use built-in types over custom types.

15:08 Use caching and memoization through LRU cache, which is a built-in thing into Python.

15:14 Local variables and local aliases while, when looping. This is something I didn't expect. This

15:20 is something like if you've got a multiple dot, dot, dot, dot, something into a function call,

15:26 creating a local copy of that makes things run faster.

15:30 Every traversal of that dot is expensive in Python, whereas like C++, not so much.

15:35 Yeah. Especially if it's in a loop. So I use functions. I don't understand why this was there.

15:41 Kind of duh, but you know.

15:43 Well, apparently the variable lookup in a function scope is faster than a global variable lookup or

15:50 something like that that he was talking about. So by forcing all the variables into the function

15:55 scope, they actually come out faster. So there's all these like little weird edge cases.

15:58 Yeah. I don't have any code that's not in a function. So don't repeatedly access attributes

16:03 in loops. Okay. There's a similar sort of thing. One of the things I didn't realize is the F

16:07 strings have been tuned to be very fast. So if you're doing string formatting, use f-strings over

16:12 other things.

16:12 How many of you out there are using f-strings these days?

16:14 Right on. Like that's five, 10 times faster. I don't know. There was a thing by Raymond Hedinger

16:20 that's mentioned in that article.

16:22 Yeah. So it's way, way faster than the other ways that it was awesome.

16:25 And then use generators because I added at least experiment with generators because generators

16:30 are really about saving memory. But if you're really dealing with some large data, memory saving

16:36 could result in speed up. So I would say throw those in and see if it's faster.

16:40 As soon as you start hitting that page file, you're done.

16:42 I love generators. I throw them everywhere.

16:45 I do too.

16:46 Anyway, I think this was an interesting article on speeding up Python. And I warned people against

16:51 a premature optimization though. So, but it's fun.

16:54 Perfect. Yeah. This is a really good one. I like it. And it's a good follow on to the

16:57 other ones we have. Brian, yeah, you're here. So you spoke about CDK, the cloud development

17:04 kit from AWS. One of the big gives I have with working with the cloud is I work from home.

17:11 I want to go work in a coffee shop. Maybe I'm traveling. I want to work from the hotel and

17:14 the internet's bad. I still want to be able to work on my code and know the internet is not

17:19 available. Whoops. I guess my app won't run anymore. Right? Well, that is a problem, which

17:26 I mostly solve by avoiding the cloud directly. But there's another cool project called local

17:31 stack. Talked about Moto before, which is a way to like mock out AWS. And this is actually

17:37 built on Moto, but it actually does quite a bit more. This comes to us from Graham Williamson

17:41 and Jan Gazda. So thank you both for sending this in. And basically what it is, is it's

17:47 like a local AWS, not just a little bit, like a lot of local AWS. It has S3. It has SES for

17:56 simple email. It has EC2. It has DynamoDB as Lambda has elastic search, all of that stuff locally.

18:02 He showed us like tons of a huge list though. They have all that stuff.

18:06 It has a bunch. I don't know how many it has, but I would say it's somewhere on the order of,

18:11 I don't know, 2025 different services.

18:14 Probably the most common ones.

18:15 Yeah. Yeah. Probably the most common ones. And then apparently there's also like some kind of pro

18:18 thing I've not used, but then you get more services if you buy the pro version, but the lesser one,

18:24 I guess is open source, which is pretty cool.

18:25 That's neat. That's great. Like if you're on an airplane or something.

18:28 Yeah. Or you just, you want to have like a little local dev environment.

18:31 You don't have to pay for that, even though it's less than pennies, but yeah.

18:34 It depends what you're doing, I guess. But yeah. So yeah, basically it brings together some of these

18:38 tools. It brings together Modo. It brings together this thing called Dynalite and puts sort of a

18:43 unifying layer on top of it. So it's pretty cool. A lot of it runs in Docker. It helps to kind of

18:49 get a repeatable experience there.

18:50 That sounds neat.

18:51 Yeah, absolutely. All right. Well, that's it for our main items, everyone. Got any extras you want to

18:56 share with folks?

18:56 Well, I just, I don't know if we've covered this before. I saw an advert for the Python job board

19:03 on python.org.

19:05 Yeah. Yeah. I saw that. I hadn't seen it before, but yeah, apparently there's the, there's now,

19:10 yeah, we're joking around. We're laughing because the internet's not quite cooperating. That's fine.

19:16 We don't need it. Who needs the internet? What did I say about the cloud? What did I just say,

19:19 Ryan? You better hope you're not trying to test something right now. Carry on. Yeah. So the job board is

19:25 cool. Right. I hadn't noticed it either, but it's on python.org.

19:27 I don't know if you have to pay for stuff, but you can just list jobs. So that's cool.

19:31 Yeah. And Python's in demand. People want to have jobs writing code in Python, right? Like,

19:36 you know.

19:37 Do you have any extras for us?

19:38 I do have a couple. Let me pull these up here for the audience as well. So I have pictures

19:43 because some of these are very, very graphical. So there's this really cool one. You never asked him

19:47 probably heard of the guy who created Python. So he was interviewed, he's Dutch. He was interviewed by this

19:51 Dutch newspaper about programming. And the title, my Dutch is a little off, but it's like Python is

19:58 half my life. Right. So I worked on a Python for half my life or something like this. And they said,

20:02 we're going to, this is like a developer thing. Let's put some code and show some Python.

20:07 Yeah.

20:07 What code do we see there?

20:09 I don't know. Is it JavaScript?

20:10 Document.getElementByID. Yeah. Not so much. Not so much. That was a pretty interesting little

20:17 thing that actually happened. The next one that's pretty interesting. I don't have a picture for it.

20:21 It would just be like a bar that's rusting or something, but no, it's pretty cool.

20:24 Though Microsoft, you know, they're all about C and C++, right? Like Windows is based on C and C++.

20:30 They are actually been doing experimentations with Rust and they're coming out with a Rust

20:35 based programming language for like rewriting things like Windows and Rust. That's a pretty big

20:41 jump for Rust. And the reason is Rust is especially good at memory management and memory ownership. So

20:49 things like buffer overflow vulnerabilities and stuff just go away in Rust, which is like,

20:55 you know, every first Tuesday, here's the seven buffer overflows. They're going to like lose all

21:00 your data if you don't patch by the next two days that you get. And like, they're trying to avoid that,

21:04 I'm guessing. So that sounds interesting.

21:05 Do you know Rust yet?

21:06 I've looked at Rust. It looked like C. I said, I'm going to go back to Python.

21:09 I mean, not exactly, but it looked like C-ish.

21:12 Maybe I should take a look at it.

21:14 Yeah, it's pretty interesting. Two more quick things. So I'm doing a webcast of Python for

21:18 the .NET developer, kind of interactive one hour thing at Crowdcast on the Crowdcast platform. I

21:23 think that'll be fun. So links in there. It's free to sign up. People can check that out. And then

21:27 Reuven Lerner was talking to him today and he has a new free course that he released called

21:32 Ace Python Interviews. So people are out there looking for a Python job. Here's like 50

21:38 little exercises and questions answered live and like live coding responses to 50 interview questions

21:46 that are explained.

21:47 Reuven's a really cool guy. So I think that'd be cool to look at.

21:50 Yeah, yeah, absolutely. It looks really good. And that was also free. So no harm, no foul there.

21:54 People can check that out.

21:54 And I've got a job opening. So if anybody's looking, I'm mostly, last time I interviewed was for Python

22:01 person. So I'm probably just going to take some of these things and convert them to C++.

22:07 So, you know. So if you want to pass Brian's interview, it may be a good idea to take this

22:11 course. Don't tell my boss.

22:12 Yeah, yeah. All right. So are we ready for a joke? We always end up podcasts with a joke.

22:16 Yeah.

22:16 This one's very visual. So I'm going to put this up on the screen for you as well.

22:19 And this is really like a sort of infographic. I'm a fan of infographics. And this one helps you

22:24 understand like the different types of jobs in software development, which can be very tricky,

22:28 right? Like what is the difference between a lead developer and a full stack developer and a coder?

22:33 Well, here we go. So it says, there's this person and pretty much it's the same looking

22:37 person for every job description. And it says there's a coder. There's a little caption that

22:43 says he writes. Software engineer, he writes code. Lead developer, he writes code. DevOps,

22:49 well, he writes code. Infrastructure is code, right? Data engineer, actually, what does he do?

22:56 He writes code.

22:56 He writes code. Full stack developer, he writes code alone.

22:59 Computer programmer, he writes code too. Sysadmin, he writes, this is actually a guy eating donuts

23:07 with a big beard and looking very disheveled. It says he writes, in fact, we don't really know

23:12 what he does. All right. Well, that's the joke for Dave. I guess that's the podcast for today as

23:17 well. So thanks a lot. Yeah. Thanks a bunch. And thank you everyone.

23:20 Bye.

23:24 Thank you for listening to Python Bytes. Follow the show on Twitter via at Python Bytes. That's Python Bytes as in B-Y-T-E-S. And get the full show notes at

23:34 Pythonbytes.fm. If you have a news item you want featured, just visit Pythonbytes.fm and send it our

23:39 way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Okken,

23:44 This is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.