Transcript #272: The tools episode

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to

00:03 your earbud. This is episode 272, recorded February 23rd, 2022. I'm Michael Kennedy.

00:11 And I'm Brian Okken.

00:12 And I'm Calvin Hendricks-Parker.

00:14 Hey, Calvin. So good to have you here.

00:16 I'm excited to be back.

00:17 Yeah, it's great to have you back. I also want to say this episode is brought to you by

00:20 Fusion Auth, a new sponsor. Thank you so much to them for supporting the show. Check them out at

00:25 pythonbytes.fm/fusionauth. More on that later. Calvin, it's been a while.

00:29 Since you've been here, but you're a frequent guest. How about just tell quickly people about

00:34 yourself? Sure. I've been a almost, I guess, nearly lifelong Pythonista at this point. I'm going back

00:40 a long ways, but started a company back in 1999 called Six Feet Up, where I am the CTO and co-founder.

00:46 And we are all things Python and all things cloud. So we're doing some cool stuff there. I've been

00:51 very involved in some open source projects, like the Plone Foundation for the Plone CMS. We're very

00:56 involved in Django as well. I'm sponsoring the Django Software Foundation. So super excited to be

01:02 involved in open source and all things Python. Right on. So if I was a company, a person said,

01:06 I need a Python app, I need some help with it, or maybe someone to even build it, I might reach out

01:10 to you all and you might build it for me. Yeah. No, we'd love to talk about those kinds of opportunities.

01:14 The harder and more challenging, the better. Yeah. Those are the fun ones. Yeah.

01:19 I agree. I completely agree. I like easy. All right. Well, I think this one you got here and this

01:28 for the first one, Brian, maybe not so easy. That's what's up with your mocking. Why is it not working?

01:32 Well, your mocks. So this is a great, actually, this is a rabbit hole, but well, mocks are a rabbit hole.

01:40 So Ned Batchelder, great guy wrote, he writes a lot of great stuff. Also maintains coverage.

01:47 But he wrote an article called Why Your Mocks Still Don't Work, which is a reference to an earlier

01:56 article he wrote called Why Your Mock Doesn't Work. And he wrote that in 2019. So if you haven't read this

02:04 first, so I'm going to go back and recommend both of these. So Why Your Mocks Don't Work,

02:09 or Why Your Mock Doesn't Work, is an excellent article. It starts talking about just like,

02:17 well, to think about how mocks work, you really have to understand Python name spaces and names and how

02:24 imports work and all of that stuff. And maybe you don't think you should have to, but you kind of do.

02:30 So in imports, those are not entirely obvious, like the way that that happens compared to other

02:38 languages, right? Where you just say, I'm using this library. There's like, it's way more direct.

02:42 So Ned starts off with like this really great example of just basically two variables,

02:47 X and Y pointing to a number. And if you assign into each other, but you don't really point at another

02:54 variable. You point at the thing that the variable is pointing to if you assign X to Y or Y to X or

02:59 something. And this hat, this does have to do with importing because the names that you import are

03:06 just kind of variable names that point to something. So he talks about namespaces and, and where, where

03:12 things point to talks about importing and the difference, mostly the difference between from,

03:18 from foo import thing instead of import foo and reference it as foo thing. Those are completely different

03:27 names within, within your Python application. So talk walks through that, talks about it and why.

03:34 So if you've got a value from one module pointing to an original value and then another one pointing to

03:42 the same threat thing, if you mock the wrong place, you're not going to get what you think is right.

03:49 So basically, and this is terrible to discuss over podcast. So I recommend looking at this,

03:56 because it's a really great example of how mocks work and, and why, why they act the way they do.

04:04 And you'll, you'll be able to fix a lot of your problems. So this is a good recommendation,

04:08 but the one that we're trying to talk about right now, why your mocks still doesn't work has to do with

04:14 this cool decorator thing that mock. I don't remember when a unit test mock added this, but there's a patch decorator.

04:21 Actually, I'm going to link to the show notes somewhere. We talk about the patch decorator and

04:27 it's, it's pretty neat. Actually, I'm jumping all around. Sorry. But if you just say like patch

04:34 something, what happens on your test? Like in this example, he's got two, you got a patch, cool feature

04:41 cool feature dot logger. And you're not patching it with anything with this decorator. What happens is

04:50 you're, you get these like other variables that you can add as parameters, mock prep and mock logger

04:57 that you can use those to change it. You can change the behavior, the, you can use asserts on it. You can

05:03 change the return values on these through that. It's a, just a, this is a way to get a handle into your

05:10 mock object. It's pretty cool. and, but the trick is they have to be in the right order.

05:17 And this example looks like it's fine, but you've got logger at the bottom decorator,

05:22 which is really the first and prep as the above decorator. And it's the, it's the first one,

05:29 Brian, it says the first line says this prep one. The second one says logger.

05:33 Yeah. But that's not the way it happens. So the, the catch here and the punchline is, it goes,

05:39 but when you're reading decorators, it goes bottom up, and, left to right.

05:45 Like onions.

05:46 Sure.

05:47 Right. Like there's layers around your code and like the inner layers, the first one it sees.

05:53 The second, so if you, the way he has it originally written, it just fails. It doesn't work.

05:58 the second punchline is unless you've got a real good reason to use, auto spec because auto spec

06:05 would have caught this problem like right away. And, what auto spec does is it says the thing that

06:11 you're mocking, you have to call it with the same interface that the thing had in the first place.

06:17 and, so if you've got a, a class or a module that has a function in it, you can't call

06:24 other functions. Whereas normal mocks, you can call whatever you want. so the default is free

06:30 for all. What you really should do is auto spec equals true. And really a lot of people, me included,

06:36 wish that that was the default to begin with, but for historical reasons, it can't be because it

06:41 would break other stuff. But anyway. Yeah. Yeah. if it's not the default to start with,

06:48 if you put that in to say now only behave as if you are the thing that I say you're pretending to be,

06:54 that's going to crash a ton of stuff. Yeah. So, yeah. And then the third and final punchline is,

06:58 just don't use mocks. even he Brent mentions it in both articles, avoid using mocks if you can.

07:09 So absolutely. Felix out in the audience says tiny feature with a lot of power, this auto spec.

07:14 Yes. Very much like Yoda would put it. I love it. All right. What do we got next? Calvin thoughts on,

07:19 on mocking real quick before we move on. I'm, we are just diving into that and I don't do a lot of

07:24 the test work myself. So I'm sure Brian is the super expert on all things, by test, of course. So

07:31 I don't have a huge opinion here on specific mocking. Although I think it is important for folks to

07:36 realize that, yeah, decorators go from like that closest to the function out. It was an important,

07:42 call out there. Yeah. The other one was from thing import something. And then you patch the thing

07:48 inside there. It's kind of that, that I actually, that was news to me. That's very interesting.

07:51 All right. Also news to me, please, please tell me about this. Chris may thank you, Chris may for

07:56 sending this over and wow. I don't know if you've heard about please folks, PLS as an LS

08:04 replacement for Linux and macOS, but wow, this thing is cool. was this, had either of you heard

08:10 about this? Not until today. And I went and installed it as part of my demo so I could see it because it

08:15 looks pretty darn awesome. So this is an LS replacement and I know there are other LS replacements

08:19 that already exists. So you can do more things, but this is a developer focused one written in Python.

08:25 That's pretty darn awesome. So if you go look at the image I linked to, or just go check out the site,

08:31 you can see that if you say, so I've alias LS to PLS. And if I just say LS, it shows you the contents of

08:40 your directory folders on top. And then it has icons for the types of files that they are. So I did an LS,

08:47 there's a Git ignore. So it has a Git branch icon. There's a license with the law. There's a markdown

08:52 file. There's a Python file, the Python logo, but it goes beyond that. Like it understands your Git

08:57 ignore and the files that are considered hidden are the Git ignored ones as well. And it won't show you,

09:04 even if you do like a hidden file listening, it won't show you things like the .es underscore store

09:11 on macOS because that's in the Git ignore. So the structure that it gives you is related to that.

09:17 Also for Python, if you have a virtual environment, it'll treat that directory as hidden. And because

09:23 the directory is hidden, like it's sort of suppressed in its visibility. So there's really cool features

09:29 around this that have to do with basically saying, all right, you're a developer, you're listing files

09:35 that are probably developer like. Now what, now what do we do? So another part that's cool is you can do

09:41 and please --details, which I alias to L because that's the LS like equivalent. And in this

09:48 world, it shows you the same types of things, but it also shows you the size in the human terms. So

09:55 instead of like one zero one, one, one, two, one, one, seven, one, one, one, two, it would say that's

10:00 10.2 gigabytes, which is nice, but it also shows you the Git status in the listing. So like it has a dash

10:05 M next to a file I modified and so on. What do you all think of this?

10:09 I think it's excellent. The, I tried to use it and failed and you've helped, helped me understand why,

10:16 because you, it has, you have to have fonts, it's like some special fonts installed, right?

10:22 So one of the questions when I posted this week was people like, how does it have custom icons?

10:28 What magic is this? Cause this will work over, like if I SSH into somewhere and install please on the

10:33 server, as long as my local terminal is set up correctly, this will work. And like, how does this

10:38 work? Well, the way it works is you have to have your terminal font set to a nerd font. So I've talked

10:43 about nerd fonts before, but all these nerd fonts have all these special icons in them. And long as your

10:50 active terminal font is one of the nerd fonts, then you get all these cool behaviors. If you don't,

10:55 you get squares, which are less than awesome.

10:58 My only complaint with it is that I wish it would have emulated more of the flags from standard LS

11:05 so that it just kind of work a light. I installed it. The first thing I did was like, please dash AL

11:10 and I got like a no, no L flag.

11:13 We don't know what that is.

11:14 Yeah. I didn't know what that was.

11:15 Yeah. It's kind of its own thing. That's why I did the alias on it. And I'm like, okay, well,

11:20 I'll just, I'll do these things. Another thing that's interesting, it has a, you can set up these,

11:25 these YAML configuration files that control how it looks. And then you can put those kind of like

11:32 node JS does with node modules. Like you can put it at different or the project.json, you can put it at

11:37 different levels. And if you go into a certain project that you had of a configuration file in

11:43 there somewhere, it'll pick up that configuration and then use that to like customize how it looks

11:48 for those. So that's kind of an interesting thing as well.

11:51 That is awesome. I mean, we need more, we need more like fun stuff in our terminals as developers.

11:56 I totally love this. This is absolutely the way to get people hooked on using the terminal in the

12:02 console. Yes. And I was thinking about like server management, which we'll get to later,

12:07 right, Calvin? Yeah. Yeah. But if you want to go into the server and so I'm going to SSH in and do a

12:13 thing like this is your entire user interface. Yeah. To that. Sure world. And so, yeah, yeah,

12:19 this is your world. So this is a way to bring like a little bit more UI information rather than just raw

12:24 white text listing a files. I feel like there's like actually a lot of value in this. Oh, totally.

12:31 Awesome. Let's see a Dean out in the audience and former. Yes. I thought my life was complete until

12:37 today. I didn't know this was missing from my life, but it is. Yes. Please more terminal stuff.

12:43 Absolutely. Well, speaking of more terminal stuff. Yeah. Speaking of more terminal stuff. Yeah. Take it away.

12:51 All right. So we over at six feet up, we've been very focused on the developer experience. And so

12:57 once a week we get together and talk about just, we can do a called code review, but it really,

13:02 it's more of a show and tell. And this is something I showed off last week at our code and review and

13:07 show and tell, which is a terminal that is GPU based and written in Python. And it's called Kitty.

13:13 If you've not used it, it actually is a super awesome, super fast. Basically, I don't know. I found it to

13:20 just be a super smooth experience. And the reason I had revisited it was I'd been using Tmux forever.

13:25 And before that I was using screen and I wanted to like, just have the ultimate power tools available

13:30 to me as a, as a developer. And as a, like my primary mode of operation is basically hanging out

13:35 in the shell or an editor all day long. So I want the best tools possible. So I'd highly recommend

13:40 folks check out Kitty. It is absolutely tuned for performance. And you may ask, well, why the heck would

13:46 I want a GPU enabled terminal is just showing me text is because I want the most performance

13:51 possible out of my, my system when I am using it. And so I actually was going to give a quick

13:55 little demo here of Kitty. This is actually using PLS. And obviously, yeah, I see it.

14:01 I have the right.

14:03 And what is this? Oh, my, is this on my posh also? What do you got for the,

14:06 Oh, for the, the, the bullet train down there. So it's actually bullet train core.

14:10 it's not maintained anymore. That's probably my next, my next venture is going to be replacing

14:16 that line with some new or something like that. Exactly. Some new, you know, system for showing

14:21 my awesome, like prompt up there. But yeah, if you get the right fonts installed and all that Kitty

14:26 supports all that stuff seamlessly, it's very, very fast. And one really cool thing, since we're all

14:32 like Pythonistas and people who are listening to this would totally relate. You can extend Kitty

14:38 with kittens that are basically Python plugins for Kitty. so if you actually, I'll, I'll give

14:44 you a quick example. So if I run Kitty, I, it is, I love, but I love it. I kind of love it actually.

14:48 So you just, you invoke Kitty with the, the kitten flag and say like what kitten you want to use. In this

14:54 case, I'm going to use iCat. I'm going to just basically echo out a Slack emoji that we use for Python.

15:01 Oops. Oh, oh no. Is this like going to be like the, total like demo guys, not, not being very kind

15:07 to me now.

15:08 I love that your, your prompt has the Python version and stuff in there. That's great.

15:13 Oh yeah. If you're in a specific like pyenv area, it definitely shows up. Let me just,

15:18 let me just like try it here. iCat Python. Oh, there we go. So I must've just been in the wrong

15:24 directory, but so you can actually show graphics on the screen. So if I want to look at an image real

15:29 quick, I don't need to go to finder and open up preview and like do a quick look on an image.

15:33 I can actually like, you know, quick look on any image I want. And I, one of the things I did want

15:38 to show off, I go back to the director. I was, people who are listening, this looks like a full

15:42 high res image in the terminal. Yeah. Based on a PNG. So I used iCat, which is a kitten to display an

15:48 image directly in the terminal. And now if I wanted to do show even something coolers, because one of the

15:52 features of Kitty is it's basically got a graphic subsystem in here. And if I wanted to like, look at my

15:57 markdown files, normally if I'd use bat to look at my markdown, like for the readme that's in here,

16:02 you know, you get the kind of content highlighted markdown, kind of cool, not, you know, not super

16:08 great. But if I do MD cat, which is Kitty enabled, it actually leverages the Kitty subsystem here.

16:14 I can actually review my markdown file with images in line. So you can see that the XKCD cartoon is

16:22 actually embedded into the readme of that document. And it does a little better job of like coloring and

16:27 highlighting the syntax. So if you want to preview markdown documents without again, not going to

16:31 preview or not going to like rendering it to HTML and viewing your browser, you can actually view it

16:36 right in your terminal with the images shown in line, which is super awesome. And one last thing,

16:42 one more trick I'll show you. There is a Kitty kitten for viewing diffs in a rich tool. So if I do

16:48 get a diff tool, I've configured with my, you know, dot get config to configure an alternate diff tool for

16:56 diff. And now I get this really beautiful, you know, high res graphic representation of my diff,

17:02 my files. So really kind of pretty, you can see there's two, two diffs, two different files,

17:07 I can page through the different files and it shows me, you know, side by side, you know, diffs.

17:11 Is that still in the terminal?

17:13 That is still in terminal. I did not leave my terminal.

17:15 It looks like a new window.

17:16 It did. It looked like a new window, but it's actually all in terminal. So you can stay in

17:19 your terminal, stay at your keyboard and all the, all the shortcuts are super nice. One last thing.

17:25 The diff just sold me. So yeah, you can split your windows, obviously like, you know, other kinds of

17:32 terminal emulators, like if you're using Tmux, but then it's got all the layouts built in. So if I

17:37 wanted like horizontal or vertical or a grid, or if I wanted to like, just get to the current one I'm on,

17:44 you've got all that available to you with nice shortcut keys and you can just, I got rid of Tmux and

17:49 started just going all in on kitty only.

17:51 Okay. I got a question. So I've always been confused about this whole windowing thing because

17:55 I just opened another terminal. so.

17:59 Oh, Oh, I see what your question is. I don't want to open another terminal. I don't want to reach for my

18:04 mouse when I have to like go between terminal windows. So I'd rather have like multiple tabs and then have

18:09 splits. And so I can have like a paging going on in one window or another window. I can like, you know,

18:14 then edit these things, make them like shorter or taller or split them in a different way.

18:18 So I feel like just keeping my hands on a keyboard keeps me more productive as a developer,

18:24 as opposed to reaching over for the mouse. I know it's not that far away, but I feel like it breaks

18:29 that flow. If you're, you're typing away at code and you want to like quickly, like open up a tail

18:33 for a log and like a same window right below where you're running the process. You're just,

18:38 you know, a couple of keystrokes away and you've got that open and going.

18:41 Cool. Nice. Yeah. Yeah. Oh, that's really cool. So I know we're not to the joke section yet,

18:47 but you know, the joke about how do you generate a random string, put a new computer science

18:50 student inside of them and ask them to exit. Yep. So I think the new version is put four of them in

18:57 these different panes controlled here and ask them to exit the top right one.

19:01 Here, I'll do it right now.

19:04 Even more random number or word.

19:07 There we go. Boom. Did it.

19:10 I'm going to try kitty. This looks great.

19:12 It's super configurable. It supports all the nerd fonts, all the color schemes. I've got,

19:17 you know, just got, there's a bunch of cool plugins for it. I've got a search plugin,

19:20 which searches through your back through your terminal. And so if I was, there's nothing in

19:25 that specific one, but if I searched through here and wanted to search backward, it uses FCF to do

19:31 searching backwards. So if I was looking for like LS, you see it highlighted the word LS or 2020.

19:35 And then if I had multiple ones and I could just arrow back up between them.

19:40 So it's all built in.

19:42 Yeah. Yeah. Really cool. Also really cool is our new sponsor, Fusion Auth. Thank you

19:48 them for sponsoring this episode. So let me just tell you really quickly about them. They're an

19:53 authentication and authorization platform built for devs, by devs. It solves the problems of building

19:59 essential user security without adding the risk and distraction from your main app. Fusion Auth has all

20:05 the features you need to, for great support at a price that won't break bank. And you can either do

20:11 self-hosting or you can get a fully managed solution running in any AWS region. So if you've got a side

20:17 project that needs a custom login registration or multi-factor authentication, social logins, or user

20:22 management, you can download Fusion Auth and get the community edition for free. And the best part is you can have

20:28 unlimited users and there's no credit card or subscription required for that. So check them

20:33 out at pythonbytes.fm/Fusion Auth or just click the link in your podcast player show notes and,

20:40 let them know you came from us. I want that t-shirt. Thank you, Fusion Auth. That's cool. I know.

20:43 Yeah. You got, get a cool t-shirt, cat slash et cetera slash password, D. Yeah. Very cool. Nice. I love when you get a cool t-shirt from our sponsors. All right. Well, Sue doesn't like cats. I know. And kitties. And kitties. And kitties. I can tell that you're already a fan of kitties. well, so, I, I don't know if, kitty is parallelized, but,

21:10 Yeah. So I want to talk about parallels and parallelization say that three times. anyway, I found this article, by, Jamie Welta. Cool. Last name actually. but it's called futures and easy parallelization. And, I was like, you know, it was a pretty short article and I was like, it can't be that easy. but so this isn't talking about, AIO stuff, ACE or asyncio. It's, it's talking about

21:40 this one is talking about thread futures with threads. And this is, it's a, it's pretty cool. The idea is you've got just maybe you've got, and it starts off with a simple example. I just have some work that I want done and I want it done on, on different threads or different processors. So, this, this example, brings up a thread pool, a thread pool executor, and then runs them all at the same time. does an executor submit?

22:10 And it's just a small snippet of code. It's just a handful of lines. And then I tried this out. So this example actually, is kind of boring. It's just doing, like a power X to the power of two, or X squared. I'm on a couple of things. I think that's what star star is, isn't it? Is that power of, I don't remember.

22:28 Yeah. X to the power of two X. Yeah. X squared.

22:31 X squared. So, the, the other example seems a bit, a bit more, down to earth and that's, and yes, on our screen, on the, on the YouTube screen is just, it's the entire program here. it's just, you're taking a few, a few websites and a couple of pages to go to.

22:51 And, and then actually just slurping those down with requests and grabbing something about them. And this example just, as the result, whether it's a 200 or something like that, but it's a really short example.

23:04 And you've got parallelization going on. And I played with this, just downloaded it. The only, we'll have it in our show notes. The only error on this that I had to do is it's using, it's using time in here is just like a debug thing. And in the example, doesn't import time. So you have to add, add that import time. And it runs just fine like this. And this is a pretty quick way to add parallelization for some quick task.

23:29 So I kind of like it. There's occasionally, especially like I would do it. there's a lot of huge log files I have to parse or, or big data files that I'm looking for stuff on or grabbing, grabbing error logs off of different systems. And this would be a great example to just grab, grab them all at the same time and pull them in. So.

23:48 Yeah. Yeah. Yeah. That's nice. You know, one thing that's cool about the, the futures you get back from the thread pool executor is you can say dot result and it blocks. Whereas on asyncio, it throws an exception and says it's not done rather than just blocking.

24:03 Oh yeah. Yeah. And that's the article kind of talks about that. It's one of the simplest things is you, you tell the executors, you call executor submit and that gets the jobs ready, but that doesn't block. Those are, you can, you can submit as many as you want.

24:18 And then in the example, he's just using few like future dot result, in a, in, in a list comprehension and that for each of those result that'll block until the next one's done. And you know, this one's doing it in an order of which one you submitted it, that might not be the order they finish in, but you don't really care because you just want to wait till they're all done anyway. So.

24:42 Yeah, exactly. If you block on the first one, it's not done. The second one might finish first, but it'll be done by the time you get to them. Yeah. Yeah. Yeah. And this is a lot more natural for folks who may not be used to like asyncio too.

24:54 Oh, I've been deep in the asyncio world.

24:56 Well, it's all built natural here.

24:59 And that's, that's.

25:01 And I have scars now, let me tell you.

25:03 That's exactly why I wanted to bring up this article is because there's a lot of stuff where like a, like a, maybe a DevOps person or something.

25:10 They're not, they're not writing async programs, but they might have a async need or asynchronous needs that can be solved with a simple, some simpler code.

25:19 So.

25:20 Yeah. This is very, very elegant, easy to understand.

25:23 Yeah. Nice article.

25:25 Good one. All right.

25:25 Well, I want to talk about databases and more tools.

25:29 I feel like this is just the tool focused.

25:31 Like the tool episode.

25:32 It is. It's full time.

25:34 All right.

25:35 So I did an episode on talk Python with Emily Morehouse, glyph and Hennick and Hennick pointed out this thing called PG mustard.

25:45 Have you heard of this?

25:46 No.

25:47 Have you?

25:47 Oh, well, I just listened to that excellent episode and heard about it.

25:51 So now, you know, but other than that, like I had not heard of it, but one of the challenges, so many websites, I just don't understand why the world works this way.

26:00 But you go to the website and it just spins and spins and the clunkier, the more internal the thing looks, the more likely it is to take 10 seconds to do whatever it's doing.

26:09 Right. But, you know, it's doing some database query, probably without an index.

26:14 Maybe they gave it an index, but the index is not being used because it's actually filtering first on some other random thing or whatever.

26:20 So databases have this cool feature to say, given a query, explain how you're going to execute this.

26:27 MongoDB has this.

26:28 Postgres has this and so on.

26:30 OK, so that's that comes out as text.

26:33 What if you could have a better way that gave you advice?

26:35 And that's what this tool here is, which does cost money.

26:38 Just to be clear, it's a commercial thing, but I thought it was cool enough that I wanted to point it out to people.

26:43 So what you do is you give it your basically a select statement and you ask it to explain it and it will it'll break down the explain statement into different sections and tell you this part is really good.

26:59 This part of of the query could be improved and so on.

27:03 And then it gives you so like it'll it'll show you this beautiful visual way of explaining it and you can dive into it.

27:10 And if you click on it, it'll tell you things like, OK, this is a nested loop.

27:14 And on this part of the query, it took one hundred and fifty two milliseconds.

27:17 You got one hundred rows back.

27:18 And then it actually describes the situation, why it's good or bad.

27:23 So, for example, it says you got five stars because you discarded one point three million rows.

27:29 That makes it faster.

27:30 But you only got three point two stars because the row estimate was out by a factor of 42.

27:36 You know, try to get that from text, right?

27:38 This is this is super helpful if you've got a slow site that you want to say, OK, this page is slow.

27:45 These are the three queries that run when we pull this page.

27:48 Why are they slow?

27:49 How can I make them better?

27:50 So it's you know, we talked about the regex one on one thing, how it like kind of guided you through and gave you recommendations.

27:55 This is like the database equivalent.

27:57 This is nice.

27:58 This is awesome.

27:59 I can't believe I've never seen this before.

28:01 Well, I know I have.

28:02 I can't either.

28:03 I've never seen it either.

28:04 Now, from the from the interview, it sounded like it would recommend how you could change your query to make it better.

28:11 Is that something it has in there or did I imagine that?

28:14 I believe so.

28:15 And I believe so.

28:15 If you like open up this 3.2 stars, I think it'll give you descriptions about what could be better in there.

28:23 So it can give you performance advice, including high index potentials or index efficiency, disk operations.

28:30 So like you've got to read too much off this to answer this question.

28:33 Poor cash performance, excessive heat fetches, read efficiencies, glossy bitmap scans and on and on and on.

28:42 Right.

28:42 So it's pretty cool.

28:44 It runs on Postgres 9.6 or newer.

28:47 I hope you're newer than that 9.6.

28:49 That's that's pretty great.

28:51 It supports that far back.

28:52 Yeah.

28:53 Yeah.

28:53 Anyway, it is a paid tool.

28:56 But if you could for ninety five dollars, make your website ten times faster and people have been complaining and complaining, you don't have to rewrite anything.

29:04 You just put in like a slightly different hint or index or change the order of a query like that.

29:10 That's that's worth a lot.

29:11 I think I'm just going to try to learn this and then set up a page to say I'm a database optimization consultant and just run this in the background and say, yeah, look, five hundred dollars an hour.

29:23 I will absolutely come in there.

29:24 Yeah.

29:25 And I have this new business proprietary set of tools that I've I just I can't talk about it.

29:30 But you let me in there and magic's going to happen.

29:33 I shouldn't have said that out loud.

29:36 Yeah.

29:36 Yeah.

29:36 Well, I guess that the market will be swamped with these folks.

29:40 Yeah.

29:40 Anyway, I thought this was cool.

29:42 So I wanted to give it a shout out.

29:43 Cool.

29:44 Nice.

29:45 That is super cool.

29:45 Yeah.

29:46 All right.

29:47 Calvin, you got the last one.

29:48 All right.

29:49 Last one up.

29:49 Continuing the tools parade.

29:51 Another tool.

29:52 This is also cross platform.

29:54 I didn't mention that before.

29:55 One of the reasons I really like Kitty was the fact I can use it anywhere.

29:58 Linux, BSD, Windows, Mac.

30:01 There are downloads for all those platforms.

30:04 This one doesn't support Windows, but it does support Linux, Mac and FreeBSD.

30:08 It's BPYTOP.

30:10 I used to be a longtime user of Glances.

30:13 And if you didn't know what Glances was, you were also missing out.

30:16 Because Glances is an awesome way to see what's going on in your system.

30:19 Like what I was being used, how much memory is being used, how much swath is being used.

30:23 And BPYTOP is kind of the next generation of that.

30:26 So I will show a quick little demo of this one as well.

30:29 So for those of you who weren't familiar with Glances, this is what Glances looks like.

30:34 Sorry, a little interruption there.

30:39 But while I'm live here on the podcast, this is Glances.

30:42 It's kind of like more tech, very tech-y, kind of old school looking.

30:46 But if you just PipX install, and that's how I typically install these kinds of tools.

30:50 Sorry about that.

30:53 It's all good.

30:54 PipX is the homebrew.

30:58 Oh, man.

30:59 It's amazing.

30:59 Python things, right?

31:00 You just PipX install BPYTOP.

31:03 If you don't have PipX installed, you should install PipX because that'll get you access to all the stuff and your path all set up.

31:11 But now I just use BPYTOP.

31:13 And you get this nice colorized view of a dashboard.

31:19 It's all really just laid out well.

31:22 And then all the, you can kind of see on here, there's like characters that are shaded in a different color.

31:28 If you, you know, hit those characters, you'll be able to like resort.

31:33 Or if you see the little numbers, I can hide and show the CPU or the memory graph if I don't care about that one so much.

31:40 So for people who are just listening, here's a terminal app that has like task manager.

31:47 Oh, yeah.

31:48 Or activity monitor levels of sort of graphs going on of like, here's the CPU over time.

31:53 Here's the running processes sort of by CPU.

31:56 Here's the network.

31:58 This is probably more useful than activity monitor, honestly.

32:01 Oh, I think it is.

32:02 I mean, what's nice is you get that trending metrics over time.

32:05 So you can see by CPU core or by like kind of aggregated CPU view, how, you know, if you're seeing spikes or what.

32:13 And this is really useful if you're on a server and like something's periodically happening, you're not sure what.

32:18 And you can kind of track down like either IO issues or CPU spikes.

32:22 And you can kind of see if they're becoming very like periodic.

32:25 Maybe they're happening every minute or every like five minutes, like on the dot.

32:29 And you're like, oh, that's weird.

32:30 There's something like maybe it's this cron job.

32:32 So it helps you track that, track those kinds of things down.

32:34 You can also inspect the processes so you can arrow up and down and you can actually like hit return and see like what CPU specifically or memory like a specific process is.

32:43 You can like dig down into a specific process and see what core it's actually running on.

32:47 It's just, again, for tracking down performance issues.

32:51 I'm just using it locally on my own laptop right here.

32:53 But I've used this numerous to every machine I log into for, you know, customers or production type stuff.

32:59 They're still using virtual machines.

33:01 This is absolutely installed so that when they're like, oh, something's slow or something's doing weird.

33:05 Like I just go fire these up and take a look real quick to get kind of a snapshot in real time what's going on.

33:11 I've done that with glances.

33:12 Yeah.

33:13 That was used to my go to.

33:14 But I just found B5 Top was like the super powered version of glances.

33:18 Interesting.

33:19 You've moved over.

33:19 Yeah, the graphs are way better.

33:21 Yeah.

33:22 Like you have progress bars or like meter bars that are graphical in glances if you make it wide enough.

33:28 But they don't have over time.

33:29 They're just like snapshots.

33:30 And these are like beautiful gradient colors.

33:32 Like I kind of just expanded the net one so you can see the traffic going across my card.

33:38 You can also switch back and forth between different interfaces on the system.

33:41 So if you've got multiple network interfaces, you can see the aggregate or just for a specific interface.

33:47 Yeah, super helpful when trying to track down, you know, weird, you know, in quotes, weird issues.

33:52 How's it run on Kitty?

33:53 It runs great on Kitty.

33:55 The performance is amazing.

33:57 That's another thing I didn't show you is like you can actually see with the NVIDIA SMI tool.

34:01 Like there is Kitty using up 20 megabytes of my GPU memory on there.

34:06 Oh my goodness.

34:07 No, that's pretty awesome.

34:09 Speaking of beautiful, Alvaro says B5 Top has themes.

34:13 Yes.

34:14 They use the Darkula theme.

34:16 Yeah, I love the fact that anything's got themes I can customize like, you know, Kitty or even B5 Top or whatever my IDE is.

34:23 I just, I trick that stuff out.

34:25 This is my environment.

34:26 I want to be as comfortable and as productive as possible.

34:29 So the more customizable, the better.

34:31 And the more emojis, the better.

34:32 Love emojis.

34:33 Yes.

34:34 There's something to be said about it.

34:37 If you sit down and you're like, I am excited.

34:39 Look how cool this is.

34:40 I'm excited to play with this.

34:41 Yeah.

34:41 You're like, oh, I'm using like an old version of Bash with nothing else installed.

34:44 This is not as much fun as I envisioned it to be.

34:46 Well, the only issue is when you sit down with someone who's like, they fire up text edit to like edit some text.

34:52 You're like, what are you doing?

34:53 Like, come on.

34:55 Like, let me show you some cool tools.

34:56 Like, let me get you up to speed on here.

34:58 Yeah.

34:59 Yeah.

34:59 Text edit being the notepad equivalent if you're not a Mac person.

35:02 Yeah.

35:03 That's maybe it's a word pad.

35:05 You know, maybe it's a word pad level.

35:06 Like, that's probably where it, which is worse.

35:08 I think that's worse than notepad.

35:10 It is.

35:10 Because you're going to get weird, corrupted characters that you're not going to know.

35:13 Yeah.

35:13 Oh, man.

35:14 One of the, especially with, with working from home now, a lot of people are, are working with kids around that walk by and you want your job to look awesome.

35:24 So that they're interested in what you're doing.

35:26 Just saying.

35:27 Yeah.

35:28 I'm sure my kids think I'm a hacker.

35:29 You're not, wait, you're not a hacker.

35:34 Well, I hack on code.

35:36 Yes.

35:36 The, the original, the original.

35:40 Exactly.

35:40 The original meaning of that word.

35:42 Yeah.

35:42 Brian, time for some extras.

35:46 I do have a big extra that I'm really excited about.

35:49 Yes.

35:50 The book is, which I've been talking about for about 18 months.

35:54 And if you add that to the previous 18 months, about three years of this podcast has been talking about this book.

35:58 But anyway, so it's, there's no longer a beta flag on it.

36:04 It's not in beta anymore.

36:05 So it's, it's off to the printers.

36:07 And, and then to celebrate it being officially released.

36:12 It is, there's a coupon code that we'll link to this page in the show notes, but it's a coupon code.

36:19 I don't know how long this is going to be good for, but this is for 40% off the ebook.

36:24 So this is exciting.

36:25 And I don't have a physical copy yet.

36:28 I'm still waiting for mine to get delivered.

36:30 Hopefully it'll be in the next couple of weeks.

36:32 So, yeah.

36:34 Anyway, that's, that's one of my extras.

36:36 The other extra I wanted to bring up is a pie camp.

36:39 Spain is happening.

36:41 April 15th through the 18th.

36:44 And man, this looks fun.

36:46 It's like actual camping.

36:48 I love the idea.

36:50 Well, like, I don't know if it's actual camping, but there's a, admission includes, four days and three nights, including accommodations and breakfast, lunch and dinner provided.

37:01 This is a pretty amazing.

37:03 I know, Brian, I see karaoke on that list too.

37:06 Karaoke talks.

37:07 Yeah.

37:08 Games, ping pong.

37:09 Ah, this looks great.

37:11 You have to give your talk in song form.

37:14 Yeah.

37:14 Oh man.

37:14 No one, no one would want to see me do that.

37:16 No one.

37:17 Or me either.

37:18 So this looks great.

37:19 It does look like a lot of fun.

37:21 Fantastic.

37:24 Calvin, you got anything extra you want to throw out there?

37:26 I do.

37:26 coming up next month and just a little over a month is the Python web conference.

37:31 Oh, I've got the, let me pull up the slide for it.

37:33 You've got the screen.

37:34 I do have the screen for it.

37:36 I had even, even pre-planned for this to be ready to roll.

37:39 So it is Python web conference time.

37:41 this is our fourth.

37:42 Got any good speakers or what?

37:43 Fourth annual event.

37:44 we've got some amazing speakers.

37:47 who are going to join us this year.

37:49 So I'll actually bring that up because I'm really proud of this group.

37:52 if you scroll down through here, there is just a amazing, bunch of people who have

37:59 signed on for this amazing adventure with all of us, over here.

38:03 So definitely check it out.

38:04 It's going to be way bigger than it has been in previous years.

38:07 So this is the fourth year we've run it.

38:08 And I believe we've got 90 speakers this year.

38:11 We are doing, five tracks across five days.

38:16 So there's two app dev tracks, a culture track, a cloud track, and a pie data track.

38:21 so there is something for everybody.

38:23 get your tickets.

38:25 Now it is going to be a ton of fun.

38:27 We will start, you know, getting things cranked up a couple of days beforehand.

38:30 We've getting, you know, the Slack channels all set up and people can start basically hanging

38:34 out.

38:34 And we're going to have some cool socials.

38:36 I know we've got one of our speakers is going to give a mindfulness, social.

38:40 So if you want to come and learn how to decompress as a developer, and she's going to actually

38:45 be one of our keynotes about burnout.

38:46 but she's going to, you know, give a practical example during one of the socials that I'm super

38:51 excited to try out.

38:52 Nice.

38:53 Fantastic.

38:53 Yeah.

38:54 I see a bunch of the people in the speaker list have been here on the show.

38:57 Yeah.

38:58 These should, these should, these should not be strangers to, especially this guy

39:01 right here.

39:01 He's definitely not a stranger.

39:02 I don't know about that guy.

39:03 He's shady.

39:03 Definitely shady.

39:04 Well, I was, I'm noticing a lot of these speakers from either this, show or talk Python

39:10 or, testing code.

39:11 They've been, a lot of people have been one of those friendly faces, a lot of friendly

39:15 faces on here.

39:16 And again, great, great group of people.

39:18 They're all super excited to participate in the conference.

39:20 They're all super excited to like hang out with everybody and like, just be a part.

39:24 Yeah.

39:25 I like that you're putting, the social links up on the page so that people can check

39:29 that out instead of having to Google for them or something.

39:31 Yeah.

39:32 It's all about the people for me.

39:33 I mean, I'm, I'm, I love being a community builder and, and putting this together for folks.

39:38 Oh, look, there's another amazing speakers in the, in the audience right now.

39:41 Fantastic.

39:44 Yeah.

39:44 That is awesome.

39:45 Well, how about, no, I don't mean to cut you off, but I was just curious if Michael had

39:50 any extras.

39:50 You, you know that I do.

39:52 All right.

39:54 I got a couple.

39:54 Let me, let me tell you about this little app I got, which I meant to do a little video

39:59 so I could show you.

40:00 I'm a huge fan of macOS and, I really enjoy working there.

40:04 I love the, like the terminal tools are like server stuff, but it's not, you know, you got

40:10 all the nice little tools and whatnot.

40:12 One of the things that I absolutely just don't understand is switching between windows is like

40:19 nearly impossible if it's the same app.

40:21 If I got one web browser set of tabs and another, I'm going to cycle between them like command

40:26 tab, the alt tab equivalent has no effect on that.

40:29 Right?

40:30 Like why is this?

40:30 So I found this cool app called witch that lets you do all sorts of stuff.

40:34 like, you map it like alt tab instead of command tab and it'll pull up.

40:40 It's very similar, but then it'll, you can even like switch between tabs within a browser.

40:44 So I want to switch to Vivaldi, but onto this tab of that Vivaldi.

40:49 So how, wherever it lives on what Vivaldi window, I don't care.

40:52 I just want to go to that tab, stuff like that.

40:54 Super, super cool.

40:56 It's just cycle between the last use window instead of the last use app.

41:00 And there's just a, the customizability of it is insane.

41:03 Like it's, it's truly a crazy.

41:06 And what does it cost?

41:07 It costs $14 once.

41:08 So if that frustrated you, check that out.

41:11 Number two is I did this video called don't use loops.

41:15 Or do you actually need loops in Python?

41:17 It was really to say like some of the time you can use comprehensions of various types.

41:22 That already talked about.

41:23 But in response to that, someone said, oh, I don't really think there's any difference

41:28 between using a list comprehension and a for loop.

41:31 They're the same.

41:32 Like, how could you even tell me that they're different?

41:34 Well, one import dis from, from dis import dis, dis like, disassemble and you'll see a big difference.

41:40 But two, I put together an example that for 10 million times, basically adds the numbers one to 10 million,

41:46 even numbers one to 10 million to a list using a for loop and then using a list comprehension.

41:52 And it is about 25% faster to do the list comprehension than the loop, which isn't going to change people's world probably,

42:00 but it's, you know, something to consider.

42:02 Yeah.

42:02 So I'll link into a very small gist there.

42:05 I just.

42:07 My, that peeve of mine.

42:10 If you're going to do a loop, at the very least, don't do for I in length of something.

42:16 That's C.

42:18 That's not Python.

42:19 Yes, please.

42:21 Or, or create a number.

42:23 Count equal, you know, I equals zero.

42:26 While I lessen this, I plus plus on the inside, right?

42:29 Like, yeah, there's a lot of bad variations.

42:32 Yeah.

42:32 Those are, those are, I intentionally put things like that in interview questions

42:37 to try to see if people are really Python programmers or if they're.

42:40 Yeah, exactly.

42:41 And it might be fine that you're a C programmer coming into Python, but sometimes people will be dishonest with you during the interviews.

42:48 Like, oh yes, I use Flask all the time.

42:50 All right, how about you create a hello world view and run that?

42:54 I can't, I can't do that.

42:56 Okay, well.

42:57 Then you probably don't use Flask all the time.

42:59 You might use it sometimes.

43:00 They're not, not eight hours a day like you told me.

43:03 All right.

43:04 Another thing, another similar little gist thing is I was working on using a database API

43:12 that is async only, but I want to use it in the web app that is not async at all.

43:18 How do you do that?

43:19 All right.

43:19 We talked about like the result and blocking and all how painful that is.

43:22 So I came up with this little gist that was working great in production, in dev.

43:27 So this really simple thing, you can just wrap up an async call and say, run it.

43:33 It internally manages a little loop and it calls run async.

43:36 So you've got like a database async call.

43:40 You can just say, you know, go to the call and just say, run whatever, get the thing async

43:46 with the parameters, right?

43:47 So it's not like a decorator.

43:48 You just call it really simple.

43:50 Well, in practice, what I found trying to deploy this to a website was the database

43:56 back end was doing weird stuff with like what thread it's running on.

43:59 The web server, MicroWhisgi was like shuffling around like the order of when stuff ran on different

44:05 threads and it was freaking out the event loop.

44:07 And you get all these errors about like, this thing has become detached from its asyncio loop

44:13 or it came from one loop and it's trying to continue on another loop.

44:16 Just like, oh no, what is all this?

44:18 So I ended up coming up with a massively crazier version that people can check out that basically

44:23 coordinates all the work to a background thread, runs it all in the same place and puts it back.

44:28 It works fabulously.

44:30 It is horrifying.

44:31 So you can take it for what it is.

44:33 Anyway, as part of this conversation, maybe it works really well, but it looks really bad.

44:41 Bill Jones.

44:42 So from court, I believe sent over a thing that said, one of the problems with asyncio is if it's

44:49 already running and then you call async version, which internally happens to use the same pattern,

44:54 it's going to crash and say it's already running weird.

44:58 So there's this thing called nest IO, which allows you to basically have a reentrancy.

45:04 So if you get the runtime error, this event loop is already running.

45:07 Well, if it is just run, you know, whatever.

45:09 But that's the error you get.

45:11 So this will allow you to basically re like continue on in the same loop.

45:16 All right.

45:16 Those are all my extras.

45:17 I thought those were all a little fun.

45:20 Nice.

45:20 I can't believe combining threading and asyncio in the same little submodule is brave.

45:26 I did not get to that position willingly.

45:29 Not at all.

45:33 Like, but everything I had tried, it didn't matter.

45:36 And people say, oh, you should use asyncio dot run that manages it for you.

45:39 Yeah.

45:40 Except for that.

45:41 It wasn't working in the weird web servers that are doing all sorts of threading tricks and

45:45 just right.

45:46 Like it was the only thing that worked.

45:48 And so there it was.

45:49 All right.

45:50 That was not funny, but maybe I've got something funny for you.

45:53 You ready for a joke?

45:55 Yes.

45:55 Always.

45:56 Okay.

45:57 So this one is about mistakes that people make with testing for truthiness versus assignment.

46:07 And so it's a cartoon and there's these humans being ripped apart by robots.

46:12 It says, oh no, the robots are killing us.

46:14 And someone asks why, but why we never programmed to do this.

46:18 And then there's like a computer with some code on the screen in the background.

46:23 You see robots killing people.

46:24 And it says, it has actually the code for the robot.

46:27 It says, void, interact with humans.

46:30 If is crazy robot equals true, kill humans.

46:34 Else be nice to humans.

46:36 But it's an assignment, not an equality.

46:39 It's a single equals instead of a double equals.

46:41 End of the world has come because of that.

46:45 You've just assigned it to be a crazy killer robot.

46:46 Crazy murdering robot.

46:48 Yes, exactly.

46:49 Yeah.

46:50 Nice.

46:50 Anthony would save us from this cartoon apocalypse by saying this, you know, remember your unit test.

46:58 Yeah.

46:59 And beta testers.

47:00 Why do we keep losing QA people?

47:04 I just don't understand.

47:05 Where do they go?

47:07 I don't know.

47:08 Just ship it to beta.

47:10 Yeah.

47:10 Yeah, exactly.

47:12 Exactly.

47:13 What could go wrong?

47:13 Segment your population.

47:14 A-B test this stuff.

47:15 Anyway.

47:18 No, thanks.

47:19 Nice.

47:20 But thank you, Calvin, for coming on the show this time.

47:23 It was totally fun.

47:23 Love it.

47:24 Yeah, absolutely.

47:25 Brian.

47:25 Thank you.

47:26 Good to be here.

47:26 That was a good time.

47:27 Yeah.

47:27 Always.

47:28 See you later.

47:29 Yep.

47:29 Bye.

47:29 Bye, everyone.

47:30 Thanks for listening to Python Bytes.

47:32 Follow the show on Twitter via at Python Bytes.

47:35 That's Python Bytes as in B-Y-T-E-S.

47:38 Get the full show notes over at Pythonbytes.fm.

47:41 If you have a news item we should cover, just visit Pythonbytes.fm and click submit in the navbar.

47:46 We're always on the lookout for sharing something cool.

47:49 If you want to join us for the live recording, just visit the website and click live stream

47:53 to get notified of when our next episode goes live.

47:56 That's usually happening at noon Pacific on Wednesdays over at YouTube.

48:01 On behalf of myself and Brian Okken, this is Michael Kennedy.

48:04 Thank you for listening and sharing this podcast with your friends and colleagues.