Transcript #272: The tools episode

00:00 Hello and welcome to Python Bytes where we deliver Python news and headlines directly to your earbud.

00:04 This is episode 272, recorded February 23rd, 2022.

00:10 I'm Michael Kennedy.

00:11 And I'm Brian Okken.

00:12 And I'm Calvin Hendricks-Parker.

00:14 Hey Calvin, so good to have you here.

00:16 I'm excited to be back.

00:17 Yeah, it's great to have you back.

00:19 I also want to say this episode is brought to you by Fusion Auth, a new sponsor.

00:22 Thank you so much to them for supporting the show.

00:24 Check them out at pythonbytes.fm/fusionauth.

00:27 More on that later.

00:28 Calvin, it's been a while since you've been here, but you're a frequent guest. How about just tell quickly people about yourself?

00:35 Sure. I've been a almost, I guess, nearly lifelong Pythonista at this point. I'm going back a long ways, but started a company back in 1999 called Six Feet Up, where I am the CTO and co-founder, and we are all things Python and all things cloud. So we're doing some cool stuff there. I've been very involved in some open source projects, like the Plone Foundation for the Plone CMS. We're very involved in Django as well, sponsoring the Django Software Foundation. So super excited to be involved in open source and all things Python.

01:04 Right on. So if I was a company or person and said, I need a Python app, I need some help with it, or maybe someone even build it, I might reach out to you all and you might build it for me.

01:11 Yeah. No, we'd love to talk about those kinds of opportunities.

01:14 The harder and more challenging, the better.

01:17 Yeah. Those are the fun ones.

01:18 Yeah. I agree. I completely agree.

01:21 I like easy.

01:22 - Well, I think this one you got here and this for the first one, Brian, maybe not so easy.

01:30 - Not so easy.

01:31 - What's up with your mocking?

01:32 Why is it not working?

01:33 - Well, your mocks.

01:34 So this is a great, actually, this is a rabbit hole, but well, mocks are a rabbit hole.

01:40 So Ned Batchelder, great guy, he writes a lot of great stuff, also maintains coverage.

01:48 But he wrote an article called, "Why Your Mocks Still Don't Work," which is a reference to an earlier article he wrote called "Why Your Mock Doesn't Work," and he wrote that in 2019.

02:02 So if you haven't read this first, so I'm gonna go back and recommend both of these.

02:07 So "Why Your Mocks Don't Work," or "Why Your Mock Doesn't Work," is an excellent article.

02:14 It starts talking about just like, - Well, to think about how mocks work, you really have to understand Python namespaces and names and how imports work and all of that stuff.

02:27 And it, maybe you don't think you should have to, but you kind of do.

02:30 So--

02:31 - And imports, those are not entirely obvious.

02:35 Like the way that that happens, - No. - compared to other languages, right, where you just say, I'm using this library.

02:40 There's like, it's way more direct.

02:42 - So Ned starts off with like this really great example of just basically two variables, X and Y pointing to a number.

02:50 And if you assign to each other, but you don't really point at another variable, you point at the thing that the variable is pointing to if you assign X to Y or Y to X or something.

03:00 And this does have to do with importing because the names that you import are just kind of variable names that point to something.

03:09 So he talks about namespaces and where things point to, talks about importing and the difference, mostly the difference between from foo import thing instead of import foo and reference it as foo thing.

03:25 Those are completely different names within your Python application.

03:31 So talk, walks through that, talks about it and why.

03:35 So if you've got a value from one module pointing to an original value, and then another one pointing to the same thing, If you mock the wrong place, you're not going to get what you think is right.

03:49 So basically, and this is terrible to discuss over podcast.

03:54 So I recommend looking at this because it's a really great example of how mocks work and why they act the way they do and you'll be able to fix a lot of your problems.

04:07 So this is a good recommendation.

04:09 But the one that we're trying to talk about right now, why your mock still doesn't work, has to do with this cool decorator thing that Mach, I don't remember when a unit test Mach added this, but there's a patch decorator.

04:22 Actually, I'm gonna link to the show notes somewhere.

04:25 We talk about the patch decorator and it's pretty neat.

04:29 Actually, I'm jumping all around, sorry.

04:31 But if you just say like patch something, what happens on your test?

04:37 Like in this example, he's got two, You got a patch cool feature dot expensive preparation and patch cool feature dot logger.

04:47 And you're not patching it with anything with this decorator.

04:49 What happens is you get these like other variables that you can add as parameters, mock prep and mock logger, that you can use those to change it.

04:59 You can change the behavior, the, you can use asserts on it.

05:03 You can change the return values on these through that.

05:07 It's just a, this is a way to get a handle into your mock object.

05:11 It's pretty cool.

05:11 And, but the trick is they have to be in the right order.

05:17 And this example looks like it's fine, but you've got logger at the bottom decorator, which is really the first, and prep as the above decorator.

05:27 And--

05:28 - It's the first one, Brian.

05:29 It says the first line says this prep one, the second one says logger.

05:33 - Yeah, but that's not the way it happens.

05:35 The catch here and the punchline is, when you're reading decorators, it goes bottom up and left to right.

05:45 >> Like onions.

05:46 >> Sure.

05:48 >> There's layers around your code and the inner layer is the first one it sees.

05:53 >> The way he has it originally written, it just fails, it doesn't work.

05:58 The second punchline is, unless you've got a real good reason to use AutoSpec because AutoSpec would have caught this problem like right away.

06:07 And what AutoSpec does is it says the thing that you're mocking, you have to call it with the same interface that the thing had in the first place.

06:16 And so if you've got a class or a module that has a function in it, you can't call other functions, whereas normal mocks, you can call whatever you want.

06:29 So the default is free for all.

06:31 What you really should do is auto spec equals true.

06:34 And really, a lot of people, me included, wish that that was the default to begin with.

06:38 But for historical reasons, it can't be because it would break other stuff.

06:42 But anyway.

06:43 Yeah.

06:44 Yeah.

06:45 If it's not the default to start with, if you put that in to say now only behave as if you are the thing that I say you're pretending to be, that's going to crash a ton of stuff.

06:55 Yeah.

06:55 So yeah.

06:56 And then the third and final punch line is is just don't use mocks.

07:01 (laughing)

07:03 Even he mentions it in both articles, avoid using mocks if you can.

07:10 - Absolutely.

07:11 Felix out in the audience says, "Tiny feature with a lot of power, this auto spec." - Yes.

07:14 - Very much like Yoda would put it, I love it.

07:16 All right.

07:17 - What do we got next?

07:18 - Thoughts on mocking real quick before we move on?

07:21 - We are just diving into that and I don't do a lot of the test work myself.

07:26 So I'm sure Brian is the super expert on all things pytest, of course, so I don't have a huge opinion here on specific mocking, although I think it is important for folks to realize that yeah, decorators go from like that closest to the function out.

07:41 It was an important call out there.

07:43 - Yeah, the other one was from thing import something, and then you patch the thing inside there.

07:49 It's kind of that, that I actually, that was news to me, that's very interesting.

07:52 All right, also news to me, please, please tell me about this, Chris May.

07:55 Thank you, Chris May, for sending this over.

07:58 And wow, I don't know if you've heard about, please folks, PLS as an LS replacement for Linux and macOS, but wow, this thing is cool.

08:09 Was this, have either of you heard about this?

08:11 - Not until today.

08:11 And I went and installed it as part of my demo so I could see it, 'cause it looks pretty darn awesome.

08:16 - So this is an LS replacement.

08:17 And I know there are other LS replacements that already exist, so you can do more things.

08:22 But this is a developer focused one written in Python That's pretty darn awesome.

08:26 So if you go look at the image I linked to, or just go check out the site, you can see that if you say, so I've aliased ls to pls.

08:36 And if I just say ls, it shows you the contents of your directory, folders on top, and then it has icons for the types of files that they are.

08:45 So I did an ls, there's a git ignore, so it has a git branch icon.

08:49 There's a license with the law.

08:51 There's a markdown file.

08:52 There's a Python file, the Python logo, but it goes beyond that, like it understands your gitignore and the files that are considered hidden are the gitignored ones as well.

09:03 And it won't show you, even if you do like a hidden file listing, it won't show you things like the .es_store on macOS because that's in the gitignore.

09:15 So the structure that it gives you is related to that.

09:17 Also for Python, if you have a virtual environment, it'll treat that directory as hidden.

09:22 And because the directory is hidden, like it's sort of suppressed in its visibility.

09:27 So there's really cool features around this that have to do with basically saying, all right, you're a developer, you're listing files that are probably developer-like.

09:37 Now what, now what do we do?

09:39 So another part that's cool is you can do an please dash dash details, which I alias to L 'cause that's the LS like equivalent.

09:48 And in this world, it shows you the same types of things, but it also shows you the size in human terms.

09:55 So instead of like 101121171112, it would say that's 10.2 gigabytes, which is nice.

10:02 But it also shows you the get status in the listing.

10:04 So like it has a dash M next to a file I modified and so on.

10:09 What do you all think of this?

10:10 - I think it's excellent.

10:12 I tried to use it and failed and you helped me understand why because you have to have fonts, like some special font installed, right?

10:22 - So one of the questions when I posted this tweet was people like, how does it have custom icons?

10:28 What magic is this?

10:29 'Cause this'll work over, like if I SSH into somewhere and install please on the server, long as my local terminal is set up correctly, this will work.

10:37 And they're like, how does this work?

10:38 Well, the way it works is you have to have your terminal font set to a nerd font.

10:43 So I've talked about nerd fonts before, but all these nerd fonts have all these special icons in them and long as your active terminal font is one of the nerd fonts, then you get all these cool behaviors.

10:54 If you don't, you get squares, which are less than awesome.

10:59 >> My only complaint with this, I wish it would have emulated more of the flags from standard LS, so that it just work alike.

11:07 I installed it, the first thing I did was like, please dash AL and I got like a no L flag.

11:13 >> We don't know what that is.

11:14 >> Yeah, I didn't know what that was.

11:16 >> Yeah, it's its own thing.

11:18 And that's why I did the alias on it.

11:19 And I'm like, okay, well, I'll just, I'll do these things.

11:22 Another thing that's interesting, it has a, you can set up these, these YAML configuration files that control how it looks.

11:30 And then you can put those kind of like Node.js does with Node modules.

11:34 Like you can put it at different, or the project.json, you can put it at different levels.

11:38 And if you go into a certain project that you've had of a configuration file and there's somewhere, it'll pick up that configuration and then use that to like customize how it looks for those.

11:49 So that's kind of an interesting thing as well.

11:52 - That is awesome.

11:52 I mean, we need more like fun stuff in our terminals as developers.

11:56 I totally love this.

11:59 This is absolutely the way to get people hooked on using the terminal in the console.

12:03 - Yes, and I was thinking about like server management, which we'll get to later, right Calvin?

12:08 - Yeah, yeah.

12:09 - But if you wanna go into the server and so I'm gonna SSH in and do a thing, like this is your entire user interface to that place.

12:17 >> It's your world.

12:18 >> Yeah, this is your world.

12:19 This is a way to bring a little bit more UI information rather than just raw white text listing of files.

12:28 I feel like there's actually a lot of value in this.

12:30 >> Totally.

12:31 >> Awesome. Let's see a Dean out in the audience and former.

12:35 I thought my life was complete until today.

12:37 I didn't know this was missing from my life, but it is.

12:40 Yes.

12:41 Please, more terminal stuff.

12:44 Absolutely.

12:45 Speaking of more terminal stuff.

12:48 Speaking of more terminal stuff, yeah, take it away.

12:51 All right.

12:52 So we over at 6PF have been very focused on the developer experience.

12:56 And so once a week we get together and talk about just, we do a code review, but really it's more of a show and tell.

13:03 This is something I showed off last week at our code and review in Show and Tell, which is a terminal that is GPU-based and written in Python.

13:12 It's called KITTY. If you've not used it, it actually is a super awesome, super fast.

13:18 Basically, I found it to just be a super smooth experience.

13:21 The reason I had revisited it was I'd been using Tmux forever and before that I was using Screen.

13:27 I wanted to just have the ultimate power tools available to me as a developer and my primary mode of operation is basically hanging out in the shell or an editor all day long, so I want the best tools possible.

13:39 I'd highly recommend folks check out KITTI.

13:42 It is absolutely tuned for performance.

13:44 You may ask, "Why the heck would I want a GPU-enabled terminal that's just showing me text?" It's because I want the most performance possible out of my system when I'm using it.

13:54 I was going to give a quick little demo here of KITTI.

13:58 This is actually using PLS.

14:00 - I'm there, yeah, I see it, fantastic.

14:02 - I have the right--

14:03 - And what is this, oh my, is this oh my posh also?

14:06 What do you got for the--

14:06 - Oh, for the bullet train down there.

14:08 So it's actually bullet train core.

14:11 It's not maintained anymore.

14:12 That's probably my next venture is gonna be replacing that line with some new--

14:17 - Starship or something like that.

14:18 - Yeah, exactly, some new system for showing my awesome prompt up there.

14:23 But yeah, if you get the right fonts installed and all that, KITTI supports, all that stuff seamlessly, It's very, very fast.

14:30 And one really cool thing, since we're all like Pythonistas and people who are listening to this would totally relate, you can extend kitty with kittens that are basically Python plugins for kitty.

14:42 So if you actually, I'll give you a quick example.

14:45 - It's terrible. - If I run kitty, I love it, but I love it.

14:47 - I kind of love it, actually.

14:49 - So you just, you invoke kitty with the kitten flag and say like what kitten you wanna use.

14:54 In this case, I'm gonna use iCat.

14:55 I'm gonna just basically echo out a Slack emoji that we use for Python.

15:02 Oops, uh-oh.

15:03 Oh no, is this gonna be like the total demo guys not being very kind to me now?

15:08 - I love that your prompt has the Python version and stuff in there, that's great.

15:14 - Oh yeah, if you're in a specific like pyenv area, it definitely shows up.

15:18 Let me just try it here.

15:20 I cat Python.

15:22 Oh, there we go.

15:23 So I must've just been in the wrong directory.

15:24 But so you can actually show graphics on the screen.

15:27 So if I want to look at an image real quick, I don't need to go to Finder and open up Preview and like do a quick look on an image.

15:34 I can actually like, you know, quick look on any image I want.

15:36 And one of the things I did want to show off, I go back to the director.

15:40 - For people who are listening, this looks like a full high res image in the terminal, based on a PNG.

15:46 - So I used iCat, which is a kitten, to display an image directly in the terminal.

15:49 And now if I wanted to do show even some coolers, because one of the features of KITTI is it's basically got a graphic subsystem in here.

15:55 If I wanted to look at my markdown files, normally if I'd use bat to look at my markdown, like for the readme that's in here, you get the content highlighted, markdown, cool, not super great.

16:08 But if I do MDCat, which is KITTY enabled, it actually leverages the KITTY subsystem here.

16:14 I can actually review my markdown file with images in line.

16:19 You can see that the XKCD cartoon is actually embedded into the readme of that document.

16:25 It does a little better job of coloring and highlighting the syntax.

16:28 If you want to preview Markdown documents without again, not going to preview or not going to rendering it to HTML and viewing your browser, you can actually view it right in your terminal with the images shown in line, which is super awesome.

16:41 One last thing, one more trick I'll show you.

16:44 There is a kitty kitten for viewing diffs in a rich tool.

16:48 If I do get diff tool, I've configured with my .gitconfig to configure an alternate diff tool for diff.

16:57 And now I get this really beautiful high-res graphic representation of my diff, my files.

17:03 So really pretty. You can see there's two diffs, two different files.

17:07 I can page through the different files and it shows me side-by-side diffs.

17:11 Is that still in the terminal?

17:13 That is still in the terminal. I did not leave my terminal.

17:15 It looks like a new window.

17:16 It did. It looked like a new window, but it's actually all in terminal.

17:19 so you can stay in your terminal, stay at your keyboard, and all the shortcuts are super nice.

17:24 One last thing.

17:26 Oh, the diff just sold me, so...

17:28 Yeah, and you can split your windows, obviously, like other kinds of terminal emulators, like if you're using Tmux, but then it's got all the layouts built in, so if I wanted, like, horizontal or vertical or grid, or if I wanted to, like, just get to the current one I'm on, you've got all that available to you with nice shortcut keys, and you can just, I got rid of TMUX and started just going all in on KDE only.

17:51 - Okay, I got a question.

17:53 So I've always been confused about this whole windowing thing because I just opened another terminal.

17:57 So--

18:00 - Oh, I see what your question is.

18:01 I don't wanna open another terminal.

18:03 I don't wanna reach for my mouse when I have to like go between terminal windows.

18:07 So I'd rather have like multiple tabs and then have splits.

18:10 And so I can have like a paging going on one window or another window.

18:14 I can then edit these things, make them shorter or taller, or split them in a different way.

18:19 I feel like just keeping my hands on a keyboard keeps me more productive as a developer as opposed to reaching over to the mouse.

18:26 I know it's not that far away, but I feel like it breaks that flow.

18:29 If you're typing away at code and you want to quickly open up a tail for a log in the same window right below where you're running the process, you're just a couple of keystrokes away and you've got that open and going.

18:41 >> Cool. Nice.

18:42 >> Yeah.

18:43 >> That's really cool. I know we're not to the joke section yet, but you know the joke about how do you generate a random string, put a new computer science student inside of them and ask them to exit.

18:53 >> Yeah.

18:54 >> I think the new version is put four of them in these different panes controlled here and ask them to exit the top right one.

19:01 >> I'll demo that right now.

19:04 >> Even more random number or word.

19:07 >> There we go. Boom, did it.

19:10 (laughing)

19:11 - I'm gonna try kitty, this looks great.

19:13 - It's super configurable, it supports all the nerd fonts, all the color schemes, I've got, you know, there's a bunch of cool plugins for it.

19:19 I've got a search plugin which searches through your, back through your terminal.

19:23 And so if I was, there's nothing in that specific one, but if I search through here and wanted to search backward, it uses FZF to do searching backwards.

19:32 So if I was looking for like LS, you see it highlighted the word LS or 2020.

19:36 And then if I had multiple ones I could just arrow back up between them.

19:39 - Nice.

19:40 - So it's all built in. - Fantastic.

19:42 Yeah, now really cool.

19:43 Also really cool is our new sponsor, Fusion Auth.

19:47 Thank you them for sponsoring this episode.

19:50 So let me just tell you really quickly about them.

19:52 They're an authentication and authorization platform built for devs by devs.

19:57 It solves the problem of building essential user security without adding the risk and distraction from your main app.

20:04 Fusion Auth has all the features you need for great support at a price that won't break the bank.

20:10 And you can either do self-hosting or you can get a fully managed solution running in any AWS region.

20:16 So if you've got a side project that needs a custom login registration or multi-factor authentication, social logins or user management, you can download FusionAuth and get the community edition for free.

20:27 And the best part is you can have unlimited users and there's no credit card or subscription required for that.

20:32 So check them out at pythonbytes.fm/fusionauth or just click the link in your podcast player show notes and let them know you came from us.

20:41 - I want that T-shirt.

20:42 That's cool.

20:43 - I know.

20:44 Yeah, you got to get a cool T-shirt, cat slash et cetera slash pow password D.

20:50 Very cool.

20:51 - Nice.

20:52 - I love when you get a cool T-shirt from our sponsors.

20:56 All right.

20:57 - Well Susan likes cats.

20:58 - I know, and kitties.

20:59 - And kitties.

21:00 - I can tell that you're already a fan of kitties.

21:04 - Well, so I don't know if Kitty is parallelized, but yeah, so I want to talk about parallels.

21:12 And parallelization, say that three times.

21:15 Anyway, I found this article by Jamie Welta, cool last name, actually, but it's called "Futures in Easy Parallelization." And I was like, you know, it was a pretty short article and I was like, it can't be that easy.

21:32 But so this isn't talking about AIO stuff or async IO.

21:37 It's talking about, this one is talking about thread futures with threads.

21:44 And this is, it's pretty cool.

21:48 The idea is you've got just, maybe you've got, and it starts off with a simple example.

21:53 I just have some work that I want done and I want it done on different threads or different processors.

21:59 So this example brings up a thread pool, a thread pool executor, and then runs them all at the same time.

22:08 Does an executor submit?

22:10 And it's just a small snippet of code, just a handful of lines.

22:14 And then I tried this out.

22:15 So this example actually is kind of boring.

22:17 It's just doing like a power X to the power of two or X squared on a couple of things.

22:23 I think that's what star star is, isn't it?

22:25 Is that power of?

22:27 I don't remember.

22:28 Yeah.

22:29 X to the power of two.

22:30 X, yeah, X squared.

22:31 - X squared.

22:32 So the other example seems a bit more down to earth and that's, and yes, on our screen, on the YouTube screen is just, it's the entire program here.

22:46 It's just, you're taking a few websites and a couple of pages to go to and then actually just slurping those down with requests and grabbing something about them.

22:58 And this example just as the result, whether it's a 200 or something like that.

23:03 But it's a really short example and you've got parallelization going on.

23:07 And I played with this, just downloaded it.

23:09 The only, we'll have it in our show notes.

23:11 The only error on this that I had to do is it's using, it's using time in here as just like a debug thing.

23:18 And in the example, it doesn't import time.

23:20 So you have to add that import time and it runs just fine like this.

23:24 And this is a pretty quick way to add parallelization for some quick task.

23:29 So I kind of like it.

23:30 There's occasionally, especially like I would do it.

23:33 There's a lot of huge log files I have to parse or big data files that I'm looking for stuff on or grabbing error logs off of different systems.

23:44 And this would be a great example to just grab them all at the same time and pull them in.

23:49 - Yeah, yeah, that's nice.

23:50 You know, one thing that's cool about the futures you get back from the Threadpool executor, as you can say dot result and it blocks, whereas on async IO it throws an exception and says it's not done rather than just blocking.

24:04 - Oh yeah, yeah, and that's, the article kind of talks about that.

24:07 It's one of the simplest things is you tell the executors, you call executor submit and that gets the jobs ready, but that doesn't block.

24:15 Those are, you can submit as many as you want.

24:19 And then in the example, he's just using like future.result in a list comprehension.

24:28 And that for each of those result, that'll block until the next one's done.

24:32 And this one's doing it in an order of which one you submitted it.

24:35 That might not be the order they finish in, but you don't really care because you just wanna wait till they're all done anyway.

24:41 So just keep doing it. - Yeah, exactly.

24:43 If you block on the first one, it's not done.

24:45 The second one might finish first, but it'll both be done by the time you get to them.

24:49 - Yeah. - Yeah.

24:50 - Yeah, and this is a lot more natural for folks who may not be used to like async IO too.

24:55 - Oh, I've been deep in the async IO world.

24:57 - Well, it all built natural here.

25:00 - And that's exactly--

25:01 - Well, and I have scars now, let me tell you.

25:03 - That's exactly why I wanted to bring up this article is 'cause there's a lot of stuff, we're like maybe a DevOps person or something.

25:10 They're not writing async programs, but they might have async need or asynchronous needs that can be solved with a simple, some simpler code.

25:19 So.

25:20 This is very, very elegant.

25:22 Easy to understand.

25:23 Yeah.

25:23 Nice article.

25:25 Good one.

25:25 All right.

25:25 Well, I want to talk about databases and more tools.

25:29 I feel like this is just the tool focused tool episode.

25:32 It is.

25:33 It's cool time.

25:34 All right.

25:34 So I did an episode on talk Python with Emily Morehouse, glyph and Henek and Henek pointed out this thing called PG mustard.

25:45 Have you heard of this?

25:46 No.

25:47 Have you?

25:47 Oh, well, I just listened to that.

25:49 Excellent.

25:49 - Well, thank you.

25:50 So now you know, other than that, like I had not heard of it, but one of the challenges, so many websites, I just don't understand why the world works this way, but you go to the website and it just spins and spins.

26:01 And the clunkier, the more internal the thing looks, the more likely it is to take 10 seconds to do whatever it's doing.

26:08 Right, but you know, it's doing some database query, probably without an index.

26:12 Maybe they gave it an index, but the index is not being used 'cause it's actually filtering first on some other random thing.

26:18 filtering first on some other random thing or whatever.

26:20 So databases have this cool feature to say, given a query, explain how you're gonna execute this.

26:27 MongoDB has this, Postgres has this and so on.

26:31 Okay, so that comes out as text.

26:33 What if you could have a better way that gave you advice?

26:35 And that's what this tool here is, which does cost money.

26:39 Just to be clear, this is a commercial thing, but I thought it was cool enough that I wanted to point it out to people.

26:43 So what you do is you give it your, basically a select statement and you ask it to explain it.

26:51 And it'll break down the explain statement into different sections and tell you, this part is really good, this part of the query could be improved and so on.

27:03 And then it gives you, so like it'll show you this beautiful visual way of explaining it and you can dive into it.

27:11 And if you click on it, it'll tell you things like, okay, this is a nested loop and on this part of the query, it took 152 milliseconds, you got 100 rows back.

27:19 And then it actually describes the situation, why it's good or bad.

27:24 So for example, it says you got five stars because you discarded 1.3 million rows, that makes it faster, but you only got 3.2 stars because the row estimate was out by a factor of 42.

27:37 You know, try to get that from text, right?

27:38 This is super helpful if you've got a slow site that you wanna say, okay, this page is slow, These are the three queries that run when we pull this page.

27:48 Why are they slow?

27:49 How can I make them better?

27:50 So it's, you know, we talked about the Regex 101 thing, how it like kind of guided you through and gave you recommendations.

27:56 This is like the database equivalent.

27:57 - This is nice.

27:58 - This is awesome.

27:59 I can't believe I've never seen this before.

28:01 - Well, I know, I can't either.

28:03 I'd never seen it either.

28:04 - Now from the interview, it sounded like it would recommend how you could change your query to make it better.

28:11 Is that something that has in there?

28:12 Or did I imagine that?

28:14 I believe so, and I believe so.

28:15 If you like open up this 3.2 stars, I think it'll give you descriptions about what could be better in there.

28:23 So it can give you performance advice, including high index potentials, poor index efficiency, disk operations.

28:30 So like you've got to read too much off this to answer this question.

28:33 Poor cache performance, excessive heap fetches, read efficiencies, glossy bitmap scans, and on and on and on, right?

28:43 So it's pretty cool, it runs on Postgres 9.6 or newer.

28:48 - I hope you're newer than that 9.6.

28:49 That's pretty great, it supports that far back.

28:53 - Yeah, yeah.

28:54 Anyway, it is a paid tool, but man, if you could for $95 make your website 10 times faster and people have been complaining and complaining, you don't have to rewrite anything, you just put in like a slightly different hint or index or change the order of a query, like that's worth a lot I think.

29:12 I think I'm just gonna try to learn this and then set up a page to say I'm a database optimization consultant and just run this in the background and say--

29:21 - Yeah, look, $500 an hour.

29:23 I will absolutely come in there.

29:25 - Yeah, it's a new business idea.

29:25 - And I have this proprietary set of tools that I just, I can't talk about it, but you let me in there and magic's gonna happen.

29:34 - I shouldn't have said that out loud.

29:36 - Yeah, well, the market will be swamped with these folks.

29:40 - Yeah.

29:41 Anyway, I thought this was cool, so I wanted to give it a shout out.

29:44 - Cool, nice. - That is super cool.

29:46 - Yeah, all right, Calvin, you got the last one?

29:48 - All right, last one up, continuing the tools parade.

29:52 Another tool, this is also cross-platform.

29:54 I didn't mention that before.

29:55 One of the reasons I really like KITTY was the fact I can use it anywhere.

29:58 Linux, BSD, Windows, Mac, there are downloads for all those platforms.

30:04 This one doesn't support Windows, but it does support Linux, Mac, and FreeBSD.

30:09 It's bpytop.

30:11 I used to be a longtime user of Glances.

30:13 And if you didn't know what Glances was, you were also missing out.

30:16 Because Glances is an awesome way to see what's going on in your system, like what I/O is being used, how much memory is being used, how much swaps being used.

30:23 And BPyTop is kind of the next generation of that.

30:26 So I will show a quick little demo of this one as well.

30:30 So for those of you who aren't familiar with Glances, this is what Glances looks like.

30:34 I got, sorry, a little interruption there.

30:39 But while I'm live here on the podcast, this is Glances.

30:42 It's kind of like more tech, very techsy, kind of old school looking.

30:47 But if you just pipx install, and that's how I typically install these kinds of tools.

30:51 It will start, sorry about that.

30:53 - pipx is the homebrew Python thing, right?

31:00 - You just pipx install bpytop.

31:04 If you don't have pipx installed, you should install pipx because that'll get you access to all the stuff and your path all set up.

31:11 But now I just use BPyTop and you get this like nice, colorized view of like a dashboard with all really like just laid out well.

31:22 And then all of the, you can kind of see on here, there's like characters that are shaded in a different color.

31:29 If you hit those characters, you'll be able to like resort or if you see the little numbers, I can hide and show the CPU or the memory graph if I don't care about that one so much.

31:41 - So for people who are just listening, here's a terminal app that has like task manager or activity monitor levels of sort of graphs going on of like here's the CPU over time, here's the running processes sorted by CPU, here's the network.

31:58 This is probably more useful than activity monitor, honestly.

32:01 - Oh, I think it is.

32:02 I mean, what's nice is you get the trending metrics over time. So you can see by CPU core or by like kind of aggregated CPU view, how you know if you're seeing the spikes or what this is really useful if you're on a server and like something's periodically happening, you're not sure what and you can kind of track down like either I. O. Issues or CPU spikes. You can kind of see if they're becoming very like periodic.

32:26 Maybe they're happening every minute or every like five minutes like on the dot and you're like, oh, that's weird. There's something like maybe it's this cron job. So it helps you track that track those kinds of things down.

32:34 You can also inspect the processes.

32:36 So you can arrow up and down and you can actually like hit return and see like what CPU specifically or memory like a specific process is.

32:43 You can like dig down into a specific process and see what core it's actually running on.

32:48 It's just, again, for tracking down performance issues, I'm just using it locally on my own laptop right here, but I've used this numerous, every machine I log into for, you know, customer or production type stuff, if they're still using virtual machines, this is absolutely installed so that when they're like, "Oh, something's slow," or "Something's doing weird," I just go fire these up and take a look real quick to get kind of a snapshot in real time what's going on.

33:11 - I've done that with Glances.

33:13 - Yeah, that was used to be my go-to.

33:15 But I just found Beefy Top was like the super powered version of Glances.

33:18 - Interesting, you've moved over.

33:19 Yeah, the graphs are way better.

33:21 - Yeah.

33:22 - Like you have progress bars or like meter bars that are graphical in Glances if you make it wide enough, but they don't have over time, they're just like snapshots.

33:31 And these are like beautiful gradient colors.

33:33 Like I kind of just expanded the net one so you can see the traffic going across my card.

33:38 You can also switch back and forth between different interfaces on the system. So if you've got multiple network interfaces, you can see the aggregate or just for a specific interface.

33:47 Yeah, super helpful when trying to track down weird, in quotes, weird issues.

33:52 How's it run on KITTI?

33:54 It runs great on KITTI. The performance is amazing.

33:57 That's another thing I didn't show you is like you can actually see with the NVIDIA SMI tool, like there is kitty using up 20 megabytes of my GPU memory on there.

34:07 My goodness.

34:08 No, that's pretty awesome.

34:09 Speaking of beautiful, Alvaro says B pi top has themes.

34:13 Yes, they use the dark yellow theme.

34:16 Yeah, I love the fact that anything's got themes I can customize like, you know, kitty or even be pi top or my my whatever my ID is.

34:24 I just I trick that stuff out.

34:25 This is my environment.

34:26 to be as comfortable and as productive as possible.

34:29 So the more customizable, the better.

34:31 And the more emojis, the better.

34:32 Love emojis.

34:33 - Yes.

34:34 (laughing)

34:36 There's something to be said about it.

34:37 If you sit down and you're like, I am excited.

34:39 Look how cool this is.

34:40 - Look how cool it is.

34:41 - Yeah.

34:41 - Yeah.

34:42 - I'm using like an old version of Bash with nothing else installed.

34:45 This is not as much fun as I envisioned it to be.

34:47 - Well, the only issue is when you sit down with someone who's like, they fire up text edit to like edit some text.

34:52 You're like, what are you doing?

34:53 Like, come on, like, let me show you some cool tools.

34:56 Let me get you up to speed on here.

34:58 >> Yeah. TextEdit being the notepad equivalent if you're not a Mac person.

35:02 >> Yeah.

35:02 >> That's maybe it's a WordPad.

35:05 Maybe it's a WordPad level.

35:06 That's probably where it, which is worse.

35:08 I think that's worse than notepad because you're going to get weird corrupted characters that you're not going to know.

35:13 >> Yeah.

35:13 >> Oh, man.

35:15 >> Especially with working from home now, a lot of people are working with kids around that walk by, and you want your job to look awesome so that they're interested in what you're doing.

35:26 Just saying.

35:27 >> Yeah, I'm sure my kids think I'm a hacker.

35:30 >> You're not a hacker?

35:35 >> I hack on code, yes.

35:37 >> The original.

35:39 >> Exactly, the original meaning of that word.

35:42 >> OG hacker.

35:43 >> Yeah.

35:44 >> Brian, time for some extras.

35:46 >> I do have a big extra that I'm really excited about.

35:49 Yes, the book which I've been talking about for about 18 months.

35:54 And if you add that to the previous 18 months, about three years of this podcast has been talking about this book.

35:59 But anyway, so it's, there's no longer a beta flag on it.

36:04 It's not in beta anymore.

36:06 So it's off to the printers.

36:07 And then to celebrate it being officially released, there's a coupon code that will link to this page in the show notes, but it's a coupon code.

36:19 I don't know how long this is going to be good for, but this is for 40% off the ebook.

36:24 So this is exciting.

36:26 And I don't have a physical copy yet.

36:28 I'm still waiting for mine to get delivered.

36:30 Hopefully it'll be in the next couple of weeks.

36:32 So yeah, anyway, that's one of my extras.

36:36 The other extra I wanted to bring up is a PyCamp Spain is happening April 15th through the 18th.

36:44 And man, this looks fun.

36:46 - It's like actual camping.

36:49 I love the idea.

36:50 >> I don't know if it's actual camping, but admission includes four days and three nights, including accommodations and breakfast, lunch, and dinner provided.

37:01 This is pretty amazing.

37:03 >> I know, Brian, I see karaoke on that list too.

37:06 >> Karaoke talks?

37:08 >> Yes.

37:08 >> Board games, ping pong? This looks great.

37:11 >> You have to give your talk in song form.

37:14 >> Yeah.

37:14 >> Oh, man. No one would want to see me do that.

37:16 >> No one.

37:17 >> Or me either.

37:18 This looks great. It does look like a lot of fun Fantastic Calvin got anything extra you want to throw out there?

37:26 I do coming up next month and just a little over a month is the Python web conference Oh, I've got the let me pull up the slide for it. You've got the screen I do have the screen for it I had even even pre-planned for this to be ready to roll. So it is Python web conference time This is our four good speakers or what fourth annual event we've got some amazing speakers who are going to join us this year.

37:49 So I'll actually bring that up because I'm really proud of this group.

37:52 If you scroll down through here, there is just an amazing bunch of people who have signed on for this amazing adventure with all of us over here.

38:03 So definitely check it out.

38:04 It's going to be way bigger than it has been in previous years.

38:06 So this is the fourth year we've run it.

38:09 And I believe we've got 90 speakers this year.

38:11 We're doing five tracks across five days.

38:16 So there's two app dev tracks, a culture track, a cloud track, and a Pi data track.

38:21 So there is something for everybody.

38:23 Get your tickets now.

38:25 It's going to be a ton of fun.

38:27 We will start getting things cranked up a couple days beforehand.

38:30 We're getting the Slack channels all set up and people can start basically hanging out and we're going to have some cool socials.

38:36 I know we've got one of our speakers is going to give a mindfulness socials. So if you want to come and learn how to decompress as a developer, and she's going to actually be one of our keynotes about burnout.

38:46 But she's going to give a practical example during one of the socials that I'm super excited to try out.

38:52 Nice. Fantastic.

38:53 Yeah, I see a bunch of the people in the speaker list have been here on the show.

38:57 Yeah, these should not be strangers to especially this guy right here.

39:01 He's definitely a stranger. I don't know about that guy. He's shady.

39:03 Definitely shady.

39:04 >> Well, I'm noticing a lot of these speakers from either this show or Talk Python or Test and Code.

39:11 They've been, a lot of people have been one of those.

39:14 >> A lot of friendly faces on here.

39:16 Again, great group of people.

39:18 They're all super excited to participate in the conference.

39:20 They're all super excited to hang out with everybody and just be a part.

39:24 >> Yeah. I like that you're putting the social links up on the page so that people can check that out instead of having the Google forum or something.

39:31 >> Yeah. It's all about the people for me.

39:34 I love being a community builder and putting this together for folks.

39:38 Now look, there's another amazing speakers in the audience right now.

39:42 >> Fantastic.

39:44 >> That is awesome.

39:45 >> Well, how about, no, I don't mean to cut you off, but I was just curious if Michael had any extras.

39:51 >> You know that I do. I got a couple.

39:54 Let me tell you about this little app I got, which I meant to do a little video so I could show you.

40:00 I'm a huge fan of macOS, And I really enjoy working there.

40:04 I love that like the terminal tools are like server stuff, but it's not, you know, you got all the nice little tools and whatnot.

40:12 One of the things that I absolutely just don't understand is switching between windows is like nearly impossible if it's the same app.

40:21 If I got one web browser set of tabs and another, I'm going to cycle between them, like command tab, alt tab equivalent has no effect on that, right?

40:30 Like, why is this?

40:30 So I found this cool app called which that lets you do all sorts of stuff.

40:34 Like, you map it like alt tab instead of command tab and it'll pull up.

40:40 It's very similar, but then it'll, you can even like switch between tabs within a browser.

40:44 So I want to switch to.

40:45 The Vivaldi, but onto this tab of that Vivaldi.

40:49 So how, wherever it lives on what Vivaldi window, I don't care.

40:52 I just want to go to that tab, stuff like that.

40:54 Super, super cool.

40:56 Just cycle between the last use window and say the last use app.

41:00 and there's just a, the customizability of it is insane.

41:03 Like it's truly crazy and what does it cost?

41:07 It costs $14 a month.

41:08 So if that frustrated you, check that out.

41:12 Number two is I did this video called Don't Use Loops.

41:16 Or do you actually need loops in Python?

41:18 It was really to say like some of the time you can use comprehensions of various types.

41:22 That I already talked about.

41:23 But in response to that, someone said, oh, I don't really think there's any difference between using a list comprehension and a for loop.

41:31 They're the same.

41:32 Like, how could you even tell me that they're different?

41:34 Well, one, import dis from dis, import dis, dis like a disassembly, and you'll see a big difference.

41:40 But two, I put together an example that for 10 million times, basically adds the numbers one to 10 million, even numbers one to 10 million to a list using a for loop and then using a list comprehension.

41:53 And it is about 25% faster to do the list comprehension than the loop, which isn't going to change people's world probably, but it's, you know, something to consider. Yeah. So I'll link into a very small gist there.

42:06 I just, my, my people of mine, if you're going to do a loop at the very least, don't do for I in length of something. That's C, that's not Python. So >> Yes, please. Or create a number, i equals zero while i less than this i plus plus on the inside.

42:29 There's a lot of bad variations.

42:31 >> Yeah. I intentionally put things like that in interview questions to try to see if people are really Python programmers or if they're just C programmers.

42:42 >> It might be fine that you're a C programmer coming into Python, but sometimes people will be dishonest with you during the interviews.

42:48 like, oh, yes, I use flask all the time. All right. How about you create a Hello World view and run that? I can't do that. Okay, well, then you probably don't use flask all the time. You might use it sometimes or not, not eight hours a day, like you told me. All right. Another thing, another similar little gist thing is I was working on using a database API that is async only, but I want to use it in the web app that is not async at all?

43:18 How do you do that?

43:19 Right, we talked about like the result and blocking and all how painful that is.

43:22 So I came up with this little gist that was working great in production, in dev.

43:27 So this really simple thing, you can just wrap up an async call and say, run it.

43:33 It internally manages a little loop and it calls run async.

43:36 So you've got like a database async call.

43:40 You can just say, you know, go to the call and just say run, whatever, get the thing async with the parameters, right?

43:47 So it's not like a decorator, you just call it really simple.

43:50 In practice, what I found trying to deploy this to a website was the database backend was doing weird stuff with like what thread it's running on, the web server, microWSGI was like shuffling around like the order of when stuff ran on different threads and it was freaking out the event loop and you get all these errors about like, this thing has become detached from its async IO loop or it came from one loop and is trying to continue on another loop, just like, oh no, what is all this?

44:18 So I ended up coming up with a massively brazier version that people can check out that basically coordinates all the work to a background thread, runs it all in the same place and then puts it back.

44:29 It works fabulously, it is horrifying.

44:31 So you can take it for what it is, anyway.

44:34 As part of this conversation, - Maybe it works really well, but it looks really bad.

44:40 Bill Jones, so from court, I believe, sent over a thing that said, "One of the problems with async I/O "is if it's already running "and then you call a sync version "which internally happens to use the same pattern, "it's gonna crash and say it's already running." Weird.

44:58 So there's this thing called nest I/O, which allows you to basically have a re-entrancy.

45:05 So if you get the runtime error, this event loop is already running.

45:08 Well, if it is just run, you know, whatever.

45:10 But that's the error you get.

45:11 So this will allow you to basically re like continue on in the same loop.

45:16 All right, those are all my extras.

45:17 I thought those were all a little fun.

45:20 - Nice.

45:20 - I can't believe combining threading and async IO in the same little sub module is brave.

45:27 - I did not get to that position willingly.

45:30 (both laughing)

45:32 Not at all.

45:33 Like, but everything I had tried, It didn't matter and people say, oh, you should use asyncio.run, that manages it for you.

45:40 Yeah, except for that it wasn't working in the weird web servers that are doing all sorts of threading tricks.

45:45 And just, right, like, it was the only thing that worked and so there it was.

45:50 All right, that was not funny, but maybe I've got something funny for you.

45:54 You guys ready for a joke?

45:55 - Yes, always, definitely.

45:57 - So, this one is about mistakes that people make with testing for truthiness versus assignment.

46:07 And so it's a cartoon and there's these humans being ripped apart by robots.

46:12 It says, "Oh no, the robots are killing us." And someone asks, "Why?" "But why?

46:17 "We never programmed to do this." And then there's like a computer with some code on the screen.

46:22 In the background, you see robots killing people.

46:24 And it says, it has actually the code for the robot.

46:27 It says, "Void, interact with humans.

46:31 If is crazy robot equals true, kill humans else be nice to humans.

46:36 >> But it's an assignment, not a quality.

46:39 >> It's a single equals instead of a double equals.

46:42 End of the world has come because of that.

46:45 >> You just assigned it to be a crazy killer robot, crazy murdering robot.

46:48 >> Yes, exactly.

46:49 >> Yeah, nice.

46:51 >> Anthony would save us from this cartoon apocalypse by saying this, remember your unit tests.

46:58 >> Yeah, and beta testers.

47:01 Why do we keep losing QA people?

47:04 I just don't understand.

47:06 Where do they go? I don't know.

47:09 >> Just ship it to beta.

47:11 >> Yeah, exactly.

47:13 >> Segment your population, AB test this stuff.

47:16 >> Anyway.

47:18 >> No, thanks.

47:19 >> Nice.

47:20 >> But thank you, Calvin, for coming on the show this time.

47:23 >> It was totally fun. Love it.

47:24 >> Yeah, absolutely. Brian.

47:26 >> Thank you. That was a good time.

47:27 >> Yeah.

47:27 >> All right.

47:28 >> Always.

47:28 >> See you later.

47:29 >> Yeah. Bye.

47:29 >> Bye, everyone.

47:30 Thanks for listening to Python bytes. Follow the show on Twitter via @Pythonbytes. That's Python bytes as in B Y T E S. Get the full show notes over at Python bytes.fm. If you have a news item we should cover, just visit Python bytes.fm and click Submit in the nav bar. We're always on the lookout for sharing something cool. If you want to join us for the live recording, just visit the website and click live stream to get notified of when our next episode goes live. That's usually usually happening at noon Pacific on Wednesdays over at YouTube.

48:01 On behalf of myself and Brian Okken, this is Michael Kennedy.

48:05 Thank you for listening and sharing this podcast with your friends and colleagues.