Transcript #289: Textinator is coming for your text, wherever it is

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 289, recorded June 21st, 2022.

00:09 I'm Brian Aachen.

00:10 Hey, I'm Michael Kennedy.

00:12 And I'm Gina Heuske.

00:13 Welcome, Gina. I'm so glad that you could join us for the show.

00:16 Well, I'm still very, very honored that I'm allowed to be on here because I've been a long-time listener.

00:22 And yeah, so being on here is absolutely amazing for me.

00:25 That's very cool. So tell us a little bit about yourself before we jump in.

00:29 Yeah, well, so I think my claim to fame most likely is that I'm the creator and maintainer of Octoprint,

00:35 which is basically a web front end slash print server for 3D printers that I've been now maintaining for almost a decade, actually.

00:45 So this December, it will be a decade.

00:47 And it's written in Python, which is why I have gotten more and more interested into the language over the course of the past decade.

00:55 And also now hang out on conferences when I get the chance.

01:00 And there is no pandemic going on right now.

01:02 And in general, just kind of fell in love with the language, I got to admit.

01:06 I used to be a Java developer.

01:08 I'm very glad that I no longer am.

01:13 And now I'm a full-time open source maintainer, actually, because I'm in the very lucky position that people apparently love Octoprint enough that donations and such and sponsorships and such generate enough revenue that it can work for me.

01:27 Oh, that's awesome.

01:28 Very cool.

01:28 Pretty awesome.

01:29 That is so fantastic.

01:30 Yeah.

01:31 I still don't know why it works, but hey, I'm taking it.

01:36 Well, Michael, you got our first topic today.

01:39 I do.

01:40 I want to combine some things that we've covered before as a way to introduce something new.

01:46 So way back when we talked about something called Mangita.

01:51 Have I mentioned that I like MongoDB?

01:53 I think this might be something I've been talking about.

01:55 Like, I love MongoDB.

01:56 I think it's great.

01:57 Love working with it from Python.

01:58 It works super duper well for me.

02:00 Now, one of the things I'm envious of on the relational side is SQLite, which ships with Python, and you don't have to start up a server or anything.

02:08 Just say, here's a file.

02:09 That's my database.

02:10 Let me point database tools like SQLAlchemy and stuff at it, right?

02:14 With MongoDB, we haven't really had anything like that.

02:17 But then we did talk about this thing called Mangita, which is Mangita is to MongoDB as SQLite is to SQL.

02:23 So that's pretty awesome.

02:25 You pip install MongoDB, which has a beautiful little character for its icon.

02:31 And you pip install this thing, and then you can just create a disk connection or a memory connection or stuff like that.

02:38 It's not incredibly fast, but you know what?

02:41 Fast enough, right?

02:42 Fast enough for simple apps, for demo apps, or if you're teaching a class or something like that.

02:46 It's sort of my primary, basically demos, YouTube video demos and class examples and stuff.

02:53 I don't want to have people set up a whole database server they maybe have never set up.

02:56 Just to play with an example, right?

02:58 So here's a cool way, except for it only does like the low-level MongoDB API.

03:02 So I'm a big fan of Beanie.

03:04 I reached out to Roman Wright and said, hey, do you know of anything that will allow Beanie to work with something like Mangita?

03:12 And he said, not really.

03:14 So I'll make one.

03:15 Where did I put it?

03:17 Oh, no.

03:17 If I copied.

03:18 No, I.

03:18 Yeah.

03:19 So I had the wrong one selected.

03:20 He did Beanie, the ORM.

03:22 And I said, I want to use this ORM against the disk database, much like you would SQLAlchemy against SQLite.

03:28 He said, no, I don't know of one of those.

03:29 But what if I made Beanieta for Mangita?

03:32 And so I love the naming here.

03:35 That's why I wanted to put them all together.

03:37 So Beanieta is a local DB-like database ready to work with Beanie, right?

03:44 So basically, all you have to do to work with this is import client from there, create a client, paste that, give it a directory just like you would SQLite, pull a database off of it, and then just call initialize Beanie like you normally would, but pass it this Beanieta database.

04:02 So it just works off the file system.

04:03 So now you've got a local, mostly compatible MongoDB disk-based in-process thing that you can use for simple MongoDB examples.

04:13 It has some things it doesn't support, links, which are like following references across documents, aggregations, which is a whole crazy data analytics aspect of MongoDB, and unions, and other things that Mangita doesn't support.

04:26 So I'm pretty sure.

04:28 Let's just double check here.

04:29 Yeah, so it basically just builds that wrapper on top of Mangita.

04:33 And the real challenge that was tricky here was Beanie is only async, and Mangita is only sync.

04:38 So you've got to somehow put those together.

04:41 Anyway, thank you, Roman Wright, for doing this.

04:43 It looks really cool.

04:43 And we could come up with a joke for this, but I think that's Benita's.

04:49 Oh.

04:50 Oh, Brian, that was solid.

04:53 That hurts.

04:56 Yeah.

04:58 Yeah.

04:58 Sorry.

05:00 Not sorry.

05:01 Okay.

05:02 So I think that I'd like to talk to people about using good coding practices, especially in research and science.

05:11 So I ran across Patrick Minolt.

05:16 It's an online book, really.

05:18 It's called The Good Research Code Handbook.

05:21 And he says it's for grad students, postdocs, and PIs, which I had to look up, was principal investigator, who didn't do a lot of programming as part of their research.

05:30 And I think it's also just kind of a good thing for really anybody that's coming into coding from a different field and wants to jumpstart some good practices.

05:39 It's actually really kind of great.

05:42 He's got a little roadmap, which is neat.

05:44 I'll click on that roadmap.

05:45 A little small here.

05:46 But it goes through.

05:48 It's got a journey mapped out there.

05:50 I love it.

05:51 Yeah, it's nice.

05:51 But kind of a visual of where you're going to go with this.

05:55 But he talks about setting yourself up for success using Git and virtual environments and projects and even packaging and cookie cutter, which is cool.

06:05 Talks about style guides and keeping things clean and removing dead code.

06:11 Some coding practices like separating concerns and separating your pure functions from side effect functions, which I thought that some people don't really touch on right off the bat, but it's a good practice.

06:25 Even talks about testing, although the focus is really around unit testing and sort of some hand waves around in-to-end testing.

06:31 But still, it's nice.

06:33 And then even gets into documentation and social aspects like doing pairing and peer reviews and getting involved in open source and community.

06:43 It's actually just really kind of a pretty solid look for people.

06:48 Sounds great.

06:49 Yeah, that's really great.

06:50 I think it's, yeah.

06:51 Go ahead, Gina.

06:52 No, I just wanted to say it sounds great.

06:54 Because I sometimes have people joining Octoprint who are pretty much newcomers to the whole coding aspect, but are interested in writing a plugin or something like that.

07:04 So that might be a good resource for them to just direct them to.

07:08 I had, you know, I'm, go ahead, Brian.

07:11 I had somebody contacted me once and try to, I wish I knew about this before because somebody contacted me and said, we don't really need coding training, but we kind of aren't used to working together well.

07:22 We're all individual people.

07:24 And now we're working as a group and we don't know how to do that.

07:27 And I don't know if that's here, but a lot of this around this is sort of good practices for working well with others.

07:34 Michael?

07:35 Yeah.

07:36 What I was going to say is a lot of people, especially in this research field, learn to code in a jit fashion, like just enough learning of coding to get the problem solved.

07:47 And then they just, they have to keep moving on and it's easy to find yourself just stumbling into wrong patterns of, well, everything's into one huge file and it doesn't take like command line argument inputs.

07:59 It just hardcodes the values in.

08:01 So you can't reuse it.

08:02 It's not a module you can import or a function you can call.

08:05 And it does no error checking.

08:07 These are, yes, exactly.

08:09 No, no testing.

08:10 All of that kind of stuff would be really valuable for people who are coming from that angle.

08:14 So definitely a good thing to check out.

08:15 And I love it.

08:16 Goodresearch.dev.

08:17 What a sweet domain name.

08:19 There's also one of the neat things about the writing style is he's talking about trying to just free yourself from time.

08:30 So not free yourself from time, but as a researcher, you're in a hurry.

08:35 There's a lot of stuff to do.

08:36 And a lot of the stuff he's trying to get people to do is about offloading information so you don't have to keep it in mind.

08:44 So he talks about short-term memory and long-term memory and how, you know, like unit tests and inline code comments are good to, you know, short-term reminders in the moment.

08:56 And then project documentation and test suites are about keeping that knowledge long-term.

09:03 And that's kind of a, I never really thought about that before of short-term versus long-term information.

09:08 But it's kind of a neat angle as well.

09:10 So anyway.

09:11 Yeah, that is clever.

09:14 So where are we going next?

09:15 We have something that I brought along.

09:17 And given that I come from a 3D printing background now, I thought I would start with something that can be applied to 3D printing.

09:25 And that is a Python library for building parametric 3D CAD models.

09:30 It is based on, for those of you who have heard of the open source CAD solution called FreeCAD.

09:37 It's based on the same CAD kernel, on the same modeling kernel, and allows you to, yeah, programmatically design stuff.

09:48 And that is pretty amazing if you want to do it parametric.

09:51 So, for example, imagine that you have some kind of case for an electronics project or something.

09:56 And you need screws from, screws, screw holes at certain distances, but maybe you want to be able to scale that up or down or whatnot or things like this.

10:06 And for stuff like this, something like this is absolutely amazing.

10:10 From the ground up, CAD query is only a library, but it also comes with, as it says here, it also comes with a QT-based GUI called CQ Editor.

10:22 Nice.

10:22 Which I tried out and worked great.

10:27 The only problem was it came with an older version of CAD query that did not yet support some stuff that I wanted to play around with.

10:32 So what I did instead was I tried the Jupyter extension and just threw that up on my NAS in a Docker container.

10:39 And that works absolutely wonderfully.

10:41 In the browser, it gives you a 3D model view of the stuff that you're currently modeling, automatically refresh whenever you change something.

10:48 And the way things look is, this is the GUI, by the way, you simply create, you define a work plane, then you create a box on this, for example.

11:00 Then you say to the top face in Z direction, define that as a work plane, put a hole through that with a diameter.

11:09 So the whole API is pretty intuitive as well.

11:11 Also well documented on here.

11:14 And if you scroll a bit through the examples, you can see that there is way more possible than just planes with holes.

11:23 But things like, where was it?

11:27 Somewhere there was a...

11:29 Right.

11:31 A flask or the aforementioned parametric enclosure, a Lego brick.

11:36 So for all of that, there is examples, example code there.

11:41 And yeah, it really surprised me how quick I got this up and running because I had a very specific use case that I wanted to try to run through that.

11:50 And that was inserts for one of these part case thingies where you can put inserts into sort screws and stuff.

11:58 And that worked flawlessly with that.

12:01 I had this finished in maybe half an hour or something like that.

12:04 So really great package, works nice, has everything that you expect from a basic cut solution and also works in a browser.

12:13 So if you want to go fully parametric and already know Python, then give this a look as well as free cut.

12:19 Very cool.

12:20 Yeah, this looks super.

12:21 Yeah, I have a friend who is doing a lot of Python and CNC machines.

12:28 He has a guitar company and it gets these wood blocks and shaves out guitars and necks and things that I'm not familiar with.

12:36 Would this be applicable to something like that as well?

12:39 Not just 3D printing, but if you've got to define the polygons of it, basically?

12:44 The objects that it can output are STL files, object files.

12:48 I think step export is also available.

12:50 So all the stuff that you find commonly in the CNC and 3D printing world.

12:55 So I could just design something with this, export the STL, throw it in a slicer for my 3D printer,

13:00 or alternatively throw a resulting step file or something into whatever workflow preparation I do for my CNC router, I guess.

13:11 So I'm not that familiar with CNC.

13:13 Yeah, yeah.

13:14 I think this huge CNC machine, I think it takes STL files.

13:17 I'm pretty sure.

13:18 Well, then.

13:19 I'll have to check.

13:20 But that sounds like it could be really relevant.

13:21 A couple of pieces of feedback.

13:23 Daniel Kras says it's nice because he hates the GUI workflow of FreeCAD.

13:30 Yeah.

13:31 The funny thing is CAD query originally started out as a FreeCAD plug-in, actually, as far as I see.

13:36 So the CAD query 1 was FreeCAD, still built into FreeCAD.

13:40 CAD query 2 is now standalone and just uses the kernel and all that.

13:43 So really nice.

13:45 For people who used to use OpenSCAD, that one is the next level and really nice.

13:50 And then Brandon Branner, hey, Brandon, says, can you import this into a slicer for 3D printing?

13:56 Yeah, that was what I meant with you can export the STL and then throw that into the slicer.

14:00 Yeah, that's what I thought.

14:01 Perfect.

14:01 Awesome.

14:02 This is a great find.

14:03 I love it.

14:03 And I also, I love that you come with your experience and your view of Python and what's super interesting,

14:09 because I would have never picked this, but it's very cool.

14:13 And a lot of people are into it, right?

14:14 It's just like, I'm like, oh, FastAPI.

14:16 Yeah, it's stuff like that.

14:16 Yeah.

14:17 Different point of view, I guess.

14:19 I want to do some 3D printing.

14:21 It'll be fun.

14:22 It is, but be careful that you don't start a hobby project that then takes over your life

14:26 that sometimes happens, I've heard.

14:27 Well, so that's why I put that off, because I know that I don't need another one.

14:33 Brian, I heard you have some awesome 3D Star Wars stuff.

14:40 And maybe you could like build in, like you could print some additional stuff to go with it.

14:44 I could.

14:45 Yeah.

14:46 Or just buy it.

14:47 Yeah, there you go.

14:48 Or you could just buy it.

14:49 I just actually bought a part.

14:52 So I needed a camera mount.

14:53 So I had a camera that the one I'm using now, there's other applications where I want to use

14:59 that I don't, I didn't have a mount for a tripod mount.

15:02 And there was somebody on Etsy that it's obviously a 3D printed thing that you can buy.

15:08 So Etsy is full of people making useful things with 3D printers.

15:12 I think that's neat.

15:14 And $10 to somebody else to do it for me?

15:17 What a deal.

15:18 I'm happy to report that I'm currently looking into a camera that is sitting on a mount that I designed and printed myself.

15:25 That's neat.

15:26 Wow.

15:27 That's awesome.

15:27 Yeah.

15:28 So that mounts to a VESA monitor arm and then has a little slide that I can actually slide up and down to adjust the height.

15:36 And I also have some of these angle brackets that are with tripod mounts on the up and down side that I can use to adjust the tilt and stuff.

15:45 And yeah.

15:46 So after I did this, that thing finally set where it is supposed to sit.

15:50 So win-win.

15:52 Awesome.

15:54 That's awesome.

15:54 And just for people listening who are not watching, Gina has the best video setup of all three of us.

15:59 She looks great.

15:59 Yeah.

16:00 It's a really good studio setup.

16:01 All right.

16:02 And then parting thought on this one.

16:04 Henry Schreiner.

16:05 Hey, Henry.

16:05 Says, 3D work from Python has always been tricky.

16:07 So this looks great.

16:09 Right?

16:10 You ready for a fun one?

16:12 Yeah.

16:12 Let's switch to you.

16:13 All right.

16:14 I got something cool.

16:15 So previously I spoke about TextSniper.

16:18 This is a Mac app and it lives in the menu bar, like up by the clock.

16:23 And you just run it and you can, on Mac, you can hit Command Shift 4 and select a region and screenshot it.

16:28 This one adds Command Shift 2.

16:30 And if you select a region, it will capture the text out of whatever is behind it.

16:34 So for example, I was on a meeting on Zoom and somebody put something up and like, oh, I want to have notes on that.

16:40 And I thought, oh, I could type it.

16:41 Wait a minute.

16:41 Command Shift 2, swipe on Zoom.

16:43 Boom.

16:43 I have notes to the whole page instantly.

16:45 Right?

16:46 This is a cool app.

16:46 It's like 11 bucks for the Mac.

16:48 This I sort of randomly mentioned that I thought it was cool.

16:51 This is not what I'm talking about.

16:52 What I want to talk about is Brett, Brett Turnbull said, hey, I heard Michael talk about this TextSniper thing I just described on Python Bytes.

17:00 That's neat.

17:01 I bet I could build this in Python.

17:02 So he did.

17:03 Nice.

17:04 Isn't that dope?

17:06 So if we jump over here to the, I love the name.

17:10 I got so many good names today.

17:12 I mean, I didn't get them.

17:13 I just gathered them up.

17:14 So we have TextSniper for that thing that grabs the text.

17:17 He created the Textinator.

17:20 And it's a simple macOS status bar menu bar app that automatically detects text within screenshots.

17:26 So instead of adding a new hotkey that does its own thing, you just do Command Shift 4, copy it, and then it hooks into macOS through PyOBJC, figures out a screenshot was taken, grabs it, real quickly uses the neural engines in macOS and the Apple Silicon and stuff to do the text processing.

17:45 And then drops out the answer.

17:47 And if you look at the whole app, the entire app end to end is 450 lines of code.

17:51 That's brilliant.

17:53 Yeah.

17:53 Very cool.

17:54 And if you look at this, like, let's see, oh, boy, there's a lot in it.

17:57 So you can see it's, you know, from Foundation, from Coco, you know, Import, NS Notification Center, and all these different things is what it's basically using to listen for events of screenshots being taken and then feed those off into the various places that it needs to go.

18:14 So what do you think?

18:16 I think it's awesome.

18:17 And Rhett's in the audience.

18:19 And he says, oh, yeah.

18:21 Hey, Rhett.

18:21 Thanks for the shout out.

18:23 And then, of course, somebody says, just for Mac, power.

18:27 I'm both weasel on that.

18:28 I'm afraid.

18:29 Gina, what do you run for your OS?

18:32 Actually, Windows.

18:34 Yeah, to the big surprise of many.

18:36 But I'm also a gamer.

18:37 So I got tired of constantly dual booting.

18:42 So, yeah.

18:43 That's definitely a challenge.

18:45 So there's actually a video showing you how it works.

18:48 And it uses something I've built apps with as well called Rumps, ridiculously uncomplicated macOS menu bar apps or something like that.

18:57 And so it's really an interesting way where it's just kind of like clicking together a couple of cool things like the NS query for Spotlight with PyOBGC, the vision VN recognized text request, and like just building on top of macOS and Rumps, which is pretty cool.

19:15 You do have to give it special permissions because it has to be able to monitor your system globally.

19:20 So there's a few steps you got to follow.

19:21 But well done, Rhett.

19:22 I love the ambition here, and it looks good.

19:25 But this is incredible.

19:26 What I love about this is it's just one thing, but it's a small application.

19:30 So, I mean, potentially just you could come up with all of these pieces.

19:34 You could do other stuff, too, and have just a small example to try to learn off of.

19:39 That's pretty neat.

19:40 Indeed.

19:41 Let me just make a real quick suggestion out there.

19:45 Maybe.

19:46 I'm not sure.

19:47 I don't see it over here, but over on the releases side, on my Rumps app, I did this, and I put over in the releases, you can come over here, and you can just download a .app zipped, but it just unzips as a .app file.

19:59 So get that with some Py install.

20:01 No, Py2.

20:02 Use Py2 app on it and make it one cool further step there.

20:08 And maybe automate that with GitHub Actions.

20:11 Yes, that's a very good point.

20:12 Absolutely.

20:12 So you don't have to constantly keep doing it.

20:15 Yes, very good point.

20:17 Push to a branch, have it do some magic.

20:19 Magic.

20:20 Or click on release and have it do some magic.

20:23 That's always wonderful to watch in the log.

20:28 Yeah.

20:28 You know what I find is that people that set that stuff up, they release apps like this more frequently.

20:33 They're like, oh, it's just one feature, but I don't mind pushing the button, or I can just push to that branch.

20:37 And the more that you have to do it manually, make sure you don't mess up stuff, and don't forget to bump the version, and you just do it a lot less frequently.

20:44 So there's some really cool knock-on effects of that recommendation.

20:46 I went through a lot of pain to automate testing and release tasks in Octoprint, which also involved, by the way, flashing physical hardware.

20:58 And so I built a lot of stuff so that all of that is more or less at least semi-automated, because that takes so much work out of every single release.

21:07 That's absolutely insane.

21:08 So, yeah, automation.

21:10 It's great.

21:10 Yeah.

21:11 I do have to add.

21:13 So, yeah, there's a couple.

21:14 I don't maintain that much stuff, but I have to admit that I am reluctant to bump versions, because then I'm like, I got to push it out to PyPI then, and I haven't automated that through GitHub Actions yet.

21:27 So, yeah, I got to get on that stick.

21:29 So, okay.

21:30 I'd like to talk about locks, actually not using locks.

21:36 So, this is an article called Handling Concurrency Without Locks from Haki Benita.

21:43 And it's just sort of a nice, actually, this is intimidated at first by flipping and getting ready to read this article, but it was really well written.

21:55 So, the idea is it starts with a Django application, and it's a URL shortener, actually, you were just talking about.

22:03 But there's, so there's like an idea that you've just sort of walks through it.

22:08 The idea is you have something that creates a unique URL, but that uniqueness is checked because you keep a database of it.

22:17 And so, you check to make sure it's unique, and then you create it, make sure it's unique, and then store it in the database.

22:23 But that check has to hit the database, and you've got a read-write thing, and there's a concurrency problem.

22:30 I know there's lots of ways to get around it, but it's a reasonable use case to just think about.

22:35 But there's a lot of places within Django or within web, actually, and within actually all applications where if you're using a database, there's this concurrency thing might be a problem.

22:46 And you can get around it with locks or other things, but the discussion kind of goes through basically broadening it up a bit and just talks about collision problems and locking problems in general.

22:58 And with some nice diagrams, I can't remember where they are on the page, but nice horse.

23:02 But it's kind of a good, slow walk.

23:08 Here are some diagrams.

23:09 And then, okay, going on, continuing on with the article, the idea is like once you know it's unique, you're storing in the database, the database is already going to, if you set it up right, it's already going to make sure that he is unique.

23:24 So instead of making sure you can do it ahead of time and then doing it and making it, then checking for database errors, just assume that it's right and throw it into the database and then watch for any problems.

23:36 So basically using the database as the uniqueness indicator.

23:41 And then he talks about how to do that and then how to structure code a little bit to deal with those issues and then recover from it gracefully if that happens.

23:54 And then the side effect is you're faster because you're not putting the concurrency locks within your application.

24:04 You're depending on that uniqueness and that control of the database, which is already fine-tuned to allow multiple access and keep track of that.

24:17 Just utilize that.

24:18 So it's a neat reminder to everybody to utilize the tools under you, I guess.

24:24 Yeah.

24:25 Good reminder.

24:26 Do you do much database or Django stuff?

24:31 Me?

24:32 No.

24:32 I think I've never so far even touched Django and I try to avoid databases like the plague.

24:41 Yeah.

24:41 I'm a huge enthusiast for flat files.

24:44 And yeah, I don't know.

24:46 I can do that stuff.

24:48 They are quite valuable, right?

24:48 Yeah.

24:49 It's just, it takes so much complexity out of the system.

24:53 If you can somehow avoid having to run a database server or an SQLite file somewhere or a Benita file.

25:01 Yeah.

25:01 Yeah.

25:01 I've been working on a project, which I'm not really talking about yet, but it's, I've been thinking, well, could this just be a bunch of local JSON files?

25:09 You know, I'll talk about that later, but yeah, it's, yeah.

25:12 Could it just be?

25:12 Maybe.

25:13 If it's not, if it's 20 records or something, maybe just a JSON file is all I need.

25:17 I mean, sometimes there's no way around it, but.

25:19 Yeah.

25:19 I've got a project with a very small database and it's in, it fits, it's a very tiny text file as we, as we started out.

25:29 But, but the problem always is as long as soon as you get more than one user, you have concurrency problems.

25:35 Even if the application isn't multi-threaded or multi-processed and, and we're running into that.

25:43 So switching to, it does feel weird to have to switch to a database just for that.

25:48 And you, I mean, you can get around it also.

25:50 You can put the, you can just say, Hey, if I don't already have a database, I can put the concurrency in, in the server so that, so that that is handled there.

26:00 You can do that, but.

26:01 Could be sure.

26:01 Yeah, for sure.

26:03 My example is a client side, like desktop type story.

26:07 So there's only going to be one user.

26:08 There might be two people hitting the keyboard, but it'll be one in one app.

26:12 Cool.

26:12 But I also just really like thinking about, well, if you have a database and you have an ORM,

26:17 in what ways could you just leverage that?

26:20 And then instead of, like you said, well, let's, you know, from threading, import re-entered locks, the R lock, and then take those.

26:26 And then you do pay the price for every successful time.

26:29 And if it's one in a thousand or more that you're going to hit that problem, you know,

26:33 pay that price and that little bit of complexity of catching like the integrity error and then reporting the error instead of reverse.

26:40 That's great.

26:40 Yeah.

26:41 All right.

26:42 What do we have next?

26:44 Yeah.

26:46 Next, we have something that I have not yet had a chance to test out myself, sadly, but which I could have used roughly five or six years ago.

26:54 Really hard.

26:55 So I thought maybe I should make a mental note now and share it here.

27:00 So I will remember the next time that something like Tatsu exists.

27:04 And Tatsu is a library with which, sometimes English is hard, you can generate parsers for Python.

27:15 So Python-based parsers based on eBNFs.

27:18 So I think the E was for extended and BNF is Bakus-Naur form.

27:23 So a very structured way to define a grammar of a language of a file format of whatnot.

27:30 And textual, I should add, though maybe also binary, but that sounds like a lot of pain.

27:38 And that looks like something like, I'm looking for a good example.

27:44 Let me quickly...

27:46 Yeah.

27:48 So, for example, this is a grammar.

27:51 And then you turn that into a slightly annotated grammar.

27:58 And in the end, you run this through Tatsu.

28:02 And then Tatsu spits out something which builds an AST out of your thing that you put into it that is supposed to be parsed.

28:11 And apparently, Tatsu cannot only do that.

28:13 So it can do, on the one hand, in this kind of way that we know from the RE module, from the regular expression module in Python,

28:22 where you can compile a pattern and then reuse it during the duration of your code.

28:26 But you can also generate Python code itself out of it so that you can then import it as a module and reuse it and reuse it.

28:32 Maybe also adjust it, but I don't know how well that works.

28:34 And apparently, it is also compatible to Antler grammars, which is a name I haven't heard since university.

28:40 So that is nice, I guess, for people who have grammars lying around that are written in Antler, I think, version 4 or something like that.

28:48 And yeah, so this is really one of these things that I wish I had found sooner when I had a desperate need to generate a parser from an EBNF,

28:57 because writing an EBNF is sometimes way easier than writing a parser.

29:01 And yeah, now I have it.

29:04 But now it's too late.

29:07 For people who are listening, the EBNF, the grammar file, it looks a little bit like a YAML definition, a tiny bit, a little bit like a little bit of regex.

29:17 But you basically specify the rules of the language and then you can take that and run with it, right?

29:22 Yeah.

29:22 In a way, you basically say you have an expression and that expression consists of this other components.

29:29 And then these components are defined further down.

29:31 And so you dig deeper and deeper into the syntax and build up the definition from that, basically.

29:36 Like if you've ever seen one of these JSON grammar railroad diagrams, it also goes a bit like this, just in textual form.

29:45 Yeah.

29:46 Once I was teaching a class and it's...

29:48 Go ahead, Brian.

29:48 No, I just...

29:50 I was getting PTSD from my language classes in college.

29:53 Yeah.

29:54 That's admittedly also where I learned about this stuff.

29:57 But for some reason, I...

29:59 Yeah.

29:59 Back then it was horrible.

30:00 Now I kind of like it.

30:01 It's weird.

30:02 Yeah.

30:03 It just makes it so easy to well define something.

30:07 Yeah.

30:07 And no pressure.

30:08 And now I see the advantages because it makes it so much easier to define structured text and stuff that you need to use in machine communication.

30:17 So if I had had an EBNF or if I had had that like 10 years ago when I started an Octoprint, I would have written an EBNF for the...

30:26 Yeah.

30:27 Basically for the G-code stuff that runs between a printer and Octoprint to communicate over the serial interface.

30:33 And then it would have been way easier to pass that the way it is now.

30:36 I had to do all of this by hand.

30:38 And yeah.

30:39 But maybe I can rewrite it with this.

30:43 We'll see.

30:43 Yeah.

30:45 It seems useful if you wanted to create a simple, higher level scripting language that controls your app.

30:52 Right?

30:52 If, say, you were writing code for scientists in a lab and those scientists were not developers, but you could say like, okay, you can enter a couple of simple expressions and it'll like move the robot around or it'll like upload the results or, you know, something like that.

31:06 Right?

31:07 You could probably build that with this.

31:08 And that seems neat.

31:09 And then the other thing that's noteworthy here is this is 3.10 or above.

31:13 And I'm sure that's because it's the peg parsers match effectively the switch statement that's in Python 3.10.

31:20 True, true.

31:21 Right?

31:22 Also, with regards to data science, I could also imagine this helps maybe with parsing weird data formats that you have in Textual that fall out of some legacy software that is running on some ancient sun cluster or something that is connected to your measuring devices.

31:40 So I could imagine that could also help there.

31:42 Good idea.

31:43 Right.

31:43 Is that it for our items?

31:44 I think it is.

31:45 Do you have anything extra?

31:47 All right.

31:48 Of course.

31:49 Of course, I've got some extras.

31:51 Let's see.

31:51 Me too.

31:52 Let me pull up something real quick.

31:52 All right.

31:53 I know.

31:54 Fantastic.

31:55 All right.

31:55 I'll throw mine out.

31:57 So not here.

31:58 In a recent episode, we covered, where was it?

32:02 It was the PEP 690.

32:04 Oh, okay.

32:05 285.

32:06 Thank you.

32:06 Yes, that's it.

32:07 So we talked about PEP 690 and Lazy Imports, and this was recommended by Itamar.

32:12 This comes out of the Cinder project and other things.

32:16 Since then, I've had Barry Warsaw, Armand Bravo, and Carl Meyer on Talk Python to discuss this, which was fun.

32:23 And they've also written this up as a formal blog post.

32:26 So people were interested in that, and they wanted to just go dive in.

32:29 I just want to point out that that's a pretty intense dependency graph right there.

32:33 It's this black cloud of dots all connected and stuff.

32:36 So there's some really interesting things from Instagram's use of this and whatnot.

32:41 Very neat.

32:41 People can check that out.

32:42 There was an expedited release of Python 3.11 Beta 3 because of an incompatibility with pytest.

32:49 Brian, can you imagine?

32:50 Yeah.

32:51 It was fun.

32:53 I do know about this.

32:55 Yeah.

32:55 So I do.

32:56 I found it amusing.

32:57 In one of the announcements, it said, apparently, pytest is well used.

33:02 No, can't be.

33:05 No way.

33:08 I think it was a joke, but it made me laugh.

33:11 Yeah.

33:11 Yeah.

33:12 But the fix was no joke.

33:13 So there's a quick fix that rolled out for people if they want to check that out.

33:16 And then last thing, I kind of threw out a teaser for this episode saying, I just did something completely silly and weird.

33:23 And I think I might like it.

33:25 I'm not sure.

33:25 So for a long, long time, I've used DuckDuckGo.

33:28 Right.

33:29 And I really think that DuckDuckGo is fantastic.

33:32 And I may just keep using that.

33:33 But let me give a little credit here.

33:36 Daniel Herthom sent us a message and said, you know, this isn't really Python related, but if I know Michael, I bet he will love this ad-free privacy-respecting search engine called Kagi.com.

33:49 Have either of you heard of Kagi?

33:50 I had not.

33:51 Never.

33:52 No.

33:53 And so it's pretty interesting.

33:56 I went and I listened to or read.

33:59 I listened with my eyes.

34:00 I listened to this interview.

34:01 Watched this interview.

34:02 There's an audio version, so I keep stumbling on that.

34:05 But Vladimir Perlovak is the founder of Kagi.

34:10 It was founded in just March.

34:12 So it's really brand new.

34:13 Talks a lot about the motivation for creating the search engine.

34:16 And basically, he says, I think the ad tech, which I think maybe should be called more like surveillance capitalism or something like a little more serious.

34:25 But ad tech, putting all these ads and tracking in front of my kids.

34:27 I don't want my kids to grow up thinking that just like being completely tracked and having zero privacy is like the way of the world.

34:33 So there's a pretty interesting conversation here on what they built.

34:36 But the idea is it's a search engine that is a premium search engine.

34:40 So instead of having ads, users have to pay for it.

34:42 So I'm going to do an experiment for us, Brian.

34:44 I'm going to live with this for a month, and then I'll give you a report on what I think.

34:47 I think that'd be great.

34:48 Yeah.

34:49 Yeah.

34:49 So check this out, for example.

34:51 Pep690 lazy imports.

34:54 What happens if we put that up there?

34:56 So comes up with the peps page.

34:59 And check that out.

35:00 Number three.

35:01 And then probably somewhere a little farther down, since it's not in the title, is Python bytes, I would suspect.

35:06 But like these results are pretty interesting.

35:08 But check this out.

35:09 So I can come over here and I can hover over next to one of these.

35:12 And it says, oh, this is on this page.

35:14 This one doesn't have that much information about it.

35:17 Some of them.

35:18 Let's see.

35:19 Maybe this one should have it.

35:20 So before you even visit it, it'll say there's four trackers.

35:23 It's this ranked by traffic.

35:24 It'll use HTTPS.

35:25 It's fast or not.

35:26 It lands in this category.

35:28 And you can upvote and downvote things.

35:30 You can push things up and down.

35:32 That is a power feature.

35:33 For example, isn't that cool?

35:35 You can say W3 schools.

35:36 I never, ever want you to show up ever, ever again.

35:37 Exactly.

35:37 That was my thought.

35:38 Yes.

35:40 It's so horrible.

35:41 They're so good at SEO and they're so bad at writing meaningful, helpful stuff.

35:45 So you could just, when I got my first W3C result, schools result, I just did block that.

35:50 And so you can, for example, go to like the docs on python.org and say, you know what?

35:56 That one's higher for me because if I search for Python stuff, I'd kind of want to get the

35:59 official discussion.

36:00 Yeah.

36:01 The other thing you can do is cool is you can have these lenses.

36:03 So you're saying, I'm only interested in programming stuff right now.

36:06 So it takes you to like GitHub and stuff like that.

36:10 And then you say, you know what?

36:11 I'm just actually, what I want is just more results.

36:14 Just basically search GitHub for all of this.

36:16 And the final thought here that's cool is it deranks.

36:21 So, you know, I showed you like different sites.

36:23 So here, let's put, let's put something embarrassing for a company that needs to be embarrassed.

36:27 CNN.

36:27 So if you hover over this, how many block?

36:29 There's 43 ad detectors on this thing.

36:31 43.

36:32 43.

36:33 43.

36:33 Here's what they do.

36:35 They are down ranking and down voting, like lowering the SEO rank of things that have tons

36:42 of ad trackers and things like that.

36:45 So if you, if you're overwhelmed like this, where it's orange or red, like that's a, that lowers your rank.

36:49 So it's like an anti ad tracker result surfacer.

36:53 Anyway, what do you all think?

36:56 I'm tempted, but I'll let you test it first.

36:59 Yeah.

37:00 Give me a month.

37:03 I didn't think I was going to like it that much.

37:04 It's, it's pretty nice.

37:06 So I paid 10, it's $10 a month, which seems a little high.

37:09 I think they are getting their results by buying API access to Google and Bing and then resurfacing

37:15 those results to you.

37:17 in, in this privacy respecting way.

37:20 Yeah.

37:21 Actually, I kind of think it's brilliant.

37:23 And I, I want to try it too, because at first I was like, I don't want to pay for a search,

37:28 but I, you know, I know I try to tell my kids, if you're not, if you're not the cuss, if you're

37:35 not paying for it, then you're not the customer or the product.

37:37 so yes, absolutely.

37:40 Absolutely.

37:40 And, and even if it wasn't for that, that the customization options are just like amazing.

37:46 Like, I don't know.

37:47 Yeah.

37:48 The fact that you can say, I prefer more of this site, less of that site.

37:51 Yeah.

37:51 It's, it is pretty neat.

37:53 So, I really, I really appreciate that Daniel sent this in and I'm going to live with

37:58 it for a month and I will let you know, but the maps doesn't work very well.

38:01 They don't work very well, but you know, you can always just go to somewhere else.

38:04 A final thought.

38:05 The whole reason I switched to Vivaldi over Firefox is I didn't want to bring other things

38:10 like ad block and you have ad blocking tracker blocking and stuff there, but then you have

38:14 to get extensions.

38:14 And then those extensions send their data back.

38:18 And then sometimes they'll like, well, these ads paid to be let through our ad blocker.

38:22 And there's just like still kind of a tracking level.

38:24 So with, with Vivaldi, you know, you get like total blocking without third party things.

38:31 So with this, you kind of get a search engine that is like a good, good match to that.

38:35 So anyway, I think it's, I think it's cool.

38:37 And, anyway, I just wanted to give it a quick shout out here.

38:39 Cause it seems fun.

38:40 funny thing.

38:42 I think that's it for everything.

38:43 Okay.

38:44 Funny thing.

38:45 I, in this, the chat, eco V or Eki, Eki Voo, says I use a, I use a custom Firefox

38:53 extension to block W3 schools.

38:56 That's interesting.

38:57 I didn't know that existed, but.

38:59 Yes.

38:59 I installed that as well.

39:00 And I, Eki Voo, that is an absolutely worthwhile thing to do.

39:05 I totally agree.

39:06 So I'm with you.

39:07 I used to use that as well.

39:07 I also use something that like, I'm, it was, it was called, I'm okay with cookies and it

39:12 would automatically accept all cookie disclaimers, you know, the cookie warnings.

39:16 It would just say, okay.

39:17 And there's, I saw almost none of them.

39:18 And then I just blocked them on my network.

39:20 So it doesn't matter if I accept them or not.

39:21 well let's jump to Gina's extra stuff.

39:25 It's only some small stuff.

39:27 So one thing, can you share my, thank you.

39:32 so one thing is something that, I mentioned before that I went through a lot of pain to automate

39:37 a bunch of stuff about the testing and Octoprint, which involves some raspberry pi cluster here

39:42 that I can flash via, a Python tool and stuff like this.

39:46 And I also needed voltage monitoring for this raspberry pi cluster, because I don't know how

39:52 familiar you are, you are with raspberry pies, but they can be a bit, let's say,

39:57 selective about what power supplies they like.

40:01 And, they start browning out, easily and stuff.

40:05 So power monitoring is very helpful.

40:06 And there are these cheapish 30, 40 bucks, USB, power and voltage monitoring dongles

40:15 that you can just plug, between two USB cables, and then it will measure everything.

40:20 And they also come with Bluetooth.

40:22 And I found a tool that can, lock the data from this Bluetooth interface, via

40:28 PI BlueZ.

40:28 It's, it's written in Python from these UM24C, UM25C or UM34C dongles in order to, to allow

40:36 you to track the voltage and power over a course of, yeah, whatever.

40:41 So I have to set up, lock this out, throw that into an influx DB, throw it up on a Grapana

40:46 dashboard and get warnings when something underpowers.

40:48 And that is great for sleeping, about the test cluster situation.

40:54 And the other thing is something that reached me actually just last night in the shape of

40:59 a tweet.

40:59 And that is that there is now another node JS PI PI distribution that you can just depend

41:06 your, Python apps on that will then pull in for you a complete node JS distribution

41:12 with NPM with node and with NPX.

41:14 So if you have like me, for example, if you happen to build a web server that also ships with

41:20 a web UI built in, then now you have a way to ensure that, everything that you need

41:27 in order to build that web UI is also there and can also maybe make this part of a build

41:32 process that you build into your, set up PI or something like this as well.

41:36 So that was a really nice thing to discover.

41:40 I still considered alpha, but I tried it briefly this morning and, really great.

41:45 That's cool.

41:46 That's really cool.

41:47 That is fantastic.

41:48 Yeah.

41:49 That's super interesting.

41:50 you could pair that with PI script and have it do a little electron thing and then

41:56 just run that run Python, write Python for your little web app that runs on the front

42:00 end.

42:01 Yeah.

42:01 Or if you do something like, so for example, I still use Cypress JS for end to end testing

42:07 in Octoprint, which means that, I now need a node JS environment and a Python environment

42:12 in order to have all developer dependencies together.

42:14 This would solve this problem because then I could just make this part.

42:19 Maybe I could even fire it up as part of the PI test suite.

42:22 I don't know, but, would be interesting.

42:24 Certainly to test that out.

42:26 Definitely.

42:26 Very cool.

42:27 Brian, it sounds like a PI test fixture needs to be put in place.

42:30 If you want to do the, if you want to do that, that would be great.

42:34 I'll put it on my to-do list right after 3d printing.

42:39 yeah.

42:40 I, I know the problem.

42:42 we're kind of, since I'm running, we're running a little long, I'm going to skip

42:47 my extras till next week.

42:49 so let's, maybe jump to something funny.

42:52 Nathan Ashberger.

42:54 It's a tweet.

42:54 and it's like, you pretend you're getting a call and, and the, and the phone says, Hey,

43:00 I'm from somewhere.

43:01 we built the first AI driven sales platform.

43:04 And I wanted to talk to you about your sales needs.

43:06 he says, wait, are you an AI?

43:09 no, well, we can both appreciate the irony here, right?

43:13 So I don't know if this was real or not, but it just made me laugh.

43:19 That was funny.

43:20 So I love it.

43:21 Yeah.

43:22 Shouldn't they be having their AI call you with their AI platform?

43:25 It's subtle.

43:26 Yeah.

43:27 Okay.

43:27 So let's go.

43:28 Maybe yours is funnier.

43:29 Let's go to yours, Michael.

43:30 my, well, mine is as a, it's a collection of jokes.

43:34 So maybe one of them will be this, this comes to us from Brian's skin, who, as I said, has

43:39 been on the show before.

43:40 Thank you, Brian.

43:41 And these are, you know how, like you've got the Mars Rover badge and you've got the Arctic

43:46 vault badges on GitHub.

43:48 These are ones that were considered and potentially rejected.

43:50 Okay.

43:50 Okay.

43:51 So let's see what's over here.

43:52 These are, they were considered.

43:55 I feel like David Letterman.

43:56 With his, his like, ad stuff.

43:59 Okay.

43:59 So one of them is the vital contributor over 100 issue comments consisting of just plus

44:05 ones or thumbs up emoji.

44:06 There's the Sith Lord who wiped out someone else's commit by force pushing to the main branch.

44:12 You could have that badge.

44:13 I have that.

44:14 The procrastinator.

44:15 Do you heard?

44:16 You could be the procrastinator.

44:19 Create a repository with a single init commit with just the readme.md and never touch it for

44:23 five years.

44:26 My personal favorite is the secret Santa accidentally commit a secret API key to a public repository.

44:32 That's good.

44:34 We have the monkey rich.

44:35 Yeah.

44:36 The monkey rich is, makes a commit directly to the main branch that breaks the build.

44:40 this is fine.

44:42 It has a dog in a room that's full of fire.

44:44 We've all seen this, main meme in some form or other.

44:47 I'm sure this is fine.

44:48 Over 1000 open issues on a public repository that you own.

44:52 Ouch.

44:53 Fine.

44:57 Let's wrap it up with the last one.

44:58 Works on my machine.

44:59 The works on my machine certification suggests user error in at least 10 issue threads without

45:04 attempting to reproduce in user runtime environments.

45:06 All right.

45:08 These are pretty good.

45:08 That would be hard to automatically detect though, right?

45:11 It would.

45:13 Yeah.

45:13 Some of these you got to self-assign, I think.

45:15 Yeah.

45:15 Yeah.

45:16 Yeah.

45:16 Yeah.

45:17 But yeah, I don't know if you saw that the GitHub recently actually rolled out more achievements

45:22 that you can now try to hunt.

45:23 So it's not anymore just the Arctic stuff, but there's now stuff like Merch Shark and PR Shark,

45:30 I think they called it.

45:31 And so there are several, they are hidden.

45:34 You have to figure out.

45:35 Yeah.

45:35 Yeah.

45:35 Yeah.

45:35 They, I think last Thursday or so they rolled it out, which was also when, this, this,

45:41 repo, I think popped up or at least popped up.

45:44 I also, it also went past me on my Twitter feed and, I immediately shared it around with

45:49 people because yeah, it's, it's, it's good.

45:52 It's cute.

45:52 It's kind of sad that some of them didn't make the cut.

45:55 Didn't make the cut.

45:56 I think so for sure.

45:57 Yeah.

45:57 This is fine.

45:58 People definitely, definitely need that.

46:00 This is fine.

46:00 Absolutely.

46:01 Yeah.

46:02 All right, Brian.

46:03 So is that it for us?

46:04 I think it is.

46:05 It was a fun episode.

46:06 Thank you, Michael.

46:07 Thank you, Gina.

46:08 Absolutely.

46:08 Thank you.

46:09 Yep.

46:09 Bye everyone.

46:10 Thanks for being here, Gina.

46:11 Thank you for having me.