Transcript #289: Textinator is coming for your text, wherever it is

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 289, recorded June 21st, 2022.

00:10 I'm Brian Okken.

00:11 Hey, I'm Michael Kennedy.

00:12 And I'm Gina Häuskel.

00:13 Welcome, Gina. I'm so glad that you could join us for the show.

00:16 Well, I'm still very, very honored that I'm allowed to be on here because I've been a long-time listener and yeah, so being on here is absolutely amazing for me.

00:26 That's very cool. So tell us a little bit about yourself before we jump in.

00:29 Yeah, well, so I think my claim to fame most likely is that I'm the creator and maintainer of Octoprint, which is basically a web frontend/print server for 3D printers that I've been now maintaining for almost a decade, actually.

00:45 So this December, it will be a decade.

00:47 And it's written in Python, which is why I have gotten more and more interested into the language over the course of the past decade and also now hang out on conferences when I get the chance and there is no pandemic going on right now. And in general, just kind of fell in love with the language, I got to admit, I used to be a Java developer. I'm very glad that I no longer am. And now I'm a full time open source maintainer, actually, because I'm in the very lucky position that people apparently love OctoPrint enough that donations and such and sponsorships and such generate enough revenue that it can work for me.

01:28 >> Pretty awesome.

01:29 >> That is so fantastic.

01:30 >> I still don't know why it works, but hey, I'm taking it.

01:36 >> Well, Michael, you got our first topic today.

01:39 >> I do. I want to combine some things that we've covered before as a way to introduce something new.

01:47 Way back when, we talked about something called Manguita.

01:52 If I mentioned that I like MongoDB, I think this might be something I've been talking about.

01:55 I love MongoDB. I think it's great.

01:57 Love working with it from Python.

01:58 It works super duper well for me.

02:00 Now, one of the things I'm envious of on the relational side is SQLite, which ships with Python and you don't have to start up a server or anything.

02:08 You just say, here's a file.

02:09 That's my database.

02:10 Let me point database tools like SQLAlchemy and stuff at it.

02:14 Right?

02:14 With MongoDB, we haven't really had anything like that.

02:17 But then we did talk about this thing called Mongita, which is Mongita is to MongoDB as SQLite is to SQL.

02:24 So that's pretty awesome.

02:25 You pip install Mongita.

02:26 which has a beautiful little character for its icon.

02:31 And you pip install this thing, and then you can just create a disk connection or a memory connection or stuff like that.

02:39 It's not incredibly fast, but you know what?

02:41 Fast enough, right?

02:43 Fast enough for simple apps, for demo apps, or if you're teaching a class or something like that, it's sort of my primary, basically, demos, YouTube video demos and class examples and stuff.

02:53 I don't want to have to people set up a whole database server.

02:55 they maybe have never set up just to play with an example, right?

02:58 So here's a cool way, except for it only does like the low level MongoDB API.

03:02 So I'm a big fan of Beanie.

03:04 I reached out to Roman Wright and said, "Hey, do you know of anything that will allow Beanie to work with something like Monguita?" He said, "Not really, so I'll make one." Now, where did I put it?

03:17 Oh no, if I copied, no, I, yeah.

03:19 So had the wrong one selected.

03:20 He did Beanie the ORM.

03:22 And I said, "I wanna use this ORM against the disk database, much like you would SQLAlchemy against SQLite.

03:28 Said, no, I don't know of one of those, but what if I made Binita for Mongita?

03:33 (laughs)

03:34 And so I love the naming here.

03:36 Why would I put them all together?

03:37 So Binita is a local DB-like database ready to work with Beanie, right?

03:45 So basically all you have to do to work with this is import client from there, create a client, paste that, give it a directory just like you would SQLite, pull a database off of it, and then just call initialize Beanie like you normally would, but pass it this Beanita database.

04:02 So it just works off the file system.

04:03 So now you've got a local, mostly compatible MongoDB disk-based in-process thing that you can use for simple MongoDB examples.

04:13 It has some things it doesn't support, links which are following references across documents, aggregations, which is a whole crazy data analytics aspect MongoDB and unions and other things that Monquita doesn't support.

04:26 So I'm pretty sure let's just double check here.

04:29 Yeah, so it's basically just builds that wrapper on top of Monquita.

04:33 And the real challenge that was tricky here was Beanie is only async and Monquita is only sync.

04:39 So you've got to somehow put those together.

04:41 Anyway, thank you, Roman Wright, for doing this.

04:43 It looks really cool.

04:44 And we could come up with the joke for this, but I think that's Benitas.

04:49 >> Oh, Brian, that was solid.

04:54 >> That hurts.

04:57 >> Yeah.

04:59 >> Sorry. Not sorry.

05:02 I think that I'd like to talk to people about using good coding practices especially in research and science.

05:11 Ran across Patrick Minoltz.

05:16 It's an online book really, It's called the good research code handbook.

05:21 And he says it's for grad students, postdocs and PIs, which I had to look up was principal investigator, who didn't do a lot of programming as part of their research.

05:31 And I think it's also just kind of a good thing for really anybody that's coming into to coding from a different field and wants to jumpstart some good practices.

05:39 It's actually really kind of great.

05:42 He's got a little roadmap, which is neat.

05:44 I'll click on that roadmap.

05:46 little small here, but it goes through.

05:48 >> It's got a journey mapped out there. I love it.

05:51 >> Yeah, it's nice. But a visual of where you're going to go with this, but he talks about setting yourself up for success using Git, and virtual environments, and projects, and even packaging a cookie cutter, which is cool.

06:06 Talks about style guides, and keeping things clean, and removing dead code.

06:11 Some coding practices like separating concerns, and separating your peer functions from side effect functions, which I thought that some people don't really touch on right off the bat, but it's a good practice.

06:25 Even talks about testing, although the focus is really around unit testing and sort of some hand waves around indent testing, but still it's nice.

06:34 And then even gets into documentation and social aspects like doing pairing and peer reviews and getting involved in open source and community.

06:44 It's actually just really kind of a pretty solid book for people.

06:48 Sounds great.

06:49 Yeah, that's really great.

06:50 I think it's, yeah, go ahead, Gina.

06:52 No, I just wanted to say it sounds great.

06:55 Because I sometimes have people joining OctoPrint who are pretty much newcomers to the whole coding aspect, but are interested in writing a plugin or something like that.

07:04 So that might be a good resource for them to just direct them to.

07:08 I had, you know, I'm...

07:10 Go ahead, Brian.

07:11 I had somebody contact me once and try to, I wish I knew about this before because somebody contacted me and said, we don't really need coding training, but we kind of aren't used to working together.

07:22 Well, we're all individual people and now we're working as a group and we don't know how to do that.

07:27 And I don't know if that's here, but a lot of this around this is sort of good practices for working well with others.

07:34 Michael.

07:35 Yeah.

07:36 What I was going to say is a lot of people, especially in this research field, learn to code in a JIT fashion, like just enough learning of coding to get the problem solved and then they just, they have to keep moving on.

07:50 And it's easy to find yourself just stumbling into wrong patterns of, well, everything's into one huge file and it doesn't take like command line argument inputs, it just hard codes the values in.

08:01 So you can't reuse it.

08:02 It's not a module you can import or a function you can call and.

08:05 It does no error checking.

08:07 These are yes, exactly.

08:09 No, no testing.

08:10 all of that kind of stuff would be really valuable for people who are coming from that angle.

08:14 So definitely a good thing to check out.

08:15 And I love it.

08:16 Good research.dev.

08:17 What a sweet domain name.

08:19 There's also one of the neat things about the writing style is he's talking about, trying to just free yourself with, from time.

08:30 So, not free yourself from time, but, as a researcher, you're in a hurry.

08:35 And there's a lot of stuff to do.

08:36 And a lot of the stuff he's trying to get people to do is about offloading information so that you don't have to keep it in mind.

08:44 So he talks about short-term memory and long-term memory and how, you know, like unit tests and inline code comments are good to, you know, short-term reminders in the moment.

08:56 And then project documentation and test suites are about keeping that knowledge long-term.

09:02 And that's kind of, I never really thought about that before short term versus long term information. But it's kind of a neat angle as well. So anyway, yeah, that is clever. So where we got next, we have something that I brought along. And given that I come from a 3d printing background. Now, I thought I would start with something that can be applied to 3d printing. And that is Python library for building parametric 3d cut models. It is based on for those of you who have heard of the open source CAD solution called FreeCAD.

09:37 It's based on the same CAD kernel on the same on the same modeling kernel and allows you to programmatically design stuff. And that is pretty amazing if you want to do it parametric. So for example, imagine that you have some kind of case for an electronics project or something and You need screws from screws, screw holes at certain distances, but maybe you want to be able to scale it up or down or whatnot, or things like this. And for stuff like this, something like this is absolutely amazing. From the ground up, CatQuery is only a library, but it all also comes with, as it says here, it also comes with a Qt based GUI called CQ Editor, which I tried out and worked great. The only problem was it came with an older version of CatQuery that did not yet support some stuff that I wanted to play around with. So what I did instead was I tried the Jupyter extension and just threw that up on my NAS in a Docker container and that works absolutely wonderfully. It gives you in the browser, gives you a 3D model view of the stuff that you're currently modeling, automatically refresh whenever you change something. And the way things look is, this is the GUI, by the way, you simply create, you define a work plane, then you create a box on this, for example, then you say to the face, to the top face in z direction, define that as a work plane, put a hole through that with a diameter. So the whole API is pretty intuitive as well, also well documented on here. And if you scroll a bit through the examples, you can see that there is way more possible than just planes with holes, but things like, where was it?

11:27 Somewhere there was a, right, a flask, or the aforementioned parametric enclosure, a Lego brick.

11:37 So for all of that, there is example code there.

11:41 And yeah, it really surprised me how quick I got this up and running because I had a very specific use case that I wanted to try to run through that.

11:50 And that was inserts for one of these part case thingies where you can put inserts into solid screws and stuff.

11:59 And that worked flawlessly with that.

12:01 I had this finished in maybe half an hour or something like that.

12:04 So really great package, works nice, has everything that you expect from a basic cut solution and also works in a browser.

12:13 So if you want to go fully parametric and already know Python, then give this a look as well as free cut.

12:19 Very cool. Yeah, this looks super.

12:21 Yeah, I have a friend who is doing a lot of Python and CNC machines.

12:28 He has a guitar company and it gets these wood blocks and shaves out guitars and necks and things that I don't I'm not familiar with.

12:36 Would this be applicable to something like that as well?

12:38 I mean, just 3D printing, but if you got to define the the the polygons of it, basically, the objects that it can output are STL files, object files, I think step export is also available. So all the stuff that you find commonly in the CNC and 3D printing world. So I could just design something with this, export the STL, throw it in a slicer for my 3D printer, or alternatively throw a resulting step file or something into whatever workflow preparation I do for my CNC router, I guess. So I'm not that - I'm not that familiar with CNC, but--

13:14 - Yeah, yeah.

13:15 I think this huge CNC machine, I think it takes STL files, pretty sure.

13:18 - Well, then.

13:19 - I'll have to check, but that sounds like it could be really relevant.

13:22 Couple of pieces of feedback, Daniel Kras says, nice because he hates the GUI workflow of FreeCAD.

13:30 - Yeah, the funny thing is, CatQuery originally started out as a FreeCAD plug-in, actually, as far as I see.

13:36 So CatQuery 1 was FreeCAD, still built into FreeCAD.

13:40 CatQuery 2 is now standalone and just uses the kernel and all that.

13:43 So really nice.

13:45 For people who used to use OpenSCAD, that one is the next level and really nice.

13:50 - And then Brandon Branner, hey Brandon, says, "Can you import this into a slicer for 3D printing?" - Yeah, that was what I meant with, you can export the STL and then throw that into the slicer.

14:00 - Yeah, that's what I thought.

14:01 Perfect.

14:02 Awesome, this is a great find.

14:03 I love it.

14:04 And I also, I love that you come with your experience and your view of Python and what's super interesting, 'cause I would have never picked this, but it's very cool and a lot of people are into it, right?

14:14 It's just like, I'm like, oh, FastAPI, yeah, it's stuff like that.

14:17 - Yeah. - Yeah.

14:18 - Different point of view, I guess.

14:20 - I wanna do some 3D printing.

14:21 It'll be fun.

14:22 - It is, but be careful that you don't start a hobby project that then takes over your life.

14:26 That sometimes happens, I've heard.

14:28 - Well, so that's why I put that off because I know that I don't need another one, so.

14:36 Brian, I heard you have some awesome 3D Star Wars stuff.

14:40 And maybe you could like build in, like you could print some additional stuff to go with it.

14:44 - I could, yeah.

14:46 Or just buy it.

14:47 - Yeah, there you go.

14:48 Or you could just buy it.

14:49 (both laughing)

14:50 - I just actually bought a part.

14:52 So I needed a camera mount.

14:54 So I had a camera that, the one I'm using now, there's other applications where I wanna use that I didn't have a mount for a tripod mount.

15:03 And there was somebody on Etsy that it's obviously a 3D printed thing that you can buy.

15:09 Etsy is full of people making useful things with 3D printers.

15:13 I think that's neat.

15:14 Ten bucks to somebody else to do it for me, what a deal.

15:18 >> I'm happy to report that I'm currently looking into a camera that is sitting on a mount that I designed and printed myself.

15:25 >> That's neat.

15:27 >> Wow, that's awesome.

15:28 >> Yeah. That mounts to a VESA monitor arm and then has a little slide that I can actually slide up and down to adjust the height.

15:36 And I also have some of these angle brackets that with tripod mounts on the up and down side that I can use to adjust the tilt and stuff.

15:46 And yeah, so after I did this, that thing finally set where it is supposed to sit.

15:51 So win-win.

15:52 - Awesome.

15:54 That's awesome.

15:55 And just for people listening who are not watching, Gina has the best video setup of all three of us.

15:59 She looks great.

16:00 (Gina laughs)

16:01 It's a really good studio setup.

16:02 All right, and then parting thought on this one, Henry Schreiner, hey Henry, says 3D work from Python has always been tricky, so this looks great.

16:10 Right, you ready for a fun one?

16:12 - Yeah, let's switch to you.

16:14 - All right, I got something cool.

16:15 So previously I spoke about TextSniper.

16:19 This is a Mac app and it lives in the menu bar, like up by the clock, and you just run it and you can, on Mac, you can hit Command + Shift + 4 and select a region and screenshot it.

16:29 This one adds Command + Shift + 2 and if you select a region, it will capture the text out of whatever is behind it.

16:35 So for example, I was on a meeting on Zoom and somebody put something up and like, oh, I wanna have notes on that.

16:40 And I thought, oh, I could type it.

16:41 Wait a minute, Command + Shift + 2, swipe on Zoom.

16:43 Boom, I have notes of the whole page instantly, right?

16:46 This is a cool app.

16:47 It's like 11 bucks for the Mac.

16:48 This I sort of randomly mentioned that was thought it was cool.

16:51 This is not what I'm talking about.

16:52 What I wanna talk about is Brett, Brett Turnbull said, "Hey, I heard Michael talk about this tech sniper thing "I just described on Python Bytes.

17:00 "That's neat.

17:01 I could build this in Python.

17:02 So he did.

17:03 Nice.

17:04 Isn't that dope?

17:06 So if we jump over here to the, I love the name.

17:10 I got so many good names today.

17:12 I mean, I didn't get them.

17:13 I just gathered them up.

17:14 So we have text sniper for that thing that grabs the text.

17:17 He created the textinator and it's a simple macOS status bar, mini bar app that automatically detects text within screenshots.

17:26 So instead of, adding a new hotkey that does its own thing, you just do Command Shift 4, copy it, and then it hooks into macOS through PI OBJC, figures out a screenshot was taken, grabs it real quickly, uses the neural engines in macOS and the Apple Silicon and stuff to do the text processing and then drops out the answer.

17:47 And if you look at the whole app, the entire app end to end is 450 lines of code.

17:51 That's brilliant.

17:53 Yeah.

17:53 Very cool.

17:54 If you look at there's like, let's see, there's a lot in it.

17:57 So you can see it's, you know, from foundation, from cocoa, you know, import, NS notification center and all these different things is what it's, basically using to, listen for events of screenshots being taken and then feed those off into the various places that it needs to go.

18:14 So what do you think?

18:16 I think it's awesome.

18:17 And, Rhett's in the audience and he says, Oh yeah.

18:21 Hey Rhett.

18:21 Thanks.

18:22 Thanks for the shout out.

18:23 And then of course somebody says, just for Mac, power.

18:27 - I'm a little wheezy on that.

18:28 - I'm afraid.

18:29 Gina, what do you run for your OS?

18:33 - Actually Windows, yeah, to the big surprise of many, but I'm also a gamer, so I got tired of constantly dual booting, so yeah.

18:44 - That's definitely a challenge.

18:45 So there's actually a video showing you how it works, and it uses something I've built apps with as well called Rumps, ridiculously uncomplicated macOS and you've R apps or something like that.

18:58 And so it's really an interesting way where it's just kind of like clicking together a couple of cool things like the NS query for a spotlight with Pio BGC, the vision being recognized text request and like just building on top of macOS and Rumps, which is pretty cool.

19:15 You do have to give it special permissions 'cause it has to be able to like monitor your system globally.

19:20 So there's a few steps you gotta follow, But well done, Rhett.

19:22 I love the ambition here and it looks good.

19:25 >> But this is incredible.

19:26 What I love about this is it's just one thing, but it's a small application.

19:31 So I mean, potentially, just you could come up with all of these pieces, you could do other stuff too, and have just a small example to try to learn off of.

19:39 That's pretty neat.

19:40 >> Indeed. Let me just make a real quick suggestion out there.

19:45 Maybe, I'm not sure if this is, I don't see it over here, but if over on the releases side, On my rumps app, I did this and I put over in the releases, you can come over here and you can just download a .app zipped, but it just unzips as a .app file. So hit that with some pi install, no, pi2, use pi2 app on it and make it one cool further step there.

20:08 And maybe automate that with GitHub actions.

20:10 Yes, that's a very good point. Absolutely.

20:12 So you don't have to constantly keep doing it.

20:15 Yes, very good point. Push to a branch, have it do some magic.

20:19 - Or click on release and have it do some magic.

20:23 That's always wonderful to watch in the log.

20:28 - Yeah, you know what I find is that people that set that stuff up, they release apps like this more frequently.

20:33 They're like, oh, it's just one feature, but I don't mind pushing the button or I can just push to that branch.

20:37 And the more that you have to do it manually, make sure you don't mess up stuff and don't forget to bump the version.

20:42 And you just do it a lot less frequently.

20:44 And so there's some really cool knock-on effects of that recommendation.

20:47 I went through a lot of pain to automate testing and release tasks in OctoPrint, which also involved by the way, flashing physical hardware.

20:58 And so I built a lot of stuff so that all of that is more or less at least semi-automated because that takes so much work out of every single release.

21:07 That's absolutely insane.

21:09 So yeah, automation.

21:10 It's great.

21:11 Yeah, I do have to add.

21:13 So yeah, there's a couple, I don't maintain that much stuff, but I have to admit that I am reluctant to bump versions because then I'm like, I gotta, I gotta push it out to a, out to PI PI then.

21:25 And I haven't automated that through GitHub actions yet.

21:27 So yeah, I gotta get on that stick.

21:30 So okay.

21:31 I'd like to talk about, locks actually not using locks.

21:36 So this is an article called handling concurrency without locks from Haki Benita.

21:43 And it's, it's just sort of a nice, actually this is intimidated at first by reading like flipping and getting ready to read this article, but it was really well written.

21:55 So the idea is it starts with a Django application and it's a URL shortener, actually you were just talking about, but there's, so there's like an idea that you've just sort of walks through it.

22:07 Of the idea is you have something that creates a unique URL, but that uniqueness is checked because you keep a database of it.

22:17 And so you check to make sure it's unique and then you create it, the check it, make sure it's unique and then store it in the database.

22:24 But that check has to hit the database and you've got to read write thing and there's a concurrency problem.

22:29 I know there's lots of ways to get around it, but it's a reasonable use case to just think about.

22:35 But there's a lot of places within Django or within web actually, and within actually all applications where if you're using a database, there's this concurrency thing might be a problem.

22:46 And you can get around it with locks or other things, but the discussion kind of goes through basically broadening it up a bit and just talks about collision problems and locking problems in general.

22:58 And with some nice diagrams, I can't remember where they are on the page, but nice horse.

23:02 But it's a kind of a good slow walk.

23:08 Here's some diagrams.

23:09 And then, okay, going on, continuing on with the article, the idea is like, once you know it's unique, you're storing in the database, the database is already going to, if you set it up right, it's already gonna make sure that that key is unique.

23:25 So instead of making sure you can do it ahead of time and then doing it and making it, then checking for database errors, just assume that it's right and throw it into the database, and then watch for any problems.

23:36 So basically using the database as the uniqueness indicator.

23:42 He talks about how to do that and then how to structure code a little bit to deal with those issues and then recover from it gracefully if that happens.

23:54 Then the side effect is you're faster because you're not putting the concurrency locks within your application, you're depending on that uniqueness and that control of the database, which is already fine-tuned to allow multiple access and keep track of that, just utilize that.

24:19 So it's a neat reminder to everybody to utilize the tools under you, I guess.

24:25 - Yeah. - Good reminder.

24:27 - Yeah, do you do much database or Django stuff?

24:32 - Me, no.

24:33 I have never, I think I've never so far even touched Django and I try to avoid databases like the plague.

24:39 - Yeah.

24:41 - I'm a huge enthusiast for flat files.

24:44 Yeah, I don't know.

24:46 I can do that stuff. - They are quite valuable, right? - Yeah.

24:49 It's just, it takes so much complexity out of the system if you can somehow avoid having to run a database server or an SQLite file somewhere, or a Benita file.

25:00 - Yeah, yeah, I've been working on a project, which I'm not ready to talk about yet, but I've been thinking, well, could this just be a bunch of local JSON files?

25:10 I'll talk about that later, but yeah, could it just be, maybe, but if it's 20 records or something, maybe just a JSON file is all I need.

25:17 - I mean, sometimes there's no way around it, but.

25:20 - I've got a project with a very small database And it's in, it fits, it's a very tiny text file as we, as we started out.

25:29 But, but the problem always is as long as, as soon as you get more than one user, you have concurrency problems.

25:35 Even if the application isn't multi-threaded or multi-process you've, and, and we're running into that.

25:43 So switching to, it does feel weird to have to switch to a database just for that.

25:48 And you, I mean, you can get around it also.

25:50 you can put the, you can just say, Hey, if I don't already have a database, I can put the currency in, in the server so that, so that that is handled there.

26:00 You can do that, but.

26:01 Could be sure.

26:02 Yeah, for sure.

26:03 my example is a client side, like desktop type story.

26:07 So there's only going to be one user.

26:09 There might, there might be two people hitting the keyboard, but it'll be one in one app.

26:12 Cool.

26:13 But I also just really like thinking about, well, if you have a database and you have an ORM in what ways could you just leverage that?

26:20 And then instead of like you said, well, let's, you know, from threading import re-entry locks, the R lock, and then take those, and then you do pay the price for every successful time.

26:29 And if it's one in a thousand or more that you're gonna hit that problem, you know, pay that price and that little bit of complexity of catching like the integrity error and then reporting the error instead of reverse.

26:40 That's great.

26:41 - Yeah. All right.

26:42 What do we have next?

26:45 Yeah, next we have something that I have not yet had a chance to test out myself, sadly, but which I could have used roughly five or six years ago, really hard, but did not have access to then.

26:57 So I thought maybe I should make a mental note now and share it here.

27:00 So we'll remember the next time that something like Tatsu exists and Tatsu is a library with which sometimes English is hard.

27:10 you can generate parsers for Python, so Python-based parsers based on eBNFs. So I think the E was for extended and BNF is Bacchus-Navar form, so a very structured way to define a grammar of a language of a file format of whatnot. And textual, I should add, though maybe also binary, but that sounds like a lot of pain.

27:37 And that looks like something like, I'm looking for a good example.

27:44 Let me quickly, yeah, so for example, this is a grammar and then you turn that into a slightly annotated grammar.

27:57 And in the end you run this through Tatsu and then Tatsu spits out something which builds an AST out of your thing that you put into it that is supposed to be parsed.

28:11 And apparently Tatsu cannot only do that, like, so it can do on the one hand in this kind of way that we know from the RE module, from the regular expression module in Python, where you can compile a pattern and then reuse it during the duration of your code, but you can also generate Python code itself out of it so that you can then import it as a module and reuse it and reuse it, maybe also adjust it, but I don't know how well that works.

28:35 And apparently it is also compatible to Entler grammars, which is a name I haven't heard since university.

28:40 So that is nice, I guess, for people who have grammars lying around that are written in Entler, I think version four or something like that.

28:48 And yeah, so this is really one of these things that I wish I had found sooner when I had a desperate need to generate a parser from an EBNF, because writing an EBNF is sometimes way easier than writing a parser.

29:02 And yeah, now I have it, but now it's too late.

29:06 - For people who are listening, the eBNF, the grammar file, it looks a little bit like a YAML definition, a tiny bit, a little bit like a little bit of regX, but you basically specify the rules of the language and then you can take that and run with it, right?

29:22 - Yeah, in a way you basically say you have an expression and that expression consists of this other components and then these components are defined further down.

29:31 And so you dig deeper and deeper into the syntax and build up the, the, the definition from that, basically.

29:36 Like if you've ever seen one of these Jason grammar railroad diagrams, it also goes a bit like this, just in textual form.

29:45 Yeah.

29:46 Once I was teaching a class and it's, go ahead, Brian.

29:48 No, I just, I was getting PTSD from my language classes in college.

29:53 Yeah.

29:54 That's admittedly also where I learned about this stuff, but for some reason I, yeah, back then it was horrible.

30:00 Now I kind of like it. It's weird.

30:02 Yeah.

30:04 It just makes it so easy to well define something.

30:07 Yeah, and no pressure.

30:08 And now I see the advantages because it makes it so much easier to define structured text and stuff that you need to use in machine communication.

30:17 So if I had had an EBNF, or if I had had that like 10 years ago, when I started on OctoPrint, I would have written an EBNF for the, Yeah, basically for the for the G code stuff that runs between a printer and OctoPrint to communicate over the serial interface, and then it would have been way easier to pass that the way it is now I had to do all of this by hand.

30:38 Yeah, but maybe I can rewrite it with this.

30:43 We'll see.

30:43 Yeah, it seems useful if you wanted to create a simple, higher level scripting language that controls your app, right?

30:52 If, say, you are writing code for scientists in a lab, and those scientists we're not developers, but you could say like, okay, you can enter a couple of simple expressions and it'll like move the robot around or it'll like upload the results or, you know, something like that.

31:07 Right.

31:08 You could probably build that with this.

31:09 And that seems neat.

31:10 And then the other thing that's noteworthy here is this is 3.10 or above.

31:13 And I'm sure that's because it's the peg parsers match the effectively the switch statement that's in Python 3.10.

31:21 Right.

31:22 Also with regards to data science, I could also imagine this helps maybe with parsing weird data formats that you have in Textual that fall out of some legacy software that is running on some ancient sun cluster or something that is connected to your measuring devices.

31:39 So I could imagine that could also help there.

31:43 - Good idea.

31:43 Right, is that it for our items?

31:45 - I think it is.

31:46 Do you have anything extra?

31:47 - All right, of course.

31:49 Of course I've got some extras.

31:51 Let's see. - Me too.

31:52 - Let me pull up something real quick.

31:53 All right.

31:53 I know, fantastic.

31:55 All right, I'll throw mine out.

31:57 So not here, in a recent episode, we covered, where was it?

32:02 It was the-- - 285?

32:03 - PEP 690.

32:04 - Oh, okay.

32:05 - 285, thank you.

32:06 Yes, that's it.

32:07 So we talked about PEP 690 and lazy imports, and this was recommended by Itamar.

32:12 This comes out of the Cinder project and other things.

32:16 Since then, I've had Barry Warsaw, Armand Bravo, and Carl Meyer on DocPython to discuss this, which was fun.

32:23 And they've also written this up as a formal blog post.

32:26 So people were interested in that and they wanted to just go dive in.

32:29 I just want to point out that that's a pretty intense dependency graph right there.

32:33 It's just black cloud of dots all connected and stuff.

32:36 So there's some really interesting things from Instagram's use of this and whatnot.

32:41 Very neat, people can check that out.

32:42 There was an expected release of Python 3.11 beta three because of an incompatibility with pytest.

32:49 Brian, can you imagine?

32:51 - Yeah.

32:52 (laughing)

32:53 It was fun.

32:54 - Do you know about this?

32:55 - Yeah, so I do.

32:56 I found it amusing.

32:57 In one of the announcements, it said, "Apparently pytest is well used." (laughing)

33:04 - No, can't be.

33:06 No way.

33:08 - I think it was a joke, but made me laugh.

33:11 - Yeah, yeah.

33:12 But the fix was no joke.

33:13 So there's a quick fix that rolled out for people if they wanna check that out.

33:16 And then last thing, I kind of threw out a, >> A teaser for this episode.

33:21 I just did something completely silly and weird, and I think I might like it, I'm not sure.

33:26 For a long time, I've used DuckDuckGo.

33:29 I really think that DuckDuckGo is fantastic, and I may just keep using that.

33:34 But let me give a little credit here.

33:37 Daniel Herthome sent us a message and said, "This isn't really Python related, but if I know Michael, I bet he will love this ad-free privacy a respecting search engine called kaggy.com.

33:49 Have either of you heard of Kaggy?

33:51 I had not.

33:51 - Never.

33:53 - No, and so it's pretty interesting.

33:56 I went and I listened to, or read, I listened with my eyes, I listened to this interview, watched this interview, there's an audio version, so I keep stumbling on that, but Vladmir Hrlovac is the founder of Kaggy, was founded in just March, so it's really brand new.

34:14 Talks a lot about the motivation for creating the search engine.

34:16 And basically, he says, I think the ad tech, which I think maybe should be called more like surveillance capitalism or something like a little more serious, but ad tech, putting all these ads and tracking in front of my kids.

34:27 I don't want my kids to grow up thinking that just like being completely tracked and having zero privacy is like the way of the world.

34:33 So there's a pretty interesting conversation here on what they built.

34:36 But the idea is it's a search engine that is a premium search engine.

34:40 And so instead of having ads, the users have to pay for it.

34:42 So I'm gonna do an experiment for us, Brian.

34:44 I'm gonna live with this for a month and then I'll give you a report on what I think.

34:47 - I think that'd be great.

34:48 - Yeah.

34:49 - Yeah, so check this out for example, pep 690 lazy imports.

34:55 What happens if we put that up there?

34:57 So comes up with the peps page and check that out, number three.

35:01 And then probably somewhere a little farther down so it's not in the title is Python bytes, I would suspect.

35:06 But like these results are pretty interesting but check this out.

35:09 So I can come over here and I can hover over next to one of these and it says, Oh, this is on this page.

35:15 This one doesn't have that much information about it.

35:17 Some of them, let's see, maybe this one should have it.

35:20 So before you even visit it, it'll say there's four trackers.

35:23 It's this rank by traffic.

35:24 It'll use HTTPS, it's fast or not.

35:26 It lands in this category and you can upvote and downvote thing.

35:30 You can push things up and down.

35:32 - That is a power feature.

35:33 - And you can say like, for example, isn't that cool?

35:35 You can say W3 schools, I never ever want you to show up ever ever again.

35:37 - Exactly, that was my thought.

35:39 (laughing)

35:40 - Yes, it's so horrible.

35:42 they're so good at SEO and they're so bad at writing meaningful, helpful stuff.

35:45 So you could just, when I got my first W3C result, schools result, I just did block that.

35:50 And so you can, for example, go to like the docs on python.org and say, you know what, that one's higher for me because if I search for Python stuff, I'd kind of want to get the official discussion.

36:01 - Yeah.

36:01 - The other thing you can do is cool is you can have these lenses.

36:04 So you're saying I'm only interested in programming stuff right now.

36:06 So it takes you to like GitHub and stuff like that.

36:10 And then you say, you know what, just actually what I want is just more results.

36:14 Just basically search GitHub for all of this.

36:17 And the final thought here that's cool is it de-ranks.

36:21 So, you know, I showed you like different sites.

36:23 So here, let's put something embarrassing for a company that needs to be embarrassed, CNN.

36:27 So if you hover over this, how many block, there's 43 ad detectors block on this thing.

36:31 43, 43.

36:34 Here's what they do.

36:35 They are down ranking and down voting, like lowering the SEO rank of things that have tons of ad trackers and things like that.

36:45 So if you if you're overwhelmed like this, where it's orange or red, like that's that lowers your rank.

36:50 So it's like an anti ad tracker result surfacer.

36:53 Anyway, what do you all think?

36:56 I'm tempted, but I'll let you test it first.

36:59 Yeah, give it give me a month.

37:03 I didn't think I was going to like it that much.

37:04 It's it's pretty nice.

37:06 So I paid 10, it's $10 a month, which seems a little high.

37:09 I think they are getting their results by buying API access to Google and Bing and then resurfacing those results to you in this privacy respecting way.

37:21 - Actually, I kind of think it's brilliant.

37:23 And I want to try it too, because at first I was like, I don't want to pay for a search, but I know I'm--

37:29 - Yes, I know, I went through the process, Brian.

37:31 - Yeah, I tried to tell my kids, if you're not paying for it, then you're not the customer, you're the product.

37:38 So.

37:39 - Yes, absolutely, absolutely.

37:41 - And even if it wasn't for that, that the customization options are just like amazing.

37:46 Like, I don't know.

37:48 - Yeah, the fact that you can say, I prefer more of this site, less of that site.

37:51 Yeah, it is pretty neat.

37:53 So I really appreciate that Daniel sent this in and I'm gonna live with it for a month and I will let you know, but the maps doesn't work very well.

38:01 They don't work very well, but you can always just go to somewhere else.

38:04 A final thought, And the reason I switched to Vivaldi over Firefox is I didn't want to bring other things like ad block and you have ad blocking, tracker blocking and stuff there, but then you have to get extensions and then those extensions send their data back.

38:18 And then sometimes they'll like, well, these ads pay to be let through our ad blocker.

38:22 And there's just like still kind of a tracking level.

38:24 So with, with Vivaldi, you know, you get like total blocking without third party things.

38:31 So with this, you kind of get a search engine that is like a good, good match to that.

38:35 So anyway, I think it's, I think it's cool.

38:38 And anyway, I just wanted to give it a quick shout out here because it seems fun.

38:41 a funny thing.

38:42 I think that's it for everything.

38:44 Okay.

38:45 Funny thing I, in the chat, eco V or E K E V says I use a, use a custom Firefox Ox extension to block W3 schools.

38:56 That's interesting.

38:57 I didn't know that existed, but yes, I installed that as well.

39:01 And I, Ikabu, that is an absolutely worthwhile thing to do.

39:05 I totally agree.

39:06 So I'm with you.

39:07 I used to use that as well.

39:08 I also use something that like, it was called, I'm okay with cookies and it would automatically accept all cookie disclaimers.

39:15 You know, the cookie warnings.

39:16 It would just say, okay.

39:17 And there's almost none of them.

39:19 And then I just block them on my network.

39:20 So it doesn't matter if I accept them or not.

39:22 - Well, let's jump to Gina's extra stuff.

39:26 - It's only some small stuff.

39:28 One thing, can you share my, thank you.

39:32 So one thing is something that I mentioned before that I went through a lot of pain to automate a bunch of stuff about the testing and OctoPrint which involves some Raspberry Pi cluster here that I can flash via a Python tool and stuff like this.

39:47 And I also needed voltage monitoring for this Raspberry Pi cluster because I don't know how familiar you are with Raspberry Pis, but they can be a bit, let's say, selective about what power supplies they like and they start browning out easily and stuff.

40:05 So power monitoring is very helpful.

40:07 And there are these cheapish 30, 40 bucks USB power and voltage monitoring dongles that you can just plug between two USB cables and then it will measure everything.

40:20 And they also come with Bluetooth.

40:22 And I found a tool that can lock the data from this Bluetooth interface via Pi BlueZ, it's written in Python, from these UM24C, UM25C or UM34C dongles in order to allow you to track the voltage and power over a course of, yeah, whatever.

40:41 So I have to set up, lock this out, throw that into an InfluxDB, throw it up on a Grapana dashboard and get warnings when something underpowers and that is great for sleeping in peace at night about the test cluster situation.

40:55 And the other thing is something that reached me actually just last night in the shape of a tweet.

41:00 And that is that there is now another Node.js PyPI distribution that you can just depend your Python apps on that will then pull in for you a complete Node.js distribution with NPM, with Node and with NPX.

41:14 So if you have, like me, for example, if you happen to build a web server that also ships with a web UI built in, And then now you have a way to ensure that everything that you need in order to build that web UI is also there and can also maybe make this part of a build process that you build into your setup pie or something like this as well.

41:37 So that was a really nice thing to discover.

41:40 I still considered alpha, but I tried it briefly this morning and really great.

41:46 That's cool.

41:47 That's really cool.

41:48 That is fantastic.

41:49 Yeah, that's super interesting.

41:51 You could pair that with PyScript and have it do a little electron thing and then just run that, run Python, write Python for your little web app that runs on the front end.

42:01 - Yeah, or if you do something like, so for example, I still use Cypress JS for end-to-end testing in OctoPrint, which means that I now need a Node.js environment and a Python environment in order to have all developer dependencies together.

42:15 This would solve this problem because then I could just make this part, maybe I could even fire it up as part of the Pytest suit. I don't know, but it would be interesting certainly to test that out.

42:26 Definitely. Very cool.

42:27 Brian, it sounds like a Pytest fixture needs to be put in place.

42:30 If you want to do that, that would be great.

42:34 I'll put it on my to-do list right after 3D printing.

42:39 Ah, yeah. I know the problem.

42:41 Since we're running a little long, I'm going to skip my extras until next week.

42:50 So let's maybe jump to something funny.

42:52 Nathan Ashberger.

42:53 It's a tweet.

42:54 and it's like, you pretend you're getting a call and, and the, the phone says, Hey, I'm from somewhere.

43:01 we built the first AI driven sales platform and I wanted to talk to you about your sales needs.

43:06 he says, wait, are you an AI?

43:09 no.

43:10 Well, we can both appreciate the irony here, right?

43:13 So I don't know if this was real or not, but it just made me laugh.

43:19 That was funny.

43:20 So I love it.

43:21 Yeah.

43:22 Shouldn't they be having their AI call you with their AI platform?

43:25 It's subtle.

43:25 Yeah.

43:27 Okay.

43:27 So let's go to, maybe yours is funnier.

43:29 Let's go to yours, Michael.

43:30 my, well, mine is as a, it's a collection of jokes.

43:34 So maybe one of them will be this, this comes to us from Brian skin, who, as I said, has been on the show before.

43:40 Thank you, Brian.

43:41 And these are, you know how, like you've got the Mars rover badge and you've got the Arctic vault badges on GitHub.

43:48 These are ones that were considered and potentially rejected, okay?

43:51 - Okay.

43:52 - So let's see what's over here.

43:53 These are, they were considered.

43:55 I feel like David Letterman with his like ad stuff.

43:59 Okay, so one of them is the vital contributor, over 100 issue comments consisting of just plus ones or thumbs up emoji.

44:07 There's the Sith Lord who wiped out someone else's commit by force pushing to the main branch.

44:12 You could have that badge.

44:13 - I have that, I think.

44:14 - You have the procrastinator badge.

44:16 You could be the procrastinator, create a repository with a single init commit with just the readme.md and never touch it for five years.

44:26 My personal favorite is the secret Santa, accidentally commit a secret API key to a public repository.

44:32 That's good.

44:34 We have the monkey rich.

44:35 Yeah, the monkey rich makes a commit directly to the main branch that breaks the build.

44:41 This is fine.

44:42 It has a dog in a room that's full of fire.

44:44 we've all seen this meme in some form or other, I'm sure.

44:48 This is fine.

44:49 Over 1000 open issues on a public repository that you own.

44:52 - Ouch.

44:53 (both laughing)

44:56 - Fine, let's wrap it up with the last one.

44:58 Works on my machine.

44:59 The works on my machine certification suggests user error in at least 10 issue threads without attempting to reproduce in user runtime environments.

45:07 All right, these are pretty good.

45:08 - That would be hard to automatically detect though, right?

45:12 - It would, yeah.

45:13 - The last one.

45:14 - Rest maybe. - You gotta self-assign, I think. - Yeah, yeah, yeah.

45:17 - Yeah. - But yeah, I don't know if you saw that GitHub recently actually rolled out more achievements that you can now try to hunt.

45:23 So it's not anymore just the Arctic stuff, but there's now stuff like Merge Shark, and PR Shark, I think they called it.

45:31 And so there are several, they are hidden.

45:34 You have to figure out. - Oh, that's cool.

45:35 - Yeah, yeah, I think last Thursday or so, they rolled it out, which was also when this repo, I think popped up or at least popped up.

45:44 I also it also went past me on my Twitter feed and I immediately shared it around with people because it's good.

45:52 >> That was good.

45:52 >> It's kind of sad that some of them didn't make the cut.

45:55 >> I know. I think so for sure.

45:57 >> Yeah.

45:58 >> This is fine. People definitely need that this is fine.

46:00 >> Absolutely.

46:02 >> Yeah.

46:02 >> Hi Brian. Is that it for us?

46:04 >> I think it is. It was a fun episode.

46:07 Thank you, Michael. Thank you, Gina.

46:08 >> Absolutely.

46:08 >> Thank you.

46:09 >> Yeah. Bye everyone. Thanks for being here, Gina.

46:11 Thank you for having me. Bye!