WEBVTT 00:00:00.100 --> 00:00:05.200 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to 00:00:05.200 --> 00:00:12.420 your earbuds. This is episode 485, recorded on Tuesday, June 23rd. I'm Calvin Hendryx-Parker. 00:00:12.680 --> 00:00:13.500 And I'm Michael Kennedy. 00:00:13.900 --> 00:00:18.900 Thanks, Michael, for being here with me today. I'm excited to be your co-host for Python Bytes. 00:00:19.640 --> 00:00:26.340 We've got a lot going on. We're going to dive right in. But first, make sure you go check out 00:00:26.360 --> 00:00:29.200 our sponsors. It is sponsored by us in our work. 00:00:29.960 --> 00:00:32.360 Michael's great work over at the Talk Python courses. 00:00:32.500 --> 00:00:34.840 So check out Talk Python's coursework over there. 00:00:35.500 --> 00:00:40.040 And if you are in the Midwest tomorrow, since this does go live today, 00:00:41.260 --> 00:00:45.800 AWS Community Day Midwest is tomorrow, June 24th in downtown Indianapolis. 00:00:46.020 --> 00:00:49.440 Six Feet Up is sponsoring and there'll be two Sixies presenting. 00:00:49.740 --> 00:00:54.100 So you've got, that's actually what Six Feet Up people refer to themselves as. 00:00:54.520 --> 00:00:57.960 So if you want to come check out two of us presenting at the conference tomorrow, check that out. 00:00:58.460 --> 00:01:05.840 If you want to connect with the hosts, you can find Michael and myself at all the various social spots on Mastodon, Bluesky. 00:01:06.860 --> 00:01:07.740 Do you still use X? 00:01:08.040 --> 00:01:09.000 I do still use X. 00:01:09.010 --> 00:01:14.200 And I updated our show notes template so that we have our X links and LinkedIn. 00:01:14.940 --> 00:01:15.300 Perfect. 00:01:15.470 --> 00:01:17.240 Yeah, because I'm probably a little more active on LinkedIn. 00:01:17.870 --> 00:01:21.700 But maybe this is a nudge or a motivation that I should go be active over on the other platforms too. 00:01:22.260 --> 00:01:25.820 And you can check out the show on Mastodon and Bluesky at those various links. 00:01:26.040 --> 00:01:27.180 And X there as well. 00:01:27.590 --> 00:01:34.220 So join us on YouTube for the live version of the show at pythonbytes.fm/live to be part of the audience. 00:01:35.120 --> 00:01:41.860 Usually we record Tuesday at 7 a.m. Pacific, 10 a.m. Eastern for those of us on the East Coast. 00:01:41.910 --> 00:01:43.360 I guess we cover both the coasts, Michael. 00:01:44.060 --> 00:01:46.980 And then older video versions are available there as well. 00:01:47.400 --> 00:01:54.380 Finally, if you want a bonus digest of every week's show notes from an email form, just add your name 00:01:54.520 --> 00:01:59.320 and email to our friends of the show list. And we'll never share that email address with anyone 00:01:59.400 --> 00:02:04.340 else. You'll just get great sets of links and show notes from the team over here at Python Bytes. 00:02:04.760 --> 00:02:08.259 All right, Michael, let's kick it off. You've got the first topic up here. 00:02:08.740 --> 00:02:12.060 And let me tell you, I think this one is going to resonate with you. 00:02:12.740 --> 00:02:15.940 Yeah, I already saw why I peeked at some of the headlines. I was like, this is interesting. 00:02:16.740 --> 00:02:18.160 Backing up Docker volumes. 00:02:18.820 --> 00:02:21.360 So maybe we should do a quick, what the heck? 00:02:21.700 --> 00:02:22.300 What are Docker volumes? 00:02:23.160 --> 00:02:26.500 So if you do Docker a lot, you know exactly what Docker volumes are. 00:02:26.520 --> 00:02:31.140 But Docker's a little bit one of those things like, oh, it seems so complicated, so weird. 00:02:31.340 --> 00:02:33.200 And it's just really different than what I'm used to doing. 00:02:33.220 --> 00:02:34.100 And then you work on the Go. 00:02:34.440 --> 00:02:36.580 That's kind of like Linux, but just in a file. 00:02:36.980 --> 00:02:37.520 You know what I mean? 00:02:38.120 --> 00:02:41.860 But one of the things that is different about Docker is it's very transient. 00:02:42.360 --> 00:02:46.380 It's not quite Nix, that it can't be changed once it's fired up. 00:02:46.680 --> 00:02:50.300 but it's generally ephemeral, right? 00:02:50.390 --> 00:02:51.520 I start up a Docker container, 00:02:52.100 --> 00:02:54.520 my app or whatever app is running there might write to it, 00:02:54.530 --> 00:02:56.380 but that's right inside the Docker container. 00:02:56.910 --> 00:02:58.240 And then when it goes away, 00:02:58.900 --> 00:03:00.520 if I restart it, rebuild it, especially, 00:03:01.340 --> 00:03:03.300 then whatever it wrote is just gone. 00:03:03.600 --> 00:03:04.440 Sometimes you want that. 00:03:04.740 --> 00:03:06.920 But say if you want your web server logs to stick around, 00:03:07.000 --> 00:03:07.480 you don't want it. 00:03:08.020 --> 00:03:10.500 If you want your database data to stick around, 00:03:10.620 --> 00:03:11.380 you really don't want it. 00:03:11.700 --> 00:03:15.800 So there's a lot of times that you want a permanent storage, 00:03:16.300 --> 00:03:19.920 but an ephemeral operated system and code and so on for Docker. 00:03:20.560 --> 00:03:22.180 So what you can create is a Docker volume, 00:03:22.800 --> 00:03:27.260 which is a managed persistent virtual hard drive type thing 00:03:28.100 --> 00:03:31.040 associated with that particular Docker container. 00:03:31.440 --> 00:03:33.440 And you can move around theoretically afterwards, right? 00:03:33.640 --> 00:03:36.000 But typically you say, okay, my Docker setup, 00:03:36.300 --> 00:03:39.660 this database server gets a volume for its data, something like that. 00:03:39.820 --> 00:03:41.580 Yeah, that's one of my favorite features of Docker 00:03:41.760 --> 00:03:43.520 is the immutability of the containers. 00:03:43.980 --> 00:03:46.780 And if people respect that, things go a lot smoother generally. 00:03:47.020 --> 00:03:47.640 Yes, exactly. 00:03:47.790 --> 00:03:52.660 Like, well, what I typically do, Calvin, when my Docker doesn't work, I just log into the container. 00:03:53.480 --> 00:03:56.240 And then I like get pulled a new version and just like read. 00:03:56.450 --> 00:03:56.940 No, just kidding. 00:03:57.280 --> 00:04:02.680 This is this is the modern equivalent of change mod 777 across all your files. 00:04:03.140 --> 00:04:03.500 Exactly. 00:04:03.610 --> 00:04:04.100 It's not working. 00:04:04.150 --> 00:04:04.880 That always fixes it. 00:04:05.200 --> 00:04:05.740 Just fix it. 00:04:06.020 --> 00:04:06.920 I just open all the ports. 00:04:07.010 --> 00:04:07.860 It says it can't get to it. 00:04:08.200 --> 00:04:08.580 What's the problem? 00:04:08.760 --> 00:04:08.960 Passwords. 00:04:09.320 --> 00:04:09.500 Exactly. 00:04:10.380 --> 00:04:13.280 So what do you do with those volumes, Michael? 00:04:14.680 --> 00:04:16.299 So you've got to back that data up. 00:04:16.530 --> 00:04:18.440 I mean, unless you just don't like yourself. 00:04:18.820 --> 00:04:21.260 You've got to back up your data because if something goes wrong, 00:04:21.940 --> 00:04:24.000 from a whole host of reasons, things can go wrong. 00:04:24.440 --> 00:04:25.640 You need a backup of your data. 00:04:25.800 --> 00:04:27.640 That's just living in servers, right? 00:04:27.960 --> 00:04:28.100 Yeah. 00:04:28.100 --> 00:04:28.740 Or just computers. 00:04:29.640 --> 00:04:32.880 So a lot of times what you can do is you can run some kind of command, 00:04:33.160 --> 00:04:34.320 like I use MongoDB. 00:04:34.580 --> 00:04:35.980 A lot of these people, I also use Postgres. 00:04:36.210 --> 00:04:39.179 For either of those, there's a command you can run that says backup this 00:04:39.200 --> 00:04:45.520 and like back it up to the host or somewhere like that and then send it off to a encrypted storage 00:04:45.540 --> 00:04:52.240 or something. But if it's just the hard drive, the volume that you want to back up specifically, 00:04:52.780 --> 00:04:59.800 that's way more challenging. So Brian Weber, thanks Brian, sent over this idea called this Rust 00:05:00.380 --> 00:05:06.360 utility called Docker Volume Backup. And the idea is you can back up Docker volumes to in the, 00:05:07.060 --> 00:05:10.020 all the ways I can imagine I would possibly want to back them up. 00:05:10.630 --> 00:05:11.800 So you can back them up locally. 00:05:12.200 --> 00:05:13.940 You can back them up to an S3 bucket. 00:05:14.150 --> 00:05:18.500 And when you say S3, that means not just AWS that you can learn about tomorrow, 00:05:19.700 --> 00:05:25.120 but many of the cloud storage providers have adopted some kind of S3 compatible API. 00:05:25.740 --> 00:05:29.080 So you could back it up to, I don't know, Digitalization or something if you want as well. 00:05:29.640 --> 00:05:33.780 WebDev, Azure Blob Storage, which you probably could just treat as S3 directly. 00:05:34.580 --> 00:05:37.340 Dropbox, Google Drive, some SSH compatible storage, 00:05:37.500 --> 00:05:39.060 I'm guessing SCP or something like that. 00:05:39.400 --> 00:05:40.240 All those things. 00:05:40.700 --> 00:05:41.980 And how sweet is that? 00:05:42.400 --> 00:05:46.900 So the idea is when you've got a Docker Compose type of setup, 00:05:47.420 --> 00:05:50.720 you can go and add a separate service, right? 00:05:50.880 --> 00:05:51.720 In your Compose. 00:05:52.090 --> 00:05:54.300 And it just says the image is this project, 00:05:54.800 --> 00:05:55.840 this Docker volume backup, 00:05:56.320 --> 00:05:57.900 and you give it environment files 00:05:58.400 --> 00:05:59.540 and then it can just back up. 00:05:59.900 --> 00:06:00.880 That's really easy. 00:06:01.200 --> 00:06:03.380 Really, if it's easy to do, you will do it. 00:06:03.620 --> 00:06:04.480 Yes, it's nice, right? 00:06:04.820 --> 00:06:07.960 So you basically just add it as a separate service. 00:06:08.540 --> 00:06:11.740 And then in your ENV, you give it a schedule and a destination. 00:06:12.420 --> 00:06:14.000 And then you're on the ENV settings, right? 00:06:14.120 --> 00:06:14.740 The backup settings. 00:06:15.080 --> 00:06:16.200 You just give it those kinds of things. 00:06:16.400 --> 00:06:18.640 And if you've got a home lab and you've got a NAS on there, 00:06:18.860 --> 00:06:20.600 most NASs have S3 compatible storage. 00:06:20.840 --> 00:06:25.280 So you can keep it all, I'm kind of touting the local first operation again, 00:06:25.780 --> 00:06:26.900 keeping all your data, your data. 00:06:27.440 --> 00:06:28.100 Yeah, absolutely. 00:06:28.320 --> 00:06:31.060 So this thing is super small, written in Go, 25 megs. 00:06:31.500 --> 00:06:38.120 So there was also a Docker volume backup Docker shell based image, and this is 1/20th the size. 00:06:38.280 --> 00:06:38.840 So that's cool. 00:06:39.190 --> 00:06:45.040 let's see, you can set up a cron job in your composed setup, or you can just CLI run it. 00:06:45.350 --> 00:06:47.620 It, this is, this is part of the stuff that I thought was really cool. 00:06:47.620 --> 00:06:49.080 It rotates away your old backups. 00:06:49.170 --> 00:06:50.580 You don't have to worry about that kind of stuff. 00:06:50.580 --> 00:06:55.160 You say like, just keep me, give me a backup every day or for a month, you know, or, or 00:06:55.380 --> 00:06:55.940 something like that. 00:06:55.940 --> 00:06:56.840 And then you move it on. 00:06:57.400 --> 00:07:03.300 local GPT encryption, like at rest encryption of your backups, which I think is super important 00:07:03.700 --> 00:07:06.240 because the last thing you want to do is like, well, I want to make sure I back it up. 00:07:06.280 --> 00:07:08.520 So it doesn't, if things get hacked, it doesn't go wrong. 00:07:08.560 --> 00:07:10.740 And then that actually being the thing that gets stole. 00:07:11.120 --> 00:07:13.840 Get notification on finished infill build, filled runs. 00:07:14.520 --> 00:07:20.000 you can have it stop the container before it does the snapshot and start it back up. 00:07:20.520 --> 00:07:24.559 So if you're doing like, I don't know, a SQL light sort of thing as your storage 00:07:25.140 --> 00:07:26.540 and you don't need that much uptime, 00:07:26.880 --> 00:07:30.700 you want to make sure it's not in some like half-committed journaling state 00:07:30.900 --> 00:07:31.520 or whatever, you know. 00:07:32.040 --> 00:07:34.540 You can run custom commands during the backup lifecycle. 00:07:34.540 --> 00:07:35.220 That was my next question. 00:07:35.340 --> 00:07:36.380 Can you run a custom command? 00:07:36.660 --> 00:07:39.700 And here's the one that maybe go start going down this list. 00:07:40.880 --> 00:07:44.340 It has Docker Swarm support plus ARM64 and ARMv7, 00:07:44.680 --> 00:07:46.860 which means it runs on Raspberry Pi Home Labs. 00:07:47.280 --> 00:07:47.860 Oh, that's nice. 00:07:49.360 --> 00:07:50.740 It's hitting all the check marks. 00:07:51.440 --> 00:07:52.720 Yeah, it's pretty calm, right? 00:07:52.860 --> 00:07:53.220 It's all right. 00:07:54.100 --> 00:07:54.700 That's very cool. 00:07:54.920 --> 00:07:59.000 This also gives me a chance to shout out Idlembyte's search. 00:07:59.560 --> 00:08:02.160 So people come over and search for like Docker backup. 00:08:02.700 --> 00:08:08.900 And Brian pointed out like, oh, if you search for volume until today, it came back with no results. 00:08:08.940 --> 00:08:09.760 And he thought it was broken. 00:08:09.980 --> 00:08:12.600 But actually, he said, it's so fast, I thought it didn't work. 00:08:12.660 --> 00:08:13.920 It felt kind of like a rough thing. 00:08:14.340 --> 00:08:18.320 But I actually put a ton of energy into this custom search engine that I built for this. 00:08:19.680 --> 00:08:22.320 If you're interested in what we cover some of them, please check out our search engine. 00:08:22.460 --> 00:08:23.460 And you can even add it. 00:08:24.780 --> 00:08:26.260 Now, this is like next level right here. 00:08:26.270 --> 00:08:28.440 You can add it as a custom search engine to your browser. 00:08:28.550 --> 00:08:30.600 So you just say PB space and search our site. 00:08:31.190 --> 00:08:33.780 So you can have Python bites all the things. 00:08:34.340 --> 00:08:35.680 Actually, I was looking for that this morning. 00:08:35.900 --> 00:08:36.820 Because I was like, did we cover this? 00:08:36.930 --> 00:08:38.159 Have we talked about this in the past? 00:08:38.460 --> 00:08:39.880 So that's super cool. 00:08:40.140 --> 00:08:45.140 Now, that's awesome that they went and looked at your site to see if this thing existed 00:08:45.270 --> 00:08:48.440 and didn't find it and scratched that itch and made that thing happen. 00:08:49.240 --> 00:08:49.960 So very cool. 00:08:50.700 --> 00:08:52.680 Well, speaking of making things happen. 00:08:52.840 --> 00:08:56.140 Before we move on, I just want to give a shout out to Chris out there. 00:08:56.820 --> 00:09:01.120 It says, backing up a volume is surprisingly hard when you think about application state, 00:09:01.420 --> 00:09:02.540 stateful data, restore points. 00:09:03.060 --> 00:09:03.240 Yes. 00:09:03.640 --> 00:09:04.500 So absolutely. 00:09:04.860 --> 00:09:06.660 And this is a really nice option there. 00:09:06.740 --> 00:09:07.820 Yeah, that is very true. 00:09:08.040 --> 00:09:10.600 Like I said, if you make a thing easy to do, it will get done. 00:09:10.980 --> 00:09:15.120 If it is hard to do, it will go into that queue of things still to do. 00:09:16.160 --> 00:09:16.520 All right. 00:09:16.760 --> 00:09:19.920 Well, thanks for that tip on backing up my Docker volumes 00:09:20.470 --> 00:09:21.400 into S3 compatible storage. 00:09:22.280 --> 00:09:24.980 But the next one up here actually is kind of exciting 00:09:25.480 --> 00:09:29.040 because it's not just a release of Pyodide 314, 00:09:29.530 --> 00:09:32.520 which you'll notice a new numbering scheme there going on. 00:09:32.780 --> 00:09:36.700 The real news here is actually the fact that PEP783 00:09:37.220 --> 00:09:38.240 has made this possible. 00:09:39.020 --> 00:09:42.280 The Pyodide maintainers were basically hand building 00:09:43.060 --> 00:09:45.999 300 plus packages for your browser 00:09:46.020 --> 00:09:47.060 to use Python in the browser. 00:09:47.120 --> 00:09:48.780 So this is all about using Python in the browser. 00:09:49.600 --> 00:09:51.800 If you've ever thought about why do I have to use JavaScript 00:09:52.020 --> 00:09:54.920 to make fun web apps, you can actually use Python 00:09:55.160 --> 00:09:56.660 to do that with tools like Piodide. 00:09:57.320 --> 00:10:00.140 And now that the 783 has actually been accepted, 00:10:00.260 --> 00:10:03.180 you can use cibuildwheel and put wheels, 00:10:03.540 --> 00:10:06.560 like compiled binary packages onto PyPy 00:10:06.560 --> 00:10:08.520 so that if you write an app for the browser, 00:10:08.760 --> 00:10:11.040 someone can quickly download and get the latest versions 00:10:11.800 --> 00:10:13.440 of the dependencies you may be including 00:10:13.700 --> 00:10:14.780 in that application itself. 00:10:15.400 --> 00:10:18.120 So again, I don't know, they may be seeing hopefully 00:10:18.440 --> 00:10:22.120 a surge of browser-based Pyodide packages 00:10:22.540 --> 00:10:26.340 onto PyPI or PyPI coming up here in the future, 00:10:26.700 --> 00:10:29.860 but that's actually really cool to see that ease of use now. 00:10:30.180 --> 00:10:31.880 And so there's a whole tool chain available 00:10:32.540 --> 00:10:34.360 to build these packages and make them available. 00:10:34.680 --> 00:10:36.280 A couple other changes that have happened 00:10:36.340 --> 00:10:39.700 in this 314 release, you'll notice the numbering change. 00:10:39.980 --> 00:10:44.060 There's a version jump from 0.29 to 314. 00:10:44.400 --> 00:10:49.060 that's intentional. Basically, you're going to have the Pyodide version track the Python version 00:10:49.170 --> 00:10:55.880 that it closely matches with. So if you're looking for the 3.12 version of Pyodide, that's the older 00:10:56.160 --> 00:11:02.000 0.29. But moving forward, you'll quickly spot and know which version of Python you're getting when 00:11:02.000 --> 00:11:07.059 you install and use Pyodide in your projects based on the version number of Pyodide itself. So you'll 00:11:07.080 --> 00:11:10.520 get 314 tracks to 3.14. 00:11:11.700 --> 00:11:12.720 So for binary compatibility, 00:11:13.320 --> 00:11:14.660 it's locked within the Python cycle, 00:11:15.040 --> 00:11:17.200 meaning the packages you build today won't break 00:11:17.380 --> 00:11:18.640 with the next Pyodide release. 00:11:18.940 --> 00:11:21.500 So another big benefit to this piece here. 00:11:21.740 --> 00:11:24.300 They've also brought back in some packages 00:11:24.660 --> 00:11:27.080 that were taken out of the standard library 00:11:27.700 --> 00:11:29.420 for Pyodide to make the packaging smaller. 00:11:30.100 --> 00:11:33.300 Things like SQLite3, SSL, and LZMA 00:11:33.740 --> 00:11:35.840 are back in the default standard library, 00:11:36.460 --> 00:11:38.640 which is nice, so you don't have to basically install 00:11:38.960 --> 00:11:40.780 something like SQLite if you're using that 00:11:41.020 --> 00:11:42.520 in your browser-based application. 00:11:43.840 --> 00:11:46.300 There's some bug fixes and other things in there. 00:11:46.440 --> 00:11:49.960 There's also experimental TCP socket support in Node.js, 00:11:50.340 --> 00:11:53.280 and you can now connect Pyodide to a real database. 00:11:54.030 --> 00:11:55.480 So that's kind of a wordy, 00:11:55.880 --> 00:11:57.220 if you've got Python running in the browser, 00:11:57.340 --> 00:11:59.780 but you can't connect to a database or to sockets, 00:12:00.140 --> 00:12:01.780 you were kind of limited in what you could do 00:12:01.900 --> 00:12:03.120 with local storage, for example, 00:12:03.540 --> 00:12:07.620 but now that opens up a wider spectrum of things you can do. 00:12:07.870 --> 00:12:11.120 It looks like they've tested it on MySQL, Postgres, and Redis 00:12:11.710 --> 00:12:12.620 when running on server side. 00:12:13.080 --> 00:12:16.480 So it definitely blurs the lines between Python in the browser 00:12:16.780 --> 00:12:19.180 and Python runtime anywhere WASM runs, 00:12:19.480 --> 00:12:21.080 which I think is, again, super cool. 00:12:21.270 --> 00:12:22.000 So I don't know, what do you think? 00:12:22.060 --> 00:12:24.840 Are you building some web-based Python apps, Michael? 00:12:25.100 --> 00:12:29.340 I would love a Vue.js-like equivalent 00:12:29.940 --> 00:12:33.200 or something along those lines built on top of PyScript, 00:12:33.460 --> 00:12:36.460 which can run Pyodide or even MicroPython, 00:12:36.630 --> 00:12:37.700 which I think would be super neat. 00:12:37.940 --> 00:12:41.140 I built some examples actually when I built a PWA 00:12:41.740 --> 00:12:43.440 based on PyScript when it first came out 00:12:43.490 --> 00:12:44.740 and that was based on Pyodide. 00:12:45.180 --> 00:12:47.840 And it worked really, really well, but I have not. 00:12:48.120 --> 00:12:49.180 I've not been doing it lately. 00:12:49.420 --> 00:12:51.060 You guys see a lot more variations 00:12:51.440 --> 00:12:52.740 with all your projects that you work on. 00:12:52.860 --> 00:12:53.120 How about you? 00:12:53.610 --> 00:12:55.260 I mean, we've got some interesting stuff going on 00:12:55.480 --> 00:12:56.400 in the appliance world, 00:12:56.920 --> 00:12:58.760 but this might actually open the door to, 00:12:59.560 --> 00:13:01.459 again, more of those at-edge applications 00:13:01.460 --> 00:13:04.260 where connectivity may be limited to, 00:13:05.040 --> 00:13:06.800 I just see a lot of special use cases 00:13:06.980 --> 00:13:09.360 that open up when you can run Python in the browser. 00:13:09.520 --> 00:13:10.820 I've always wanted Python in the browser. 00:13:11.310 --> 00:13:13.360 And when PyScript came out, what was that back? 00:13:13.560 --> 00:13:14.680 Oh, it's been a few years back now. 00:13:15.080 --> 00:13:16.860 That was exciting and kind of big news, 00:13:17.040 --> 00:13:19.220 but obviously it was slow in the early days. 00:13:19.900 --> 00:13:21.740 There was limitations, but this, 00:13:22.110 --> 00:13:23.020 I think this is a big move 00:13:23.060 --> 00:13:24.180 that's going to open up a lot of options 00:13:24.390 --> 00:13:26.780 for folks to build either embedded applications 00:13:27.480 --> 00:13:29.020 or things that can kind of stand on their own 00:13:29.400 --> 00:13:30.480 and run at the edge. 00:13:31.100 --> 00:13:36.640 i'm excited i think it's going to offer a lot of interesting uh moves there yeah uh i'm very 00:13:36.780 --> 00:13:40.760 excited about um about wasm i think it's going to be i think it's going to be cool i think it's 00:13:40.760 --> 00:13:45.900 going to be super cool uh now let's continue on you've been talking a lot about tools uh lately 00:13:46.040 --> 00:13:51.500 you've got another one here i never stop we don't stop talking about the tools so i won't talk about 00:13:51.920 --> 00:14:00.079 won't stop exactly i want to talk about the nbcli so this is a command line interface or automation 00:14:00.100 --> 00:14:05.660 people but the primary reason for bullying is ai agents and notebooks specifically the envy 00:14:05.790 --> 00:14:12.740 obviously is for notebooks so this is a jupyter ai contrib project and i'm sure you've noticed 00:14:12.770 --> 00:14:19.260 this calvin but so many projects now have some form of cli as kind of the final integration 00:14:19.700 --> 00:14:24.700 pointer the ultimate integration should be the ultimate integration point yeah i mean the 00:14:24.960 --> 00:14:29.319 motivation allows like well we our agents can't get to our data and it doesn't know the authenticated 00:14:29.340 --> 00:14:34.020 API is too complicated. So what can we just do to make it simple? But it opens up so much more, 00:14:34.450 --> 00:14:41.240 right, to just have a CLI for whatever you're working on. And so, so here's one for Jupyter 00:14:41.380 --> 00:14:46.120 notebooks. And if you do Marimo, like hang tight, I got something there as well. But the idea here 00:14:46.560 --> 00:14:54.700 is that instead of having the agents try to parse the IPYNB files, which are just gnarly JSON, 00:14:54.960 --> 00:15:00.980 especially if like lots of output is in there, you know, it's like there's your actual code and your 00:15:01.220 --> 00:15:06.240 cell definitions, but then there's maybe 10,000 lines of output to the next cell. That's not a 00:15:06.350 --> 00:15:11.720 great place for agents to be working. Right. Right. Context windows are a precious space. 00:15:12.760 --> 00:15:18.040 Exactly. And you're, you're messing with that. And also it's hard for it to actually kind of see 00:15:18.480 --> 00:15:23.940 the ambient variables and stuff like that that are in the kernel. So the idea here is with this CLI, 00:15:23.960 --> 00:15:26.700 you get more direct access to those types of things. 00:15:27.240 --> 00:15:29.320 So let me over here. 00:15:30.140 --> 00:15:31.220 First of all, experimental. 00:15:31.560 --> 00:15:32.440 So heads up. 00:15:32.620 --> 00:15:35.260 It is Rust-based, so it's likely you can see VC funding. 00:15:35.440 --> 00:15:36.180 That's super good. 00:15:36.780 --> 00:15:38.800 But the idea is that it can read, write, execute, 00:15:39.100 --> 00:15:41.680 and search Jupyter Notebooks and just skip that whole thing. 00:15:41.780 --> 00:15:45.540 It doesn't necessarily need to parse the IPYMB files, 00:15:45.760 --> 00:15:47.700 and it even works without a Jupyter server. 00:15:48.280 --> 00:15:51.400 So it can just talk directly. 00:15:51.720 --> 00:15:52.720 So it works with files directly, 00:15:52.780 --> 00:15:57.120 And it talks to the kernels using zero MQ, which is super cool. 00:15:57.480 --> 00:16:03.660 And instead of using that token heavy JSON or just markdown, it uses at at cell and at 00:16:03.760 --> 00:16:06.420 at output sentinels with inline metadata. 00:16:06.700 --> 00:16:13.440 So as I said, less wasted context and you compose these things and it even ships with 00:16:13.440 --> 00:16:14.140 an agent skill. 00:16:14.330 --> 00:16:17.440 This is another thing I'm starting to see a lot of and have been working on. 00:16:17.730 --> 00:16:20.380 I should have prepared an announcement on this that I'm not really talking about, but I got 00:16:20.440 --> 00:16:20.840 some stuff over. 00:16:21.500 --> 00:16:22.280 Some skills. 00:16:22.540 --> 00:16:23.320 I got skills. 00:16:23.820 --> 00:16:31.900 So if you want to install this, you can just npx skills install jupyter-a-dash-contrib-nb-cli 00:16:32.480 --> 00:16:33.480 to install it with Claude Code. 00:16:33.800 --> 00:16:38.060 Yeah, I think that's smart because MCP is not the answer to everything. 00:16:38.460 --> 00:16:43.500 I think there's a lot of value in skills and CLI combined together because, again, preserving 00:16:43.640 --> 00:16:48.519 that context window in a tool like this opens up a lot for folks who are doing data science 00:16:48.520 --> 00:16:50.920 and they don't know how to write or want to write an MCP, 00:16:51.040 --> 00:16:53.860 but they're definitely a whole in on AI and agentic and LLMs. 00:16:54.220 --> 00:16:54.400 Yeah. 00:16:55.740 --> 00:16:59.340 And MCPs basically say, hey, AI, here are some tools you can use, 00:16:59.430 --> 00:17:03.000 but they don't set the context of, here's why you need to use the tools. 00:17:03.260 --> 00:17:04.920 Here's the ways in which they work together. 00:17:05.100 --> 00:17:07.640 You know, like you need an agent or a skill. 00:17:07.959 --> 00:17:10.520 You need a skill to kind of set the stage, even if it's an MCP, 00:17:10.730 --> 00:17:13.420 like, hey, you can use all this stuff in the MCP, by the way. 00:17:13.699 --> 00:17:14.920 Makes me think I should improve my skills. 00:17:15.170 --> 00:17:15.280 Okay. 00:17:15.900 --> 00:17:17.980 So I said I got skills, but we'll see. 00:17:18.380 --> 00:17:20.120 there's an article about it by 00:17:21.089 --> 00:17:22.680 so you check that out 00:17:23.199 --> 00:17:24.900 and if you're a Marimo person 00:17:25.360 --> 00:17:26.620 I am going to have 00:17:27.160 --> 00:17:28.940 Trevor on from the Marimo team 00:17:29.250 --> 00:17:31.060 to talk about Marimo pair 00:17:31.560 --> 00:17:32.500 which is a similar 00:17:32.630 --> 00:17:34.600 not the same idea but a pretty similar idea 00:17:34.820 --> 00:17:36.920 of basically how you get the agents 00:17:37.440 --> 00:17:38.840 inside the kernel instead 00:17:39.630 --> 00:17:40.880 of just having them work 00:17:40.920 --> 00:17:42.300 with the files and so on 00:17:42.410 --> 00:17:44.880 so look forward to that 00:17:45.020 --> 00:17:45.740 over to you 00:17:45.740 --> 00:17:46.259 I like that 00:17:46.340 --> 00:17:50.740 I like that. The Marimo pair of aspects. I've become a Marimo, like I mentioned in the last 00:17:50.870 --> 00:17:56.320 episode of Marimo fan. Okay. So pairing well with something like that last tool you just mentioned 00:17:56.500 --> 00:18:02.940 is that a lot of agents and a lot of these agentic workflows are lacking the ability to recall memory 00:18:03.240 --> 00:18:09.020 in an efficient way. So this next piece here is actually a tool you can add into your agentic 00:18:09.300 --> 00:18:17.260 workflow to do AI memory. So, and more than just memory, memory that learns. So thinking about this 00:18:17.440 --> 00:18:21.320 memory problem is actually kind of hard. AIs forget everything between sessions, obviously. 00:18:21.820 --> 00:18:27.360 So you're constantly writing project specs, implementation detail plans. You know, those are 00:18:27.360 --> 00:18:32.700 good practices, but wouldn't it be nice if it could remember like last Tuesday when you use this tool, 00:18:32.840 --> 00:18:36.840 this kind of output happened. And maybe in this context, it's actually better to use this other 00:18:36.860 --> 00:18:44.040 tool, but that kind of, that temporal, like using time and associating facts and details together 00:18:44.800 --> 00:18:49.560 isn't something that just a markdown memory.md file is going to help you out with. So there's 00:18:49.560 --> 00:18:54.520 an open source tool called Hindsight from the folks at Vectorize, and they offer a cloud version 00:18:54.520 --> 00:18:58.220 of this, but you can run it locally. You can install it with Python. It's got, I think, 00:18:58.460 --> 00:19:04.960 adapters for all the major like Python, JavaScript, Node, et cetera. But basically what they're saying 00:19:05.000 --> 00:19:06.980 is simple vector search is not enough. 00:19:07.090 --> 00:19:08.960 Even if you had a ChromaDB locally 00:19:09.600 --> 00:19:11.400 with your documents all loaded into it, 00:19:11.770 --> 00:19:13.300 that isn't enough to solve the problem 00:19:13.740 --> 00:19:16.360 about what did you do last spring, for example, 00:19:16.600 --> 00:19:18.040 like the example they give here. 00:19:18.430 --> 00:19:19.720 That requires temporal reasoning 00:19:19.730 --> 00:19:21.280 and not just semantic similarity. 00:19:21.780 --> 00:19:24.100 So hindsight attempts to add in 00:19:25.440 --> 00:19:28.580 this extra layer of temporal retrieval, 00:19:28.630 --> 00:19:31.260 which runs semantic keyword graph 00:19:31.290 --> 00:19:33.179 and temporal search in parallel 00:19:33.180 --> 00:19:36.340 to store, retrieve, and reason over those memories. 00:19:36.550 --> 00:19:39.600 So you actually get kind of three main functions. 00:19:39.710 --> 00:19:41.040 There's three, it's a simple API. 00:19:41.600 --> 00:19:44.040 There's just retain, recall, and reflect. 00:19:44.370 --> 00:19:46.580 And then it uses this temper technique 00:19:46.770 --> 00:19:49.960 to do the search across all in parallel. 00:19:50.420 --> 00:19:51.720 And then it brings it all together 00:19:51.770 --> 00:19:54.420 and it automatically consolidates the related facts 00:19:55.050 --> 00:19:58.720 into durable observations instead of piling up duplicates. 00:19:58.890 --> 00:20:01.520 So again, preserving that context window, 00:20:01.820 --> 00:20:03.340 putting in what is most valuable, 00:20:03.720 --> 00:20:06.480 when it means the most to what you're doing, 00:20:06.940 --> 00:20:08.860 it's probably the next step in memory. 00:20:08.870 --> 00:20:12.480 I think this is, again, a next front that has to be conquered 00:20:12.820 --> 00:20:15.900 is giving these agents better recall and memory. 00:20:16.280 --> 00:20:17.860 I think a lot of people got a little taste of this 00:20:17.960 --> 00:20:20.000 with OpenClaw, but OpenClaw's default memory, 00:20:20.110 --> 00:20:22.640