#496: A lake house in Seattle

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 496, recorded Tuesday,

00:09 September 15th.

00:09 I'm Michael Kennedy.

00:10 And I'm Calvin Hendryx-Parker.

00:12 This episode is brought to you by Logfire from Pydantic.

00:15 If you want observability for your apps and your AI agents,

00:19 Logfire is the business.

00:21 I will be telling you more about them later.

00:23 Find the link at the top of the show notes.

00:25 Follow us on the socials.

00:27 All the various things you can think of are there on the episode page as well.

00:32 And sign up for the newsletter.

00:33 I just sent out the most recent one a couple days ago.

00:37 It was a little bit late.

00:38 Sorry, folks.

00:38 But really cool stuff that we add, like extra information that doesn't even appear in the show that helps you get a little more out of the show.

00:45 Yep.

00:46 I love all the context it adds.

00:47 I do, too.

00:48 I do, too.

00:48 I'm like, well, that's pretty good.

00:50 We found some good stuff here.

00:51 I would say that this newsletter that we're writing here, it shouldn't go extinct.

00:57 But some things might need to go extinct.

01:01 What's going on here?

01:02 So I found, so this is a blog post from, what's Eddie's last name?

01:10 Hold on, it's down here at the bottom of his copyright.

01:11 Eddie Atkinson.

01:13 He gave a talk at the most recent latency conference.

01:18 So it's actually a talk from last year, but I think he kind of brought it back a little,

01:21 evergreened it into a blog post last week about pandas that should go extinct.

01:26 and we're not talking about the cute little fluffy things that are used for international diplomacy,

01:32 but the Python data frame library.

01:36 Michael, how many times have you thought you had big data,

01:39 only to find out you were ready to defunstrate your laptop because pandas was the problem?

01:43 You know what? It's happened.

01:45 It's happened.

01:45 I'm going to get my dictionary real quick, and then I'm going to know that that happened.

01:50 I think the issue is a lot of folks really don't have truly big data problems.

01:54 I mean, we've done some big data projects in the past, which were 10,000 tables,

01:59 petabytes of data.

02:00 That's truly big data.

02:01 Most folks probably lie in the medium-sized data,

02:05 but Pandas definitely tops out.

02:06 I mean, he does some interesting benchmarks in here, gives a couple good code examples,

02:11 actually shows a really interesting post from Amazon Redshift team,

02:16 where they were looking at the composition of many of the tables that are out there in the Redshift environment.

02:21 If anybody's going to have a good view on what the size of data is and what big data could be,

02:26 they're probably the ones to look at that.

02:28 But if you look at this chart, they basically say on a continuum of data size,

02:33 most folks start over here in Excel.

02:36 You've got like under a gigabyte, around a gigabyte of data.

02:39 About that point in time, Excel falls over.

02:41 It's probably time to pick up another tool to handle that.

02:44 And a lot of people reach for pandas because I think there's just a lot of

02:47 built up inertia or momentum in the community around the pandas and data frames and it's an easy

02:53 ui it's been taught in a lot of universities so there's just not a lot of like need to kind of

02:58 move out of that space because there's a lot of good code examples a lot of blog posts have been

03:01 produced a lot of data science is based on pandas but they're really based on data frames and there's

03:06 more than one library out there to handle data frames and probably do it more efficient so if

03:11 you actually looked at the chart here they're basically saying when you get up into like the 10

03:15 10 gigabyte range for data sizes.

03:19 Pandas is probably still pretty good, but then there's a gap.

03:21 It falls off somewhere between 10 and 100 gigabytes of data.

03:25 And 100 gigabytes of data these days is not unfathomable.

03:28 You can easily go find sample data sets that are in that realm, in that range, in that size.

03:33 And so the next thing they reach for is typically a commercial tool like Databricks,

03:37 Snowflake, Dask, or some of these other things

03:40 that are like Spark.

03:41 So you're distributing the memory of that data set across many machines

03:45 or maybe even across one very large machine, but doing it in a distributed manner.

03:50 Most people probably don't need to go that far.

03:52 Like most people probably are still sitting in the range

03:54 where you can see on this chart that Polars handles.

03:56 Polars can handle straight up into 100 gigabytes of data

04:01 easily on a single machine.

04:03 And then DuckDB takes it to the next step, which actually kind of fitting for this episode.

04:08 I think this will be an interesting episode because there's a lot of information here

04:11 about DuckDB later on in the show.

04:13 But he kind of goes on again and shows that basically

04:17 the average size of a row in Redshift is about a kilobyte.

04:20 Every Redshift cluster has like 10 machines in it.

04:22 They're capable of guzzling 8 gigabytes per second

04:25 from S3, but really in actuality,

04:28 almost 95% of the tables in Redshift contain fewer than 100 gigabytes of data.

04:34 Most people are still in the range of just using a single machine with DuckDB

04:38 or even just Polars, which is probably similar,

04:40 simpler to maintain and manage, but it's above the reach of pandas.

04:44 That's why the post is kind of going on about pandas needing to go extinct.

04:47 Another interesting bit, again,

04:49 kind of good code examples in here.

04:51 When we get down into some of the tables for the performance,

04:56 what strikes you here,

04:57 Michael, on their memory usage?

04:59 The pandas library, we're talking about 30, this is one of

05:02 those examples, I can't remember which one, but four minutes basically for the duration of the

05:06 processing, 38 gigabytes of RAM.

05:08 If you get into pullers, that gets halved, 18 gig of RAM.

05:11 And if you go into DuckDB to do the same operation,

05:14 five seconds at 1.93 gigabytes of RAM.

05:18 So even the-

05:20 It's like nine to 20 times as much, yeah.

05:24 Yeah, I mean, an iPhone could do this operation

05:27 against 100 gigabytes of data.

05:29 It's fine though, 'cause you can just get more memory,

05:31 memory's cheap these days.

05:32 Memory's cheap.

05:33 Totally cheap, totally, totally cheap.

05:35 So I just think folks need to put this one to bed.

05:39 Pandas was probably a good way to start, but you'll notice in here,

05:42 like the Polars notation or syntax, really, really similar.

05:46 I like the DuckDB syntax.

05:49 I think he's got some examples in here where he reads in and does some more operations.

05:53 He does a couple against some larger machines, then falls back into an older laptop,

05:57 like a framework 13, to show that this is still useful as a developer tool.

06:03 So I think here, Pollard versus DuckDB kind of comes down

06:06 to your workload, your experience, and your preference.

06:09 It's a good post.

06:09 I really liked all the code snippets he links over

06:13 into the GitHub where you can actually try it yourself.

06:16 It goes against the New York City taxi data set.

06:19 It's got a ton of data in there, so it's fun to play with.

06:22 And you can see that we want to wait minutes or do you want to wait seconds?

06:25 And would you want to use all your memory for this?

06:28 And actually, another thing that Pollard

06:30 and DuckDB did much better was utilizing the CPU.

06:33 PurePandas in this case was using like a multi-core machine,

06:37 146% CPU, where if you go to PureDuckDB,

06:40 there were over 800% CPU.

06:42 So obviously eight to 10 cores are being fully utilized as opposed to

06:47 basically one and one and a half cores.

06:49 Yeah, that's awesome.

06:50 I feel like this is a pretty data heavy episode for the data science crew out there. It wasn't on purpose, but we ended up that

06:57 way.

06:57 I have a little bit of a real-time follow-up for you, Calvin,

07:00 for people who...

07:02 Yeah, I was gonna ask the one last bit in here.

07:04 He does mention Apache Arrow, and if you've not played with it,

07:07 it allowed him to switch back and forth between pandas,

07:09 Polars, and DuckDB without having to reload or copy the data into RAM.

07:14 So you could actually do the same operation with each of the libraries

07:17 without actually having to take the data back out of RAM.

07:19 So check out Arrow, which is really cool.

07:23 It's kind of a little bonus side bit that was in the blog post.

07:27 So data folks who got medium-sized data, This is going to be a godsend for you.

07:33 Yeah.

07:33 I think Arrow is the foundation of pandas too, if I remember correctly,

07:36 and also a polar.

07:37 So that's pretty sweet.

07:39 Yeah.

07:39 My real-time follow-up here.

07:40 It was a real-time follow-up here.

07:42 Oh.

07:43 If you were working with one of these and you want to switch to the other,

07:47 I had Marco Garale on, really on Talk Python a while ago to talk about narwhals.

07:53 And narwhals is a facade adaptive layer that speaks native polars,

07:58 but also talks pandas.

08:01 So if you want to try like, oh, let's see what we're doing.

08:04 You know, you could use this as a intermediate layer to kind of swap that out a little more

08:07 easily than rewrite and everything.

08:09 Yeah.

08:09 I think people just need to drop pandas.

08:11 I mean, it was great.

08:13 It was great 10 years ago.

08:15 Yeah.

08:17 Yeah.

08:18 I have some funny jokes, but let's carry it.

08:20 Let's move on.

08:22 Let's move on to Pydantic Pint.

08:24 So Pydantic Pint is really interesting.

08:27 Do you know Pint?

08:28 Are you familiar with Pint?

08:29 actually i've never i've never used pint so pint we've covered that on the show back in the day

08:34 and pint is interesting because if you're i mean all you got to do is say mars lander sample return

08:40 whatever and it's like the 100 million dollar plus fail because somebody used feet and somebody used

08:46 neighbors or something like that right and so pint lets you do math in python with units attached

08:52 which is pretty cool right so i can say instead of just having a distance i have 42 i have 42

08:58 kilometers and you can say like two miles to whatever and so on so it it basically mean forces

09:04 you to work in units right a lot of times we don't do this as regular programmers but if you do

09:10 anything scientific well there you go right so that's the background on pint but what i want to

09:15 talk about is actually not pint it's called pidantic pint because pidantic is an awesome

09:21 library that lets you validate the inputs and parse them and everything whenever you read some sort of

09:27 JSON, right? Like if it's FastAPI or just a JSON file or whatever,

09:31 you need database, SQL model, all those things. Yeah.

09:35 So Pydantic Pint takes this idea and adds units to your data validation

09:41 libraries.

09:42 So instead of saying I have a box that has a length and a width,

09:46 I could say I have a box that has a length and a width that is a Pint quantity and the validation is to convert it to

09:53 meters.

09:53 So even if you parse something that says feet or centimeters or whatever,

09:57 it will show up correctly.

09:59 What do you think?

10:00 That's definitely handy.

10:01 Yeah.

10:02 It's the kind of thing that's like, you don't have to,

10:04 you're not going to use it a lot unless you're really in,

10:06 you know, some kind of engineering or something.

10:07 Oh my God, this is so good.

10:09 It's like so perfect.

10:10 This has got to solve so many, like you said, small mistakes that end up in huge damage.

10:16 A hundred percent. So yeah.

10:17 And to be able to validate with it too.

10:19 Yeah. Just automatically, right. Just all the Pydantic validations have a Pydantic base model

10:23 And it parses over to whatever it is.

10:25 And if you put, I don't know, liters into the length,

10:29 well, liters, you can't convert liters to meters.

10:32 So I don't know.

10:34 And it kind of fits perfectly under the Pydantic scope

10:37 of the data validation and serialization.

10:40 Like it just, there's natural,

10:42 like this should exist and they made it exist.

10:45 Yeah, it's really cool.

10:45 So you can have say a FastAPI endpoint that just automatically just takes units

10:49 and automatically converts units.

10:50 And yeah, it's a really nice one there.

10:52 Speaking of really nice, now this transition here has nothing to do with the sponsorship,

11:00 the previous one.

11:01 They just happen to do great stuff in open source code too.

11:04 But Pydantic also happens to create Logfire, which I told you about at the beginning.

11:09 So let me go ahead and tell you about our sponsorship offer,

11:13 Logfire, not Pydantic Pint, which is not even from them, but it's based on Pydantic.

11:17 So here's the deal.

11:18 It's 2 a.m.

11:19 Your AI agent failed.

11:21 Was it the model?

11:22 A tool call?

11:22 the database, just the general unreliability of,

11:26 hey, I think a new model is coming. So the current one starts breaking periodically.

11:29 So most observability tools, they can't tell you because

11:33 they only see part of your stack.

11:34 PyDentaglogfire sees all of it.

11:36 One trace across your agents, LMs, APIs, and databases down to the infrastructure,

11:40 services, Kubernetes, hosts.

11:41 It's built on open telemetry with SDKs for Python,

11:45 TypeScript, and Rust.

11:47 And it works with any OTEL compatible language.

11:50 Every prompt, token count, and cost right next to your vector searches and API calls.

11:54 You query everything with Postgres compatible SQL to understand what your app is doing,

12:00 and so can your coding agent.

12:02 It can use the same way because it talks SQL really well.

12:05 If you connect it to the MCP server,

12:07 your agent can also figure out what is going on. So stop guessing,

12:10 read the trace, Pydenic, Logfire.

12:11 AI, it is still just engineering, even if it's weird engineering

12:16 these days.

12:16 So visit pythonbytes.fm/logfire today and sign up.

12:20 Get 10 million records free every month,

12:22 no credit card required.

12:23 You can even click, and I really like this, there's a little

12:26 copy of this text, onboard you with your agent.

12:30 Click that and it gives you a prompt you can drop in to Claude Code or Codex or whatever and it automatically knows what to do to set up Logfire

12:36 in your app. So thank you to Pydantic for supporting the show.

12:39 Calvin, I know you're a big fan of the visibility into the token.

12:43 Yeah.

12:43 I was just curious now if copy the setup prompt is the new

12:47 pipe to bash, like pipe some curl to bash.

12:51 Yes.

12:51 This is the replacing that. I think it is.

12:54 That's why I was getting it. Yeah. No, no. I think it is. And it's amazing.

12:57 I have some stuff that I'm working on. I'm like, oh, this is like,

13:01 this idea is perfect.

13:02 I love it so much.

13:03 So yeah, pretty cool. Thanks to Pydantic for sponsoring the show.

13:06 Thank you.

13:07 And let's jump over to your topic next. You've got in the wrong order in the other. So what's next?

13:13 Well, speaking of Pydantic,

13:16 this comes from Bob Builderboss, a friend of the show. I know you've had him on numerous

13:22 times for other events and things, but this one is about how to run,

13:27 how Rust code becomes something you can import.

13:30 I think it's interesting that we can, if people are complaining about performance,

13:35 the first news article I had about getting rid of pandas and bringing back in with polars and

13:41 DuckTV was about performance.

13:43 This is similarly veined.

13:45 Like if I've got a very computationally intense data structure or function that's happening in my program,

13:53 it'd be sure be nice if I could maybe replace it out with a Rust version of that, but have it act natively inside of my Python code.

14:00 So this blog post from Bob goes over basically what Pydantic V2 does, which is a data validation

14:06 library that most Python apps are using these days.

14:09 It's actually a Rust extension under the covers that does the work.

14:12 Its core, Pydantic core, is all built with Py03,

14:16 the same tool chain we're going to use here in this example.

14:18 So it goes over some examples of basically you write a normal Rust

14:23 module, you annotate it with some specific macros,

14:26 and then you'll actually be able to import that

14:28 into your Python code fairly naturally.

14:32 I think there's basically the Rust parser is incredibly fast.

14:37 Python never touches anything until the boundary crossing.

14:41 So if you call for some data, that all happens over in Rust.

14:44 The thing that I think the article covers that's really important is that

14:48 if you pull back that data across the boundary,

14:51 those Rust results get turned into Python objects.

14:55 And so you're going to want to think carefully about how you bring back parts of that, because

14:59 maybe you're only interested in a small piece of what is coming back and you don't need to

15:03 populate.

15:04 Say, for example, 100,000 JSON values means that you're going to get 100,000 Python dictionary

15:09 objects after the parsing is all done and everything gets passed back.

15:12 You don't incur the penalty until you cross that threshold back into Python land.

15:16 Maybe you don't need all 100,000, but there's some other operation you can do to get down

15:21 to just the pieces you need.

15:22 So what's nice is errors cross a boundary too.

15:26 So if Rust runs into errors, those come back as Python exceptions.

15:30 So it makes it easy to debug and figure out what's going on.

15:33 So you get clean Python semantics while still leveraging Rust.

15:38 So basically for anyone porting Rust, if you're returning a scalar,

15:41 don't sweat it.

15:42 If you're returning a big structure, profile the boundary.

15:44 And if that's a real cost, you want to switch over and maybe do more of the algorithm on the Rust side.

15:51 So much like you can use C or other languages in,

15:57 I don't know if you can, can you use Ruby to do this kind of thing?

15:59 I don't know if you can or not, but you definitely use Rust.

16:03 And I'm kind of excited about that.

16:04 I know you've been doing some coursework on it.

16:06 And it sounds like Bob has also made some learning materials to lead folks through.

16:10 It just feels like there's a really nice friendship between the Rust communities and the Python communities

16:16 and all the niceties that have been put in place

16:19 to allow us to use Rust almost natively over in the Python world.

16:23 So thanks, Bob, for the awesome post about that.

16:26 He's, again, code examples in here,

16:28 kind of explains to the Python folks who have never touched Rust

16:32 what the function signatures look like, which I appreciate because breaking it down

16:36 and telling me what each of those pieces means that I can now pretty easily read some Rust code

16:40 and understand what's going on because it doesn't look terribly foreign to me,

16:44 but it's just different enough, but this goes over a good usage of what each of those pieces mean for you.

16:50 Yeah, it's pretty surprisingly similar to Python, honestly.

16:52 Yeah, yeah, and incredibly fast,

16:55 but you get to think about things a little differently

16:56 because of the way it manages memory, and I think that's the big differentiator.

17:01 What was that cartoon with the guys?

17:03 Like, I'd gladly pay you on Tuesday.

17:05 It was Popeye.

17:07 Popeye, that was Popeye, right?

17:08 Yeah, it was.

17:08 I mean, I was thinking borrow checkers.

17:10 It was Wimpy, who will glad you pay you Tuesday for a hamburger today.

17:15 Yeah, that's the difference of Rust is you've got the borrow checker.

17:20 Always checking.

17:20 Yep, yep.

17:21 Always checking.

17:22 So yeah, I don't know.

17:23 You've been doing a little more with Rust and Python and teaching some folks these things.

17:28 Yeah, yeah, a little bit, a little bit.

17:30 I have two follow-ups here.

17:33 So you talked about DuckDB, but the question is, do you have a lake house?

17:39 Do you have a lake?

17:40 I mean, a lake house.

17:41 So we've heard of data lakes, which is a place you just kind of dump a ton of like an insane.

17:47 This is like back to your big data thing, DuckDB thing.

17:50 You just dump a bunch of data into this data lake and you figure it out.

17:53 Well, that's grown up a little bit.

17:55 And now there's this thing called DuckDB, but it's an implementation,

18:01 an example of what's called an open lake format.

18:04 Who knew?

18:04 Do you know?

18:05 I didn't know that.

18:07 We've done lake house implementations.

18:08 I didn't know there was an open lake format now.

18:10 So the story is, what if we could use S3 to scale our data access?

18:17 S3 scale is pretty large.

18:19 If you can read stuff off the file system instead of out of memory,

18:21 you can scale that tremendously large.

18:23 In the open lake story is, well, if you put file formats in S3

18:29 that everything could read, like maybe JSON files that tell you what the files mean, you could read them first.

18:35 Here's where the data lives in each piece.

18:36 And then parquet files.

18:38 Yeah.

18:38 Or zipped CSV.

18:40 don't know take your pick right it could be whatever so i just had the folks from duck lake on which is

18:45 a duck db implementation story of uh this open lake format well then i get this message here saying

18:54 yep guess what aws i'm sure you know this is a i did i did see this one come by yeah yeah and

18:59 in duck labs just offered basically the h1 is bad let me read let me read the first sentence today

19:06 we are announcing that Amazon has signed an agreement to acquire Duck Labs,

19:11 the Amsterdam based company behind the open source analytical database DuckDB.

19:16 I'll put a link to the announcement and I thought, well,

19:19 hmm, what does this even mean?

19:21 And I didn't, I wasn't entirely sure.

19:22 So I, I like, I went and did some looking here. I'm like,

19:26 there's, there's actually a lot of pieces in play.

19:29 So let me lay it out and I'll tell you what part AWS acquired,

19:32 what part didn't. Okay. So first of Well,

19:34 thanks to Dylan McConnell who sent this in.

19:36 What does this mean for the DuckDB ecosystem?

19:38 So first of all, DuckDB, which you gave a shout out before,

19:41 twice really, is the open source in-pros analytical SQL engine,

19:47 MIT licensed.

19:48 It's like SQLite, but for columnar data,

19:51 which if you're And way more, and way, way more.

19:53 Yeah.

19:54 Anything you pointed at becomes SQL queryable. It's

19:56 amazing.

19:57 Yeah, yeah, yeah. So you can say pointed at a pandas data frame and then

20:00 do SQL queries against your Panda. Like there's a bunch of plugins.

20:03 It's far beyond just a database, but it's an in-process sort of data processing engine,

20:10 much like that.

20:11 So the IP of this is not owned by any company.

20:14 It's held in a DuckDB foundation.

20:17 Oh, good. Thank goodness.

20:18 That sounds good.

20:19 That is good.

20:19 It was spun out of CWI Amsterdam from the folks who mentioned that article.

20:25 Good, but we'll come back to it.

20:27 Then there's Duck Labs.

20:28 And the story is Amazon AWS has acquired Duck Labs.

20:33 This is the company formerly branded DuckDB Labs, founded over five years ago by Hannes

20:39 Mullison and Mark Roosevelt to give DuckDB a stable home,

20:44 bootstrapped, grew to 30 people.

20:46 Now they can go chill on their island, which congrats to them.

20:49 That's awesome.

20:50 Because DuckDB really has taken over, right?

20:52 Then there's DuckLake, which I mentioned earlier.

20:56 It's one of three projects by DuckLab.

20:59 And there's DuckDB, there's DuckLake.

21:01 And then there's an API for working with this called Quack.

21:05 Check out the Talk Python episode.

21:07 But it's an open lake, lake house format.

21:10 All the waterfowl puns are great.

21:13 It is.

21:14 And we actually on the podcast had a fun conversation about like,

21:17 do you need a more serious name?

21:18 Like RPC for your data lake.

21:21 Like, no, they're like, we're calling it Quack.

21:23 Come on now.

21:23 And then also we have MotherDuck, which I think I thought Mother Duck was online version of DuckDB.

21:31 But no, that is a separate Seattle company selling serverless hosted DuckDB.

21:36 And originally it was started in partnership with DuckDB Labs

21:39 and they worked closely with Hannes and Mark for years

21:42 and even contracted Duck Labs for some of the engineering.

21:45 So now, what does this all mean?

21:48 So the foundation owning DuckDB is awesome, but it has three directors,

21:52 the people who own Duck Labs.

21:55 So there's a bit of a, how much independence is it really going to have?

22:00 There are some other folks, other governance and so on there.

22:05 But, you know, it's cool.

22:06 There's a foundation.

22:07 It's not super independent of DuckDB at the moment.

22:09 Maybe it will be, though, after this.

22:11 Mother Duck immediately moved into the business that Duck Labs vacated.

22:15 They now sell enterprise support for DuckDB and so on.

22:19 Everyone expects an AWS DuckDB service, probably a Duck Lake as well.

22:24 already used s3 right but maybe just a little more formal yeah there's s3 query and some adjacent

22:30 like technologies that sound like this may maybe this will augment or replace what's really nice

22:35 about ducklapse is it runs a local duckdb or a local postgres server and a lot of the chatty api

22:42 that would come from an open table format and the metadata now all happen in the database and then

22:46 it just fetches and reads the files yeah that's pretty cool yeah we can't beat physics if we can

22:51 keep the data, the bits where they're at physically and bring the compute to it.

22:56 That's the win.

22:56 Yeah.

22:57 So the duck lake angle actually is probably the most interesting one because AWS heavily

23:02 committed to iceberg through S3 tables, which is a competitor,

23:07 at least a competitor competing concept to a duck lake.

23:11 So yeah, check it out.

23:12 I think AWS just got better.

23:15 We'll see what that means for the rest of the world.

23:16 What do you think? I mean, you're on the inside of this a little bit.

23:19 Yeah, a little bit, but you know, they've,

23:21 we've had mixed reviews on their handling of open source,

23:23 but luckily they don't have any control over the open source other than they've just bought out the founders who are on the board of the open source foundation.

23:31 It's still separate enough of an entity.

23:33 I don't think there's a conflict here.

23:35 I mean, it's going to be good for the project.

23:36 The project is already incredible.

23:37 Like the DuckDB stuff is, it's just like,

23:39 if someone had thought about SQLite and said, I need to grow all these other features that handle all kinds of crazy data and do give me native like JSON access and functions.

23:49 And it's a really great platform for building cool little utilities or talking to giant chunks of data,

23:55 as we saw in the first article.

23:56 Yeah, yeah, yeah, absolutely.

23:57 And use barely any memory.

23:59 I mean, that's why this will run and work.

24:01 Yeah, the DuckDB part is really interesting on that.

24:03 And then the Duck Lake is like insane.

24:05 You know, you could have terabytes of Parquet files all broken in little bits.

24:09 No big deal.

24:10 No big deal.

24:11 NBD, NBD.

24:12 Yeah, I love it.

24:13 Well, how about some extras?

24:15 Well, I will continue on my beating the uv drum.

24:21 The latest release, every week we got something new, the latest release from the uv folks,

24:25 we get code signing on Mac and Windows.

24:29 So the binaries are now officially signed, code signed.

24:33 Again, I think this is all coming together,

24:37 ensuring we can secure the software supply chain part of this.

24:40 So I'm excited to see that release too.

24:42 I was hoping I wouldn't see a uv thing this week, but sure enough, it popped up in my feed.

24:47 And I was like, I have to mention it because they just keep making everything better and better and better.

24:52 So uv is now code signed.

24:55 So you can trust that it came from the right source on your own machine if you're on Mac and Linux.

25:01 Yeah, that's excellent.

25:02 Excellent, excellent.

25:03 Yeah.

25:03 I see it's still going.

25:04 Oh my gosh, the code signing is such a pain these days.

25:07 It used to be you could just build an EXE or a.app and you could just go here, try my app.

25:12 How dangerous is that?

25:15 I know.

25:15 That's how the world used to be, though.

25:17 Well, we used to have, what, R login with no password to remote machines?

25:22 What could go wrong?

25:23 It's fine.

25:24 It's fine.

25:24 Trust.

25:25 You've got to have a lot of trust.

25:27 Why would somebody do something mean to computers?

25:29 I don't know.

25:29 I don't know.

25:30 I remember in Windows 95, we had a bunch of them at a university I worked at.

25:35 Plugged straight in to Ethernet.

25:38 And the Ethernet, everyone got its own IP address.

25:41 And guess what?

25:41 That thing got taken over pretty quickly.

25:44 The university I was at, they all had public IP addresses in the labs too.

25:48 Yeah, it didn't go well.

25:49 No.

25:49 It did not go well.

25:51 Speaking of things that need to be patched and updated, check this out.

25:54 So there's two things that involve restarts.

25:56 But I have a MacBook Pro, M5 Pro.

25:59 Very nice.

26:00 Love it.

26:00 I got it this summer or earlier, maybe spring.

26:02 I don't know, whenever I got it.

26:03 And it came with a power supply.

26:05 One of those power bricks.

26:06 the power brick had to reboot the other day to to update itself i was sitting there working and i

26:12 saw this article come up come by say apple releases a firmware update for the 140 watt usbc power

26:19 adapter which is the one that runs the macbook and it's almost that that's almost actual size

26:23 right there it's so big yeah it's it's a beefy boy it is uh yeah i think this is even a little

26:29 small this big old picture of it it's it's it's heavy but i was i read this article and i was

26:33 sitting there working and my Mac, when it comes off of power,

26:37 it dims the monitor.

26:39 Yeah.

26:39 So I'm, I'm just, you know, and I was an hour or two later, I was sitting there and everything goes dim for a

26:43 second.

26:44 The little power just disconnects.

26:46 And then two, two, three,

26:48 five seconds later, something like that,

26:49 power comes back, brightness comes back. I'm like, I just,

26:53 my power break just rebooted.

26:55 What in the world is going on here?

26:56 That's crazy.

26:57 I've, I've, I've replaced all my Mac power books.

27:00 I've got, not that I'm trying to be an ad for Anchor

27:03 or anything like that, but the Anchor Prime has 160 watt, like itty bitty little power brick

27:09 that, because I travel quite a bit,

27:10 it has four USB-C ports on it and they can all deliver a combined sum of 160 watts.

27:18 So I can full bore charge my MacBook Pro M4 Max

27:22 and my iPad and my phone all at the same time.

27:25 That's beautiful.

27:26 And it's smaller than that brick.

27:28 Yeah, I'm also a fan of the Anchor stuff.

27:30 This one, since I had it anyway, I just plugged it into the wall part of the house.

27:33 And just if I'm in that part of the house, I just grabbed that cord.

27:37 But yeah, normally if I travel, I have an Anker that's actually a power brick, a little battery.

27:41 And it has two USB things and it'll do not quite as high as yours,

27:45 but pretty high.

27:46 And it's super nice because it's also a power brick, right?

27:49 So if I need to charge up, it'll even charge the MacBook,

27:53 but then just get it and just plug it into the wall.

27:54 And then it just becomes a power thing.

27:56 I'm also a fan of this Anker stuff.

27:58 Okay, a couple more extras really quick here.

28:00 we've got do not play yeah so the story of vs code the official documentary is out have you

28:07 watched this no i have not watched this it's an hour and 38 minutes and i'm here for it okay

28:12 all right it's got a lot of people that uh maybe you didn't see coming like Eric gamma for example

28:17 you know thinking back to the gang of four patterns and all that kind of stuff because

28:22 he was apparently involved in the early days um so yeah cool we had uh cult repo do the python

28:27 documentary we had them do the jet brains document or intellij documentary and here's the vs code one

28:33 i'm really loving these like high quality production i mean these are these are nice

28:38 little nice videos they yeah a lot of people behind it um i mean there's such there's an

28:43 audience for all these things i'm here for it too i love the fact that like the underdogs can feel

28:49 they are important for us and we can now hear more of the story about how some of these things came

28:54 about yeah it's really interesting i mean vs code has taken over so much and then yeah anyway and the

29:00 origins are way way more uh less ambitious let's say it's cool to check out so it also has over

29:06 half a million views so there is an audience for this apparently there's absolutely audience okay

29:09 think speaking of things that make you reboot yesterday last night yesterday mac os golden

29:14 gate ios golden gate watch os 27 golden gate all those things came out so uh-oh did you upgrade i

29:21 I did. Why wouldn't I mean, I'm like, oh, let's go.

29:23 I'm not yet upgraded on this.

29:25 I'm usually of that opinion, but lately I may wait a month or until a.1 or.1 to come out.

29:32 I spent one day working with it and so far it's okay.

29:35 Okay.

29:35 I'm going to upgrade then on your full recommendation.

29:38 Well, I've not upgraded my MacBook or my streaming computer.

29:41 I only recorded my main desktop.

29:44 So we'll see.

29:45 Ask me next week. Ask me how I feel about it.

29:47 You're right.

29:48 Honestly, the one thing to be a little careful about as developers is the Rosetta 2.

29:52 Yeah.

29:53 That's going away.

29:54 So your ability to run Intel compiled stuff, you might think, Michael, why would I run Intel

29:59 compiled stuff?

30:00 Like, ooh, Docker.

30:01 Certain Docker things only have Intel versions.

30:03 So that's going to be a mega hassle.

30:04 I mean, that's pretty rare.

30:06 People have cross-compiled most of this stuff.

30:07 Because when was the last time an Intel Mac was released?

30:10 Yeah, but let's suppose I'm deploying to an x86 server.

30:15 Yeah.

30:15 And I want to test something.

30:17 I think it's gotten a lot better.

30:19 It definitely has gotten better.

30:20 But it used to be, the thing is, I don't know.

30:22 It used to be certain stuff would only work in an x86 version of Linux.

30:26 Oh, I remember this.

30:27 Yeah.

30:27 But that was like three, four, five years ago when I was really dealing with that.

30:31 Actually, it was when I was dealing with like Databricks and trying to coordinate that stuff.

30:36 Nice.

30:36 So this one still has it, but the one after it, whatever that's called,

30:39 won't.

30:39 So this is like your last safe upgrade if you're worried about the other thing.

30:43 So did Apple actually deliver some AI features this time?

30:46 Well, I'll tell you what.

30:47 the new Siri caught me off guard.

30:48 I'm like, oh yeah, I did.

30:50 I did actually upgrade the phone.

30:51 And I guess it has the new Siri because it sounded,

30:53 I asked it something like set a timer and it said something completely different

30:56 that I'm used to.

30:57 And it sounded better.

30:58 I'm like, oh wait, I haven't had a chance to test it though.

31:01 All right.

31:01 It did set the timer like a champ.

31:03 Let me tell you.

31:04 Well done.

31:05 Way to go.

31:07 All right.

31:07 Let's talk a joke.

31:09 Okay.

31:09 Speaking of, you know, the new Siri is supposed to be agentic.

31:12 So the joke is we're agentic now.

31:14 We're an agentic startup.

31:15 You ready?

31:16 This is how you, there's certain things you've got to position yourself.

31:19 I was just watching an ad because I started watching football yesterday.

31:24 And normally ads are excluded from my life, but apparently not in football,

31:28 American football.

31:29 And there's some ad for Zoom that Zoom is an AI company.

31:33 They're not about meetings anymore.

31:34 Nope.

31:35 They can book the thing so that your dry cleaning gets picked up.

31:39 They can do a slideshow.

31:40 Like what?

31:41 Okay.

31:42 So everyone's got to be some kind of AI thing now.

31:44 So here's the joke.

31:48 i changed all of our loading dot dot dot states to thinking dot dot dot we're an agentic startup now

31:55 perfect i'm gonna get right on that yeah get right in there like you can there's so much vc

31:59 money to be had from this go for it discombobulating i'm thinking oh oh i hate that so much about

32:06 Claude Code it drives me crazy that it's got all these random little words yeah the reason i don't

32:10 like it is I don't, it feels like it's made for someone with ADHD who just can't possibly let it

32:16 just be for like five seconds.

32:18 And so if I'm like doing something else, I'd look over and like word

32:21 starts. I'm like, Oh, maybe it's no, it's not done. It's like, Oh, maybe it's done.

32:24 Oh no, no. It's just still like randomly.

32:26 Like, could it just have the little icon go?

32:29 No, no, no, no.

32:30 It's thinking is combobulating.

32:32 It's wording.

32:33 I don't know. What is it doing?

32:34 That's why you need her to exactly.

32:36 That's, that's, that's the story for another episode.

32:38 All right. Sounds good.

32:39 All right. Well, thanks as always for being here, Calvin.

32:42 And thank you everyone for listening.

32:43 We'll talk to you soon.

32:44 Yeah. Bye.