Brought to you by Talk Python Courses and Six Feet Up consulting

#501: A Nod to Sir-Mix-Alot

Published Tue, Oct 20, 2026, recorded Fri, Oct 16, 2026
0:00
00:37:33
Watch this episode on YouTube
Play on YouTube
Watch the live stream replay

About the show

Sponsored by us! Support our work through:

Michael #1: Nod: pause your Python program and open a notebook at any line

Nod (Notebooks-on-Demand) is a JupyterLab extension by Eric Rawn that drops a notebook into a running Python program. Add a notebook() call anywhere, run the program with nod, and execution pauses there with a notebook containing all variables at that point in the call stack. Edits go back to the source file with a button, with cells converted to code and markdown to comments via Jupytext.

  • It answers the classic 'this would be easier in a notebook' debugging moment without copying snippets out to files.
  • Like a breakpoint with a notebook inside: the left panel walks the call stack and notebook state switches with it
  • Install is pip install nodpy plus nod --install-kernel, then run with nod python -m myfile
  • The round trip back to source is the differentiator; markdown cells become comments, cells become newlines, and notebooks are disposable since the source file recreates them
  • nodLog() stashes variable values across calls to explore later; the article also pitches notebook() for stepping through AI-generated code line by line
  • Early stage: repo from December 2025 with 16 stars and essentially one contributor, so expect rough edges

Calvin #2: Polars 2.0 ships: streaming engine by default, spill-to-disk, and a row-order gotcha

  • Polars 2.0 released Oct 6, 2026 (Ritchie Vink). The post says it's "not intended to be a big feature release", but it bundles a few big changes.
  • LazyFrame.collect now defaults to the streaming engine, with large memory and speed gains on most queries.
  • Breaking: streaming doesn't guarantee row order for join, group_by and unpivot. Use maintain_order=True if order matters. This is why it's a major version.
  • Out-of-core (spill-to-disk) is on by default. It spills at about 80% of RAM ("may need tuning") with a 64 GB default disk budget. It covers sort, window functions and many expressions. Joins and group-bys are planned, not shipped.
  • The new native Map dtype (Arrow MapType) replaces the old List(Struct(key, value)) reading. It comes with map.get, map.contains_key, map.len, map.keys and map.values.
  • Stricter by design: errors should surface up front and implicit handling of data mismatches is opt-in. collect_schema() catches schema mismatches without reading data.
  • SQL is now a first-class interface, with join reordering, better common-subplan elimination, and dynamic predicates/bloom filters.
  • Benchmarks are self-run TPC-H/TPC-DS derived numbers against DuckDB 1.5.6 and 2.0 alpha, and DataFusion 54. They are not official TPC results.
  • Polars was fastest on all but one benchmark. At 192 threads its constant overhead hurts small data (SF10 TPC-DS is 1.8x slower), and a fix is hoped for in the next release.
  • Roadmap, not released: out-of-core joins and group-bys, better scaling at high core counts, Polars Cloud, and early GeoPolars work.
  • Upgrade guide: https://docs.pola.rs/releases/upgrade/2/

Michael #3: django-cachalot caches your ORM queries and invalidates them for you

django-cachalot is a Django package that caches ORM queries and invalidates them automatically when the underlying data changes. Install it, add one line to INSTALLED_APPS, and read queries run through your cache backend with no code changes. The trick is that it caches whole tables, so any write flushes that table's cached queries, making it a strong fit for read-heavy sites and a poor one for frequently updated tables. It supports Django 4.2 through 6.1 and Python 3.8 to 3.14.

  • ORM caching that normally takes careful engineering drops in with pip install and one settings line, invalidation included.
  • Near-100x on joins: the README reports nearly 100x speedups for initial deployments heavy on select_related and prefetch_related.
  • It caches the entire table, so a hot table with frequent writes invalidates constantly; docs suggest staying under 50 modifications per minute and mixing in cache-machine for hot paths.
  • Setup is two lines, plus caveats: sync clocks across servers sharing one cache, and run a manage.py command after restoring a database outside Django.

Calvin #4: Towards async transactions in Django

  • Carlton Gibson, The Stack Report, Oct 9, 2026. It's about getting async with transaction.atomic() into Django, hoped for 6.2 and not a commitment. Status check: both PRs are still open as of Oct 19-20, with no approvals and no milestone.
  • The problem: asyncio tasks share one thread and so one DB connection. A slow transaction that rolls back also discards a sibling task's quick write. Today transactions go through sync_to_async, so everything runs serially on one thread.
  • Design: async with transaction.atomic() opens a nested ThreadSensitiveContext, which gets its own worker thread and connection. On exit the connection is cleaned up and control returns to the request's connection. Siblings keep using the request's connection in autocommit, so their writes survive a rollback.
  • Only the task that opened the block may nest further atomic() blocks. Any other task raises TransactionManagementError, and this is enforced.
  • Subtasks inside a transaction are discouraged, since the connection is serial and parallel queries gain little. If you need them, use structured concurrency (TaskGroup or an awaited gather()) so they finish before the block exits. That rule is not enforced yet, and Gibson wants Django to catch late queries before 6.2.
  • Caveats: each async transaction uses an extra connection for its duration, so pools may need tuning. The advice to disable CONN_MAX_AGE for async still applies.
  • Origin: the Django on the Med sprint in Pescara (hosted by Paolo Melchiorre). An expert panel on Day 1 included Lily Acorn, Simon Charette, Jacob Walls, and Charlie Denton and Sam Searles-Bryant of Kraken Tech (authors of django-subatomic, which is being integrated into Django). Caleb Hattingh then investigated the design and opened both PRs.
  • PRs: Django #22074 ("Fixed #33882 -- async with transaction.atomic") has review comments from four people, with the latest commit on Oct 19 adding a test for a late child-task query. asgiref #596 (ThreadSensitiveContext changes) had Gibson's review on Oct 19 and follow-up edits on Oct 20.

Extras

Calvin:

  • Jazzband is Shutting Down (announced in March), but there are alternatives like https://django-commons.org/
  • Treemd — nice TUI for viewing Markdown https://github.com/Epistates/treemd Michael:
  • Talk Python Membership
  • Talk Python Visual topic guides
  • Quick Follow Up to Episode #499 where we shared PEP 824 None coalescing operator, we also have PEP 823 (None Aware access operators) the successor to PEP 505. - via Nick Muoh and Rafael
  • Free Threaded Polars, but staled on PyPI?

Joke: LGTM (Looks Good to Me) - Claude Opus 5.5 music video

Episode Transcript

Collapse transcript

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to

00:04 your earbuds.

00:05 This is episode 501, recorded October 20th, 2026.

00:10 I'm Calvin Hendryx-Parker.

00:11 And I'm Michael Kennedy.

00:12 Great to have you, Michael.

00:13 This show is sponsored through by us, our continual support through our own work.

00:18 Actually, check out Michael's great work over

00:21 at the Talk Python community.

00:23 He's got some great new courses up, some really cool stuff.

00:26 Some people who've been collaborating on the various courses that are on Talk Python.

00:31 And then if you want to support Six Feet Up, so I'm co-founder and Six Feet Up CTO,

00:35 and we do really fun projects with people who've got hard problems and need cool things solved.

00:42 So check out sixfeetup.com and then connect with us over on the socials.

00:45 You can find Michael and myself over on the various regulars, Mastodon,

00:49 Bluesky X, and LinkedIn.

00:51 And then the show itself is on Mastodon Bluesky X.

00:54 And then you can also join us for the live stream. So if you're here with us right now,

00:58 you're on the live stream.

00:59 We're at pythonbytes.fm/live.

01:02 And then if you want the handcrafted, not handcrafted,

01:04 but the bespoke show notes delivered to your inbox,

01:08 the artisanal show notes delivered to your inbox,

01:12 join our friends of the show list and we will never share your email.

01:17 But if we wanted to take a pause,

01:18 Michael, what would that look like? Maybe a little nod, a little nod,

01:22 A little nod, if you will.

01:24 Oh, yeah.

01:24 A little nod.

01:25 So let's talk about notebooks.

01:31 Now, I don't know how you work.

01:34 I could take a guess.

01:35 But I don't know how if you've got a problem and you're like debugging it.

01:38 And you're like, oh, I'm not really sure what I'm going to do here.

01:41 There's different things.

01:42 There's the time-tested print location 1.

01:47 Now I'm in location 1A.

01:49 Now I'm location, you know, the bisection method of like you just start printing out stuff all around the area you're trying to understand.

01:56 And, you know, there's a place for that.

01:58 But maybe we got better tools.

02:00 I don't know.

02:00 I would hope so.

02:01 I mean, I remember the day I discovered PDB and the magic.

02:05 Yeah, exactly.

02:05 Set a breakpoint.

02:06 Let's go in.

02:07 Yeah.

02:07 Yeah.

02:08 You could use, you know, VS Code,

02:10 PyCharm, one of them, and run it in a debugger potentially.

02:14 But there's this project called Nod, Notebooks on Demand.

02:17 So if your style of exploration is notebooks already and you're writing some code and you're like, oh,

02:23 this part is not working.

02:24 I need to kind of debug it. I need to understand it. I need to play with it.

02:28 Then you can use this nod or notebook on demand, which is super cool by Eric Rahn.

02:33 And the advantages that I see here are if I'm somehow able to explore my code with a notebook,

02:39 it's not just a debugger will let you sort of see what's there, step forward,

02:43 step backwards.

02:44 but you can actually explore different ideas.

02:46 Like, well, what if I had a function that did this?

02:49 Or what if I actually had a query that ran it differently?

02:51 Let me write that and see what happens.

02:53 So here's how this works.

02:54 What you do is you import nodpy and you get the notebook.

02:58 Then whenever you want to basically set a break point,

03:01 you just say notebook.

03:03 I don't know, break.

03:04 I might do import notebook as notebook break or something.

03:08 But anyway, you just call notebook and that will pause your code

03:11 And it fires up or attaches to a Jupyter notebook.

03:14 So you've got to install a Jupyter extension that basically lets it listen for these incoming events.

03:20 And when they do, it will break into a notebook instead of to a debugger.

03:24 And you're just in your function as a cell.

03:27 You can run your function.

03:28 You can explore it.

03:29 There's the other functions that are nearby that you can play with.

03:32 You can write other pieces, right?

03:35 Like a new cell.

03:35 You could write a new cell and then go play with that.

03:37 What do you think?

03:38 I love it.

03:38 I think folks who are used to notebooks but are a little intimidated by, say, the debugger,

03:43 this is way more of a natural feel.

03:46 It's weird because notebooks break your mind a little bit

03:49 because you can run the cells out of order.

03:51 You can jump back up and go to a previous thing

03:54 and play there and come back down and play here.

03:57 And so you can kind of lose yourself a little bit.

03:59 But this gives people that capability to express that creativity and debugging

04:05 by using various cells to play.

04:08 It's interesting, like when you're the most stressed out and things are the most not working,

04:13 being in a comfortable place makes a bigger difference than I think just looking at a calm time from the outside.

04:18 Why is this not working?

04:19 Like, oh, OK, but this thing, this thing, I'm really comfortable.

04:22 Let's go.

04:22 And now I understand, right?

04:24 Like a friend of mine had this saying about debugging and code,

04:29 about basically code complexity in general.

04:31 I'll share people like Jason said, debugging is always a little bit harder than writing code.

04:38 you know because there's all the complexities of what's going on so if you write code that's at the

04:43 limit of your capability of like understanding complexity or like the situation then you might

04:49 not be able to debug it right it's like yeah you should always leave a little bit of a buffer in

04:53 there of of that and i feel like this takes that down a notch right because it gives you

04:58 a better understanding during that harder debugging time yeah well it was like the difference between

05:03 going from PDB to, say, PyCharm for a debugger.

05:06 I remember in PDB, traversing up the call stack

05:10 and down into things just required you kind of holding a lot of that in your head. You didn't

05:14 get a good visual model of it, where if you use PyCharm, you could just kind of click through the

05:18 stack and you see the rest of the stack still sitting on the screen. There's the call stack right there.

05:21 Exactly.

05:21 You can just see it.

05:22 Yeah. And so this gives you that same comfort level and visual

05:27 anchoring when debugging. So I love the fact that they have built this for the notebook natives

05:33 I agree.

05:34 And I believe there's a couple other out there.

05:36 But one of the things that's interesting about this one is the markdown cells become comments,

05:42 cells become new lines, and the notebooks you can just throw away because they get saved into the source file if you want.

05:49 So if you run it again, it kind of recreates the exploration that you set up.

05:53 Oh, that's really nice.

05:54 It can persist from run to run.

05:57 Yeah.

05:58 So it's a pretty new project.

05:59 Don't expect it to be super polished.

06:01 But people well, check it out.

06:02 I think it's certainly worth a try.

06:04 Yeah, it looks really cool.

06:05 We should definitely check that out.

06:07 Definitely.

06:08 Over to you.

06:08 All right.

06:09 Well, are you ready for the new thing,

06:12 the hot new release of Polars?

06:14 Polars 2.0 has shipped.

06:16 And streaming engine is on by default.

06:19 There's a lot of new features.

06:21 That was the reason for the major bump in version

06:23 is because there's going to be some changes and some things you're going to need to do

06:28 as you upgrade to your new version of Polars.

06:31 But one other thing to mention here is really, really important is the performance.

06:36 I know we've mentioned in previous episodes, we've talked about Polars performance

06:39 versus things like DuckDB.

06:41 And in the blog post of the release here by Richie,

06:44 they've got some performance numbers here because SQL is a first class citizen now

06:50 in the Polars environment.

06:51 So if you are doing things like you would have done

06:53 in DuckDB, for example,

06:54 maybe you're more comfortable there because of the SQL environment,

06:58 and you love the speed of DuckDB, look at these numbers.

07:01 This is on some AWS instances here on a C7 4X large,

07:05 16 vCPUs and 32 giga RAM.

07:07 That is a large.

07:08 That is a very large.

07:09 They're running two different benchmarks here, the TPCH SF10 and the SF100.

07:15 And you can see here, Polars completes it in 3.2 seconds

07:19 compared to DuckDBDB, which is fast.

07:21 I mean, we all know compared to what we read in our,

07:24 went over in the last episode, DuckDB is at 4.44 seconds on that same benchmark.

07:30 And then they basically did the DS version of that benchmark as well and saw that we've got

07:34 Polars at 12.59 seconds where the current release of DuckDB is at 15.92.

07:40 The alpha of DuckDB 2.0 still slower at 14.3.

07:44 That's not saying don't use DuckDB but I'm just saying Polars is making a

07:47 real run for performance here with some of the new enhancements.

07:50 That streaming capability really minimizes memory usage.

07:55 And when you do use heavy memory on these various pullers examples,

07:59 you'll get the new, well,

08:01 new, basically the out of core or spill to disk is on by default.

08:05 So if you start hitting, for example,

08:08 by default, it's set to 80% of your RAM usage,

08:10 it will spill out to disk for these larger operations.

08:15 And it supports joins and group buys are not supported yet. They're planned but not shipped,

08:22 but it does support sort window functions and many expressions.

08:26 So those things will actually continue running even if you would have gotten an out of memory

08:30 or maybe the Ohm killer would have come and killed your process.

08:33 You can actually make these bigger datasets work

08:37 by leveraging that spill to disk feature of it.

08:40 What a neat idea because the examples you got on the screen

08:44 do have a lot of memory, but at the same time,

08:46 memory is expensive even before all this crazy stuff, right?

08:49 Relative to how much you can have.

08:51 Like memory on a server is more expensive than an NVMe disk that is not as fast, but still really fast.

08:59 But still pretty darn fast.

09:00 I mean, you're basically kind of going back a couple of years in memory technology, but you're gaining the ability to finish the work.

09:07 So I think that may be more important.

09:10 Yeah, exactly.

09:10 A little detail.

09:12 Oh, all of a sudden I went from 16 gigs of RAM to 500 and I don't know,

09:16 30 or whatever, right?

09:17 Yep.

09:17 Yeah, so one of the things here, again, I mentioned the streaming engine is on by default,

09:21 but that has a side effect.

09:22 So one of the things you need to consider when you're upgrading your code from 1.0

09:25 or the one version to the two version is that order of operations

09:30 or order of the data is not guaranteed in the streaming version

09:33 unless you set maintaining order to true.

09:35 So there's certain row level or row order default operations

09:39 that might want that.

09:40 Be sure you check out the upgrading guide before you switch up to 2.0.

09:45 And then there's also the new map data type called map type.

09:48 So it's now the arrow map type supported in the Polars map data type. So if you're in the arrows game,

09:56 which is actually kind of cool because arrow gives you that in-memory transportable data type that you can move between

10:04 various processes, Polars now supports the map type there, which is basically like a dictionary

10:09 that's super powered in the Polars world.

10:12 That's kind of the big, big news here.

10:15 There's last words, I don't know, coming out in months, but I think the benchmarks are the thing

10:21 speaks for itself on this release.

10:23 It is incredibly fast performance.

10:26 This is another thing I just I love about the Python community is people may claim Python slow and maybe should be using

10:33 Go or Rust. I don't know. I think you can there's lots of great work going on to make sure that

10:38 Python is still a first class citizen and maintains that performance that is needed to do great

10:43 production work with these tools.

10:45 Yeah.

10:45 If you've got a large scale application so frequently it's

10:49 There's a little part of it that is performance critical and 90% of it that's not as well.

10:55 And you can bring different tools to bear on that 10%.

10:58 Yep.

10:59 Yep.

10:59 There's an upgrade guide.

11:00 So if you go to the Polars documentation under the releases upgrade slash two.

11:05 So make sure you check that out before performing any kind of an update or point your LLM agent

11:10 at that upgrade guide and say, make it happen.

11:12 You probably get better results by doing that.

11:14 Yes.

11:15 A hundred percent you will.

11:16 Yeah.

11:16 Yeah.

11:17 I'm sure you will.

11:17 So to your performance comment, I don't think that this had anything to do with free threaded Python or any concurrency on the consumer side.

11:26 No, I don't think so.

11:27 Nope, not yet.

11:28 But it certainly looks like Holders itself internally is using multiple threads.

11:34 Otherwise, why would you get a much better result on the much larger server,

11:38 right?

11:38 Unless it's just pure memory.

11:39 So you're still in a sense, this is like...

11:41 Yeah, number cores wouldn't matter unless you're using that.

11:43 Yeah, exactly.

11:44 Exactly.

11:44 So it's like, hey, we don't have this great threading story, or do we?

11:48 We don't have this great threading story on Python, but still, here's the part where you actually care.

11:53 Yep.

11:53 You know, jamming on it anyway.

11:55 All right.

11:55 So pick the right libraries.

11:57 And that'd be good.

11:57 Speaking of performance, Michael, what have we cached more?

12:01 Would you say if you could cache a lot?

12:03 I love that name of this project.

12:06 Django cache a lot.

12:08 Yeah.

12:09 A little bit of CERM makes a lot.

12:11 A real throwback, right?

12:12 I'm here for it all day long.

12:13 I am.

12:13 So am I.

12:14 Oh my gosh, what a music genre that was back in the day.

12:18 That was fun.

12:19 So this project, 1.4 thousand stars,

12:24 called Django Cash-A-Lot.

12:26 I think we've got to just throw it out there, the requisite joke.

12:28 Three things, two, I already broke it.

12:31 There's two things that are hard in computer science.

12:33 Right.

12:34 Naming things, cache and validation and off by one errors,

12:37 right?

12:37 Yeah.

12:37 But the fact that cache and validation is like one of the first two before the joke even goes off the rails,

12:43 is it's interesting because it absolutely is not easy.

12:47 So there's different types of caching that you might want to do.

12:51 One way to cache is just I'm going to put some kind of front end

12:55 or pre-compute or something in front of a page.

12:58 So like on this page, it's got the issues,

13:02 counts, it's got the number of watchers,

13:04 the forks, all that kind of stuff.

13:06 GitHub could just say, well, we're just going to pre-render all of that stuff.

13:09 And when you ask, here are your bytes back, right?

13:12 That's probably the most effective, but it also has a lot of challenges.

13:15 For example, what if a new issue comes up while I'm sitting on this page?

13:19 Should that refresh?

13:20 Maybe.

13:20 What about my little picture up here?

13:22 If you just straight up cache the whole thing, Calvin logs in.

13:26 He doesn't want to see me, and I certainly don't want him accessing my stuff.

13:29 You know what I mean?

13:30 So there's just a lot of issues with caching that much.

13:33 So one way you can cache that's really interesting is just say,

13:36 well, what if everything I asked for the database was already,

13:40 the queries were ran, the results were given back, they were parsed, and they were turned into ORM objects,

13:46 and then those things were held on to, right?

13:48 That could be a much better option where you say, give me the details of this page, that part is cached,

13:54 give me the details of the user, that part is not cached,

13:57 and then render it, right?

13:59 So that's kind of where this thing lives.

14:01 It basically caches your ORM queries, and then here's the joke part.

14:05 Hopefully this part is not the joke, but there's a part that is challenging,

14:09 the invalidation. So this automatically invalidates them,

14:13 which I think is pretty interesting.

14:15 Notices when your tables change or something like that.

14:17 And hey, this cache is now invalid.

14:19 That's really, really nice.

14:21 Yeah.

14:22 Because otherwise you're spending all the time and

14:24 mechanics to figure that out.

14:25 Yeah. You know, sometimes you've got to subscribe to changes

14:29 if your database server gives you those kinds of things.

14:32 I don't know.

14:33 It's pretty tricky.

14:39 So the edge cases are here though there's probably still some not funness i believe it plugs into

14:44 the django um just caching built-in caching yeah stuff right for the orm because it points out hey

14:51 if you have multiple servers doing the caching i know multiple web servers and i bet servers also

14:57 would just mean web workers as in web garden anyway with a common cache server uh server

15:02 make sure they have their clock synchronized because they're gonna have to use uh each other's

15:07 numbers and so on.

15:08 But that seems like a solved problem these days.

15:10 If you don't have clock synchronization, you've got lots of other problems happening too.

15:14 Exactly.

15:15 Exactly.

15:16 SSL or TLS won't work.

15:17 Details.

15:18 Details.

15:19 Also, if you run a script that somehow changes the database outside of that,

15:24 so it needs to run manage.py.

15:26 So I imagine that what's happening is there's a last modified

15:29 on the cache server.

15:31 And every time you run a query, it goes, when was this last modified?

15:33 And if that changes, then it's just going to do a full query.

15:36 Otherwise, it'll just keep pulling.

15:37 I'll bet you that that's it.

15:38 Because otherwise, if you just use the ORM directly through this other script, it would

15:43 have changed it.

15:43 But you've got to run this manage pi command on it.

15:46 That's what I'm guessing.

15:47 You've got to be in the Django world, the Django machinery for it.

15:51 Exactly.

15:52 I've had this problem before.

15:53 If you run Redis caches in Redis command, like Redis through Django, like accessing it,

16:00 not just as a cache, but using it directly.

16:02 and you try and do something outside of that Redis,

16:04 there's some subtleties there that you've got to be really, really careful of.

16:07 So it sounds like this is similar.

16:08 Yeah, yeah.

16:09 It's using Django ORM technology, not the database technology or the cache technology,

16:16 right?

16:17 Like I said, like subscribing to changes to a table or whatever.

16:20 So one of the things that Warren's about is if you have a high rate of database modifications,

16:24 then that'll keep invalidating your cache.

16:26 So maybe it won't help that much.

16:29 The win, maybe a lose.

16:31 Exactly.

16:32 You're just adding complexity, but maybe every read effectively is going to just run anyway.

16:38 So there's a couple of limits.

16:39 People can check them out, different setups and so on.

16:42 But I think it's a cool project.

16:43 So yeah.

16:45 If you're looking to kind of gain a performance gain real quick there, that's an easy one to try and see if it fits your use case.

16:53 Caching is such a bit of magic tixie dust that you can sprinkle on.

16:57 But the challenge is it's the complexity and the invalidation.

17:00 Yeah.

17:01 Yeah, because you don't want to serve the stale data like you mentioned at the beginning.

17:04 Yeah.

17:05 That's interesting.

17:06 Performance is tricky.

17:07 Cash it.

17:08 Yeah.

17:08 Yeah, performance is tricky.

17:10 And speaking of performance, some people may think asynchronous is the answer to all things in performance.

17:17 But Carlton Gibson actually had a good bit of work done at the Django on the Med sprint.

17:23 So if you're not familiar with code sprints, people get together from time to time and will work on the open source projects.

17:29 And one of the outcomes from the latest sprint was a lot of work toward making async

17:34 with transaction atomic come to Django, hopefully in 6.2,

17:38 which would be pretty incredible because that's the next release of Django.

17:41 And this is one of the pieces that's been missing

17:44 for a while in the Django ecosystem is the ability to use the ORM in an asynchronous fashion.

17:49 Because currently ORM, you have to still wrap it

17:52 with async to async version of that.

17:55 And so he's got some example code here showing basically what this is all about.

18:00 But this is also a very, very tricky problem.

18:03 Asynchronous operations and concurrency, where multiple things can happen in any order,

18:08 don't combine well typically with database transactions,

18:10 which are necessarily well ordered, as he mentions here in the blog post.

18:15 There's a PR open as of, I think, yesterday and today

18:19 with no approvals and no milestones yet.

18:21 So if you're excited about transactions and the Django ecosystem and ORMs,

18:26 you may want to go check out the current work that Carlton did based on this sprint.

18:30 He made quite a bit of forward progress.

18:33 And again, I think that this is not an easy problem

18:36 or else it would have been solved long ago.

18:38 We've had asynchronous views in Django now for years and years.

18:42 And I think that, like I mentioned here,

18:44 Django has required using the ORM's sync API for transactions specifically.

18:49 If you aren't using a transaction, if you didn't wrap it in a begin and a commit,

18:52 which is what you kind of put a wrap a transaction into

18:54 because you're running multiple SQL commands at one time

18:58 or multiple ORM operations at one time, you have to use this sync to async wrapper

19:03 to make that all go correctly.

19:05 But now if we get this async with transaction atomic,

19:09 Django, it could actually make this work.

19:11 I just think that he puts a little ad in here for the Django on the Med sprint,

19:16 which I highly recommend people going to sprints.

19:17 It's a really great way to get involved in the community.

19:19 But if you wanna be able to use this version of the code,

19:23 There's again a PR.

19:24 You can check out the thread.

19:26 You can see how it works for your specific workloads.

19:28 I think a lot of this is going to be workload specific

19:31 and you're heading into kind of extreme performance area here

19:35 when you're talking about asynchronous operations on transactions.

19:39 This could be a big win for your specific workloads.

19:42 It could also be a big headache if it doesn't operate as expected

19:45 or you're not paying close enough attention to the order of operations

19:48 for some of these things, which it goes over with in the post.

19:51 It's a really good post.

19:52 They had a fun time there on the Mediterranean.

19:55 Not terrible going for bicycle rides along the sea.

19:57 That looks all right.

19:58 They got quite a bit of work done.

20:00 He actually had, it looks like he collaborated with Caleb Haddington,

20:03 who wrote a book on that.

20:05 So you actually click over here, the Using Async in Python.

20:09 So someone who knew quite a bit about the async story in Python.

20:13 And so they spent literally three days working through this and ended up with a pull request

20:19 against the ASGIE ref and on the Django bits to make this all go.

20:23 That's pretty optimistic.

20:24 It seems like the right brains got together at the right time,

20:26 which is again the reason why we do things like sprints where people get together in person in the same place,

20:32 and usually they're in pretty nice places,

20:33 to work on awesome code like this.

20:36 So you can see here, again, pretty extensive,

20:39 a couple poll requests for people to go review,

20:42 and lots of thanks, and lots of great things

20:44 happened at Django on the Met.

20:45 So I don't know, how do you feel about asynchronous transactions

20:48 to a database?

20:49 I feel okay with it, I think.

20:51 I think I feel okay with it.

20:53 I had Calvin, I had Carlton, sorry, Carlton. I had Carlton on.

20:58 He and I look the same.

21:00 Yeah, I know.

21:01 Well, it's the C.

21:03 So I had him on to talk about updates to Django's async story.

21:08 And it sounds like Django's async story is really strong these days.

21:11 It's just down to a couple of little hard problems to solve,

21:16 like transactions.

21:17 I'm a big fan of async.

21:19 I think it unlocks a bunch of different deployment stories and operational stories that are really powerful.

21:26 It doesn't always solve a ton of problems, but it can.

21:30 Yeah, well, I feel like this keeps Django more than relevant in today's day and age of things like FastAPI and everyone basically touting full asynchronous stories.

21:42 Django is still a very, very valid option when it comes to a web framework.

21:45 And Frank Wiles and I did a presentation at this year's DjangoCon comparing FastAPI with Django and Ninja,

21:52 which Ninja is an API layer on top of Django that does async really, really well.

21:56 and the performance is obviously FastAPI still faster,

21:59 but the performance is really, really good for Django

22:01 if you're using things like Ninja because it leverages this asynchronous capabilities

22:06 that folks like Carlton have worked really, really hard

22:09 to make sure that Django has a good story for.

22:11 Yeah.

22:11 You were talking about Django Ninja, right?

22:13 Yeah, yeah, Django Ninja.

22:14 Yeah, which is super neat.

22:15 Yeah, the analogous to FastAPI.

22:18 So I think it's really powerful.

22:21 One thing I would throw out there though, this is not even necessarily related to async,

22:26 but it could be in the mix here.

22:28 So if you're worried about transactions and performance,

22:31 and this is a big caveat, but if it's true,

22:34 and you're using ORMs, you could use a design pattern called optimistic concurrency,

22:39 which itself is already a pretty interesting thing.

22:42 Like ORMs don't necessarily always work that well with transactions.

22:47 Right.

22:47 Because I'm going to do a query, I'm going to get a user object.

22:49 I'll return that user object to some other part of the code.

22:52 If it makes a change to it, it needs to save it.

22:54 Like you don't want to let the entire request run within a single transaction,

22:58 right?

22:58 These need to be like, I'm doing the data stuff and then I'm done.

23:01 But if you say like, well, we're going to enforce concurrency by just like the beginning

23:04 of a web request and the end of the web request becomes a transaction.

23:07 You're just saying I'm having a single threaded process end to end, which is not great.

23:11 So optimistic concurrency puts a last modified hash on the result.

23:16 And when you try to save it, it's going to say update the username to name where the modified

23:22 hash is equal to the one that I have.

23:23 And if that changes, like, oh, it was changed behind you,

23:26 you know, you could catch that, requery it, make the, you know what I mean?

23:29 Like that can get you some better concurrency and some better safety if you're using an ORM, right?

23:34 Like that's a pretty big story.

23:36 But I don't know.

23:36 There's a lot going on here.

23:37 I don't know how I feel about ASync transactions because I try to avoid using too many transactions already.

23:44 Usually you don't.

23:45 But, you know, if you're running a bank,

23:46 you want them.

23:47 Right.

23:47 things can get out of order and, and then you could end up with a strange right history based on performance.

23:53 If some slow query opens the first transaction and there's a fast one that

23:56 comes behind it, making a right,

23:59 you've got to make sure you handle that.

24:00 So it looks like we're working on it.

24:02 Yeah.

24:03 Yeah.

24:03 And you got to start talking like transaction, uh,

24:06 concurrency levels within trans, like read committed,

24:08 read uncommitted, all the different things you can set on a database.

24:11 It gets, all right.

24:12 I always, that part's still hard.

24:14 That part's still hard.

24:14 I can't do it.

24:15 We just got to talk.

24:16 Let's just talk about something.

24:16 Let's go talk about our extras.

24:18 How about some extras, Michael?

24:19 What do you got?

24:19 Well, let's see here.

24:20 I already got them all queued up.

24:22 So first extra, this comes to us from Christian Klaus,

24:26 as you can see here, right now here, at Mike C. Kennedy,

24:29 FYI update on Calvin's first topic, free threaded wheels for polars.

24:36 Oh, wow.

24:37 Very quick.

24:37 So we've already got, well, there's a very interesting story and people out there may

24:43 know a little better than I do.

24:44 So I only have like very sparse GitHub comments to go on.

24:47 So please give me some leeway to get it right here.

24:50 So this is an open issue to add free-threaded wheels,

24:53 like add the A-B-I-T wheels to PyPI.

24:57 So it says currently Polars builds on the free-threaded Python build.

25:01 So you can use it, but there's no binary wheels.

25:04 It's going to be like a slow install build or rust and so on.

25:09 But to be clear, you can use Polars 2 on free-threaded Python.

25:13 Just the installing of it is a bit tricky.

25:16 So this issue is like, can we make that better?

25:19 It's open.

25:19 That's the part that's interesting, that it's not closed.

25:22 But if you go down here somewhere, if I look for PyPI,

25:27 it says, however, before we can do it, we still have to wait on getting more space from PyPI

25:32 because PyPI has said that you're using too much space

25:36 and we're not going to allow you to upload anything anymore.

25:39 That's a new one.

25:40 So you can see the request here.

25:41 If you jump over to it and you wait for GitHub, because GitHub has become slow the last few years,

25:46 not on a red sentence fault.

25:48 But over on PyPI, it says, project limit request, pullers runtime compatibility,

25:53 50 gigabytes, please.

25:56 And it's asking for, this is by Richie Vink,

25:59 the creator, as you mentioned before.

26:02 And it said, help, we can't upload anymore.

26:04 So this is an interesting problem because...

26:07 Yeah, obviously they're trying to avoid abuse of PyPI.

26:10 Right. And just trying to pay for it. You know, they're asking for 50 gigs. That's a lot of space.

26:16 At the same time, if you have a popular project that has a big binary built component like the rest part of this,

26:23 and you want to support all the different things, right?

26:26 I want to support free threaded wheels on ARM, you know, like all the different things that you might build for that starts to explode out into a bunch of combinatorial options that take a lot of space.

26:36 right so i'm sympathetic to it somebody it's just i don't know i can't remember where it was um so

26:41 it's like do do we really need pi pi yes you really need it to be or do do we really no do

26:47 we really need old releases on pi pi i'm like yes yes yes you do please anyway thank you christian

26:53 klaus for sending this in this is um it's about polars it ties into your items but it's also just

26:59 an interesting difficulty of the day you know there's still things going on behind the scenes

27:04 absolutely okay two more things i want to cover real quick let's do this one next so i've i started

27:11 working on these write-ups that are basically telling like guides visual guides to talk python

27:17 episodes that tell this the technical story of what the episode was about so for example we've got

27:24 let's see the probably the funnest one is this eve because it's a game so it's got maybe 10 to 15

27:31 graphics.

27:31 Got a nice little reader view.

27:32 You can do like full screen or you can, you know,

27:36 a little, it's like a little Instapaper-ish because it's,

27:39 it's a decent amount of reading. So it goes through and it says,

27:41 you know, okay, what is this? It talks about like the problem with nice pictures,

27:46 tells the story of, of what all the, what's going on. And just,

27:49 it has this like visual storytelling component of each,

27:53 um, of each episode.

27:55 And so I've been working on this. I don't have that many

27:57 out.

27:57 I think I've got six out, but I just, I was fascinated with like,

28:01 could I make a visual storytelling thing?

28:04 That's just not, it's not just narration,

28:06 right? It's, it's, let's look at the

28:07 different things discussed in the episode and kind of make a technical article out of them.

28:12 Yeah.

28:12 Well, and people learn differently.

28:14 Some people want to hear the podcast.

28:15 Some people want to read the blog posts and some people would love this visual representation,

28:19 which is, I like how it themed or how you theme the graphics,

28:23 you know, in line with Eve, you know, as opposed to. Yeah.

28:25 Thanks. There's a bunch of really cool graphics in here.

28:29 It's very sci-fi looking.

28:31 Yeah.

28:31 And each one's a little bit of a different theme.

28:34 I'll go to the one that we just did together, the state of the art in Python.

28:40 And it's got some really interesting pictures.

28:43 Check it out. If you just go to Talk Python, hover over episodes,

28:47 it drops into the visual guides and you can go and see all of them.

28:50 Like I said, there's a handful of them.

28:51 It's not just a podcast anymore.

28:53 No, it's so much more.

28:55 So much more.

28:57 And you've got so much more.

28:59 Then, remember how I lamented?

29:01 And I wish I knew this.

29:02 Two people sent this in.

29:04 Nick Moua.

29:04 Hey, Nick.

29:05 And Raphael Bosen and said, Michael,

29:07 you kind of went off on PEP 824 about like, why is it just so focused?

29:12 Couldn't it have this other thing that you wanted?

29:14 And you're like, no, I don't really want it.

29:16 What are you talking about?

29:16 You're insane.

29:17 Well, if I had just gone back at one in PEP numbers,

29:22 8pep823 is in fact that null coalescing operator that I was looking for right like a.b question mark

29:29 dot c d is the complicated sort of deal here that if I'm trying to access something in a chain

29:37 like x.y.z.whatever if any of those is none I need to just stop and say it's none or stop and

29:43 raise an exception like that that that is not none type has no attribute whatever this is what that's

29:50 about. So that's pretty interesting.

29:51 And I think I'm here for it.

29:53 Nice.

29:53 Yeah.

29:54 Last thing. And this one is, it's a big one.

29:56 This is really fun.

29:58 This is really, yeah, this is a big one.

29:59 So I'm launching Talk Python membership,

30:03 and I'm hoping that we can create a really nice community over

30:06 there that has a couple of aspects.

30:08 So I've been thinking for a long time, like,

30:10 couldn't I offer something like a little bit more for people,

30:13 listeners of the podcast and this podcast too, that's one thing I could offer that seems super obvious, like, hey, you get ad free episodes,

30:19 or you get episodes ahead of time.

30:21 It's like, well, you know, that's great,

30:23 but that's not that great of it.

30:24 Like that's not a huge benefit.

30:25 So sure, Talk Python membership does come with ads,

30:28 but I wanted to see what else I could offer.

30:30 So it comes with a nice Discord community.

30:33 A bunch of the Talk Python guests are already in the community.

30:36 So you can come hang out with many of the people that you've heard on the show.

30:40 Yeah.

30:40 Calvin, for example.

30:41 It's been one of the reasons I've now keeping Discord open

30:44 more regularly.

30:46 That's awesome.

30:46 There's enough gravitational pull because of the work you're doing there

30:49 I've keep it open now.

30:50 Thank you.

30:51 That's awesome.

30:51 Yeah, I do too.

30:52 And it's really, really, it's a neat area to be in.

30:54 And if you're not a gamer, just Discord is easier.

30:57 It's easier than Slack.

30:58 It's not just for gamers.

30:59 It's actually just an easier Slack.

31:01 I agree.

31:02 And one of the main things I wanted to focus on is live events.

31:06 So it comes with a meetup.

31:09 If you would like to go to a Python meetup or a regional conference or something like that,

31:13 but your area doesn't have one, you can't make it, or you just want to go to more,

31:17 You could go and I'm running a monthly tech session where it's like a conference style presentation,

31:24 but then sort of a mix and mingle, just hang out on Discord video channels with whoever gave the presentation and everyone else,

31:31 me included.

31:31 So all those things.

31:33 And I'm also going to add a preview, like a summary type of thing to the episode.

31:39 So a separate sidecar podcast feed.

31:42 Like here's an hour and 20 minute podcast episode, but here's a four minute.

31:46 What's it about?

31:47 kind of the cliff notes.

31:48 And if you find that interesting, then you can go listen to the full one,

31:51 right?

31:51 Kind of let you, you know, protect your time a little bit,

31:53 but keep up on the ones that, I don't know, you don't have the time to listen.

31:56 I love it.

31:57 I'm here for it.

31:57 I'm all about community.

31:58 And so this is a great, great move.

32:00 I'm glad you're doing this.

32:01 Thanks.

32:02 Thank you so much.

32:02 So folks, support all the work.

32:05 Talkpython.fm.

32:06 Click on membership right at the top.

32:08 Check it out.

32:08 Be part of it.

32:09 Hope you're there.

32:09 Cool.

32:10 Well, thank you for all that.

32:11 That's an amazing extra to end on there for you.

32:13 I've got a couple extras that we want

32:17 is the fact that Jazz Band is shutting down.

32:19 That was announced back in March, but there was a renewed discussion a week ago here on Reddit.

32:25 And one of the things that came out of that is,

32:26 by the way, there's Django Commons

32:28 for things like what Jazz Band used to do.

32:30 So if you wanted to come out and have a place where you can find

32:34 kind of more of the Commons supported projects for Django,

32:37 make sure you check out django-commons.org.

32:40 You can join the Commons, you can see the projects,

32:42 you can see here, you know, pretty heavy hitter projects,

32:45 Django Valkey, so support for Valkey, things like Tailwind CLI,

32:50 DRF stuff.

32:52 One of my favorites, Django FSM 2

32:55 is now part of the Commons projects.

32:57 So tons more that are joining in there, all one home,

33:00 which means they have a nice, safe place to live.

33:02 So check out the Django Commons.

33:04 The other thing, this is not necessarily Python related,

33:07 but boy, is it an awesome, fun little tool I found.

33:10 Since I'm doing more and more of my agentic work

33:13 on remote machines and a lot of the output maybe markdown files,

33:17 I would love to have a very rich way to view markdown in the terminal.

33:22 So I went and searched for a TUI slash CLI markdown navigator and I found TreeMD.

33:27 I believe it is in Rust.

33:29 Rust? Yeah, 99.2% Rust.

33:32 But it is a really nice markdown viewer, which gives you obviously mouse usage. A lot of the TUI apps,

33:39 the big benefit is you can use the mouse in your terminal.

33:42 It gives you an outline.

33:43 It gives you obviously a rich rendering.

33:46 If your terminal supports images,

33:48 it will see inline images inside the markdown files.

33:51 Wow.

33:52 Yeah, that's cool.

33:52 See by their screenshot doesn't show that off because that's actually a really fancy

33:56 feature of this.

33:57 You can jump in the outline.

33:59 I'll make that bigger maybe.

34:00 Yeah, there you go. You can jump in the outline to various sections.

34:03 You can also pre-filter a markdown file from the

34:06 command line to say, show me all the Rust snippets or Python code snippets in there.

34:11 And then it'll basically filter the document down to just those snippets and give you a navigator of the various

34:16 snippets. Cause you're like, I'm looking for a code sample in this markdown.

34:19 Just show me only those pieces.

34:21 And you can do that with a tree MD. So I know again, it's not a, not a Python tool,

34:24 but boy, is it a part of my new productivity toolkit of great things that I have installed

34:30 to basically remote work remotely with my agents and the outputs that are coming from them.

34:36 So another cool thing that I'll mention, remote documents.

34:39 You can type treeMD and like a GitHub repo reference to owner slash repo,

34:44 and it will get to open the readme immediately.

34:47 Or you can just put in any URL to a raw markdown file on the web,

34:51 and it's now open in your local viewer in your command line.

34:54 So you can be remote.

34:55 You know, I've been doubling down on my usage of herdr

34:59 and my framework desktop that's over here so that I can run my Pi sessions on my CloudCo sessions.

35:03 And now I can view all my markdown files.

35:06 so that was a cool one i think it's a great a great find it looks really nice so two things first one

35:11 thing i've learned there are so many projects by running this show there are so many projects out

35:16 there that are a UI focused sort of thing and they sometimes don't even have one screenshot what are

35:20 you doing you're literally the thing you're about this you are about pictures why do you have no

35:27 picture right there's only words here and it's got themes too oh yeah i love it like nine different

35:32 themes.

35:34 Yeah.

35:34 And the other, I think, is our mission is to bring tools to Python developers,

35:38 not necessarily.

35:39 I don't think they don't have to be Python tools.

35:42 It doesn't have to be written in, compiled in or whatever,

35:45 Python, in order for it to be something awesome that we can use. So

35:48 I think it's great.

35:49 Cool.

35:50 All right, Michael.

35:51 Would you say that it looks good for you?

35:53 It does look good. Oh, no, it started playing.

35:57 All right, so the joke this week comes to us from this guy named Calvin sent this in to me.

36:04 No, Calvin shared this with me earlier, and it's, boy, this thing captures the zeitgeist so well, doesn't it?

36:10 It's called Looks Good to Me, LGTM,

36:13 Claude Opus 5.5.

36:16 It starts on a Friday.

36:18 It's this kinetic typography, a lot of moving words sort of interesting.

36:23 Really well done, I think, on the graphics.

36:25 And it talks about basically the story of the manager who gets super psyched about AI and just makes everyone use it.

36:34 Yeah.

36:34 So go give it a listen.

36:35 It's a fun little music video.

36:37 I believe it was generated by AI itself, too,

36:40 which is kind of fun.

36:40 Oh, it's got to be.

36:41 It's got to be.

36:42 Very meta.

36:44 You've got to be careful because once you go down this path, I believe it will send you down a path of other Opus 5 videos.

36:51 I'll set the stage.

36:52 So there's a Slack.

36:53 I mean, it's not actually Slack, but you know it's Slack.

36:55 and Brad, VP of Velocity, Vice President of Velocity.

37:00 Hey team, quick one.

37:01 Before the weekend, leadership is super excited about AI.

37:04 Off it goes.

37:05 It's like, ooh.

37:07 Give it a listen.

37:09 You'll definitely laugh.

37:10 It's very fun.

37:11 You'll get a giggle.

37:12 Yeah, you will.

37:13 You know what?

37:13 I think this is a perfect bookend as well, Calvin.

37:16 We started with Sir Mix-a-Lot and we end with LGTM.

37:22 Let's go.

37:22 Looks good to me.

37:24 All right, Michael.

37:25 This episode looks good to me.

37:27 Let's give it a wrap.

37:28 Let's give it a wrap.

37:29 Oh, my gosh.

37:30 All right.

37:30 Bye, all.

37:31 See y'all later.


Want to go deeper? Check our courses




Subscribe to Python Bytes