Transcript #217: Use your cloud SSD for fast, cross-process caching

00:00 - Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 217, recorded, what is it, January 19, 2021.

00:12 I'm Brian Okken.

00:13 - I'm Michael Kennedy.

00:14 - And I'm Ogie Moore.

00:15 - Welcome, thanks for joining us.

00:16 - Thanks for having me. - Thanks for having me.

00:18 - Yeah, thanks for coming.

00:19 - Who's first?

00:20 Michael's first.

00:20 - I'm first.

00:21 You wanna talk about caching?

00:22 I got some cool stuff to talk about with caching.

00:24 So I recently got a recommendation from Ian Maurer, who was talking about genetics and biology over on Talk Python, I think 154, so a while back, but he pointed out this project called Python Disk Cache.

00:38 And it just seems like such a cool project to me.

00:42 So one of the big problems, or not problems, one of the trade-offs or the mix of resources we have to work with when we're running stuff in the cloud so often has to do with limited RAM, limited memory in that regard, and limited CPU, but usually have a ton of disk space.

00:57 For example, on my server, I think I've got like using five gigs out of 25 gigs, but I've only got, you know, two or four gigs of RAM, right?

01:06 But one of the things you can do to make your code incredibly fast is to cache stuff that's expensive, right?

01:12 If you're gonna do a complicated series of database queries, maybe just save the result and refresh it every so often or something like that, right?

01:18 Well, this library here is kind of the simplest version of one of these caches.

01:23 Like people often recommend memcached, They talk about Redis.

01:28 You might even store something in your database and then pull it back out.

01:31 And all those things are fine.

01:32 They just have extra complexity.

01:33 Now I have a separate database server to talk to if I didn't have one before.

01:37 I've got a Redis caching server now I gotta share.

01:40 What if you just use that extra hard disk space to make your app faster?

01:44 A lot of these cloud systems like Linux, for example, they have SSDs for the hard drive.

01:49 So if you store something and then read it back, it's gonna be blazing fast, right?

01:52 So, disk cache is all about allowing you to do, you know, put this thing in the cache, to get it from the cache, but it actually stores it in the file system.

01:59 That's pretty cool, right?

02:00 - Yeah.

02:01 - Yeah, so it's super easy to use.

02:03 You can just come up here and say, import disk cache, just to get an item, I just say cache, like a dictionary basically, and to put it back, same thing.

02:11 You give it a key and a value.

02:12 It is basically like a dictionary, but it persists across runs.

02:16 It's multi-threaded, multi-process safe, and all those kinds of things.

02:20 So, incredibly cool.

02:22 It's pure Python.

02:23 It runs in process.

02:24 So there's not like a server to manage.

02:26 It has a hundred percent test coverage, hours of stress testing.

02:29 It's focused on performance.

02:30 And it actually, Django has a built-in caching API in Django and you can plug this into Django.

02:36 So when people say cache with my thing, even third-party apps and stuff, you can automatically start using this, which is pretty awesome.

02:42 It has support for eviction.

02:44 So last, most recently used, first and so on.

02:49 you can tag things and say these can get evicted sooner and whatnot.

02:54 So really, really nice, incredibly easy to use.

02:56 I definitely recommend people check it out because very nice.

02:59 It has different kinds of data structures that you can work with like a fan out cache, a Django cache, a regular cache and so on.

03:05 So if you wanna work with some code and it's possibly going to run in multiple processes or it's gonna start and then restart, start and stop and then run again and you wanted to not have to recompute everything, this cache, I guess.

03:17 - Evictions are evictions like on hold for 2020.

03:21 - Yeah, well, because of COVID, you're gonna need more dissipate now, just kidding.

03:25 (laughing)

03:27 - Now this looks cool.

03:27 So what's, one of the things I was confused about is it's a key, it's called the cat disk cache, but what's the difference between that and just like a key value store database?

03:37 - Well, those, the key value store database in practice would be no different.

03:41 So, but you have a set, well, but you have a separate server.

03:46 Like there is a server process that runs somewhere that you have to have a, like a connection string and stuff too, that you talk to it in this way.

03:52 This is like, I have a file and I use the same API to talk to it.

03:56 So instead of having another server to manage, another place to run it, you just say like, let me just put it on the SSD and that's probably quite fast.

04:03 - Cool, yeah.

04:04 - And then we got a quick question here.

04:06 Brandon asks, do they talk about any way to scale this out, say multiple servers behind a load balancer?

04:11 I did not see anything.

04:12 I'm pretty sure as far as I can tell that it's local, just like sort of a per machine type of thing, not a, but it does go across processes, but it doesn't, I haven't seen anything talking about multi-machine.

04:24 I guess you could set up a, like a microservice, but at that point you might as well just have Redis.

04:28 - Yeah, yeah.

04:29 Redis is kind of on my list of things to try here pretty soon too.

04:33 - Yeah, absolutely.

04:35 - Another thing I want to check out is some of the, well, I like Tommel lately.

04:41 - Yeah, Tommel's great.

04:42 - That was great.

04:43 - I heard that it reached 1.0.

04:45 - Yeah, so it's at 1.0 now.

04:48 And I think that they were kind of headed there anyway.

04:53 So I was looking through the changelog.

04:55 Looks like they had several release candidates.

04:58 And anyway, we'll talk about it a little bit.

05:01 So it's at 1.0 now.

05:04 I mean, a lot of us don't really understand, maybe I'm speaking for myself, don't really get what all the specification means.

05:11 I just use it.

05:12 It just works. It's easy.

05:14 And one of the things I use it for is the pyproject.toml file.

05:18 It's mostly what I use it for.

05:20 But pyproject.toml is taking off, and this is at 1.0, so what does this mean?

05:26 I'm hoping that this means that we have like a Python package built into the Python that parses toml.

05:33 - Yeah, now the language is stable, right?

05:35 - Yeah.

05:36 - Maybe it means I need to learn more about toml.

05:38 - Maybe.

05:40 But I think there's talk about it.

05:41 I'm not sure what the state of it is.

05:43 Maybe we could get Brett or somebody to talk about it.

05:46 But in the meantime, if you want to play with 1.0, with Python, I think there might be limited choices.

05:54 So I went out and looked.

05:55 There's a page on the project page that shows, it's like down at the bottom, it shows the different projects that implement the various versions of TOML.

06:07 And there's one project, So there's a C++ project that--

06:11 or a handful of C++ that support the 1.0.0, the most recent version of TOML.

06:18 And then various support levels for different other things.

06:23 There's a 1.0.0 release candidate one that's supported by TOMLkit.

06:28 So TOMLkit is a Python project that looks--

06:31 and I think that that might be sufficient to try out most of the features, the new features.

06:36 And then--

06:36 >>Oh, nice.

06:38 Then there's what I would think of as just the Toml project in Python.

06:42 That one, it supports 0.5.0.

06:46 So I'm not sure what's going on there.

06:48 It'd be great if it would support the latest.

06:51 But then I'm like, what does that mean?

06:52 What's different between 0.5.0 and 1.0?

06:56 And so I went and looked at the changelog.

06:58 There's three things that jump out that look like they're new, really changes.

07:03 One of them is leading zeros in exponent parts of floats are permitted.

07:07 So, okay.

07:09 Then allowing raw character tabs in basic strings and multi-line basic strings, that seems reasonable.

07:16 And then the difficult one might be allowing heterogeneous values in arrays, which that's cool.

07:24 And then, yeah, so apparently it wasn't there before.

07:27 - Yeah, but none of those seem like super common stuff that's gonna be a big breaking change.

07:32 Like, oh, well, of course we use heterogeneous types in here.

07:34 Like we're just gonna mix it up and have random stuff in our array, right?

07:37 It seems like it's probably still the built-in or the pure Python one is probably decent still.

07:43 - Right, and I need the, I guess, there's a whole bunch of these that are listed as clarify, like clarifies, but it is a specification, so clarify might be very important, but I'm not sure how important that is.

07:55 It probably affects the implementation, but I'm putting this out because I'd like to hear from people that know more than I do about this and how this affects Python and if we should care about it.

08:06 - Yeah, yeah, for sure.

08:07 That's very cool to see it coming along and it definitely lends some support to the whole PyProject, Toml stuff.

08:12 Yeah, hey, before we move on to Augie's first topic, Martin Boris asks, "Just wondering, is this disk cache thing I mentioned, "is it a simple way to share data "between uvicorn and gunicorn workers?" Yes, exactly.

08:24 That's exactly why it matters because it goes across the worker processes or across worker process in general, across multi-processes and a consequence of multiple worker processes.

08:34 'Cause normally you would either cache in process memory, so you gotta do it like 10 times, you've got it all fanned out, different processes running, so this will solve that for sure.

08:42 And then one for you, Brian, for Magnus Carlsen.

08:45 - Yeah, does pep621--

08:48 - The Toml spec, whatever the pep is for that.

08:50 - Specify the version of Toml to use?

08:52 I don't know, I'll have to ask Brett about that too.

08:55 - Yeah, I don't know either, sorry.

08:56 All right, Augie, what you got?

08:57 - Well, I'm here, well, thank you for inviting me again.

09:01 This is actually, you have two consecutive weeks of hosting mechanical engineers as your guest on the podcast.

09:07 - Why not?

09:08 - So thanks for being inclusive.

09:12 But I wanted to talk about PyQt Graph, which is not new, but it's--

09:17 - Yeah, people maybe don't know though, so tell them about it.

09:19 - Yeah, absolutely.

09:20 So PyQt Graph is a plotting library, but it's a little different from the likes of Matplotlib and the variants or derivatives from that or a bouquet.

09:32 YQT Graph uses the Qt framework and it's meant for embedding interactive plots within GUI applications.

09:42 And as a consequence of using it as the Qt, you can actually get some really high performance out of it, which is, which Matplotlib is absolutely phenomenal for generating plots for publications or, you know, for static media on websites.

09:58 But the moment you try and do anything like with mouse interactions, you might be in for for a bit of a tough time.

10:03 - But with this you're running on native, with Qt you're running natively on the OS, right?

10:10 - Absolutely, yeah, you're running, yeah, there's no client server relationship like you would get with a bouquet, which you might need in some certain situations.

10:18 But, and anyway, so part of the PyQt Graph library is, which I guess I should identify that I am a maintainer of, but is that we actually bundle an example application.

10:32 So if you're ever curious about the library and its capabilities, you know, and don't feel like reading through dozens of pages of documentation, you can just run this example app, which I have on the screen share, and it shows you the list of various--

10:43 - And this comes with PyQtGraph, right?

10:46 - Yes, yeah, it's bundled in the library.

10:48 So if you pip install PyQtGraph, you get this.

10:50 And here's some of the basic, you know, plots, but, yeah, and as you can see, you get our mouse interactivity going, and we can do zoom behavior.

11:01 But what's really cool about this library is that example here, basic plotting is generating with this code right here.

11:09 All those plots was in this, I can't tell how many lines, maybe 70 lines total.

11:14 >> Yeah.

11:14 >> But anyway, within this editor here, you can change any of the code and experiment with yourself.

11:20 Here on the tab, you see all these different items.

11:23 It does 2D, we have some 3D capability, which you need the PyOpenGL library for.

11:28 This one is just maybe a dozen lines of code, but you have a couple of plots here and then just with the mouse interactivity, we can sub-select, or here you can get our crosshairs and get information about what's the data points underneath the mouse.

11:42 For an analysis tool, it can be incredibly powerful.

11:48 If you're generating tools for any engineering or scientific analysis where you you want the user to be able to interact with the data in some way, zoom in, zoom out, things like that, or a Pikey graph might be a really good option for you.

12:03 - Yeah, absolutely.

12:04 Can you run the basic plotting thing one real quick?

12:07 - Oh yeah, of course.

12:08 - So when I was looking at this, the thing that stood out to me was while it looks, like the graphs are beautiful and they look good, the first couple of layers, like I could probably do that in bokeh or plot layer, map plot layer, something like that, right?

12:22 but the nice interaction between multiple graphs as you zoom in one, the other goes in, or that super high frequency yellow one that's people listening, it's like refreshing, you know, many, many times a second, right?

12:34 Getting high frame rates out of those like Jupyter notebooks sounds tricky.

12:39 - Yeah, and I'm actually really glad you brought up high frame rates.

12:42 I'm actually on the verge of merging a pull request to integrate a Coupy support, which is the CUDA number arrays for some of the image data and on some of our benchmarks we're showing being able to go from maybe 20 frames per second of images up to over 150 frames per second, which at that point monitors can't keep up, but you lessen the CPU load substantially.

13:08 - Yeah, that's fantastic.

13:09 We got a comment question from the Anthony Shah.

13:13 - I use the built-in grapher app in macOS.

13:20 I do not know what the built-in grapher app is.

13:23 So I am afraid I don't know how to answer that.

13:27 - You don't know if it can replace or not.

13:29 I don't know either, but yeah.

13:30 - But PyQt Graph has a couple dependencies.

13:35 You need some Qt bindings, and right now we support Qt 5, 5.12 and newer.

13:41 Up until very recently, PyQt Graph supported virtually any Qt bindings you could install, even going back a decade, which eventually I had to put an ax to that.

13:50 That was just too much work.

13:52 And so we support Qt 5.12 or newer.

13:58 We don't support Qt 6 yet, although there is a pull request in to add support for PySide 6, which was discussed on this show just two weeks ago.

14:07 - It just came out, right?

14:08 - Right, it just came out, and which I'm really thankful for contributors, you know, that are submitting these pull requests.

14:15 I often feel bad that I can't keep up with the rate that they're coming in, but it's still appreciated.

14:21 - Are you looking for contributors to the project?

14:24 - Yeah, absolutely.

14:26 Not just contributors to the code, but also people that are willing to look over pull requests or willing to test out pull requests mainly.

14:35 With the plotting library, sometimes testing can be really difficult 'cause like visual artifacts, like how do I test for that, right?

14:45 And so sometimes a big chunk of our testing is, well, does this break or does this look right?

14:51 And having somebody else verify that kind of stuff is a really big help.

15:00 So if you're interested in this, and feel free to reach out to me directly or take a look at our issue tracker or pull request tracker.

15:12 Yeah, and I guess the last thing I should say is it's primarily used in scientific and engineering applications.

15:18 It's periodically I go through to get log and I look at like the email addresses that people are contributing to and NASA Ames Research Center and a bunch of places like that.

15:29 But I get a kick out of that.

15:33 - Yeah, that's super, super cool.

15:34 Nice, thanks for sharing that and good work on it.

15:36 - Well, another cool thing is Linode and they're sponsoring this episode.

15:40 Thank you, Linode.

15:41 Simplify your infrastructure cut your cloud bills in half with Linode's Linux virtual machines. Develop, deploy, and scale your modern applications faster and easier. Whether you're developing a personal project or managing larger workloads, you deserve simple, affordable, and accessible cloud computing solutions.

15:58 As listeners of Python Bytes, you get a $100 free credit. You can find all about those details at pythonbytes.fm/linode. Linode also has data centers around the world with the same simple and and consistent pricing regardless of location.

16:14 Choose the data center nearest to your users.

16:16 You also receive 24/7, 365-day human support with no tiers or handoffs, regardless of your plan size.

16:25 You can choose shared and dedicated compute instances, or you can use your $100 credit on S3-compatible object storage, managed Kubernetes, and more.

16:34 If it runs on Linux, it runs on Linode.

16:38 Visit pythonbytes.fm/linode and click on the create free account button to get started.

16:45 - Awesome, thanks for supporting the show, Linode.

16:47 Ah, okay, Brian, I wanna cover something that comes to us from two listeners.

16:51 This comes from Jim Kring, who pointed out some really interesting aspects how Python is being used in this whole parlor social media kerfuffle, and a great article by my good friend and fellow Portlander, Mark Little.

17:07 So let's go over the article first.

17:10 So you guys heard there was basically an attempt to overthrow the US government, do you guys hear that?

17:14 That was lovely, god, what idiots.

17:17 So a lot of the people who were there got kicked off of official social media, and they went to this site called Parler.

17:23 So Parler, according to Wikipedia, is an American alt-tech micro-blogging and social media networking service, and it has a significant user base of Donald Trump supporters, conservatives, conspiracy theorists, and right-wing extremists.

17:37 Not my words, that's Wikipedia.

17:38 So a lot of the people who stormed the Capitol tried to get into Congress and stop the counting of the votes they decided to live blog it on their personal accounts.

17:49 But a lot of them were no longer on Twitter and whatnot, although some were, so they were on Parler.

17:56 And they probably came to realize, you know, it's probably not a good idea of showing me charging into the Capitol as like hundreds of people are being arrested and charged with federal crimes, right?

18:06 At the same time, Parler was getting kicked off of Apple's App Store for the iOS.

18:12 They were getting kicked off of the Google Play Store.

18:14 They were getting banned in a lot of places.

18:16 So there was this, hacker's not the right person, the sort of data savior person, I guess you could say, who came along and realized it would be great if we could download all of that content and save it and hand it over to journalists at say like ProPublica, hand it over to the FBI and so on.

18:32 It turns out it wasn't very hard to do.

18:34 There was a couple of things, if you look through the Ars Technica article about how the code behind Parler was a coding mess.

18:43 And I've tried to figure out what technology was used to implement it, and I just couldn't find that anywhere.

18:47 Anyway, it says, "The reason this woman was so successful "at grabbing all this data," which she got like 1 million videos and a whole bunch of pictures, there's a whole host of mistakes.

18:59 So the public API for it used no authentication.

19:03 Let me rephrase that, restate that.

19:05 The public API is zero authentication, no rate limiting, nothing.

19:09 Just, yeah, sure, we'll just, go ahead.

19:10 - There you go.

19:11 (laughing)

19:12 - You have it all.

19:13 Secondly, when a user deleted their post, the site didn't remove it, it just flagged it as deleted so it would show up in the feed, which in and of itself is not necessarily bad, but you pair that with every post was an auto-incrementing ID, which meant you could just enumerate.

19:28 You're like, oh, I'm on post 500.

19:30 Well, let's see what 501 is.

19:31 It doesn't matter if it's deleted, give me that.

19:34 That's crazy, right?

19:35 So she wrote a script in Python to go download it.

19:40 And you can actually see like, here's all the videos and all the stuff and their IDs and whatnot.

19:45 And in here, this is the one that Jim sent over.

19:48 If you look, there's a gist here that shows you how do you download a video from Parler?

19:53 Let's go down and find, is it here?

19:55 No, maybe it's not there.

19:56 I think it might be back.

19:57 There's a part where it shows the, how do you download it with Python and so on.

20:02 So you just go through and like, you know, screen scrape it traditional Python right there.

20:06 So apparently Python was used to free and capture all of this.

20:10 Oh, another thing that they did in Parler that made it easy to get was when you upload videos and images to places like Twitter, they'll auto strip the exif, like the geolocation and whatnot from the images.

20:23 Now they don't need it, just post it, right?

20:24 So like geolocation, camera name, all that kind of stuff is all in there.

20:30 So there's just a bunch of badness.

20:33 They've been since kicked off of AWS, because you know, crimes.

20:37 And now they're apparently trying to get hosted in a server in Russia, is that right, Augie?

20:43 - Yeah, there was a, actually I think there's an article on Ars Technica that went up this morning that they're somewhat partially online on some Russian infrastructure, which--

20:53 - Yeah, they're only partially online, because I looked and it, they're like, it says something like, well, we're trying to come back.

20:59 Here's a couple of posts.

21:02 It's not, yeah, it's not all the way back, right?

21:04 They're experiencing technical difficulties, as in the world hates them and is trying to make them go away.

21:10 So I'm not here to try to make this a political statement or anything like that.

21:14 That's not why I covered the story.

21:15 I covered it 'cause I thought it's very interesting, both the security side and how people were able to leverage Python to sort of grab this stuff before it's gone.

21:23 Some of the journalists were asking, like, is there a more accessible way to get the data?

21:27 they're like, yes, we're going to build, the woman who got it is like, we're going to build some better way for you to get it.

21:32 But right now, it's like I had to run into the burning building and grab the files before they were gone.

21:36 >> Yeah. The other thing I want to point out about the story is it's not like Parler was lacking funding to develop these schools.

21:43 They had, from what I understand, they had significant financial backing.

21:47 >> Yeah.

21:48 >> Whether they did not have the technical expertise, the time, I don't know.

21:52 But I'm really curious as more fallout comes from this.

21:57 There's going to be some good stories from a technical standpoint on here.

22:02 Absolutely.

22:03 Well, pretty, pretty insane.

22:04 All right, Brian, let's move on to something more devy, developer, web devy.

22:09 Well, you know, maybe if you want to scrape the web or something else.

22:13 Absolutely.

22:14 Yeah, we've got a suggestion from Douglas Nichols.

22:18 Thanks, Douglas.

22:20 Best of the web development with Python.

22:22 So we've seen...

22:23 I would put Parler not in that list.

22:26 So we've seen the best of lists like this before.

22:30 I'm kind of a fan of them.

22:32 But one of the things I liked about this is the icons are nice.

22:37 So there's a whole bunch of different icons that are used to help.

22:40 You can see the likes or the lows and stuff of different projects.

22:45 And then there's icons for--

22:46 you can search for Flask projects or things like that.

22:49 That's nice.

22:50 But it's a pretty big comprehensive list.

22:53 We've got web frameworks, HTTP client servers, authorization tools, URL utilities, open API, GraphQL, which is nice to see.

23:03 There's even web testing and markdown listed, how to access third-party APIs.

23:09 But then near the end, I really liked seeing there's a bunch of utilities sections.

23:14 So there's a Flask utilities and FastAPI and Pyramid and Django utilities, which are really neat.

23:20 And what I really was pleased to see was that even though FastAPI is, what, a couple years old now?

23:26 There's a whole bunch of FastAPI projects that are there to make FastAPI easier, like using SQLAlchemy or coming up with the contributions thing or--

23:39 Yeah, fantastic.

23:40 Different React, how to use React with it, things like that.

23:43 So yeah, nice if you're trying to check out--

23:46 want to look at different tools that are available for web development with Python.

23:51 This might be a good place to peruse.

23:53 I feel like that's one of the big challenges in general, you know, with people coming into Python or getting into new framework, it's like there's 500 libraries to do a thing.

24:02 Yes.

24:03 Which one should I use?

24:04 Not, can I find a library, but there's too many, right?

24:06 Yeah.

24:07 Yeah.

24:08 So do you have a suggestion for that?

24:09 Well, I think these awesome lists are super good, right?

24:11 Because they're somewhat vetted and whatnot.

24:14 I recommend, so like, for instance, if I was, if I was building a, well, it's harder now, But if I was building something new with the web development or web interface or something, and I didn't have which framework to pick is one of the starter things.

24:28 It's the people I have around me as resources.

24:31 I know that you know about Pyramid, but you're also fairly knowledgeable about FastAPI.

24:40 I know some people that are Django friendly and know quite a bit about Django.

24:45 If you've got a couple of friends that already know one of these big hitters, I would go with that so that you can ask them questions.

24:53 - Well, maybe even you don't pick the same thing, but you could ask, like, you chose this one.

24:58 Tell me, you looked at a lot of the other ones.

24:59 Why did you pick that?

25:01 Yeah.

25:01 - Oh, yeah, yeah, that's a good idea.

25:03 - Yeah, for sure.

25:04 Like maybe FastAPI makes sense for me, it doesn't make sense for you, but you can then see why it made sense for me and not for you or whatever.

25:09 - Yeah.

25:10 - Yeah, absolutely.

25:11 All right.

25:12 - All right, I up now?

25:14 - Yeah, you're up.

25:14 - So, Mr. Shah being in the audience here is a bit of a surprise, but one of the things I wanted to talk about is, I'm going to butcher this.

25:24 I pro I apologize.

25:25 pigeon.

25:27 I think it's pigeon.

25:28 Oh my goodness.

25:30 Not.

25:31 Okay.

25:31 Yes.

25:32 What a wonderful name.

25:33 And, in, I've been fascinated by this.

25:38 and so what a pigeon is, is, this is feel so awkward to talk about somebody else's project when they're in the audience here.

25:47 - It's a JIT extension of CPython that compiles Python code using the .NET 5 CLR.

25:54 And what's been fascinating to me about this is this is like a whole area of software that I have absolutely no experience with.

26:04 Like I know nothing about, but I've been following what Anthony's been talking about on Twitter about it, and he's been explaining what he's doing, explain, you know, along the way in these Twitter size increments that I feel like I'm able to follow along with the attendant and I found that this project absolutely fascinating and I'm seeing like the rates of improvement over time and I've just been absolutely blown away.

26:32 And so I think this has been absolutely amazing and I really hope that, I'm really curious. So One of the benchmarks that Anthony's been using is his own Python implementation of the nbody problem, which is, which is sort of funny that it's come up 'cause I've been wanting to do an nbody plotting example in PyQt Graph, and so now, and of course, this has been sort of on my to-do for some time, so now I'm curious if I should even attempt to, or if it's even remotely possible to try and integrate those functionalities together.

27:08 - Yeah, that's cool.

27:10 And go ahead.

27:11 - No, no, go ahead.

27:12 - Oh, sorry.

27:13 The other things that I've, the other thing that I've recently used for some extension or made use of, but is not particularly new, is the numpy's underscore or dunder array function, functionality, which is specified in nef18.

27:32 And what that allows for is using numpy methods on not necessarily NumPy arrays.

27:43 So for example, with Coupy, you can use like the NumPy methods that would operate on the array, but use it on a Coupy array.

27:53 And this is not limited to Coupy.

27:55 There's other libraries that offer this functionality too, but this makes it so much easier to integrate various libraries together with really having minimal code impact and having near identical APIs.

28:12 And earlier I was talking about the pull request for giving Koopai support into PyQt Graph and this functionality, which was implemented in Koopai, but it's made the integration so much easier.

28:24 - Nice, 'cause you guys are already implemented with NumPy and it's just like, we're just gonna go through this layer basically.

28:28 - Yeah, I mean, there's some other gotchas that you have to have, right, with handing stuff off to the GPU and stuff like that.

28:34 "Yeah, no, that's..." But the actual size of the diff was not that big, you know, for what you would think.

28:41 - Well, and you think what it means to run on a CPU or run on a GPU.

28:45 Like, that's a very different whole set of computing and assumptions and environments and, right, and so on.

28:53 And to make that a very small merge is crazy.

28:56 - Right, yeah, no, it's fantastic.

29:00 Yeah, as I said, it's nothing new.

29:02 This functionality has existed, is been enabled by default in NumPy since version 1.17, which I believe is almost coming up on two years old now.

29:12 But this is the first time I've made use of this functionality or I've been impacted by this functionality directly, and I'm so appreciative of it.

29:19 - Yeah, fantastic.

29:21 And that's super cool.

29:22 I've not really found a reason for me to work with Coupy or anything like that, but just really excited about the possibilities for people for who it does matter, you know?

29:31 How about you, Brian? - Yeah, absolutely.

29:32 Yeah, I actually, I always, every time I hear about it, I write a note down and say, "Oh, I got to check this out. Looks neat." Absolutely.

29:40 Well, there we go. There's our six items. Do you have anything extra for us, Michael?

29:45 This almost could be an extra, extra, extra, extra. You're all about it. So I'm just going to throw a few things out really quick. One, I got my new M1 not long ago and actually had to send in my old laptop. Its battery was dying, its motherboard was dying, all sorts of things.

30:00 So I had to put it in a box and send it away.

30:02 I'm like, I don't really want to put my data in here.

30:05 So I just formatted that as well.

30:06 So now I have two brand new computers.

30:08 I'm trying to think like, all right, what kind of getting bugged by how much spying, monitoring, observation, all these different companies are doing.

30:15 So I've started running just Firefox, but also, you know, when things a lot of times, like for example, StreamYard, I can't use a green screen on Firefox.

30:22 I have to use Chrome, it says.

30:24 I'm like, I don't really want to use Chrome, but I want a green screen, so here I am.

30:28 So I've started using Brave.

30:29 Whenever something says I have to have Chrome, I started using Brave, which is a more privacy-focused browser.

30:34 So I thought that was interesting.

30:35 And just turning on a VPN like all the time, just to limit people observing, not that I really need to keep anything super secret.

30:43 Two conferences are coming out with calls for proposals that are due quite soon.

30:48 So the Python Web Conf got some calls for proposal.

30:53 The conference is actually March 24th.

30:57 That order is not quite right.

30:59 Is it 22nd to 26th?

31:01 if you, if you look at their site, like the, the days that it's on are like sort of not in order anyway.

31:07 end of March, there's a cool online conference.

31:09 They did this last year, six feet up did, and they're doing it again this year.

31:13 I'm actually speaking here.

31:15 Brian, are you speaking there?

31:16 The webcom?

31:17 Yeah.

31:17 No.

31:18 Well, there's a call for paper, so you could be on YouTube.

31:20 Yeah.

31:22 And I think they expanded it out to be like five days or something.

31:24 So there'll be a lot of content, which is very cool.

31:26 So I'll be giving a talk on Python memory deep dive there, I believe.

31:30 And then the big one, PyCon.

31:32 PyCon is virtual again this year, but the call for proposals has gone out and is they're due February 12th.

31:38 So if you want to be part of PyCon, you know, get out there and send something in.

31:42 Are you going to submit something?

31:44 I will probably do it.

31:45 Yeah.

31:45 it means I got more work to do, but yeah, I think I'll do it.

31:49 you got any plans?

31:51 I'll probably submit some something, maybe three, four, five, six, seven, eight, nine, - Yeah.

31:55 - 10 proposals.

31:56 (laughing)

31:57 - The more you submit, the better chances you got.

31:59 Augie, you gonna submit to eithers?

32:01 - There's talk amongst us PyKitty Graph maintainers about doing a tutorial session at SciPy.

32:08 So I might, no, that's not listed here, but we're considering doing that, which SciPy is also virtual this year.

32:14 - Oh yeah, that'd make a lot of sense.

32:15 Yeah, that's cool, awesome.

32:16 Then final here, here, here, all about it, extra stuff, is Apple's launching a racial equity and justice initiative, which I think is pretty cool.

32:25 Basically, they're setting up centers to teach programming and other entrepreneurship skills in underserved communities, right?

32:33 And I know there's, again, a lot of political stuff around all this, but to me, I would just love to be in a world where I look around the community and it looks representative of everybody, right?

32:42 People feel included.

32:43 Like, tech is such a wonderful space.

32:44 I think this is a cool initiative.

32:47 Obviously, it could be, hopefully they deliver it in the right way.

32:50 It's not just like, "We're gonna teach everyone how to build iPhone apps." That's what the world is, right?

32:54 You know, it's a more broad sort of conversation.

32:56 I could go any which way.

32:58 And hopefully it's just a start.

32:59 Like if you look, they're saying they're donating a hundred million dollars to this cause, which is a lot of money, but it's also only eight hours of profit to Apple.

33:07 So yeah, it's got room to grow, I suppose.

33:09 Anyway, I just want to give a shout out to that as well.

33:12 That seemed pretty cool.

33:13 Hi, Brian, how about you?

33:14 More conference stuff?

33:15 - Well, Pi Cascades is, actually I don't remember when it is, but-

33:20 - February possibly.

33:20 - February, probably.

33:22 - Yep, February 20th, it starts.

33:24 And there is the schedule's up.

33:27 So I wanted to announce the schedule's there so you can check it out.

33:30 There's still tickets available and you can see what's gonna happen.

33:34 I really had fun at the in-person PyCascades and I think they did a good job for the online one in 2020.

33:42 So, and we're gonna be there.

33:44 - Yeah, we are.

33:44 We're on a panel.

33:45 - Yeah.

33:46 - Along with Ollie Spittel.

33:47 - Yeah, should be fun.

33:48 But there's--

33:49 - It should definitely be fun.

33:49 - About podcasting, but there's like another panel about writing technical books that looks good.

33:55 There's a bunch of cool talks that I'm looking forward to seeing.

33:58 - Yeah, me too.

33:59 It looks great.

33:59 I love all these online conferences that it's pretty accessible to everybody.

34:03 Last year, if we would announce this, be like, "Oh, well, I'm not in Portland, "so it doesn't matter to me." - Yeah. - Right, but.

34:08 Augie, I know you got some stuff to shout out real quick, but also a quick question, a follow-up from Anthony.

34:15 "Why is there AVX extensions "for native matrix multiplication on supported CPUs?

34:20 It'd be interesting if that extension supported the same for non NumPy arrays.

34:24 Thoughts, ideas?

34:25 Yeah, I...

34:26 The...

34:27 Yes, I'm sure you can use those extensions on...

34:30 I mean, NumPy doesn't have a monopoly on AVX extensions.

34:34 You know, it just needs...

34:35 Whatever library you use, I think it just need to...

34:37 Or it would need to be compiled with the Intel MKL BLAS extension, which is...

34:45 Goes into build systems, which is way over my head.

34:48 and, I, yeah, I used to live in the C++ world and whatnot, but I'm far from that world that you and Anthony are inhabiting these days.

34:57 Right.

34:58 So, yeah, I'm, I'm so yeah, short, I'm not sure.

35:02 but in terms of the extras, a couple of things I wanted to bring attention to is, I've been loving the Anthony explains video series and these are generated by, I'm gonna mispronounce his last name.

35:13 Anthony's style, he's been a guest on, can't remember if he's been a guest here, but I think he's been a guest on Talk Python to me.

35:20 He maintains pre-commit, he's a pytest developer and maintains-- - Anthony Sotili.

35:24 - Sotili. - Yeah.

35:26 - And I've been absolutely loving his Anthony Explains playlist series.

35:32 The other resource that I've recently found myself having to make use of is learn X and Y minutes.

35:37 Sometimes I have to write something in a text stack or in a language I have absolutely no familiarity with.

35:45 And so that resource has been absolutely amazing for the five minute overview, right?

35:52 On the real basic operations.

35:53 And then the other one is this book I've been reading, working in public.

35:58 And I think Guido plugged it a while ago on his Twitter feed, but it talks about maintaining open source projects and some of the issues arising that I think it's, I'm still not done with it, but I think it's both helpful from a maintainer point of view to, you know, for sanity checking your experiences might not be as isolated.

36:22 And I think it's helpful for new open source contributors to see what things might look like from the maintainer's perspective as well.

36:29 - I love that book. - Yeah, I've heard really good things.

36:31 Yeah, have you read it, Brian?

36:32 - It has an audio book version, so I listened to it.

36:34 And you wouldn't think like a book on open source would be good audio, but it was great.

36:39 - Yeah, fantastic.

36:40 Yeah, awesome.

36:41 All right, well, Brian, should we do a joke?

36:43 - Yes, we should.

36:44 - All right, so I put two jokes into the show notes.

36:47 One of them is a rap song, which I know Brian's especially fond of.

36:51 It's a rap song about working at the help desk.

36:55 So if you're the help desk for your company or I guess public support as well, it's probably a dual called "Here to Help." And man, it is so funny.

37:03 It's a video song on YouTube, so it doesn't really make sense to cover it, I thought I'd throw it in there as a pre-recommendation, what I'm going to actually talk about.

37:13 Augie, what do you think? I see you smiling.

37:14 >> I have to say that song was just jam after jam after jam.

37:19 >> It is. I need you to click your right mouse button.

37:23 I only have one mouth.

37:26 So here's the actual Python related joke for us.

37:31 It's a tech support thing.

37:33 Brian, why don't you be the person that needs some help?

37:36 >> Okay. Hi.

37:37 This is a chat, by the way.

37:38 Tech support, how may I help you?

37:40 Hi, I've got a problem.

37:41 Your program is telling me to get a pet snake.

37:44 I don't want one.

37:45 Excuse me?

37:46 It's giving me a message telling me I need a snake to run it.

37:49 Okay, read the message to me, please.

37:52 Python required to run the script.

37:54 That's terrible.

37:56 That is terrible.

37:57 Terribly good is what it is.

37:58 Yeah.

37:59 So, hey, I wanted to add some humor as well.

38:03 All right, do it.

38:04 So I saw this on Twitter and it was a quote from from, how do I, I don't know how to pronounce that name.

38:10 Byron Brian, I'm Byron.

38:12 I don't know.

38:12 A quote from Byron Hobart running a successful open source project is just goodwill hunting in reverse where you start out as a respected genius and you end up being a janitor who gets into fights.

38:24 Yeah, that's awesome.

38:26 And it goes right along with the book recommendation as well.

38:28 Well, that that's a good way to put a cap in.

38:32 Yep.

38:33 All right.

38:33 Well, thank you, Brian. Thank you, Augie.

38:35 Thank you for having me.