#435: Stop with .folders in my ~/

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 435, recorded June 9th, 2025.

00:10 And I am Brian Okken.

00:11 And I'm Michael Kennedy.

00:13 And this episode is sponsored by us.

00:15 So please check out all the awesome courses at Talk Python Training and lots of different options for learning pytest at pythontest.com.

00:26 So check that out.

00:26 And also thank you to our Patreon supporters.

00:28 They've been sticking around for a long time, and we really appreciate them.

00:31 If you'd like to connect with us, more about this later, but if you'd like to connect with us, please check out the links in the show notes.

00:40 Links to both Blue Sky and Mastodon are there.

00:45 And if you would like to join us live, you can head on over to pythonbytes.fm/live, and it'll show you how to do that.

00:53 There's a link at the top.

00:54 It's really easy.

00:56 And then also, when you're listening and you want to check out all the cool things that we talk about but don't want to write them down, no worries.

01:04 Just go ahead and sign up to be part of the newsletter, and we'll send those directly to your inbox.

01:11 So with that, do you have a cool item for us, Michael?

01:16 I do.

01:17 I have some stuff I want to talk about.

01:20 It's an update on things I've spoken about before, but it could have been six years ago.

01:25 I can't remember when we covered this and I for some reason didn't do a search.

01:29 So here's the deal.

01:30 You are writing an application and you're doing things like, hey, I need to have my app save some content in a file.

01:38 I don't want the user to access it.

01:40 I want my app to be able to get to it.

01:41 And that's pretty much it.

01:42 Like maybe a SQLite DB file, right?

01:45 That's not a thing that the users should be tracking down.

01:47 You don't want to put it in their My Documents on Windows or somewhere in the user profile on Mac or Linux or something like that, right?

01:55 You need to save it somewhere.

01:56 It needs to be associated with the user.

01:58 They need to have permissions right there.

01:59 So I previously talked about appdurs, the appdurs package, a small Python module for determining appropriate platform-specific directories like user directories or stuff like that, right?

02:11 So where's your config?

02:13 Where's your cache directory?

02:14 So on macOS, that's like tilde slash capital library, capital L library, cache.

02:21 Then you probably use app name, something like that.

02:23 Those kinds of things, right?

02:24 You want to get access to that and put things in there.

02:27 And it drives me nuts, Brian.

02:29 I go to my user profile.

02:31 If I want to look at hidden files, there's a ton of dot this app, dot that app settings in my user profile.

02:39 And they're not there for me.

02:40 They're there because the designers of those apps were too lazy, usually, too lazy to put that into the appropriate location on that platform, right?

02:50 There's some that maybe I should mess with.

02:51 Like, for example, OhMyZShell, you might want to go and tweak like your prompt, how that looks or whatever.

02:57 And right, you would get to that from there.

02:59 But a lot of things, they just do it because like, well, we'll just do a dot because it's hidden.

03:03 People won't see that.

03:04 Just drop it right in there.

03:05 So I implore people listening to put stuff in the right place or a better place.

03:11 Well, okay, I'm guilty of this, but I don't really think about Windows too much.

03:15 I just think about like in a Linux-y environment, the home directory is usually the right place.

03:21 well I well maybe I mean let me see what I got in here I don't think so but

03:25 let me do an an A um what do I do my hidden ones there you know I do it this way hold on I do it this way so for example um

03:33 well I'm probably doing it the wrong way there too so you're probably maybe but

03:36 there's like studio 3t which is a MongoDB GUI management tool it has all of its settings in here is that something that I should be working with should I be seeing this probably not it's like a dot 3t folder and it's got like

03:49 roaming history and crap in there i've got dot android for whenever i have that android sdk doing something

03:56 i've got um i mean i can see like i've got a dot aws maybe it's got things in there right but i have a dot matplotlib for real there's a dot matplotlib in here why why is it here i don't know there's a pip audit cache literally it's called cache should i see the cache. No, the cache is not for reading. It should be in, there's literally a cache folder on all these platforms where it goes. And maybe on Linux, it literally is just like there, but okay, fine. But on the other ones, it shouldn't be, right? Anyway,

04:25 maybe I'm ranting a bit hard on this, but you should probably do these correct things. And I wasn't even thinking about Windows, although Windows does have a different set of places things go, right? So this thing was great and I recommended appdirs.

04:39 But if you go up a little bit, it says, note, this project has officially been deprecated.

04:45 Oh, no.

04:46 No.

04:46 We were so close.

04:49 So it says, you may want to check out platformdirs, which is my recommendation.

04:53 So moving over to platformdirs, this is under toxdev, T-O-X-dev.

04:59 So full group there.

05:01 And it's a small Python module for determining the appropriate platform-specific directories, such as I just said.

05:07 However, it's better in some ways.

05:10 So it has, I don't know if you noticed before, but there's maybe three or four directories on the app doors.

05:14 This one has documents, downloads, pictures, videos, music, desktop, runtime, et cetera, et cetera.

05:19 Lots of different places that you might want to go to depending on what platform you're on, right?

05:26 It has a better typing.

05:28 It has been rewritten to use path as in path objects from pathlib and all that.

05:34 It's got CI testing against all the different platforms that it supports.

05:39 So very, very cool that this is not only continues to live, but it's actually gotten much better.

05:45 Yeah, and the TOX development group is going to keep it updated.

05:49 So cool.

05:50 Yeah, exactly.

05:51 It can't require that much support over time, right?

05:55 These things are not super dynamic, but I do really appreciate them.

05:59 Modernizing it, you know, I think that some of the changes were that basically the app just got out of sync with Python rather than out of sync with the platforms, right?

06:08 It was just like, well, it doesn't have types.

06:10 And so now we got, you know, and there's just a bunch of stuff like that.

06:12 So I'm very happy to see it and recommend platform DERS.

06:15 I already used it to massive success on some projects.

06:18 I was like, I want to use this and some cool caching things.

06:21 And it made my app so much better.

06:22 And guess what?

06:23 There's not yet another.Michael's app cache in my tilde slash.

06:28 Over to you.

06:29 I'll have to do this on some of my applications.

06:32 So anyway.

06:33 It's worth considering.

06:34 So I'm glad you brought it up.

06:35 All right.

06:37 I'd like to actually want to do a shout out to Bob Belderbos because he posted, I think he posted on several platforms, but I saw it on LinkedIn.

06:48 It says, I love make files.

06:49 They save me time and help my team work in a more uniform way.

06:54 I'd love to be able to say this.

06:55 I love make files also.

06:56 Actually, I don't love them.

06:58 I'm just used to them.

06:59 So I throw in a make file into something that makes sense.

07:02 But for a very long time, nobody else on my projects have been familiar with makefiles.

07:07 So I can't use them.

07:07 That would be me.

07:08 I'd be like, what's a makefile?

07:09 Why do we have this?

07:11 That would be in Python.

07:12 So some Python projects I use, I use Tox because you can do kind of some similar stuff with Tox.

07:18 But it is annoying that it like creates a virtual environment.

07:20 It's better with uv now because it's fast.

07:22 But still, there is, so there's a new kid on the block, or at least new to me, called, that Bob has introduced.

07:31 And it's called Poe the Poet.

07:33 And I was a little leery because I thought it maybe was too tied to the poetry project.

07:41 But even if it is, you don't have to use it with it.

07:43 I'm using it without it, of course.

07:46 But anyway, so the thing I love about this is, as an example, I'm leaving this on Bob's post because it's great.

07:53 In your pyproject.toml, that's where you define the tasks.

07:57 And this is great because I don't have to have an extra thing, an extra file in my project.

08:02 This makes sense to just put it there.

08:05 So here's an example of like, so a test action would be, you can just define it as pytest or something.

08:12 Or Cove, you can have pytest running with coverage.

08:17 The linter is rough, which is, it's funny, people still do this, but actually I remember rough better than I remember lint anymore, but you know, it's just me.

08:27 But anyway, some cool type.

08:31 Oh, he has tie check for the ty.

08:34 What?

08:35 Wow, he's on top of that.

08:37 That is super new.

08:38 Integrates well with uv.

08:39 I tried it out.

08:40 It is pretty slick.

08:41 So the Poet the Poet documentation says batteries included task runner that works well with poetry or with uv.

08:50 I'm definitely using it with uv.

08:53 There's a bunch of cool things how to define this in the documentation.

08:59 It's pretty good.

09:00 So you can, like Bob said, you just add a section called tool.po.tasks and then define a bunch of actions.

09:09 There's ways to have built-up actions like you would with a make file or tox.

09:15 Actually, I don't know if you can do this, tox.

09:17 Anyway, you'd have multiple actions that run one after another.

09:21 The sequences is pretty cool.

09:23 You can also add help text to things, which is nice.

09:28 Anyway, but having it right in, I just love it.

09:30 It's right in the PyProject.toml, and it works well with uv.

09:34 Love that.

09:35 So definitely going to switch it for a lot of projects.

09:38 And there's an example.

09:40 I'm not sure where the example is right now, but in the kind of getting started, or maybe it was Bob.

09:46 Anyway, adding it to your development dependencies within your PyProject.toml is slick because then now if you're using, hopefully you're using development dependencies within your Project.toml so that new developers can just install all the tools they need to just by using that.

10:04 So pretty cool.

10:07 Love it.

10:07 Yeah, very cool.

10:08 And by the way, did you catch that the logo of Poe the Poet is Edgar Allan Poe?

10:13 Yeah, he's the poet.

10:15 Very interesting.

10:16 Okay, next up is some data science.

10:19 There's an article I'm linking to on the new stack by Jacob Jackson.

10:27 This is that Pandas 3.0 will significantly boost performance by replacing NumPy with PyArrow as the default engine,

10:35 enabling faster loading and reading of columnar data.

10:40 So since 2.0, I think Pandas has had possible support for PyArrow, which is more of an analytics type backend than NumPy, which is more database-y, like traditional rows, like here's an entry in its internal structure,

10:57 making it much easier to ask questions like, what's the average or the max of this aspect of the data?

11:06 What is the average sales price for these million rows is way faster in a columnar orientation because you just go write down the data as it is.

11:15 It makes it slower for row type of operations, but that's less common for data science.

11:20 It also does much, much faster reading and writing of files and all sorts of things.

11:26 So I recently spoke with Reuben Lerner about the PyArrow revolution on Talk Python not too long ago,

11:33 and that was a lot of fun.

11:34 Basically, we talked about all the benefits of PyArrow and why you might consider it, But the news here is that this will be the default in 3.0 for Pandas.

11:45 When is that coming out?

11:46 Who knows, right?

11:47 Pandas have been around for like many, many, many years, and it's on version 2, so maybe it's going to take, what, another 15 years for version 3 to come out?

11:55 I don't know.

11:56 I think the perception of versions is changing over, you know, we talked about zero and all this stuff, but I think versions are starting to go faster these days, right?

12:07 Django is on five and it was just on four and it's been around a long time, but it should be out pretty soon.

12:12 So we're not exactly sure on the timing. I know some people will know better than me, but

12:15 anyway.

12:16 Also may have been like that just switching out the backend is a big enough change that they decided that's a good reason to switch to a three.

12:24 Yeah, that's a very good point.

12:25 It says over time Pyro is becoming better and better integrated with Pandas, but using it as a backend is still experimental and isn't recommended in production.

12:35 Presumably until 3.0 where it becomes the default in which it would become recommended because you'd have to work around it to know better, right?

12:42 Anyway, that's my item.

12:43 Check out Pyro.

12:44 Check out the conversation I had with Reuben about it to see what's coming for Pandas.

12:49 Cool.

12:50 All right.

12:51 I wanted to talk about data science a little bit as well.

12:56 And I'm going to point to a project called Point Blank.

13:01 It's data validation made beautiful and powerful.

13:05 At least that's the sales pitch or the sub point.

13:08 Anyway, so I don't do a lot of data science, but I really think this is a cool idea.

13:14 Even without data science, any sort of pipeline stuff of checking data within your pipeline makes sense and what to do with it.

13:25 So here's the idea.

13:27 So it says it's a powerful yet elegant data validation framework for Python that transforms how you ensure data quality.

13:34 There's a chainable API that you can use to validate data against comprehensive quality checks.

13:41 And it says visualize through stunning interactive reports, which I haven't looked at the reports yet.

13:46 But anyway, so I'm just at the top example.

13:50 You've got, you import point blank, and then you have these validation features where you can take a data set and validate it.

14:00 In this example, it says, okay, on column D, there, you know, there's validate that the values are greater than 100.

14:09 There's more than 100 values there.

14:11 And then actually on column C, validate that there's less than or equal, the values are less than or equal to five.

14:21 And then there's certain columns exist.

14:26 Make sure the date and the time exist in those columns.

14:28 So you get data frames in, or data sets and data frames, and you've got to be able to make sure that they're in the right shape, the data's in the right place.

14:39 And that's awesome.

14:41 And what do you do if they don't?

14:42 Well, there's a REPL with some validation, Git tabular report.

14:47 That's great for when you're debugging stuff and looking at it.

14:50 But what about in real time?

14:52 You're not going to want to do that.

14:53 And this is something you can use in real time to check all this because it's got these cool actions.

14:58 So there's a bunch of actions as part of this where you can set up callbacks.

15:06 It's got a default one that's built in to notify you on Slack.

15:12 I don't use Slack anymore, but there's a function called one.

15:15 So if there's any failures, you can set up your own notify function, which I love this idea of just, like, give me an API to, like, fill this in, and I can fill that in with whatever I want.

15:26 I can call it rest API.

15:27 I'm going to send myself an email or whatever.

15:29 Yeah, whatever.

15:32 I'll hook it up.

15:33 And so this idea of, like, just run this all the time, and then if there's problems with your data chain, let somebody know about it in real time.

15:41 Love that idea.

15:42 You can even log it.

15:44 Lots of stuff there.

15:45 And looks like the interface is pretty intuitive.

15:48 They've got a roadmap for additional validation methods, advanced logging capabilities, more messaging actions.

15:58 So kind of a neat project.

16:00 I love that it's set up already with a code of conduct and a governance system.

16:06 So pretty promising little project.

16:10 so that's it for that i'm really i like it i like the idea of this a lot yeah

16:14 i like it too i'm not sure um it'd be fun to to to hear somebody that's used it and see um is it something that really can be used in real time to for uh does it slow things down too much i imagine not because you're gonna do a little bit of work just after you've done a lot of work in a few places validate it a few places in the pipeline to make sure everything's still kosher.

16:39 It definitely, you know, data science has the feel a lot of times of I'm going to do a little bit of work to get things set up and I'm going to do some complex calculations on lots of data.

16:49 Yeah.

16:49 Rather than a bunch of tiny little function calls. I think here's a function, a bit of functionality I'm adding to like an API. It would be totally reasonable to say, I want that to be under 10 milliseconds for this API call. No one's going, I need my notebook to process stuff unless than 10 milliseconds.

17:06 Like if you add a little overhead, it's probably fine.

17:09 This certainly looks like it's going to be useful for unit tests as well, right?

17:13 Yeah, yep.

17:14 And checking to make sure.

17:16 Yeah, and that's the sort of thing is like, okay, you've checked all the pieces, but this is, once you put everything together, are we like losing something somewhere?

17:25 Is some part of the process filling in nulls or treating invalid data incorrectly and putting garbage in there or something?

17:35 Yeah, it's super easy to lose pieces of data on some of these things.

17:40 How is this for a tie-in onto my first extra?

17:43 Okay.

17:43 If you're working with pollers or something, you can say, I want to do some calculations, and what you get back is a new data frame.

17:49 By default, what you get back are the columns you've asked for, like the new computer columns.

17:54 But maybe what you intended is, I want to add a column to the existing data frame, not take a data frame and transform it into a new data frame that has just the stuff you asked for.

18:03 So instead of doing an operation, you would say dot width columns, then do the operation.

18:08 And it will keep the original data frame and then add on to it, right?

18:11 You would maybe like that these columns exist test.

18:15 Then you would be able to say, oh, we've lost some of the columns.

18:18 Who forgot the width columns?

18:19 Why would you know to do that, Brian?

18:21 Because you take our Polars for Power Users course that we just released last week.

18:26 Oh, nice.

18:27 Yeah.

18:28 So this is out.

18:29 It's by Chris Trudeau.

18:31 It's a really awesome course, Polarist for Power Users.

18:34 Three hours, almost just under three hours.

18:37 And you can go and learn a whole bunch of cool things, how to work with Polarist.

18:40 And Polarist uses PyArrow underneath the covers or Arrow underneath the covers.

18:44 So you can also get some cool experience with that.

18:47 So people, check this out.

18:48 It's been well-received and continuing to go.

18:51 All right, that's my only extra I have.

18:52 I have a joke, but you got any extras before we do that?

18:54 Oh, I'm loaded with extras this week.

18:57 So let's load them up so there's my point blank.

19:00 I was interested, so I haven't been using Ruff.

19:05 I use Ruff a lot, but I don't tweak it very much.

19:08 I haven't, well, there's a bunch of new rules or not really new, but there's a lot of rules that you can turn on for Ruff for checking your code.

19:18 And if you look at the rules list, there's a lot of rules that you could add.

19:23 So I was interested in this discussion on Reddit about, hey, Ruff users, what are the rules you're using and what are you ignoring?

19:33 This one was amusing to me that said, I start every project and select all and ignore none.

19:39 That's not realistic because some of them conflict with each other.

19:44 So I think this was probably a joke, but I'm not sure.

19:48 I do like, so there's some serious people that said, hey, this is my set that had like PyFlakes rules and PyCode style and warnings and errors.

19:57 Seems like a decent set.

19:59 It's just kind of a fun thing to see what other people are using and maybe check out some things.

20:07 Here's somebody else that has a different way.

20:10 Instead of picking which rules they use, it's a select all but ignore certain rules that don't make sense for them.

20:19 And yeah, just kind of an interesting discussion.

20:22 Anyway, I'm not sure if I covered this one already, but this was as of May 26th.

20:28 3.26 Python 3.14.0 beta 2 is out.

20:32 So if you're testing new Python versions, make sure to keep up on the new betas that come out.

20:38 Nice.

20:39 Yeah, I don't know.

20:39 I don't think we covered it.

20:40 Okay.

20:41 And I guess sort of a, oh, I got a couple more topics.

20:45 One of them is if you happen, like just randomly wanted to switch Mastodon servers and not lose anything, not like lose anybody that might be following you already. There is a way to do it. And we'll link to an article on how to do this on Fetty Tips. And it's not, this is not a trivial action. So I guess don't do it on an empty stomach. Looks like a lot of steps, but I'm glad somebody pointed that out. Why might you? Well, there's been a little bit of drama on Fostadon lately. And I am not the expert on this but um i'm there's enough people i trust that um are migrating away from fosdon that i'm looking into it um just haven't done it yet because yeah anyway um thoughts on fosdon i have i have

21:37 noticed people who i thought followed me i got notifications yeah on macedon that they had followed me and i think that's why i think they were moving accounts yeah

21:46 yeah i got some too i'm like well i want to follow them too and i was already following them so um So if you want, and so I guess this is public service as well.

21:55 Even if you're not considering switching, if you're getting notifications from people that they're, that you, that you're like, I thought they all already followed me.

22:03 It's probably because of something around this.

22:05 The last thing I wanted to talk about is I was, I've, I've got some, I think that my Python or yeah, pythontest.com.

22:16 That's what I got. Yeah. That's my site.

22:19 I don't know if I can trust the analytics that I was using before because I think I'm getting hit hard by AI bots and stuff.

22:26 So I did update my robots.txt, but I'm also checking out a new analytics package.

22:32 I'm trying out Fathom Analytics, and I chose them because they don't collect data on anybody.

22:41 It's just which pages were hit and which pages weren't, and they are GDPR compliant.

22:48 So, yeah.

22:50 Anyway, I'm not much more of a plug than that other than I'm trying it out.

22:54 And, yeah, we'll see.

22:55 Yeah, very cool.

22:56 That's it.

22:57 Awesome, awesome.

22:58 Yeah, I'm still loving Umami Analytics.

23:02 Again, GDPR, even better than GDPR compliant.

23:05 Like, no cookies whatsoever.

23:06 How's that?

23:07 And we self-host it, so we don't have to – we don't share the data with anybody.

23:11 So we don't have to tell you about how we're sharing our data with people.

23:14 Because we're not sharing data with people, which I think is great.

23:16 Okay.

23:17 Effortless.

23:18 It says effortless.

23:19 I'm not sure it's effortless.

23:20 I've expended effort when the migrations don't work or whatever.

23:24 But you know what?

23:25 It's fine.

23:25 It's still good.

23:26 Better than Google Analytics.

23:28 All right.

23:28 Are you ready for something fun as well?

23:30 Yes.

23:30 I'm not sure if I call this fun.

23:31 I think it's fun E in a sort of sympathy way.

23:34 So this joke is a cartoon called Programming Humor, from Programming Humor, called Emotional Damage.

23:41 And I think, Brian, you want to be the one, you want to act this down, you want to be the one that does the first square?

23:48 Sure. Hey, there's two people looking at a dog. Does your dog bite?

23:54 No, but it can hurt you in other ways. I don't know if I was going to say this. I'll do the dog.

23:58 The dog says, the feature you spent hours coding will not be deployed. Management finds it unnecessary.

24:04 And the person starts crying.

24:07 Does your dog bite?

24:08 No, but it can hurt you.

24:11 Yeah. Do you know what's more rough than software having that happen with you?

24:17 hardware people um i've got i remember this was decades ago but i was working with on a team and somebody got a was retiring and they were kind of roasting him during the retirement but um uh he was a fbj developer or known asic developer doing uh um custom chips and like he'd worked on like dozens of projects and only like five percent of them ever shipped most of them got canceled before the project end date. So that's rough.

24:49 Yeah, the lead time is definitely longer in hardware than it is on software, right?

24:53 Like from the time you work on it until the time it's out. Yeah, it's in like, you know, single digits or fractional digits of years instead of weeks or months.

25:03 I definitely think about the, I'll watch documentaries about, you know, like the Cassini probe that went to Saturn it went to?

25:11 And these people will be like, yeah, we started that 25 years ago, and now I'm done with this project.

25:16 Moving on to the next.

25:17 It's like you get two or three projects for your life.

25:19 Crazy.

25:20 Those things better not burn up in the atmosphere on their way out.

25:24 Yeah.

25:25 Well, may the dogs that talk about unshipped features stay away from you all.

25:32 Yep.

25:33 Bye.

25:33 Bye.