Transcript #280: Easy terminal scripts by sourcing your Py
Return to episode page view on github00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.
00:05 This is episode 280, recorded April 19th, 2022.
00:10 I'm Michael Kennedy.
00:11 And I'm Brian Okken.
00:12 And I'm Pat Decker.
00:13 Welcome, Pat. Good to have you here.
00:15 Well, thanks for having me on.
00:17 Yeah, it's really exciting.
00:18 You and I were chatting a bit about less new programming languages a bit over email.
00:24 And yeah, it'd be fun to just have you come on and share some of the things that you're passionate about and whatnot.
00:30 Tell people a bit about yourself.
00:31 I got started with computers way back in the day.
00:35 High school, I had an Apple II computer at school.
00:38 Eventually got a Commodore 64 at home.
00:40 I did Pascal on cards at Iowa State University.
00:44 And I did a data processing program at Kirkwood in Cedar Rapids.
00:51 And got my first job out of there about 30 years ago or so.
00:55 So I've worked mainframe and PC, Cobalt, C, and a little bit of Python here and there.
01:04 Fantastic.
01:05 And it sounds like you're doing DevOps type things today.
01:08 Yeah, the company has, like many, gone from on-site in their own data center to 80% or so running on AWS.
01:18 And so I get to share your same frustration with the AWS interface in Python.
01:23 It's not always obvious how that works.
01:26 It's updated every week.
01:28 And I know the way it is updated every week has got to be some code gen is running somewhere that regenerates it.
01:35 And it's just never quite discoverable.
01:38 Right.
01:38 It's fine once you get it working, as long as you don't touch it or have to understand it.
01:43 This is not the sign of a fantastic API, but it's cool.
01:46 You get it working in AWS.
01:48 Cool.
01:48 Well, great to have you here.
01:49 Now, Brian, before we jump over to your first topic, I just want to say this episode is brought to you by Mergify.
01:56 I am psyched about what these guys are offering.
01:58 Mergify is super cool.
02:00 It's all about automating Git collaboration.
02:03 And I have a lot to say about Git, but I want to start with Pi because I love Pi.
02:08 Banana cream pie is my favorite.
02:10 I'll take a cheesecake.
02:11 Does that count as a Pi?
02:12 Is that what we're talking about?
02:13 No, we're talking about HTT Pi.
02:19 Still a good Pi.
02:20 Yeah.
02:20 The website actually lists in their readme, tells you exactly how they wish you to pronounce it right off the bat.
02:26 H-A-T-T-Pi.
02:28 Anyway, it's a really cool tool.
02:31 It's a command line tool, and I think we've covered it before, for interacting with...
02:37 It's especially useful for APIs.
02:39 So, you know...
02:40 It's one of the first things I install on any server on my Mac if I get a new one.
02:44 Straight up, this thing is fantastic.
02:45 Yeah, yeah.
02:46 And it's so fantastic that it had 60...
02:48 This is what I want to cover.
02:49 I'm sorry to laugh.
02:51 It had 54,000 stars on GitHub, and then it lost them.
02:57 Then it didn't?
02:57 Then it didn't.
02:58 So, oof, what a cliff.
03:01 So, how many does it have now?
03:04 Right now, if we look, it's got 16.1 thousand.
03:06 Oh my gosh, it's way back.
03:08 Yeah.
03:08 Actually, this morning when I looked, it was just 16.0 thousand.
03:12 So, it's gotten 100 stars just as I was researching this.
03:17 So, anyway, the...
03:19 That's one of the...
03:21 I guess what I want to shout out to these guys of...
03:23 It's a cool tool.
03:25 If you're doing APIs, check it out.
03:27 And also, if you've started before, restart it because they deserve it.
03:32 And what happened was they accidentally put their repo private for...
03:36 Somebody was trying to do make it private, make something, some other test repo private.
03:42 And they accidentally put the main one private.
03:46 And if you're even private for a second, you lose all your stars.
03:49 So, warning.
03:51 Don't make your public stuff private.
03:53 Don't do that.
03:54 Oops.
03:55 Wow.
03:56 That is...
03:57 Yeah.
03:58 It's a good warning.
03:59 It's a great tool, but a good warning.
04:01 It does have a danger zone.
04:03 Danger zone.
04:03 This is going to happen.
04:04 But unfortunately, the warning is equivalent if there's zero followers and zero people interested
04:12 in it.
04:12 Or you have one of the most popular things on GitHub.
04:14 Like, it should be like, this is super duper important.
04:17 You're going to lose 54,000 GitHub stars.
04:19 Are you doing it?
04:20 Yeah.
04:21 Well, and also, I mean, if you work with a lot of GitHub repos and you're like, maybe
04:28 you made something public for a talk and then you're making it private because, I don't
04:32 know, the talk's over or whatever.
04:33 Or you just don't want to maintain something like, I'm just done with this thing.
04:36 If you're working with it a lot, maybe you've seen that warning so many times that you're
04:40 not really reading it anymore and double checking.
04:42 So, I guess this is just a warning.
04:45 Double check it.
04:46 If you can see that warning box, make sure you're on the right repo.
04:50 Indeed.
04:50 And Pat, what do you think about this?
04:52 Yeah, it's pretty tragic.
04:54 It's kind of like those end user license agreements where we all see them so often, they just
04:58 automatically click, I think.
05:00 Yeah, exactly.
05:01 The confirmation dialogues and stuff.
05:03 You're like, well, I have to copy this and paste that there.
05:05 I just can't continue.
05:06 So, I'm going to just, you know.
05:07 Instead of warning danger, I think people see, I want to get my work done.
05:11 So, click here to get my work done.
05:14 And then, oh, no, what happened?
05:15 Yeah.
05:16 Yeah.
05:16 Yeah.
05:17 Well, Brian, I believe you have at least helped a little bit.
05:21 David, out in the audience, starting it right now.
05:24 Oh, I forgot to start also.
05:25 I'm going to go start.
05:26 You better get in there and start.
05:27 Boom.
05:27 All right.
05:27 Plus two.
05:27 Couldn't you?
05:28 I was going to start myself, but I'm not logged in any of my browsers for some reason on
05:32 this account.
05:32 So, no, starting.
05:33 I have to start it later.
05:34 Okay.
05:35 Also, a little bit out of order, but I want to also just cover this other thing.
05:39 We are generally, not 100% of the time, because there's still a little bit of long tail stuff
05:46 happening.
05:46 But generally, we are moving our live stream of the recording, the one where David said,
05:51 I'm starting right now.
05:52 Thanks for being here, David.
05:54 That recording is now moving to Tuesdays at noon Pacific time.
05:57 So, if people want to come be part of the live show, see the video version, as well as,
06:03 you know, just make comments or whatever, not Wednesday, Tuesdays in general.
06:07 But check for the next week, couple of weeks, because it is still like some pre-scheduled
06:10 stuff.
06:11 Yeah.
06:11 So, how do people find out when it's coming up next?
06:13 I would say the best way is just to subscribe to the YouTube channel.
06:17 And then, you know.
06:18 Okay.
06:19 I hate to say it, but push the little bell for notifications.
06:22 So, it'll tell you like, hey, this thing's coming up.
06:24 And it'll let you know that we're gone live.
06:26 Yep.
06:26 Yep.
06:27 I don't have a great, better way.
06:28 Maybe we could do a mailing list, but it's a little tricky.
06:31 But usually, stick around on Tuesday Pacific.
06:34 Tuesday noon Pacific, we'll be recording.
06:36 Yeah.
06:37 If you check YouTube once a week, you'll see it's scheduled pretty far out.
06:40 And then you can get a reminder for just that one thing.
06:42 You don't have to always get messages.
06:43 Okay.
06:44 So, this is a fun one.
06:47 You mentioned some of the older programming languages.
06:49 When I was in college, I was studying chemical chemistry, chemical engineering, math, all these
06:56 different things.
06:56 And as part of that, they said, you're going to have to take a programming class.
07:00 I'm like, super.
07:00 What's it going to be?
07:01 Like C++ or something?
07:02 Remember, this is like early 90s.
07:04 So, that was a good choice then.
07:05 And they said, no, you're going to take the most important programming class you're ever
07:10 going to take in your career.
07:11 You're going to learn Fortran.
07:12 I'm like, wait a minute.
07:13 No.
07:13 No, no.
07:15 I don't want to do that.
07:16 That doesn't sound like the most important thing ever.
07:18 But nonetheless, that was one of my first courses I had to take.
07:23 I only took a couple.
07:24 Eventually got to take some C++.
07:26 But Fortran was where it started.
07:28 So, here is an article that was recommended to us by, let me make sure I get the name
07:32 right, Galene Swint.
07:34 And Galene is a PhD researcher and said, hey, here's a really interesting article.
07:39 The article I don't believe is by them, but it's about the sort of, I think it was something
07:46 in their lab they were talking about.
07:47 It says, the title is, The Counterintuitive Rise of Python in Scientific Computing.
07:52 Why is it counterintuitive?
07:53 Because Fortran is fast.
07:55 And what you want to do is process lots of data and you want to do it fast.
07:58 So, here's a really interesting story of people who were doing older stuff like Fortran and
08:04 C sort of were forced in, coerced into doing a little bit of Python and accidentally made
08:10 their supercomputer stuff go like a hundred times faster or something in Python rather than
08:16 in C or rather than Fortran, which is a counterintuitive result, right?
08:20 Yes, it is.
08:21 Yeah.
08:22 So, I think that I'll cover a few details of this article, just put some highlights.
08:26 But the reason I'm bringing an article instead of like a tool or something is I think this
08:30 would be interesting for people to share if they're in a situation where they're like,
08:34 oh, Python is no good.
08:35 This is such a slow language.
08:36 It's like doing math is like a thousand times slower than C.
08:39 I don't actually know what it is, but it's a lot, lot slower, right?
08:41 So, it goes through here.
08:43 And here's a couple of things.
08:44 It says, in our laboratory, a polarizing debate rages since, has been raging since 2010.
08:49 Summarized by the question, why are more and more time-critical scientific computations
08:53 formerly performed in Fortran now written in Python, a slower language?
08:58 Ah, Python, a slower language.
09:00 So, Python does have the reputation of being slow and especially around math.
09:04 I think it's, honestly, it's earned it.
09:07 Unfortunately, I would really love, sidebar, I would really love to see Python adopt something
09:12 along the lines of value types that can be boxed back to PyLongs, right?
09:18 So, like, on the stack locally, it's a number, but it gets complicated.
09:22 But I think that would actually solve a ton of stuff.
09:24 Anyway, so, yeah, plain Python is slower than Fortran.
09:28 But when you do computational stuff, you don't do plain Python.
09:32 What you do is you do NumPy, SciPy, Dask.
09:36 All of these things are written in C with a thin coordination layer in Python as part of
09:41 its API, right?
09:43 So, here's, like, some graph you can see on the article.
09:46 And it says, here's, like, you know, 10 to the 1 versus 10 to the minus 2.
09:51 So, is that 100 times faster if you use Cython or NumPy and so on?
09:57 So, pretty interesting.
09:58 And it also shows, guess what?
10:00 Shocker.
10:00 Be-de-de-de-de-de-de-de-de.
10:01 News alert.
10:02 Fortran is somewhat less popular than it used to be.
10:05 Oh, no.
10:08 Oh, no.
10:09 So, there's an interesting story of this project nicknamed Projector, written in, written around
10:19 2010.
10:19 So, modern, in quotes, Fortran, by somebody named Bob.
10:24 So, Bob worked on this project, and there's apparently, let me find the numbers here.
10:28 It's, like, 1.5 thousand lines of Fortran code.
10:32 So, a lot of Fortran code.
10:33 Like, a lot of complicated math bits.
10:35 The kind you kind of don't want to touch after it works.
10:38 And it's going to do, it's trying to project something.
10:42 Thousands of multiple, multi-perforations of combustion liner onto a 3D complex shape made
10:50 of millions of polygons.
10:51 So, that might get complicated when you multiply those numbers together.
10:54 You know?
10:55 And it turns out, Bob soon discovered that he was going to use NumPy and this thing called
11:03 a KD tree.
11:04 Because there's a bunch of people working in the lab that only knew Python.
11:07 So, like, here's a Python version you can use.
11:09 And he thought, this is going to be rough.
11:11 It's going to be super slow.
11:13 So, he went and ran it after he got it done.
11:16 And the Fortran version ran in 6 hours and 30 minutes.
11:19 The Python version, 4 minutes.
11:21 Wow.
11:25 So, what happened?
11:26 Is Python 100 times faster?
11:28 No.
11:29 Python is easier to work with and has many more built-in algorithms and data structures.
11:34 This KD tree thing was using a different data structure, is a different data structure that
11:40 has O log N complexity.
11:42 Whereas the Fortran version, because it's so hard to write code in, they just wrote it
11:47 in the simplest way they can make it work.
11:49 So, O N.
11:50 And when you have billions times millions times thousands, O log N wins, even if it's in a
11:55 slower language.
11:56 So, the conclusion was, Fortran would have been faster if they had implemented this KD tree
12:01 algorithm in it.
12:02 But the people working on it, they're like computational scientists, not PhD data structure type people,
12:10 right?
12:10 Like, they're already super busy just trying to do the programming and not working on, like,
12:15 cutting edge data structures and optimizations and stuff.
12:19 So, it was very easy to try one new thing in Python, which would have been very hard in
12:23 Fortran, and it was a big win.
12:25 So, anyway, I think the debate about performance, developer speed versus execution speed, but
12:31 also straight up execution speed, I think it's really interesting in Python.
12:34 And it's hard to be nuanced enough.
12:37 It's always there's some little interesting wrinkle like this.
12:40 Yeah.
12:40 And if you're really, you're going to hand code everything anyway, it's also faster in assembly.
12:46 So, just write assembly.
12:47 There's a downside to that.
12:51 Why not?
12:51 Yeah.
12:52 I've always wanted to stick to this one processor I really loved forever.
12:56 But this is interesting.
12:59 I like it.
13:00 But it sort of reads like a commercial.
13:02 So, Bob was programming scientific computing and he was not happy in Fortran.
13:08 Bob tried Python and now he's happy.
13:12 Well, I think Bob unwillingly tried Python and eventually he was happy.
13:16 Like, it sounded like he didn't want it.
13:17 He was dragged through it.
13:18 I like it.
13:19 Yeah, yeah.
13:19 I know it does read a bit like that, but I think it's an interesting story.
13:22 Pat, what do you think?
13:23 You've had, you have a bit of a historical perspective on all this.
13:26 Yeah.
13:27 I didn't go to school for chemistry, but I bet Fortran's not popular among chemistry students, is it?
13:32 No, not anymore.
13:33 I'm sure.
13:33 I did a little benchmark the other day.
13:37 I was trying to just see how fast some EFS storage was on AWS versus EBS.
13:44 And so I wrote a Python script to do that and wrote 10 million records out.
13:48 And I thought, well, maybe Python is going to be slow.
13:50 So I thought I'll just whip up a quick thing in Rust here.
13:54 And as it turned out, Python was 10 times faster at that too.
13:58 On the same exact hardware.
14:00 And I'm not sure why that was.
14:02 I may not have done something that was optimal for Rust, but there it is.
14:06 10 times faster.
14:07 Yeah, sure.
14:07 No, that's super interesting.
14:08 I've seen Python just rip across like OS type stuff like that.
14:12 It's pretty good.
14:13 All right.
14:13 Well, what's your first item, Pat?
14:15 Are we off to the metaverse?
14:16 Or is the metaverse off to us?
14:18 I stumbled across this the other day.
14:20 We've had Lucas Longa on the show, the programmer in residence.
14:26 And it looks like they're going to get another year due to a donation from a meta of $300,000 to the Python Software Foundation.
14:36 And it was specifically for that purpose.
14:38 That's a really big deal.
14:40 And like you said in the past too.
14:43 Python is very important to meta.
14:47 Facebook's parent company now.
14:49 Because much of their infrastructure is built on it.
14:52 PyTorch in particular.
14:54 They use it a lot.
14:56 So their internal implementation of Python is called Cinder apparently.
15:01 Which I think I've seen mentioned a few times.
15:04 But they would like to see continued development.
15:06 And they're going to fund it, which is awesome.
15:08 Yeah, that's really neat.
15:10 I think Cinder is a fantastic thing they've created.
15:14 And it's interesting.
15:15 It's open source in the sense of maybe the better term would be source open.
15:20 Like, here, we're going to put this out here.
15:22 We don't expect any contributions.
15:23 And we don't want to put it out there as like a project to grow on its own.
15:27 But we think it might inspire people.
15:28 So here's the source.
15:29 But what they have contained in there is amazing.
15:33 There's just so many optimizations and different things.
15:37 I think they might even have that value type thing I discussed.
15:40 And I can't remember.
15:42 But yeah, that's a really cool project that they're working on.
15:46 So this is great.
15:47 You know, I think Lucas Schelenga has gone sort of full circle, right?
15:50 He was independent.
15:52 He was at Instagram for a while, which is, you know, part of Meta.
15:55 And then left to sort of do his own thing and found his way to this developer in residence.
16:00 And now Meta is sending money so that he stays there.
16:03 That's like an interesting circle, I think.
16:05 Yeah, that's good.
16:07 It seems tenuous, though, that we have it at like one year at a time.
16:11 That we don't know if we're going to have a developer in residence longer than another year.
16:15 It's like a postdoc.
16:16 Hopefully it's a game of tag with Microsoft coming next or something.
16:20 Yeah, exactly.
16:21 That'd be great.
16:23 And I think there are a couple of organizations doing this.
16:25 So I know that there was a talk about having multiple folks sort of participating in that group,
16:31 which would be great because I know there's way more work than Lucas can do on himself.
16:35 He's quite busy.
16:36 Alvaro in the audience has a fine name proposal here.
16:40 The PSF developer in residence should be called the PSF Dunderder.
16:45 Underscore, underscore, D-I-R, underscore, underscore.
16:48 I hereby am opening a pep.
16:51 Well done, Alvaro.
16:52 No, just kidding.
16:52 I want to give him more work for something silly like that.
16:56 But that's pretty awesome.
16:56 The PSF Dunderder.
16:58 Cool.
17:00 Well, thank you, Pat.
17:01 Now, before we move on to the next item, let me tell you all about our sponsor today.
17:07 And like I said, I'm really excited about this.
17:08 I'm actually doing a bunch of work on some Git presentations.
17:11 And so I'm just trying to think about what's the workflow with Git and how can we work better,
17:18 especially doing team type stuff.
17:20 So Mergeify is a service that will help you and especially your team be way better with Git.
17:27 So it's an amazing tool to make you and your team be more productive with GitHub in particular.
17:32 It's all about leveling up for requests.
17:36 So some of the features of their platform that you hook into your repositories are automatic merges.
17:42 So you can set up conditions for an automatic merge.
17:45 And Mergeify will take care of merging the PR as soon as it's ready and passes all the details.
17:49 There's automatic updates.
17:51 So if you have multiple pull requests, it'll merge the pull requests serially on top of each other.
17:57 So there's no way to introduce a regression.
17:59 There's also merge queues.
18:01 So if you've got like a long CI process and things are slow or you've got lots of code,
18:05 you can set up priorities and have the most important ones merge fix like bug requests or bug fixes,
18:11 especially if they're security fixes.
18:13 They go right here, right at the front.
18:15 Do those now.
18:15 So that's awesome.
18:16 They also have something called backports.
18:19 So Mergeify will copy the pull requests to other branches once it's merged, if you set that up,
18:25 so that you can ship bug fixes to multiple branches, feature branches and whatnot.
18:30 Automatically or even older versions, right?
18:32 Version 1.0 of your app versus, you know, 2.0, 2.2, whatever.
18:36 Have those automatically merge those bug requests, bug fixes in.
18:41 Super cool.
18:42 So Mergeify does all these things automatically that you would otherwise have to do manually.
18:46 So you save time.
18:47 You and your team will focus on projects that matter the most.
18:50 Mergeify is coordinated with any CI and is fully integrated into GitHub.
18:55 They also have a startup program that can give your company up to $21,000 of credits.
19:00 So if you're doing a startup and you want to use them to help grow, you can reach out to them for sure.
19:05 Do that by going to pythonbytes.fm/Mergeify.
19:08 Sign up for a demo and get started.
19:10 The link is at the top of your podcast player show notes.
19:13 So just click it.
19:14 Thanks to Mergeify for supporting our show.
19:17 Very cool project.
19:18 Yeah.
19:18 All right, Brian.
19:19 What's the next one here?
19:20 Well, we were talking about data science a little bit and people using Python for data science.
19:25 But one of the things that people use when they're doing data science, plus other stuff and DevOps and whatever, is dashboards.
19:32 Dashboards in Python.
19:33 And that's one of the powerful reasons to use it.
19:36 And so there's a couple things I want to discuss around dashboards.
19:40 Both of them come via suggestions from Mark Skov-Madsen.
19:45 So thanks, Mark.
19:46 First is the easiest way to create interactive dashboard in Python.
19:50 Obviously, it's their opinion, but it does look pretty easy.
19:53 Is to use hvplot.interactive.
19:57 So this is an article that talks about how to do this.
20:01 And it really looks nice.
20:04 This panel looks pretty good.
20:06 Oh, yeah.
20:07 That's great.
20:08 Pulling data out, you got dark and light, and you've got the controls on the one side.
20:13 Interactive sliders?
20:13 Yeah.
20:13 Okay.
20:14 So this is just using some pre-canned data, but you can use, obviously, your own data.
20:21 But the article's pretty short.
20:23 And one of the things I love about it, of course, is it's got an animated GIF to show you exactly what you're getting into.
20:30 And then also a video.
20:33 So one of the authors, Sophia, just reads the, like, pretty much goes through the article and talks about it and shows the demo of everything.
20:42 So that's nice.
20:43 And it's only, like, six minutes.
20:45 I was watching it this morning.
20:46 So really cool if you want to try.
20:49 So HVPLOT is part of the HoloViz family of tools.
20:53 So a lot of people are already using that.
20:55 Or if you're not, check that out.
20:57 Cool stuff.
20:58 Yeah.
20:58 This looks great.
20:59 I always want to have good use cases for this.
21:03 And I just, I don't find myself doing lots of dashboards and stuff.
21:06 But every time I see them, I'm like, well, what can I build a dashboard for?
21:08 That would be great to have this.
21:09 Well, so I have some use cases for dashboards, but I really don't want to spend a lot of time on it.
21:14 And I think even though this is, like, six-minute video of how to get through it, I think that probably I'd have to set aside maybe a half day to figure out something to get it usable.
21:25 But it does show that once you understand what you're doing, you can throw together dashboards pretty quickly and be able to.
21:33 Well, we didn't, in the demo, one of the dashboard items was, and the controls is, like, let's say you've got a whole bunch of columns or some different elements in a row or in a column that you want to filter out.
21:47 So there's ways to filter your data so that you're looking at different stuff.
21:51 And I was like, I have an exact use case for this, so I'm pretty excited.
21:55 The other thing around plots is maybe HV plot or dashboards is maybe HV plot isn't the right thing for you.
22:03 Well, we've got another thing is the PyData Global 2021.
22:09 There's a video stream up on YouTube called Python Dashboarding Shootout in Showdown.
22:16 And so this is a whole bunch of different presenters talking about building dashboards in Python using either dash or panel or voila or streamlet.
22:28 So watch different people do it.
22:31 So if you want to watch some, use something other than HV plot, watch this video.
22:36 One of the things I love about this is if you're only looking forward to one of them, one of the commenters on this video put up hyperlinks to each of the different sections.
22:47 So you can just hop right to the video part that you want to watch.
22:50 Oh, yeah, that's great.
22:51 Yeah, looks good.
22:52 I love these dashboards.
22:53 It's definitely one of their powers.
22:54 Pat, do you do anything with them?
22:56 Sorry, Brian.
22:57 I cut you out.
22:58 We do have some pre-built dashboards to model the EKS cluster in Grafana and Elastic.
23:06 But this is another option.
23:08 It'd be good to do something ad hoc.
23:10 If it's a quick and dirty, maybe I could put one out.
23:13 So, Brian, do you just like grab a Pandas data frame and then make it interactive?
23:16 No.
23:18 Well, not quite, but it's almost.
23:21 I mean, this is built to go really closely with Pandas data frames.
23:26 So the code is, you know, you're doing some filtering, some pipeline processing, like group buys and selecting in to see what index you're going to use.
23:36 And then, yeah, it's pretty much interactive at that point.
23:40 But the code around this is building the widgets for the controls.
23:46 You got to define the UI and how people interact with it.
23:49 Got it.
23:49 Yeah, it's not.
23:50 But it's pretty easy.
23:51 I can't imagine.
23:53 I mean, it's not obvious.
23:54 So you have to kind of, it's good to watch a tutorial or something.
23:57 But it's also not difficult.
23:59 And then putting it together.
24:02 Yeah.
24:02 So most of the code in here is not the interactive plot because that's kind of already happens.
24:07 But it's the widgets.
24:09 Got it.
24:10 Cool.
24:10 Thanks.
24:11 All right.
24:12 I think you all will be excited about this.
24:14 I think this is quite a neat project.
24:16 Now, let's see.
24:17 This was sent over by Dave Chevelle.
24:20 Well, actually, it is posted over and is by Dave Chevelle.
24:23 So, I don't know.
24:24 Brian, do you ever use entry points in Python packages?
24:27 You know, so you can set up and pip install a package.
24:31 And then if you have that Python active, then you can just type a CLI command that is one
24:35 of the entry points there.
24:36 You know what I'm talking about?
24:37 Yeah.
24:37 I do that all the time.
24:38 That's great.
24:39 But there is some overhead to set it up, right?
24:42 I've got to install it as a package and whatnot.
24:45 This project called sourcepy allows you to basically turn any function into a CLI command, straight
24:52 CLI command in one line.
24:54 So, let me show you what I'm talking about.
24:56 And it doesn't involve packages and installing or anything like that.
24:58 So, imagine you've got some file here and it's got a function, just a def function.
25:03 Pygrep is the example here.
25:05 And it has a pattern, which is actually a regular expression pattern.
25:09 Note the typing.
25:10 It has grep data, which is a list of text inputs.
25:13 So, it could be a string or something, but it could also be like something piped or standard
25:19 in or something along those lines, right?
25:20 So, if I have this script that just does that work with that type assigned data and I have
25:27 sourcepy installed, I can say source this file here.
25:32 And what it will do is it'll actually add up the, it'll expose those functions like pygrep
25:38 and so on.
25:39 So, then I can just type pygrep and it'll tell me how it works.
25:42 And somewhere in here, where's the example running it?
25:45 Oh, right here, I think it is.
25:46 So, what you can do is you can just run pygrep against, now, as a function, right?
25:51 As a CLI function.
25:53 So, it has all sorts of cool features.
25:55 For example, it has this type handling.
25:58 So, type hints that we talked about can be used to coerce inputs off the command line
26:03 into like integers and patterns and I.O.
26:07 streams for like files and standard in and such.
26:09 Let's see.
26:11 It just takes basically any function like this and turns it into something you run on the
26:15 shell.
26:16 And then you can also go down and you can do like, I believe there's some sort of class
26:21 based way to work with it and so on.
26:23 But yeah, this is pretty neat.
26:25 What do you think?
26:25 I guess I'm confused at how it works with the normal source command.
26:30 That is a good question.
26:31 Like, how does it not break normal source?
26:33 Yeah.
26:34 That's a good question.
26:35 Yeah.
26:35 It is a good question.
26:38 Like, does it replace it for while you've got it installed or does it extend it somehow?
26:43 Yeah, I don't know.
26:45 I haven't looked at it enough to understand how it either coordinates or overrides source.
26:51 Sorry to put you on the spot there.
26:53 No, no, no.
26:53 It's a great question.
26:54 Yeah.
26:55 I haven't played with it, but it's really important.
26:58 You would want to know because you want to break your regular source stuff.
27:00 I'm sure you could alice it if for some reason it conflicted, but I don't know.
27:04 I think.
27:04 So I think this would be really helpful for, aside from that, if that's all working great,
27:09 then aside from that, I think it's a very useful way to have like a handful of little tools
27:15 for a project to be able to use that.
27:18 If you're trying to share it with other people, I kind of think packaging it the right way
27:24 to have entry points is probably the right way to go.
27:26 Yeah.
27:26 If you want to do a more formal, like a PIPX style, but this feels way more ad hoc, right?
27:32 Like Pat was talking about his little script that he built to test some stuff.
27:35 You could run this and then just call those functions as if they were built into Bash or
27:39 ZShell or whatever.
27:40 Definitely.
27:40 Also, also worth pointing out, it has built in native async support.
27:44 So if you have async functions and you source it, then you can just call, call it as if it
27:49 was a regular command line command and it's, it still just takes it and runs.
27:53 And Kim Benwick, who's been a co-host here before has a clever thought.
27:59 I suspect it wraps the normal source command and hands it over to the normal source.
28:04 If it isn't Python, it's a good idea.
28:06 Probably.
28:07 Yeah, probably.
28:08 Although, like I said, I haven't really looked at the mechanism there, but this looks like
28:12 a cool project.
28:13 It's got all sorts of supported types like JSON and unions and date time objects and all kinds
28:19 of stuff.
28:19 So check it out.
28:20 If you do a bunch of kind of ad hoc stuff with Python on the command prompt terminal, Pat,
28:25 how's this strike you?
28:26 I know that this might touch some of the things you're doing in your world.
28:30 Right.
28:30 Yeah.
28:31 just yesterday, one of the guys was contacting me because he needed to go and cycle through
28:35 a list of addresses to, to hit an API.
28:40 And this, this could be something you'd use like that.
28:43 Consume this file and hit the API with each example and away we go.
28:47 Yeah, absolutely.
28:48 Yeah.
28:48 This looks great for putting a little, making those little scripts you build with Python,
28:52 like literally scripts.
28:54 I know a lot of people call everything Python scripts, but this is really for those types
28:58 of apps.
28:58 A single file script.
28:59 Yeah.
29:00 Yeah.
29:00 A single file thing, not like a 20,000 line Flask app, right?
29:04 That thing starts to stretch the terms of what script might mean, but this is exactly for
29:09 those things.
29:09 Yeah.
29:10 All right.
29:10 What's your final one, Pat?
29:11 This, this is, is similar to what you just covered.
29:14 Actually, you've, you've talked, I think a little bit about it before.
29:17 Conch shell combines the best of bash shell and Python in the Linux terminal terminal.
29:22 I ran across this.
29:24 This is way more than what I covered.
29:25 This, this is like another level here.
29:27 It takes it to another level.
29:28 Yeah.
29:29 I ran across it on its boss, which is, is a nice site for open source type things.
29:36 In a nutshell, what, what it does is provide you with a new shell where as they demonstrate
29:42 here, you can use straight Python.
29:44 You can import JSON, for example, and, and define a, a, a variable or a table, print,
29:53 print that out and, format it, the length function here, but you can also then mix
30:00 in regular bash functions where we, for example, here it curls from a website.
30:06 The result comes back and the length function tells you the length of that curl.
30:10 Oh, how interesting.
30:12 And one of their famous.
30:13 That's a little bit like, almost like a Jinja Django template, but in your terminal,
30:19 right?
30:19 You put a little dollar to say, here's a, a bit of code to run on the shell.
30:24 Yep.
30:24 Um.
30:25 Shell to run here.
30:26 The one famous statement they have is, you know, I always forget how to do a for loop in,
30:29 in bash, but in Python, it's pretty easy.
30:32 and, and they have kind of a rule that it evaluates.
30:35 To Python first.
30:37 If it evaluates to Python, it executes as Python.
30:40 If it doesn't, then it more or less executes as bash.
30:43 and I do have a link we could, we could put in the show notes too, to a, a video
30:50 demonstration from, from their homepage.
30:53 and the guy does a spectacular, demonstration with auto completes and, and they have, what
31:01 they call contributions.
31:02 And, and that also begins with an X.
31:05 so it's, it's like a plugin or an add on, but it's a contribution.
31:10 it, it, it, it is get aware and it is, virtual environment aware.
31:16 it sounds like a killer environment, but it would take some getting used to.
31:21 I have a lot of muscle memory to overcome, to take full advantage.
31:26 I think.
31:27 This is cool.
31:27 Brian, have you played with us?
31:29 You know, I tried to play with it a couple of years ago, but I, but now I, and then
31:34 I haven't, but I do want to.
31:35 And I think one of the things I didn't give it a fair shake.
31:38 And I think it would be good to just say for a week, maybe I'm going to try to just use
31:43 this instead of pulling up my normal terminal just to see if I can get used to it.
31:47 That's the thing.
31:48 I think you have to immerse yourself probably.
31:49 And, and it is about five years old.
31:51 I think it's fairly mature.
31:52 Yeah.
31:53 And yet also the less I write bash scripts, the more I wish that I bash scripts were more
32:00 like Python scripts.
32:01 So now they are.
32:03 Yeah, this is great.
32:05 a good find and it's something I've also wanted to play with, but haven't.
32:10 Nice.
32:11 Brian, you got any extras you want to share with people?
32:13 I don't this week.
32:14 How about you?
32:15 I thought I didn't, but then I did.
32:17 I just want to do a quick follow-up.
32:19 Okay.
32:19 I was just thinking about a couple episodes ago where we talked about the March Madness
32:25 package tournament that Chris May had sent in.
32:28 And I was thinking, you know, it's probably done now, isn't it?
32:31 Let's see who won.
32:32 Cause I think we were at the point where we were at the elite eight, weren't we?
32:36 Or yeah, I believe we were at the elite eight.
32:39 So we had a showdown to put between NumPy and Pip.
32:42 NumPy crushed it.
32:43 We had a showdown between pandas and requests.
32:46 Pandas crushed it.
32:47 And it was an all data science finale.
32:51 And NumPy and pandas went to head, went head to head.
32:54 And it was pandas, 55% taking the winter.
32:56 So I just want to do a quick follow-up on that.
32:58 Very important news.
32:59 We have a champion.
33:00 It's good.
33:01 But try using pandas without NumPy.
33:02 That'd be tough.
33:03 There you go.
33:04 Yeah, indeed.
33:06 I was.
33:07 Pat, do you have anything else you want to get a, give a shout out to?
33:10 I do have one extra.
33:12 quick, like, it's easy to bash Microsoft, but they've been involved in.
33:17 They bought GitHub.
33:18 They, have lots of our Python code developers working for them and they have this rewards program
33:25 and they want you to use Bing search, obviously.
33:27 But what you can do is choose a, a charity to give points to.
33:33 you can also get like a $5 gift certificate to AWS or, or Amazon or something, for yourself
33:40 if you'd rather.
33:40 And, and every day you just kind of click on these little links and, and I just got
33:45 five points or 10 points and, oh, now we got to do a quiz.
33:49 I'll come back to that later.
33:50 and, and as you do this, you accumulate more points each day and those points then
33:55 can go to your charity.
33:56 And I did, the red cross for a while and I did the, the CDC during our, our lockdown
34:02 year there.
34:03 And you can also donate to the Python software foundation.
34:07 So that's some, that's a way we can all contribute.
34:10 So if you're Googling stuff with Bing, go ahead and set up the rewards.
34:14 You're Googling stuff with Bing.
34:18 If you're Googling stuff with Bing, set up the rewards to go to the PSF.
34:20 You might as well, right?
34:21 That's right.
34:22 It's fantastic.
34:22 Awesome.
34:24 All right.
34:25 Well, thanks.
34:25 That's a good one.
34:26 I got a joke for you all here and I know Brian, you do as well, but I feel like this
34:32 one, I better take this one because I don't feel you're in a position to, to make an unbiased
34:37 decision here.
34:38 Cause it's so close to your heart.
34:39 Okay.
34:39 All right.
34:40 Okay.
34:40 So this one is over on the Reddit programming humor one and it poses a question.
34:45 What seems like a straightforward question for a piece of software?
34:49 How do you exit them?
34:51 And it types, it, there's some, you know, a couple of interesting options here.
34:56 You get colon exit.
34:58 7% of the people think you type colon exit.
35:01 They're wrong.
35:01 You type colon WQ, which I guess if you want to make changes, cool.
35:06 Save your changes and exit.
35:07 You could do most people got that one.
35:09 And then you could do colon quit 7% again.
35:12 You could click the close button hard for a non UI app, but or almost 20% of the people
35:19 said, you know what?
35:20 The way you restart Vim, this horrible thing.
35:22 I can't get out of it.
35:23 I'm going to restart the computer.
35:24 I'm trapped in here.
35:27 I just, I got to get out.
35:29 I tried command Z or control Z and it just background it.
35:32 And now what's it doing?
35:33 It keeps coming back.
35:34 And I think people clicked restart the computer to be funny.
35:38 I don't.
35:38 I know they did.
35:39 I thought, and I pulled it up as a joke to be funny as well, but there you go.
35:43 So have you, I don't know the, the, the close button works for me.
35:48 I don't know.
35:49 Well, it closes the terminal, right?
35:51 Yeah.
35:51 Yeah.
35:52 so I mean, who doesn't, you got to say, do you want to terminate the running program?
35:58 Right.
35:58 You get like a warning that you're doing it wrong.
36:00 I'm with you, Brian.
36:03 I'm a boom guy.
36:03 It's not hard.
36:07 And actually I haven't, I don't use straight Vim except for on the command line.
36:11 You're right.
36:12 I guess I do that on the command line, but I do, I essentially use Vim within PyCharm
36:16 or VS Code as well.
36:17 But yeah.
36:18 Vim emulation.
36:19 Yeah.
36:19 Okay.
36:20 It's fun.
36:20 So I, I wanted to share a joke also because I just thought this was so funny.
36:25 I could stop laughing this morning.
36:26 so ran across this, tweet by, mediocre superheroes.
36:31 it's a little cartoon thing.
36:33 So, so the guy, you know, finds a genie, genie says three wishes.
36:39 Hey, can I wish for more wishes?
36:41 Not with me?
36:42 No.
36:42 Well, fine.
36:43 I wish you couldn't count.
36:45 And the genie, done.
36:49 How many wishes do you have left?
36:50 A billion.
36:51 Hmm.
36:52 That sounds right.
36:53 I love it.
36:55 anyway, that was funny.
36:58 It's an inventive way around an old joke.
37:00 So that's great.
37:03 Anyway.
37:04 oh, quick.
37:05 Oh, a bit of audience feedback as well.
37:07 Henry Schreiner says, restart the computer.
37:10 Isn't that how you get out of email?
37:11 Also true.
37:14 Yeah.
37:14 Yeah.
37:16 What, what key chord do I hit to get out of that one?
37:19 Fantastic.
37:20 It's probably control something.
37:22 Yeah.
37:22 I bet it involves a control K something somehow.
37:25 Anyway, thanks for, everything.
37:28 Brian, thanks for being here.
37:29 Thanks for the jokes and pan.
37:30 It was great.
37:30 You could join us.
37:31 Yeah.
37:31 Thanks, Brett.
37:32 Thanks for having me on guys.
37:33 Bye everybody.
37:34 Bye everyone.