« Return to show page
Transcript for Episode #266:
Python has a glossary?
00:00 Hello, and welcome to python bytes where we deliver Python news and headlines directly to your earbuds. This is episode 266. Quarter, January 12 2022. I'm Michael Kennedy.
00:11 And I'm Brian knockin. So great
00:13 to be here again. And we had this whole survey about having guests, Brian. And this week, we don't have a guest that just you mean, which I think is cool. That's all right. Yeah, yeah. Yeah. People out there listening, if they really want to be a guest, they can shoot us a message. For now. We've got so much cool things, so many cool things to speak about. We're gonna need like a glossary or an FAQ or I mean, something. Yes. Well,
00:38 I'm actually I don't know how I missed this a lot for so long. But there was a tweet by who was it Trey Hunter. At a tweet that I mentioned, it actually referred to the glossary. And I'm like, why we have a glossary? I never checked it out before. So on the python.org, website, there docstoc, Python that org, there's a glossary. And it's actually pretty cool. There's a whole bunch of stuff like, if you if you forget what abstract places base classes are, it's there. And there's so there's Python stuff, there's programming stuff,
01:14 and even defines what the three arrows mean. Yeah, like
01:17 the three errors. That's the first one default Python prompt, but also the dot, dot, dot, what is that? The ellipses? And two, three. So this threw me off once. When I first started, I was like, What's this two to three thing? That's a third party package. And it was, it wasn't obvious to me that it was built in. So that's kind of neat. But it shouldn't be an issue anymore, because everybody's on python three now. Right? So anyway, so the glossary, I just just this just a shout out that this is here. It's fun. So check it out. The other thing that this refers to the other documentation in Python a lot. And one of the things that refers to sometimes is the FAQ, and also didn't know that was there, we have an FAQ. Yeah. And it's split into a whole bunch of stuff like general Python, and programming and history and design and stuff. And like, and I ran across it, because one of the things I looked up when I followed from the glossary was this question of what's the difference between arguments in parameters? And it's something that I've always messed up. And now I think I have it, right adders are the names of things that are appear in the function definition. And the arguments or the values get passed in neat. I don't know
02:35 why sometimes people use them interchangeably, but they kind of talk about different ways of working with that data.
02:41 Yeah, but like, let's say you're new, either new to Python or new to programming. Some of these perusal some of these are the great things like, why did why did my changing list? Why also change list x? Well, this will help you understand why there's the the naming system in Python, this stuff like that. So
03:01 it's pretty great. Yeah, it talks about references and all sorts of stuff. Yeah, quite cool. Like, I didn't know that we had it. But yeah, that's, that's cool. You didn't know it. Was there? No, I did not know it was there. That's great. Okay. I mean, I didn't, I didn't know anything about it. So I want to talk about something else. I won't talk about any IO, as I'm sure you and a lot of listeners know, I'm a big fan of async. Io and async. And await, I think it really unlocks a lot of potential. When you're waiting on things. There's been a lot of analysis saying, Oh, I, I did this computational thing. And it didn't make it any faster. I made it slower. It's like, Yeah, cuz it only scales waiting, and you're not waiting. So when you're talking about waiting, it's usually has to do with IO with external systems, right, I'm waiting on the file system, I'm waiting on the database and waiting on whatever. So there's this cool library called any IO. So I indirectly learned about this from Sebastian Ramirez from fast API, because he talks about this thing called a sinker, which extends a few things that are ultimately probably gonna make it back to any IO. So any IO is an asynchronous networking and concurrency library that works on top of either async I O, which is the one we all know and love, or trio, which is similar to async. Io, but it has a larger, it has a more of an understanding of dependencies between tasks and things, how you can say, I'm going to create a set of work that is made up of these tasks. And this task is actually a child of that other task. So if I cancel the top level one canceled children, it's a little bit more complicated, but it solves this structured concurrency story that people sometimes need. So you can use this to get some libraries that will do nice things with stuff you might wait on. Right. So some of the features include there's task groups, that's the thing I was describing with parent child relationship type of things. With trio it has high level networking, TCP, UDP, an API for byte streams and object stream inter task synchronization and communication like locks and conditions and events and semaphores, worker threads, sub processes, all kinds of stuff. So you over and you can sort of see some real simple ways for it to run. So one of the things that sometimes not entirely obvious is how do you run something on async IO, because you've, you've got to make sure you've got an ace async i o event loop running. And if there's already one, you should call get loop. But if it's not one, you should create one, and so on. So this is just you know, have an async method, which can be a task and just say, you know, true dot run, or you can run and just say the back end is trio, which is pretty cool. So all sorts of cool stuff like that. And it just sort of simplifies working with these different things. If we go and look at the sockets example, you just say await async with await, connect TCP, and that allow you to do like a wait receive array, await, send, and so on. So some nice libraries that come out of any IO for doing TCP, UDP, all that kind of stuff. You know, the things you would wait on?
06:05 Yeah. So um, so if you're, if you know, you're going to use async IO with this by you anything,
06:12 I think that it has those additional higher level libraries for like talking to TCP, and bytes difference and stuff like that. And also the sub process. Yeah, thanks. I think it does have like some utility stuff on top of it. But it's pretty cool. You can say like a weight runs a process, which is pretty cool.
06:30 That's actually that's really cool.
06:32 That's I've not seen this one before. And that one kind of makes me excited now.
06:35 Yeah. That's cool. Nice. Cool.
06:37 So not a whole lot more to say about that. But if those are the types of things you're doing, then, you know, come check it out. It's a cool library.
06:43 Do you know what else was cool?
06:45 I do not. Tell me about
06:47 that we're doing something else.
06:50 Wait, oh, yes, we have got one more thing to talk about before we move on. Because we have a different number of things. I'm not sure that I'll slot in here. So what else is cool is that this episode is brought to you by data dog, thank you data dog for supporting the show. They've been big supporters of Python bytes for a really long time. So let's purchasing plus a really great T shirts. Exactly. They've got cool T shirts, I mean, I definitely want to get one of those. So data dog does a lot of things. One of their things we're focusing on now is real time monitoring. So they have a real time monitoring platform that unifies metrics, traces logs, into one tightly integrated platform. Their APM empowers developers to identify anomalies, resolve issues and improve application performance. He just finished a talk Python episode talking about running in production, and everyone there on the panel was like, You need to make sure you're monitoring in production, for things that change in your performance profile. Because you get too much data, as your infrastructure changes, as the way your app has been used changed. It could hit these scenarios and run into problems that you would just never see in testing. So if you had data dog APM, you would have caught it. So you can begin collecting stack traces, visualize them as flame graphs, organize them into profile types, such as these are the CPU, metrics, ether IO, and so on. Teams can search for specific profiles, correlate them with distributed traces, if you're doing micro services and identify it slower underperforming code for analysis and optimization. Plus, with data dog APM live search, you can perform searches across the full stream of ingest races generated by your app over the last 15 minutes. So try data dog APM for free with a 14 day trial. And you do you get that t shirt that Brian mention. So just go to a Python by Sarafem slash digit dog or click the link in your podcast player show notes. Or in this chapter. Remember, we talked about chapters and the links? I'll have this chapter as well. So thank you, dog for supporting the show. Now, let's talk about your next item. Brian?
08:51 Yeah, I think it's Vax Vex, Dax, I don't. Thanks.
08:57 Oh, there's there's people are gaining traction for the idea of putting a pronunciation on a GitHub repo for projects that are not obvious. Yeah, I saw this on Twitter. But let's do it. Let's make it happen.
09:08 So this was suggested by Glenn Ferguson. This is a library that's a high performance Python library for lazy out of core data frames. Hmm. I don't know what our core is. So I looked it up in a glossary.
09:26 So after the FAQ,
09:27 yeah. Out of core, typically refers to processing data that is too large to fit in the computer's memory. So yeah, that's what this is. So for data processing, often, you're trying to do some analysis, do some statistics, maybe explore the data a little bit, but you don't want to read it because they're huge data sets and you've got like maybe a limited computer. And so that's, that's what this this is set up to do. The the main features of it so you've got like big data sets, it also has a It has statistics like mean and some accountants, standard deviation, etc. But it also has some visualizations that are sped up from, from how they've sped things up and not kept things in memory. And they're using memory mapping and some tricks inside to try to avoid any caught memory copies, and try to do it as lazy as computation as possible. And this is actually pretty impressive. I was watching some of this, some of the demos. So there's a sai pi 2019 video, where it's the person who started this library, which is now a company also, but does a demo of this. And it's really impressive how fast things are, he's pulling things up. Because of the memory mapping. You can even have multiple, you know, multiple Jupyter Notebooks. Yeah, that's it, multiple Jupyter Notebooks looking at the same huge data set. And it doesn't slow things down even when things are working on it. It's pretty neat. So definitely think this is worth checking out. One of the things on the readme that I like is a key features. So it's instant opening of huge data files. Because it's a memory mapping the data file, it actually doesn't read, it doesn't do any reads when you read it. But when you pull some data out, it does lazy reads, jumps ahead. And it's it's pretty impressive. So got that also as an expression system so that it's kind of there is a little bit of a, so you can lazy transforms of data. So that's neat. avocor data frames, like we said, fast group by an aggregations, whole bunch of the the fast and efficient joins are interesting. I was watching looking at another comparison of pandas and Dask and other things versus Vax. And the joins of huge tables are pretty fast and seamless with here and those will blow up some projects. So yeah, this is yes, it is similar to Dask. Somebody asked a lazy list. Lazy like Dask. Yes. But
12:07 that's a good thing. Yeah.
12:10 Oh, but, yeah, a bunch of fun things. It's good to have. It isn't the same as Desco. It's worth checking out to see if maybe this one might be a good fit for you.
12:22 Yeah, it's cool. It's the lazy, it makes the magic, right? You don't have to load it all from disk. You can distribute it. There's all kinds of interesting things in the a billion sample per row operations per second. That sounds that sounds pretty good.
12:37 Yeah, the it watching the demo, it's incredible. How fast is popping up things in, in loading even even to be able to visualize things by pulling out samples on the set. So
12:49 wait a member, I heard people told me that Python was slow. So it didn't make sense to do this kind of stuff with it. What's going on here?
12:57 No, no, no. Pythons fast.
12:59 I know. So pick the right libraries. Alright, ah, one of the things that is definitely well known in the Python world is Django. I've even had people tell me, I came to become a Django developer. And so I had to learn Python, which is a really interesting perspective. Yeah, I want to talk about the Django developer survey results for the 2021 survey, because that just recently happened. So I'll highlight a couple things that are interesting over here. One of the questions was, what is the main reason you use Django? Is it both for work personal or for work? Only? 15% said, just for work. Does that seem like a lower number than you expect? Yeah, yeah, I thought more people would just like they go to work to do Django and they go home, and they, I don't know, watch Game of Thrones or something. But Django developers love it. And they, you know, they use it a lot for all sorts of things. So by far the biggest group here 66% is for using for both. So that that stood out to me. Another one that's interesting is how many people are on the latest version. So web apps often sort of get stuck in the past, because once you get them up and running, like people don't want to touch it. But 75% of the people are using a three two, which at the time of asking, I believe was the latest version.
14:09 Okay, I'm like that we're up to four know what's going on for is
14:13 in beta? I'm not sure. I don't think it's totally released. But yeah, this is still remembers from 2021. Okay, 2021. And then also, Django has this concept of the latest stable release, and then a Long Term Support release. So if you go to just the latest stable release, and it's not LTS, you may have to upgrade sooner if you want security fixes and so on. And yet 71% of people use the latest stable release, because they're upgrading frequently, I'm guessing. And then 27% on the latest LTS and 2% are just like, how do I upgrade this again? I don't know. But that's pretty interesting. And then the next question was how often so 44% of the people upgrade every stable release other people less so and it kind of breaks down? 5% I use an unsupported version of Django. I'm okay with that. databases for people doing. Django is a very strong bias to use a relational database, because much of the magic of Django depends upon the Django models, right? Like the admin section is driven by that and so many things. And those are all relational. So with that in mind, the most common database 77% of the time is Postgres SQL. She's cool. And then there's number two there surprise you, Brian?
15:24 No, and I really SQLite. Yeah,
15:27 if you got very simple deployment stories, you're just gonna put it on one server, not much data, you just want to need something relational SQL light,
15:36 well, the internal tools and stuff to exactly.
15:39 So I mean, you wouldn't run like a major tech company, SQL light, get away with it, without scars and tears. But, you know, for simple internal apps, that might just be what you
15:49 need. You're gonna make some SQLite enemies by saying that, but but if
15:53 you had 100,000 users can currently isn't quite like that might be bad.
15:58 Oh, somebody else said? Possibly, because SQLite is the default setting. So
16:03 yeah, I certainly that that is that's a big push. The other one is, do you do caching? So caching is a another layer between the database and your web app, where you get the database stuff back, and then you stash it in the memory somewhere, so that you don't have to do queries again. So they said Do do you do that? And if so, what do you use? 47%, Redis 43%. I don't do that. And then the only other really notable thing is memcached. D. So interesting there. And I guess people, if they're really interested, they come through and look, there's a lot of I don't want to go through because there's so many details, but it's like, what are your favorite components? Like models, or admins, or auth? Or what contrib apps do you find most useful, like humanized or whatever. So pretty interesting. No surprise, people are using Django templates, not Jinja as their main templates, and then look, it's a race between Pi test and unit test as the top two most common frameworks
17:03 with PI tests near above unit tests. That's pretty cool, especially since unit test is the default.
17:54 Next, we got more extras?
17:56 We got extras? Okay. Yes. Extra, extra, extra extra. So I've got so many extras. I decided to make it. One of my topics. Brian, got anything else before I go on another rant?
18:09 No, I'm just ready to listen to all these extras.
18:11 I got a bunch of good stuff. So don't let the bad guys into your web apps. Django just had security releases for 4.01. Or sorry, 40132. And 266. Out. Does that mean four? Oh is out? Yeah, it does. nobody's using it? Yeah, well, they didn't use it in the past. And how was it out? Paul Everett and I teamed up to create a course over at talk Python called static sites with Sphinx and markdown. So this course is free, everyone can go take it, all you got to do is have an account and go here. And it teaches you how to do Markdown and Spinx. And generate static sites. There's a cool little demo app that we build over here that can go into search and look around and see it like you document your code and do all kinds of stuff. It's nothing too complicated, but sort of neat to see how to use markdown with sphinx because typically, Sphinx is about restructured text. So check out the course over there. I'll put that in the show notes.
19:08 I'm going to definitely check that out. Because I've got a project that I wanted to use sphinx for, but I was a little intimidated. So
19:14 cool. Yeah, Paul does a great job of that. So and it's only an hour and 25 minutes or something. So it's not a huge investment in time. Something that's bothered me, basically ever since USBC. Was this four years or something is I need more ports on my computer, and I want them to be a b c ports because I have USB things these days because I want them to go into the ports that already have until Thunderbolt four you've not been able to get a dock that has more than one USB C or Thunderbolt port, which is super weird to me. But recently, they've come out with Gen four and I just got this thing called the CalDigit Thunderbolt four USB four element hub, or magazine is fantastic, right? I'm talking to you on my computer here and I have my 4k monitor under my PDP camera, my microphone my stream Deck, the lights, keyboard mouse track, like seven different things, including the monitor plugged in with one cable through this thing.
20:11 That's really pretty cool.
20:13 It's so sweet. So basically I has on the front, it has three, USB C Thunderbolt four and a power in. And then on the side it has the Thunderbolt that goes to the computer. And then also for USB high speed USB, A, but the good ones. So really, really cool. If you need to expand out your new, newish computer,
20:36 what do you use the plug in to the monitor,
20:39 I have a venerable to display port adapter. And so that way, if I come with my new MacBook, I can just unplug one thing from my mini, click it over and then boom, I'm ready to go. Everything's configured. I'm gonna get one of these. Yeah, they're not super cheap. They've been off for about four or five months. But they've been sold out supply chains. You know what time of what's going on with supply chains, everything, but they finally came out there on Amazon. So I linked to it over on Amazon, also linked to this video that by doc rock talking about like, what the heck is this thing? And why is it different? Alright, I also tweeted about how we use the stream deck to do our live stream, which was fun. So I shared a bunch of pictures of that, like how we like put the website. So it says how it's streaming, how we tweet automatically how we do this sharing and all that kind of stuff. I'm now going to be working on how to use that thing for software development. Like how do you use it for Jupyter Notebooks? So every button on the stream deck, which is 14 free buttons basically like how what are the 14 Jupiter operations, you'd like to have like run all cells give you a button? Or you know format with Black could be a button, all sorts of stuff. So very cool.
21:48 Oh, you just have a black button. The thing with no love?
21:51 Yes, yes. Absolutely be black as well. So anyway, people are interested in that it's there. I did a talk at pi Bay quite a while ago. Now the talk is out. Carson was kind enough to retweet that and pointed out that hey, the talk is actually out. So I'm linking to my PI Bay talk, which was an in person talk at a conference. Imagine that. In San Francisco, that was really fun. People can check that out. Speaking of conferences, we are a media sponsor of Python web conference. And so you can definitely check that out. This is one of the honestly becoming one of the bigger online conferences, it's five days, all day, you know, a lot of these online conference that I go half day a little thing here. So a lot of tracks a lot of things going on with the Python web conference. I'm also speaking there as well. Are you speaking there? No, I'm off to get you pi test and something up there?
22:41 Well, it's probably do some web stuff. So absolutely.
22:45 Absolutely. And so there's a code that you can use. It's in the in the show notes, Python bytes at PWC. Eu zero to two and I'll give you 15% off. Also in our neighborhood, sort of because it's virtual, does that still have meaning? We have pi cascades coming up and pi cascades is February 5 and sixth. So that's going to be remote so it's not really local until things settle down so people wherever it can take
23:11 what's in our timezone so good. That's true
23:14 timezone still matters. It absolutely still matters. Alright, that's it for all of all of my extras. And we have Patrick out in the audience pointing out Python, Italy is also happening in June. So that is fantastic. Yeah. Awesome.
23:29 Cool. So before we get to the joke, I wanted to to like ask you this brain teaser that like my daughter brought on the spot. So she yeah, I've totally put me on the spot. So she came home. She's She's a junior I came home she said we have this cool brainteaser. I just want to ask you just tell me what you think. So it's, it's a math problem. So I'm going to go out I'm going to buy a baseball bat and a baseball. The total for the both the baseball bat and the baseball are $1.10 It's pretty cheap. The difference is the baseball. The baseball bat is $1 more than the baseball. So how much is the baseball? So I'm not gonna You don't have to answer right now. But it tripped me up for a little bit. Like why is this difficult? It turns out it's like five cents, because of five cents plus $1 $5.10. But my brain went. It's $1 $2. And dollar said Yeah, yeah. But that's a 90 cent difference. I don't know why this is difficult. But it's a fun brainteaser to ask people and
24:33 J. Ah, funny. Very cool. Well, you know what else is funny? That feeling of joy that we get as software developers, but is mixed in with kind of remember set myself screaming at my computer yesterday like out loud because something was so frustrating. I just like how is it possible that this is not working? Like what is going on? Was it actually about programmers with some app or something? It was it's always happening. Yeah, yes, sometimes it's my fault. But anyway, so the joke is expressing that feeling. And it's the stickers is I hate programming. I hate programming. I hate programming. It works. I love it. It's like, it's like childbirth like you forget all the horror and pain, like look at my amazing app like, do you remember that? You cried for two days? Because like you couldn't get it to query the database, right in production. Yeah. And you're right, but you love it, because now it works. So
25:29 I love this. I was there this morning. I was like fighting Jenkins of trying to create a Jenkins job with four repos and different branches. And I just like I hate, I hate Jenkins. But
25:43 if it works, or when it works, when
25:46 it works, I'm like, sweet, I am the smartest person in the world to do this whole thing
25:50 at tastic. All right, well, it never ends never ends. We've been doing this a long time. And we still have these feelings, don't we? Yeah.
25:57 So well. So thanks a lot, Michael, for another great show.
26:01 Yeah, you as well. It's always fun. Thank you, everyone for listening. catch you all later. Thanks for listening to Python bytes. Follow the show on Twitter via at Python bytes. That's Python bytes as in by Ts. Get the full show notes over at Python by Sarafem. If you have a news item we should cover just visit by them by Sarafem and click submit in the nav bar, we're always on the lookout for sharing something cool. If you want to join us for the live recording, just visit the website and click Live stream to get notified of when our next episode goes live. That's usually happening at noon Pacific on Wednesdays over at YouTube. On behalf of myself and Brian Aachen, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.