Brought to you by Michael and Brian - take a Talk Python course or get Brian's pytest book

« Return to show page

Transcript for Episode #182:
PSF Survey is out!

Recorded on Wednesday, May 13, 2020.

00:00 Hello, and welcome to Python bytes where we deliver Python news and headlines directly to your earbuds. This is Episode 182, recorded may 13 2020. And I am Brian, aka Michael Kennedy. And this episode is brought to you by data dog, there's two surveys that I feel really do a good job of keeping their, you know, they have their thumb on the pulse of the community. And I would say one is the PSF JetBrains combination. That one's really good for Python in particular, and the other is the Stack Overflow survey. Well, the Stack Overflow survey recently came out a couple months ago or something like that. So now it's the PSF JetBrains survey. When I first saw this, I thought, Oh, this is last year's because it was 2019. But they do a ton of analysis on it. And it the survey was done in 2019. But it's released now. So anyway, I want to talk about some of the results in there. Because there's some interesting takes. And also want to say thank you, too, was a Mario who sent this in, because like I said, I thought it was last year, I'm still waiting for the 2020 edition. Alright, so let's talk about some of the results. First of all, one of the first questions asked was, do you primarily use Python as your main language? Or are you interested in Python, because it's some other, like, I also happened to use it in addition to JavaScript or something 84% of the people who took the survey, and that was mostly mostly folks who visited So 84% said, it's my primary language. And that's unchanged from last year. Okay. Interesting. A lot of the analysis here was, how has this changed over the last year or so? Alright, so there's a lot of interesting trends to be pulled out from here. They said, what other languages do you use? Well, there was JavaScript, which has gone down and usage, there's bash, which has gone down and usage, there's HTML, which has gone down in usage relative to last year, and c++, which has gone down relative to last year. So the people who took the survey, the same number people, so they're primarily using Python, but the other languages that are employing all the ones who are which were popular seem to be going down. And I would say this means maybe people are starting to lean more on Python, those who use it. I don't know what conclusions we should draw there. There's also some growth in some other languages as well. I'm guessing. Interesting to see all the other ones down. Yeah. Well, this is down within the Python community, right? This is not necessarily down overall, right? Yeah. So I would say we should look at the Stack Overflow survey to like, really, more like even Stack Overflow trends. But if you look at that one, it's all the other languages are relatively down as well. So interesting. Now, a lot of the divide happens around web versus data science here. And so a lot of the questions said, Are you a web developer primarily, or a data scientist, primarily? And then, if you are, what type of tools do you use? So for example, if you are a data scientists use Python, of course. But you also make larger use of c++, Java, R and C sharp as a data scientist. But if you are a web developer, you make more use of sequel JavaScript and HTML, which is, you know, no surprise, because it's hard to write the web without HTML and JavaScript. Yeah. So those are pretty interesting. And they said, 58% of the people use Python for both work and personal projects. And they said, All right, well, what do you use it for? Right? So there is a bunch of answers. And we'll give you the top four, because the average person who filled this out chose four things that they primarily use Python for, right? Like I could use it for web, and DevOps, or something like that. Right? It's not exclusive. So the top four were data analysis, which is exactly the same as last year, 59% of the respondents said that web development, this is pretty interesting. 51% of the people said they do web development, but this is down 4% year over year. Interesting. That's a big drop. And I feel like web is a big component of what the Python space is. So I don't really know what to draw from that. Like maybe just more people are sort of backfilling in the data side side, maybe, maybe the tools that they use are better. So things like streamlet and stuff, they're like, Why don't have to do web now? I'm not really sure. Yeah, machine learning 40%, that's up 1%, DevOps 39%, that's down 4% as well, which is the same number as a web development. And these are kind of similar things. So what use Python for is pretty interesting. And then more broadly, it says, What do you use it the most for web data analysis and machine learning? was down 1% as the primary thing use it for data analysis is up 1% machine learning is up 2%. Also, not surprising, I guess. But good news on the Python versus legacy Python story. 90% of people are using Python three. That's good. That's a huge change from five years ago when it was like, Oh, yeah, that there's that one weird guy that uses Python three, like everyone else. Now. More of the data scientists are using Python three as a percentage.

05:00 over the web developers, I think that's because data science, the libraries, and the tools are changing a lot. It's not like you have legacy data science like, Oh, yeah, we're using that machine library from Machine Learning Library from, like, 10 years ago, because we just don't want to change. It's like those are fundamentally changed, and you have new tools. Whereas web development, I think there's a lot of folks that are like, yeah, we're still got that, you know, Django one app going on Python two or something. I'm not surprised. There's still some Python two, but iE 10%. still seems a little high. I think it's mostly legacy code. I'm not sure. But usually like a C Python for your legacy code. Alright. And web frameworks. flask, was kind of neck and neck with Django last year. Now it's like, totally head. So 48% flask developers and 44% Django developers and everything else is pretty small. Data Science. 63% of people are using NumPy 55%. pandas. 46% matplotlib. Here's what I threw in for you, Brian. Testing. 49% of people are using pi test. Yes. Yeah. 30% of the people are using the built in unit test. And 34% are using this other framework I heard and hadn't heard of before. It's called the none framework. Yeah, what that's unfortunately high. But it's funny that it's higher than unit test. Yeah, I know. That's what I thought I saw. Pretty funny. Alright, a couple more. And we'll be done with this one. So Cloud, the cloud platforms people are using AWS is in the lead. No surprise there with 55%. What did surprise me is Google Cloud GCP and whatnot, is actually number two at 33%. of digitalocean. Shout out to them is at 22%. And then Heroku is 20%. And Azure is 19%. So pretty interesting. And then how do you run your code in the cloud? In a production environment? Do you run in containers? VMs platform as a service like Heroku or something? So containers? 47% that's pretty high. VMs 46%? And then platform as a service is 25%. What editor do you use? Pi charm. 33% VS code 24% vim 9%. Everything else is like a super small margin after that. Yeah. And that's pretty much it. Yeah. So I threw in an extra one on there at the end. The one the containers. I thought containers was a was number two below VMs. Last year. And now it's a jump to number one. Oh, yeah. Like that's probably Kubernetes. Like, posted Kubernetes clusters, right? They'll throw in their Docker images into Oh, yeah. What's the last one taken seriously. And then the tool use, they also listed a handful of things of people using 90% people using version control. That's good. 80% right. There tests that's good. Many percent code linting. And 65%. People use type hinting, which actually is a little higher than I Oh, yeah. When you go, that's nice. And about half the people using code coverage tools. 52%. Yeah, thanks for adding that I'm very familiar with legacy code. And non legacy code, which I guess you call modern, but you're gonna take it to 11. What's up? What's up with this? Yes, this level? It's hypermodern. Ooh.

08:09 This is from Claudio. Cool name. But anyway, so he wrote a Actually, it's like a book. This is an incredible series of blog posts. And he actually writes them out. They're all linked together called hypermodern. Python, and set them up in six chapters. He's got set up testing, linting, typing, documentation. And then the last chapter is ci CD. And he just wrapped it up. I've been watching this and kinda I was gonna announce it when, when he was done. It's a really fun series to learn about, you know, you've learned some of the basics of Python, but you want to, like get some best practices and take it up a notch. And I think this, this is good. It's opinionated, of course, what I actually like opinionated things, and some of the opinions I don't quite follow. Like, he likes a pie envy. I'm not really fan of pie envy, but, but that's alright. Also, poetry uses poetry I use flit are usually, but anyway, he does recommend the source layout, which is good. But for setup, there's a whole bunch of neat stuff in here and some tools that like for linting, he talks about, like a black import order. bugbear, which is fun. I don't know if we've covered that. But there's a whole bunch of tools that I haven't even heard of like, safety and dessert, and data validation with dessert and marshmallow. So I'll have to look that sort of stuff up. That looks neat. So just a good run through it. In the CI CD section. He talked about using GitHub actions, and reporting your coverage with code Cove, uploading to pi pi and using test pi pi servers and documenting and read the docs. I think this is a fairly good representation for modern projects. So yeah, covers a lot of stuff that people probably should be doing and maybe haven't taken the time to set up or dig into. Yeah, and one of the things I want to highlight also is

10:00 Incredible use of pictures, the imagery that he uses on these posts are like the cicd section was some 1970s space station or, you know, Space Colony images, and they're beautiful. So it's worth it just for the pictures. Aren't they just for the pictures? They're great. Yeah, very nice one. Well, another thing that's really nice is data dog. Indeed, yep. This episode is Python bites is brought to you by data dog. And let me ask you a question. Do you have an app in production that is slower than you like? its performance all over the place? Sometimes fast, sometimes slow. Now, here's the important question. Do you know why? Well, with data dog, you will, you can troubleshoot your apps performance with data dogs into and tracing. Here's the detailed flame graphs to identify bottlenecks and latency in that finicky app of yours. be the hero that got the app back on track at your company. Get started today with a free trial of data dog by going to Python slash data dog and even get a free t shirt. So I really like my purple data dog t shirt. I wear a little too much probably adds cute. Awesome. Yeah. Thanks, David. All this next one that I want to cover. Brian is a little bit just kind of a fun thing is Dan Bader shared this with me the other day. He's like, Man, you got to check this out. Look at this thing. And like what is it? It's called the open AI jukebox. And so it's this AI that creates music.

11:33 It doesn't just create music, it creates different genres of music in the styles of certain artists with lyrics and musical accompaniment. So it's wild, right? I mean, I had you listened to a couple of the samples and folks out there, you should just go click on the Open AI box, link in the show notes and go to the curated samples and play a couple there. How would you classify them? Right? I'd classify them as you can tell their music.

12:05 There, none of these I would pick up one ago, I rush out and buy the album right away. Yeah, neither would I mean, they're to me, they sound like sort of bad recordings of an artist at you know, maybe taking on like a phone at a live concert where you've kind of got like, not a good audio setup. There's like a little bit of a I think I remember this song bit to it, even though there's no way Yeah, but it was created by an AI which is insane. So one, they've got a country song in the style of Alan Jackson is a country singer. And you could convince me that that was Alan Jackson singing that song if I didn't know better. Oh my god. I've never heard that song. And I'm not really super familiar with his music, but I kind of know what that guy sounds like. That's probably one of his songs because it sounds like he's singing it, which is crazy. It's got Elvis Presley. It's got Katy Perry. It's got some heavy made metal and the style of rage, I'm actually going to give you guys some other crazy stuff like alternative metal in the style of disturbed or a jazz like Ella Fitzgerald. And it's really interesting how accurate these things reproduce what those artists style of singing is their voice what their voice sounds like, the style of music they write. I mean, I would definitely not want to go listen to this to like, relax or whatever. But it's as a AI example. It's pretty crazy. Yeah, and one of the things I really appreciate is the the music while you're listening to it, it kind of it shows you the words, the lyrics, while you're Yeah, those lyrics highlight. Because some of them it's easy to understand, but others like the disturbed one, it's not so much easy to understand. It's like a highlighted thing, but my brain wanted to have see the little bouncy ball. Like, I don't know, what the kids music. Yeah, exactly. So the code for this is available on GitHub, the dataset, they use, they train the model to train it, they crawl the web and curated a dataset of 1.2 million songs I want that's got to use some bandwidth to get a hold of those. And it's 600,000 of those were in English, and then it paired those with lyrics and metadata from the lyric wiki. Okay, so when found the songs, I said, Okay, we need the written text so we can teach the model what the words are, so that it can make up its own lyrics, I guess. Yeah. And then it says the the top level Transformers trained on the task of predicting compressed audio tokens, and they provide additional information like the artists and genre for each song and said, they get two advantages from that. First, it reduces the entropy of the audio prediction. So the models are able to get better audio quality for the given style. And at generation time, they're able to like guide it in the style of their choosing, like no, no, we want some like hard rock versus Elvis or whatever. Anyway, if you're into AI, this seems like a pretty wild project to check out. Yeah, definitely. Yep. I'd say it's even clear.

15:00 Curious sorry, if you're curious, you should go check it out he should. And you should also check out this next post called The Curious Case of pythons context manager. So read one delamar went through this because context managers are really important. And when you start really making some elegant, really readable code, it's good to make use of context managers, if you're not familiar with they are anytime you see with like a with open file as for something like that, that's a, that's a use of a context manager. And the final one is probably the most notable one. So when you when you leave, what it does is it hooks up the like the keeping track of the data so that when you exit the block, it can clean up after itself. And so really handy things. I've seen a lot of different tutorials on how to write them. And a lot of them also they go through the class. So in general, the punch line is used the context manager context, live context manager decorator, and use a yield statement in the middle. And that will help you out. That's really the the working way to do it. But if you want to write you write a class based one with the Dunder init, Dunder, enter, Dunder exit. I think those are good examples. One of the things I liked about this tutorial is that it went through that, and it didn't seem artificial, it seemed it's like, this is one way to do it. But it went through it pretty quickly and went through some other stuff. And it's a pretty quick tutorial. But then it gets into some really fun stuff that I really appreciated, like context managers, as decorators, and writing your own and then create so that you're not using a decorator to make a context manager, you're actually creating a decorator that is a context manager. And that's pretty interesting thing. And then wrapping things and nesting context managers with block or with Stan, and that's called have like three of them instead of just one. And rather than like having three with blocks. Now I just missed that. Yeah, yeah. And I've just nested the width blocks. But you don't have to do that you can have them all on one line, which is cool. And there's a lot of cool little tips here. And then combining them. So like creating context manager, that's really a combination of two other context managers or more, more than one, which is nice. And then you kind of get into this right. And you think, well, what am I going to do with this? Where would I use it? And so there's three examples that he lists and shows. There's a context managers with the sequel alchemy session, which is a really cool idea. Yeah, I love this one. Yeah, yeah, how to use rollback and session so that when you're testing and stuff, you can just automatically undo the thing that was in the block. Sweet idea, using it for Exception Handling so that you can end that in combination with using it as a decorator is a neat idea to have a policy for how to deal with certain exceptions during parts of your code, and then just decorate those functions with exception handling. That's pretty cool. And then last one was talking about persistent parameters across HTTP requests. So it goes from very gentle to really deep into using this. Well, pretty quick, but it's really easy to read. Well, you know, I realized reading through this and I have not been using the decorator style nearly enough. I'm always like, Oh, just add a class with an enter exit type of thing. But yeah, the context managers just get a function and do yield. It's, it's pretty clean. Yeah. reminds me a lot of pi test fixtures. Yep. Yeah, for sure. That's a great article, I like it, I'll definitely have to study that one up. So previously, we spoke about NB Dev, which takes Jupiter notebooks and allows you to do a whole bunch of cool things with them, right, you can export the stuff into a script, you can have it strip out some of the metadata, the saved output, which is like the bane of all GitHub committed notebooks, because every time you rerun it, if it's taking variable input data, it's gonna have different metadata. So every run is a conflict, a merge conflict, which is no fun. So that things have a bunch of those types of things. But I met Robert sent over a message and said, hey, that's really cool. And B dev is great. But if you're looking just to do the stripping of the metadata, there's a project called NB strip out, which is pretty clever. And yeah, you can just set it up as like a get pre commit hook. And then every time you commit your stuff to GitHub, it automatically just strips it out, so that you never run into any of those merge conflicts. Oh, hook it up as a pre commit hook. That's a great idea. Yeah, yeah. So it's, you can either run it from the command line, or once you're in a GitHub repo and you PIP installed in B strip out, you just say envy strip out dash dash install. That's it now to get a pre commit hook and it'll take care of doing all the things for Jupiter notebooks. Oh, nice. Yeah, so basically is the same as going to Jupiter sync

20:00 All output in the UI, but it just does it as you try it, you know only as you save it to get out, which is cool. And then there's also a YouTube tutorial i we've said that it's really cool to have screenshots like you eyes. Well this is also a really nice so if you go actually to the pie listing, there's like a YouTube video right there that shows you like a four minute tutorial of like, why you should care about this. And I've done that it's super useful and like, Oh, this is kind of interesting. Let me watch it like, yep, that looks useful. You should talk about it. Yeah, nice. Yeah. Anyway, so if people are working with notebooks, and they're having this merge conflict issue is the saved output, people forgetting to clear the output for they committed, don't make them remember, Just do me strip out dash dash install, and you're golden. In Episode 179, we had Greta one, which was totally a lot of fun. One of the things he brought up was the 2020 Python language summit. And I was really interesting to listen about that. But I, if you want to read more about it, there's a pretty good write up of all the topics I talked about. So we're linking to a post that has links to other posts, we've got things like should all strings become f strings and using the pig parser in replacing see pythons parser with the pig parser. and different things like that. And even some of the lightning talks, just little snippets of kind of what was talked about and like a news article feed of what's going on, I find it really interesting and helpful to be able to kind of pay attention to what's going on at the language summit and what's going on with the language going forward to keep up with everything. I also wanted to bring up that there's there's been notifications recently about there's a voting coming up for the board board of directors for the Python Software Foundation. And so the PSF and some of the board of directors did a video on what this feels like and what does it mean to do that. So we're linking to the video and a link to the information about nominations. Because nominations are open for new board members up until the 31st of May. Now that's cool. Yeah, it is a little bit mysterious to me what the PSF Board of Director folks do. I mean, I can imagine, but I don't really know for sure. So it's cool that they've got a video on talking about that. Yeah. If you know people who should be part of it nominees. Oh, that's cool. Yeah. Assuming they want to be nominated.

22:24 Awesome. All right. Well, I guess that's it for all our items. You got anything extra by they don't I'm just been working on about you. two quick things. Follow up. We talked about Austin, the profiler, which is awesome. Just CPU profiling. Also mirror memory profiling. Remember, it had the two he had the web GUI had all the different user interfaces. And one of the ways you could view it one of the many ways you could view it was through speed scope, school. So window ballwin. Bauman, sent in a script that he'd created called pi speed scope, which will let you generate one of these speed scope files that you can then visualize from Python from reading. So anyway, I'll just link over to his GitHub repo for pi speed scope, which looks pretty cool. Also, I updated our search engine, I realized that when you search for stuff over a Python byte, it would give you good results, but it would just present them kind of as if they were all equal. So now it ranks things. So I added ranking to our little search engine. So now if you search for stuff, it's more likely to give you good results.

23:33 Well, that's cool. Without ranking, isn't it just pulling up random lists of things? Well, I mean, everything fit in there, right? Like if you search for it, everything that came up, had, like if you search for, I don't know Jupiter, it would have to have Jupiter in it if it came up as a result. But for example, if Jupiter was in the title or just like a random and here's an example Jupiter notebook versus the topic is about Jupiter like they would show up in whatever order they just came back now. It's like the stuff that's about it. More specifically shows up first. Oh, very helpful. Yeah, indeed. So hopefully, I use that. I know from it. That's why I did I'm like there's this thing I know it's in here and why are there so many results? It should be right at the top get the title is basically what I search for. Yeah, exactly. So it's, it's this for us, but people can benefit as well. Definitely. All right. You got some jokes. Yeah, these are these are definitely groaners. But they're submitted by friends of the show on Twitter, so I appreciate this. So a couple of them. Due to social distancing. I wonder how many projects are migrating to UDP and away from TLS. To avoid all the handshakes? Well, we have to write Yeah, can we get a computer virus next, a chef and a vagrant walk into a bar. Within a few seconds, it was identical to the last bar they went into.

23:33 I got it so vagrant manages virtual machines and chefs helps set up configure those environments. Got it.

23:33 could not hear. So, yeah, I like how the you're leaving somewhat understanding these as an exercise to the reader and partially I'm ruining their joy of solving the problem. Yeah.

23:33 No, it's all good. I took both of these took me one example a little bit of googling to understand beautiful I like them. They're they're definitely groaners, but they do kind of make you think for a second as well, which is great. Yeah. So thanks a lot. Yeah, you bet. I think we're done. Thanks. See ya. Thank you for listening to Python bytes. Follow the show on twitter at Python bytes. That's Python bytes as in be yts and get the full show notes at Python bytes at FM. If you have a news item you want featured just visit by thumb and send it our way. We're always on the lookout for sharing something cool. This is Brian Aachen and on behalf of myself and Michael Kennedy, thank you for listening and sharing this podcast with your friends and colleagues.

Back to show page