« Return to show page
Transcript for Episode #146:
Slay the dragon, learn the Python
0:00 KENNEDY: Hello, and welcome to Python Bytes where we deliver Python news and headlines directly to your earbuds. This is Episode 146, recorded September 4, 2019. I'm Michael Kennedy.
0:10 OKKEN: And I'm Brian Okken.
0:11 KENNEDY: And we have a special guest joining us this time. Welcome back, Trey Hunner.
0:15 HUNNER: Thank you, it's nice to be here.
0:16 KENNEDY: Yeah, it's great to have you here, and it's always nice to get a new perspective and so it'll be fun to see what you're into. We've got some cool Django stuff lined up because of that, I know, but Brian, why don't you start off by telling us why there might be a forward slash in our Python function definition?
0:32 OKKEN: I sometimes forget that 3.8 isn't out already, because I've been using Python 3.8 for a while now. But it's coming soon, and one of the features for 3.8 is positional-only arguments. So positional-only arguments are a way, we have keyword-only arguments that say you have to include the keyword in the argument. We also have normal arguments that you can include the keyword or not. In 3.8, they also add positional-only, which means you can't provide the keyword and they have to just be in the correct order, kind of like C functions. So my first thought for this was that the reason for this is probably to, when you're integrating with a DLL or something, it might be just better for integration, but we ran across this argument by a guy named Sanket. It's called "Positional-only Arguments in Python" and it's just a nice little overview of what it is, but I like the examples. So the example, how you specify it is that you specify your normal arguments, and then you put a slash as one of the arguments, like a normal division slash, that isn't one of your variables. It just denotes the difference between where the positional-onlys stop. And the example he gave was a power function. And this is like a math-type function or something like that, makes a lot of sense, where the names of the arguments really don't matter, and the order kind of really does matter, because if you're thinking that the power function was x to the y, but it's really y to the x, that would be weird, so specifying 'em in the correct order is kind of necessary, so anyway, I just wanted to throw this out there because it was a good example of a place where positional-only arguments makes a lot of sense.
2:19 KENNEDY: Yeah, it's interesting. You know, when I first looked at it, I thought, well, what is the value of this? But there are a couple interesting things. One, functions that are called positional-only, without the keywords, have some kind of significant speedup, I can't remember if it was in 3.7 or if it's coming in 3.8 or something like that, like 20% quicker. So there's some kind of performance optimization for this. I also have heard that it might make other implementations like PyPy and other things easier to have compatibility with some of the lower-level stuff in there if you can have these positional-only arguments. So it's interesting. It's a feature coming in 3.8. I'm curious to see how it gets used, right? Like how many people will actually go and use it. We've had, like you said, the keyword-only arguments, which is the *, not *args but I kind of like that as a feature for sure, so maybe this one'll grow on me. Trey?
3:18 HUNNER: This is an interesting one, because I'm pretty sure this was discussed when they were planning the feature, that this is actually possible at the C layer in Python, and in fact, I brought this up while you were talking about pow, Brian. If you look at help on the built-in pow function in Python, Python, it actually has the slash in the documentation, because that slash is already there. It means something, you just can't use it as an actual Python syntax right now. So you already cannot call the pow function that's built in with anything but positional arguments. This just kind of reveals or allows this feature to be used by actual python programmers, outside of that C level. So I mean, as a teacher, I kind of feel like it's yet another thing to teach, but it's also kind of consistency, right? I can now explain the help documentation with an actual Python feature.
4:02 KENNEDY: Yeah, it makes it...
4:02 OKKEN: Nice.
4:03 KENNEDY: Yeah, like brings it up for all the regular developers, not just the core developers. Okay, interesting. Trey, I promised something about Django. What d'you got for us?
4:12 HUNNER: Yeah, django-stubs. So this is type checking in Django, so type annotations, which are, you know, a cool, I'd say new Python 3 thing. I guess they're not so new anymore, although they keep adding little edges to them. It's a pretty new library, I think. There's a blog post that I saw on this recently. Let me see when it was written. Yeah, it was, oh, just last week, I guess. And because of that, I guess it's in beta, in a sense that without actual users giving you feedback, it's only going to work for your use case maybe, especially 'cause Django's used so widely. To be clear, I don't use type annotations in my Django code, but this library might allow me to eventually, and I figure more people knowing about it means more users, more people maybe fixing little edges in there, 'cause there's so much magic kind of, I mean, as much as they say there was a big magic removal in Django, there's just a lot of corners in Django to kind of fix with annotations.
5:02 KENNEDY: Yeah, I think annotations are great. They can definitely overwhelm the code and go crazy, but judicious use here and there where you're crossing boundaries and stuff, you're like what the heck is that supposed to be? Or even something as simple as putting a type annotation on a request object, so you can say request dot, how do I get to headers again and what exactly is headers? Is it a dictionary, is it a multidict, what is it, right? So those kinds of things I think are super valuable, and I'm happy to see this.
5:32 HUNNER: Right, well, and the edges between the code. You know, if Django or a third-party library is expecting something that what you're sending it works 80% of the time, but there's that one edge case that technically you're not using yet, and no one's testing that thing you don't even know about, it could break, but a type annotation might actually hint at a test you haven't written yet.
5:49 OKKEN: So are the Django type checking, is that going to extend to APIs as well?
5:56 HUNNER: Ooh, that's a good question. Yeah, I wonder about that, 'cause it says in the article, it talks about Django REST framework, and I didn't even think about the fact that you could probably piggy back on top of this for some kind of other type checking on top of it, but I don't know, I think it is kind of more geared toward the actual programmer side of things, but I'm not certain.
6:16 KENNEDY: Yeah, that's a good one. I'm sure a lot of Django programmers out there will like, especially if you're already using type annotations, cool. So Trey, you talked about as a teacher earlier, and I've done a lot of training myself. I also have kids who I would like to have some programming literacy, not necessarily make them into little programmers, but have them be able to use programming like they might in math or something, or statistics, right, in some other aspects. So there's this cool think that I found a while ago, but it's in the news again, called "Code Combat." Have either of you heard of this?
6:49 OKKEN: I don't think so.
6:50 KENNEDY: Yeah, so "Code Combat" is a place that you can go and it's aimed at teachers and educators, but honestly, anybody who wants to help somebody who's really early stage get involved in coding with Python, this is a great opportunity. Let me tell you what it is. So it's basically like this "Dungeons and Dragons" game or something along those lines, and you go into these different worlds, and you open them up, and each world has maybe 40 or 50 puzzles or challenges you've got to cross to get through, visually, right? So you might go in there, there might be a dungeon. It's got like a maze and then a door, and then an enemy, and you might have to write some code that'll say take my hero and move them over here, have them go down then and have them pick up this thing and then find the nearest enemy and attack them and then open the door and whatnot, right? Maybe you do that just imperatively, maybe you write a while loop, 'cause there's some kind of repeating pattern in the maze they've got to do, but what's really unique about this is it's very visual, it's got all the gamification, you've won a badge, you've got a new skill, like you have to earn the while loop skill, so you get like a little, I don't know, special boots that let you run really fast and they call like while loop. I don't remember exactly what it is, but it's really quick feedback for little kids. What I like about it, though, is unlike many of these things, you write real Python code. There's a hero and you say, hero.move_right or speed up, or whatever it is. You actually type it. Now that sounds bad for younger kids who are not great at typing, or frustrating, but the autocomplete is insane for this. So if you're in the editor, and you want to say, hero, find next enemy, you could just type e and hero, find next enemy comes up with the enemy variable. It's crazy helpful in how much autocomplete that it'll give you, and there's not that much that you can do, and so it's super interesting. My daughter, who at the time I think she did this, was 10. She was all about it. She made me get her a subscription and everything, and she just played it probably for a month. Then she kind of got tired of it, but she was really engaged, and I just thought I'd throw that out there. The reason it's back in the news now is they just got $6 million in VC funding to keep up this mission.
9:02 OKKEN: Oh, nice.
9:03 HUNNER: Yeah, it's awesome.
9:03 KENNEDY: Yeah, so just a free resource. Anyone out there who's a teacher or wants to help someone else learn, or maybe you want to learn yourself, right, like it's really constrained and simple Python, but it's real Python code you write to go through the journey. It's pretty cool.
9:18 HUNNER: Right. And that style of learning really shouldn't be written off, 'cause I mean, you're anchoring some kind of thing in the game that's exciting and fun with the knowledge of that little tidbit of some Python that you learned.
9:28 KENNEDY: Yeah, and it lets you learn in a real way. You don't have to go, well, I know you did some draggy droppy stuff, and that was sort of programming, but now it's time to open up a blank text editor and you have no idea what you're doing. You've been doing that the whole way, so it really introduces you to what professional programming is. It doesn't sugarcoat it too much for you, which is what I think's cool about it.
9:51 HUNNER: Awesome.
9:52 KENNEDY: Yeah, worth checking out. All right, now before we get to the next one, I want to tell you quickly about our sponsor. So this episode is brought to you by DigitalOcean. Now DigitalOcean has previously had Postgres as a service. That's something they announced not too long ago, so if you're using Postgres, you can go sign up, say, hey, just take care of my database for me, back it up, make sure it's running, all those kinds of things. Well now, they're adding a few more things that go along with that. They just announced that they have MySQL and Redis hosting as well, so if you want to use Redis for a cache to make things go faster, check a box, and you've got your own Redis cluster. Or if you want to use MySQL instead of Postgres, also an option now. So check them out, pythonbytes.fm/digitalocean, $50 credit for new users, and lots of cool services coming online. Speaking of stuff, services and background things like Redis, Celery is definitely one that might land in that realm, Brian.
10:44 OKKEN: I picked this topic, so there was an article by Nick Janetakis called "Four Use Cases for When to Use Celery in a Flask Application." However, I think it's just really any web framework. I'm not sure if this is specific to Flask, but I like this sort of a okay, so this tool, in this case it's Celery, when would you want to use it? I picked this because I'm not sure if he hit the nail on the head or not, and I wanted to pick you and Trey's brains on this. So Celery, if people don't know, it's a module that you can run with your web application, and it helps you run asynchronous or periodic schedules code in the background, so mostly just extra task multitasking, I guess.
11:27 KENNEDY: Sometimes it's very much the wrong thing to try to run that code right in the view handler, right? Like if I've got some code that is going to take 15 seconds, you don't want to just have the user sit there and spin for 15 seconds, you want to say awesome, we're working on it, we kicked it off. Enjoy your day, right?
11:46 OKKEN: The first example, I'm going to go through the four here, the first example is just to send out an email, and this totally makes sense, because you, all the work of actually sending it out, you don't actually care as long as the central task collected enough information, you can push that onto some background task to actually get the email sent out. The user doesn't have to wait for that, right? That totally makes sense. The other three are connecting to third-party APIs, performing long-running tasks, and running tasks on a schedule. And the third-party API one, I don't actually quite get, so I was hoping that one of you could explain if this is a good idea or a bad idea, or, any thoughts?
12:28 HUNNER: Yeah, I use it for all four of these things, actually.
12:30 OKKEN: Oh, okay.
12:31 HUNNER: In Python I use it sending emails, that's the thing every Monday, 7:00 a.m. Pacific Time, I have to send hundreds of emails, and I actually was just wrestling with Celery the last couple days to get the email sending down from 10 minutes to 23 seconds, which was only possible because I realized I could make it a little bit more concurrent, 'cause you can kind of just throw more Celery at the issue sometimes. And then, third-party APIs, if you've got a thing, like a, maybe not a credit card processing, but something that could happen, it could happen in the background, you just have to go notify some analytics thing or something. The user doesn't need to wait for that, so you can just spin that off in a Celery task on a schedule, sending emails in my case was on a schedule as well, and long-running tasks, sending emails is actually a pretty long-running task for me, you know, used to be 10 minutes.
13:17 OKKEN: Is there cases where the user's waiting to see something coming back, but you still want to use Celery?
13:24 KENNEDY: Well, I would say sometimes, and I'm pretty much with Trey on all of these things. I don't use Celery. I just have like a real simple background thread that I kick stuff off to, 'cause the work that I'm doing is 10, 20 seconds, whatever, that's good enough to just not have more servers talking to each other, where there's more places they could fail. But when you're talking to a third-party API, if somebody comes and say, buys a course on my website, I absolutely want to wait until I get response, say, yes, that was approved, or no, it was denied, or whatever, right? So that one I would never kick purchasing, like a purchasing call or something like that, off to a background queue. But what I may well do is hey, I want to add this person to my mailing list. There's no reason that, they don't want a response. I think the thing is if they do not expect a response in the webpage, it's a totally good candidate for this background work, right? If you say, I can't log in, I need to reset my email, they don't expect that to happen. You can just say hey, yeah, great, we emailed you. Go check your email, right? There's always like a five-second delay anyway before it makes it through the email pipes. So it's totally easy to kick that, too, with something like Celery, send off that email but get right back. On my course site, if I put a new office hour event, there's a bunch of people, like thousands of people have signed up to get an email when I put that in there. At first, I tried to just send it out, but then it started timing out 'cause it was taking so long. So now that's a background job, and things like that.
14:53 OKKEN: Okay, cool. I guess it's a great article, then, if all these are...
14:57 KENNEDY: I think the third-party one needs a big caveat, right? Do they expect a response from that third party, like a credit card purchase, or do they not care at all about that response, like they're now on your mailing list, right? I think that's the dividing line there. Trey, what do you think?
15:12 HUNNER: I'd agree with that. I'm kind of in the same camp in the sense of when I add someone to my mailing list, that API call, they don't even know about that API call. They don't need to wait for it if that checkbox fails, whereas charging your credit card, that's got to be in the request response in my opinion.
15:26 KENNEDY: Right, that's the primary thing they're trying to do. It'll make them really nervous if they don't get some kind of feedback from that. Yeah, so you probably should test it, huh, Trey?
15:33 HUNNER: Yes, you should. The next thing we're going to talk about is pytest, which is one of Brian's favorite things, obviously. I feel like I stole this one from Brian, 'cause it's about pytest, but so this is pytest-steps is what I'm sharing, which is, I didn't really know what it was when I first looked at it, then I just kind of scrolled through and saw the code, and personally, I would use this, kind of coming from a Django perspective of my functional test. If I have a test where by necessity, I've got a lot of steps in it, you know, I have to go to the login page, log in, click here, fill out another form, do a thing, it's not a unit test, it's this multi-step process, it would be nice to kind of put little checks in there and say, did it pass this far? Or if it didn't, don't show the rest of it as failed, show it as skipped, 'cause the failure was really at this part here. We don't even know if the rest of it failed. And so this is a way to kind of break up one longer test into steps, and there's a whole bunch of different syntaxes for it. I really like the generator syntax that's on this page. The other ones I find a little bit less readable, but you're going to really have to read through the page, because there's kind of a lot of things buried in it as far as different ways you can use it.
16:38 OKKEN: Yeah, I think it's very interesting.
16:39 KENNEDY: Yeah, you're always told, well, many people tell you that you should have one assert per test, or you should be testing one thing, but also a lot of times, you've kind of got to build it up, right, so this seems like a nice way to have the infrastructure break it apart. And doesn't it do reporting based on what step it's at? I feel like it kind of gives you better information when it fails.
17:00 HUNNER: Yeah, I think Brian would know more about this, but I think it just integrates with kind of the way pytest does its reporting, and it pretends as if it has run multiple tests in a sense, kind of like how parameterization works, I think.
17:13 OKKEN: Yeah, it looks like the output just, it looks just like a parameterization, so you have the test name is the same, but each step will be listed in a bracket after the test name. Looks like it is piggy backing off the parameterization stuff. I think it's a nice implementation. I'm going to have to play with it. My fear in working with the teams is it is helpful to try to focus in and have a test that really focuses in on something and workflow tests are a red flag, but we do have workflow tests. Those are real things, and so this is a way to have workflow tests be in place. I would just say probably don't abuse it too much, but in some cases, it might be the perfect hammer.
17:54 KENNEDY: Yeah, if you've got to have it, you might as well do it with a nice too.
17:57 HUNNER: Right, right. I feel like this is, you know, it's a smaller use case in terms of those very few workflow tests, but still useful.
18:03 KENNEDY: All right, let me wrap us up with this last main topic here, and now before I tell you all what it is, let me tell you the scenario which it makes sense to use, because it came out of law, and you might think, well, I'm not a lawyer, so this is irrelevant to me, and it may be, but it might not be. So the idea is, with this thing I'm about to tell you about, if you want to conduct a survey, think SurveyMonkey or something like that, or you want to create kind of a self-guided interview process, but you want a lot of workflow and control, there's this really creative project called docassemble, and it comes out of the legal space. There's a guy named Jonathan Pile. I actually interviewed him for "Talk Python" on this, and that's coming out in a couple of weeks, something like that. Probably about the same time this episode ships, actually. And the idea is, he worked for a public defendant type place, where it wasn't a big fancy law firm, but they needed to talk to lots of people and do legally valid interviews and gathering up information, so he wrote this open-source thing called docassemble, and it can be used any time you kind of want to conduct interviews or surveys that are way better than, say, SurveyMonkey, where you have lots of flow. The way you can create the workflow is wild. You write basic Python code, and then it evaluates the Python, the execution path, so I could say, if user.is_us_citizen, then something, else, something else, and it will actually realize okay, I have to first determine if they're a US citizen, and only if they say yes do I have to ask them this other question that's inside that if statement. And the way it does it is really wild. So this thing is packed with all sorts of features. It sends SMS, email, does OCR. It has plugins. It has background tasks. We just talked about Celery. It does a bunch of background stuff, I think actually using Celery. You define the interviews in YAML and you write the flow in basic questions in Python, and it's just a really interesting way to make advanced interview workflow survey type things. So people might check that out. It's open source, and it looks pretty cool.
20:15 OKKEN: When I looked at this, I definitely want to try to come up with a reason to use it, 'cause it looks kind of fun.
20:20 KENNEDY: You can even go, and there's like a demo like try it or take a survey or whatever, and when you do that, there's at the top, there's something that shows the source for each question. It's pretty cool, it'll show you the actual YAML, and I think the Python code that's making it go, but it also shows you things like the reading level. There's a bunch of different measures for how hard or complex are these questions and terms, and so you can get it like, oh, this question's too complicated, how do we rephrase it. There's just tons of little edge cases that are super well polished. There's a plugin for Word docs so you can write them there instead of in YAML, all kinds of stuff.
20:54 HUNNER: I'm actually going to have to use this. Or at least look at it, 'cause this is something I've needed in asking surveys for my Python Morsels customers. If you're using it professionally versus if you're a hobbyist, you've got a different set of concerns and a different...
21:07 KENNEDY: Yeah exactly.
21:07 HUNNER: questions.
21:08 KENNEDY: In Python, there'd be like if user.is_hobbyist:, and like that's how you write the workflow. It's really cool, this thing.
21:14 HUNNER: Awesome.
21:15 KENNEDY: All right, cool. Well, that's our main topics. Do you guys hae any extra stuff you want to just throw out there real quick?
21:19 OKKEN: I don't.
21:20 KENNEDY: All right, well, good thing I brought some, then. Actually, so really quickly, we are at 194,740 projects on PyPI, which means we are on the edge of 200,000 packages on PyPI, which is, I think, pretty awesome. So very exciting to have that growing. I remember when I was blown away that there were like 70,000 packages.
21:43 HUNNER: That's awesome.
21:43 KENNEDY: Super cool, so I'm linking to a tweet that Raymond Hettinger put out around that. Also really quick, NumPy 1.17.0 was released. There's a bunch of mathematical stuff that if you care about cool new features and improvements probably too specific to go into, and also, Brian, you talked about Python 3.8. Python 3.8 beta 4 is now available for those who want to test on the slightly-closer-to-finished version.
22:07 OKKEN: Yeah, I love 3.8.
22:08 KENNEDY: All right, I've got a joke for you. We haven't told this one yet, have we, Brian?
22:11 OKKEN: It seems familiar.
22:13 KENNEDY: Well...
22:13 OKKEN: But let's do it anyway.
22:14 KENNEDY: But we're doing it anyway, because it was fun to have it go through your office as well. All right, knock knock.
22:19 OKKEN: Who's there?
22:20 KENNEDY: Recursive function.
22:21 OKKEN: Recursive function who?
22:21 KENNEDY: Knock knock. Yeah, how long can we go for this? So it's a good joke, but you only understand it if you know recursion, right? So how do you learn about it? Do you Google it, Brian?
22:32 OKKEN: Yeah, yeah, you can Google recursion, and then you'll see a link that says, "Did you mean recursion?"
22:37 KENNEDY: That's awesome. All right, I don't know who put this next one here, but I really like it. You have to do it, though.
22:42 OKKEN: Okay, yeah, I put it down. So hey, what's your address.
22:45 KENNEDY: 126.96.36.199.
22:47 OKKEN: No, your local address.
22:49 KENNEDY: 127.0.0.1.
22:51 OKKEN: No, your physical address.
22:52 KENNEDY: AC5EC.
23:00 OKKEN: Yeah, I love those.
23:01 KENNEDY: Yeah, that's my Mac address, yeah. Yeah, yeah, so thanks. That's a good one. All right, well, those were some good laughs, and Trey and Brian, thank you both for being here today.
23:09 HUNNER: Thanks for having me.
23:09 OKKEN: Thank you.
23:10 KENNEDY: Thank you for listening to Python Bytes. Follow the show on Twitter via @PythonBytes. That's Python Bytes as in B-Y-T-E-S. And get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Okken, this is Michel Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.