Transcript #146: Slay the dragon, learn the Python

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:04 This is episode 146, recorded September 4th, 2019.

00:09 I'm Michael Kennedy.

00:10 And I'm Brian Eichen.

00:11 And we have a special guest joining us this time.

00:13 Welcome back, Trey Hunter.

00:15 Thank you. It's nice to be here.

00:16 Yeah, it's great to have you here.

00:17 And it's always nice to get a new perspective.

00:20 And so it'll be fun to see what you're into.

00:22 We've got some cool Django stuff lined up because of that, I know.

00:25 But Brian, why don't you start off by telling us why there might be a forward slash in our Python function definition.

00:32 I sometimes forget that this is a, that 3.8 isn't out already, because I've been using Python 3.8 for a while now.

00:38 But it's coming soon.

00:40 And one of the features for 3.8 is positional only arguments.

00:45 So positional only arguments are a way, we have keyword only arguments that say you have to include the keyword in the argument.

00:53 we also have normal arguments that you can include the keyword or not.

00:59 In 3.8 they also add positional only which means you can't provide the keyword and they have to just be in the correct order, kind of like C functions. So my first thought for this was that the reason for this is probably to, you know, when you're integrating with a DLL or something, it might be just better for integration but We ran across this argument by a guy named Sanket, it's called positional arguments in Python, and it's just a nice little overview of what it is, but I like the example.

01:31 So the example, how you specify it is that you specify your normal arguments and then you put a slash as one of the arguments like a normal division slash.

01:40 That isn't one of your variables, it just denotes the difference between where the positional only stop.

01:46 The example he gave was a power function, And this is like a math type function or something like that makes a lot of sense where the names of the arguments really don't matter.

01:57 And the order kind of really does matter because if you, you know, like if you're thinking that the power function was X to the Y, but it's really Y to the X, that would be weird.

02:08 So specifying them in the correct order is kind of necessary. So anyway, I just wanted to throw this out there because it was a good example of a place where positional only arguments makes a lot of sense.

02:18 Yeah, it's interesting. You know, when I first look at it, I thought, well, what is the value of this?

02:23 But there are a couple interesting things.

02:24 One, functions that are called positional only without the keywords have some kind of significant speed up.

02:32 I can't remember if it's in 3.7 or if it's coming in 3.8 or something like that, like 20% quicker.

02:37 So there's some kind of performance optimization for this.

02:41 I also heard that it might make other implementations like PyPy and other things easier to have compatibility with some of the lower level stuff in there.

02:51 If you can have these positional only arguments.

02:54 So it's interesting.

02:55 It's a feature coming in three eight.

02:57 I'm curious to see how it gets used, right?

02:59 Like how many people actually go and use it.

03:01 We've had, like you said, the keyword only arguments, which is the star.

03:04 Not star args, but star comma argument.

03:09 whereas you have to do the keyword argument style.

03:12 And I don't see that very often, but I kind of like that as a feature for sure.

03:15 So maybe this one will grow on me.

03:17 Trey?

03:17 This is an interesting one because I'm pretty sure this was discussed when they were planning the feature that this is actually already possible at the C layer in Python.

03:27 And in fact, I brought this up while you were talking about PowBrian.

03:30 If you look at help on the built-in Pow function in Python, Python actually has the slash in the documentation because that slash is already there.

03:37 It means something.

03:38 You just can't use it as an actual Python syntax right now.

03:42 So you already cannot call the power functions built in with anything but positional arguments.

03:46 This just kind of reveals or allows this feature to be used by actual Python programmers outside of that C level.

03:52 So I mean, as a teacher, I kind of feel like it's yet another thing to teach, but it's also kind of consistency, right?

03:58 I can now explain the help documentation with an actual Python feature.

04:02 Yeah, it brings it up for all the regular developers, not just the core developers.

04:07 Okay, interesting.

04:09 Trey, I promised something about Django.

04:11 What you got for us?

04:12 Yeah, Django Stubs.

04:14 So this is type checking in Django.

04:17 So type annotations, which are, you know, a cool, I'd say, new Python 3 thing.

04:22 I guess they're not so new anymore, although they keep adding, you know, little edges to them.

04:26 It's a pretty new library, I think.

04:28 There's a blog post that I saw on this recently.

04:31 Let me see when it was written.

04:32 Yeah, it was, oh, just last week, I guess.

04:34 And because of that, I guess it's in beta in the sense that, you know, without actual users giving you feedback, it's only going to work for your use case, maybe, especially because Django is used so widely.

04:43 To be clear, I don't use type annotations in my Django code.

04:46 But you know, this library might allow me to eventually and I figure more people knowing about it means more users, more people may be fixing little edges in there because there's so much magic kind of, I mean, as much as they say there was a big magic removal in Django, there's just a lot of corners in Django to kind of fix with annotations.

05:02 Yeah, I think annotations are great.

05:04 they can definitely overwhelm the code and go crazy, but judicious use here and there, where you're crossing boundaries and stuff, like what the heck is that supposed to be?

05:15 Or even something as simple as like putting a type annotation on a request object.

05:19 So you can say request dot.

05:21 How do I get to headers again?

05:22 And what exactly is headers?

05:24 Is it a dictionary?

05:25 Is it a multi-dict?

05:25 It's like, what is it, right?

05:27 So those kinds of things I think are super valuable and I'm happy to see this.

05:31 - Right.

05:32 Well, and the edges between the code, if Django or a third party library is expecting something that what you're sending it works 80% of the time, but there's that one edge case that technically you're not using yet, and no one's testing that thing you don't even know about, it could break, but a type annotation might actually hint at a test you haven't written yet.

05:49 So are the Django type checking, is that going to extend to APIs as well?

05:56 Ooh, that's a good question. Yeah, I wonder about that. Because it says in the article, it talks about Django REST framework. And I didn't even think about the fact that you could I could probably piggyback on top of this for some kind of other type checking on top of it, but I don't know.

06:10 I think this is kind of more geared toward the actual programmer side of things, but I'm not certain.

06:15 - Okay, yeah, that's a good one.

06:16 I'm sure a lot of Django programmers out there will like it, especially if you're already using type annotations, be cool.

06:22 So, Trey, you talked about it as a teacher earlier, and I've done a lot of training myself.

06:27 I also have kids who I would like to have some programming literacy, not necessarily make them into little programmers, but have them be able to use programming like they might in math or something, or statistics, right, in some other aspects.

06:42 So there's this cool thing I found a while ago, but it's in the news again, called Code Combat.

06:47 Have either of you heard of this?

06:49 - I don't think so.

06:49 - Yeah, so Code Combat is a place that you can go, and it's aimed at teachers and educators, but honestly, anybody who wants to help somebody who's really early stage get involved in coding with Python, this is a great opportunity.

07:04 Let me tell you what it is.

07:05 So it's basically like this Dungeons and Dragons game or something along those lines.

07:11 And you go into these different worlds and you open them up and each world has maybe 40 or 50 puzzles or challenges you've got across to get through visually, right?

07:21 So you might go in there, it might be a dungeon, it's got like a maze and then a door and then an enemy.

07:27 And you might have to write some code that'll say, take my hero and move them over here, have them go down, then have them pick up this thing and then find the nearest enemy and attack them and then open the door and whatnot, right?

07:38 Maybe you do that just imperatively, maybe you write a while loop 'cause there's some kind of repeating pattern in the maze they've gotta do.

07:45 But what's really unique about this is it's very visual, it's got all the gamification.

07:51 You've won a badge, you've got a new skill, like you have to earn the while loop skill.

07:57 So you get like a little, I don't know, special boots that you run really fast that they call like, wow, I don't remember exactly what it is, but it's really like quick feedback for little kids.

08:06 What I like about it though, is unlike many of these things, you write real Python code.

08:11 There's a hero and you say hero dot move, right.

08:14 Or speed up or whatever it is.

08:17 You actually type it.

08:18 Now that sounds bad for like younger kids who are not great at typing or frustrating, but the auto complete is insane for this.

08:26 So if you're in the editor and you want to say, hero, find next enemy, you could just type E and hero, find next enemy comes up with the enemy variables.

08:36 Like it's crazy helpful on how much autocomplete that it'll give you.

08:41 And there's not that much that you can do.

08:42 And so it's super interesting.

08:44 My daughter at the time, I think she, this was 10.

08:47 She was all about it.

08:48 She made me get her a subscription and everything.

08:50 And she just played it probably for a month.

08:52 Then she kind of got tired of it, but she was really engaged.

08:55 And I just thought I'd throw that out there.

08:56 The reason it's back in the news now is they just got $6 million in VC funding to keep up this mission.

09:02 Oh, nice.

09:02 That's awesome.

09:03 Yeah.

09:03 So just a free resource.

09:05 Anyone out there who's a teacher or wants to help someone else learn, or maybe you want to learn yourself, right?

09:09 Like it's really constrained and simple Python, but it's real Python code you write to like go through the journey.

09:17 It's, it's pretty cool.

09:17 Right.

09:18 And that style of learning really shouldn't be written off.

09:20 Cause I mean, you're anchoring, you know, some kind of thing in the game that's exciting and fun with the knowledge of that little tidbit of some Python that you learned.

09:28 - Yeah, and it lets you work in a real way.

09:30 You don't have to go, well, I know you did some draggy, droppy stuff, and that was sort of programming, but now it's time to open up a blank text editor and you have no idea what you're doing.

09:40 You've been doing that the whole way.

09:42 So it's like, it really introduces you to what professional programming is.

09:48 It doesn't sugarcoat it too much for you, which is what I think's cool about it.

09:51 - Awesome. - Yeah.

09:52 Worth checking out.

09:53 All right, now before we get to the next one, I want to tell you quickly about our sponsor.

09:57 So this episode is brought to you by DigitalOcean.

10:00 Now DigitalOcean has previously had Postgres as a service.

10:03 That's something they announced not too long ago.

10:05 So if you're using Postgres, you can go sign up, say, "Hey, just take care of my database for me, "back it up, make sure it's running," all those kinds of things.

10:13 Well, now they're adding a few more things that go along with that.

10:16 They just announced that they have MySQL and Redis hosting as well.

10:20 So if you want to use Redis for a cache to make things go faster, check a box, you've got your own Redis cluster.

10:26 Or if you wanna use MySQL instead of Postgres, also an option now.

10:29 So check them out, pythonbytes.fm/digitalocean, $50 credit for new users, and lots of cool services coming online.

10:37 Speaking of stuff, services, and background things like Redis, Celery is definitely one that might land in that realm, Brian.

10:44 - I picked this topic, so there was an article by Nick Genetakis called "Four Use Cases for when to use Celery in a Flask application.

10:52 However, I think it's just really any web framework.

10:55 I'm not sure if this is specific to Flask.

10:58 But I like this sort of, okay, so this tool, and in this case it's Celery, when would you wanna use it?

11:05 I picked this because I'm not sure if he hit the nail on the head or not, and I wanted to pick you and Trey's brains on this.

11:12 So Celery, people don't know, it's a module that you can run with your web application that helps you run asynchronous or periodic schedules, code in the background.

11:23 So mostly just extra task multitasking, I guess.

11:27 - Sometimes it's super, it's very much the wrong thing to try to run that code right in the view handler, right?

11:33 Like if I've got some code that is going to take 15 seconds, you don't want to just have the user sit there and spend for 15 seconds.

11:40 You want to say, awesome, we're working on it, we kicked it off, enjoy your day, right?

11:46 The first example, I'm going to go through the four here.

11:49 The first example is to send out an email.

11:51 And this totally makes sense because you, all the work of actually sending it out, you don't actually care as long as you've, as the central task collected enough information, you can push that onto some background task to actually get the email sent out.

12:05 The user doesn't have to wait for that, right?

12:08 That totally makes sense.

12:10 The other three are connecting to third-party APIs, performing long-running tasks and running tasks on a schedule.

12:18 And the third-party API one, I don't actually quite get.

12:22 So I was hoping that one of you could explain if this is a good idea or a bad idea.

12:27 Any thoughts?

12:28 Yeah, I use it for all four of these things, actually.

12:30 I mean, in Python, I use it for all four because sending emails, that's the thing. Every Monday, 7 a.m. Pacific time, I have to send hundreds of emails.

12:40 And I actually was just wrestling with Sowie the last couple of days to get the email sending down from 10 minutes to 23 seconds, which was only possible because I realized I could make it a little bit more concurrent because you can kind of just throw more salary at the issue sometimes.

12:54 And then third party APIs, if you've got a thing like a, maybe not a credit card processing, but something that could happen, it could happen in the background, you just have to go notify some analytics thing or something, the user doesn't need to wait for that.

13:06 So you can just spin that off in a salary task on a schedule, you know, sending emails in my case was on a schedule as well.

13:12 And long running tasks, sending emails is actually a pretty long running task for me.

13:16 You know, it used to be 10 minutes.

13:18 Is there cases where like the user's waiting to see some something coming back, but you still want to use Celery?

13:24 Well, I would say sometimes and I'm pretty much with Trey on all these things.

13:29 I didn't use Celery. I just have like a real simple like background thread that I kick stuff off to because the work that I'm doing is 10, 20 seconds, whatever, like that's good enough to just not have more servers talking to each other, where there's more places they could fail.

13:44 But when you're talking to a third-party API, if somebody comes and says, "Buys a course on my website," I absolutely want to wait until I get a response, say, "Yes, that was approved," "No, it was denied," or whatever, right?

13:57 So that one, I would never kick purchasing, like a purchasing call or something like that, off to a background queue.

14:03 But what I may well do is, "Hey, I want to add this person to my mailing list." There's no reason that they don't want a response.

14:11 I think the thing is if they do not expect a response in the webpage, it's a totally good candidate for this background work, right?

14:19 Like if you say, "I can't log in, I need to reset my email." They don't expect that to happen.

14:24 You can just say, "Hey, yeah, great, we emailed you.

14:26 "Go check your email," right?

14:27 There's always like a five second delay anyway before it makes it through the email types.

14:32 So it's totally easy to kick that to something like send off that email, but get right back.

14:37 On my core site, if I put a new office hour event, there's a bunch of people, like thousands of people have signed up to get an email when I put that in there.

14:47 At first I tried to just send it out, but then it started timing out, 'cause it was taking so long.

14:51 So now that's a background job, and things like that.

14:53 - Okay, cool.

14:54 I guess it's a great article then, if all these are.

14:58 - I think the third party one needs a big caveat, right?

15:01 Like, do they expect a response from that third party, like a credit card purchase, or do they not care at all about that response, like they're now on your mailing list, right?

15:10 I think that's the dividing line there.

15:11 Trey, what do you think?

15:12 - I'd agree with that.

15:13 I've kind of been in the same camp in the sense of when I add someone to my mailing list, that API call, there's no, they don't even know about that API call.

15:20 They don't need to wait for it if that checkbox fails.

15:22 Whereas charging your credit card, like that's gotta be in the request response, that's my opinion.

15:26 - Right, that's the primary thing they're trying to do.

15:28 And like, it'll make them really nervous if they don't get some kind of feedback from that.

15:32 Yeah, so you probably should test it, huh, Trey?

15:33 Yes, you should.

15:35 The next thing we're gonna talk about is pytest, which is one of Brian's favorite things, obviously.

15:40 I feel like I stole this one from Brian 'cause it's about pytest.

15:43 So this is pytest steps is what I'm sharing, which is I didn't really know what it was when I first looked at it.

15:49 Then I just kind of scrolled through and saw the code.

15:51 And personally, I would use this kind of coming from a Django perspective of my functional test.

15:57 If I have a test where by necessity, I've got a lot of steps in it, I have to go to the login page, log in, click here, fill out another form, do a thing.

16:05 It's not a unit test, it's just like multi-step process.

16:08 It would be nice to kind of put little checks in there and say, did it pass this far?

16:13 Or if it didn't, don't show the rest of it as failed, show it as skipped, 'cause the failure was really at this part here.

16:18 We don't even know if the rest of it failed.

16:20 And so this is a way to kind of break up one longer test into steps.

16:24 And there's a whole bunch of different syntaxes for it.

16:27 I really like the generator syntax that's on this page.

16:29 The other ones I find a little bit less readable, but you're gonna really have to read through the page because there's kind of a lot of things buried in it as far as different ways you can use it.

16:37 - Yeah, I think it's very interesting.

16:39 - Yeah, you're always told, well, many people tell you that you should have one assert per test, you should be testing one thing, but also a lot of times, you've kind of got to build it up, right?

16:48 So this seems like a nice way to have the infrastructure break it apart.

16:52 And doesn't it do reporting based on what step it's at?

16:55 I feel like it kind of gives you better information when it fails.

17:00 Yeah, I think Brian would know more about this, but I think it just integrates with kind of the way pytest does its reporting.

17:05 And like, it pretends as if it has run multiple tests in a sense, kind of like how parameterization works, I think.

17:12 Yeah, it looks like the output just it looks just like a parameterization. So you have the, the test name is the same, but each step will be like listed in a bracket after the test name looks like it is piggybacking off the parameterization I think it's a nice implementation.

17:29 I'm going to have to play with it.

17:30 My fear in working with teams is it is helpful to try to focus in and have a test that really focuses on something.

17:38 And workflow tests are a red flag, but we do have workflow tests.

17:42 Those are real things.

17:44 And so this is a way to have workflow tests be in place.

17:47 I would just say probably don't abuse it too much, but in some cases it might be the perfect hammer.

17:53 Yeah, if you've got to have it, you might as well do it with a nice tool.

17:56 Right, right. I feel like this is, you know, it's a smaller use case in terms of those very few workflow tests, but it's still useful.

18:02 Yeah. All right, let me wrap us up with this last main topic here.

18:06 And now before I tell you all what it is, let me tell you the scenario which it makes sense to use because it came out of law and you might think, "Oh, I'm not a lawyer, so this is irrelevant to me." And it may be, but it might not be.

18:17 So the idea is, with this thing I'm about to tell you about, if you want to conduct a survey, think SurveyMonkey or something like that, or you want to create kind of a self-guided interview process, but you want a lot of workflow and control.

18:32 There's this really creative project called DocAssemble, and it comes out of the legal space.

18:38 There's a guy named Jonathan Pyle.

18:39 I actually interviewed him for Talk Python on this, and that's coming out in a couple weeks, something like that, probably about the same time this episode ships, actually.

18:47 And the idea is he worked for a public defendant-type place where it wasn't a big fancy law firm, but they needed to talk to lots of people and do like legally valid interviews and gathering up information.

19:01 So he wrote this open source thing called DocAssemble, and it can be used anytime you kind of want to conduct interviews or surveys that are way better than say SurveyMonkey, where you have lots of flow.

19:13 The way you can create the workflow is wild.

19:16 You write basic Python code, and then it evaluates the Python.

19:23 the execution path.

19:24 So I could say, if user.isUSCitizen, then something else, something else, and it will actually realize, okay, I have to first determine if they're a US citizen, and only if they say yes, do I have to ask them this other question that's inside that if statement.

19:42 And the way it does it is really wild.

19:44 So this thing is packed with all sorts of features.

19:47 It has sends SMS, email, does OCR.

19:51 It has like plugins, it has background tasks.

19:55 We just talked about Celery.

19:56 It does a bunch of background stuff.

19:57 I think actually using Celery, you define interviews in YAML and you write the flow in basic questions in Python.

20:05 And it's just a really interesting way to make like advanced interview workflow survey type thing.

20:12 So people might check that out.

20:14 It's open source and it looks pretty cool.

20:15 - When I looked at this, I definitely wanna try to come up with a reason to use it 'cause it looks kind of fun.

20:20 You can even go and there's like a demo, like try it or take a survey or whatever.

20:24 And when you do that, there's at the top, there's something that shows the source for each question.

20:28 It's pretty cool.

20:29 It'll show you the actual YAML, I think the Python code that's making it go, but it also shows you things like the reading level.

20:37 There's a bunch of different measures for how hard or complex are these questions and terms.

20:41 And so you can get it like, oh, this question is too complicated.

20:44 How do we rephrase it?

20:45 And there's like just tons of little edge cases that are super well polished.

20:49 Like there's a plugin for Word docs, so you can write them there instead of in YAML, all kinds of stuff.

20:54 - I'm actually gonna have to use this.

20:55 This is, or at least look at it, 'cause this is something I've needed in asking like surveys for my Python morsels customers.

21:02 If you're using it professionally versus if you're a hobbyist, you've got a different set of concerns and a different set of questions.

21:07 - Yeah, exactly.

21:08 In Python, it'd be like, if user.ishobbyist, colon, and like that's how you write the workflow.

21:13 It's really cool, this thing.

21:14 - Awesome.

21:15 - All right, cool.

21:16 Well, that's our main topics.

21:17 Do you guys have any extra stuff you wanna just throw out there real quick?

21:19 - I don't.

21:20 - All right, well, good thing I brought some then.

21:22 - Yeah.

21:23 - Actually, so really quickly, we are at 194,740 projects on PyPI, which means we are on the edge of 200,000 packages on PyPI, which is, I think, pretty awesome.

21:37 So very exciting to have that growing.

21:40 I remember when I was blown away that there were like 70,000 packages.

21:43 - Yeah. - That's awesome.

21:44 - Yeah, super cool.

21:44 So I'm linking to a tweet that Raymond Hettinger put out around that.

21:48 Also, really quick, NumPy 1170 was released.

21:52 There's a bunch of mathematical stuff that if you care about, cool new features and improvements.

21:57 Probably too specific to go into.

21:59 And also, Brian, you talked about Python 3.8.

22:01 Python 3.8 Beta 4 is now available for those who want to test on the slightly closer to finished version.

22:07 - Yeah, I love 3.8.

22:08 - All right, I've got a joke for you.

22:09 We haven't told this one yet, have we, Brian?

22:11 - It seems familiar here.

22:13 - Well. - But let's do it anyway.

22:14 - We're doing it anyway, because it was fun to have it go through your office as well.

22:18 - All right, knock knock.

22:19 - Who's there?

22:20 - Recursive function.

22:21 - Recursive function who?

22:22 - Knock knock.

22:22 (laughing)

22:24 Yeah, how long can we go for this?

22:26 So it's a good joke, but you only understand it if you know recursion, right?

22:29 So how do you learn about it?

22:30 Do you Google it, Brian?

22:31 - Yeah, you can Google recursion, and then you'll see a link that says, did you mean recursion?

22:35 (laughing)

22:37 - That's awesome.

22:39 All right, I don't know who put this next one here, but I really like it.

22:41 You have to do it, though.

22:42 - Okay, yeah, I put it down.

22:43 So, hey, what's your address?

22:45 - 192.7.7.3.

22:48 - No, your local address.

22:49 - 127.0.0.1.

22:51 - No, your physical address.

22:53 - A-C-5-E-C.

22:56 (laughing)

22:57 Give us a Mac address.

22:58 - I love this.

23:01 - Yeah, that's my Mac address, yeah.

23:02 - Yeah, yeah.

23:03 So, thanks.

23:04 - That's a good one.

23:05 All right, well, those were some good laughs, and Trey and Brian, thank you both for being here today.

23:09 - Thanks for having me. - Thank you.

23:10 - Thank you for listening to Python Bytes.

23:12 Follow the show on Twitter via @pythonbytes.

23:15 that's Python Bytes as in B-Y-T-E-S.

23:18 And get the full show notes at pythonbytes.fm.

23:21 If you have a news item you want featured, just visit pythonbytes.fm and send it our way.

23:25 We're always on the lookout for sharing something cool.

23:28 On behalf of myself and Brian Okken, this is Michael Kennedy.

23:31 Thank you for listening and sharing this podcast with your friends and colleagues.