Transcript #88: Python has brought computer programming to a vast new audience

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 88, recorded July 26, 2018. I'm Michael Kennedy.

00:10 And I'm Brian Okken.

00:11 Brian, how are you doing?

00:12 I'm doing great.

00:13 Yeah, it's a ton of good stuff that I think you found this week, and I'm looking forward to share what I got as well.

00:18 It's never a boring week in the Python world, is it?

00:20 No, it isn't.

00:21 No. Well, it's definitely not boring if your app has problems. You need to track them down.

00:27 So if you're in that space, you might want to check out Datadog over at pythonbytes.fm/Datadog.

00:33 Tell you more about them later.

00:35 Right now, I want to know about documenting code.

00:37 We do hear a lot about, I think we hear a lot about making sure that your code documentation is good.

00:45 There's the read the docs and make the docs effort, or I guess it's write the docs.

00:50 And there's quite a bit of that.

00:52 And I think a lot of that is focused on the end user user documentation.

00:57 And that's definitely important.

00:58 But there's an article on, I think it was on RealPython, called Documenting Python Code, A Complete Guide.

01:06 It's an article that describes really why you should comment and document your code.

01:10 It talks about the differences between comments and doc strings and different separate documentation.

01:17 But one of the things that I want to zoom in on is the code comments.

01:21 And I don't think very many people get really how, if they think about their comments that they put in their code, how effective they can be.

01:29 So I want to kind of zoom in on some of the things that they cover in this article on code comments.

01:34 So code comments are those things that start with a pound sign or hash.

01:37 And they're not really part of your runtime.

01:41 You can't get a hold of them at runtime.

01:43 They're intended for you and other maintainers and other developers of the code.

01:48 And some of the things that I've used them for, but I never really saw anybody point out before, is before you even start your code, you can use comments to plan what you're going to write.

01:57 And you can also, once you've written your code, review to make sure the code really matches what you planned to do.

02:04 And then sometimes those kind of comments are best to be, if your code is clear enough, just go ahead and delete those when you're done.

02:11 I like that.

02:12 I end up doing that a lot myself.

02:13 I'm like, okay, I'm not entirely sure how to do this.

02:15 So I'll sketch this out in comments.

02:17 Like, here's the five steps I think I got to do.

02:19 And then I'll just take them one at a time.

02:20 And then I always delete those comments.

02:22 Almost always.

02:23 Yeah.

02:23 The deleting part is important because if you don't do that, then you get the common pitfall of a code comment that says, retrieve web page.

02:32 And then the very next line is one line of code that's a request or something.

02:36 And it's obvious.

02:37 If it's not obvious, using those to explain intent, don't explain really how it's done, but explain why you're doing it.

02:44 And then sometimes you'll have like a complicated math algorithm or something that's fairly terse if you need to explain that better.

02:51 But we're trying to get as few comments as possible, minimizing those because you want to make your code pretty clear to begin with.

02:59 But one of the other things that I think is I've just started using quite a bit is the to-do feature.

03:04 So using code comments to the article mentions to-do, bug, and fix me type sections.

03:10 Hopefully you're not putting too many bugs and fix me's in your code.

03:13 But to-dos are, I think those are great for, because when we're in the middle of coding, it's often easy to get caught up in, well, I could make it more powerful if I added these other features.

03:25 Just go ahead and like write that as a comment.

03:27 You don't really need to fall into the trap of making your code do too much at once.

03:32 And then the article also has some other tips on best practices for how to format your code comments and stuff.

03:39 Of course, it goes on and talks about doc strings and how to utilize those.

03:44 Also using type hints for commenting and documentation.

03:48 And then goes on to do like the larger, talks about the larger types of project documentation, like using Sphinx and make docs and stuff.

03:55 Yeah.

03:56 It's a really great article and it is really in-depth.

03:58 I always learn something from the real Python articles.

04:01 It's cool.

04:01 You know, one thing about the to-do stuff that I use a lot and I really like is the editor integration.

04:08 So I can't speak to the other ones.

04:10 I don't really know how exactly they handle it.

04:13 But it's saying PyCharm, if you have hash to do colon something, there's a special pane where you can say, show me all the to-dos in this project.

04:19 And even when you try to commit, like, you know, I want to commit this stuff to GitHub.

04:24 It'll say, warning, there's three to-dos in here.

04:27 Are you sure you want to commit to GitHub with these three to-dos?

04:30 And then I think it actually even remembers those three.

04:33 But if you had another one, it'll say, oh, there's like a new to-do you haven't like approved to commit, things like that.

04:38 So it's pretty sweet.

04:39 Yeah.

04:39 I actually, I stumbled across that feature without knowing it was there.

04:43 And I'm like, oh, man, this is great.

04:45 So I like it.

04:46 Yeah.

04:46 It's cool.

04:47 Even colors the comment different if it's a to-do comment.

04:49 So people should definitely do that.

04:51 So we talked a while ago about security vulnerabilities and other types of alerts.

04:57 And we even covered how GitHub has this, like, dependency thing where you can see the dependencies, but it didn't cover Python, which was unfortunate, right?

05:07 Right.

05:07 Well, big news.

05:09 It now supports Python.

05:10 That was pretty fast.

05:11 That was.

05:12 They got on that pretty quick.

05:13 They did.

05:13 So it's pretty sweet.

05:15 If you go to any GitHub repository and you go to insights, and then there's a bunch of options on the left, you pick dependency graph.

05:22 It will actually show you all of the files.

05:25 It's either found in a pip file.lock or in a requirements.txt anywhere in the project.

05:32 Wow.

05:32 That's cool.

05:33 And it'll list them out.

05:33 And it'll say it's pinned to this version.

05:35 But warning, this version actually has a security vulnerability and is fixed by this other release of it.

05:41 So if I'm using, say, Pyramid and there happened to have been, like, a security vulnerability in that, it would let me know and it would, like, actually highlight that and give me an alert.

05:50 Okay.

05:50 Neat.

05:51 Yeah.

05:51 It's really cool.

05:52 So it had previously only worked with Ruby and JavaScript, but now they just announced it.

05:56 They said for the Ruby and JavaScript people, they've identified millions of vulnerabilities and prompted many patches, which is pretty cool.

06:03 Okay.

06:03 I'm just checking my cards project to make sure it's clean.

06:07 It looks like it's good.

06:07 Yeah.

06:08 And it's interesting.

06:08 You can actually, it's like a tree structure that you get to.

06:11 So you can go there and it'll say, suppose you're using requests, for example, right?

06:16 Then there's, like, a little Chevron expando thing by requests and it'll open up and say, well, this depends on six and char dead or, you know, whatever it depends upon.

06:25 I can't remember the exact dependency of requests, but you can actually expand it out and see, like, the transitive closure or, like, what your dependencies, dependencies, dependencies depend upon.

06:35 And, like, if there's a vulnerability deep, deep down that you didn't directly install but effectively you installed.

06:40 So it's really nice.

06:41 The dependency graph is cool.

06:43 It even has, it'll even follow a requirements underscore dev.

06:48 Yeah.

06:48 It'll get all of them.

06:49 Yeah.

06:49 And then you can also look at dependents.

06:51 So people depending on your project.

06:53 Interesting.

06:54 Right.

06:55 Right.

06:55 So if you've got, like, a public package, you can see what depends upon it.

06:58 I wonder if you go to request if that breaks the UI.

07:02 Yeah.

07:05 I'll leave that exercise to the reader because I don't want my web browser to fail.

07:10 Yeah.

07:10 Exactly.

07:11 Exactly.

07:12 Anyway, so we've seen other services like pyup.io, which I use and still recommend.

07:17 But this is a nice way to just, like, walk up to a public GitHub repository and say, well, what are its dependencies?

07:23 Is it up to date?

07:24 Things like that without actually using it or registering it.

07:27 It's cool.

07:28 Right.

07:28 So pyup, you would have to have it be on your own and you could do that.

07:32 But this allows you to look at that for any project on GitHub.

07:36 Right.

07:37 Exactly.

07:37 Does maybe this project have a security vulnerability in its dependency or not?

07:41 If it doesn't, then I'm going to use it.

07:43 If it does, maybe I'll wait or do a PR or something.

07:45 Yeah.

07:45 Nice.

07:46 Well, speaking of dependencies, I depend on pip a lot and virtual environments.

07:52 Yes.

07:52 All the time.

07:53 I try to, like, super optimize what I do in the command line around pip and virtual environments.

07:58 So much so that I have a single alias that will create a virtual environment, make sure pip and setup tools is up to date, and then activate it all in, like, a couple of characters.

08:07 I love it.

08:08 Yeah, nice.

08:08 So the virtual environments are so useful.

08:10 I was happy to see this article pop up that's called How Virtual Environment Libraries Work in Python.

08:17 And I'm just going to quote it.

08:20 It says, have you ever wondered what happens when you activate a virtual environment and how it works internally?

08:25 Here's a quick overview of the internals behind popular virtual environments, for example, virtualenv, virtualenv,wrapper,conda, and pipenv.

08:34 And, yeah, it's a really fairly quick look at how there's really not much.

08:39 I didn't know this.

08:40 So one of the things that it points out is that all of these sort of work on a little bit of a hack.

08:46 And these hacks are based on the fact that when the Python interpreter starts, when the interpreter starts, it searches.

08:53 It's trying to find a site-specific directory to find all the modules it can import and all the packages.

09:00 And so the search starts with the parent directory of the Python executable location and then continues backtracking up the path into parent directories until it reaches all the way to the root.

09:12 But it's also looking for a lib directory with os.py in it.

09:17 Apparently, it's looking for that magic module.

09:19 And then if it finds that, that's the local site packages directory.

09:24 So the hack is to let it copy the executable somewhere and then create a directory with that symbolically linked.

09:33 And then you can have other stuff symbolically linked inside, too.

09:36 And earlier versions wouldn't symbolically link everything in the site packages if you had everything.

09:42 Or a whole bunch of stuff, at least.

09:44 I'm not sure if it's everything.

09:45 In Python 3.3, it came up with a couple of tricks with a pyvnv config file that allows you to not have to copy the executable completely.

09:54 You can copy.

09:55 You can have a symbolic link to the executable.

09:58 And all of this is how the sausage is made.

10:01 But I found it fairly interesting to figure out how all these virtual environment things work.

10:06 Yeah, I think this is really helpful.

10:08 Because certainly being effective with Python is being effective managing your dependencies and stuff.

10:14 And the way you do that is virtual environments.

10:16 So this is a really good look at how that works.

10:19 Yeah, and to me, it did seem like after about 3.3, that things sort of went faster.

10:24 And now it makes sense that it's faster because they're not copying the executable into a new location.

10:29 So it's a clean look.

10:32 And I'm with you.

10:33 I've tried a whole bunch of off-the-shelf helper things.

10:37 And it's just my own handful of commands that I've come up with that I've just stuck them in my bash config file as little functions.

10:47 Because they don't do much.

10:48 I just needed to create a virtual environment and then activate it.

10:51 And then, like you said, update pip.

10:53 And what else do you update?

10:55 Set up tools.

10:55 Okay.

10:56 Yeah.

10:56 It's interesting.

10:57 There's a lot of things that try to help and manage this stuff.

10:59 And they're all pretty good.

11:01 But in reality, like you say, I just want to run these three commands and just get rolling and just keep it simple.

11:07 And I could just create that as an alias and like a bash RC or a ZSHRC and separate the three commands with a double ampersand.

11:15 And then, boom, it's one command.

11:16 You're good.

11:17 And for people like us that are doing a lot of like trying out a lot of new things, I create a lot of virtual environments.

11:23 I do agree with Kenneth Reitz and the others that are promoting pip.env that for because of all of the hashes and security reasons, for full application development, I think using pip.env is the right thing.

11:37 And keeps everybody in the project.

11:39 We're using the project on the same page.

11:41 But there's plenty of other uses for raw virtual environment or V and V that I use.

11:47 Yeah.

11:47 Pretty interesting.

11:47 One final little tidbit of news that I don't think justifies its own place.

11:53 But pip, not too long ago, went from version 9 to version 10, right?

11:57 Oh, yeah.

11:58 Yeah.

11:58 I'm glad you brought that up.

11:59 That's big.

12:00 So now, in the last week or two, I haven't really been paying attention.

12:03 But it went from 10 to 18.

12:04 Yeah.

12:05 It completely skipped.

12:07 A lot of features.

12:09 But it switched from whatever that normal versioning.

12:13 Semantic versioning.

12:14 Semantic versioning to date-based versioning.

12:16 Yes.

12:16 So I don't understand how the date works.

12:19 Maybe it's the year, but surely there's more than one release in a year.

12:22 Who knows?

12:22 Anyway, it's pretty cool.

12:24 Maybe it is 2018.

12:24 But yeah, so there's a brand new version of pip, a new versioning scheme there.

12:28 Yeah.

12:28 Cool.

12:29 Nice.

12:29 All right.

12:30 Before we get to the next one, let me tell you about Datadog, our sponsor.

12:33 So if you have infrastructure and you want to monitor it, you want to trace a request,

12:39 say, across your web tier to some microservice series of services down at the database,

12:45 put the logs all together that Datadog provides end-to-end visibility for the health and performance

12:51 of your Python apps.

12:52 You can explore metrics, distribute request traces and logs, all that in one place.

12:56 And even have a 14-day free trial.

12:58 And if you check them out at pythonbytes.fm/Datadog, set up a trial, you get a sweet,

13:03 cute Datadog t-shirt.

13:05 So that makes it worth it right there.

13:06 Yeah.

13:07 So anyway, check them out.

13:08 It really helps support the show.

13:09 Brian, have we talked about GUIs?

13:12 Yes, I think so.

13:13 Maybe.

13:14 Maybe.

13:14 Maybe.

13:14 Maybe.

13:15 So one of the really nice things about having TK Enter built in to Python is it's really easy.

13:23 It just comes along.

13:24 You don't have to do stuff.

13:25 previously to use something like Qt or I think the same may be true for WX Python.

13:31 But for Qt, it was a serious pain to get that installed so that you could use it.

13:36 It's like, well, you run this funky package on Ubuntu.

13:40 You download this installer for Mac.

13:42 And then all of a sudden, you're like, here's my simple Python file that's going to be your app

13:46 becomes like this weird extra dependency installing problem, right?

13:52 So the Qt company, the people that make Qt, the framework, and then they've taken over PySide2,

13:59 the wrapper, and they've rebranded it for Qt for Python.

14:01 Well, they've made that much, much easier.

14:04 So now if I have an application and it has like a requirements file or a pip file.lock or whatever,

14:12 pip file that says, I depend upon PySide2, which is the working package name for Qt for Python right now.

14:19 If you depend upon that, all you have to do is pip install requirements and you have it

14:25 because it ships as a wheel right off PyPy.

14:27 Okay.

14:27 Nice.

14:29 If you want to use Qt, instead of having like really painful dependencies that people have to set up

14:34 and like a little step, you want to run our app, well, click here to go to this page to set up Qt.

14:38 Now you just pip install your requirements.

14:40 Boom, you have Qt.

14:41 It's nice.

14:42 That is nice.

14:42 That's a huge thing, actually.

14:44 I was considering, basically, I'm like, I'm not using Qt.

14:46 What is required is this separate go find and install Qt so you can run my app.

14:51 Like that just means there's no way that that's reasonable for distributing that to people.

14:55 But if all you got to do is like package the dependencies, then, you know, presumably you

15:00 could package it up with like PyInstaller at this point, maybe.

15:03 So really nice.

15:05 And I want to shout out to all of the people that every time we cover this contact me and tell me that I'm mispronouncing Qt as Qt.

15:14 That this time it was Michael that pronounced it Qt instead of Qt.

15:19 I think I had it right before, didn't I?

15:21 Oh, well.

15:22 Maybe you've infected me.

15:24 No, it's, yeah, Qt for Python.

15:25 Yeah, it's cool.

15:26 Yeah.

15:26 Well, I don't know.

15:27 I think that's official.

15:28 Yeah, yeah.

15:29 I think so, too.

15:30 And also for all the people that said, yes, you've covered 17 GUI frameworks, but have

15:34 you heard of these other two?

15:35 So that's, we're definitely learning a lot about the different frameworks out there.

15:42 So this is one more advance there.

15:44 I don't think people should stop because I still don't, I think this is great, but I don't

15:48 think it's completely solved.

15:49 So people keep trying to solve it with new libraries.

15:52 Go for it.

15:52 Absolutely.

15:53 Absolutely.

15:53 All right.

15:54 What's this next one you got about exceptions here, Brian?

15:56 Okay.

15:56 Well, so exceptions, I think if, and I misspelled it, but exceptions are part of Python.

16:03 And even if they're kind of, I think they're handled sort of later when you're learning about

16:10 how to program in Python, you sort of deal with telling people how to use exceptions later,

16:15 but you're going to have to deal with them because whether or not you write your own exceptions

16:20 or utilize, raise, and try accept yourself, other, your libraries that you utilize are

16:27 utilizing or using it.

16:29 So if you don't want your program to crash, you got to figure out how to do all this.

16:32 So this is, not only is it a tutorial on how to use exceptions, it's learning how not to

16:38 handle exceptions because sometimes the right thing to do is not to catch an exception.

16:43 The right thing to do is to pass it, either ignore it completely because it'll pass up to a higher

16:49 level.

16:49 And that's part of one of the, one of the things that's this, in this article, an article called

16:55 learning not to handle exceptions, talks about not only how to use them, but how to not use them

17:00 correctly.

17:00 How do, so first it does tell you how to use them correctly.

17:03 Like the try and accept and raise.

17:05 So try and accept are obvious.

17:07 So if you try a bunch of, a bit of code, and then you can use an accept clause to catch something

17:12 right off the bat, they tell you, which is, I think, correct, is usually you don't want to

17:17 accept everything.

17:18 You want to accept a specific exception that you know how to deal with.

17:21 And then everything that you don't know how to deal with, just let it pass you up the stack.

17:26 But sometimes even if you catch it, you want to keep raising it.

17:30 So you can, it does talk about re-raising an exception with just raise without any keywords

17:35 out or any, any exception after it'll just raise the current exception.

17:38 Yeah.

17:39 I really like that is like, you have a try, do your thing, accept.

17:42 And then maybe you want to do some kind of compensating action, like roll back a transaction

17:48 or delete a temporary file.

17:50 But then you don't really want to handle it.

17:52 You just needed to say, if there's a problem, I got to fix this and then let somebody else actually

17:57 handle it.

17:57 So then you just say raise, or you could even use the raise.

17:59 One exception from another exception and chain them.

18:03 Yeah.

18:03 Yeah.

18:04 It was really nice.

18:04 And then there's other things like finally and else.

18:07 So finally can allow you to run some code, whether or not an exception runs.

18:12 And then else lets you run some code only if there are no exceptions.

18:16 And you can use both of them at the same time with the same exception.

18:19 Yeah.

18:20 Both can be utilized.

18:21 Yeah.

18:22 The article goes on and covers dealing with tracebacks, custom exceptions, and then some

18:27 best practices, and then making your exceptions more readable by adding arguments and printing

18:32 things out nicely and stuff.

18:34 So this is good.

18:35 Yeah.

18:36 It's really nice.

18:37 Great little article.

18:38 Definitely a nice find there.

18:40 So I thought I'd put here at the end of our episode today something that takes a step back

18:48 and just looks at the overall community, the ecosystem, and a bit of an interesting, I don't

18:55 know, high-level view of Python.

18:57 So last week we did our whole special episode on Guido von Rossum stepping down, right, with

19:02 Brett Cannon, Carol Willing, and sort of covered that whole news.

19:05 But another interesting thing that happened that is around that is, you know, sort of speaks

19:11 to Python's popularity is, you know, the magazine, The Economist?

19:16 Yeah, definitely.

19:17 They did a feature in The Economist on Python and Guido von Rossum.

19:21 That's pretty mainstream and huge.

19:24 That's pretty interesting, right?

19:25 Yeah.

19:26 So there's an article in The Economist called Python Has Brought Computer Programming to

19:30 a Vast New Audience, which is pretty awesome.

19:33 And it covers like the meteoric growth of Python, how it's becoming so much more popular than

19:39 many things, how, why, and where people are learning Python.

19:43 So I thought that was all quite interesting.

19:44 And then interspersed in there is sort of the history of Python with quotes from Guido himself.

19:49 So it's pretty cool.

19:50 Yeah, this is some nice publicity here.

19:52 Yeah, not too shabby.

19:54 And it's not a super long read.

19:56 But it's a pretty nice treatment.

19:58 It talks about some interesting things like some really powerful things that people are

20:02 doing with Python, some interesting ways that people are learning it.

20:05 But it also says, you know, there are some dangers to Python growing so fast as a lot of people

20:11 are coming into the industry, learning just enough, grabbing something like TensorFlow, you

20:16 know, some AI framework and sort of understanding it and then drawing conclusions from what the

20:22 machine learning algorithms, they sort of understand, tell them, you know, and run off with like half

20:27 baked ideas.

20:27 I thought that was an interesting comment.

20:29 Yeah.

20:30 Well, I mean, yeah, that is an interesting comment.

20:32 But half understanding what you're talking about and drawing conclusions.

20:36 We see that every day with like statistics and in newspapers and stuff also.

20:41 Yeah, absolutely.

20:42 I mean, even in like medicine and medical research, that stuff is rampant.

20:46 It's a lack of understanding of statistics is a problem across the board.

20:50 I haven't read the article yet.

20:51 I'm excited to read it.

20:53 But the interesting things is people learning to code isn't necessarily because you're going

20:59 to end up being a professional programmer.

21:01 It's often one of the things that helps it grow is a lot of people using it as part of

21:07 their other job, like a biologist or somebody else that is just using it to help out with

21:12 some section of their job.

21:14 And or kids, you know, tinkers just trying to play with a Raspberry Pi or something.

21:20 Yeah, it's actually that's a really good point.

21:22 They actually touch on that a little bit in the article.

21:24 They say for professionals that have long relied upon trawling through spreadsheets, Python

21:30 is especially valuable.

21:31 And they point out that Citigroup, the American Bank, has even put together a crash course for

21:37 Python for its analysts.

21:38 And the eFinancialCareers, like a website for financial jobs, found that there's like a four

21:46 time increase in the request for Python skills in the last three years among financial analysts,

21:52 not programmers.

21:52 Oh, nice.

21:53 Yeah.

21:53 So there's a lot of interesting little tidbits of stuff coming out of this article.

21:56 Yeah, cool.

21:57 Yeah, definitely worth checking out.

21:58 All right.

21:59 Well, I think that's it.

22:00 That's our six items for the week.

22:01 Brian, you got anything else you want to share with everyone while you're here?

22:04 No, I've been having my head in antiques.

22:07 So if anybody's a listener in the Portland area, hit me up and I'll tell you where I'm going

22:13 to be selling some antiques this Sunday.

22:14 So it's a completely non-Python job.

22:17 It's kind of nice every now and then to just take a step away, put the keyboard down and

22:22 do something really different so you can come back more refreshed, right?

22:24 Yeah.

22:25 You got any news?

22:26 No, not this week.

22:27 Although I'm working on something big that I will announce next week.

22:30 So I'm close to having news, but I'm not quite there yet.

22:33 All right.

22:34 Nice.

22:34 I'll bring a little extra news next week.

22:36 All right.

22:36 Well, Brian, thanks for doing this and sharing with everyone.

22:39 And thank you everyone for listening.

22:40 Thanks.

22:40 Bye.

22:41 Yep.

22:41 Thank you for listening to Python Bytes.

22:44 Follow the show on Twitter via at Python Bytes.

22:47 That's Python Bytes as in B-Y-T-E-S.

22:50 And get the full show notes at pythonbytes.fm.

22:53 If you have a news item you want featured, just visit pythonbytes.fm and send it our way.

22:58 We're always on the lookout for sharing something cool.

23:00 On behalf of myself and Brian Okken, this is Michael Kennedy.

23:04 Thank you for listening and sharing this podcast with your friends and colleagues.