Transcript #88: Python has brought computer programming to a vast new audience

00:00 Hello and welcome to Python Bytes where we deliver Python news and headlines directly to your earbuds. This is episode 88 Recorded July 26 2018. I'm Michael Kennedy. I'm Brian. Okay, Brian. How you doing? I'm doing great Yeah, it's a ton of good stuff that I think you found this week and I'm looking forward to share what I got as well It's never a boring week in the Python world. Is it? No, it isn't. No Well, it's definitely not boring if your app has problems and you need to track them down So if you're in that space, you might want to check out Datadog over at pythonbytes.fm/datadog.

00:33 I'll tell you more about them later.

00:35 Right now, I want to know about documenting code.

00:38 We do hear a lot about, I think we hear a lot about making sure that your code documentation is good.

00:45 There's the read the docs and make the docs effort, or I guess it's write the docs.

00:51 And there's quite a bit of that and I think a lot of that is focused on the end user user documentation and that's definitely important but there was a there's an article on I think it was unreal yet unreal python.

01:03 Call documenting python code a complete guide it's an article that describes you know really why you should comment and document your code.

01:11 Talks about the differences between comments and doc strings and different separate documentation.

01:17 But one of the things that I want to zoom in on is the code comments and I don't think very many people get really.

01:24 How if they think about their comments they put in their code how effective they can be so i want to zoom in on some of the things that they cover in this article.

01:32 On code comments so come answer those things that start with a pound sign or hash.

01:38 and they're not really part of your runtime, you can't get a hold of them at runtime.

01:43 They're intended for you and other maintainers and other developers of the code.

01:48 Some of the things that I've used them for, but I never really saw anybody point out before, is before you even start your code, you can use comments to plan what you're going to write.

01:58 You can also, once you've written your code, review to make sure the code really matches what you planned to do.

02:05 And then sometimes those kind of comments are best to be, if your code is clear enough, just go ahead and delete those when you're done.

02:11 I like that. I end up doing that a lot myself.

02:13 I'm like, okay, I'm not entirely sure how to do this, so I'll sketch this out in comments.

02:17 Like, here's the five steps I think I got to do.

02:19 And then I'll just take them one at a time.

02:20 And then I always delete those comments, almost always.

02:23 Yeah, the deleting part is important because if you don't do that, and then you get the common pitfall of a code comment that says, "Retrieve web page," and then the very next line is one line of code that's a request or something and it's obvious.

02:36 So if it's not obvious, you're using those to explain intent.

02:40 Don't explain really how it's done, but explain why you're doing it.

02:44 Then sometimes you'll have a complicated math algorithm or something that's fairly terse if you need to explain that better.

02:51 But we're trying to get as few comments as possible, minimizing those because you want to make your code pretty clear to begin with.

02:59 But one of the other things that I think I've just started using quite a bit of the to do feature so using code comments to the article mentions to do bug and fix me type sections.

03:10 Hopefully you're not putting too many bugs and fix me is in your code but to do is are I think those are great for because when we're in the middle of coding it's often easy to get caught up in.

03:21 well, I could make it more powerful if I added these other features.

03:25 Just go ahead and write that as a comment.

03:27 You don't really need to fall into the trap of making your code do too much at once.

03:32 And then the article also has some other tips on best practices for how to format your code comments and stuff.

03:39 Of course, it goes on and talks about doc strings and how to utilize those.

03:44 Also, using type hints for commenting and documentation, and then goes on to do like the larger talks about the larger types of project documentation like using Sphinx and make docs.

03:55 Yeah, it's a really great article and it's really in depth.

03:59 I always learn something from the real Python articles.

04:01 It's cool.

04:02 You know, one thing about the to do stuff that I use a lot and I really like is the editor integration.

04:08 So I can't speak to the other ones.

04:10 I don't really know how exactly they handle it.

04:13 But it's a in PyCharm.

04:14 If you have hash to do colon, something, there's a special pane where you can say show me all all the to-dos in this project.

04:20 And even when you try to commit, like, you know, I wanna commit this stuff to GitHub, it'll say, warning, there's three to-dos in here.

04:27 Are you sure you wanna commit to GitHub with these three to-dos?

04:30 And then I think it actually even remembers those three, but if you add another one, it'll say, oh, there's like a new to-do you haven't like approved to commit, things like that.

04:38 So it's pretty sweet.

04:39 - Yeah, I actually, I stumbled across that feature and without knowing it was there, and I'm like, oh man, this is great.

04:45 So I like it.

04:46 - Yeah, that's cool.

04:47 colors the comment different if it's a to do comment. So people should definitely do that.

04:51 So we talked a while ago about security vulnerabilities and other types of alerts.

04:57 And we even covered how GitHub has this like dependency thing where you can see the dependencies, but it didn't cover Python, which was unfortunate, right?

05:07 Right.

05:07 Well, big news. It now supports Python.

05:10 That was pretty fast.

05:11 That was.

05:12 They got on that pretty quick.

05:13 >> They did. So it's pretty sweet.

05:15 If you go to any GitHub repository and you go to Insights, and then there's a bunch of options on the left-hand pick dependency graph, it'll actually show you all of the files.

05:25 It's either found in a pip file.lock, or in a requirements.txt, anywhere in the project.

05:32 >> Wow, that's cool.

05:33 >> It'll list them out and it'll say it's pinned to this version.

05:35 But warning, this version actually has a security vulnerability and is fixed by this other release of it.

05:41 So if I'm using, say, Pyramid, and there happened to have been a security vulnerability in that, it would let me know, and it would actually highlight that and give me an alert.

05:50 - Okay, neat.

05:51 - Yeah, it's really cool.

05:52 So it had previously only worked with Ruby and JavaScript, but now they just announced it.

05:57 They said for the Ruby and JavaScript people, they've identified millions of vulnerabilities and prompted many patches, which is pretty cool.

06:03 - Okay, I'm just checking my cards project to make sure it's clean.

06:07 Looks like it's good.

06:08 - Yeah, and it's interesting.

06:09 can actually it's like a tree structure that you get to so you can go there and it'll say, suppose you're using requests, for example, right, then there's like a little Chevron expando thing by requests, and it'll open up and say, well, this depends on six and chardet or whatever it depends upon.

06:25 I can't remember the exact dependency of requests, but you can actually expand it out and see like the transitive closure or like what your dependencies, dependencies, dependencies depend upon and like if there's a vulnerability deep, deep down that you didn't directly install, effectively installed. So it's really nice.

06:41 The dependency graph is cool. It even has, it'll even follow a requirements underscore dev. Yeah, it'll get all of them, yeah.

06:49 And then you can also look at dependents.

06:51 So people depending on your project.

06:53 Interesting. Right, right, right.

06:55 So if you've got like a public package, you can see what depends upon it.

06:59 I wonder if you go to request, if that breaks the UI.

07:03 [laughter]

07:05 Yeah, I'll have to, I'll leave that I had to exercise the reader because I don't want my web browser to fail.

07:10 Yeah, exactly.

07:11 Exactly.

07:12 Anyway, so we've seen other services like pyup.io, which I use and still recommend.

07:17 But this is a nice way to just like walk up to a public GitHub repository and say, well, what are its dependencies?

07:24 Is it up to date?

07:25 Things like that, without actually using it or registering it.

07:27 It's cool.

07:28 Right.

07:29 So pyup, you would have to have it be on your own and you could do that.

07:32 this is allows you to look at that for any project on GitHub.

07:36 Right, exactly. Does maybe this project have a security vulnerability in its dependency or not? If it doesn't, then I'm going to use it. If it does, maybe I'll wait or do a PR or something. Yeah. Nice. Well, speaking of dependencies, I depend on pip a lot and virtual environments.

07:52 Yes, all the time. I, I've tried to like super optimize what I do in the command line around pip and virtual environments, so much so that I have a single alias that will create a virtual environment, make sure pip and setup tools up to date and then activate it all in like a couple of characters.

08:07 I love it.

08:08 Yeah, nice.

08:09 So the virtual environments are so useful.

08:11 I was happy to see this article pop up that's called "How Virtual Environment Libraries Work in Python." And I'm just going to quote it.

08:20 It says, "Have you ever wondered what happens when you activate a virtual environment and how it works internally?

08:25 Here's a quick overview of the internals behind popular virtual environments, for example, Env, Virtual Env Wrapper, Conda, and pip Env.

08:34 And yeah, it's a really fairly quick look at how there's really not much. I didn't know this. So one of the things it points out is that there's a, all of these sort of work on a little bit of a hack.

08:46 And these hacks are based on the fact that the Python, when the Python interpreter starts, when the interpreter starts, it searches it's trying to find the site-specific directory to find all the modules it can import and all the packages.

09:00 So the search starts with the parent directory of the Python executable location and then continues backtracking up the path into parent directories until it reaches all the way to the root.

09:12 But it's also looking for a lib directory with os.py in it.

09:17 Apparently, it's looking for that magic module.

09:20 Then if it finds that, that's the local site packages directory.

09:24 So the hack is to let it copy the executable somewhere and then create a directory with that symbolically linked.

09:33 And then you can have other stuff symbolically linked inside too.

09:36 And earlier versions wouldn't symbolically link everything in the site packages if you had everything or a whole bunch of stuff at least.

09:44 I'm not sure if it's everything.

09:45 In Python 3.3, it came up with a couple of tricks with a pyvenv config file that allows you to not have to copy the executable completely.

09:55 You can copy, you can have a symbolic link to the executable.

09:58 And all of this is how the sausage is made, but I found it fairly interesting to figure out how all these virtual environment things work.

10:06 - Yeah, I think this is really helpful because certainly being effective with Python is being effective managing your dependencies and stuff.

10:14 And the way you do that is virtual environments.

10:16 So this is a really good look at how that works.

10:19 - Yeah, and to me, it did seem like after about 3.3, that things sort of went faster and now it makes sense that it's faster because they're not copying the executable into a new location.

10:30 So it's a clean look and I'm with you.

10:33 I've tried a whole bunch of off the shelf helper things and it's just my own handful of commands that I've come up with that I've just stuck them in my little bash config file as little functions 'cause they don't do much.

10:48 I just needed to create a virtual environment and then activate it, and then like you said, update pip, and what else do you update?

10:55 - Set up tools.

10:56 - Okay, yeah.

10:56 - It's interesting, there's a lot of things that try to help and manage this stuff, and they're all pretty good, but in reality, I just, like you say, I just want to run these three commands and just get rolling and just keep it simple, and I could just create that as an alias in a, like a bash RC or a zsh RC, and separate the three commands with a double ampersand, and then boom, it's one command, you're good.

11:17 - And for people like us that are doing a lot of, like trying out a lot of new things, I create a lot of virtual environments.

11:23 I do agree with Kenneth Reitz and the others that are promoting pipenv that for, because of all of the hashes and security reasons, for full application development, I think using pipenv is the right thing and keeps everybody using the project on the same page.

11:42 But there's plenty of other uses for raw virtual environment or VNV that I use.

11:47 Yeah, pretty interesting.

11:48 One final little tidbit of news that I don't think justifies its own place, but pip not too long ago went from version 9 to version 10, right?

11:57 Oh, yeah.

11:58 Yeah, I'm glad you brought that up.

12:00 That's big.

12:01 So now in the last week or two, I haven't really been paying attention, but it went from 10 to 18.

12:04 Yeah.

12:05 It completely skipped.

12:06 A lot of features.

12:10 It switched from whatever that normal versioning...

12:13 Semantic versioning?

12:14 Semantic versioning to date-based versioning.

12:16 So I don't understand how the date works.

12:19 Maybe it's the year, but surely there's more than one release in a year.

12:22 Who knows?

12:23 Anyway, it's pretty cool.

12:24 Maybe it is 2018, but yeah, so there's a brand new version of PIP, a new versioning scheme there.

12:28 - Yeah, cool.

12:29 - Nice.

12:30 All right, before we get to the next one, let me tell you about Datadog, our sponsor.

12:33 So if you have infrastructure and you want to monitor it, you want to trace a request, say across like your web tier to some microservice series of services down at the database, and put the logs all together that Datadog provides end-to-end visibility for the health and performance of your Python apps.

12:52 You can explore metrics, distribute request traces and logs, all that in one place.

12:56 And even have a 14-day free trial.

12:58 And if you check them out at pythonbytes.fm/datadog, set up a trial, you get a sweet, cute Datadog t-shirt.

13:05 So that makes it worth it right there.

13:06 - Yeah.

13:07 - So anyway, check them out.

13:08 It really helps support the show.

13:10 Brian, have we talked about GUIs?

13:11 (laughing)

13:12 - Yes, I think so, maybe.

13:15 - Maybe.

13:15 So one of the really nice things about having TkEnter built in to Python is it's really easy.

13:24 It just comes along, you don't have to do stuff.

13:26 Previously to use something like Qt, or I think the same may be true for WXPython, but for Qt, it was a serious pain to get that installed so that you could use it.

13:36 It's like, well, you run this funky package on Ubuntu, you download this installer for And then all of a sudden you're like, here's my simple Python file that's going to be your app becomes like this weird extra dependency installing problem, right?

13:53 So the Qt company, the people that make Qt, the framework, and then they've taken over a PySide to the wrapper and they re-branded it for Qt for Python.

14:02 Well, they've made that much, much easier.

14:05 So now if I have an application and it has like a requirements file or a pip file that lock or whatever, pip file that says I depend upon PySide 2, which is the working package name for Qt for Python right now.

14:20 If you depend upon that, all you have to do is pip install requirements and you have it because it ships as a wheel right off Python.

14:29 If you want to use Qt instead of having like really painful dependencies that people have to set up and like a little step, you want to run our app, well, click here to go to this page that set up Qt.

14:38 Now you just pip install your requirements.

14:40 Boom, you have Qt, it's nice.

14:42 - That is nice.

14:43 That's a huge thing, actually.

14:44 - I was considering, basically I'm like, I'm not using Qt.

14:47 What is required is this separate go find and install Qt so you can run my app.

14:52 Like that just means that there's no way that that's reasonable for distributing that to people.

14:56 But if all you gotta do is like package the dependencies, then presumably you could package it up with like PyInstaller at this point, maybe.

15:04 So really nice.

15:05 - Yeah.

15:06 And I want to shout out to all of the people that every time we cover this, contact me and tell me that I'm mispronouncing cute as QT.

15:15 This time it was Michael that pronounced it QT instead of cute.

15:19 I think I had it right before, didn't I?

15:22 Oh well.

15:23 Maybe you've infected me.

15:24 No, it's cute for Python.

15:25 Yeah, it's cool.

15:26 Yeah.

15:27 Well, I don't know.

15:28 I think that's official.

15:29 Yeah, yeah.

15:30 I think so too.

15:31 And also for all the people that said, "Yes, you've covered 17 GUI frameworks, but have Have you heard of these other two?

15:37 So that's, we're definitely learning a lot about the different frameworks out there.

15:42 So this is one more advance there.

15:44 I don't think people should stop because I still don't, I think this is great, but I don't think it's completely solved.

15:50 So people keep trying to solve it with new libraries.

15:52 Go for it.

15:53 Absolutely.

15:54 Absolutely.

15:55 All right.

15:56 What's this next one you got about exceptions here, Brian?

15:57 Okay.

15:58 Well, so exceptions, I think, and I misspelled it, but exceptions are part of Python.

16:04 And even if they're kind of, I think they're handled sort of later when you're learning about how to program in Python, you sort of deal with telling people how to use exceptions later.

16:15 But you're going to have to deal with them because whether or not you write your own exceptions or utilize Rays and try to accept yourself, other, your libraries that you utilize are utilizing, they're using it.

16:29 So if you don't want your program to crash, you got to figure out how to do all this.

16:33 So this is not only is it a tutorial on how to use exceptions, it's learning how not to handle exceptions because sometimes the right thing to do is not to catch an exception.

16:43 The right thing to do is to pass it, either ignore it completely because it'll pass up to a higher level.

16:50 And that's part of one of the things that's in this article, an article called Learning Not to Handle Exceptions.

16:57 Talks about not only how to use them, but how to not use them correctly.

17:02 So first it does tell you how to use them correctly, like the try and accept and raise.

17:06 So try and accept are obvious.

17:07 So if you try a bunch of a bit of code, and then you can use an accept clause to catch something right off the bat, they tell you, which is I think correct, is usually you don't want to accept everything.

17:18 You want to accept a specific exception that you know how to deal with.

17:22 And then everything that you don't know how to deal with, just let it pass you up the stack.

17:27 But sometimes even if you catch it, you want to keep raising it.

17:30 So you can, it does talk about re-raising an exception with just raise without any keywords out or any exception after it'll just raise the current exception.

17:39 Yeah, I really like that is like you have a try, do your thing, accept.

17:43 And then maybe you want to do some kind of compensating action, like roll back a transaction or delete a temporary file, but then you don't really want to handle it.

17:52 You just needed to say, if there's a problem, I got to fix this and then let somebody else actually handle it.

17:57 just say raise or you could even use the raise one exception from another exception and chain them.

18:04 Yeah.

18:05 It was really nice.

18:06 And then there's other things like finally and else.

18:07 So finally can allow you to run some code whether or not an exception runs.

18:12 And then else lets you run some code only if there are no exceptions.

18:16 And you can use both of them at the same time with the same exception.

18:20 Yeah.

18:21 Both can be utilized.

18:22 Yeah.

18:23 The article goes on and covers dealing with tracebacks, custom exceptions, and then some best practices, and then making your exceptions more readable by adding arguments and printing things out nicely and stuff. So this is a good.

18:35 Yeah, it's really nice. Great little article. Definitely a nice find there.

18:40 So I thought I put here at the end of our episode today, something that is kind of takes a step back and just looks at the overall community, the ecosystem, and a bit of an interesting, I don't know, high level view of Python. So last week we did our whole special episode on Guido von Rossum stepping down right with Brett Cannon, Carol Willing and sort of cover that whole news. But another interesting thing that happened that is around that is you know sort of speaks to Python's popularity is you know the magazine The Economist? Yeah definitely. They did a feature in The Economist on Python and Guido von Rossum. That's pretty pretty mainstream and huge. That's pretty - Pretty interesting, right?

19:26 - Yeah.

19:27 - So there's an article in The Economist called Python Has Brought Computer Programming to a Vast New Audience, which is pretty awesome.

19:33 And it covers the meteoric growth of Python, how it's becoming so much more popular than many things, how, why, and where people are learning Python.

19:43 So I thought that was all quite interesting.

19:44 And then interspersed in there is sort of the history of Python with quotes from Guido himself.

19:50 So it's pretty cool.

19:51 - Yeah, this is some nice publicity here.

19:53 - Yeah, not too shabby.

19:54 And it's not a super long read, but it's a pretty nice treatment.

19:58 It talks about some interesting things like some really powerful things that people are doing with Python, some interesting ways that people are learning it.

20:06 But it also says, there are some dangers to Python growing so fast as a lot of people are coming into the industry, learning just enough, grabbing something like TensorFlow, some AI framework, and sort of understanding it.

20:20 And then drawing conclusions from what the machine learning algorithms, they sort of understand, tell them, and run off with half-baked ideas.

20:28 Thought that was an interesting comment.

20:29 - Yeah, well, I mean, yeah, that is an interesting comment, but half-understanding what you're talking about and drawing conclusions, we see that every day with statistics in newspapers and stuff also.

20:41 - Yeah, absolutely.

20:42 I mean, even in medicine and medical research, that stuff is rampant.

20:46 It's a lack of understanding of statistics is a problem across the board.

20:50 - I haven't read the article yet.

20:52 I'm excited to read it.

20:53 But the interesting thing is people learning to code isn't necessarily because you're going to end up being a professional programmer.

21:01 It's often one of the things that helps it grow is a lot of people using it as part of their other job, like a biologist or somebody else that is just using it to help out with some section of their job.

21:14 Or kids, tinkerers just trying to play with a Raspberry Pi or something.

21:20 >>Yeah, that's a really good point.

21:22 actually have touched on that a little bit in the article. They say for professionals that have long relied upon trawling through spreadsheets, Python is especially valuable. And they point out that Citigroup, the American bank, has even put together a crash course for Python for its analysts. And the eFinancial careers, like a website for financial jobs, found that there's like a four-time increase in the request for Python skills in the last three years among financial analysts, not programmers.

21:52 - Oh, nice.

21:53 - Yeah, so there's a lot of interesting little tidbits of stuff coming out of this article.

21:56 - Yeah, cool.

21:57 - Yeah, definitely worth checking out.

21:58 All right, well, I think that's it.

22:00 That's our six items for the week.

22:02 Brian, you got anything else you want to share with everyone while you're here?

22:04 - No, I've been having my head in antiques, so if anybody's a listener in the Portland area, hit me up and I'll tell you where I'm gonna be selling some antiques this Sunday, so completely non-Python job.

22:18 It's kind of nice every now and then to just take a step away, put the keyboard down, and do something really different so you can come back more refreshed, right?

22:25 Yeah.

22:26 You got any news?

22:27 No, not this week, although I'm working on something big that I will announce next week.

22:30 So I'm close to having news, but I'm not quite there yet.

22:33 All right.

22:34 Nice.

22:35 I'll bring a little extra news next week.

22:36 All right.

22:37 Well, Brian, thanks for doing this and sharing with everyone, and thank you everyone for listening.

22:41 Thanks.

22:42 Bye.

22:43 Yep.

22:44 Thank you for listening to Python Bytes.

22:45 Follow the show on Twitter via @pythonbytes.

22:47 Python Bytes as in B-Y-T-E-S. And get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Okken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.