WEBVTT

00:00:00.001 --> 00:00:05.180
Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:00:05.180 --> 00:00:10.120
This is episode 93, recorded August 30th, 2018.

00:00:10.120 --> 00:00:11.080
I'm Michael Kennedy.

00:00:11.080 --> 00:00:11.880
And I'm Brian Okken.

00:00:11.880 --> 00:00:15.680
And Brian, you know, last time we had some awesome drop the mic stuff.

00:00:15.680 --> 00:00:17.660
I think we're going to do it again, aren't we?

00:00:17.660 --> 00:00:20.840
Yeah, I think it's some really interesting things today.

00:00:20.840 --> 00:00:21.500
Absolutely.

00:00:21.500 --> 00:00:24.940
So before we get on to them, though, I just want to say thanks to Datadog.

00:00:24.940 --> 00:00:26.280
Datadog's sponsoring the show.

00:00:26.280 --> 00:00:29.380
Check them out at pythonbytes.fm/datadog.

00:00:30.000 --> 00:00:31.080
I'll tell you more about that later.

00:00:31.080 --> 00:00:39.240
Brian, a lot of people get into programming, kind of in air quotes there, by doing scripting around like the shell and stuff, right?

00:00:39.240 --> 00:00:39.880
I think so.

00:00:39.880 --> 00:00:42.860
I think that's the way some people like in the IT world are...

00:00:42.860 --> 00:00:45.560
Like Bash or even bat files.

00:00:45.560 --> 00:00:48.800
Or lots of DevOps roles.

00:00:48.800 --> 00:00:49.380
Yeah, exactly.

00:00:49.380 --> 00:00:59.080
At the point where I learned Python, I was doing a lot of C++ development and writing little utility scripts in Bash and Perl and stuff like that.

00:00:59.080 --> 00:01:06.460
And so one of the things that I did, you know, I want to try to learn Python by writing as much Python as I can.

00:01:06.460 --> 00:01:13.680
And so I started doing more and more of my utility scripts in Python.

00:01:14.020 --> 00:01:19.580
And so I think that was a good way to just try to jump in and solve little problems.

00:01:19.580 --> 00:01:23.240
I could have written it faster in Bash because I knew that or in Perl.

00:01:23.240 --> 00:01:26.040
But learning how to do things in Python is good.

00:01:26.040 --> 00:01:31.680
So it's a GitHub repo that's called replacing Bash scripting with Python.

00:01:32.500 --> 00:01:35.840
And I think it goes through a lot of stuff that...

00:01:35.840 --> 00:01:42.040
A lot of the issues that you'd need to do that you'd normally write in the little utilities and how to do that in Python.

00:01:42.040 --> 00:01:43.840
So this is fun.

00:01:43.840 --> 00:01:51.220
It talks about like dealing with files, reading and writing files, dealing with standard in and stuttered out, standard error, things like that.

00:01:51.220 --> 00:01:53.800
And writing the command line utilities.

00:01:53.800 --> 00:01:54.220
Yeah.

00:01:54.380 --> 00:01:56.140
Things like dealing with exit codes.

00:01:56.140 --> 00:02:00.880
So if you shell out to some other process, you run a command knowing whether it failed or it worked, right?

00:02:00.880 --> 00:02:01.460
Things like that.

00:02:01.460 --> 00:02:01.860
Yeah.

00:02:01.860 --> 00:02:06.480
And how to pipe between commands if you're going to pull things together.

00:02:06.480 --> 00:02:12.440
And then it doesn't jump into a lot of the really cool stuff with Python, but it's a good start.

00:02:12.440 --> 00:02:16.380
But it does like introduce people to the path library and sh...

00:02:16.380 --> 00:02:18.240
I don't know how you pronounce that.

00:02:18.240 --> 00:02:18.640
Shutil?

00:02:18.640 --> 00:02:19.460
S-H-U-T-I-L.

00:02:19.460 --> 00:02:21.500
And those are things.

00:02:21.500 --> 00:02:27.100
And then a lot of times you'll maybe using sed and awk and grep and things like that.

00:02:27.100 --> 00:02:33.100
And how to go about replacing that sort of stuff with regular expressions with regex.

00:02:33.100 --> 00:02:35.780
And so this is just fun.

00:02:35.780 --> 00:02:37.500
I like it to be able to...

00:02:37.500 --> 00:02:43.640
If you're writing a little utility and it's up to you as to what language you write it in, this might help you write it in Python.

00:02:43.640 --> 00:02:44.180
It's cool.

00:02:44.180 --> 00:02:55.720
Well, and I think one of the real key things to actually making programming stick when you're trying to learn it or even learning a new programming language, you know, some other kind of programming, is to have what you're doing be relevant.

00:02:55.720 --> 00:02:56.420
Yeah.

00:02:56.420 --> 00:02:57.360
To you, right?

00:02:57.360 --> 00:03:03.700
And so if you work in a DevOps world and you're like, well, I'm going to go learn Python, so I'm going to do this data science project.

00:03:03.700 --> 00:03:06.120
Like, that may be totally irrelevant to you.

00:03:06.120 --> 00:03:11.840
So having this as a resource for people who generally do scripting and automation type stuff, this is really cool.

00:03:11.840 --> 00:03:12.080
Yeah.

00:03:12.080 --> 00:03:13.700
And then also you're going to...

00:03:13.700 --> 00:03:14.440
The stuff you...

00:03:14.440 --> 00:03:17.560
When you're learning is stuff you start out writing isn't...

00:03:17.560 --> 00:03:20.140
Just to be frank, and it's the same with everybody.

00:03:20.140 --> 00:03:21.640
It's not that great to start with.

00:03:22.300 --> 00:03:26.340
So starting small is good, so you don't have something huge to rewrite later.

00:03:26.340 --> 00:03:27.860
True.

00:03:27.860 --> 00:03:36.400
I guess a lot of people, I know I did one in the earlier days, went through the experience of going back and reading code I wrote six months ago and going, wow, that's bad.

00:03:37.180 --> 00:03:49.120
But then that same person who looked back badly on the old code, six months further down the line, you look back and you're like, yep, that's bad too, just in a different way, right?

00:03:49.120 --> 00:03:50.840
But eventually, eventually you get there.

00:03:50.840 --> 00:03:52.860
But yeah, having the small stuff definitely helps.

00:03:52.860 --> 00:04:03.260
And I've heard a lot of people that, like in a DevOps world, where they just, you know, you start writing like one or two of your scripts in Python, and then you can just add to it.

00:04:03.260 --> 00:04:05.940
As you learn more stuff, you can do more powerful things.

00:04:05.940 --> 00:04:12.940
And having, like you said, having it be relevant to your job is important for what you're doing, for learning.

00:04:12.940 --> 00:04:13.260
Yep.

00:04:13.260 --> 00:04:14.560
It's a really cool find.

00:04:14.560 --> 00:04:16.900
I'm glad you brought it up, and I'm sure it'll help a ton of people.

00:04:16.900 --> 00:04:19.960
Now, this next topic, I'm pretty excited about.

00:04:19.960 --> 00:04:23.320
I'm honestly kind of beside myself about this one.

00:04:23.320 --> 00:04:26.440
This literally could be a game changer.

00:04:26.440 --> 00:04:30.860
And so this is called Pi Iodide.

00:04:30.860 --> 00:04:32.480
And I don't know if you've heard of it before.

00:04:32.480 --> 00:04:37.460
I had not heard of it until just beginning of this week, I think, actually.

00:04:37.460 --> 00:04:40.360
So Iodide is a project by Mozilla.

00:04:40.360 --> 00:04:44.960
And it's sort of, think of Iodide as like a competitor to Jupyter Notebooks,

00:04:45.640 --> 00:04:52.120
where the concept of Jupyter Notebooks is you have this web page, and it's like a view into a Jupyter server.

00:04:52.120 --> 00:04:57.580
And then when you execute code, that code executes on the server, which is a little sketchy,

00:04:57.580 --> 00:05:01.820
because if you're letting people run arbitrary Python code on your server, that's not amazing.

00:05:01.820 --> 00:05:05.860
So maybe you're using like some sort of Docker thing to like spin them up and shut them down.

00:05:05.860 --> 00:05:06.880
It gets complicated, right?

00:05:07.120 --> 00:05:14.580
Well, Mozilla wants to make this environment available to everybody, but in a distributed fashion.

00:05:14.580 --> 00:05:18.360
And so the goal is to bring this environment to the browser.

00:05:18.360 --> 00:05:22.980
And when you execute code, it executes on your browser client side.

00:05:22.980 --> 00:05:24.200
So you can do whatever you want.

00:05:24.840 --> 00:05:25.900
Well, that's cool.

00:05:25.900 --> 00:05:29.900
But the problem is, a lot of people want to do data science in Python.

00:05:29.900 --> 00:05:36.700
And Python, while it's had some JavaScript options for running code in the browser,

00:05:36.700 --> 00:05:39.900
they've always been like, well, you can kind of do this.

00:05:39.900 --> 00:05:42.100
And it's mostly like Python and so on.

00:05:42.100 --> 00:05:47.840
A while ago, as in two weeks ago, we asked the question of, will there be a PyBlazor?

00:05:48.040 --> 00:05:54.980
Remember, Blazor is the .NET community's attempt to bring the .NET CLR, their equivalent of CPython,

00:05:54.980 --> 00:05:59.540
into the browser with WebAssembly and make it run there.

00:05:59.540 --> 00:06:02.400
And they're making that actually work, which was kind of blowing my mind

00:06:02.400 --> 00:06:05.960
and giving me a lot of hope that maybe somebody could do it for CPython, right?

00:06:05.960 --> 00:06:06.820
And what do you think?

00:06:06.820 --> 00:06:07.440
Do you think it'll happen?

00:06:07.440 --> 00:06:09.240
Well, it's a nice dream.

00:06:09.240 --> 00:06:09.680
I agree.

00:06:09.680 --> 00:06:10.400
It's a nice dream.

00:06:10.400 --> 00:06:13.640
But sometimes dreams become reality, right?

00:06:13.640 --> 00:06:25.400
So PyIodide is an implementation, a running version of all of CPython, an exact bit of CPython,

00:06:25.400 --> 00:06:27.820
same thing, in the browser, in WebAssembly.

00:06:27.820 --> 00:06:29.600
That's so cool.

00:06:29.600 --> 00:06:31.820
Is that incredible or what?

00:06:31.820 --> 00:06:33.460
I mean, my mind is blown.

00:06:34.040 --> 00:06:42.000
So I reached out to one of the contributors, Michael D., and asked him, like, this is like

00:06:42.000 --> 00:06:44.840
really CPython running in WebAssembly in the browser.

00:06:44.840 --> 00:06:46.180
And he said, yeah.

00:06:46.180 --> 00:06:46.600
Yeah.

00:06:46.600 --> 00:06:48.400
So Michael works for Mozilla.

00:06:48.400 --> 00:06:49.760
This is a Mozilla project.

00:06:49.760 --> 00:06:56.140
And they're trying to bring all of CPython, at least the data science part of it, into the

00:06:56.140 --> 00:06:56.420
browser.

00:06:56.420 --> 00:06:58.840
Now, there's a few things that it won't do.

00:06:58.840 --> 00:06:59.980
Like, you can't open a socket.

00:07:00.420 --> 00:07:03.720
Obviously, Tkinter doesn't work, because what does that mean in the browser, right?

00:07:03.720 --> 00:07:07.000
So there's some parts of CPython that are not there.

00:07:07.000 --> 00:07:11.920
But if it's permitted in the browser, it's identical to CPython.

00:07:11.920 --> 00:07:13.400
And that, I think, is amazing.

00:07:13.400 --> 00:07:14.180
That is amazing.

00:07:14.180 --> 00:07:15.400
I can't wait to play with this.

00:07:15.400 --> 00:07:15.880
I can't either.

00:07:15.880 --> 00:07:20.460
And so they also want to have the data science tooling there.

00:07:20.460 --> 00:07:25.600
And so things like NumPy, Matplotlib, and so on, they have heavy dependencies upon C.

00:07:26.100 --> 00:07:30.500
So you can't just bring it over and run it in this WebAssembly version of CPython.

00:07:30.500 --> 00:07:36.660
So you have to take the C code that supports things like NumPy and Matplotlib and get them

00:07:36.660 --> 00:07:39.120
to compile to WebAssembly themselves.

00:07:39.120 --> 00:07:40.760
So they've also done that.

00:07:40.760 --> 00:07:47.320
So you have native NumPy, native Matplotlib on native CPython client side in the browser

00:07:47.320 --> 00:07:48.640
with WebAssembly.

00:07:48.640 --> 00:07:49.480
And it's fast.

00:07:49.480 --> 00:07:51.340
I'm still trying to get my head around this.

00:07:51.340 --> 00:07:52.540
How could it be fast?

00:07:52.540 --> 00:07:55.940
I mean, it's like binary stuff executing.

00:07:55.940 --> 00:07:57.380
And it's also not that big, right?

00:07:57.380 --> 00:08:02.840
The download, I noticed using the web tools, when you go and interact with this page, it

00:08:02.840 --> 00:08:06.140
downloads like CPython.wasm, right?

00:08:06.140 --> 00:08:07.580
The extension for WebAssembly.

00:08:07.580 --> 00:08:09.900
And it's 2.5 megs, right?

00:08:09.900 --> 00:08:13.440
And in my connection, that's like one and a half seconds to get that thing up and running.

00:08:13.440 --> 00:08:17.480
And then it doesn't have to parse it like it does like two megs of JavaScript.

00:08:17.480 --> 00:08:19.520
You don't make sense of that, right?

00:08:19.520 --> 00:08:20.620
Effectively compile it.

00:08:20.680 --> 00:08:22.760
It just takes it and runs it because it's already compiled.

00:08:22.760 --> 00:08:23.680
Oh, okay.

00:08:23.680 --> 00:08:24.480
That's okay.

00:08:24.480 --> 00:08:25.400
So that's not nothing.

00:08:25.400 --> 00:08:26.860
But that's pretty cool.

00:08:26.860 --> 00:08:27.340
It's not nothing.

00:08:27.340 --> 00:08:29.380
But it's not that bad.

00:08:29.380 --> 00:08:30.420
And you could cache.

00:08:30.420 --> 00:08:34.080
Somebody could set up a CDN for that thing and cache it like, say, Mozilla.

00:08:34.080 --> 00:08:34.540
Yeah.

00:08:34.540 --> 00:08:36.140
Like, say, Mozilla.

00:08:36.140 --> 00:08:39.860
So I think it's awesome.

00:08:39.860 --> 00:08:44.860
Tim, I want to just say thank you, Mozilla and Michael and team for working on this because

00:08:44.860 --> 00:08:46.580
this is amazing.

00:08:46.580 --> 00:08:48.320
And this could really, really change.

00:08:48.320 --> 00:08:52.980
It could really expand where Python is broadly applicable.

00:08:53.180 --> 00:09:02.380
Now, I think it's worth pointing out that this generally is built with the concept of making data science work in the browser, right?

00:09:02.380 --> 00:09:07.080
Whereas this PyBlazer thing, this Blazer, my imaginary PyBlazer thing that I made up,

00:09:07.400 --> 00:09:14.840
that was around building single-page apps, more of the web side of programming than the data science programming side of programming.

00:09:15.880 --> 00:09:23.080
But this thing does have extensions to let you work with HTML and the DOM and events and stuff like that.

00:09:23.080 --> 00:09:32.720
So you could, it's not a long shot to go from this to having an interactive, rich application for a web app in Python.

00:09:32.720 --> 00:09:33.280
Yeah.

00:09:33.280 --> 00:09:37.560
So it's still not quite where we want, but, like, really cool.

00:09:37.560 --> 00:09:38.300
It's really cool.

00:09:38.300 --> 00:09:41.080
And, I mean, it's totally working for what they're trying to do.

00:09:41.080 --> 00:09:50.840
But I'm just thinking, like, if I want to, say, replace, like, a React thing or an AngularJS thing with Python,

00:09:50.840 --> 00:09:56.180
there might be a few more steps to get, like, that angle of stuff working.

00:09:56.180 --> 00:10:00.520
But it shouldn't be that far because we already have the CPython runtime working,

00:10:00.520 --> 00:10:03.660
which is really what you need, and the integration with HTML.

00:10:03.660 --> 00:10:04.140
Yeah.

00:10:04.140 --> 00:10:05.160
So this is cool.

00:10:05.160 --> 00:10:07.600
I want a lot of people to grab this and run with it.

00:10:07.600 --> 00:10:07.880
Yeah.

00:10:07.880 --> 00:10:11.120
I think this really opens up the possibility for some amazing stuff.

00:10:11.120 --> 00:10:13.740
So, you know, once again, thank you, Mozilla.

00:10:13.740 --> 00:10:15.740
And I also, I just want to reflect a little bit.

00:10:15.740 --> 00:10:19.920
I think it's really ironic or interesting coincidence.

00:10:19.920 --> 00:10:26.300
At PyCon 2018, as we mentioned when we talked about the Blazor thing, I think it was Tim Callahan.

00:10:26.300 --> 00:10:28.300
I'm still forgetting the name.

00:10:28.300 --> 00:10:35.120
But somebody from Mozilla came and gave an impassioned presentation as the opening keynote of,

00:10:35.120 --> 00:10:44.440
please, let's find a way to use WebAssembly to get Python more capable in the web space and specifically in the browser and client side.

00:10:44.440 --> 00:10:52.820
And I think it's just pretty awesome that Mozilla, another team in Mozilla, actually is the one that implemented this.

00:10:52.900 --> 00:10:54.700
So that's really, it's really amazing.

00:10:54.700 --> 00:11:00.360
So these guys working on PyIodide, they may want to document their stuff, right?

00:11:00.360 --> 00:11:01.220
Yeah.

00:11:01.220 --> 00:11:04.060
You're so much better at the transition thing than I am.

00:11:04.060 --> 00:11:05.400
I'm like, oh, next topic.

00:11:05.400 --> 00:11:08.400
I personally like, I like Markdown.

00:11:08.400 --> 00:11:16.380
I write, even when I have to use, to be honest, when I have to use Microsoft Word, I write in Markdown and then I convert it to Word.

00:11:16.580 --> 00:11:22.620
Anyway, I don't know why I was talking about Word, but so restructured text is not something that comes naturally to me.

00:11:22.620 --> 00:11:27.160
But there's a lot of Python documentation that's written in restructured text.

00:11:27.160 --> 00:11:35.680
So if you're going to contribute to an existing project, you can't say, well, I can't do any of the documentation because it's all in restructured text.

00:11:35.680 --> 00:11:36.560
Convert it to Markdown.

00:11:36.560 --> 00:11:37.380
Just use Markdown.

00:11:37.380 --> 00:11:39.800
No, we're not rewriting all of this in Markdown.

00:11:39.800 --> 00:11:40.020
Right.

00:11:40.180 --> 00:11:45.020
So whether or not you like it, restructured text is part of the Python world.

00:11:45.020 --> 00:11:50.380
So this is a wonderful, there's a wonderful article from Simon Will.

00:11:50.380 --> 00:11:51.940
Oh, I always had his name wrong.

00:11:51.940 --> 00:11:54.460
I always thought it was Wilson, but it's Simon Willison.

00:11:54.460 --> 00:11:58.100
A subset of restructured text worth committing to memory.

00:11:58.100 --> 00:12:00.460
And that's a noble goal.

00:12:00.460 --> 00:12:04.380
But instead of committing it to memory, I'm just going to bookmark this article.

00:12:04.780 --> 00:12:08.880
But he's got the necessary things for easy documentation.

00:12:08.880 --> 00:12:14.760
You have paragraphs, headings, bulleted lists, how to do italics and things like that.

00:12:14.760 --> 00:12:24.740
And then, of course, for documenting code, you need how to do inline code and code snippets and images and internal references and things like that.

00:12:24.740 --> 00:12:33.220
And there's a lot covered here, but one of the issues that I always had was the tutorials on restructured text.

00:12:33.220 --> 00:12:35.220
Restructured text can do a lot.

00:12:35.220 --> 00:12:36.680
And it's kind of overwhelming.

00:12:36.680 --> 00:12:42.120
And you don't need all of that to understand enough to help document a Python project.

00:12:42.120 --> 00:12:43.120
So this is great.

00:12:43.120 --> 00:12:43.740
Yeah, it's cool.

00:12:43.740 --> 00:12:45.920
These are the five or six things you need.

00:12:45.920 --> 00:12:50.620
Headings, links, bulleted lists, numerical lists, images, things like that, right?

00:12:50.620 --> 00:12:52.180
Yeah, and then how to get your code in there.

00:12:52.180 --> 00:12:53.020
Yeah, exactly.

00:12:53.020 --> 00:12:54.200
It's great.

00:12:54.200 --> 00:13:00.220
I'm going to make use of this one that you're sharing here because I'm working on a project where I have to write a lot of restructured text.

00:13:00.220 --> 00:13:04.580
And, yeah, I basically have doubled my knowledge of restructured text by reading that.

00:13:04.580 --> 00:13:06.460
Yeah.

00:13:06.460 --> 00:13:09.580
Because like you, I do markdown most of the time.

00:13:09.580 --> 00:13:20.040
So to be honest, most of the time when I needed to do something in restructured text, I would just find somebody else that did something like that and copy it and not know why I was doing these things.

00:13:20.040 --> 00:13:23.420
So having a little primer and tutorial is good.

00:13:23.660 --> 00:13:25.500
Yeah, it's great.

00:13:25.500 --> 00:13:29.220
Speaking of finding things, let's talk about Datadog really quick.

00:13:29.320 --> 00:13:35.980
So if you have issues, performance problems, bugs, things like that with your application, you should check out Datadog.

00:13:35.980 --> 00:13:43.640
So they're sponsoring this episode.

00:13:43.640 --> 00:13:51.300
And they have this new feature called Trace Search and Analytics.

00:13:51.300 --> 00:14:03.100
around performance for things like show me all of the operations that this customer or this user did or all the requests that go to this URL and aggregate the performance of that, which is pretty awesome.

00:14:03.100 --> 00:14:05.900
So start troubleshooting and optimizing your Python apps today.

00:14:06.140 --> 00:14:10.020
The free trial and Datadog will send you a free cute little t-shirt.

00:14:10.020 --> 00:14:12.600
So check them out at pythonbytes.fm/Datadog.

00:14:12.800 --> 00:14:16.800
So I have a theme for the rest of the items I've chosen, Brian.

00:14:16.800 --> 00:14:17.220
Okay.

00:14:17.220 --> 00:14:22.660
So have you ever heard that some software has security problems people can break into it?

00:14:22.660 --> 00:14:23.580
Yeah.

00:14:24.480 --> 00:14:28.320
You know, even though Python's not compiled, Python code can have it too, right?

00:14:28.320 --> 00:14:29.300
I assume so.

00:14:29.300 --> 00:14:29.720
Yeah.

00:14:29.720 --> 00:14:36.140
Well, our friend Anthony Shaw told us about this cool project, a little app you can run.

00:14:36.140 --> 00:14:38.080
It's really a linter, basically.

00:14:38.080 --> 00:14:40.880
You run against your program called Bandit.

00:14:40.880 --> 00:14:41.320
Okay.

00:14:41.540 --> 00:14:45.920
So Bandit is designed to find common security issues in Python code.

00:14:45.920 --> 00:14:59.620
So what it does is you point it at your project and it'll load up each file into an abstract syntax tree, run a bunch of analysis over that AST, and then it'll generate a report.

00:14:59.620 --> 00:15:06.860
And it'll tell you things like you're misusing the Telenet lib or you're using eval or you're doing try except pass and eating errors.

00:15:07.240 --> 00:15:11.520
So opening a subprocess with popen with shell equals true.

00:15:11.520 --> 00:15:14.780
I actually don't know what's wrong with that, but I should probably figure that out.

00:15:14.780 --> 00:15:15.140
Yeah.

00:15:15.140 --> 00:15:16.540
But apparently it's bad.

00:15:16.540 --> 00:15:17.860
Shouldn't do it.

00:15:17.860 --> 00:15:19.000
And that's error 602.

00:15:19.000 --> 00:15:23.700
So it'll tell you, you know, for example, like I wouldn't know that that's necessarily bad.

00:15:23.700 --> 00:15:26.180
I know I've heard it before, but I don't remember why it's bad.

00:15:26.180 --> 00:15:28.120
So it's pretty cool.

00:15:28.120 --> 00:15:34.260
And it just has a whole bunch of errors it'll find and, you know, security lapses it'll find and tell you about them.

00:15:34.260 --> 00:15:36.960
So this is something that you, it's like a stack analysis.

00:15:36.960 --> 00:15:38.880
It's not, it's not running your code.

00:15:38.880 --> 00:15:39.900
It's looking at.

00:15:39.900 --> 00:15:40.880
No, it's, it's static.

00:15:40.880 --> 00:15:41.200
Yeah.

00:15:41.200 --> 00:15:41.980
It's static analysis.

00:15:41.980 --> 00:15:42.880
Oh, that's cool.

00:15:42.880 --> 00:15:43.340
Yeah.

00:15:43.340 --> 00:15:43.720
Isn't that nice?

00:15:43.720 --> 00:15:43.940
Yeah.

00:15:43.940 --> 00:15:44.680
Yeah.

00:15:44.680 --> 00:15:49.360
It's like, I see you did a try except and you just ate that error and that can actually,

00:15:49.360 --> 00:15:51.600
you know, cause other types of problems.

00:15:51.600 --> 00:15:54.440
Like maybe you should have validated something, but instead you just keep going.

00:15:54.440 --> 00:15:55.000
Things like that.

00:15:55.000 --> 00:15:59.940
So this might be good even for non, I mean, it's definitely important for, for Python projects

00:15:59.940 --> 00:16:06.480
that, that have a interface to the web or like a web application, but you could use this for

00:16:06.480 --> 00:16:07.300
a lot of stuff.

00:16:07.300 --> 00:16:08.260
Not just that.

00:16:08.260 --> 00:16:08.620
Sure.

00:16:08.620 --> 00:16:09.020
So.

00:16:09.560 --> 00:16:09.760
Yeah.

00:16:09.760 --> 00:16:09.960
Yeah.

00:16:09.960 --> 00:16:13.540
It definitely lets you know when you're doing things wrong in like a different category,

00:16:13.540 --> 00:16:14.740
say than probably flake eight.

00:16:14.740 --> 00:16:15.200
Yeah.

00:16:15.200 --> 00:16:15.880
Neat.

00:16:15.880 --> 00:16:16.140
Yeah.

00:16:16.140 --> 00:16:19.620
And because it's, you know, you could easily integrate this type of thing into a continuous

00:16:19.620 --> 00:16:20.880
build integration pipeline.

00:16:20.880 --> 00:16:21.460
Yeah.

00:16:21.780 --> 00:16:23.160
I'll definitely have to check this out.

00:16:23.160 --> 00:16:24.080
Cool.

00:16:24.080 --> 00:16:24.360
Yeah.

00:16:24.360 --> 00:16:27.040
This is, it's, it even probably has a little testing angle to it somehow.

00:16:27.040 --> 00:16:27.520
Maybe.

00:16:27.520 --> 00:16:28.060
Yeah.

00:16:28.060 --> 00:16:29.040
Maybe.

00:16:29.040 --> 00:16:29.920
Awesome.

00:16:29.920 --> 00:16:30.280
All right.

00:16:30.280 --> 00:16:31.040
What you got next for us?

00:16:31.040 --> 00:16:35.620
Well, last week we talked about what Netflix was up to with Jupyter Notebooks.

00:16:35.620 --> 00:16:40.520
So I was, and I mentioned that I've kind of been trying to learn more about them.

00:16:40.520 --> 00:16:48.580
So one of the, we had somebody submit a, another, it's another repo, but it's, it's a whole

00:16:48.580 --> 00:16:49.260
bunch of notebooks.

00:16:49.600 --> 00:16:54.120
So it's a, what he's put together is learn Python three.

00:16:54.120 --> 00:16:55.760
I think I forgot the wrong link here.

00:16:55.760 --> 00:16:58.780
Learn Python three with a Jupyter Notebook.

00:16:58.780 --> 00:17:03.300
And so I was actually playing with this morning and it was just a blast.

00:17:03.300 --> 00:17:10.920
So we, we have a, it's got beginner and intermediate Python concepts and you just sort of like can

00:17:10.920 --> 00:17:11.540
walk through it.

00:17:11.540 --> 00:17:16.680
And one of the things I had to learn, of course, is to at first is just to how to, how to do

00:17:16.680 --> 00:17:16.960
this.

00:17:16.960 --> 00:17:23.520
How to clone the, I cloned the repo and just ran and then installed Jupyter and then ran,

00:17:23.520 --> 00:17:26.420
ran the, just say, what is it?

00:17:26.420 --> 00:17:31.460
You just say Jupyter Notebook or something and it opens up a web browser and then you can click

00:17:31.460 --> 00:17:34.940
different notebooks and run them and, and see what they do.

00:17:35.100 --> 00:17:39.220
And the thing that really intrigued me that the bone that he put in there for me to try

00:17:39.220 --> 00:17:42.660
to grab is there's two topics on testing.

00:17:42.660 --> 00:17:49.660
And I've been asked this before of like, how do you, can you run a pytest within a notebook?

00:17:50.360 --> 00:17:52.580
And I didn't know if you could or not.

00:17:52.580 --> 00:17:58.260
I, I heard rumors, but there's two examples in here of how to, two notebooks that have

00:17:58.260 --> 00:17:59.400
tests embedded in them.

00:17:59.400 --> 00:18:04.780
So you can, you can run your tests within, so you can define a, define a function or something

00:18:04.780 --> 00:18:09.540
in a different cell and then write a test for it in another cell and then run it.

00:18:09.540 --> 00:18:10.440
I think that's really cool.

00:18:10.440 --> 00:18:14.180
And I had no idea how to run pytest either in there.

00:18:14.180 --> 00:18:18.180
And I always sort of saw that as like one of the weaknesses of notebooks.

00:18:18.180 --> 00:18:21.580
I have lots of respect for them, but one of the things I don't really like is it's hard

00:18:21.580 --> 00:18:24.960
to do like, how do I run tests against this code, this code?

00:18:24.960 --> 00:18:29.100
How do I like get a code coverage report of the code?

00:18:29.100 --> 00:18:31.240
My Jupyter Notebook is doing things like that, right?

00:18:31.240 --> 00:18:35.120
It seems much easier when you have a set of files that you can like call directly.

00:18:35.120 --> 00:18:35.560
Yeah.

00:18:35.560 --> 00:18:40.300
And I'm still not, so this is running pytest within the notebook and I'm not, I think there's

00:18:40.300 --> 00:18:45.460
also a way that I haven't figured out yet to take a notebook and run, if there's tests

00:18:45.460 --> 00:18:48.080
inside of it, run the run tests against it.

00:18:48.080 --> 00:18:48.340
Yeah.

00:18:48.340 --> 00:18:54.320
There may be in the stuff that Netflix talked about last week, allow you to basically take

00:18:54.320 --> 00:18:57.620
a notebook and execute it as if it were a function with an input and an output.

00:18:57.620 --> 00:19:01.580
And like that in some sense would open up a testing angle to itself, right?

00:19:01.580 --> 00:19:02.520
Definitely.

00:19:02.520 --> 00:19:06.620
And especially if you like save the output somewhere, that'd be great.

00:19:06.620 --> 00:19:08.580
Anyway, it was just really fun.

00:19:08.580 --> 00:19:10.080
It was a, it's fun to look around.

00:19:10.160 --> 00:19:17.020
The other thing that I like in, as an example is there's the examples of how to do pip install

00:19:17.020 --> 00:19:17.940
within a notebook.

00:19:17.940 --> 00:19:22.460
It has those at the top because it doesn't assume that you've installed pytest to begin

00:19:22.460 --> 00:19:22.660
with.

00:19:22.660 --> 00:19:23.400
Oh yeah, that's cool.

00:19:23.400 --> 00:19:24.420
And it shows that.

00:19:24.420 --> 00:19:31.500
And then it's using another pip installable package called the ipytest, which is how it's

00:19:31.500 --> 00:19:33.080
running pytest within the notebook.

00:19:33.080 --> 00:19:33.900
So it's cool.

00:19:33.900 --> 00:19:34.920
That is really cool.

00:19:34.920 --> 00:19:36.000
Nice find.

00:19:36.000 --> 00:19:36.620
All right.

00:19:36.620 --> 00:19:37.140
Last one.

00:19:37.200 --> 00:19:38.440
I said I was on a bit of a theme.

00:19:38.440 --> 00:19:45.320
So we talked about Bandit and Bandit lets you find code that is being either using a bad

00:19:45.320 --> 00:19:50.220
library or is using a bad pattern, like tracks that pass, things like that.

00:19:50.220 --> 00:19:56.520
This next one takes a different angle on the same sort of problem around security and integrity

00:19:56.520 --> 00:19:57.380
of your application.

00:19:57.380 --> 00:19:59.740
And it's called detect secrets.

00:19:59.740 --> 00:20:06.680
So their sort of tagline there is an enterprise friendly way of detecting and preventing secrets

00:20:06.680 --> 00:20:07.220
in code.

00:20:07.220 --> 00:20:09.320
And this is actually released from Yelp.

00:20:10.080 --> 00:20:15.860
So on one hand, like using code badly, that's a problem, but also put hard coding passwords

00:20:15.860 --> 00:20:21.200
and other things like that into your code can be a problem, right?

00:20:21.200 --> 00:20:22.460
Yeah, definitely.

00:20:23.000 --> 00:20:29.200
And so the enterprise friendly aspect, I think, comes from the fact that it says, all right,

00:20:29.200 --> 00:20:31.500
look, we assume that your code is just messed up.

00:20:31.500 --> 00:20:33.140
There's all sorts of badness in here.

00:20:33.140 --> 00:20:37.520
What do you do if you find yourself in a hole and things keep getting worse?

00:20:37.520 --> 00:20:39.880
You know, first thing you do is stop digging, right?

00:20:39.880 --> 00:20:42.040
Before you get out of the hole, you just stop making it worse.

00:20:42.040 --> 00:20:47.880
So there's a lot of mechanisms in detect secrets to say, let's not make this worse.

00:20:47.880 --> 00:20:50.920
Let's prevent new secrets from going into the code base.

00:20:51.120 --> 00:20:55.720
Now we can start to peel off, like start to like remove some of these things that are

00:20:55.720 --> 00:20:56.620
hard coded in there.

00:20:56.620 --> 00:21:02.300
But let's go and set a baseline and go, all right, if we get any new detections of secrets

00:21:02.300 --> 00:21:03.920
being entered, that's going to fail the build.

00:21:03.920 --> 00:21:05.860
But the way it is now, we're going to accept it.

00:21:05.860 --> 00:21:10.200
And then we can like make pass after pass to actually take away some of those and reset

00:21:10.200 --> 00:21:10.720
the baseline.

00:21:10.720 --> 00:21:12.080
Okay, interesting.

00:21:12.080 --> 00:21:12.780
Yeah, it's pretty.

00:21:12.780 --> 00:21:18.000
Also, there's some things that look like they're secrets that should be detected, but you've

00:21:18.000 --> 00:21:20.540
inspected the code and said, no, this one's fine.

00:21:20.540 --> 00:21:21.300
Yeah, exactly.

00:21:21.300 --> 00:21:26.020
And that baseline thing will let you say, I approve of these, but not anything else.

00:21:26.020 --> 00:21:26.400
Yeah.

00:21:26.400 --> 00:21:27.040
Cool.

00:21:27.040 --> 00:21:27.780
Which is pretty cool.

00:21:27.780 --> 00:21:30.940
And they also show you how to set it up as a git commit hook.

00:21:31.940 --> 00:21:36.020
So as you know, from the testing space, one of the challenges is like, you can have awesome

00:21:36.020 --> 00:21:41.240
tests, but if no one runs them or worse, one or two people on the team are very passionate

00:21:41.240 --> 00:21:41.880
about them.

00:21:41.880 --> 00:21:46.440
And a whole bunch of other people hate them and refuse to run them no matter how much you

00:21:46.440 --> 00:21:47.080
talk about it.

00:21:47.580 --> 00:21:54.120
The people who care about the test suffer from the neglect that the people who explicitly

00:21:54.120 --> 00:21:54.800
ignore it, right?

00:21:54.800 --> 00:21:58.600
Maybe that person enters some problem that'll fail a test, but they don't run their test,

00:21:58.600 --> 00:21:59.280
so they don't know.

00:21:59.280 --> 00:22:00.180
You get it.

00:22:00.180 --> 00:22:02.880
You try to run your test and then you're hit with their problems as well, right?

00:22:03.140 --> 00:22:09.740
So this git commit hook here basically means everyone on the team is going to be basically

00:22:09.740 --> 00:22:11.440
running this as part of a check-in.

00:22:11.440 --> 00:22:14.240
The whole idea around git commit hooks is interesting.

00:22:14.240 --> 00:22:19.760
There's a lot of interesting things people are doing, like making sure that there's a plaque

00:22:19.760 --> 00:22:24.420
has a git commit hook as part of it or attached to it so people can make sure their code is still

00:22:24.420 --> 00:22:25.620
black before they check it in.

00:22:25.620 --> 00:22:27.280
Yeah, that's the one I was thinking of as well.

00:22:27.280 --> 00:22:33.000
And it can even automatically reformat your code to the group's format as part of the check-in,

00:22:33.000 --> 00:22:33.200
right?

00:22:33.200 --> 00:22:35.360
I'd be curious to know if there's any other...

00:22:35.360 --> 00:22:40.000
I'd like to have people let us know if there's other git commit hooks that we haven't talked

00:22:40.000 --> 00:22:41.760
about that are helping their teams out.

00:22:41.760 --> 00:22:43.960
Maybe they'd help other people out too, so...

00:22:43.960 --> 00:22:47.340
Yeah, it'd be fun to collect like a list of use cases of what people are doing.

00:22:47.340 --> 00:22:47.680
Yeah.

00:22:47.680 --> 00:22:51.740
Yeah, if you're doing something cool with a git commit hook, check out pythonbytes.fm

00:22:51.740 --> 00:22:54.300
slash 93 and put a comment at the bottom.

00:22:54.300 --> 00:22:58.120
That'll be in the show notes, you know, sort of below the show notes for everyone to come

00:22:58.120 --> 00:22:58.340
see.

00:22:58.340 --> 00:23:01.480
All right, well, that's all of our items for this week.

00:23:01.700 --> 00:23:04.640
Brian, you got anything personal you want to share before we're out of here?

00:23:04.640 --> 00:23:12.400
No, I guess pytest came out with a new version the other day, which kind of blew me away because

00:23:12.400 --> 00:23:17.660
I had submitted a very minor bug, but I had submitted a bug to the project.

00:23:17.660 --> 00:23:22.820
And I only submitted it like five days ago, and it's already now in a fixed and in a release.

00:23:22.820 --> 00:23:24.240
So that's amazing.

00:23:24.400 --> 00:23:25.460
That's a really good turnaround time.

00:23:25.460 --> 00:23:25.860
That's awesome.

00:23:25.860 --> 00:23:26.900
Yeah, that's fun.

00:23:26.900 --> 00:23:28.360
Anything with you?

00:23:28.360 --> 00:23:29.180
Got any news?

00:23:29.180 --> 00:23:30.480
Nothing I'm ready to talk about.

00:23:30.480 --> 00:23:35.380
I'm working on a couple of new projects, one that I'm doing restructured text for, a couple

00:23:35.380 --> 00:23:40.240
of new courses are coming down the pipe, things like that, but nothing ready to talk about

00:23:40.240 --> 00:23:40.680
just yet.

00:23:40.680 --> 00:23:41.500
All right.

00:23:41.500 --> 00:23:43.540
So how's that for a long-winded no?

00:23:43.540 --> 00:23:46.840
Well, it was fun talking to you.

00:23:46.960 --> 00:23:47.680
Yeah, you as well.

00:23:47.680 --> 00:23:49.600
Thanks for everything, and see you later.

00:23:49.600 --> 00:23:50.000
All right.

00:23:50.000 --> 00:23:50.180
Bye.

00:23:50.180 --> 00:23:52.260
Thank you for listening to Python Bytes.

00:23:52.260 --> 00:23:54.820
Follow the show on Twitter via at Python Bytes.

00:23:54.820 --> 00:23:57.720
That's Python Bytes as in B-Y-T-E-S.

00:23:57.720 --> 00:24:01.120
And get the full show notes at pythonbytes.fm.

00:24:01.120 --> 00:24:05.480
If you have a news item you want featured, just visit pythonbytes.fm and send it our way.

00:24:05.480 --> 00:24:08.200
We're always on the lookout for sharing something cool.

00:24:08.200 --> 00:24:11.580
On behalf of myself and Brian Okken, this is Michael Kennedy.

00:24:11.580 --> 00:24:15.200
Thank you for listening and sharing this podcast with your friends and colleagues.