Transcript #93: Looking like there will be a PyBlazor!

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 93, recorded August 30th, 2018.

00:10 I'm Michael Kennedy.

00:11 And I'm Brian Okken.

00:11 And Brian, you know, last time we had some awesome drop the mic stuff.

00:15 I think we're going to do it again, aren't we?

00:17 Yeah, I think it's some really interesting things today.

00:20 Absolutely.

00:21 So before we get on to them, though, I just want to say thanks to Datadog.

00:24 Datadog's sponsoring the show.

00:26 Check them out at pythonbytes.fm/datadog.

00:30 I'll tell you more about that later.

00:31 Brian, a lot of people get into programming, kind of in air quotes there, by doing scripting around like the shell and stuff, right?

00:39 I think so.

00:39 I think that's the way some people like in the IT world are...

00:42 Like Bash or even bat files.

00:45 Or lots of DevOps roles.

00:48 Yeah, exactly.

00:49 At the point where I learned Python, I was doing a lot of C++ development and writing little utility scripts in Bash and Perl and stuff like that.

00:59 And so one of the things that I did, you know, I want to try to learn Python by writing as much Python as I can.

01:06 And so I started doing more and more of my utility scripts in Python.

01:14 And so I think that was a good way to just try to jump in and solve little problems.

01:19 I could have written it faster in Bash because I knew that or in Perl.

01:23 But learning how to do things in Python is good.

01:26 So it's a GitHub repo that's called replacing Bash scripting with Python.

01:32 And I think it goes through a lot of stuff that...

01:35 A lot of the issues that you'd need to do that you'd normally write in the little utilities and how to do that in Python.

01:42 So this is fun.

01:43 It talks about like dealing with files, reading and writing files, dealing with standard in and stuttered out, standard error, things like that.

01:51 And writing the command line utilities.

01:53 Yeah.

01:54 Things like dealing with exit codes.

01:56 So if you shell out to some other process, you run a command knowing whether it failed or it worked, right?

02:00 Things like that.

02:01 Yeah.

02:01 And how to pipe between commands if you're going to pull things together.

02:06 And then it doesn't jump into a lot of the really cool stuff with Python, but it's a good start.

02:12 But it does like introduce people to the path library and sh...

02:16 I don't know how you pronounce that.

02:18 Shutil?

02:18 S-H-U-T-I-L.

02:19 And those are things.

02:21 And then a lot of times you'll maybe using sed and awk and grep and things like that.

02:27 And how to go about replacing that sort of stuff with regular expressions with regex.

02:33 And so this is just fun.

02:35 I like it to be able to...

02:37 If you're writing a little utility and it's up to you as to what language you write it in, this might help you write it in Python.

02:43 It's cool.

02:44 Well, and I think one of the real key things to actually making programming stick when you're trying to learn it or even learning a new programming language, you know, some other kind of programming, is to have what you're doing be relevant.

02:55 Yeah.

02:56 To you, right?

02:57 And so if you work in a DevOps world and you're like, well, I'm going to go learn Python, so I'm going to do this data science project.

03:03 Like, that may be totally irrelevant to you.

03:06 So having this as a resource for people who generally do scripting and automation type stuff, this is really cool.

03:11 Yeah.

03:12 And then also you're going to...

03:13 The stuff you...

03:14 When you're learning is stuff you start out writing isn't...

03:17 Just to be frank, and it's the same with everybody.

03:20 It's not that great to start with.

03:22 So starting small is good, so you don't have something huge to rewrite later.

03:26 True.

03:27 I guess a lot of people, I know I did one in the earlier days, went through the experience of going back and reading code I wrote six months ago and going, wow, that's bad.

03:37 But then that same person who looked back badly on the old code, six months further down the line, you look back and you're like, yep, that's bad too, just in a different way, right?

03:49 But eventually, eventually you get there.

03:50 But yeah, having the small stuff definitely helps.

03:52 And I've heard a lot of people that, like in a DevOps world, where they just, you know, you start writing like one or two of your scripts in Python, and then you can just add to it.

04:03 As you learn more stuff, you can do more powerful things.

04:05 And having, like you said, having it be relevant to your job is important for what you're doing, for learning.

04:12 Yep.

04:13 It's a really cool find.

04:14 I'm glad you brought it up, and I'm sure it'll help a ton of people.

04:16 Now, this next topic, I'm pretty excited about.

04:19 I'm honestly kind of beside myself about this one.

04:23 This literally could be a game changer.

04:26 And so this is called Pi Iodide.

04:30 And I don't know if you've heard of it before.

04:32 I had not heard of it until just beginning of this week, I think, actually.

04:37 So Iodide is a project by Mozilla.

04:40 And it's sort of, think of Iodide as like a competitor to Jupyter Notebooks,

04:45 where the concept of Jupyter Notebooks is you have this web page, and it's like a view into a Jupyter server.

04:52 And then when you execute code, that code executes on the server, which is a little sketchy,

04:57 because if you're letting people run arbitrary Python code on your server, that's not amazing.

05:01 So maybe you're using like some sort of Docker thing to like spin them up and shut them down.

05:05 It gets complicated, right?

05:07 Well, Mozilla wants to make this environment available to everybody, but in a distributed fashion.

05:14 And so the goal is to bring this environment to the browser.

05:18 And when you execute code, it executes on your browser client side.

05:22 So you can do whatever you want.

05:24 Well, that's cool.

05:25 But the problem is, a lot of people want to do data science in Python.

05:29 And Python, while it's had some JavaScript options for running code in the browser,

05:36 they've always been like, well, you can kind of do this.

05:39 And it's mostly like Python and so on.

05:42 A while ago, as in two weeks ago, we asked the question of, will there be a PyBlazor?

05:48 Remember, Blazor is the .NET community's attempt to bring the .NET CLR, their equivalent of CPython,

05:54 into the browser with WebAssembly and make it run there.

05:59 And they're making that actually work, which was kind of blowing my mind

06:02 and giving me a lot of hope that maybe somebody could do it for CPython, right?

06:05 And what do you think?

06:06 Do you think it'll happen?

06:07 Well, it's a nice dream.

06:09 I agree.

06:09 It's a nice dream.

06:10 But sometimes dreams become reality, right?

06:13 So PyIodide is an implementation, a running version of all of CPython, an exact bit of CPython,

06:25 same thing, in the browser, in WebAssembly.

06:27 That's so cool.

06:29 Is that incredible or what?

06:31 I mean, my mind is blown.

06:34 So I reached out to one of the contributors, Michael D., and asked him, like, this is like

06:42 really CPython running in WebAssembly in the browser.

06:44 And he said, yeah.

06:46 Yeah.

06:46 So Michael works for Mozilla.

06:48 This is a Mozilla project.

06:49 And they're trying to bring all of CPython, at least the data science part of it, into the

06:56 browser.

06:56 Now, there's a few things that it won't do.

06:58 Like, you can't open a socket.

07:00 Obviously, Tkinter doesn't work, because what does that mean in the browser, right?

07:03 So there's some parts of CPython that are not there.

07:07 But if it's permitted in the browser, it's identical to CPython.

07:11 And that, I think, is amazing.

07:13 That is amazing.

07:14 I can't wait to play with this.

07:15 I can't either.

07:15 And so they also want to have the data science tooling there.

07:20 And so things like NumPy, Matplotlib, and so on, they have heavy dependencies upon C.

07:26 So you can't just bring it over and run it in this WebAssembly version of CPython.

07:30 So you have to take the C code that supports things like NumPy and Matplotlib and get them

07:36 to compile to WebAssembly themselves.

07:39 So they've also done that.

07:40 So you have native NumPy, native Matplotlib on native CPython client side in the browser

07:47 with WebAssembly.

07:48 And it's fast.

07:49 I'm still trying to get my head around this.

07:51 How could it be fast?

07:52 I mean, it's like binary stuff executing.

07:55 And it's also not that big, right?

07:57 The download, I noticed using the web tools, when you go and interact with this page, it

08:02 downloads like CPython.wasm, right?

08:06 The extension for WebAssembly.

08:07 And it's 2.5 megs, right?

08:09 And in my connection, that's like one and a half seconds to get that thing up and running.

08:13 And then it doesn't have to parse it like it does like two megs of JavaScript.

08:17 You don't make sense of that, right?

08:19 Effectively compile it.

08:20 It just takes it and runs it because it's already compiled.

08:22 Oh, okay.

08:23 That's okay.

08:24 So that's not nothing.

08:25 But that's pretty cool.

08:26 It's not nothing.

08:27 But it's not that bad.

08:29 And you could cache.

08:30 Somebody could set up a CDN for that thing and cache it like, say, Mozilla.

08:34 Yeah.

08:34 Like, say, Mozilla.

08:36 So I think it's awesome.

08:39 Tim, I want to just say thank you, Mozilla and Michael and team for working on this because

08:44 this is amazing.

08:46 And this could really, really change.

08:48 It could really expand where Python is broadly applicable.

08:53 Now, I think it's worth pointing out that this generally is built with the concept of making data science work in the browser, right?

09:02 Whereas this PyBlazer thing, this Blazer, my imaginary PyBlazer thing that I made up,

09:07 that was around building single-page apps, more of the web side of programming than the data science programming side of programming.

09:15 But this thing does have extensions to let you work with HTML and the DOM and events and stuff like that.

09:23 So you could, it's not a long shot to go from this to having an interactive, rich application for a web app in Python.

09:32 Yeah.

09:33 So it's still not quite where we want, but, like, really cool.

09:37 It's really cool.

09:38 And, I mean, it's totally working for what they're trying to do.

09:41 But I'm just thinking, like, if I want to, say, replace, like, a React thing or an AngularJS thing with Python,

09:50 there might be a few more steps to get, like, that angle of stuff working.

09:56 But it shouldn't be that far because we already have the CPython runtime working,

10:00 which is really what you need, and the integration with HTML.

10:03 Yeah.

10:04 So this is cool.

10:05 I want a lot of people to grab this and run with it.

10:07 Yeah.

10:07 I think this really opens up the possibility for some amazing stuff.

10:11 So, you know, once again, thank you, Mozilla.

10:13 And I also, I just want to reflect a little bit.

10:15 I think it's really ironic or interesting coincidence.

10:19 At PyCon 2018, as we mentioned when we talked about the Blazor thing, I think it was Tim Callahan.

10:26 I'm still forgetting the name.

10:28 But somebody from Mozilla came and gave an impassioned presentation as the opening keynote of,

10:35 please, let's find a way to use WebAssembly to get Python more capable in the web space and specifically in the browser and client side.

10:44 And I think it's just pretty awesome that Mozilla, another team in Mozilla, actually is the one that implemented this.

10:52 So that's really, it's really amazing.

10:54 So these guys working on PyIodide, they may want to document their stuff, right?

11:00 Yeah.

11:01 You're so much better at the transition thing than I am.

11:04 I'm like, oh, next topic.

11:05 I personally like, I like Markdown.

11:08 I write, even when I have to use, to be honest, when I have to use Microsoft Word, I write in Markdown and then I convert it to Word.

11:16 Anyway, I don't know why I was talking about Word, but so restructured text is not something that comes naturally to me.

11:22 But there's a lot of Python documentation that's written in restructured text.

11:27 So if you're going to contribute to an existing project, you can't say, well, I can't do any of the documentation because it's all in restructured text.

11:35 Convert it to Markdown.

11:36 Just use Markdown.

11:37 No, we're not rewriting all of this in Markdown.

11:39 Right.

11:40 So whether or not you like it, restructured text is part of the Python world.

11:45 So this is a wonderful, there's a wonderful article from Simon Will.

11:50 Oh, I always had his name wrong.

11:51 I always thought it was Wilson, but it's Simon Willison.

11:54 A subset of restructured text worth committing to memory.

11:58 And that's a noble goal.

12:00 But instead of committing it to memory, I'm just going to bookmark this article.

12:04 But he's got the necessary things for easy documentation.

12:08 You have paragraphs, headings, bulleted lists, how to do italics and things like that.

12:14 And then, of course, for documenting code, you need how to do inline code and code snippets and images and internal references and things like that.

12:24 And there's a lot covered here, but one of the issues that I always had was the tutorials on restructured text.

12:33 Restructured text can do a lot.

12:35 And it's kind of overwhelming.

12:36 And you don't need all of that to understand enough to help document a Python project.

12:42 So this is great.

12:43 Yeah, it's cool.

12:43 These are the five or six things you need.

12:45 Headings, links, bulleted lists, numerical lists, images, things like that, right?

12:50 Yeah, and then how to get your code in there.

12:52 Yeah, exactly.

12:53 It's great.

12:54 I'm going to make use of this one that you're sharing here because I'm working on a project where I have to write a lot of restructured text.

13:00 And, yeah, I basically have doubled my knowledge of restructured text by reading that.

13:04 Yeah.

13:06 Because like you, I do markdown most of the time.

13:09 So to be honest, most of the time when I needed to do something in restructured text, I would just find somebody else that did something like that and copy it and not know why I was doing these things.

13:20 So having a little primer and tutorial is good.

13:23 Yeah, it's great.

13:25 Speaking of finding things, let's talk about Datadog really quick.

13:29 So if you have issues, performance problems, bugs, things like that with your application, you should check out Datadog.

13:35 So they're sponsoring this episode.

13:43 And they have this new feature called Trace Search and Analytics.

13:51 around performance for things like show me all of the operations that this customer or this user did or all the requests that go to this URL and aggregate the performance of that, which is pretty awesome.

14:03 So start troubleshooting and optimizing your Python apps today.

14:06 The free trial and Datadog will send you a free cute little t-shirt.

14:10 So check them out at pythonbytes.fm/Datadog.

14:12 So I have a theme for the rest of the items I've chosen, Brian.

14:16 Okay.

14:17 So have you ever heard that some software has security problems people can break into it?

14:22 Yeah.

14:24 You know, even though Python's not compiled, Python code can have it too, right?

14:28 I assume so.

14:29 Yeah.

14:29 Well, our friend Anthony Shaw told us about this cool project, a little app you can run.

14:36 It's really a linter, basically.

14:38 You run against your program called Bandit.

14:40 Okay.

14:41 So Bandit is designed to find common security issues in Python code.

14:45 So what it does is you point it at your project and it'll load up each file into an abstract syntax tree, run a bunch of analysis over that AST, and then it'll generate a report.

14:59 And it'll tell you things like you're misusing the Telenet lib or you're using eval or you're doing try except pass and eating errors.

15:07 So opening a subprocess with popen with shell equals true.

15:11 I actually don't know what's wrong with that, but I should probably figure that out.

15:14 Yeah.

15:15 But apparently it's bad.

15:16 Shouldn't do it.

15:17 And that's error 602.

15:19 So it'll tell you, you know, for example, like I wouldn't know that that's necessarily bad.

15:23 I know I've heard it before, but I don't remember why it's bad.

15:26 So it's pretty cool.

15:28 And it just has a whole bunch of errors it'll find and, you know, security lapses it'll find and tell you about them.

15:34 So this is something that you, it's like a stack analysis.

15:36 It's not, it's not running your code.

15:38 It's looking at.

15:39 No, it's, it's static.

15:40 Yeah.

15:41 It's static analysis.

15:41 Oh, that's cool.

15:42 Yeah.

15:43 Isn't that nice?

15:43 Yeah.

15:43 Yeah.

15:44 It's like, I see you did a try except and you just ate that error and that can actually,

15:49 you know, cause other types of problems.

15:51 Like maybe you should have validated something, but instead you just keep going.

15:54 Things like that.

15:55 So this might be good even for non, I mean, it's definitely important for, for Python projects

15:59 that, that have a interface to the web or like a web application, but you could use this for

16:06 a lot of stuff.

16:07 Not just that.

16:08 Sure.

16:08 So.

16:09 Yeah.

16:09 Yeah.

16:09 It definitely lets you know when you're doing things wrong in like a different category,

16:13 say than probably flake eight.

16:14 Yeah.

16:15 Neat.

16:15 Yeah.

16:16 And because it's, you know, you could easily integrate this type of thing into a continuous

16:19 build integration pipeline.

16:20 Yeah.

16:21 I'll definitely have to check this out.

16:23 Cool.

16:24 Yeah.

16:24 This is, it's, it even probably has a little testing angle to it somehow.

16:27 Maybe.

16:27 Yeah.

16:28 Maybe.

16:29 Awesome.

16:29 All right.

16:30 What you got next for us?

16:31 Well, last week we talked about what Netflix was up to with Jupyter Notebooks.

16:35 So I was, and I mentioned that I've kind of been trying to learn more about them.

16:40 So one of the, we had somebody submit a, another, it's another repo, but it's, it's a whole

16:48 bunch of notebooks.

16:49 So it's a, what he's put together is learn Python three.

16:54 I think I forgot the wrong link here.

16:55 Learn Python three with a Jupyter Notebook.

16:58 And so I was actually playing with this morning and it was just a blast.

17:03 So we, we have a, it's got beginner and intermediate Python concepts and you just sort of like can

17:10 walk through it.

17:11 And one of the things I had to learn, of course, is to at first is just to how to, how to do

17:16 this.

17:16 How to clone the, I cloned the repo and just ran and then installed Jupyter and then ran,

17:23 ran the, just say, what is it?

17:26 You just say Jupyter Notebook or something and it opens up a web browser and then you can click

17:31 different notebooks and run them and, and see what they do.

17:35 And the thing that really intrigued me that the bone that he put in there for me to try

17:39 to grab is there's two topics on testing.

17:42 And I've been asked this before of like, how do you, can you run a pytest within a notebook?

17:50 And I didn't know if you could or not.

17:52 I, I heard rumors, but there's two examples in here of how to, two notebooks that have

17:58 tests embedded in them.

17:59 So you can, you can run your tests within, so you can define a, define a function or something

18:04 in a different cell and then write a test for it in another cell and then run it.

18:09 I think that's really cool.

18:10 And I had no idea how to run pytest either in there.

18:14 And I always sort of saw that as like one of the weaknesses of notebooks.

18:18 I have lots of respect for them, but one of the things I don't really like is it's hard

18:21 to do like, how do I run tests against this code, this code?

18:24 How do I like get a code coverage report of the code?

18:29 My Jupyter Notebook is doing things like that, right?

18:31 It seems much easier when you have a set of files that you can like call directly.

18:35 Yeah.

18:35 And I'm still not, so this is running pytest within the notebook and I'm not, I think there's

18:40 also a way that I haven't figured out yet to take a notebook and run, if there's tests

18:45 inside of it, run the run tests against it.

18:48 Yeah.

18:48 There may be in the stuff that Netflix talked about last week, allow you to basically take

18:54 a notebook and execute it as if it were a function with an input and an output.

18:57 And like that in some sense would open up a testing angle to itself, right?

19:01 Definitely.

19:02 And especially if you like save the output somewhere, that'd be great.

19:06 Anyway, it was just really fun.

19:08 It was a, it's fun to look around.

19:10 The other thing that I like in, as an example is there's the examples of how to do pip install

19:17 within a notebook.

19:17 It has those at the top because it doesn't assume that you've installed pytest to begin

19:22 with.

19:22 Oh yeah, that's cool.

19:23 And it shows that.

19:24 And then it's using another pip installable package called the ipytest, which is how it's

19:31 running pytest within the notebook.

19:33 So it's cool.

19:33 That is really cool.

19:34 Nice find.

19:36 All right.

19:36 Last one.

19:37 I said I was on a bit of a theme.

19:38 So we talked about Bandit and Bandit lets you find code that is being either using a bad

19:45 library or is using a bad pattern, like tracks that pass, things like that.

19:50 This next one takes a different angle on the same sort of problem around security and integrity

19:56 of your application.

19:57 And it's called detect secrets.

19:59 So their sort of tagline there is an enterprise friendly way of detecting and preventing secrets

20:06 in code.

20:07 And this is actually released from Yelp.

20:10 So on one hand, like using code badly, that's a problem, but also put hard coding passwords

20:15 and other things like that into your code can be a problem, right?

20:21 Yeah, definitely.

20:23 And so the enterprise friendly aspect, I think, comes from the fact that it says, all right,

20:29 look, we assume that your code is just messed up.

20:31 There's all sorts of badness in here.

20:33 What do you do if you find yourself in a hole and things keep getting worse?

20:37 You know, first thing you do is stop digging, right?

20:39 Before you get out of the hole, you just stop making it worse.

20:42 So there's a lot of mechanisms in detect secrets to say, let's not make this worse.

20:47 Let's prevent new secrets from going into the code base.

20:51 Now we can start to peel off, like start to like remove some of these things that are

20:55 hard coded in there.

20:56 But let's go and set a baseline and go, all right, if we get any new detections of secrets

21:02 being entered, that's going to fail the build.

21:03 But the way it is now, we're going to accept it.

21:05 And then we can like make pass after pass to actually take away some of those and reset

21:10 the baseline.

21:10 Okay, interesting.

21:12 Yeah, it's pretty.

21:12 Also, there's some things that look like they're secrets that should be detected, but you've

21:18 inspected the code and said, no, this one's fine.

21:20 Yeah, exactly.

21:21 And that baseline thing will let you say, I approve of these, but not anything else.

21:26 Yeah.

21:26 Cool.

21:27 Which is pretty cool.

21:27 And they also show you how to set it up as a git commit hook.

21:31 So as you know, from the testing space, one of the challenges is like, you can have awesome

21:36 tests, but if no one runs them or worse, one or two people on the team are very passionate

21:41 about them.

21:41 And a whole bunch of other people hate them and refuse to run them no matter how much you

21:46 talk about it.

21:47 The people who care about the test suffer from the neglect that the people who explicitly

21:54 ignore it, right?

21:54 Maybe that person enters some problem that'll fail a test, but they don't run their test,

21:58 so they don't know.

21:59 You get it.

22:00 You try to run your test and then you're hit with their problems as well, right?

22:03 So this git commit hook here basically means everyone on the team is going to be basically

22:09 running this as part of a check-in.

22:11 The whole idea around git commit hooks is interesting.

22:14 There's a lot of interesting things people are doing, like making sure that there's a plaque

22:19 has a git commit hook as part of it or attached to it so people can make sure their code is still

22:24 black before they check it in.

22:25 Yeah, that's the one I was thinking of as well.

22:27 And it can even automatically reformat your code to the group's format as part of the check-in,

22:33 right?

22:33 I'd be curious to know if there's any other...

22:35 I'd like to have people let us know if there's other git commit hooks that we haven't talked

22:40 about that are helping their teams out.

22:41 Maybe they'd help other people out too, so...

22:43 Yeah, it'd be fun to collect like a list of use cases of what people are doing.

22:47 Yeah.

22:47 Yeah, if you're doing something cool with a git commit hook, check out pythonbytes.fm

22:51 slash 93 and put a comment at the bottom.

22:54 That'll be in the show notes, you know, sort of below the show notes for everyone to come

22:58 see.

22:58 All right, well, that's all of our items for this week.

23:01 Brian, you got anything personal you want to share before we're out of here?

23:04 No, I guess pytest came out with a new version the other day, which kind of blew me away because

23:12 I had submitted a very minor bug, but I had submitted a bug to the project.

23:17 And I only submitted it like five days ago, and it's already now in a fixed and in a release.

23:22 So that's amazing.

23:24 That's a really good turnaround time.

23:25 That's awesome.

23:25 Yeah, that's fun.

23:26 Anything with you?

23:28 Got any news?

23:29 Nothing I'm ready to talk about.

23:30 I'm working on a couple of new projects, one that I'm doing restructured text for, a couple

23:35 of new courses are coming down the pipe, things like that, but nothing ready to talk about

23:40 just yet.

23:40 All right.

23:41 So how's that for a long-winded no?

23:43 Well, it was fun talking to you.

23:46 Yeah, you as well.

23:47 Thanks for everything, and see you later.

23:49 All right.

23:50 Bye.

23:50 Thank you for listening to Python Bytes.

23:52 Follow the show on Twitter via at Python Bytes.

23:54 That's Python Bytes as in B-Y-T-E-S.

23:57 And get the full show notes at pythonbytes.fm.

24:01 If you have a news item you want featured, just visit pythonbytes.fm and send it our way.

24:05 We're always on the lookout for sharing something cool.

24:08 On behalf of myself and Brian Okken, this is Michael Kennedy.

24:11 Thank you for listening and sharing this podcast with your friends and colleagues.