Transcript #93: Looking like there will be a PyBlazor!

00:00 Hello and welcome to Python bites where we deliver Python news and headlines directly to your buds. This is episode 93 recorded August 30th 2018 I'm Michael Kennedy. I'm Brian. I can and Brian, you know last time we had some awesome drop the mic stuff I think we're gonna do it again, aren't we? Yeah, I think it's some some really interesting things today. Absolutely So before we get on to them, though I just want to say thanks to data dog data dog sponsoring the show check them out at Python bytes FM slash data dog I'll tell you more about that later.

00:31 Brian, a lot of people get into programming, kind of in air quotes there, by doing scripting around like the shell and stuff, right?

00:39 I think so.

00:40 I think that's the way some people like in the IT world are.

00:42 Like Bash or even bat files?

00:45 Or lots of DevOps roles.

00:48 Yeah, exactly.

00:49 At the point where I learned Python, I was doing a lot of C++ development and writing little utility scripts in bash and Perl and stuff like that.

00:59 And so one of the things that I did, you know, I want to try to learn Python by writing as much Python as I can.

01:07 And so I started doing more and more, more and more of my utility scripts in Python.

01:14 And so I think that was a good way to just try to jump in and solve, solve little problems.

01:19 I could have written it faster in bash because I knew that or in Perl, but learning how to to do things in Python is good.

01:26 So there's a, it's a GitHub repo that's called Replacing Bash Scripting with Python.

01:32 And I think it goes through a lot of stuff, a lot of the issues that you'd need to do, that you'd normally write in the little utilities, and how to do that in Python.

01:42 So this is fun.

01:44 Talks about dealing with files, reading and writing files, dealing with standard in and standard out, standard error, things like that, and writing the command line utilities.

01:54 - Yeah, things like dealing with exit codes, so if you shell out to some other process, you run a command knowing whether it failed or it worked, right?

02:01 Things like that?

02:01 - Yeah, and how to pipe between commands to, if you're gonna pull things together.

02:07 And then, it doesn't jump into a lot of the really cool stuff with Python, but it's a good start.

02:12 But it does, like, introduce people to the path library and sh, I don't know how you pronounce that, shoot-ill, S-H-U-T-ILL.

02:20 And those are things.

02:21 And then a lot of times you'll maybe using sed and awk and grep and things like that and how to go about replacing that sort of stuff with regular expressions with regex.

02:34 And so this is just fun.

02:35 I like it to be able to, if you're writing a little utility and it's up to you as to what language you write it in, this might help you write it in Python.

02:43 It's cool.

02:44 And I think one of the real key things to actually making programming stick when you're trying to learn it or even learning a new programming language, you know, some other kind of programming is to have what you're doing be relevant to you, right?

02:57 And so if you work in a DevOps world and you're like, "Well, I'm going to go learn Python, so I'm going to do this data science project," that may be totally irrelevant to you.

03:06 So having this as a resource for people who generally do scripting and automation type stuff, this is really cool.

03:11 Yeah, and then also you're going to, the stuff you, when you're learning is stuff you start outriding isn't, just to be frank, and it's the same with everybody, it's not that great to start with.

03:22 So starting small is good, so you don't have something huge to rewrite later.

03:26 True.

03:27 I guess a lot of people, I know I did one in the earlier days, went through the experience of going back and reading code I wrote six months ago and going, "Wow, that's bad." But then that same person who looked back badly on the old code, six months farther down the line, you look back and you're like, "Yep, that's bad too, just in a different way," right?

03:49 But eventually, you get there.

03:50 But yeah, having the small stuff definitely helps.

03:53 And I've heard a lot of people that, like in a DevOps world, where they just, you know, you start writing like one or two of your scripts in Python, and then you can just add to it.

04:03 As you learn more stuff, you can do more powerful things.

04:06 And having, like you said, having it be relevant to your job is important for what you're doing, for learning.

04:13 Yep.

04:14 I'm glad you brought it up and I'm sure to help a ton of people. Now this next topic I'm pretty excited about. I'm honestly kind of beside myself about this one. This literally could be a game-changer. This is called Pi Iodide and I don't know if you've heard of it before. I had not heard of it until just beginning of this week I think actually. So Iodide is a project by Mozilla and it's sort of think of iodide is like a competitor to Jupyter notebooks where the concept of Jupyter notebooks is you have this web page and it's like a view into a Jupiter server and then when you execute code that code executes on the server which is a little sketchy because if you're letting people run arbitrary Python code on your server that's not amazing so maybe you're using like some sort of Docker thing to like spin them up and shut them down it gets complicated right? Yeah. Well Mozilla wants to make this environment available to everybody but in a distributed fashion. So the goal is to bring this environment to the browser and when you execute code it executes on your browser client side so you can do whatever you want. Well that's cool but the problem is a lot of people want to do data science in Python and Python Python, while it's had some JavaScript options for running code in the browser, they've always been like, well, you can kind of do this.

05:39 And it's mostly like Python and so on.

05:42 A while ago, as in two weeks ago, we asked the question of will there be a pie blazer?

05:48 Remember blazer is the dotnet communities attempt to bring the dotnet CLR, their equivalent of CPython into the browser with WebAssembly and make it run there.

05:59 And they're making that actually work, which was kind of blowing my mind and giving me a lot of hope that maybe, maybe somebody could do it for CPython, right?

06:06 And what do you think, do you think it'll happen?

06:07 - Well, it's a nice dream.

06:09 - I agree, it's a nice dream.

06:10 But sometimes dreams become reality, right?

06:14 So PyIodide is an implementation, a running version of all of CPython, an exact bit of CPython, same thing, in the browser, in WebAssembly.

06:28 - That's so cool.

06:30 - Is that incredible or what?

06:31 I mean, my mind is blown.

06:34 So I reached out to one of the contributors, Michael D, and asked him, like, this is like really CPython running in WebAssembly in the browser.

06:45 And he said, yeah, yeah.

06:46 So Michael works for Mozilla, this is a Mozilla project, and they're trying to bring all of CPython, at least the data science part of it, into the browser.

06:57 Now, there's a few things that it won't do, like you can't open a socket.

07:00 Obviously, TK enter doesn't work, 'cause what does that mean in the browser, right?

07:03 There's still, there's some parts of CPython that are not there, but if it's permitted in the browser, it's identical to CPython, and that, I think, is amazing.

07:13 - That is amazing, I can't wait to play with this.

07:15 - I can't either, and so they also wanna have the data science tooling there, and so things like NumPy, Matplotlib, and so on, they have heavy dependencies upon C, So you can't just bring it over and run it in this WebAssembly version of CPython.

07:30 So you have to take the C code that supports things like NumPy and Matplotlib and get them to compile to WebAssembly themselves.

07:39 So they've also done that.

07:40 So you have native NumPy, native Matplotlib on native CPython client-side in the browser with WebAssembly.

07:48 And it's fast.

07:49 I'm still trying to get my head around this.

07:51 How could it be fast?

07:52 I know.

07:53 I mean, it's like binary stuff executing, and it's also not that big, right?

07:57 The download, I noticed using the web tools, when you go and interact with this page, it downloads like CPython.Wasm, right?

08:06 The extension for WebAssembly, and it's 2.5 megs, right?

08:09 And on my connection, that's like one and a half seconds to get that thing up and running.

08:14 And then it doesn't have to parse it like it does like two megs of JavaScript.

08:17 You don't make sense of that, right?

08:19 Effectively compile it, it just takes it and runs it because it's already compiled.

08:22 Okay, that's okay. So it's not nothing, but that's pretty not nothing, but it's it's not it's not that bad. And you know, you could cash.

08:30 Somebody could set up a CDN for that thing and cash it like say Mozilla. Yeah, like say Mozilla.

08:37 So I think it's awesome. I want to I want to just say thank you Mozilla and Michael and team for working on this because this this is amazing. And this could really, really change.

08:48 it could really expand where Python is broadly applicable. Now I think it's worth pointing out that this generally is built with the concept of making data science work in the browser, right? Whereas this pie blazer thing, this blazer, my imaginary pie blazer thing that I made up, that was around building single page apps, more of the website of programming than the in the data science programming, site programming.

09:15 But this thing does have extensions to let you work with HTML and the DOM and events and stuff like that.

09:23 So you could, it's not a long shot to go from this to having an interactive, rich application for a web app in Python.

09:33 - Yeah, so it's still not quite where we want, but like really cool.

09:37 - It's really cool.

09:38 And I mean, it's totally working for what they're trying to do, But I'm just thinking like if I want to say replace like a React thing or an AngularJS thing with Python, there might be a few more steps to get like that angle of stuff working.

09:56 But it shouldn't be that far because we already have the CPython runtime working, which is really what you need and the integration with HTML.

10:03 Yeah, so this is cool.

10:05 I want a lot of people to grab this and run with it.

10:07 Yeah, I think this really opens up the possibility for some amazing stuff.

10:11 So once again, thank you, Mozilla.

10:13 And I also I just want to reflect a little bit, I think it's really ironic or interesting coincidence at PyCon 2018.

10:22 As we mentioned, when we talked about the Blazor thing, I think it was Tim Callahan, I'm still forgetting the name, but somebody from Mozilla came and gave an impassioned presentation as the opening keynote of, please, let's find a way to use WebAssembly to get Python more capable in the web space and specifically in the browser and client side and I think it's just pretty awesome that Mozilla, another team in Mozilla, actually is the one that implemented this. So that's really amazing. So these guys working on PyIodide, they may want to document their stuff, right? Yeah, you're so much better at the transition thing than I am. I'm I'm like, oh, next topic.

11:06 I personally like Markdown.

11:08 I write, even when I have to use, to be honest, when I have to use Microsoft Word, I write in Markdown and then I convert it to Word.

11:16 Anyway, I don't know why I was talking about Word.

11:19 So restructured text is not something that comes naturally to me, but there's a lot of Python documentation that's written in restructured text.

11:27 So if you're gonna contribute to an existing project, you can't say, well, I can't do any of the documentation 'cause it's all in restructured text.

11:35 Convert it to Markdown.

11:36 - Just use Markdown.

11:37 No, we're not rewriting all of this in Markdown.

11:39 - Right, so whether or not you like it, restructured text is part of the Python world.

11:45 So this is a wonderful, there's a wonderful article from Simon Will, oh, I always had his name wrong.

11:52 I always thought it was Wilson, but it's Simon Willison.

11:55 A subset of restructured text worth committing to memory.

11:58 And that's a noble goal, but instead of committing it to memory, I'm just going to bookmark this article.

12:04 But he's got the necessary things for easy documentation.

12:08 You have paragraphs, headings, bulleted lists, how to do italics and things like that.

12:14 And then, of course, for documenting code, you need how to do inline code and code snippets and images and internal references and things like that.

12:24 And there's a lot covered here, but one of the issues that I always had was like the tutorials on restructured text.

12:34 Restructured text can do a lot, and it's kind of overwhelming, and you don't need all of that to understand enough to help document a Python project.

12:42 So this is great.

12:43 Yeah, it's cool.

12:44 These are the five or six things you need.

12:46 Headings, links, bulleted list, numerical list, images, things like that, right?

12:50 Yeah, and then how to get your code in there.

12:52 Yeah, exactly.

12:53 It's great.

12:54 I'm gonna make use of this the one that you're sharing here because I'm working on a project where I have to write a lot of restructured text and Yeah, I basically have doubled my knowledge of restructured text by reading that Yeah, and I mean cuz like you I do markdown most the time so most to be honest most of the time when I needed To do something in restructured text I would just find somebody else that did something like that and copy it and not know why I was doing these things So having a little little primer and tutorial it's good Yeah, it's great. Speaking of finding things, let's talk about Datadog really quick. So if you have issues, performance problems, bugs, things like that with your application, you should check out Datadog. So they're sponsoring this episode. And it's a monitoring platform that brings together metrics, logs, request traces, all in one place. And they have this new feature called trace search and analytics. And it lets you break down the usage of your Python app around performance for things like show me all of the operations that this customer or this user did or all the requests that go to this URL and aggregate the performance of that which is pretty awesome. So start troubleshooting optimizing your Python apps today with a free trial and Datadog will send you a free cute little t-shirt so check them out at pythonbytes.fm/datadog. So I have a theme for the rest of this rest of the items I've chosen Brian. Okay. So have you ever heard that some software has like security problems people can break into it? Yeah. You know even though Python's not compiled, the Python code can have it too right? I assume so yeah. Well our friend Anthony Shaw told us about this cool projects little app you can run this so they're really a linter basically you run against your program called Bandit. Okay. So Bandit is designed to find common security issues in Python on code. So what it does is you point it at your project and it'll load up each file into an abstract syntax tree, run a bunch of analysis over that AST, and then it'll generate a report. And it'll tell you things like you're misusing the telnet lib, or you're using eval, or you're doing try except pass and eating errors. Opening a sub process with P open with shell equals true. I actually don't know what's wrong with that, but I should probably figure that out. Yeah. But apparently it's bad. Shouldn't do it. And that's error 602. So it'll tell you, you know, for example, like I wouldn't know that that's necessarily bad. I know I've heard it before, but I don't remember why it's bad. So it's pretty cool. And it just has a whole bunch of errors it'll find and, you know, security lapses it'll find and tell you about them. So this is something that you, it's like a static analysis. It's not, it's not running your code. It's looking at... No, it's, it's - Static, yeah, it's static analysis.

15:42 - Oh, that's cool.

15:43 - Yeah, isn't that nice?

15:44 - Yeah, neat.

15:44 - Yeah, it's like, I see you did a try, accept, and you just ate that error, and that can actually cause other types of problems, like maybe you should have validated something, but instead you just keep going, things like that.

15:55 - So this might be good even for non, I mean, it's definitely important for Python projects that have an interface to the web or a web application, but you could use this for a lot of stuff, not just that.

16:08 - Sure.

16:09 - Yeah, yeah, it definitely lets you know when you're doing things wrong in like a different category, say, than probably Flake 8.

16:14 - Yeah, neat.

16:15 - Yeah, and because it's, you know, you can easily integrate this type of thing into a continuous build integration pipeline.

16:21 - Yeah, I'll definitely have to check this out.

16:23 Cool.

16:24 - Yeah, this is, it even probably has a little testing angle to it somehow.

16:27 - Maybe, yeah.

16:28 - Maybe, awesome.

16:30 All right, what you got next for us?

16:31 - Well, last week we talked about what Netflix was up to with Jupyter Notebooks.

16:35 So I was, and I mentioned that I've kind of been trying to learn more about them.

16:41 So one of the, we had somebody submit another, it's another repo, but it's a whole bunch of notebooks.

16:49 So it's a, what he's put together is learn Python three, I think I've got the wrong link here, learn Python three with a Jupyter notebook.

16:59 And so I was actually playing with this this morning and it was just a blast.

17:03 So we have, it's got beginner and intermediate Python concepts.

17:08 And you just sort of like can walk through it.

17:11 And one of the things I had to learn, of course, is to, at first, is just to how to do this.

17:17 How do you clone the, I cloned the repo and just ran, and then installed Jupyter and then ran the, just say, what is it?

17:26 You just say Jupyter Notebook or something, and it opens up a web browser.

17:30 And then you can click different notebooks and run them and see what they do.

17:35 And the thing that really intrigued me, that the bone that he put in there for me to try to grab is, there's two topics on testing.

17:43 I've been asked this before of like, can you run pytest within a notebook?

17:50 I didn't know if you could or not, I heard rumors, but there's two examples in here of how to notebooks that have tests embedded in them.

17:59 So you can define a function or something in a different cell and then write a test for it in another cell.

18:09 I think that's really cool and I had no idea how to run pytest either in there and I always sort of saw that as like one of the weaknesses of notebooks.

18:18 I have lots of respect for them but one of the things I don't really like is it's hard to do like how do I run tests against this code, how do I get a code coverage report of the code my Jupyter notebook is doing, things like that.

18:31 It seemed much easier when you have a set of files that you can like call directly. Yeah And I'm still not so this is running pytest within the notebook and I'm not I think there's also a way I haven't figured out yet to take a notebook and run if there's tests inside of it run the run tests against it Yeah, there may be in the stuff that Netflix talked about last week allow you to basically Take a notebook and execute it as if it were a function with an input and an output and like that in some sense would open up a testing angle by itself, right?

19:02 Definitely, and especially if you save the output somewhere, that'd be great.

19:07 Anyway, it was just really fun. It's fun to look around.

19:10 The other thing that I like as an example is there's the examples of how to do pip install within a notebook.

19:18 It has those at the top because it doesn't assume that you've installed pytest to begin with.

19:22 Oh yeah, that's cool.

19:23 And it shows that.

19:24 And then it's using another pip installable package called the Ipytest, which is how it's running pytest within the notebook.

19:33 So it's cool.

19:34 That is really cool.

19:35 Nice find.

19:36 All right, last one.

19:37 I said I was on a bit of a theme.

19:38 So we talked about bandit.

19:40 And bandit lets you find code that is being either using a bad library or is using a bad pattern like tracks a pass, things like that.

19:50 This next one takes a different angle on the same sort of problem around security and integrity of your application.

19:57 And it's called detect secrets.

20:00 So their sort of tagline there is an enterprise friendly way of detecting and preventing secrets in code.

20:07 And this is actually released from Yelp.

20:10 So on one hand, like using code badly, that's a problem, but also put hard coding passwords And other things like that into your code can be a problem, right?

20:21 Yeah, definitely.

20:23 And so the enterprise friendly aspect, I think, comes from the fact that it says, all right, look, we assume that your code is just messed up.

20:31 There's all sorts of badness in here.

20:34 What do you do if you're you find yourself in a hole, and things keep getting worse, you know, first thing you do is stop digging, right before you get out of the hole, you just stop making it worse.

20:42 So there's a lot of mechanisms in detect secrets to say, let's not make this worse.

20:48 Let's prevent new secrets from going into the code base.

20:51 Now we can start to peel off like start to like remove some of these things that are hard coded in there.

20:57 But let's go and set a baseline and go alright, if we get any new detections of secrets being entered, that's going to fail the build.

21:04 But the way it is now we're going to accept it.

21:06 And then we can like make pass after pass to actually take away some of those and reset the baseline.

21:11 Okay, interesting.

21:12 Yeah, it's pretty also pretty much some things that look like they're secrets that should be detected, but you've inspected the code and said no, this one's fine. Yeah, exactly. And that baseline thing will let you say I approve of these, but not anything else. Yeah, cool, which is pretty cool. And they also show you how to set it up as a git commit hook.

21:31 So as you know from the testing space, one of the challenges is like you can have awesome tests but if no one runs them or worse, one or two people on the team are very passionate about them and a whole bunch of other people hate them and refuse to run them no matter how much you talk about it. The people who care about the tests suffer from the neglect that the people who explicitly ignore it, right? Maybe that person enters some problem that'll fail a test but they don't run their tests so they don't know. You get it, you try to run your test and then you're hit with their problems as well, right?

22:02 So this Git commit hook here basically means everyone on the team is going to be basically running this as part of a check-in.

22:11 The whole idea around Git commit hooks is interesting.

22:14 There's a lot of interesting things people are doing, like making sure that, like there's a plaque has a Git commit hook as part of it or attached to it.

22:23 So people can make sure their code is still black before they check it in.

22:25 Yeah, that's the one I was thinking of as well.

22:27 And it can even like automatically reformat your code to the groups format as part of the check in, right?

22:33 I'd be curious to know if there's any other, I'd like to have people let us know if there's other Git commit hooks that we haven't talked about that are helping their teams out.

22:42 Maybe they help other people out too.

22:43 So yeah, it'd be fun to collect like a list of use cases of what people are doing.

22:47 Yeah, yeah.

22:48 If you're doing some cool to get commit hook, check out Python bytes, FM slash 93 and put a comment at the bottom.

22:54 that'll be in the show notes, sort of below the show notes for everyone to come see.

22:59 All right, well, that's all of our items for this week.

23:01 Brian, you got anything personal you wanna share before we're out of here?

23:04 >> No, I guess pytest came out with a new version the other day, which was kind of blew me away because I had submitted a very minor bug, but I had submitted a bug to the project.

23:18 And I only submitted it like five days ago, and it's already now in a fixed and in a release.

23:22 So that's amazing.

23:24 - That's a really good turnaround time, that's awesome.

23:26 - Yeah, that's fun.

23:27 Anything with you, got any news?

23:29 - Nothing I'm ready to talk about.

23:30 I'm working on a couple of new projects, one that I'm doing restructured text for, couple of new courses are coming down the pipe, things like that, but nothing ready to talk about just yet.

23:41 - All right.

23:42 - So how's that for a long-winded no?

23:43 (laughing)

23:45 - Well, it was fun talking to you.

23:47 - Yeah, you as well, and thanks for everything, and see you later.

23:49 - All right, bye.

23:50 - Thank you for listening to Python Bytes.

23:52 Follow the show on Twitter via @pythonbytes, that's Python Bytes as in B-Y-T-E-S.

23:58 And get the full show notes at PythonBytes.fm.

24:01 If you have a news item you want featured, just visit PythonBytes.fm and send it our way.

24:06 We're always on the lookout for sharing something cool.

24:08 On behalf of myself and Brian Auchin, this is Michael Kennedy.

24:12 Thank you for listening and sharing this podcast with your friends and colleagues.