Brought to you by Michael and Brian - take a Talk Python course or get Brian's pytest book


« Return to show page

Transcript for Episode #151:
Certified! It works on my machine

Recorded on Friday, Oct 4, 2019.

00:00 KENNEDY: Hello, and welcome to Python Bytes, where we deliver python news and headlines directly to your earbuds. This is Episode 151, recorded October 2, 2019. I'm Michael Kennedy.

00:10 OKKEN: And I'm Brian Okken.

00:11 KENNEDY: And this episode is brought to you by DigitalOcean. Check them out at pythonbytes.fm/digitalocean, get $50 credit for new users. More on that later. Brian, how you doing?

00:19 OKKEN: I am really good.

00:20 KENNEDY: Super, we are on PDX Python Meetup West, PDX West Python Meetup, whatever you're calling it's, eve, aren't we?

00:29 OKKEN: Yeah, it's just right around the corner. Although, it'll probably already have happened by the time this comes out.

00:36 KENNEDY: Yeah, it may, it's going to be tight. So, well I'm sure it went wonderfully. So, got all those speakers lined up and everything. I'm really excited to have that as part of our community around here, who knows, maybe we can put it online somehow, but no plans for that it sounds like. You know, I hear a lot about containers and, when I first heard of Docker, and things like that, I thought Docker was, I thought they had created containers, like they had come up with this idea of this other kind of isolation, relative to, say, a virtual machine and, looking in to it, it turns out that's not the only option and they actually didn't even come up with it, they just bundled it up and packaged it nicely.

01:14 OKKEN: I think what Docker has done is very cool, but I have yet to find a use case where I needed it, so far, and there are other options, and one of our friend of the show, Matt Layman, has written a article called "Python Alternative to Docker". The article uses a tool called Shiv from LinkedIn, which is a hilarious name. We actually did mention it briefly on Episode 114, but I didn't know much about it and so this is kind of a neat article. It talks about that Shiv uses zipapp, and I think we've talked about that before, that you can, and then I think there's a few tools that utilize that, that...

01:52 KENNEDY: Right, things like Pex and stuff, yeah.

01:54 OKKEN: Yeah. You can zip up a Python application with all its dependencies and I don't think that, at that point, it includes Python, does it?

02:03 KENNEDY: I don't think so, and that was actually super disappointing to me, that it didn't just deliver a thing that ran, you still have to have Python, but it's like, you don't have to install the app, long as you have Python, you can run it, was my understanding.

02:15 OKKEN: This article is talking about, and this totally makes sense, you've got a continuous integration build pipeline where you're making changes, and maybe you're updating dependencies, but, on the deployment machine, you're not deploying, you don't have to update Python all the time, you're just updating the thing that you package. And that's kind of a neat thing about Shiv, is all your dependencies, maybe you're updating dependencies or maybe you're updating your own application or fixing bugs or whatever, that can all be put together and tested as one deliverable unit, and then just that can be updated to a server machine. But then, all the steps of, okay, that sounds, in concept, that sounds great, but what are all the steps I have to go through to actually get that done? And Matt has done that, he just basically walked through all the steps to try to get that done, and I thought it was really great. He does talk about packaging a little bit, because you kind of have to talk about packaging when you're talking about deployment, so this is kind of fun. He does do a shout out to, if you didn't use Shiv or Docker, what would you do? What are all the little steps you'd have to do? Which just makes it sound like something new to either hire somebody else to do.

03:27 KENNEDY: Like, that's not, I'm not doin' that.

03:29 OKKEN: Yeah, but he also does shout out to platform as a service, things like Heroku and PythonAnywhere. And I think that's cool that, there's other options, if you want to just throw money at it, you just use one of these. And these actually aren't that expensive if you're trying to just save time, that's a fine trade-off but, if you want to do this, this is a neat idea. Also, Docker's a big hammer. I mean, a lot of people love it and that's great, but bringing up with a lot of people that you want to try to deploy with Docker and, if it's a thing like this, like you're just deploying an application, that might be overkill.

04:04 KENNEDY: It seems like it's a middle ground, like I'm packaging up, not just my Python code, but my Python dependencies and a lot of the setup around that. You still have to have Python there but, beyond that, it's not really much to it. If you need to depend on other things, like, 'I need Redis running on this port and I need Postgres like that, and then I also want this other thing in Linux doing something,' you're like, all the sudden, then you've outgrown bundling Python more cleverly, but if you don't need that it sounds like a pretty cool way to do it.

04:34 OKKEN: Yeah, like in this example, the example Matt gives is a Django application that uses Gunicorn or Gunicorn, and so that, all those dependencies are bundled together. It's not a special kind of Django app with a huge back-end that's necessary, but, although, I mean Django uses a database, so I'm not sure.

04:53 KENNEDY: Yeah, yeah, interesting. While you're talking about Docker, I'm a fan of Docker in general but, kind of like you, I don't do a ton with it day to day, I just, the stuff I do is just not that complicated. But, I recently read an article in ZDNet, that site actually still exists, apparently, and... It actually says, this is surprising to me, sort of, kind of like I was surprised when GitHub got acquired, I realized afterwards, 'Oh, they're actually financially in deep trouble and maybe this is good rather than just going away.' The article says Docker is indeed in trouble, a leaked internal memo, the Docker CEO reveals that the troubled company is looking for more money, and they talk about pushing through with perseverance despite lack of clarity on the future and stuff like that, so, anyway just something to think about.

05:40 OKKEN: Definitely, and I read that, and I forgot that Docker was a company.

05:43 KENNEDY: Yeah, exactly. And I don't even know if it matters if Docker as a company goes away, I mean there are parts of Docker that are important like the registry and stuff like that, but you could totally use Docker if you just had the tools, but, still, it's... Always interesting to see these, how this is playing out, and something you want to consider if you're going to bet your project on it, right?

06:04 OKKEN: Yeah, definitely.

06:04 KENNEDY: So, I have a bit of a theme for this week as well, and it has to do with helping a scientist do Python stuff a little bit better. So this first thing I want to talk about is an article that's kind of a how-to from this woman named Anna Nowogrodzki, I'm going to go with that, sorry if I messed up your name, Anna. And it comes to us by a recommendation from a listener named Jason Thomas, thanks Jason. And the title is "How To Support Open-Source Software and Stay Sane".

06:34 OKKEN: Nice.

06:35 KENNEDY: And this is referring to scientists who develop libraries like NumPy, or some form of scikit-learn, or something like that, and then they release it to the world and then, all the sudden, they've got all this software development work and have a hundred GitHub issues they got to review and they're like, "I was just trying to give back a little bit, what just happened?" So, the article has some interesting quotes, so I'm going to do a little bit of quoting from it because I think it's pretty revealing. Anyway, it says, "The premise to the article is that releasing lab-built open-source software involves mountains of unforeseen work for the developers." You think you're done when you release it and it kind of starts. But it opens up saying, "On the 10th of April, astrophysicists announced that they had captured the first ever image of a black hole. This was exhilarating news, but none of the giddy headlines, other than us, mention that the image would not have been possible without open-source software." And the image was created by Matplotlib, a Python library for graphing data as well as other components of the open-source Python ecosystem. Here's where it gets interesting, just five days later, the US National Science Foundation, the NSF, rejected a grant proposal to support that ecosystem, saying that the software lacked sufficient impact.

07:50 OKKEN: What even?

07:52 KENNEDY: What even, right? So, like I said, it's kind of a how-to, it talks about where you're, as a scientist, if you're creating software like Matplotlib or something, where do you go to get funding? Where have people been turned down? What has worked? And so on. It also talks a lot about using some software development, software engineering techniques for having successful open-source projects that grow without killing you. It's pretty good, it talks about the challenges of not knowing, as a scientist, how you might not have fluency with git, you might not have a continuous integration. Unit testing might not be a thing that you are super skilled in, and it mentions places like the eScience Institute at the University of Washington, the Software Carpentry workshops, places like that where folks can go get that training. And then, also talks a little bit about when do you keep your software going, and when do you decide it's absolutely, we're not going to work at it anymore, what else are we going to go build? So if you're a scientist out there, or you work with scientists and you want to help them get rolling with Python and releasing their projects in a way that they'll stay sane, check that out, I think it's pretty good.

09:00 OKKEN: It's a continuous problem.

09:01 KENNEDY: Yeah, it is, and some of the recommendations are things like: make sure you have continuous integration, make sure you have version control, make sure you have testing, and one that I really liked was that you should focus on code readability over peak performance.

09:14 OKKEN: Oh, definitely, always.

09:15 KENNEDY: Right, yeah, 'cause you're going to have to fix bugs and, I had a friend who, soft friend, who had this saying that was really interesting that, "Debugging software is harder than writing software", generally speaking. So, if you write software at the edge of the complexity that you can handle, you probably can't debug it. So you need to step it back a notch, take it a little bit simpler and then when it goes wrong you could still deal with it. I like that saying. So, before we get to the next one, though, let me tell you about DigitalOcean, so, DigitalOcean, all of our infrastructure runs there when you get your MP3s that you hear our voices on, those came through a whole bunch of stuff that's happening over at DigitalOcean, and they have two new services that I've been telling you all about for a little while, but, want to keep telling you about it because they're good. If you want to use MySQL or Redis they now have those as a service, they're just go flip a switch and turn 'em on, that's in addition to Postgres as a service as well. So, definitely comin' along. Speaking of the softwares as a service, over at the DigitalOcean side I'm still a fan of fire me up a VM and install some stuff and let it roar, but, that's all good. Check them out at pythonbytes.fm/digitalocean, get $50 credit for a new user and they're doing good work, so we can recommend them.

10:28 OKKEN: Nice.

10:29 KENNEDY: So, you found another license in addition to MIT or GNU?

10:33 OKKEN: Yes.

10:33 KENNEDY: GPL?

10:34 OKKEN: I don't know what to make of this, so I wanted to bring it up just to hear your thoughts as well. So, there's a license, and I can't remember where I ran across it, but I think probably Twitter or something, I spend way too much time on Twitter. It's a license called the Hippocratic License, like that doctors have the Hippocratic Oath.

10:51 KENNEDY: Right, right, first, promise to do no harm, things like that?

10:54 OKKEN: Yeah, I mean I know it's way more than that, but that's all average people remember, like me. So this is a interesting, it derives from the MIT License and it adds a restriction, and the restriction reads, "The software may not be used by individuals, corporations, governments or other groups for systems or activities that actively, or knowingly, endanger, harm, or otherwise threaten the physical, mental, economic or general well-being of individuals or groups in violation to the United Nation Universal Declaration of Human Rights." Interesting. That's a interesting thing, and I like, I kind of like it, my first reaction is, wow, that's great. You can say, "I'm going to let everybody use this software, but I don't want it to be used to harm people." Completely fair, but I don't know if anybody, the people that are intentionally harming people, they don't care. Do you really think people that are in this are really going to go, "Oh, yeah we really can't use this software to guide our..." Does this cover military? I don't know.

11:59 KENNEDY: Also a lot of this has to do with human well-being. That's up for debate, and like a perspective, well, we had to start the war to help people or, obviously if you're on the other side of that, you're not helping us. This is not helping. Thank you, for not, for blowing us up. It's really hard to interpret, and I think you're right that it is very challenging because the folks who are most likely to abuse it, it's like the hackers are like, "Yeah, well I was going to steal that $10 million from the bank, but its got the wrong license." It does still send a message that the person who works on the project cares, and it's not necessarily a negative even if it's not super enforceable. I'm sure that things about the MIT License, people break. I don't even know what you've got to do to break it, but you probably could.

12:44 OKKEN: I actually think it's an interesting thing to have around. I could definitely see people coming up with different sorts of alternatives to the MIT License, also, not necessarily maybe this plus other stuff. I know people that intentionally would never work for the military, or work for a government or a military contractor. Because of, they don't want to. And I think it'd be completely legitimate for somebody to say, to have a license that said, my open-source software doesn't, I don't want that to work for these people either, so. But I haven't seen that.

13:19 KENNEDY: Not putting any judgment on those things like working for Boeing or whatever. If you wanted a license like that, I'd think you would have to be more explicit.

13:29 OKKEN: It's a slippery slope, with the things that I kind of agree with, I'm like, 'Oh, yeah, well that's completely reasonable.'

13:34 KENNEDY: Yes, I know.

13:35 OKKEN: But if people released software that said this is okay as long as you don't live in Oregon, because I don't like Oregonians. That would be lame, but, people can do that.

13:44 KENNEDY: It would definitely be lame. That's why you set up a server in California and you just remote in.

13:52 OKKEN: I'm vacationing permanently in Oregon.

13:55 KENNEDY: Exactly, exactly. It's an interesting find. Yeah, so, the next one like I said, I'm on a bit of a theme here and this next one is also about helping scientists do more, but more computational people. So this one is a real Python how-to on why and how to make the switch from MATLAB to Python.

14:15 OKKEN: Oh, this is great.

14:16 KENNEDY: Yeah, it's really, really detailed. Like standard, real Python stuff. I'm not entirely sure how many pages it is, but it's quite a few. So, it's a good, thorough, it's almost like a mini book on this, as Dan and his crew are used to doing. So it basically compares the philosophy of MATLAB to Python, closed-source to open-source, things like that. Then it walks people through how to set up Python and then just, you use these computational libraries in MATLAB, here's your Python ones. You did these operations like graphing, here's how you do it in Python. And so, if you're out there working on a team that uses MATLAB, and I know there are a ton of them out there, I've worked with some even around here, a great team who was moving from MATLAB to Python, who I worked with for a few days, like helping to speed that transition and things like that. There's a ton of these science-like groups, data science, not quite, maybe, data science, but on their way toward that world, who I think would really benefit from that. I, long ago, even used MATLAB for a little bit of work at this research lab that spun off in to a company, and did wavelet decomposition of real-time eye measurements. 250 hertz per, you know, 250 samples a second, wavelet decomposition is a way harder Fourier transform. But it's funky stuff. Anyway, to do that, it was fine but the little toolbox, I think its called, the wavelet toolbox that you got, was $2,000 per user. That's a lot of money when you can pip install something that also does that.

15:57 OKKEN: So there's MATLAB used a lot in our company as well, or there used to be. I don't know what it is now. But similar sorts of things, there's engineering. I'm more familiar with communication algorithms using FFTs and other communication transforms being mocked up in MATLAB before they go to ASICs or something. Yeah, this is very common and I'm really glad to see more and more people moving to Python from MATLAB.

16:25 KENNEDY: Yeah, absolutely, I think it's just another one of those slices of the technology ecosystem where tons of people are leaving that space and coming over to Python because the great computational support that's developing. Things like JupyterLab and all the NumPy, SciPy type of things. Also, in this article, I learned that there apparently is this thing called GNU Octave, which is a free and open-source clone of MATLAB that looks pretty good, actually.

16:48 OKKEN: Oh, interesting.

16:48 KENNEDY: Yeah, not that we want to encourage people to just go do that, they should come do Python, but this is still pretty interesting, I didn't know that exists.

16:55 OKKEN: Yeah, definitely.

16:56 KENNEDY: Have we spoken about GUIs? I feel like we should actually cover that sometime.

17:00 OKKEN: Yeah, so I was almost felt guilty not covering this, but it came up in Episode 143 already, but Nicholas Tollervey has got a project called PyperCard. I mean, I remember HyperCard. I never really used it, back in the day, the dark ages. But PyperCard is kind of this neat little thing and, did you use HyperCard ever?

17:22 KENNEDY: No, I have some friends who did, but I've never used it.

17:24 OKKEN: It was a Mac only thing, and it was this thing where you kind of have a whole, it's designed as if there's a whole bunch of different cards, and a GUI window was just a card idea, and then you could print whatever you wanted on the card and then there would possibly be buttons or something on the card, and if you pushed one of the buttons, you go to a different card. And so you can think of an application as a deck of cards. Or a choose your own adventure-type game. And that's where a lot of really complicated choose your own adventure games were made, on HyperCard. And so PyperCard's a similar thing, just in Python and it started out with Adafruit, but it's now gone, Nicholas has made it, you can just run it on your, anywhere. And you can just pip install it. And I was playing with it the other day, and at first I dismissed it, but when I had him on Test & Code on Episode 89 and we talked about it a little bit, and I didn't realize this. So one of the ideas is you have, basically a simple list of what's in the card and then what card to go to next with the different buttons and inputs and stuff like that. But if you, instead of saying which card to go to, you give it a function to go to, you can run any Python code as long as that function returns a string to tell you which card to go to next. So that makes it actually pretty powerful to build up some fairly complex logic very simply. Anyway, it's fun to work with, I was even thinking of some work business uses that I could use this for, and then get it mocked up really quick.

18:59 KENNEDY: That's cool, looks interesting. It feels a little bit like a visual workflow executor. Like I go down this step, then I do this step, then there's the decision and then I do this step, or that step, and it's all represented by these cards. That's pretty cool.

19:12 OKKEN: He has an example of a Celsius to Fahrenheit Converter, but it also, I mean there's a lot of little simple conversion math stuff, applications in businesses, that would be a good use for it. And its also uses that user interface thing that I always forget, but it was like, nevermind, I can't remember what it is. It's one that you don't see very many examples of.

19:34 KENNEDY: Cool, yeah. It looks like a nice project.

19:36 OKKEN: Cool.

19:37 KENNEDY: You know, something we don't talk a whole lot about on this show is Node.js.

19:39 OKKEN: I don't know if we ever have, have we?

19:41 KENNEDY: You know, I think the extent to which we talked about Node.js was maybe some security issue in npm and how, is it analogous thing over a PyPI. Maybe, like that's the extent of it. So, Node.js, obviously, is a pretty big server-side framework for running JavaScript code on the server, and you know the stories like, hey, if I got to write JavaScript on the front end, I might as well write it in the back end. That's not my philosophy, mine is more like, how do I minimize the amount of JavaScript I write? Let's now also do that on the back end. I mean, not that I hate JavaScript but I don't want to overdo it. Anyway, there's a cool project that brings some integration between Python and Node. So imagine you're working on some project or you're on some team, and they're like, "We've chosen Node." You're like, sigh, sad face. But you still want to use some Python or maybe you want to use some kind of Python library like NumPy or TensorFlow, whatever, something that doesn't have a JavaScript equivalent. So you can use this thing called PyNode and what it does is it's a bridge between Node.js and Python that allow you to call Python code directly from JavaScript.

20:53 OKKEN: Oh, wow.

20:54 KENNEDY: So, this is on the server. Like if it was on the client side then, who knows, we'd have to use something like Skulpt or Brython or, and then it would convert the Python to JavaScript, like that's a different thing, this is the CPython interpreter with everything that Python can do on the server, but you can interact with it, call it and get return values from those functions in Node.js. Like you call in to Python and you get your answer back.

21:18 OKKEN: Nice.

21:19 KENNEDY: So that's pretty cool, and the API for working with it's not too bad, you have to basically, in Node.js you would require stuff in to, load it up, so you'd require PyNode and then you get your PyNode, you could say start the interpreter, which will, I'm guessing, embed CPython in to the node process and then you just call functions. The way you directly call 'em is kind of crummy, you say pynode.call("add", 1, 2) to call the function add by passing one and two to it, and then you get a call back from that. I think you could do some interesting stuff by wrapping this up in to a JavaScript class. So you just say, create it, and then you just start, it has a .add that takes a 1 and a 2, and you could put like a facade that makes this much simpler, do a little async/await to get rid of the call back, potentially, things like that. But, yeah, it's definitely got some potential there, and if you're working in Node.js space and you're like, 'Ah, I really wish I had Python', well, PyNode.

22:18 OKKEN: It's great, I mean there's more and more projects where they really aren't one language, you're trying to do lots of different things or you're trying to migrate from one thing to another. Use the language that works the best for wherever you're at.

22:29 KENNEDY: And similar to the last one that I covered, not trying to send people away to that thing, not to node or to Octave or whatever, just acknowledging, just what you said, that there are these environments where you don't always get to pick Python, here's another environment where you can now blend these together.

22:45 OKKEN: Yeah, there's lots of languages that are good, none of 'em are as good as Python, but, just sayin'.

22:51 KENNEDY: That's right, alright, well those are the major topics that we're covering for the week. Do you have anything extra to cover?

22:56 OKKEN: No, although, putting together a meetup is more work than I expected.

23:01 KENNEDY: I appreciate you doing that, I'm sure that it is.

23:02 OKKEN: I did submit that there's like call for proposals out, and I did submit to a couple of conferences, so we'll see.

23:08 KENNEDY: Awesome, good luck on that. The call for proposals for PyCascades and also, I think, main PyCon, right?

23:14 OKKEN: Yeah, PyCon's open, Tennessee's open, a couple others are open.

23:17 KENNEDY: Super. I actually have nothing extra to share. Except for I have a joke.

23:21 OKKEN: Oh, good, I didn't bring a joke.

23:22 KENNEDY: It's not exactly a joke, but it's definitely something that's very funny. So have you ever heard the excuse that, 'Ah, man, this works on my machine'?

23:31 OKKEN: Yeah, it's closing a defect with cannot reproduce.

23:34 KENNEDY: Exactly, dude, this is working, why does this not work for you? Who knows what it is, it could be the version of some library that's installed, it could be permissions, who knows, but there is a, 'It works on my machine' certification program that I want to tell you about, okay?

23:49 OKKEN: Okay.

23:50 KENNEDY: This is a real program, and it has like an honors badge, it says, "Works on my machine", it's kind of golden, you can put that on your wall when you get certified, and they're real steps, so this is not new... Pretty old certification, but I thought it'd be fun and we haven't talked about it before. So there's some specific steps that you got to follow, and the steps here are talking about a compiled language, so maybe it slightly varies with Python, but, still, I think it's funny. So, step one is you have to compile your application. Getting the latest version from any recent, or any recent changes, from other developers, is purely optional and not required for certification, okay? Step two is you launch the application or website that's just been compiled. Step three is you cause, you're going to like this part as a person who's in to pytest and stuff, you cause one code path in the code that you're checking to be executed. The preferred way to do this is with Ad hoc manual testing of the simplest possible case for the feature in question. You may omit this step if the code changed with less than five lines, or if in the developers professional opinion, the code change could not possibly result in an error. And, finally, you now check your code in to version control, congratulation, you're certified. It's nice, right?

23:50 OKKEN: Yeah.

23:50 KENNEDY: There's probably a lot of certified people, they just need to claim credit they've done this, I'm sure.

23:50 OKKEN: Yeah, they just don't know.

23:50 KENNEDY: Well, they can now get the star and put it on their wall. All right, well, thanks for being here, as always, Brian, and sharing all this news with everyone.

23:50 OKKEN: Well thank you, bye.

23:50 KENNEDY: You bet, bye. Thank you for listening to Python Bytes. Follow the show on Twitter via @PythonBytes, that's Python Bytes as in b-y-t-e-s. And get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way, we're always on the lookout for sharing something cool. On behalf of myself and Brian Okken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.

Back to show page