Transcript #44: pip install malicious-code

00:00 Hello and welcome to Python Bytes where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 44 recorded on September 19th, 2017. I'm Michael Kennedy.

00:12 And I'm Brian Okken.

00:13 And it's been a big news for a big week news as a man, Brian.

00:16 Yeah, very big.

00:17 Yeah, we've got, I would say the most listener feedback and requests to cover a particular topic, which we're going to jump right into as the first thing. But before we do, let's just say thanks to Datadog. They are sponsoring this episode, and they have some others and they've got some great tools and even a way to get a free t shirt at Python by stuff m slash data dogs. So we'll talk more about them later. Why don't you tell everyone what the big news is, apparently there's malicious libraries found on pipe. Yeah, right. So pip install virus not so joyful as the Pippin saw anti gravity would make it right. It actually I think scared people more than than the real threat. But let's talk about it. Yeah, you know what, I didn't see what the actual code vulnerabilities what the thing was other than sort of a proof of concept stuff. So I don't know how big of a deal this is in terms of actual viruses and malicious code. But it certainly shows the door is open for somebody to sling in some very bad things.

01:15 So the story is that there were a number of malicious libraries found on PyPI. So these are basically packages that you would pip install, but they either did some sort of typo squatting, or they grabbed the name of something that was already in the standard library. So for example, people might try to use URL lib and didn't import it, right? And so they get an error cannot find library URL lib. And so then they go type pip install URL lib. Well, guess guess what? That actually goes out to PyPI and grabs a thing. And I think there's a misspelling like URL lib with one l not two, but they would grab those things and they would put those packages up there. And to be even more devious, what they did is they actually took the implementation and put it into those libraries, so that it would actually work like it should.

02:08 But it was so you you might not notice, right? You pip install the thing, you import the thing it works. But the problem is that the setup PY the actual setup code that installs or executes during setup, like when this is installing, that was where the viruses or the malicious code lived. And so that's bad. I look into it that the code that they were putting in there, there was this little it said, proof of concept, no harm, no foul or But it was collecting a username and your host IP address and sending that to some server in China.

02:44 Absolutely. So I think what the best write up on this was done by Dan Goodwin, I think, on Ars Technica.

02:52 So that's the primary link here to that article and the conversation.

02:56 I find Ars Technica to be the best place for the comments to actually be really meaningful.

03:01 meaningful. So there's a great bunch of things in there. But let's let's cover a little bit more of the details. There's a Slovak security authority that actually discovered these packages, they discovered these packages, and then send a message to the Python package authority, and they took those down right away. Alright, so those are supposed to be gone. But that doesn't obviously get them off of your servers, get them off of your developer workstation, if you pip installed something bad.

03:29 Right.

03:30 And there's actually a message from the PSF, they did an official response to this.

03:34 And, you know, we talked several times about the fragility of PyPI.

03:39 And just how we're depending upon this thing that there's really not a lot of resources put into.

03:43 All right, we talked about Donald stuffed, and I've had him on talk Python and things like that.

03:47 And so the PSF said, this is just a part of what they said is unlike some language packaging management systems, PyPI does not have any full time staff devoted to it.

03:56 It's a volunteer run project with only two active admins.

03:59 As such, it doesn't currently have the resources for some of the proposed solutions, such as actively monitoring new projects, like inspecting code as it gets uploaded.

04:10 Historically, and by necessity, we've relied on a reactive system to take down potentially malicious projects as we've become aware of them.

04:17 Does that make you feel better, Brian?

04:18 - Not really, no.

04:20 - No, it doesn't make me feel very good either.

04:22 It's like, well, if someone notices a virus, of course we'll take that down.

04:25 But other than that, like, good luck is basically what they're saying.

04:29 So there's some interesting comments, like I said, on that, that ars technica article, and I've linked to four of them.

04:35 One of them, this is actually, I've been thinking about how you deal with this, like, do you digitally sign these things?

04:40 And then, like, everyone's gonna get a key.

04:41 And then how do you know what a bad actors key gets used, which regenerate it, get it, there's a lot of issues with getting like trusted keys, right, sort of SSL style.

04:51 But this guy, girl goes by Hugh Hugh Hugh says, what if pip gets more paranoid?

04:59 So if you say pip install the thing, and there's a very slight misspelling or slight change to that, that is more much more popular, it'll actually instead of just install it, give you a list of things and say, it looks like you might be trying to install this other thing that's way more popular than this thing.

05:16 And that might be really interesting.

05:17 Like if the thing you're installing has two downloads and the thing you were trying to get had half a million downloads, maybe it will just say like error, you need to say like force it or something to that effect.

05:28 So what do you think about that?

05:30 - I'm a little uneasy with that.

05:32 Giving preference to popular projects just because they're popular, I don't know if that's the, maybe we're swinging too far.

05:38 - Yeah, possibly.

05:39 There's actually some stats on all the downloads of the bad packages.

05:42 They were not really bad, they were like really quite small numbers, There's some graphs and stuff.

05:47 There's a person on the comment section that said their name was Stestag.

05:52 He said, "I'm sitting on a lot of the misspellings of common package names." So that's pretty cool.

05:58 Apparently, I've created packages that do nothing, that are like typos.

06:03 So typo squatters can actually do various stuff with it.

06:06 There's an undergrad, I think in Germany, who studied this capability and said, "Actually, there's this problem." it was like a year ago, they had sort of said, look, this can be a real problem. But you know, we can, I guess feel a little bit better in that he also did the same thing to Ruby.

06:22 And he also did the thing, same thing to NPM for node j s. So it's kind of a common theme that there's this challenge across all the official package repositories. Yeah. And the one of the notes also was that people like trying to install a something that's part of the standard library, it shouldn't come from PyPI.

06:42 - Absolutely.

06:43 - So there has been a change to the warehouse to not allow new, or have new packages that have the same name as standard library packages have to go through the approval process for that.

06:54 - Yeah, and you link to a PR pull request to 409 on PyPI/warehouse.

07:01 And that's pretty interesting, that conversation.

07:05 So I see how people are talking about solving the problem, which ones are there, how to deal with ones that are already there, but those are actual backports of say, like, somebody wants to bring async IO to Python two or to like a lower version of Python three, then maybe they put that package up there and it's, it would look like one of these bad named things, but they said the solution that they're considering is basically you can't create new ones without some sort of admin being involved to say, yeah, I see what you're doing and it's okay.

07:32 But the ones that exist, they won't like kill them off or anything.

07:34 Yeah.

07:35 And one of the big example of that is, for instance, mock is in the standard library as of Python 3, but in Python 2, it was separate.

07:45 So I guess mock is really part of the unit test library.

07:48 Right, but it has a legitimate place both in the standard library for Python 3 and on PyPI.

07:53 Yeah, there are some legitimate backports that show up.

07:56 So there's legitimate reasons to have the same name.

07:58 So that's a pretty nice segue to this news that Jonas Newbert sent us about the new version of PyPI, which is called Warehouse, and it might be finally moving.

08:10 Yeah, and actually, so this was great.

08:13 Jonas sent us an email and essentially he did almost all of my research for me, which I love that.

08:19 Thank you, Jonas.

08:20 Feel free to do that, anybody.

08:21 So he was writing an article, he was talking about the research he did for a topic when he wrote a blog post which we have a link to called "Publishing your first PyPI package by and for the absolute beginner." And it's a pretty nice quick article.

08:38 It talks about, well anyway, one of the things he talked about when he emailed us is things have changed and so a lot of the tutorials that are out there aren't valid anymore.

08:47 For instance, let's see, the pypi.org is no longer, it used to be read-only when we were we're just playing with it.

08:54 But now it's really where you go through to publish packages, you write to there.

09:00 The old APIs at pypi.python.org/pypi are disabled, so you have to use the new one.

09:07 - Right, and if you have one of those hidden .pypirc files that you can configure like your package, username, password, URL, and so on, you have to change that URL, right?

09:18 - If you're already done packages and pushed them up before, some of this will make sense and some of it won't, but if you read Jonas's article, all of it will make sense.

09:26 - Yeah, absolutely.

09:27 And I also had some good news, like things like Markdown support is coming for the readme.md files, yeah.

09:34 - That would be great, yeah.

09:36 I'm looking forward to, I refuse to write restructured text, so when I need it, I convert it from Markdown.

09:42 - There you go.

09:44 Yeah, yeah, that's great.

09:44 So this is good news.

09:46 - A couple of things, one of the other things that I thought was interesting is that apparently, I didn't know this, that you could change some aspects on the old PyPI, some aspects of your project, like the description or something.

09:57 There was a way to change that through the web interface or through the API without changing your package itself.

10:04 And a lot of those have been closed down and you really have to just re-upload your stuff if you wanna make quite a few changes.

10:12 And I actually think that's the way you should do it anyway.

10:14 So that's all right.

10:16 - Yeah, that sounds good to me.

10:18 I've been long waiting for pypi.org to be the thing.

10:23 And it's just a nicer interface.

10:25 It's built in Pyramid, which is kind of cool.

10:26 And I know that it's like a huge revision of a very, very old and sort of kludgy code.

10:31 So it will also open up PyPI for more contributions and collaboration with other people.

10:38 - Yeah, and I'd really like for them to, I think it's totally usable now.

10:41 I'd really like to have them take down the red notification at the top that makes it look like a warning.

10:47 I don't think we need that anymore.

10:49 - Yeah, it feels like it's gonna go pretty soon.

10:52 But yeah, definitely that should move to the old one and it should just stay, it should be gone from the new one, right?

10:57 I'm ready for the switch to happen.

10:59 I understand that pip actually references, you know, pipi.org and such for its URLs internally on something.

11:05 So it's kind of there anyway, but it's not, I don't know, it feels a little gradual.

11:11 - And apparently the one holdout is you have to, right, currently still you have to create your user account on the old website.

11:20 - Maybe that's why that red bar is still there.

11:21 - Maybe. - Maybe.

11:22 All right, so last week we had a lot of fun talking about David Beasley's fun of Reinvention, right?

11:28 - Yeah, I love that.

11:29 Yeah, I love to talk too.

11:30 If anybody hasn't watched that, go back and watch that.

11:32 - Yeah, we're basically linking to it again 'cause it was awesome.

11:35 - One of the things he did really well was he had these really cool live, he was live coding during the presentation and he had some cool backgrounds and stuff and we have no idea how to do what David did.

11:47 We asked him and he won't share it yet.

11:49 - Yeah, and if anyone knows, go to pythonbytes.fm/44 and add a comment at the bottom so we can all figure out how that cool trick was done.

11:56 - Yeah, definitely.

11:57 But for now, you can do live coding.

12:00 I like live coding in a presentation, but it can go wrong if things go wrong.

12:04 So I went out, I have a presentation that's coming up and I was thinking about whether I wanted to do this.

12:10 And so I found a few links talking about it, about advice.

12:14 One of them is basically advice for live coding and it's basically practice a lot and have a backup plan.

12:22 I guess that's the real meat of it.

12:24 And then also one thing is while you're coding a lot, it might be fun for you just to code but you have to talk at the same time.

12:31 So if you can't talk and code at the same time, maybe it's not for you.

12:34 So if you wanna have the same effect but not live code, So there's a couple other articles called Not Quite Live Coding and Avoiding Live Coding.

12:43 They're kind of cool.

12:44 They're talks about basically how you can do like GitHub labels or get labels to pull in new parts of your code if you want to watch it.

12:53 And my favorite.

12:54 - Right, you can basically go from like tag to tag to tag and then talk about the new code that's appeared without actually typing it.

13:01 Although I'm with you, I'm for the live coding.

13:03 That is the most legit, but like, these are fallbacks and I think that's not bad.

13:07 The last one is supposedly a bit of work.

13:10 I'm going to have to try this out, is doing a fade in.

13:13 So you've got all your code showing up on a slide.

13:16 But instead of showing a huge eye diagram of a whole bunch of code, and nobody knows really--

13:21 are they supposed to just read all the code at once?

13:24 Is to fade in the code a snippet at a time, highlight the piece that you're talking about, and then for the next slide or the next fade in, fade in the new piece of code.

13:34 And I hadn't actually seen how to do that before, but it talks about using reveal.js and some other tricks to do that.

13:43 - Yeah, that's a really nice effect.

13:44 If you're gonna have code up there or even lots of text in any sort of presentation, definitely don't just blast it all up there.

13:51 Let it come in piece by piece or somehow indicate the little sections you're talking about, and that definitely makes it more engaging, for sure.

14:01 - I brought this up also today 'cause I was curious about your choice.

14:05 It sounds like you like live coding as well, watching that at least.

14:07 - Yeah, I'm definitely for the live coding.

14:09 Like if people do it well, like when it goes bad, it kind of makes me squirm and be uncomfortable, but done well, I think like, you as an audience member, if you see something being presented and then you actually saw every step of it and then in the end you see the outcome, you're like, well, I saw every bit of it.

14:27 There was nothing that was crazy there and now it's doing this.

14:30 Like I feel like I could totally do that.

14:32 There's nothing sort of scary about it anymore once you see it done live.

14:37 And I think a lot of times you can skip over that and just sort of like fling pieces of code together and then you're like, well, yeah, but those are slides.

14:45 Maybe this is way harder than it sounds.

14:47 You know, if you see it done live, you kind of know how hard it is.

14:49 - Yeah, I agree.

14:50 I think I'm gonna opt for something almost there at first.

14:55 - Yeah, of course.

14:56 - And I'd also like to hear from my listeners to see, I'd like to hear like some live coding horror stories, and also some tips for how to do some Python live coding if anybody has any cool tools to share, that'd be great.

15:10 - Yeah, sounds awesome.

15:12 All right, before we get to our next topic, let's talk about Datadog.

15:15 So they're sponsoring the show and they're doing really cool stuff.

15:18 So if you have performance or bottlenecks in your application, that may be in your code, but it might be just somewhere in the whole stack that you're using.

15:26 So like let's say you have a Python web framework, a web app running Flask and it's built upon Mongo and it's Skeleton on Ubuntu running Nginx and MicroWizky.

15:35 With Datadog, you can actually monitor all of those pieces as a whole.

15:40 So that's super powerful if you wanna understand like really why your app's slow and not just why your Python code is slow.

15:46 So they have a great getting started tutorial and you can check that out, get a free Datadog t-shirt.

15:51 So just visit pythonbytes.fm/datadog and see what they've got to offer.

15:55 It's pretty cool.

15:56 - Yeah, and thank you Datadog for keeping the show rolling.

15:59 All right, let's talk, speaking of web, let's talk a little bit about REST.

16:03 - Okay.

16:03 - All right, so I mentioned Flask, I mentioned Pyramid, there's Django, of course.

16:07 Those are the three sort of high-level web frameworks.

16:10 And they're great, they're good for building web applications, there's extensions, or even they themselves are good for building RESTful services.

16:19 But there's two really interesting web API frameworks Python that a listener suggested we talk about and I'm excited to talk about them.

16:28 So there's these two called one is Falcon and one is Hug.

16:32 First of all, those are pretty good names for frameworks, right?

16:34 Yeah, they're pretty good.

16:35 I've heard of Hug, but I've never heard of Falcon.

16:37 Yeah, so I just had the Falcon guys on Talk Python to me last week on episode 129.

16:43 And that is a super low level, really high performance, RESTful framework.

16:49 So they call it a bare metal Python web API for building very fast backends and microservices.

16:57 And they don't see it as competing with those frameworks I mentioned, but they see it as more complimentary, like you write your app in that and if you need like that super fast little service, you use this.

17:06 And it even works on pi pi for extra extra speed boost.

17:10 So that's cool.

17:11 And you can use Falcon and it's really, really low level.

17:13 And then there's hug, which is actually a web, web service restful API built upon Falcon.

17:21 So they're sort of you want hug is using Falcon for his low level capabilities, but then hug is like a simplification on top of these API.

17:29 So you can do really interesting stuff with hug, like, you just put a decorator onto a function and all of a sudden it becomes an API that you can work with might be a method on a class, but you can work with that really simply.

17:45 And one of the unique things about it is it comes with built in self-documenting APIs, right, so it will like tell you can ask it what your functions are, and it'll give you a description.

17:55 And they're exposed over, you can expose them in different ways.

17:59 So maybe I have an API that I can access over HTTP.

18:03 But I could also make that a Python package where it exposes that API and make it like a command line thing where it exposes that as a command line thing.

18:11 those are all the same bits of code just exposed differently with Hug.

18:13 - Oh, that's cool.

18:14 - Yeah, that's pretty neat, right?

18:15 - Yeah, I gotta try that out.

18:16 - So if you're building RESTful services, give these two things a look depending on which level you wanna work at.

18:21 They're kinda neat.

18:23 All right, but you might wanna test those, right?

18:25 - You should test them.

18:26 So if you are testing them, you might want to test them in multiple environments.

18:31 And so, Talks would be a good thing.

18:33 - Yeah, we got a conversation, had a nice conversation with some listeners on Twitter.

18:36 Like, "Hey, what is Talks?

18:37 "Will you tell us what Talks is?" So Brian, tell me what Talks is.

18:40 - Well, yeah, first off, we're not gonna, we're gonna give a little sneak peek on what TOCS is, but I think it does quite a bit.

18:47 So I reached out to one of the TOCS developers, Oliver Bestweller, and he has agreed to come on Test and Code to have a longer conversation.

18:57 We haven't scheduled that yet, but we'll let you know when it's up.

19:00 But for now, TOCS, and this is a quote from Oliver, the name of the TOCS automation project derives from testing out of the box.

19:10 I didn't know that before I read this.

19:12 But it aims to automate and standardize testing in Python.

19:16 It's conceptually above pytest or whatever else you use and serves as a command line front end.

19:23 I think of it similar to something like a Travis CI or something that you could do on the command line.

19:29 - Right, it lets you pick different versions of Python.

19:32 So you could say Python 2.7 and Python 3.5.

19:36 And it basically depends upon pytest or something like that, right?

19:40 It'll orchestrate running your tests on pytest in those environments, for example.

19:45 - Yeah, and one of the things that I really like about it is it's when you are distributing something, you're not just, it's not just your code that you need to test, it's also the packaging and installation process and all of that.

19:55 You want to make sure that all that works.

19:57 And so essentially what it does in this normal, this is the normal use model, is to list a handful of Python versions.

20:05 And then what Tox will do is use your setup.py file to create a source distribution, and then create a virtual environment, and then install dependencies, and then install your package, and then run the tests, and then do all of that for each of the different Pythons.

20:22 So using different versions of Python to run the setup, all the way through running the tests.

20:27 - Yeah, that's really cool.

20:28 - And that's really, if you let it do all that, you have to wait for it, it's slower because you're creating that distribution every time.

20:35 and other things.

20:37 But there are, I left, there's a couple of links in the show notes on how to, on some tips and patterns and that are, you can speed things up if you need to, but just having this ability just at your desktop in the command line is really great for testing your stuff.

20:52 - Yeah, that's really cool.

20:53 And I believe there was something to do with Python 2 and that original vulnerability stuff that people discovered on PyPI, right?

21:03 like the vulnerable code only ran on Python 2 or something, right?

21:06 And that's how they discovered it?

21:07 - I think that's the case.

21:09 I don't have--

21:09 - I don't have it pulled up either, but yeah.

21:11 - A source to verify that, but I, like on Twitter, somebody said, oh yeah, and we found this because of talks and testing this stuff on Python 3.

21:20 - Yeah, that's beautiful.

21:22 All right, awesome.

21:23 So last one, I wanna talk about Legacy Python a little bit as well.

21:26 So there's Flake 8, right, which is a linter and talks about your code and tells you you're doing right and wrong, things like that.

21:33 There's a, I think it's a plugin called Flake 8 Tidy Imports.

21:38 And so one of our listeners said, "Hey, I added this cool feature to Tidy Imports and I thought it was pretty cool so I thought I'd highlight it here." People who are moving to Python 3, you might want to check this out.

21:49 So you can declare Python 2 to 3 as a band module import in Flake 8.

21:56 And then it'll go through and actually find any of the modules that would have worked in Python 2 but not in Python 3. For example, mock, right? So you used to say import mock but now you would just use import unitest.mock as mock or something like this, right? So it would actually give you that warning like in Python 3 you don't use mock anymore, you use unitest.mock and it gives you like a nice useful message not just this was not you shouldn't use this anymore but here's the thing to use instead as you do this upgrade. So it kind of shames people a little bit for using the old stuff which is good. Yeah, I really like I like it, actually I use that as well.

22:30 - That's great.

22:31 Very nice, and I have a bonus one for us actually.

22:33 I wanna throw it in really quick.

22:34 - Okay.

22:35 - So Jesse Davis from MongoDB, did a PyMongo driver, stuff like that.

22:39 He actually is the organizer for PyGotham.

22:43 So that is the PyCon for New York City.

22:46 And he's really into helping and mentoring people, especially people who are new speakers.

22:51 So he's running this project where he's trying to raise money to hire a speaking coach to work with and mentor first-time speakers who he's getting to come speak at PyGotham.

23:02 And he's trying to raise $1,200, and it turns out just like today, yeah, as of today, he's raised his goal, but I'm sure that he can do more if he had some more money.

23:11 So I'm linking to his article called "Help Me Offer Coaching to First-Time PyGotham Speakers," which I thought was a cool project, and I'm happy to spread the word for Jesse 'cause it's great to have more people coming in to the community.

23:24 - Yeah, I think that things like this are awesome and I like covering it anyway.

23:28 And I asked him to maybe write up something after the conference, but I'd like to hear how that goes.

23:34 I'd like to hear from the people that got coached and how the process went, if it helped things.

23:40 - Yeah, that'd be really cool.

23:41 Sort of retrospective, like was this actually useful?

23:44 Like what did you learn?

23:45 Right, like to see if it's something we should keep doing as a community, yeah.

23:48 - And then other conferences, and I don't have any links right now, but some conferences do mentors for submitting your proposal.

23:57 So a talk proposal, they'll have a mentor program so you can work with somebody to build up your proposal in the first place, which I think is--

24:05 - That's kind of the first step to being a first-time speaker.

24:08 - Okay, cool. - Awesome.

24:09 Well, good job, Jesse.

24:10 How about you, what other news you got?

24:12 Have you forgotten about your book and you're just relaxing, living life again?

24:15 - It's printing, no, I haven't forgotten.

24:18 But I am relaxing a lot more.

24:21 and there's sunshine outside.

24:22 I'm going outside more, which is good.

24:25 Not sunshine today. - You're actually seeing the outside.

24:27 - Yeah, but I'm seeing the outside.

24:28 - Yeah, that's awesome.

24:29 - But the physical, you can order them now.

24:32 Apparently they're printing and shipping, so that's awesome.

24:35 - Yeah, very good, very good.

24:36 That's great to hear.

24:38 So remember last week I talked about adding Switch to Python, and I said I'll put it up on GitHub?

24:42 - Yeah, and you did.

24:43 - I did, and I would say about 75% of the people said it was awesome, so cool, and 25% of people said, "Please, no, don't do this." But you can't please everyone and it's not changing the language.

24:55 It's just a package on GitHub.

24:58 You can do whatever you want with it.

24:59 So anyway, it was actually in the top Python trending packages on GitHub out of all Python packages.

25:06 Sorry, repos.

25:07 - Wow, really?

25:08 - Yeah, yeah, last week it was pretty awesome.

25:09 - That's great.

25:10 - And it had like 175 comments on Reddit or something.

25:15 So it's an interesting set of conversations that comes up around it.

25:19 So that was a follow up to last week, where I talked about that.

25:23 And then also I'm writing a free MongoDB course that's gonna compliment my paid MongoDB course, right?

25:29 Like a short one that's an intro sort of thing.

25:31 So people can, there's a link at the bottom of the show and those people can sign up to get notified.

25:36 That'll probably be out.

25:37 I finished writing that this week, like this morning, and I'll probably have that out in a few weeks.

25:41 - That's great.

25:42 - Yeah, should be fun.

25:43 All right, well, Brian, thanks for doing all the research or having our listeners do some research for you. It was really fun to talk about this. And if you guys have thoughts, especially on the PyPI security thing, go to pythonbytes.fm/44 and add your thoughts at the bottom. This is kind of a big deal.

26:00 >> Yeah. And thanks everybody for helping come up with ideas for the show. We always appreciate it.

26:06 >> Yep. Keep it coming. Very much appreciated. All right. Bye, Brian. Bye, everyone.

26:11 Thank you for listening to Python Bytes. Follow the show on Twitter via @PythonBytes. That's pythonbytes as in b-y-t-e-s. And get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Aukin, this is Michael Kennedy.

26:32 Thank you for listening and sharing this podcast with your friends and colleagues.