Transcript #37: Rule over the shells with Sultan

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 37, recorded July 31st, 2017. I'm Michael Kennedy.

00:11 And I'm Brian Okken.

00:12 And we've got a bunch of sweet stuff lined up for you.

00:14 Brian, let's talk about contributing to Python.

00:18 If there's something in CPython itself we want to change, and we don't like how it works, maybe we just go change it.

00:24 It is open source, right?

00:25 Right, it is open source. And there is and has been a developer's guide for a long time for CPython.

00:31 But the URL has changed recently. I think I saw that recently.

00:36 But it's at devguide.python.org.

00:41 And I bring it up mostly because the URL changed, and I like that URL.

00:46 But also, it is how to contribute to Python.

00:49 But I was looking over the initial page where it drops in.

00:54 And there's some really cool links in there.

00:57 And one of them is just for everybody.

01:00 It's how to contribute to open source.

01:02 And another one is building welcoming communities.

01:05 And those are both at opensource.guide.

01:08 But they're pretty good articles.

01:11 I also liked basically the structure of some of the getting started.

01:15 They have a whole bunch of, I think it's a good onboarding thing.

01:18 Even if you're not going to contribute to CPython, I think it's a nice outline of things to include in any large open source project to try to get people onboarded fairly quickly.

01:30 Yeah, for sure.

01:31 We've got, like, here's how you propose changes to Python itself.

01:35 They've got some nice things that take you through the whole, like, get flow style.

01:40 You know, the lifecycle of a pull request, running and writing tests, talking about coverage.

01:45 You know, there's a lot of good stuff here.

01:46 Yeah, and talking about the issue tracker.

01:48 And apparently there's an experts index, which is kind of native, like, who do you send issues to.

01:53 And then even things like how to run the tests and stuff.

01:57 So it's nice.

01:58 Yeah.

01:59 Yeah, very nice.

02:00 Cool.

02:00 So if you have an idea for CPython, the thing is still changing a lot.

02:05 Like, we just covered recently the 362 changes.

02:08 And that was pretty extensive for just a little, you know, tiny release to, like, basically a maintenance release.

02:14 So how would you like to rule over your shell, Brian?

02:18 Rule over my shell.

02:20 Yeah.

02:20 In fact, not just ruling over your shell, but over many of the shells.

02:24 Like, even the shells that are in the outer kingdoms.

02:26 So the thing that I want to talk about next is called Sultan.

02:30 And it says, command and rule over your shell.

02:33 And so basically you can run shell commands using this library, this package called Sultan in a Pythonic manager.

02:41 So the simplest thing, like, let's suppose you want to install tree.

02:46 So you can type tree and you get this cool, like, ASCII hierarchical view of your files.

02:51 And you would just say, you know, pip install Sultan.

02:52 And then you would say, you know, pip install Sultan.

02:54 And then you would say, create a Sultan object.

02:56 And you can say, Sultan.sudo yum install tree is a string.

03:00 And then say, dot run.

03:02 And it will actually go and it will run that command as sudo on your system.

03:08 Right?

03:09 That's pretty cool.

03:10 And you can do things like work with aptitude or yum or ls or all the shell commands.

03:17 Right?

03:17 And they also have a way to work with it inside of a with block, a context manager, so that it can clean up with itself.

03:24 It runs a slightly different command that I think has some sort of better cleanup.

03:30 I don't actually know all the details with yum.

03:33 So anyway, it's pretty cool.

03:35 It even works remotely.

03:36 So I could, like, say, Sultan.load and give it a host name and say to do that as sudo and then run that remotely on the server.

03:45 And if there's any errors or whatever, it brings all that information back and prints it out for you in the warnings, I think.

03:51 I think it would be useful.

03:52 I think it right off the bat, some of the automation stuff that just large projects have, like build scripts and whatever.

04:00 I'd like to have them in Python, but sometimes I don't really know how to do it within Python itself.

04:07 But this is kind of neat.

04:09 I'll have to check this out.

04:10 Yeah, it's pretty cool.

04:11 I think it's pretty focused on the use case that, you know, people have, like, running stuff over SSH, remote servers, a lot of shell stuff.

04:19 But if you find yourself doing that a lot and you want to do more of it in Python, here's a pretty cool library.

04:23 It's really simple.

04:24 So what is yum?

04:25 Yum is a package manager.

04:27 I don't remember which distribution it's for, but, like, Ubuntu has aptitude, you know, like apt-get.

04:32 Yum is for some of the other distributions.

04:34 For some others.

04:35 Okay.

04:36 Got it.

04:36 Yeah, I don't remember.

04:37 I probably just haven't run into that yet.

04:39 Yep.

04:39 Okay.

04:40 It's tasty.

04:40 Okay.

04:43 Next, I just want to talk about a tool that helped me a lot recently.

04:48 And I've been working on the book, which I'm not working on it anymore because it's turned into my editor.

04:56 I've been using Sublime to write it and write all the code for it.

05:01 But one of the things I wanted to do was to, I got harassed by not being PEP 8 compliant in a couple places.

05:09 So I wanted to quickly go through and just check everything.

05:11 Once, of course, I can use Flake 8 just on the command line.

05:15 But once I know that there's a bunch of files with errors, what's the fastest way to go fix them?

05:20 And for me, it was a Sublime Text plugin called Flake 8 Lint.

05:24 And I know there's a lot of Sublime Text linters out there.

05:29 But this one, like, did everything I needed it to do just right off the bat and was super easy.

05:35 It just installed it and it just worked.

05:38 So I highly recommend Flake 8 Lint.

05:41 It's got all the, it's got Flake 8, of course, and it's also got PyDoc style and some naming conventions, import order and stuff.

05:49 Yeah, I think having these automated tools makes tons of sense, right?

05:53 Like, your editor should do as much of the work for you as possible.

05:57 So definitely a good recommendation there.

06:00 So one example is I would bring up, if I knew that there was a directory with a bunch of files that had problems,

06:06 I would bring it up, open up a file.

06:09 And the default, I didn't ever change it.

06:11 The default is just to check it on save.

06:13 When I would save the file, it would check everything and then also pop up a pop-up window with all the errors.

06:21 So I could, if I could just hit enter, it would take me to like the very first error and I could just walk through all the errors.

06:27 It was great.

06:28 Nice.

06:28 And it's not just, just purely following like the PEP 8 rules.

06:34 Like, for example, it has a cyclomatic complexity checking, which is pretty sweet.

06:38 So that's like, hey, I've got a highly, highly nested 500 line method.

06:45 What's wrong with that?

06:49 It only has 20 if blocks, loops, whatever.

06:51 It would complain about that if it goes over 10.

06:54 That's cool.

06:55 All right.

06:55 So how would you be like, how would you like to, you know, travel time and be sucked into, say, like a magic wormhole?

07:01 I'm pretty excited to hear about magic wormhole.

07:03 Magic wormhole is pretty cool.

07:05 Maybe you guys have heard of this.

07:07 So the problem magic wormhole is trying to solve is I want to get things, be this text, be this a small file, be this a huge file from one computer to the other safely.

07:17 So encrypted in a way that other people can't get to and things like that.

07:22 So magic wormhole is basically a command line thing where you just type wormhole and it will come up with like a code that your computer has.

07:36 Right.

07:36 So you say, I want to create a wormhole and send this file and it'll come up with a code.

07:39 If you give that code to someone else, they can run wormhole receive on their command line, enter that code.

07:48 And then over an encrypted connection that will automatically connect the two computers, even if they both have firewalls blocking it.

07:56 There's like some sort of intermediary server that will link them together.

08:03 And it'll just do, from what I understand, end-to-end encryption between these two things.

08:08 So if you need to get something and you don't want to put it somewhere public like Dropbox or you don't want to put it somewhere that's hard to erase or just hard to actually physically transfer like a USB drive, you can wormhole it.

08:21 And there's even a PyCon presentation on this at PyCon 2016.

08:26 So you can check it out.

08:27 What do you think about that, Brian?

08:29 That looks pretty cool.

08:30 I don't think I've got a use for it, but I want to have a use for it.

08:33 I know.

08:34 Well, you know, like suppose you want to get something to some server you're maintaining, maybe, or say you're on vacation and you want to send something back.

08:43 You don't want to send it over Dropbox.

08:44 I can see some good uses for it.

08:46 They talked about things like here's a way that you might be able to share like SSH keys that you don't want to put anywhere, but you've got to get them somewhere.

08:52 Things like that.

08:54 Yeah.

08:54 Yeah, it's pretty cool.

08:56 The codes, one of the things I thought was a nice touch was the codes are short and human pronounceable.

09:01 Oh, wow.

09:01 Yeah.

09:02 So they had some nice examples.

09:03 Let me see if I can find it.

09:05 Yeah.

09:05 So like the one that they have an example on the website is 7-crossover-clockwork.

09:11 So pretty long password, if you will, but also you can tell it to someone over the phone.

09:18 So the idea is like this is when you have the ability to communicate with a person, but your computer and their computer don't necessarily have a way to directly connect.

09:27 Oh, great.

09:27 So you don't have to like email them the secret code or something non-secure.

09:32 Yeah.

09:32 I emailed you the ISO.

09:34 It was only three gigs.

09:35 I'm sure it'll be fine.

09:36 Thanks for that.

09:36 And finally, there's actually an API that's part of this.

09:41 So it's not just a command line tool.

09:43 You can actually make wormhole capabilities in your own applications by just by importing the library.

09:50 Okay.

09:51 Now that sounds even more fun.

09:53 I don't know what to do with it, but it sure sounds fun.

09:55 It sure does.

09:55 Yeah.

09:56 So take us back to some fundamentals.

09:58 One of the things that I think we really don't do enough with as developers is virtual environments.

10:04 What do you think?

10:04 It's one of those things that I forget that it's hard to figure out the first time.

10:09 So virtual environments are something, if you're not using virtual environments, you need to be.

10:15 And there's an older tutorial, but this is from realpython.com.

10:21 And it's a Python virtual environments, a primer.

10:24 It covers what they are.

10:26 Why do we need them?

10:27 How do you use them?

10:28 Of course.

10:29 And then it also covers a few extra tools that a lot of people use, like Pyvenv for that came with Python 3.6 and virtual env wrapper.

10:39 I point to it because it's a pretty good, this article is a pretty good description of really the, why do we need them and how and what they are for?

10:47 If you're not understanding of what they're, why you need them.

10:50 This is a good, good document to talk about it.

10:53 The thing that I ran into recently was I was, I was trying to help somebody and install something.

11:00 And it turned out that the problem that they were, that they were having was they were not using virtual environments.

11:06 And when they would pip install something, they weren't able to import it from within the tests.

11:12 And the problem was that their pip was pointing to a different Python than the py test was using a different Python.

11:19 So when you pip install something, it goes into the virtual environment or the, the site packages for that version of Python, not for all of Pythons.

11:29 So yeah, he was pip installing it with Python.

11:33 His pip was 2.7 and it goes into 2.7 folder then.

11:38 And for me, that's the main reason why I, one of the main reasons why I use it is because once you use virtual environments, your pip and your Python and your py test and everything are all using the same Python and they don't get confused.

11:51 Yeah, it's great.

11:52 I use it a lot to basically, basically make it very, very clear what versions a particular web app of mine is running and what I need to deploy and things like that.

12:02 I think your point about it's easy to forget why this should ever be hard.

12:07 Like it's just second nature for those of us who've been doing a long time, but your example is perfect.

12:14 Like helping someone out who is somewhat new.

12:16 Like I thought Python was simple.

12:18 Why is this not working?

12:19 What is this?

12:20 That happens all the time.

12:22 And the, all the extra stuff like virtual and wrapper and py environment and all these, these extra things, those are great for the people that work with them.

12:31 But if you, if you try these out and they don't work for you, it might not be your fault or maybe it is, but I haven't been able to get any of these extra things to work.

12:40 I've tried virtual and wrapper.

12:41 It's never worked for me.

12:43 So I don't know what I'm doing wrong, but just the straight, straight virtual environments work great.

12:48 So that's what I use.

12:49 Yeah.

12:49 Yeah.

12:49 That's, that's cool.

12:50 Nice to have a little reference there.

12:52 So last thing I want to touch on is an interesting article that actually highlights a couple of projects.

12:58 I think there's four projects that highlights.

13:00 Okay.

13:00 Yeah.

13:00 And the idea is Rust, Rust, the programming language from Mozilla is a pretty interesting alternative to C, right?

13:10 It's, it's kind of a systems level programming language like C is, but it has a type system that helps eliminate memory leaks.

13:20 It has proper interfaces.

13:21 It has better type inference, better support for concurrency, better support for first class functions, those kinds of things.

13:27 And so compared to C, it's a pretty nice language.

13:31 And so when you think of a lot of the important Python packages, many of their important internal low level pieces are written as C extensions, right?

13:43 Yeah.

13:43 Yeah.

13:43 So could those be written in a better language than something like C?

13:48 How about Rust?

13:50 So here's four projects that make it easier to implement the, what would be the C extensions, the native compiled hotspot type of code, but do it in Rust instead of in C.

14:03 So the first one is Rust dash CPython.

14:07 So that's an interesting, just describes what it is.

14:11 So it's a set of bindings for Rust for the CPython runtime.

14:15 It allows Rust to connect to CPython using its application binary interface, which is a Rust thing.

14:21 And you can run Python programs from Rust or work with Python objects directly, the representations of Python objects directly in Rust.

14:33 So this is actually for if you want to, if you're a Rust programmer and you want to hook into CPython and control it from the outside.

14:39 There's Py03, which is a way for, for people to write software, Rust software with bindings of Python in both directions.

14:51 So a Rust program can interface with Python objects and the Python interpreter, and it can expose Rust methods to the Python program the same way like a C module does.

15:05 All right, so that's pretty cool.

15:07 Then there's Snake, S-N-A-E-K, which is a kind of early stage project, but it lets developers create Rust libraries that are loaded dynamically into Python as needed.

15:18 And they don't have to be statically linked against Python's runtime, which is kind of cool.

15:23 And I think it pulls it off using CFFI, which is the C interoperator for PyPy, the alternate interpreter JIT compiler for Python, rather than C types.

15:35 So if you don't have some old library that is based on C types, like say in C and you want to upgrade, then this might be a pretty sweet option.

15:43 And then finally, there's actually a cookie cutter template, and it says, the goal of this project is to be able to produce a binary distribution, a wheel, which will not require the end user to actually compile Rust code themselves, which everyone would appreciate that, right?

16:00 So here's a bunch of cool options if you are like, I really don't want to write this in C, but it probably should be written, you know, not in Python, but in some other way.

16:09 Like, maybe try Rust if you're looking to check that out.

16:12 That actually looks pretty cool.

16:13 Have you done any Rust yet?

16:14 No, I've, it's one of the languages I want to learn that I'm kind of intrigued by.

16:20 I looked at Go a little while ago.

16:23 I'm not a huge fan of Go, but people sort of compare these a lot.

16:26 So I want to check out Rust.

16:28 I think it's pretty different than Go.

16:29 Yeah, it's definitely something on my to-do list.

16:31 Yeah, cool.

16:32 Check this out.

16:33 Yeah.

16:34 Do you guys have any uses for Rust with all the hardware stuff you're doing?

16:38 I'm sure we could come up with, invent some reason to use Rust.

16:42 There you go.

16:43 Perfect.

16:43 All right.

16:45 How about you?

16:45 Any, any personal news here at the end of the show?

16:48 Just that I'm not writing anymore.

16:50 Congratulations.

16:51 That's awesome.

16:52 How's it feel?

16:52 It feels incredible.

16:53 Yeah.

16:54 To go from, I'm, I'm writing a book to, I wrote a book is great.

16:59 So.

16:59 Yeah.

16:59 That's cool.

17:00 Was your wife and kids like, Hey, there's some random guy in our house.

17:04 What's going on?

17:04 Oh wait, no, it's just Brian.

17:06 It's dad.

17:06 Yeah.

17:07 No, I've tried to, I've tried to limit the writing to, to things, obscene things like early in

17:14 the morning and late at night.

17:15 But yeah.

17:16 So punish yourself instead of your family.

17:18 Yeah, exactly.

17:18 So.

17:19 Cool.

17:20 All right.

17:21 Well, I am packing my bags and I'm going on vacation.

17:25 So either we might be a little bit intermittent or, you know, there's, there's some talk that

17:31 we might have some guests co-hosts coming in here.

17:33 We'll see.

17:34 so if you don't see some podcasts in the next few weeks, don't worry, it'll come back,

17:40 but you might see them anyway with, some new faces.

17:43 So yeah, that would be awesome.

17:44 So either way, great chat with you, Brian.

17:46 Thanks everyone for listening.

17:47 All right.

17:48 Thanks.

17:48 Yep.

17:48 Bye.

17:50 Thank you for listening to Python Bytes.

17:52 Follow the show on Twitter via at Python Bytes.

17:54 That's Python Bytes as in B-Y-T-E-S.

17:57 And get the full show notes at pythonbytes.fm.

18:01 If you have a news item you want featured, just visit pythonbytes.fm and send it our way.

18:05 We're always on the lookout for sharing something cool.

18:08 On behalf of myself and Brian Okken, this is Michael Kennedy.

18:11 Thank you for listening and sharing this podcast with your friends and colleagues.