Transcript #37: Rule over the shells with Sultan

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:06 This is episode 37, recorded July 31st, 2017.

00:11 I'm Michael Kennedy.

00:11 And I'm Brian Okken.

00:12 And we've got a bunch of sweet stuff lined up for you.

00:15 Brian, let's talk about contributing to Python.

00:18 Like, if there's something in CPython itself we want to change, and we don't like how it works, maybe we just go change it. It is open source, right?

00:25 Right, it is open source and there is, is and has been a developers guide for a long time for CPython.

00:32 But the URL has changed recently.

00:34 I think I saw that recently, but it's at devguide.python.org.

00:42 And I bring it up mostly because the URL changed and I like that URL, but also it is how to contribute to Python.

00:49 I was looking over the the initial page where it drops in and There's some really cool links in there and one of them is just for everybody It's a how to contribute to open source and another one is building welcoming communities And those are both that open source guide But they're they're pretty good articles. I also liked basically the structure of some of the getting started They have a whole bunch of, I think it's a good onboarding thing.

01:19 Even if you're not contributing to going to contribute to CPython, I think it's a nice outline of things to include in any large open source project to try to get people onboarded fairly quickly.

01:30 Yeah, for sure.

01:31 We've got like, here's how you propose changes to Python itself.

01:36 They've got some nice things that take you through the whole like, get flow style, you the lifecycle of a pull request, running and writing tests, talking about coverage, there's a lot of good stuff here.

01:47 - Yeah, and talking about the issue tracker, and apparently there's an experts index, which is kind of native, like who you send issues to, and then even things like how to run the tests and stuff.

01:57 So it's nice.

01:58 - Yeah, yeah, very nice.

02:00 Cool, so if you have an idea for CPython, the thing is still changing a lot.

02:05 Like we just covered recently the 362 changes, And that was pretty extensive for just a little, you know, tiny release to, like a basically a maintenance release.

02:15 So how would you like to rule over your shell, Brian?

02:18 - Rule over my shell, yeah.

02:20 - In fact, not just ruling over your shell, but over many of the shells, like even the shells that are in the outer kingdoms.

02:27 So the thing that I want to talk about next is called Sultan, and it says, "Command and rule over your shell." And so basically you can run shell commands using this library, this package called sultan in a Pythonic manager.

02:42 So the simplest thing, like let's suppose you want to install tree, so you can type tree and you get this cool like ASCII hierarchical view of your files.

02:51 And you would just say, you know, pip install sultan and then you would say create a sultan object because a sultan.sudo yum install tree is a string.

03:01 And then say dot run, and it will actually go and it will run that command and pseudo on your system.

03:08 Right, that's pretty cool.

03:11 And you can do things like work with aptitude or yum or LS or all the shell commands, right.

03:18 And they also have a way to work with it inside of a with block a context manager so that it can clean up with itself, it runs a slightly different command that I'm, I think has some sort of better cleanup.

03:30 actually know all the details with yum. So anyway, it's pretty cool. It even works remotely.

03:37 So I could like say sultan.load and give it a hostname and say to do that as sudo and then run that remotely on the server. And if there's any errors or whatever, it brings all that information back and prints it out for you in the warnings, I think. I think it'd be useful. I think it right off the bat, some of the automation stuff that just large projects have, like build scripts and and whatever I'd like to have in Python, but sometimes I don't really know how to do it within Python itself.

04:07 But this is kind of neat.

04:09 I'll have to check this out.

04:10 - Yeah, it's pretty cool.

04:11 I think it's pretty focused on the use case that people have, like running stuff over SSH, remote servers, a lot of shell stuff.

04:19 But if you find yourself doing that a lot and you want to do more of it in Python, here's a pretty cool library.

04:23 It's really simple.

04:24 - So what is yum?

04:25 - Yum is a package manager.

04:27 I don't remember which distribution it's for, But like Ubuntu has aptitude, you know, like apt-get, yum is for some of the other distributions.

04:34 - For some other, okay, got it.

04:36 - Yeah, I don't remember.

04:37 - I probably just haven't run into that yet.

04:39 So, okay. - Yep, it's tasty.

04:40 (laughing)

04:43 - Okay.

04:44 Next, I just wanna talk about a tool that helped me a lot recently.

04:48 And I've been working on the book, which I'm not working on it anymore because it's turned into my editor.

04:56 but the i've been using sublime to write it and write all the code for it but the uh...

05:02 one of the things i wanted to do was to i got harassed by not being uh... pep8 compliant in a couple places so i wanted to quickly go through and just check everything once of course i can use flake8 just on the command line but once i know that there's a bunch of files with errors what's the fastest way to go fix them and for me it was a sublime text plugin called flake8 lint And I know there's a lot of Sublime Text linters out there, but this one did everything I needed it to do just right off the bat and was super easy.

05:35 It just installed it and it just worked.

05:38 So highly recommend Flakegate Lint.

05:41 It's got Flakegate, of course, and it's also got PyDoc style and some naming conventions, import order, and stuff.

05:50 Yeah, I think having these automated tools makes tons of sense, right?

05:53 like your editor should do as much of the work for you as possible.

05:58 So definitely a good recommendation there.

06:00 So one example is I would bring up--

06:02 if I knew that there was a directory with a bunch of files that had problems, I would bring it up, open up a file.

06:09 And the default-- I didn't ever change it.

06:11 The default is just to check it on Save.

06:14 When I would save the file, it would check everything and then also pop up a pop-up window with all the errors.

06:21 So if I could just hit enter, it would take me to the very first error and I could just walk through all the errors.

06:27 It was great.

06:28 - Nice, and it's not just purely following the PEP8 rules.

06:34 For example, it has cyclomatic complexity checking, which is pretty sweet.

06:38 So that's like, hey, I've got a highly, highly nested 500 line method.

06:45 What's wrong with that?

06:46 (laughing)

06:49 only has 20 if blocks, loops, whatever.

06:53 It would complain about that if it goes over 10.

06:54 That's cool.

06:55 All right, so how would you like to travel time and be sucked into, say, like a magic wormhole?

07:01 - I'm pretty excited to hear about magic wormhole.

07:03 - Magic wormhole is pretty cool.

07:05 Maybe you guys have heard of this.

07:07 So the problem magic wormhole is trying to solve is I wanna get things, be this text, be this a small file, be this a huge file, from one computer to the other safely.

07:17 So encrypted in a way that other people can't get to and things like that.

07:22 So magic wormhole is basically a command line thing where you just type wormhole and it will come up with like a code that your computer has, right?

07:36 So you say, I want to create a wormhole and send this file and it'll come up with a code.

07:39 If you give that code to someone else, they can run wormhole receive on their command line, enter that code and then over an encrypted connection that will automatically connect the two computers even if they both have firewalls blocking it.

07:56 There's like some sort of intermediary server that will link them together.

08:03 And it'll just do, from what I understand, an encryption between these two things.

08:08 So if you need to get something and you don't want to put it somewhere public like Dropbox or you don't want to put it somewhere that's hard to erase or just hard to actually physically transfer like a USB drive, you can wormhole it.

08:21 And there's even a PyCon presentation on this at PyCon 2016.

08:27 So you can check it out.

08:28 What do you think about that, Brian?

08:29 That looks pretty cool.

08:30 I don't think I've got a use for it, but I want to have a use for it.

08:34 I know.

08:35 Well, you know, like suppose you want to get something to some server you're maintaining maybe, or say you're on vacation and you want to send something back, you don't want to send it over Dropbox.

08:45 I can see some good uses for it.

08:46 They talked about things like here's a way that you might be able to share like SSH keys that you don't want to put anywhere, but you got to get them somewhere.

08:53 Things like that.

08:54 Yeah.

08:55 Yeah, it's pretty cool.

08:56 The codes, one of the things I thought was a nice touch was the codes are short and human pronounceable.

09:01 Oh, wow.

09:02 Yeah.

09:03 So they had some nice examples.

09:04 Let me see if I can find.

09:05 Like the one that they have an example on the website is 7-crossover-clockwork.

09:12 So pretty long password if you will, but also you can tell it to someone over the phone.

09:18 So the idea is like this is when you have the ability to communicate with a person, but your computer and their computer don't necessarily have a way to directly connect.

09:27 Oh great.

09:28 So you don't have to like email them the secret code or something unsecure.

09:32 Yeah.

09:33 I emailed you the ISO.

09:34 - It was only three gigs, I'm sure it'll be fine.

09:36 Thanks for that.

09:37 (laughing)

09:38 And finally, there's actually an API that's part of this.

09:41 So it's not just a command line tool, you can actually make wormhole capabilities in your own applications by just by importing the library.

09:50 - Okay.

09:52 - Now that sounds even more fun.

09:53 I don't know what to do with it, but it sure sounds fun.

09:55 - It sure does, yeah.

09:56 - So take us back to some fundamentals.

09:58 One of the things that I think we really don't do enough with as developers is virtual environments.

10:04 What do you think?

10:04 It's one of those things that I forget that it's hard to figure out the first time.

10:09 So virtual environments are something--

10:12 if you're not using virtual environments, you need to be.

10:15 And there's an older tutorial, but this is from realpython.com.

10:22 And it's Python Virtual Environments, a primer.

10:25 It covers what they are, why do we need them, how do you use them, of course.

10:29 And then it also covers a few extra tools that a lot of people use like pyvenv, for that came with Python 3.6, and virtualenvwrapper.

10:39 I point to it because it's a pretty good, this article's a pretty good description of really why do we need them and what they are for if you're not understanding why you need them, this is a good document to talk about it.

10:54 The thing that I ran into recently was I was trying to help somebody and install something, And it turned out that the problem that they were having was they were not using virtual environments.

11:06 And when they would pip install something, they weren't able to import it from within the tests.

11:12 And the problem was that their pip was pointing to a different Python than the pytest was using a different Python.

11:19 So when you pip install something, it goes into the virtual environment or the site packages for that version of Python, not for all of Pythons.

11:30 So he was pip installing it with Python.

11:33 His pip was 2.7, and it goes into 2.7 folder then.

11:38 And for me, that's the main reason why--

11:41 one of the main reasons why I use it is because once you use virtual environments, your pip and your Python and your pytest and everything are all using the same Python, and they don't get confused.

11:51 Yeah, it's great.

11:52 I use it a lot to basically make it very, very clear what versions a particular web app of mine is running and what I need to deploy and things like that.

12:03 I think your point about it's easy to forget why this should ever be hard, like it's just second nature for those of us who've been doing it a long time.

12:12 But your example is perfect, like helping someone out who is somewhat new.

12:16 Like I thought Python was simple.

12:18 Why is this not working?

12:19 What is this?

12:20 That happens all the time.

12:23 And all the extra stuff like virtual envwrapper and Py environment and all these extra things, those are great for the people that work with them.

12:32 But if you try these out and they don't work for you, it might not be your fault. Or maybe it is.

12:37 But I haven't been able to get any of these extra things to work.

12:40 I've tried virtual envwrapper.

12:42 It's never worked for me.

12:43 So I don't know what I'm doing wrong.

12:45 But just the straight virtual environments work great.

12:48 So that's what I use.

12:49 - Yeah, that's cool.

12:50 Nice to have a little reference there.

12:53 So last thing I want to touch on is an interesting article that actually highlights a couple of projects.

12:58 I think there's four projects it highlights.

13:00 - Okay. - Yeah.

13:01 And the idea is Rust, Rust the programming language from Mozilla, is a pretty interesting alternative to C, right?

13:10 It's kind of a system level programming language like C is, but it has a type system that helps eliminate memory leaks.

13:20 It has proper interfaces.

13:21 It has better type inference, better support for concurrency, better support for first class functions, those kinds of things.

13:28 And so compared to C, it's a pretty nice language.

13:32 And so when you think of a lot of the important Python packages, many of their important internal low level pieces are written as C extensions, right?

13:43 Yeah.

13:44 Yeah.

13:45 So could those be written in a better language than something like C?

13:49 How about Rust?

13:50 So here's four projects that make it easier to implement what would be the C extensions, the native compiled hotspot type of code, but do it in Rust instead of in C.

14:04 So the first one is Rust-CPython.

14:08 So that's an interesting, just describes what it is.

14:12 So it's a set of bindings for Rust for the CPython runtime.

14:15 It allows Rust to connect to CPython using its application binary interface, which is a Rust thing.

14:23 And you can run Python programs from Rust or work with Python objects directly, the representation to Python objects directly in Rust.

14:33 So this is actually for if you want to, if you're a Rust programmer and you want to hook into CPython and control it from the outside.

14:40 There's Py03, which is a way for people to write Rust software with bindings of Python in both directions.

14:51 So a Rust program can interface with Python objects in the Python interpreter, and it can expose Rust methods to the Python program the same way like a C module does.

15:05 All right, so that's pretty cool.

15:07 Then there's Snake, S-N-A-E-K, which is a kind of early stage project, but it lets developers create Rust libraries that are loaded dynamically into Python as needed, and they don't have to be statically linked against Python's runtime, which is kind of cool.

15:23 I think it pulls it off using CFFI, which is the C interoperator for PyPI, the alternate interpreter JIT compiler for Python, rather than C types.

15:36 If you don't have some old library that is based on C types, like say in C and you want to upgrade, then this might be a pretty sweet option.

15:44 Then finally, there's actually a cookie cutter template.

15:47 It says, "The goal of this project is to be able to produce a binary distribution, a wheel, which will not require the end user to actually compile Rust code themselves," which everyone would appreciate that, right?

16:00 Here's a bunch of cool options if you are like, "I really don't want to write this in see but it probably should be written in you know not in Python but in some other way like maybe try Rust if you're looking to check that out. That actually looks pretty cool. Have you done any Rust yet? No I've it's one of the languages I want to learn that I'm kind of intrigued by. I looked at Go a little while ago I'm I'm not a huge fan of Go but people sort of compare these a lot so I want to check out Rust it's I think it's pretty different than Go. Yeah it's definitely something on my to-do list, so check this out.

16:34 - Yeah, do you guys have any uses for Rust with all the hardware stuff you're doing?

16:38 - I'm sure we could come up with, invent some reason to use Rust.

16:42 - There you go, perfect.

16:43 All right, how about you, any personal news here at the end of the show?

16:48 - Just that I'm not writing anymore.

16:50 - Congratulations, that's awesome.

16:52 How's it feel?

16:52 - It feels incredible, yeah.

16:54 To go from I'm writing a book to I wrote a book is great.

16:59 - Yeah, that's cool.

17:01 Was your wife and kids like, hey there's some random guy in our house, what's going on?

17:04 Oh wait, no, it's just Brian.

17:06 - It's dad, yeah.

17:07 (laughing)

17:09 No, I've tried to limit the writing to obscene things like early in the morning and late at night.

17:16 - Yeah, so punish yourself instead of your family.

17:18 - Yeah, exactly.

17:19 - Cool.

17:21 All right, well I am packing my bags and I am going on vacation.

17:25 So either we might be a little bit intermittent Or there's some talk that we might have some guest co-hosts coming in here.

17:34 - We'll see.

17:35 So if you don't see some podcasts in the next few weeks, don't worry, it'll come back.

17:40 But you might see them anyway with some new faces.

17:43 - Yeah, that would be awesome.

17:44 So either way, great chatting with you, Brian.

17:46 Thanks everyone for listening.

17:47 - All right, thanks.

17:48 - Yep, bye.

17:49 Thank you for listening to Python Bytes.

17:52 Follow the show on Twitter via @pythonbytes.

17:54 That's Python Bytes as in B-Y-T-E-S.

17:58 and get the full show notes at pythonbytes.fm.

18:01 If you have a news item you want featured, just visit pythonbytes.fm and send it our way.

18:05 We're always on the lookout for sharing something cool.

18:08 On behalf of myself and Brian Okken, this is Michael Kennedy.

18:12 Thank you for listening and sharing this podcast with your friends and colleagues.