Transcript #26: How have you automated your life, or CLI, with Python?

00:00 Hello, and welcome to Python Bytes, where we deliver Python news and headlines directly

00:05 to your earbuds.

00:06 This is episode 26, recorded May 16, 2017.

00:10 I'm Michael Kennedy.

00:11 And I'm Brian Okken.

00:12 And we're here to deliver a bunch of cool news around the Python community.

00:17 But before we do, I'd like to say thanks, Brian.

00:19 I'd like to say thanks to Rollbar, because once again, they're sponsoring this episode.

00:23 Yay, Rollbar.

00:24 Yay, Rollbar.

00:25 So they're going to be at PyCon, so be sure to drop by their booth and tell them that

00:28 we sent you.

00:28 All right.

00:30 Let's talk about the terminal.

00:31 Yeah.

00:31 When I think of command line interface, I mean, I use like the REPL and other terminal interfaces.

00:37 But most of the time when I'm writing applications that I think of as command line, they just

00:44 have, you can launch them from the command line with flags and stuff like that.

00:47 But if you wanted to do an interactive, like an interactive terminal application, like a

00:54 REPL or something, I don't know how to do that.

00:57 And this is, I ran across this two-part series article, a two-part series.

01:02 And the first part was just actually talking about, and this is on opensource.com.

01:08 And the first article talks about four different applications that have good interface characteristics.

01:14 And it started with the MySQL interface and the Python REPL, which were good, but both could

01:21 be better.

01:22 And the author talked about BPython as a adds auto-completion to Python.

01:28 Oh, BPython.

01:29 Interesting.

01:30 I've never heard of BPython.

01:32 I use PTpython.

01:33 Well, the article also mentions PTpython.

01:35 So I was going to try both of them out and see which one I liked better.

01:38 Oh, okay.

01:39 You'll have to give us a report.

01:40 Yeah.

01:40 There's a, apparently for MySQL, there's a MyCLI that adds context-aware completion.

01:47 I don't spend enough time in a MySQL to worry about that.

01:51 But then a couple others, if you're a Postgres person, there's somebody, something called

01:57 PGCLI that adds fuzzy search.

02:01 And then you talked about bash.

02:02 Like we spend our time on the bash command line and some of the, one replacement is called

02:07 phish that supposedly has better search history.

02:10 Oh, nice.

02:10 Yeah, that's cool.

02:11 I've heard phish is good.

02:12 I use omizeh.

02:13 And also conch is a Python-based one, which is cool as well for the shell.

02:19 Yeah.

02:21 So that was the first part.

02:22 And the second part of the series talks about how to do this sort of stuff with Python.

02:27 So he talks about actually taking four different libraries.

02:31 There's one called the prompt toolkit that you can use that to add command line, command

02:37 history, a REPL-like interface to your application.

02:40 And you can add things like command history and auto-suggestion and auto-completion, which

02:46 sounds cool, actually.

02:47 Yeah, that actually does sound pretty cool.

02:49 And then click, which I'm familiar with using click for like options and flags, but apparently

02:56 it also has a pager and the ability to launch an editor outside of it, which would be cool.

03:04 That sounds cool.

03:04 Yeah, yeah.

03:05 Very cool.

03:05 And then a couple more pigments for syntax highlighting and fuzzy finder, which is a way to make fuzzy

03:13 suggestions on whatever somebody types in.

03:16 That's cool.

03:17 And the nice thing is that it talks, there's an example on how to include, to hook up fuzzy

03:22 finder and prompt toolkit to make the completion look nice.

03:27 It's a well-written series.

03:29 And when I add interactive stuff to an application, maybe I'll check this out.

03:34 Yeah, it definitely looks cool.

03:35 You know, CLIs are great for automating things, right?

03:38 So I found a Reddit conversation.

03:41 It's not linking to an article, but it's a conversation that I thought was just outrageous for some of

03:46 the things and interesting and inspiring for others.

03:48 And it's called, how have you automated your life with Python?

03:51 So this person like throws this out as a general discussion on Reddit.

03:55 And, you know, some people come and say, you know, it is really, there's something magical

03:59 about writing code that interacts with the physical world.

04:02 And so some of them are totally straightforward.

04:04 So there's like somebody that said, hey, I take the train to work or something.

04:08 So I run a script every five minutes between five and five 30, which scrapes the train websites

04:12 and sends me a notification of my train.

04:14 It's on a delay or on time or canceled or something like that.

04:17 That's pretty cool, right?

04:18 Yeah.

04:18 Yeah.

04:18 Like to your Slack channel, like you better get going.

04:20 Your train's showing up early or it's late or whatever.

04:22 Yeah.

04:23 And then, so that was pretty standard, right?

04:24 Like there's another one that's a little more playful.

04:26 Like a person wrote a script to check if a nearby ice cream shop is stocking their favorite

04:31 flavor by screen scraping their menu.

04:33 Oh, nice.

04:34 Is my, is my ice cream in stock?

04:36 Yes.

04:37 All right.

04:37 I'm going to get some.

04:38 That person is passionate about their ice cream.

04:40 Yeah.

04:41 Well, have you been to Salt and Straw?

04:42 Yes.

04:43 I'd like something like that for Salt and Straw.

04:45 That'd be great.

04:45 Yeah.

04:45 Yeah.

04:46 For sure.

04:46 Like I wouldn't want the ice cream flavor.

04:48 I would want the line length.

04:49 Oh yeah, definitely.

04:50 Like, so those of you who are coming here for PyCon, Salt and Straw is like this, you know,

04:55 you probably heard of Voodoo Donuts, which is like a iconic donut place in Python.

04:59 People, in Portland, you'll see people with like these pink boxes all over the place.

05:03 Salt and Straw is like the ice cream equivalent.

05:05 Even in the rain, there'll be like a 30 person line outside.

05:09 It's crazy.

05:09 Yeah.

05:10 And Blue Star is better than Voodoo, but.

05:12 Whoa.

05:12 You're going to start some fights here.

05:14 Let's just move on.

05:14 So let me tell you, like, so these are all pretty standard.

05:17 Like there was somebody that said, I have a script that generates weekly status emails

05:21 off my get commit messages and creates time sheets in Harvest based on that.

05:26 Harvest is time tracking.

05:27 Like I spent three hours on this.

05:29 Wow.

05:30 That's pretty cool.

05:30 The one that like made me laugh the most.

05:33 And I don't think this is meant to be all creepy, but it could come across that way.

05:38 It's like, I recently wrote a quick Python script that tells me when my girlfriend comes

05:41 home.

05:41 Like, wait, what?

05:43 And it says it sniffs the DHCP traffic on the Wi-Fi network and looks for her Wi-Fi ID.

05:49 Interesting.

05:49 So her phone gets home, it auto joins to Wi-Fi and it sends them a note.

05:53 Like, hey, girlfriend's home.

05:54 Clean up the sink before you get yelled at.

05:56 I don't know.

05:56 Something like that.

05:57 A few weeks ago, we did one on like notifying you when your friends order pizza.

06:01 Yes, exactly.

06:01 This is kind of like that.

06:02 It's like, that's pretty interesting, but it also kind of weird.

06:05 But anyway, there's a bunch more.

06:08 I only grabbed some of them.

06:09 That's pretty interesting.

06:10 If you've automated something amazing, you know, go and stick it up there, throw it into

06:13 the chat, the discuss on this episode page.

06:17 Yeah.

06:17 Speaking of automating your life a little bit, I ran across this article and it's by the

06:24 Pi Bytes people and it's called Building a Simple Birthday App with Flask SQLAlchemy.

06:30 And the kind of the idea, which I loved behind it is they were tired of relying on Facebook

06:38 to know when their friends, when all the birthdays are coming up.

06:41 And I totally agree with that.

06:43 And also it's just kind of a fun example for how to, for building a Flask application,

06:49 which actually might be useful for people.

06:51 And I used to have a phone app that did this and I don't know what happened to that.

06:55 So I think I might actually check this out to build.

06:58 I've never played with SQLAlchemy, so maybe I'll try this out.

07:01 Yeah.

07:01 Oh, that sounds really fun.

07:02 Nice.

07:03 So yeah, yeah, that definitely is another way to automate your life.

07:06 And what we have coming up is also a, maybe an automated spelling, you might say.

07:11 But before we do, let me tell you guys about Rollbar.

07:14 As you all know, I use Rollbar for all of our sites.

07:17 Rollbar is constantly watching.

07:19 And if something crashes on the site, I get an immediate notification, maybe in Slack,

07:22 maybe my phone, all sorts of stuff.

07:25 So if you run web applications, you, or even apps that you distribute, you owe it to yourself

07:31 to work with Rollbar, right?

07:33 Go to rollbar.com slash Python bytes, and you can sign up.

07:36 It's super easy to add like to Pyramid, Flask, Django.

07:39 They have quick search for all of them.

07:40 It's like really, really new work.

07:42 And those guys are going to be at PyCon, just like Brian and me.

07:45 And they are inviting you to stop by their booth, get some swag, check them out, get a demo,

07:51 things like that.

07:51 Awesome.

07:51 Yeah.

07:52 I'll check out their booth at PyCon.

07:53 How about you?

07:53 Oh, definitely will.

07:54 I'll try to give them some of my stickers.

07:56 Yeah.

07:56 I'm definitely going to bring a sticker over there as well.

07:58 We're coming, Mike.

08:00 We're coming.

08:00 All right.

08:00 So the thing that I have next is this blog post that got sent to us.

08:06 And it's really interesting.

08:09 So at first I thought, okay, this is like kind of a somewhat interesting project that might

08:16 be worth talking about, but it's probably just something really basic, someone learning

08:19 a program.

08:20 But in fact, I think it has a lot of interesting lessons contained.

08:24 So the idea is that the article is spelling with elemental symbols.

08:29 So we know that the periodic table has a bunch of one to two character words or letters representing

08:37 elements, right?

08:39 So the idea is, can you take a regular word and spell it atomically?

08:48 So, you know, N, A, like capital N, lowercase a, capital H, right?

08:54 Something like that, right?

08:55 So can you take the stuff that's on the periodic table and make words out of it, right?

09:00 So that's basically what this does.

09:01 And it sounds kind of like, kind of interesting, but there's a couple of things that are worth

09:06 looking at.

09:07 One is you can look at a word and you can break it up into this piece and say, okay, well,

09:12 this is consists of like, maybe it's a four letter word, consists of one, one, one, one

09:19 possible letter arrangements, or it could be two, one, one, or one, one, one, two, right?

09:25 Like there's, there's different ways in which you might pull those letters into the chemical

09:29 elements, right?

09:30 But if you look at the length of a word, say there's 10 characters in a word, how many

09:36 possible combinations of that are there?

09:38 Would you guess?

09:39 Yeah, I'm really bad at math.

09:41 Don't remember.

09:41 Well, not so much the number, but the formula for it, like the concept for it.

09:47 It turns out that the number of these arrangements based on the number of characters is the Fibonacci

09:53 sequence.

09:53 Oh, wow.

09:54 Okay.

09:55 Isn't that crazy?

09:55 So if you've like a five letter word, there's whatever the sixth value of the Fibonacci numbers

10:01 is.

10:02 That's how many possible options are.

10:03 10 letters of the 11th Fibonacci and so on.

10:06 It's like N plus one, basically.

10:07 Isn't that crazy?

10:08 Yeah.

10:08 Just, I want to randomly grab stuff off the periodically table Fibonacci.

10:13 Okay.

10:13 So that's pretty interesting, but it turns out like the guy was doing a bunch of work to sort

10:18 of break these apart.

10:20 The really computational part is given a word, how many of these Fibonacci numbered partitionings

10:26 are there, right?

10:26 That's where most of the work was.

10:27 So he said, okay, well, I tried to optimize it in a couple of ways and he does a lot of

10:31 interesting performance profiling.

10:32 So you can learn how to profile your code there, which is really cool.

10:35 And it turns out, you know, it was really slow doing that.

10:39 So he's like, all right, well, what else can I do?

10:41 Maybe I can use memoization.

10:43 Have you heard of memoization?

10:45 You know what this is?

10:46 Don't quiz me on it, but yes.

10:47 Okay.

10:47 Yes.

10:48 All right.

10:49 So if you have a function that is stateless, right?

10:53 It just works on its input parameters and then it gives you back a number or a value based

10:59 solely on the input.

11:00 You can use this idea called memoization to say, basically cache the input arguments and

11:06 then the potentially expensive output.

11:09 So if you get, you know, a word of length seven, well, you know, you can, you know how, what

11:16 like the seventh Fibonacci arrangement of these things is and you don't ever have to compute

11:20 it again.

11:20 So he's like, well, let's try this to make it faster.

11:22 And it turns out that it did make it faster, but like 30%.

11:26 So it went for like whatever his data set it was.

11:27 It went from like 22 minutes to 60 minutes or, or something to this effect.

11:32 So that was better.

11:33 And then he said, all right, let's rethink this algorithm, switch to a directed acyclic

11:38 graph using recursion.

11:40 And the complexity went from O2 to the N to O of N and the time to compute all these,

11:46 these sort of weird spellings, if you will, went from 16 minutes to 10 seconds.

11:52 Wow.

11:53 So, which is really cool.

11:54 So he takes this pretty easy to understand problem and, and works through it and does profiling,

11:59 complexity analysis, memoization, recursion graphs.

12:03 And there's just a lot of stuff to learn.

12:04 And if you're kind of getting started with some of these ideas, this is a very approachable

12:09 way to, to get at some of it.

12:11 That's what I like about this love about this article is it's not, it's not really that you

12:16 really have this need for this application.

12:18 It's this kind of a problem.

12:21 And how do you approach it from a computer science standpoint to, and an engineering standpoint

12:26 of faster is better and getting down to it.

12:29 It's pretty cool.

12:29 Yeah, it is really cool.

12:30 And it's, there's something super rewarding about writing code that goes, it takes 22 minutes

12:35 to run and getting that down to 10 seconds.

12:38 Yeah.

12:38 You're just like, yes, I beat you.

12:40 I beat you problem.

12:42 Now you just got to get it to run on GPUs and get it sub, sub second.

12:46 Sub second.

12:47 Yeah.

12:47 So speaking of, helping people learn and get started, you've got something cool for us.

12:53 Yeah.

12:53 Well, actually I'm not sure if it's cool or not.

12:55 I ran across, another Reddit comment thread and this one is a discussion about

13:01 a, an IDE called, I think it's Tony Thonny, T H O N N Y.

13:08 Any guesses on how to pronounce that?

13:11 I'm going, I'm going with Thonny.

13:12 But I'm not sure.

13:13 So that there's a discussion about whether or not about the merit of this particular IDE.

13:18 And I haven't tried it.

13:19 I have to, to be real honest, but I have mixed feelings about encouraging beginner IDEs.

13:25 I guess mostly I don't know if it's a good idea or not.

13:28 I think, parting, learning how to use a decent editor is kind of part of how to learning

13:33 how to be a programmer.

13:34 So actually I was just curious about what you thought about beginner editors.

13:38 I did look at this and I also had mixed emotions about it.

13:43 It was like interesting, but I'm not entirely sure that I would recommend it.

13:48 And I, I don't really know why, but I definitely think it's, it's quite interesting to see what

13:52 it does.

13:53 So I'll just, there's, there's like a little video and stuff.

13:56 You guys can click the link and check it out.

13:57 But imagine, an IDE where you have like a Python file and you type in it and then to

14:04 the right of it, like a little tab that says, these are all the variables that are in your

14:07 code.

14:08 Right.

14:09 So that's kind of cool that you can actually see the variables that are at work while you're

14:15 writing your code.

14:16 Now, obviously we don't need that these days, but you know, when you're new, like those things

14:21 get lost.

14:22 Right.

14:22 Yeah.

14:23 And so that's really nice.

14:24 It has like a single button debugging experience.

14:27 Okay.

14:28 Which is pretty cool.

14:29 And then it has like, while you're debugging it, it has a shell and you can like type stuff

14:33 in the shell, like, like a REPL basically.

14:35 And it'll like, let you explore the state of the program, I believe, if I'm understanding

14:41 it correctly.

14:42 So those are all pretty interesting things.

14:43 Okay.

14:44 Maybe I'm changing my mind because like ID, full blown ID, like Visual Studio or something,

14:48 or even PyCharm might be a little overwhelming for somebody just starting out, but learning

14:55 the concepts that an ID might be able to give you, maybe that's a good thing.

14:58 Yeah.

14:59 Maybe it is.

14:59 Maybe it is.

15:00 I mean, one thing that's cool about it is it comes with Python 3.6 built in and it's a

15:04 Windows app.

15:04 Okay.

15:05 Right.

15:05 So you just install it and you open it up and then boom, you have Python 3.6.

15:09 You don't have to think about, oh my God, VCVars.bat.

15:12 Where's the path?

15:13 Why is this not working?

15:14 You know what I mean?

15:14 Like all that, that grief just kind of goes away and that's pretty cool.

15:17 Yeah.

15:17 It probably doesn't have Vim key bindings though, but.

15:20 No, probably just Emacs.

15:21 Probably Emacs?

15:23 You think so?

15:23 I don't know.

15:26 I don't know.

15:26 Probably none.

15:27 It's probably just a GUI thing.

15:29 So.

15:29 Yeah, probably.

15:30 But no, it looks, you know, I'm not going to do anything with it, but it definitely looks

15:34 interesting if you're trying to teach people, especially maybe kids.

15:38 I'm not entirely sure, but I think it might be good for that group.

15:41 And the idol is kind of in that same category.

15:42 But I guess I brought this up here because I'd like to hear some comments from our listeners

15:47 as to whether they encourage people to use beginner IDEs when they're teaching programming or not.

15:54 Yeah, for sure.

15:55 Leave a note on the end of the show notes, the show page, pythonbytes.fm/26 for this one.

16:01 All right.

16:02 Last thing.

16:02 I have some salvation for the PDF people out there.

16:07 So a lot, especially if you're in data science or doing web scraping and trying to get data,

16:12 often it's bundled up in these reports, which are in PDFs.

16:16 And the support for actually going into these PDFs and getting them out in Python has not been good.

16:22 It's really, you know, I've had a lot of people say, I've tried this, I've tried this, I've tried this.

16:27 None of them work.

16:28 Do you have any ideas, Michael?

16:29 And I say, nope, no ideas.

16:31 So there's recommendation on Twitter that I thought was really good from Dan Schorstein.

16:36 And he said, hey, you should check out this thing called PDF Plumber.

16:41 So you can go in and you can extract graphs.

16:43 You can debug your processing of the PDF file visually.

16:51 Oh, wow.

16:51 So if you're trying to extract a table, you can go and say, you'll highlight the thing I'm looking for like this.

16:57 And it'll literally show you a PNG with like rectangles around the pieces you're getting.

17:00 Which is pretty cool.

17:02 So you can extract a bunch of tables.

17:05 You can work with the shapes.

17:07 You can group stuff together.

17:08 Somebody did like a master's thesis on this.

17:11 And a lot of those ideas are brought into it.

17:13 So if you're out there doing data science and or something where the data is wrapped up in PDFs and you've been having a hard time getting at it, check out PDF Plumber.

17:21 Great.

17:21 Awesome.

17:22 Yeah.

17:22 Yeah.

17:22 Very good.

17:23 So that's our news for this week.

17:26 Anything you've got, what, like a day to finish the book?

17:29 How's that?

17:29 No, I had last Wednesday.

17:33 Do you have two days?

17:34 No, I turned it in already.

17:35 You did?

17:36 Yeah.

17:37 Oh, man.

17:37 Congratulations.

17:38 How does that feel?

17:39 It's not done yet.

17:40 So the first five chapters will be, five chapters?

17:44 Yeah.

17:44 The first five chapters will be released unofficially in stealth mode on the 17th.

17:50 Awesome.

17:50 And that's tomorrow.

17:51 Just in time for PyCon.

17:53 Or either that or today or yesterday, depending on when people get this.

17:58 It could have been three weeks ago if you listened to this very lately.

18:00 Yeah.

18:02 Awesome.

18:03 But definitely by the time I'm hanging out at PyCon, it'll be available.

18:07 That's cool.

18:07 Official release is supposed to be next.

18:11 So this is the Python testing with pytest.

18:14 Even as is, the first five chapters are like 10 times better than anything I've ever read before.

18:19 Oh, that's awesome.

18:20 Yeah.

18:21 Congrats.

18:22 How about you?

18:22 What's going on?

18:23 Well, I am really looking forward to being at PyCon.

18:26 I'm going to be there Thursday night.

18:28 We'll have our booth there.

18:29 And Thursday, Friday, Saturday, Sunday, we'll have our booth set up.

18:33 People come hang out with us, come talk to us.

18:35 And we'll be doing some open sessions and things like that.

18:37 So it'll be a lot of fun.

18:38 Yeah.

18:39 So I'm definitely looking forward to that, being there with you and all the other podcasters.

18:43 And I actually just finished recording my RESTful HTTP services in Pyramid course.

18:49 And that's all totally done.

18:51 I'm just waiting on the final edits to publish that to the world.

18:53 And I'm now working on my MongoDB for Python developers course.

18:57 So I'm frantically writing as fast as I can before PyCon overwhelms me.

19:02 Oh, nice.

19:02 Okay.

19:03 Yeah.

19:03 Those are both going to be fun.

19:04 Great.

19:05 Cool.

19:06 All right.

19:07 Well, that's all I got.

19:08 That's about it for me.

19:08 Yeah.

19:09 Yeah.

19:09 Same here, man.

19:11 Well, welcome back to the US.

19:12 And I will see you and everyone else at PyCon.

19:15 Yeah.

19:15 See you in a couple of days.

19:16 All right.

19:16 Bye.

19:17 Bye.

19:17 Thank you for listening to Python Bytes.

19:21 Follow the show on Twitter via at Python Bytes.

19:23 That's Python Bytes as in B-Y-T-E-S.

19:26 And get the full show notes at Python Bytes.fm.

19:29 If you have a news item you want featured, just visit Python Bytes.fm and send it our way.

19:34 We're always on the lookout for sharing something cool.

19:36 On behalf of myself and Brian Okken, this is Michael Kennedy.

19:40 Thank you for listening and sharing this podcast with your friends and colleagues.