Transcript #26: How have you automated your life, or CLI, with Python?

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:06 This is episode 26, recorded May 16, 2017.

00:11 I'm Michael Kennedy.

00:12 And I'm Brian Okken.

00:13 And we're here to deliver a bunch of cool news around the Python community.

00:17 But before we do, I'd like to say thanks, Brian.

00:19 I'd like to say thanks to Rollbar, because once again, they're sponsoring this episode.

00:23 Yay, Rollbar!

00:24 Yay, Rollbar! So they're going to be at PyCon, so be sure to drop by their booth and tell them that we sent you.

00:29 All right, let's talk about the terminal.

00:31 Yeah, when I think of command line interface, I mean, I use like the REPL and other terminal interfaces.

00:38 But most of the time when I'm writing applications that I think of as command line, they just have, you can launch them from the command line with flags and stuff like that.

00:48 But if you wanted to do an interactive terminal application like a REPL or something, I don't know how to do that.

00:58 is I ran across this two-part series article a two-part series and the first part I was just actually talking about and this is on open source.com and the first article talks about four different applications that have good interface characteristics and started with the MySQL interface and the Python REPL which were good but both could be better and the author talked about BPython as a as a adds auto completion to Python.

01:28 - Oh, BPython, interesting.

01:30 I've never heard of BPython.

01:32 I use PTPython.

01:33 - Well, the article also mentions PTPython.

01:35 So I was gonna try both of them out and see which one I liked better.

01:38 - Oh, okay.

01:39 You'll have to give us a report.

01:40 - Yeah.

01:41 There's a, apparently for MySQL, there's a MyCLI that adds context aware completion.

01:48 I don't spend enough time in a MySQL to worry about that.

01:52 But then a couple others, If you're a Postgres person, there's somebody, something called PGCLI that adds fuzzy search.

02:01 And then you talked about bash, like we spend our time on the bash command line and some of the, one replacement is called phish that supposedly has better search history.

02:10 - Oh, nice.

02:11 Yeah, that's cool.

02:12 I've heard phish is good.

02:13 I use omyssh.

02:13 And also conch is a Python based one, which is cool as well for the shell.

02:20 - Yeah.

02:20 So that was the first part.

02:22 And the second part of the series talks about how to do this sort of stuff with Python.

02:27 So he talks about actually taking four different libraries.

02:31 There's one called the prompt toolkit that you can use that to add a REPL-like interface to your application.

02:41 And you can add things like command history, and auto suggestion, and auto completion, which sounds cool, actually.

02:47 Yeah, that actually does sound pretty cool.

02:49 Click, which I'm familiar with using Click for options and flags.

02:56 But apparently it also has a pager and the ability to launch an editor outside of it, which would be cool.

03:04 That sounds cool.

03:05 Yeah, very cool.

03:06 And then a couple more pigments for syntax highlighting and Fuzzy Finder, which is a way to make fuzzy suggestions on whatever somebody types in.

03:16 That's cool.

03:17 The nice thing is it talks, there's an example on how to include to hook up Fuzzy Finder and prompt toolkit to make the completion look nice.

03:28 This is a well written series and when I add interactive stuff to an application, maybe I'll check this out.

03:34 Yeah, it definitely looks cool.

03:36 You know, CLIs are great for automating things, right?

03:39 So I found a Reddit conversation, it's not linking to an article, but it's a conversation I thought was just outrageous for some of the things and interesting and inspiring for others.

03:49 And it's called how have you automated your life with Python.

03:52 So this person like throws this out as a general discussion on Reddit.

03:55 And, you know, some people come and say, you know, it is really there's something magical about writing code that interacts with the physical world.

04:03 And so some of them are totally straightforward.

04:05 So there's like somebody that said, Hey, I take the train to work or something.

04:08 So I run a script every five minutes between five and 530, which scrapes the train websites and sends me a notification if my train is on a delay or on time or canceled or something like that.

04:17 That's pretty cool, right?

04:18 Yeah.

04:19 Yeah, like to your Slack channel, like you better get going.

04:20 Your train's showing up early or it's late or whatever.

04:23 Yeah.

04:24 And then, so that was pretty standard, right?

04:25 Like there's another one that's a little more playful, like a person wrote a script to check if a nearby ice cream shop is stocking their favorite flavor by screen scraping their menu.

04:33 Oh, nice.

04:34 Is my, is my ice cream in stock?

04:36 Yes.

04:37 All right.

04:38 I'm going to get some.

04:39 That person is passionate about their ice cream.

04:40 Yeah.

04:41 - Have you been to Salt and Straw?

04:43 - Yes.

04:43 - I'd like something like that for Salt and Straw.

04:45 That'd be great.

04:46 - Yeah, yeah, for sure.

04:47 Like I wouldn't want the ice cream flavor.

04:48 I would want the line length.

04:49 - Oh yeah, definitely.

04:51 - Like, so those of you who are coming here for PyCon, Salt and Straw is like this, you know, you probably heard of Voodoo Donuts, which is like a iconic donut place in Python, people, in Python, in Portland.

05:01 You'll see people with like these pink boxes all over the place.

05:04 Salt and Straw is like the ice cream equivalent.

05:06 Even in the rain, there'll be like a 30 person line outside.

05:09 It's crazy.

05:10 And BlueStar is better than Voodoo.

05:12 - Whoa, you're gonna start some fights here.

05:14 Let's just move on.

05:15 So let me tell you, so these are all pretty standard.

05:17 Like there was somebody that said, I have a script that generates weekly status emails off my git commit messages, and creates timesheets in Harvest based on that.

05:26 Harvest is time tracking, like I spent three hours on that.

05:29 - Wow. - That's pretty cool.

05:31 The one that made me laugh the most, and I don't think this is meant to be all creepy, but it could come across that way, Like I recently wrote a quick Python script that tells me when my girlfriend comes home.

05:41 Like, wait, what?

05:42 And it says it sniffs the DHCP traffic on the Wi-Fi network and looks for her Wi-Fi ID.

05:49 Interesting.

05:50 So her phone gets home, it auto joins to Wi-Fi and it sends them a note like, "Hey, girlfriend's home. Clean up the sink before you get yelled at." I don't know, something like that.

05:57 A few weeks ago, we did one on like notifying you when your friends order pizza.

06:01 Yes, exactly. This is kind of like that. It's like that's pretty interesting, but it's also kind of weird.

06:07 But anyway, there's a bunch more. I only grabbed some of them. That's pretty interesting. If you've automated something amazing, you know, go and stick it up there, throw it into the chat, the discuss on this episode page. Yeah. Speaking of automating your life a little bit, I ran across this article and it's by the Pi Bytes people. And it's called "Building a Simple Birthday App with Flask SQ Alchemy." And the kind of the idea, which I loved behind it is they were tired of relying on on Facebook to know when their friends when all the birthdays are coming up. And I totally agree with that. And also, it's a just kind of a fun example for how to for building a flask application, which actually might be useful for people. And I used to have a phone app that did this. And I don't know what happened to that. So I think I might actually check this out to build.

06:58 I've never played with SQL comey. So maybe I'll try this out.

07:01 Yeah, that sounds really fun. Nice. So yeah, yeah, that definitely it's another way to automate your life. And what we have coming up is also a maybe an automated spelling, you might say.

07:12 But before we do, let me tell you guys about rollbar. As you all know, I use rollbar for all of our sites. rollbar is constantly watching. And if something crashes on the site, I get an immediate notification, maybe in Slack, maybe my phone, all sorts of stuff. So if you run web applications, you or even apps that you distribute, you owe it to yourself to work with rollbar right go to robot.com slash Python bytes and you can sign up it's super easy to add like the pyramid flask Django they have quick search for all of them it's like really really no work.

07:42 Those guys are going to be at PyCon just like Brian and me and they are inviting you to stop by their booth, get some swag check them out, get a demo things like that.

07:51 Awesome.

07:52 Yeah, I'll check out their booth at PyCon.

07:53 How about you?

07:54 Oh, definitely will.

07:55 I'll try to give them some of my stickers.

07:56 Yeah, I'm definitely gonna bring a sticker over there as well.

07:59 We're coming Mike we're coming.

08:00 All right, so the thing that I have next is this blog post that got sent to us.

08:07 And it's really interesting.

08:09 So at first I thought, okay, this is like kind of a somewhat interesting project that might be worth talking about, but it's probably just something really basic, someone learning a program.

08:20 But in fact, I think it has a lot of interesting lessons contained within it.

08:25 The idea is that the article is spelling with elemental symbols.

08:31 We know that the periodic table has a bunch of one to two character words or letters representing elements, like Na for sodium, H for hydrogen, and so on.

08:44 The idea is, can you take a regular word and spell it atomically?

08:50 You know, N A, like capital N, lowercase A, capital H, right?

08:55 Something like that, right?

08:56 So can you take the stuff that's on the periodic table and make words out of it, right?

09:00 So that's basically what this does.

09:01 And it sounds kind of like, kind of interesting, but there's a couple of things that are worth looking at.

09:08 One is you can look at a word and you can break it up into this piece and say, okay, well, this is consist of like, maybe it's a four letter word, consists of one, one, one possible letter arrangements or it could be two one one or one one one two, right?

09:25 Like there's there's different ways in which you might pull those letters into the chemical elements, right?

09:31 But if you look at the length of a word, say there's 10 characters in a word, how many possible combinations of that are there?

09:39 Would you guess?

09:40 Yeah, I'm really bad at math.

09:41 Don't know.

09:42 Well, not so much the number but the formula for it like the the concept for it.

09:47 It turns out that the number of these arrangements based on the number of characters is the Fibonacci sequence.

09:54 Oh, wow.

09:55 Okay.

09:56 Isn't that crazy?

09:57 So if you've like a five letter word, there's whatever the sixth value of the Fibonacci numbers is, that's how many possible options are.

10:04 Ten letters of the 11th Fibonacci and so on.

10:06 It's like N plus one, basically.

10:07 Isn't that crazy?

10:08 Yeah.

10:09 Just I want to randomly grab stuff off the periodically table Fibonacci.

10:12 Okay.

10:13 So that's pretty interesting.

10:14 But it turns out, the guy was doing a bunch of work to break these apart.

10:20 The really computational part is, given a word, how many of these Fibonacci numbered partitionings are there?

10:26 That's where most of the work was.

10:27 So he said, "Okay, well, I tried to optimize it in a couple ways." And he does a lot of interesting performance profiling, so you can learn how to profile your code there, which is really cool.

10:36 And it turns out it was really slow doing that.

10:39 So he's like, "All right, well, what else can I do?

10:42 Maybe I can use memoization.

10:44 Have you heard of memoization?

10:45 You know what this is?

10:46 Don't quiz me on it, but yes.

10:47 Okay.

10:48 Yes.

10:49 All right.

10:50 So if you have a function that is stateless, right, it just works on its input parameters and then it gives you back a number or a value based solely on the input, you can use this idea called memoization to say basically cache the input arguments and then the potentially expensive output.

11:09 So if you get a word of length 7, well, you know how the 7th Fibonacci arrangement of these things is, and you don't ever have to compute it again.

11:20 So he's like, "Well, let's try this to make it faster." And it turns out that it did make it faster, but like 30%.

11:25 So it went for, like, whatever his data set it was, it went from like 22 minutes to 60 minutes or something to this effect.

11:32 So that was better.

11:33 Then he said, "All right, let's rethink this algorithm.

11:36 Switch to a directed acyclic graph using recursion." The complexity went from O2 to the N to O of N.

11:45 The time to compute all these weird spellings, if you will, went from 16 minutes to 10 seconds.

11:53 Which is really cool.

11:54 He takes this pretty easy to understand problem and works through it and does profiling, complexity analysis, memoization, recursion, graphs, and there's just a lot of stuff to learn.

12:05 If you're kind of getting started with some of these ideas, this is a very approachable way to get at some of it.

12:11 - That's what I like about this, love about this article, is it's not really that you really have this need for this application.

12:19 It's this kind of a problem, and how do you approach it from a computer science standpoint, and an engineering standpoint of faster is better, and getting down to it.

12:29 It's pretty cool.

12:30 - That's really cool.

12:31 And there's something super rewarding about writing code that goes, takes 22 minutes to run, and getting that down to 10 seconds.

12:38 - Yeah.

12:39 - You're just like, yes, I beat you.

12:41 I beat you, problem.

12:42 Now you just gotta get it to run on GPUs and get it sub-second.

12:46 - Sub-second, yeah.

12:47 - So speaking of helping people learn and get started, you've got something cool for us.

12:53 - Yeah, well actually, I'm not sure if it's cool or not.

12:56 I ran across another Reddit comment thread, And this one is a discussion about an IDE called, I think it's Tony, Thony, I don't know, T-H-O-N-N-Y.

13:09 Any guesses on how to pronounce that?

13:11 - I'm going with Thony.

13:12 - But I'm not sure.

13:13 So there's a discussion about whether or not about the merit of this particular IDE, and I haven't tried it, I have to be real honest.

13:21 But I have mixed feelings about encouraging beginner IDEs.

13:25 I guess mostly I don't know if it's a good idea or not.

13:28 I think learning how to use a decent editor is kind of part of learning how to be a programmer.

13:35 So actually, I was just curious about what you thought about beginner editors.

13:38 I did look at this and I also had mixed emotions about it.

13:43 It was like interesting, but I'm not entirely sure that I would recommend it.

13:48 I don't really know why, but I definitely think it's quite interesting to see what it does.

13:53 So there's like a little video and stuff.

13:56 guys can click the link and check it out.

13:58 But imagine an ID where you have like a Python file and you type in it.

14:03 And then to the right of it, like a little tab that says these are all the variables that are in your code.

14:08 Right?

14:09 So that's kind of cool that you can actually see the variables that are at work while you're writing your code.

14:16 Now, obviously, we don't need that these days.

14:19 But you know, when you're new, like those things get lost, right?

14:22 Yeah.

14:23 And so that's really nice.

14:24 It has like a single button debugging experience, which is pretty cool.

14:29 And then it has like while you're debugging, it has a shell, and you can like type stuff in the shell, like a REPL basically.

14:37 And it'll like let you explore the state of the program, I believe, if I'm understanding it correctly.

14:42 So those are all pretty interesting things.

14:43 Okay, maybe I'm changing my mind because like full blown ID like Visual Studio or something or even PyCharm might be a little overwhelming for somebody just starting out.

14:54 But learning the concepts that an ID might be able to give you, maybe that's a good thing.

14:59 Yeah, maybe it is.

15:00 Maybe it is.

15:01 I mean, one thing that's cool about it is it comes with Python 3.6 built in and it's a Windows app.

15:05 Okay.

15:06 Right?

15:07 So you just install it and you open it up and then boom, you have Python 3.6.

15:09 You don't have to think about, oh my God, vcvars.bat, where's the path?

15:13 Why is this not working?

15:14 You know what I mean?

15:15 Like all that grief just kind of goes away and that's pretty cool.

15:17 Yeah, probably doesn't have them Vim key bindings though.

15:20 No, probably just Emacs.

15:22 Probably Emacs, you think so?

15:26 I don't know, I don't know.

15:27 Probably none.

15:27 It's probably just a GUI thing.

15:29 Yeah, probably.

15:30 But no, it looks--

15:32 I'm not going to do anything with it, but it definitely looks interesting if you're trying to teach people, especially maybe kids.

15:38 I'm not entirely sure, but I think it might be good for that group.

15:41 And IDLE is kind of in that same category.

15:43 But I guess I brought this up here because I'd like to hear some comments from our listeners as to whether they encourage people to use beginner IDEs when they're teaching programming or not.

15:54 Yeah, for sure.

15:55 Leave a note on the end of the show notes, the show page, pythonbytes.fm/26 for this one.

16:02 All right, last thing.

16:03 I have some salvation for the PDF people out there.

16:08 So a lot, especially if you're in data science or doing a web scrape, you're trying to get data often it's it's bundled up in these reports, which are in PDFs.

16:17 And the support for actually going into these PDFs and getting them out in Python has not been good.

16:23 It's really, you know, I've had a lot of people say I've tried this, I've tried this, I've tried this, none of them work.

16:28 Do you have any ideas, Michael?

16:29 And I say, Nope, no ideas.

16:31 So there's recommendation on Twitter that I thought was really good from Dan Shorstein.

16:38 he said, "Hey, you should check out this thing called PDF Plumber." So you can go in and you can extract graphs, you can debug your processing of the PDF file visually.

16:51 Oh, wow.

16:52 So if you're trying to extract a table, you can go and say, you'll highlight the thing I'm looking for like this, and it'll literally show you a PNG with like rectangles around the pieces you're getting, which is pretty cool.

17:03 So you can extract a bunch of tables, you can work with the shapes, you can group stuff together.

17:09 Somebody did like a master's thesis on this, and a lot of those ideas are brought into it.

17:14 So if you're out there doing data science and or something where the data is wrapped up in PDFs, and you've been having a hard time getting at check out PDF plumber.

17:21 Great.

17:22 Awesome.

17:23 Yeah, yeah.

17:24 Very good.

17:25 So that's our news for for this week.

17:26 Anything you've got what, like a day to finish the book?

17:29 How's that?

17:30 No, I had last Wednesday.

17:33 You have two days.

17:34 No, I turned it in already.

17:36 You did?

17:37 Yeah.

17:38 Oh man, congratulations.

17:39 How does that feel?

17:40 It's not done yet.

17:41 So, the first five chapters will be five chapters.

17:44 Yeah, the first five chapters will be released unofficially in stealth mode on the 17th.

17:50 Awesome.

17:51 And that's tomorrow.

17:52 Just in time for PyCon.

17:53 Or either that or today or yesterday, depending on when people get this.

17:58 It could have been three weeks ago if you listen to this very lately.

18:02 Yeah, but definitely by the time I'm hanging out at PyCon, it'll be available.

18:07 That's cool.

18:08 So official release is supposed to be next.

18:11 So this is the Python testing with pytest.

18:15 Even as is, first five chapters are like 10 times better than anything I've ever read before.

18:19 So awesome.

18:20 Oh, that's awesome.

18:21 Yeah.

18:22 Congrats.

18:23 How about you?

18:24 What's going on?

18:25 Well, I am really looking forward to being at PyCon.

18:26 I'm going to be there Thursday night, we'll have our booth there.

18:30 And Thursday, Friday, Saturday, Sunday, we'll have our booth set up, people come hang out with us, come talk to us, and we'll be doing some open sessions and things like that.

18:37 So it'll be a lot of fun.

18:38 Yeah.

18:39 So I'm definitely looking forward to that, being there with you and all the other podcasters.

18:43 And I actually just finished recording my RESTful HTTP services and pyramid course.

18:50 That's all totally done.

18:51 I'm just waiting on the final edits to publish that to the world.

18:53 I'm now working on my MongoDB for Python developers course.

18:57 So I'm frantically writing as fast as I can before PyCon overwhelms me.

19:02 Oh, nice.

19:03 Okay.

19:04 Yeah, those are both going to be fun.

19:05 Great.

19:06 Cool.

19:07 All right.

19:08 Well, that's all I got.

19:09 That's about it for me.

19:10 Yeah.

19:11 Stay here, man.

19:12 Well, welcome back to the US and I will see you and everyone else at PyCon.

19:15 Yeah, see you in a couple of days.

19:16 All right.

19:17 Bye.

19:18 Bye.

19:19 Thank you for listening to Python Bytes.

19:21 Follow the show on Twitter via @pythonbytes, that's Python Bytes as in B-Y-T-E-S.

19:27 And get the full show notes at PythonBytes.fm.

19:30 If you have a news item you want featured, just visit PythonBytes.fm and send it our way.

19:35 We're always on the lookout for sharing something cool.

19:37 On behalf of myself and Brian Auchin, this is Michael Kennedy.

19:41 Thank you for listening and sharing this podcast with your friends and colleagues.