Transcript #13: Python making the move to GitHub and Dropbox is stepping back from Pyston

00:00 This is Python Bytes, Python headlines and news delivered directly to your earbud.

00:05 It's episode 13 on Monday the 13th.

00:08 Oh my God, recorded February 13th, 2017.

00:11 Hey Brian, I want to start by saying thanks and hello.

00:14 I want to say thanks to a new sponsor of the show.

00:18 We have a new sponsor. That's wonderful.

00:19 We do. And I want to say thank you to Metis.

00:21 They're a data science training company.

00:23 They have some boot camp type stuff that's really excellent.

00:26 And I'll tell everyone about that later.

00:28 I kind of like to learn about alternate Python implementations and where they are these days.

00:34 We had heard about Piston, which is spelled P-Y-S-T-O-N, developed by Dropbox, and an announcement came out at the end of last month saying that Piston is no longer to be supported by Dropbox.

00:48 Yeah, so this is a really interesting development.

00:52 It's a little unfortunate.

00:54 I don't really know how to interpret as a positive or negative.

00:58 I mean, it could be just...

00:59 There was an experiment.

01:00 We're moving on to the next experiment.

01:01 But what it was is an alternate version implementation.

01:06 I have a hard time calling it an interpreter when it's actually JIT compiled, but alternate interpreter for Python.

01:12 And the idea was it supported Python 2.7, and it did JIT compilation instead of interpreting.

01:20 So its main goal was to be a very fast but highly compatible alternate implementation of Python 2.7.

01:29 Yeah, and so there's a couple blog posts, one on the Piston blog and one by Kevin Modzelewski and one on his own site kevmod.com.

01:39 And it looks like there's lots of reasons around it, but the general flow of it is that the cost and benefit. It cost more than they were hoping and it benefited them not as much as they were hoping. Yeah, so we look at the different implementations of Python. We've got obviously CPython and its versions but we've got Piston, we've got Unladen Swallow, we've got Cython, we've got PyPy, PyPy, IronPython, Jython. There's a bunch of different ones and they all have these trade offs. And so they were actually able to make piston pretty quick in the the benchmark story. So they got it up to 95% faster than CPython two seven. But they also ran into other other drawbacks. And they found like actually when it ran the Dropbox code itself, it was only 10% faster, but it put a lot more memory pressure on it because it does GC instead of reference counting.

02:38 I've got mixed feelings about it. I like that the project was out in the open, open source, and people could watch this experiment. I wasn't really thrilled that they were targeting 2.7 to start with. The outcome of them just, I guess, not working on it anymore. It totally makes sense, but it is, I guess, it's just a thing. It's interesting.

02:59 Yeah, I think it's a case of a couple of things. I mean, nobody said it, but I feel like, you know, Moving to Python 3 is also causing a rethinking of this as well.

03:09 If they're going to be moving to Python 3, then they'd have to redo this work and so on.

03:12 One of the problems or criticisms, mild criticisms, let's say, was that this was a from scratch thing. It didn't build on existing stuff. It didn't build on PyPy because PyPy wants to be fast at the cost of compatibility and to run a multi-million line Python application that is this Dropbox compatibility is like super, super critical.

03:33 So they're like, well, we can't have any incompatibility, but it wasn't directly based on CPython, even though it took a bunch of pieces in.

03:41 And so, I don't know, it was like this sort of other experiment where the thing that gives me the most hope right now is what Brett Cannon and Dino Veland are working on with Pigeon, P-Y-G-I-O-N at Microsoft, where they're trying to add like JIT plugin capabilities to the existing CPython.

03:58 Yeah, there is some more CPython news.

04:01 There's big CPython news.

04:03 CPython has just as of three days ago moved.

04:09 It used to be long, long ago in 2006, it moved to Subversion, SVN.

04:16 And from 2006 to 2011, it was on Subversion.

04:19 Then it moved to Mercurial.

04:21 What's kind of interesting because that's when Git came out in Mercurial.

04:24 They both, all this whole distributed version control system story kind of happened around then. And Mercurial is interesting because it's written in Python, like, for example, Facebook does some really interesting things with Mercurial. And so it, it seemed like a natural home for Python to live on Mercurial. But, you know, GitHub has really been the place where people go to do open source, not just a container for open source stuff, like source for juice to be or something. But there's like a whole social ecosystem, where everybody comes and collaborates in GitHub.

04:58 And so the big news is, as of Friday, Python, CPython, and a few of its pieces have moved to GitHub.

05:06 I think that's great.

05:07 And I think that one of the great things about it is the culture around GitHub.

05:11 Whether you're a pro GitHub person or not, that's where a lot of people look for opportunities to help out in open source projects.

05:17 It's a higher incentive for people to get involved with helping to maintain CPython.

05:22 But yeah, I think that that's actually the main motivation, which is really interesting.

05:25 So there's a link to the mailing list where they talk about it.

05:29 We'll put that in the show notes.

05:30 There's obviously the link to the repository in GitHub.

05:32 There's a Reddit thread where people are talking about this.

05:36 And a lot of people are bemoaning the fact that it's it's not on something totally open source.

05:42 It's not on something that is based on Python, like it used to be on a source control system built on top of Python.

05:50 And while that's cool, like you shouldn't, my opinion is you shouldn't do that at the cost of people loving to work on the project.

05:58 So the real thing that you want to read to get the deep insight on this is at snarky.ca.

06:04 So this is Brett Cannon's blog, and it's the history behind the decision to move Python to GitHub.

06:09 And he talks about how he worked with a bunch of people to come to this decision.

06:14 And he says, you know, back in 2007, it became really obvious that working on the source code and doing code reviews and collaboration was actually a burden.

06:24 There were people submitting, you know, patches and stuff, and they would sit there for maybe years before somebody actually reviewed it.

06:33 Like how excited would you be to work on a project to contribute to open source if you submit a patch and it doesn't get looked at for a year, right?

06:40 And so the workflow on GitHub is just so much easier for pull requests and reviewing it.

06:47 And so basically Brett said, like, I want it to be so easy that I could take my tablet and sit on the beach where I have some kind of internet connectivity and review pull requests and chill out.

06:57 Like that's just not how it works now.

06:58 Like you got to have a machine with SSH keys and all sorts of stuff.

07:00 So the main driver behind this is trying to make it easier for core developers to be more responsive to people who contribute to the project, which I think is super positive.

07:10 Yeah, I think it's, I'm really excited about it.

07:12 And actually, I had never really considered contributing myself, but I definitely consider it more now.

07:18 Yeah, I definitely consider it more accessible now.

07:21 And I think this is absolutely the right move, even if it's breaking some conventions, like it's moving off of Python as the thing that supports it.

07:28 It's moving off a fully, fully open source thing to like this somewhat closed thing, which is GitHub.

07:35 But still, I think it's great.

07:37 So remember last time when we talked about the top 10 Python articles of 2016?

07:44 Yes.

07:45 Yeah.

07:46 So I said, let's just take a survey of the kind of category of these things.

07:49 And I would say like a full two thirds or more of those were data science.

07:55 Data science is definitely important in Python.

07:57 So I want to tell you about our sponsor, Metis.

07:59 So Metis is a data science training company based in New York, San Francisco, Chicago and Seattle.

08:04 And they provide full-time immersive boot camps and evening courses that you can come to.

08:10 If you have a full-time job, you can go like two nights a week for six weeks and take classes on all sorts of data science things, machine learning and things like that.

08:19 If you are wanting to learn more about data science, especially in a hands-on way, you live in New York or San Francisco, are you willing to spend some time there?

08:29 out thisismetis.com/talkpython and maybe we'll feature your article next year.

08:35 Yeah, that's actually sounds pretty interesting.

08:37 Yeah, I would love to go do more data science.

08:40 So TensorFlow, they've got classes on that all sorts of cool stuff.

08:43 So check it out thisismetis.com/talkpython.

08:46 All right, you know what they do a lot of in data science, a lot of functional programming, a lot of expressions, things like that, right?

08:53 Yeah, actually, I don't know.

08:55 I'm not a data science person.

08:58 The ones I've hung around seem to.

08:59 - There's an article called Using Functional Programming in Python Like a Boss, Generators, Iterators, and Decorators.

09:05 And these are generators and, okay, all three of those, generators, iterators, and decorators are things that trip up intermediate Python people.

09:13 Beginner people, just ignore 'em for now.

09:16 But this article is a Jupyter notebook, which I like this trend of people just publishing Jupyter notebooks as articles.

09:26 It actually is good.

09:28 And there's some good diagrams in here, although the use of really large images, it's kind of annoying to me, but it's okay.

09:37 Anyway, this article talks about, it also talks about just basically what a function is and what things can act like functions.

09:47 And then talks about generators, iterators, and containers and how they all work together to make your code more expressive.

09:54 And it seems like kind of an esoteric sort of thing to think about, but I think it's good.

09:59 And this is a good, I like this article about thinking about this stuff.

10:02 >> Yeah, I really think this is one of those things that when you come to Python from other languages, it takes you a while to get into these, right?

10:10 Like the concept of a generator is quite different to somebody coming from C or I think Java.

10:17 I don't think Java has this, things like that.

10:20 But once you get the zen of them, they're amazing.

10:23 and they make such a big difference, but it's really a nice mental shift.

10:29 Yeah, so I really think it's great that he's covering it.

10:32 - Also one of those topics that I don't think I remember when I finally grokked what people were talking about, but it took like three or four different articles talking about it for it to sink in for me.

10:43 So here's another one.

10:44 - Yeah, excellent, yeah, so we'll link to that one.

10:45 And by some strange coincidence, just so everybody knows, the way that we pick things for the show is I go find a couple of interesting things, Brian goes and finds a couple of interesting things, we throw them together, and then we just talk about them.

10:59 And it happens to be the very next one is the same person in a Jupyter notebook, but it's actually a totally different topic, but it's presented in a similar way.

11:08 So this one is called, It's Meta Classes All the Way Down.

11:12 This is a quick but fairly deep dive into what metaprogramming is.

11:18 And it starts out by saying, look, metaprogramming is a technique for writing computer programs that can treat themselves as data.

11:24 So you can enter spanked, generate, and or modify them while they're running.

11:28 And that's definitely Python.

11:29 You can at runtime create types.

11:33 You can add functions or even remove features from classes and objects, right?

11:39 But mostly metaprogramming comes down to metaclasses where you can actually control what the class does.

11:48 So there's all sorts of interesting examples of how you might use them.

11:51 Starts out with an interesting quote saying, "Look, metaclasses are deeper magic "than 99% of the users should ever worry about.

12:00 "If you wonder whether you need them, you don't." (both laughing)

12:04 So if you don't, like that was a quote by Ted Peters.

12:06 If you don't deeply understand like, "Yeah, this is totally, I'm gonna solve this "with a metaclass, like you probably shouldn't mess with it." But they are the power that makes a lot of the magic inside Python happen.

12:17 So a couple of examples.

12:19 One is Django ORM.

12:20 we could substitute SQLAlchemy ORM or PWE ORM models, right?

12:24 So when you define like a model in Django or SQLAlchemy, you put what looks like column types with database constraints as the fields of the class.

12:32 But then when you interact with them at runtime, they look like strings and integers and dates, right?

12:36 And that transition, that sort of two-faced behavior of these things that they can be used in a database query in one way, but then act as normal objects as others, that's metaclasses.

12:46 Another example, there's a cool example about using them to make abstract classes and abstract functions that have to be implemented by things that inherit it and so on.

12:55 Again, this is a nice Jupyter notebook, it has some cool pictures, but it really does help you understand this stuff pretty well with nice examples.

13:03 - Yeah, yeah, I like this too.

13:05 It's funny that we both picked the same person's article.

13:07 - It's a small world.

13:08 - Yeah, I think we're kind of going on the stuff that most people don't need train right now, but next up is lambdas.

13:18 And speaking of another thing that I think newbies get a little confused by, but there are lambda functions.

13:27 And really, lambda functions are just unnamed functions that you can put the entire expression on one line, and so they're convenient in a lot of places.

13:39 So here's an article from Dan Bader titled Lambda Functions in Python, What Are They Good For?

13:45 And what I like about this article is it's just a fairly quick jump into it.

13:50 I think everybody that thinks of themselves as a little bit more than just a beginner in Python should at least understand Lambda functions.

13:57 And the thing I really like about this article is it talks about places to not use them.

14:03 Because when you finally get your head around Lambdas, all of a sudden it's your hammer and everything looks like a nail.

14:11 He talks about a couple places like creating class methods and other places where that maybe that you shouldn't use a lambda for that you can but don't.

14:21 And then also there's some places where list comprehensions and dictionary comprehensions are are wonderful and they're way more expressive but you could use lambdas in those cases.

14:32 Yeah, absolutely.

14:33 I think the trick with lambdas that make them so wonderful is like there's little places in your code, you're like, if I could just pass a behavior here, I don't want to go out of where I am and write a function and then like give it a name and then say the name here I just want to go.

14:47 Yeah, sort by this reversed or her, you know, grab this value out and transform it or something.

14:54 And lambda functions are beautiful for that.

14:57 They're not so beautiful for writing your whole program.

14:59 Yeah, actually, sort is one of those.

15:02 One of those places where one of the parameters is you can pass in a key, or how to find the key. And it's, I've used lambdas a lot there because I don't really need a named function. I just need a function to pass in and do the work. Yeah, I've got a list of customers I want to sort by last name. So lambda C goes to C dot last name. Done. Beautiful. Yeah, it's great. So you wanted to do a quick follow-up on Gepronto. That's our super super fast web framework, right? Theoretically. Yep. Yeah, that's what we were talking about that last week. But I ran across a Reddit thread that we'll post in the show notes just in it is titled "Sticking with Flask versus Switching to one of the new faster frameworks" and the Gepronto author actually posted on this and said yeah if you're looking at Gepronto please read the readme there because I've noted that it's not ready for production basically. So there's he's not even he's something about I'll just read it he's a state in the readme you should not build anything serious with Gibranta now because it's gonna probably eat your laundry. Remember it's hand-coded in C and this needs a lot of testing. On top of that I plan to do several iterations of API changes in a largely incompatible way. I hope though it's gonna make a serious player in the field one day. I applaud the author for putting that out there. He's basically developing in the open, but he's not promising to stick with the same model or the same API and I think that's completely fair and he's pretty open about it. Yeah, that's really great of him and it's, you know, he seriously is optimizing this stuff in some really interesting but out there ways and so and all that part is done in C, right? And so it probably needs a little bit of careful validation before it becomes a thing that you really depend on for your e-commerce site or whatever.

17:02 But it's definitely, it's got so much promise, that project.

17:05 So I hope he gets there.

17:06 Oh, we're already done.

17:07 So there we go.

17:09 That's it.

17:10 That's the news we got for this week.

17:12 Anything you want to share with everyone while we're wrapping things up, Brian?

17:15 No, nothing right now.

17:16 All right.

17:17 Awesome.

17:18 Well, I'm cranking out some classes.

17:20 I just am pretty much finished up my Consuming Restful Services course.

17:25 So stay tight, hang tight for that.

17:27 going to be out in maybe a week or so.

17:30 >> All right.

17:31 Cool.

17:32 >> Yeah.

17:33 Fun to be building new things for everyone.

17:34 All right.

17:35 Well, thank you everyone for listening.

17:36 Brian, thanks for chatting with me today.

17:39 >> Thank you.

17:40 >> Yep.

17:41 See you all later.

17:42 Thank you for listening to Python Bytes.

17:44 Follow the show on Twitter via @pythonbytes.

17:46 That's Python Bytes as in B-Y-T-E-S.

17:49 And get the full show notes at pythonbytes.fm.

17:52 If you have a news item you want featured, just visit pythonbytes.fm and send it our way.

17:57 a lookout for sharing something cool.

18:00 On behalf of myself and Brian Okken, this is Michael Kennedy.

18:03 Thank you for listening and sharing this podcast with your friends and colleagues.