Transcript #103: Getting to 10x (results for developers)

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 103, recorded November 6, 2018.

00:10 I'm Michael Kennedy.

00:11 And I'm Brian Okken.

00:12 Hey Brian, how you doing?

00:13 I am great today.

00:14 Yeah, it's another wonderful day. A lot of cool news in the Python space.

00:18 I have something that I think you all will really like. I'm looking forward to sharing that.

00:22 Cool.

00:23 Yeah, and I know you got some good ones, so I'm looking forward to talking about those as well.

00:26 Well, before we do, let's say thank you to DigitalOcean.

00:29 Check them out at pythonbytes.fm/digitalocean.

00:31 Get $100 free credit for new users.

00:35 I'll tell you more about them later.

00:36 Right now, you've got kind of this magical C++ Python combination thing queued up for us, right?

00:42 - I do, and I just, we found out about this because a listener, I think it's Sebastian Schrebert, wait, Sebastian Schrizard, is it Brizard?

00:55 Sorry about your last name.

00:57 - Thank you, Sebastian, for sending that in.

00:59 Brian, if anyone else wants to send in some news, you can also butcher their name in honor of them sending that in, right?

01:05 - Yeah, it's a tradition. - As I would as well.

01:07 I do a lot too.

01:09 It's becoming a tradition.

01:10 Thank you, Sebastian.

01:11 All right, tell us about this thing.

01:11 - Well, I think because of the flames in the logo, they actually intended to be pronounced Phoenix.

01:17 It's F-E-N-I-C-S.

01:20 It's a project, I'm glad that Sebastian sort of translated this for us.

01:24 It's a open source computing platform for solving partial differential equations.

01:31 This is actually really cool.

01:32 And I'm gonna quote right from their site.

01:35 "Phoenix enables users to quickly translate scientific models into efficient finite element code with the high level Python and C++ interfaces to help you get started.

01:47 It's got powerful capabilities for experienced programmers, but it's easy to get started.

01:52 and it runs on multiple platforms from laptops to high-performance clusters.

01:58 And it actually looks really pretty cool for anybody dealing with partial differential equations.

02:04 It's a NumFocused-backed project, so there's money behind it, which is cool.

02:09 - NumFocus seems like it's everywhere these days.

02:11 - Yeah, well, especially, and it's good to highlight that, to say that this is a backed project because it's a, these sorts of things, you wouldn't want to depend on and then have them go away.

02:23 - Right, some open source projects, depending upon them, are kind of like getting a puppy, right?

02:29 They're cute and fun, but then you gotta walk 'em and stuff like that if they get dropped or don't work the way you want.

02:35 - Yeah, and Sebastian said, "Right off the bat, "it looks cool, but there's some features inside "that you might not know about right off the bat." So I'm gonna quote an email that he sent us.

02:46 He said, "Phoenix is in fact a C++ project with a full-featured Python interface, the library itself generates C++ code on the fly and be called on the fly from Python.

02:58 It's almost magical.

02:59 Under the hood, it uses SWIG and recently moved to PyBind11.

03:06 I guess the architecture that was set up to achieve this level of automation might be useful for other situations.

03:13 - Yeah, so that's crazy.

03:14 You write Python code, this thing writes C++ code, and then calls it all dynamically at runtime.

03:19 - Yeah, that's amazing.

03:22 And from the project website, being able to develop the algorithm locally on whatever computer you're on, even a laptop or desktop, and then deploying the same code to run in parallel on thousands of processes, that's just awesome.

03:41 - Yeah, I think there's a lot of cool stuff happening here.

03:43 I mean, not everyone is solving, using finite element methods to solve PDEs, right?

03:48 I understand that's a limited group, but there's a lot of projects that may find what this project is doing interesting from a performance perspective and a dynamic meets compiled language perspective.

03:59 - Yeah, and it's also one more example of problems being solved in Python that you wouldn't have thought you could solve with Python because they're just take too much high-performance computing.

04:12 - That's a super interesting point.

04:13 So often I hear people who are not that familiar with Python say, "Well, Python's slow." I'm like, hold on.

04:19 What do you mean that Python is slow?

04:21 Like, you have to say doing this operation in Python is slow because there's so many variations.

04:27 I mean, it can be, well, Python is slow in CPython, so you can use PyPy, but it could be way more interesting.

04:32 Like, well, yes, but you would actually use, say, some kind of library that has C-level compilation elements, like SQLAlchemy or NumPy or something.

04:44 And so when you actually talk about that, like you're doing C, you're not doing Python at the hot spots.

04:49 And then you get way out there with things like Dask and like this and so on.

04:53 And it's pretty awesome.

04:55 >> Yeah, definitely.

04:56 It's neat to see.

04:57 >> You're kind of a fan of regular expressions, aren't you?

04:59 >> Yes, I like regular expressions.

05:01 >> Are you a fan of cursive language, like the fancy calligraphy type stuff?

05:05 Like do you write that way often?

05:06 >> Well, I like it.

05:08 I don't write that way, but.

05:10 >> I don't write it that way either.

05:11 But this next project I want to talk about is, its goal I think primarily is to make regular expressions more easy to indicate their intent and easier to maintain.

05:25 I like to joke that regular expressions are kind of a write-only language.

05:29 You know, you write them but then you can't read them anymore but they're magic and they do their thing, right?

05:34 So we got this project called cursive_re.

05:37 And re is of course the regular expression module in Python, right?

05:41 - Yeah.

05:41 Chris Patty of podcasting at fame and it's actually created by Bogdan Popa.

05:48 My term to hopefully take a shot at getting the name not too wrong.

05:52 But this is a library for doing regular expressions in modern Python.

05:57 So 3, 6 and above.

05:59 And the idea is instead of writing in the string symbols like you normally would, you know, bracket nine, zero dash nine dot, you know, close bracket dot that sort of thing. You write in this higher level language of combinators, they call them.

06:18 And then that overrides the operators in Python to generate a regular expression.

06:24 So this is like all sounds kind of wonky and crazy. But if you see in an example, it's super clear. So you can go and define say, like a hex color. And the way define it as you say I would like to create a you say beginning of line plus this hash symbol plus group repeated hex digit or repeated other hex digit exactly three times plus end of line and you write it in these things that are symbolic of what regular expressions do and they can be you know sort of ordered and and together and added together and so on and then if you call the string representation of them, you get the actual regular expression.

07:05 Okay, so it's a library to build regular expressions.

07:09 Yeah, it's a library to build regular expressions.

07:11 The way Bogdan describes it is he says, "It's a tiny Python library made up of these combinators to help you write regular expressions that you can read and modify six months down the line." Yeah, yeah, definitely one of the problems with regular expressions is they're terse, and that's good and bad.

07:31 They're tutors sometimes.

07:33 - Yeah, sometimes.

07:34 And then the other problem is there's a lot of special symbol.

07:38 Regular expressions are basically nothing but symbols.

07:40 They're like a symbol exploded and its guts came out all over this text.

07:45 (laughing)

07:46 But the problem is some of those symbols have to be escaped if you actually wanna search for them, but it's always kinda hard to know, well, which ones do I have to escape?

07:55 So it also does things like if you tell it, I'm looking for the text of square bracket, square bracket, it'll escape properly in regular expression format, the text representation of that, right?

08:05 'Cause bracket normally means something else, like it's a set of characters or something like that.

08:10 - Yeah.

08:11 - So I really like that it's sort of a safe way as well, like it's a more, you talk about what you want and if it has to be escaped or whatever, it does.

08:19 So yeah, it's a cool example. - I like this.

08:21 This is great.

08:22 - Yeah? - Yeah.

08:22 - I do too, I can certainly see myself using this if I'm writing regular expressions.

08:27 It's great.

08:28 - Speaking of seeing yourself.

08:30 - Seeing yourself, yes.

08:32 Actually, for a long time I've been following and paying attention to what Adrian Rosebrock, this is even an easy one and I even massacred this.

08:40 Adrian Rosebrock, he has a site called PyImageSearch.

08:44 And essentially Adrian is teaching people about OpenCV and Python and actually a lot of AI stuff and doing some really cool things with Python and like cameras and webcams and stuff, and even on Raspberry Pis and stuff, and doing lots of neat things.

09:04 And I don't have a particular article to point to, we just haven't covered it before, I don't think, and people should know about it.

09:10 So Adrian has both paid and free resources to teach people all about computer vision, and I think he's doing a cool job.

09:18 - Yeah, he's doing a real cool job, and there's so many great examples over there.

09:22 I think OpenCV is great, And this is probably the best resource for OpenCV and Python Intersected, right?

09:28 I get emails from him every now and then with boxes detecting things running around on videos or something like that.

09:35 It's great.

09:36 So if you've got to do anything with computer vision, and you want to use Python to do it, and why wouldn't you want that?

09:41 Then this is a cool place, right?

09:42 Yeah.

09:43 And he has some, like, tons of cool projects that he's done over the years of, you know, hooking up a Raspberry Pi with a camera to detect people coming to your door and stuff like that and it's some cool stuff.

09:56 And then you brought up that he had one of the most successful Kickstarters ever?

09:59 Yeah, he did a Kickstarter called Deep Learning for Computer Vision with Python.

10:04 And yeah, it's definitely one of the most successful Kickstarters ever.

10:07 So if you want to check out that book, that's really good.

10:10 I think he has some videos that are coming along with it, but I linked to the Kickstarter as well.

10:14 Yeah, it did okay.

10:15 Okay.

10:16 (laughing)

10:19 - Speaking of doing very, very well, I want to tell everyone about DigitalOcean before we move on.

10:24 And I decided it's time to think about this a little bit differently.

10:27 Think about DigitalOcean and your hosting and stuff a little bit differently.

10:30 So, you know, most of us, I think you brought this up, you're not Netflix, you're not Google, you're not Facebook, right?

10:36 Remember that?

10:37 You're not LinkedIn.

10:38 You don't need these crazy architectures.

10:39 And yet, some of the most popular hosting platforms out there, you know, like AWS or Azure, They are built with thousands and thousands of knobs so that you could be Netflix with 50,000 servers running continuous chaos experiments and all that crazy stuff, right?

10:57 But if you are actually just building what 99.9% of us are, more standard applications, then all that stuff is just overhead and complexity and cost.

11:09 So you can join companies like Slack, WeWork, Docker, GitLab, and of course us over at DigitalOcean and pay like five times less than what you would for AWS or Azure, right?

11:21 So for example, bandwidth is one cent instead of nine per gigabyte.

11:24 Servers are five bucks instead of 50 or whatever a reserved instance is over at AWS.

11:29 It's not just about price.

11:31 Keep it simple.

11:32 Use what you need.

11:33 Don't have all these crazy, crazy features that you probably don't actually need 'cause you're not building Netflix or Facebook or LinkedIn.

11:40 Anyway, try them out at pythonbytes.fm/digilution.

11:43 Get $100 credit for new users and see why we all love it for their infrastructure.

11:48 - Awesome, thanks.

11:49 - Indeed.

11:51 So, Brian, this next one, have you watched this video here that I'm about to talk about?

11:57 - I don't think I have, maybe I have.

11:59 - All right, so while we're talking, click this and open it up.

12:03 - Okay.

12:03 - And I'll tell people about it, and maybe mute your YouTube that's going.

12:06 - Okay.

12:07 - Okay, so this thing that I wanna talk about is a visualization of Python development from original origin, way back in the day, up till 2012.

12:16 And it's using this underlying system called Gorse, G-O-R-S-E, Gorse, I'm guessing.

12:23 And the idea is Gorse is a visualization library that visualizes trees and stuff like this, like graph trees, not real trees.

12:30 And what you can do is you can point it at a source repository.

12:35 That could be SVN, it even supports CVS, but Git and Mercurial.

12:39 And you can point it there, and it will do a animated over time visualization of that source library.

12:46 So not just Python, any of these repositories you can point at.

12:50 And it'll have the files as they grow, the size of the repository, and these little animated characters come in and start editing files and interacting with it.

12:59 So if you play this video that we're linking to, this is a gorse animation of Python development up till 2012.

13:07 And you see it starts out and Guido's cruising around, adding little bits, then a few more people come, and then if you start skipping farther and farther ahead, it gets just like madness at the end, like people are swirling all over the place, and it's just a really great way to like, see the growth of Python visually through animation, at least in terms of who's participating in building it.

13:27 What do you think?

13:28 - Yeah, it is cool, and one of the fun things to see is that there's sometimes some people that just sort of sit around one area, which makes sense, and then other people that fly around and edit all sorts of stuff.

13:42 - Yeah, it's really interesting, right?

13:43 And you can see people appear and then they'll fade away, they'll come and make some contributions and then they'll leave the scene.

13:49 So anyway, I think this is really cool and there's not a lot to take away from it other than it's just nice to appreciate it.

13:55 I would say watch the first minute and then just skip minute by minute and watch a little bit 'cause it's 14 minutes, you don't really wanna watch the whole thing.

14:02 - I think I do.

14:03 (laughing)

14:04 - You could just leave it running for the rest of the show.

14:06 - Yeah.

14:07 So here's what I think.

14:08 I call out to the audience.

14:10 One, who wants to build this for 2012 to present on Python?

14:15 Again.

14:16 And I think this would make an amazing lightning talk if you built that video, and then you went up there and just did a four minute animation.

14:23 Wouldn't that be cool?

14:24 It'd be cool to just have that going on in the background while you did some other talk.

14:28 Right, yeah.

14:29 Or run this between sessions at a conference.

14:33 I think it'd be great.

14:34 Anyway, I think people will appreciate checking it out.

14:37 So one of the things, Brian, that you hear a lot in software development is that there's often a wide range of skills and productivity between developers.

14:46 And I've done a lot of training and interacted with literally thousands of people in person.

14:51 And I think it's something of a myth, but I think largely there's a lot to this.

14:57 There's some people that just fly and they're just focused, and others that just kind of bounce around the keyboard randomly.

15:04 What do you think?

15:04 - So I know that it's a bit controversial.

15:06 There is this notion of a 10x developer.

15:09 And often there's a backlash against it also.

15:13 But I think people think of it as like somebody that's really 10 times better than the average good developer.

15:20 And I don't think that's it at all.

15:21 I think that it's just a notion that there is sometimes orders of magnitude between the most effective person in a organization and the least effective.

15:31 And I don't know how you argue against that.

15:34 If you've ever been at large organizations, it just is, at least, maybe it's not 10 times, but there's definitely, there's a lot to it.

15:42 - It's a multiplicative factor, I would say for sure.

15:45 - Yeah, and so this is, regardless of what you wanna take away, there's some good advice in this article.

15:50 There's a article I'm linking to that's what any developer can learn from the best.

15:56 And I think these are good things.

15:57 So one of the things, the idea around it is just This isn't magical and it isn't something that is only you.

16:05 It isn't just about skills and hard skills.

16:08 It's other stuff, too.

16:10 And there's a clear path to excellence.

16:12 People are not born great developers.

16:15 They get there through focused, deliberate practice.

16:17 And here's a few traits.

16:19 They just listed some traits of things that they they see in good developers versus not so great.

16:26 So great developers are a few of the traits are problem solving.

16:31 They're skilled at what they're doing.

16:32 They're mentors and teachers.

16:34 They're excellent learners and passionate about stuff.

16:38 So the problem solver bit, I think, is really interesting because often the 10x or the multiplier doesn't come in from that they do the same work faster.

16:48 It sometimes is they can just look at things differently because of experience, because of playing around with lots of different things and say, "Oh, let's just solve this problem differently." And it just gets done faster because they take a different approach.

17:05 I was talking with somebody about databases recently, and there's some, for instance, there's some problems that can be solved with graph databases easily that are almost impossible with a relational database.

17:18 It's just using the right tool for the right job sometimes.

17:20 >> Yeah, and I think a lot of that is the, it's not that knowing that means you're 10 times smarter or 10 times better, that means you are curious enough to keep looking and to continuously be on the lookout for, well, okay, I now totally know all about relational databases.

17:38 What else can I learn?

17:39 How do I compare it?

17:40 And I think that's one of the biggest traits that I've noticed among people that have some number of multiplier of efficiency or skills is not that they are necessarily more skilled or have more natural talent, but they're just continuously learning and really passionate about it.

18:00 And they're just always picking up these little things that help you at each step, right?

18:04 Yeah, and there's an honesty of just like, this isn't just about developers.

18:08 I think it's in every field.

18:11 For instance, somebody that's working with tools and stuff, somebody that needs to hammer in a lot of nails might be smart enough to go, "Hey, I should go get a nail gun.

18:22 I could do this a lot faster with a nail gun.

18:24 - You know what that makes me think of?

18:25 Like, you could say, I need to have the rings on my engine or my transmission fixed on my car, and you take it to a regular place, you're like, great, we'll have it back in three weeks.

18:36 You go watch something like IndyCar or Formula One, they'll pull in and they'll change the transmission in two laps.

18:42 Those are both mechanics.

18:45 - Yeah.

18:46 - Those are not the same.

18:46 - Right, exactly.

18:48 The deliberate practice is important, And I think some people forget that the difference between being an average or a below average developer and being an above average developer is mostly just deciding that you're going to do that and setting aside some of your time in your life to pick something you wanna improve and then go and do it.

19:10 Figure out what you wanna learn and go do it.

19:12 So, anyway.

19:14 - I guess what I would say as a takeaway from this article is it's really interesting.

19:18 It's numerically based, right?

19:20 There's a lot of, it's based on a survey the guy did with like a thousand folks or something.

19:24 And most importantly, it's about a growth mindset, right?

19:27 It's not to say, well, there's these people and they're just smarter than you, or they're smarter than other people or whatever.

19:32 It's here's how those people got that way and you can do it too.

19:37 And I think that's the right message.

19:39 - Right, and the person that wrote the article said this is from a lot of, he's taught a lot of people and there's a lot of people that want to be better but they don't know how to.

19:47 So this is just sort of some direction on what things to work on.

19:51 So, anyway. - Yeah, it's a good article.

19:53 Yep, and a nice find.

19:55 Let's close this out with a bit of chaos, huh?

19:57 - Sure, way to change tradition, right?

20:00 - That's right.

20:01 So you've heard of, I'm sure you've heard, I'm pretty sure you've been out on your show, if I remember correctly.

20:06 You've talked about the Chaos Monkey and things like that, right?

20:10 - Yeah, Casey Rosenthal on the show.

20:13 - Yeah, he's from Netflix, right?

20:14 - He used to be, yes.

20:16 - Okay, so I think the idea of the chaos monkey originated there.

20:19 Tell us what the chaos monkey is.

20:20 - Well, I'm probably gonna get this wrong, but there's this notion of taking parts of your system and intentionally breaking parts or shutting down, especially in a distributed system, taking some nodes and just killing them every once in a while and seeing how your system recovers from it.

20:36 - Yeah, exactly.

20:37 So the idea is if you build an architecture, both in infrastructure and software, that is supposed to take durability.

20:45 It's supposed to work if this part of your cloud goes down, or it's supposed to work if one of your database nodes goes down.

20:52 That's the theory.

20:53 But then there's the reality of how does it actually behave if one of your nodes goes down, one of your machines reboots, random stuff like that, right?

21:00 So the Chaos Monkey runs around in production, killing off processes, servers, et cetera.

21:07 And then in production, it does it.

21:11 And then you just have to build, because you know the Chaos Monkey's coming.

21:14 And it means like, well, a standard failure is like nothing 'cause the chaos monkey is a madman, right?

21:19 He's running around all the time.

21:20 So this is not a bad philosophy for large organizations or large bits of software.

21:26 However, you're not Netflix probably, people listening.

21:29 So how are you gonna build that up, right?

21:31 How are you gonna create these things?

21:32 So there's this cool thing I found called Chaos Toolkit.

21:37 So Chaos Toolkit is a library built in Python that will help create these Chaos Monkey-like things.

21:46 - Cool. - So, Chaos Engineering is what they call it, is the discipline of experimenting on distributed systems in order to build confidence in the system's capability to withstand turbulent conditions in production.

21:57 All right, so we talked about the Chaos Monkey and the friends, right?

22:00 There's other types of chaos things at Netflix.

22:04 But here's a way that you can easily build those types of experiments and systems, And it integrates with Kubernetes, AWS, Google Cloud, Microsoft Azure, some other things like that.

22:16 So just to give you a sense of what it can do, like if you look at the AWS API, it'll say you can do things like go to AWS Lambda and call a delete function concurrency that removes the concurrency limit on any specific Lambda.

22:30 Or you can just call stop instance on an EC2 instance or whatever you want.

22:35 And presumably it's gonna put that back, I'm not entirely sure.

22:38 but I guess you probably got to call start instance again on it or set, you know, add function concurrency or set something like that.

22:46 All right, but there's this infrastructure to help you change these types of settings and these types of things around your cloud providers and, you know, make sure your system can take it.

22:54 - Nice, yeah.

22:56 You cannot plan for the best, so you plan for what you can for.

23:00 - That's right.

23:01 Yeah, so you build it so that it doesn't have to be perfect and then you're in pretty good shape.

23:05 The stuff that the folks at Netflix are doing is insane, So like they take it to another level.

23:09 - Well, yeah, one of the things I remember talking with them about is the reason why they test and do all this in production is because their system is so large, you can't, it's essentially the world.

23:21 You cannot have a test bed that's similar enough to their real world environment.

23:29 So they don't have that luxury.

23:32 - Right, so they just test it in production.

23:34 And I've not seen Netflix really go down, So I'm going to say they're doing it.

23:38 All right, that's it for our main news.

23:39 I have a couple of quick ones.

23:40 You got anything you want to share, Brian?

23:41 - Yeah, just on the Python Bytes episode 100, one of the things we talked about was pyproject.toml, and I wanted to take a deep dive.

23:51 So the last episode of Test & Code is me talking to Brett Cannon, talking about that for almost an hour or so.

23:58 - Oh, nice, I definitely want to check that out.

24:00 That's a good one.

24:01 - So how about you?

24:02 - I have two other things that I don't think justify a whole segment, but I just want to throw them out there.

24:08 Remember a while back we talked about that there was potentially some experiments or something where some college researcher had put some malicious, potentially malicious libraries on PyPI?

24:19 - Yeah.

24:19 - Well, it turns out recently someone actually put malicious libraries on PyPI.

24:24 So this is October 27, 2018, and apparently 12 packages were discovered with various levels of vulnerabilities and stuff.

24:31 So I'm linking to that and people should check that out.

24:34 If you don't know how to spell Django, you're going to have a bad time, by the way.

24:39 Yeah.

24:40 So it's a lot of this typo squatting, like, oh, I forgot the J, so now I have a virus, something like that.

24:46 Yeah, that's sort of lame.

24:48 Yeah, it's upsetting.

24:49 At least they've done some work over at PyPI to block properly spelled things that are not actually packages, like re, for example, right?

24:57 The built-in stuff.

24:58 Well, yeah, that's one of the things that was an issue, is people trying to pip install things that are in the standard library.

25:06 You don't have to do that.

25:07 - Yes, exactly.

25:08 Exactly, don't do that.

25:10 Then the other one was just a quick little Twitter message that someone sent my way, and it's pretty cool.

25:17 This guy, not the one who sent it to me, I'm sorry, I don't remember who sent it, I should have written it down, but this person called Xtrek, something to that effect, on Twitter, decided to go scanning the standard library source code for interesting things like, what is the longest class name?

25:35 What is the longest function name?

25:36 And so on.

25:37 So they found out that actually in CPython, the longest class name is 200 characters, just the letter A 200 times, which is some kind of test case.

25:49 But for the real ones, there's one called test mutually exclusive optionals and positions mixed parent as the longest class name is there.

25:59 In the longest function name, they believe it's test_parser_regression_special_character_in parameter column of docstrain first line, which is 84 characters long.

26:12 And then there's other examples.

26:15 Someone says there's actually a test ctypes that has 33 million characters in it.

26:21 Anyway, it's an interesting thread if you're just wondering what the really long names are.

26:27 Yeah.

26:28 it almost could compete with Java names.

26:30 - Yeah, exactly.

26:32 Standard types right there.

26:34 Awesome, all right, well, definitely fun to share all this news with you, Brian.

26:38 As always, thanks for doing it.

26:39 - All right, thank you, bye.

26:40 - You bet, bye.

26:41 Thank you for listening to Python Bytes.

26:43 Follow the show on Twitter via @pythonbytes.

26:46 That's Python Bytes as in B-Y-T-E-S.

26:49 And get the full show notes at pythonbytes.fm.

26:52 If you have a news item you want featured, just visit pythonbytes.fm and send it our way.

26:56 we're always on the lookout for sharing something cool.

26:59 On behalf of myself and Brian Okken, this is Michael Kennedy.

27:03 Thank you for listening and sharing this podcast with your friends and colleagues.