Transcript #61: On Being a Senior Engineer

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:04 This is episode 61, recorded January 10th, 2018.

00:10 I'm Michael Kennedy.

00:11 And I'm Brian Okken.

00:12 And we have, again, some really cool stuff we want to share with you.

00:15 The best of what we found this week in the Python space.

00:18 Before we get to it, though, I want to say thank you to DigitalOcean.

00:21 This podcast, these MP3 files, everything is coming to you directly out of DigitalOcean servers.

00:27 I'll tell you more about that later.

00:28 Right now, I want to talk about dictionaries.

00:30 Yeah, I want to talk about dictionaries, too.

00:32 So thanks to Raymond.

00:33 This is a, I'm going to reference Brandon Rhodes, actually, for later on.

00:38 But Raymond Hedinger is probably the person that I have to thank for telling me how cool dictionaries are in the first place.

00:46 Because he's talked about them at different PyCons in the past.

00:49 And basically, I mean, tons of Python is built on top of dictionaries.

00:54 But one of the things that I did not know was how awesome key sharing is.

01:01 And we had one of our listeners on Twitter, Ned Letcher, keyed us in on this and said,

01:08 Hey, if you haven't checked it out, you should go look at Brandon Rhodes'

01:11 dictionary even mightier talk from PyCon 2017.

01:16 And so I did.

01:17 And yeah, the key sharing thing is like something I didn't know about, which is, it's totally cool.

01:23 It's kind of funky to think that it's not that way until you really think about how it works.

01:29 Right.

01:29 So when you create a class, and you create an instance of it, an object, every one of those has a Dunder dict.

01:35 Right.

01:35 And what goes in the keys are the names of the fields of the class.

01:39 And what goes for the values are, well, the values.

01:42 Right.

01:42 So when you say in Dunder init, you say self.name equals this, self.id equals that.

01:47 Those are putting entries in that dictionary.

01:49 Right.

01:50 Yeah.

01:50 But the thing is, if I create a million of those, I have a million times the string name as the key and a million times the string id as the id.

02:01 So basically, every instance you create stores a separate copy of the names, not just the values, but the names of the fields, which can be super problematic, right?

02:10 The names of the fields are, and in Brandon's talk, he talks about, since it's a dictionary, there can be hash collisions for figuring out where to put it.

02:20 And so each of that happens for every single object has to figure that out.

02:24 I think it was, what, Python 3.3, 3.4, where they changed it to where, so here's the trick.

02:31 As long as you put all of the fields that you're going to use for your class objects, you initialize them with something in your init function, then all of your objects can share the same dictionary for the key lookup.

02:47 Yeah, that's really cool.

02:48 So I think that's quite awesome, and it should really help with performance as well, especially memory-wise.

02:52 Yeah, it's a little bit of a little weird thing of, like, you have to kind of think about how this is implemented.

02:57 But Brandon does a really good job of walking you through it, so that it's obvious how the savings is.

03:04 And apparently from the PEP 4.12 that this was started with, for object-oriented programs, the memory reduction is like 10% to 20% less, using quite a bit less memory.

03:16 So that's definitely worth it.

03:17 Oh yeah, that's always a nice benefit there.

03:19 So I have an interesting project recommended by a listener as well called Python Hunter.

03:26 Have you heard of this?

03:28 Well, just because it was recommended to us, but I haven't looked at it yet.

03:31 Yeah, so it's pretty interesting.

03:33 The idea is it's a little bit like code coverage, and it's a little bit like a debugger, in that you, like, enable it on your code.

03:41 And then as you interact with the code, you get a report.

03:44 But what you actually get here is it will give you, basically, as your program runs, it'll show you the internals of, basically, any module or package you want to point it at as it executes.

04:01 So there's an example where they turn it on, and they call os.path.join, and it shows you, basically, all the internal implementation running, like, printed into the console, intermixed with your output.

04:13 Oh, interesting.

04:15 Yeah, so if you were like, I would like to explore what this does when I call this function, and I don't want to root through all the source code and try to figure out what piece this is referring to.

04:24 You can just turn this on, call the function, and it'll show you the path through the external code it took.

04:29 Oh, okay.

04:30 Yeah, so pretty interesting.

04:32 It has some integration with debuggers and things like that.

04:36 So you can also override it.

04:37 And it doesn't happen for every bit of code you call.

04:40 You point it at a module.

04:41 Like, say, I'm trying to work on, let's say, POSIX path.

04:46 I want to understand my interaction with that module.

04:49 Then you call os.path.join, and off it goes.

04:51 So, yeah, pretty sweet, actually.

04:53 Like, for instance, if I want to mock out my request interaction to external databases or external sites,

05:00 I could point it at requests, for instance, to find out exactly what my application is doing with requests.

05:06 Yes, exactly.

05:07 And you would say, oh, this is the part where it goes to the network.

05:09 That's the one thing I have to mock out.

05:11 The rest I can just ignore.

05:13 That would be a great tool, especially for people that are trying to isolate bits of their application.

05:18 Yeah, that's cool.

05:18 So this was recommended to us by EvenPage.

05:20 So thank you for that.

05:21 And it's also kind of interesting that it is built on Cython.

05:25 So it's nice, high performance.

05:27 It's always cool to see Cython making an appearance.

05:30 Nice.

05:31 So, you know, I always feel like when I'm working with Bash, I could do a better job.

05:36 And in fact, I don't use Bash that much these days.

05:38 I use ohmyzsh.

05:39 But same thing.

05:41 There's all this stuff about the shell I could do better.

05:43 You use zsh?

05:44 Use ohmyzsh.

05:45 I love it.

05:46 Ohmy.

05:46 Ohmy.

05:47 Oh, okay.

05:47 Yeah, yeah.

05:48 It's really got some nice integration with Git, the history and the search and the autocomplete,

05:53 like a tabbed selection of various things.

05:56 If you type a few characters and hit tab, it'll show you the list of all the things that you've done involving that.

06:02 It's kind of cool.

06:02 Okay.

06:03 I'll check that out sometime.

06:04 But I think more people use Bash, actually.

06:06 So I went through a phase where I started out learning KSH in college and in my first job.

06:13 And then I tried Bash and I went to Zshell for a while.

06:17 But it had support issues on whatever operating system that I was using at the time.

06:21 So Bash is just everywhere.

06:23 So we can just use it as is.

06:25 The place where you have to jump through a few hoops is on Windows.

06:29 So I wanted to bring up, before we get into this article, there's an article called 10 Things I Wish I'd Known About Bash,

06:35 which is a really great little quick start on things that might speed up.

06:40 It's just some fun things.

06:42 There's somebody trying to sell a book also, but it's only like five bucks.

06:46 So go for it, dude.

06:47 But kind of things that I'd always wanted to know, like what the difference is between back ticks or the dollar sign paren replacing bits in your command.

06:56 Just if it is, they're identical.

06:58 It's just one's a lot easier to read if you have a whole bunch of them.

07:01 Yeah.

07:01 Okay.

07:02 Very cool.

07:02 And then, you know, like globbing versus regular expressions.

07:05 That's something that confused me at first.

07:08 So that's a good thing.

07:09 Dealing with exit codes.

07:11 The number four is the one I really like the most because that one confused me is when you're using if statements,

07:17 what's the difference between bracket and bracket bracket?

07:20 And I'm not going to talk about it here, but these are some good things.

07:24 I also wanted to bring up, I spend a lot of time trying to hunt for a good bash application for Windows.

07:32 And right now, what I'm using the most is Git for Windows.

07:36 And it has Git Bash is embedded in it.

07:39 I think that's great, even for people that are not using Git, because it's just a good bundle of all of your bash tools.

07:47 And it just always works.

07:48 So I'm pointing everybody at that now, even if they're not using Git.

07:52 We did have a recommendation from one of our listeners a long time ago about a tool called Commander, I think.

07:58 Yeah, I'd say Commander, yeah.

07:59 Yeah, cmder.net.

08:01 And I think there's a reference on here that it's, I think it's related to Git for Windows somehow.

08:07 So not sure.

08:09 But if you don't really use all the, if you don't want to try Git for Windows,

08:13 but you still want some emulator working on Windows, check out Commander also.

08:18 Yeah, and it has the MonoKai theme.

08:21 It actually looks really nice.

08:22 Commander looks quite cool.

08:23 I saw it on my Windows 10 machine.

08:24 One more thing I'll throw on your list here while you're talking about these is,

08:29 I don't even remember what version of Windows 10 has this.

08:32 It's not the original, but maybe the one last fall.

08:36 I can't remember what came out, but they've added the Ubuntu subsystem for Windows,

08:41 which is pretty cool.

08:43 But you have to turn that on in like the turn Windows features on and off.

08:46 It's off by default.

08:47 And I'm glad you brought that up because I want to try to revisit that.

08:50 I did try those and I had issues with being able to launch external applications.

08:56 Yeah, they're like really isolated.

08:58 Like the Ubuntu part and the Windows part, they don't really communicate very well.

09:03 And so it's not the same, I don't think.

09:05 It's not like you have to do that a lot, but I often just want to open up my text editor

09:10 from the command line.

09:11 Yeah, exactly.

09:12 Things like that.

09:12 Cool.

09:13 All right.

09:13 Yeah, those are all nice things to check out, especially on Windows.

09:15 It needs the most help with the shell, actually.

09:18 Cool.

09:19 So before we get to the next one, which is a bunch of pretty pictures that I hope will help

09:24 people understand the definition of micro frameworks versus, you know, batteries included

09:29 frameworks, things like that.

09:30 I just want to tell people about DigitalOcean.

09:32 Those guys are doing awesome work over at do.co slash Python.

09:37 So check that out there.

09:39 This is where I have all my servers.

09:41 I probably have eight servers for the various things all working together now, failover and

09:46 database servers and whatnot.

09:47 But yeah, it's great, super easy.

09:50 30 seconds.

09:51 You got a Ubuntu machine or what other.

09:53 There's a bunch of different distributions and pre-configured servers.

09:57 Go over there, set them up, check it out.

09:59 They're doing a bunch of great stuff and it's definitely affordable and reliable and fast.

10:02 Awesome.

10:03 Yeah.

10:03 Cool.

10:03 So there's this thing called Snake Food, which is a tool written by Martin Blaze and it creates

10:11 Python dependency graphs.

10:13 So I've got one package or one module or one class that depends upon some other thing and

10:20 it'll like put them in a graph and draw lines between them.

10:22 So that's pretty cool.

10:24 I haven't done a whole lot with it.

10:26 You can combine that with GraphViz and it'll create these nice visualizations of Python

10:31 code bases.

10:31 Oh, nice.

10:33 So that's pretty cool.

10:33 And this is not a brand new item, but I had seen it a while ago.

10:37 It's from the Grok code guys.

10:39 I think I had them on Talk Python like episode eight or something.

10:42 So it's been a while.

10:43 It wasn't about this, but they took this Snake Food plus GraphViz and they turned it on a variety

10:50 of the frameworks that we know and love.

10:52 And it's really interesting to pull up the pictures and just look at the relative complexity.

10:58 So if somebody tells you a bottle is fast because it's simple, but it doesn't do very much for

11:03 you.

11:03 So you've got to bring a lot of pieces in.

11:05 Django is much more full featured and more complex, but it has a lot of things built in.

11:11 And Flask is somewhere in the middle and Pyramid, just a little more than that.

11:15 You can look at these pictures and get a sense.

11:17 Do you have the pictures pulled up?

11:18 Yeah, definitely.

11:19 You can totally see that that's the case.

11:22 Bottle is like incredibly simple.

11:24 Like I'm actually blown away that it is as simple as it is, but you can just tell like

11:28 straight away by looking at it.

11:29 It's so elegant.

11:30 There's no cycles.

11:31 Yeah, there's no cycle.

11:32 It's an acyclic directed graph.

11:35 That's quite interesting, isn't it?

11:36 Yeah.

11:36 Cool.

11:37 So yeah, and then Django, how would you describe Django?

11:40 It's like if you clean out a comb after it's been used for a year, that's what it looks

11:45 like.

11:46 Yeah, it's like a little bit like one of those inkblot tests.

11:49 Yeah.

11:49 And Flask and Pyramid, they're sort of on the same scale.

11:53 Pyramid's got a little bit more going on, but they're definitely not simple, but they're

11:57 not insane, right?

11:58 They're not like Django level of interaction.

12:00 Then they go and they do this for Celery and RQ as well.

12:04 And you could tell Celery is way more complicated.

12:07 It's more complicated than Pyramid or Flask, by the way.

12:10 But it's also more complicated by ways than RQ.

12:13 Let's see, what else do they cover?

12:15 They cover Twisted, and Twisted is pretty insane.

12:18 It's probably more tied together than Django, I would say.

12:23 Mercurial, quite interesting as well.

12:26 Request is in here.

12:27 Request has got some pretty nice look, actually.

12:29 Yeah, it's oddly elegant and pleasing.

12:32 It is.

12:32 IPython, not so pleasing.

12:33 No.

12:35 Anyway, I recommend people pull up this snake fooding link and just have a quick glance

12:39 through the pictures.

12:40 I think it's quite insightful.

12:41 And it gives you the technique for applying this to other code bases as well.

12:45 Yeah.

12:45 These would be fun things to just print on your wall and tell people that you're using it

12:50 as a reference.

12:51 Yeah, absolutely.

12:52 Pretty cool.

12:53 Yeah, I just follow the graph when I need to know how things work.

12:56 Yeah.

12:59 Your next item is about being a good senior engineer.

13:03 The new year, it's 2018 now.

13:06 There was somebody that posted on Twitter that I'm terrible about references, said, hey, it's

13:11 the new year, you should go read this again.

13:12 So this is a 2012 article called On Being a Senior Engineer.

13:18 It's from a website called Kitchen Soap, which is funny.

13:21 But anyway, I think there's just a lot of really good advice.

13:25 And I just went through and pulled out a bunch of the headlines.

13:28 But it's some things that I've been thinking about lately as I'm hiring people and thinking

13:36 about always with evaluations and stuff like that.

13:39 What does it mean?

13:40 What does an engineer versus a senior engineer mean?

13:43 And these, a lot of traits, I think, fit well.

13:46 I think these are all really, really good.

13:49 I mean, maybe you don't necessarily cover them all, but don't practice CYAE.

13:54 Yeah, cover your ass, engineer.

13:56 Exactly.

13:57 Oh, sorry.

13:57 No, no, exactly.

13:58 That's good.

13:58 So, I mean, I think that's a really good point.

14:01 Like a lot of people who, when they're starting out, they're trying to make sure they're never

14:05 perceived as not knowing everything, right?

14:08 Right.

14:08 By the time you're a senior engineer, you've been beaten with the fact that you don't know

14:11 stuff and new stuff is coming along that you also don't know continuously, right?

14:15 One of the top ones is seek out constructive criticism of your designs.

14:20 And yeah, you're just trying to find other perspectives to make sure you haven't missed

14:24 something.

14:24 One of the things I really like is lift the skills and expertise of those around you.

14:28 Yeah.

14:29 Well, you and I are definitely trying to do that.

14:31 So that's good.

14:32 I think a lot of people also try to like keep the recognition for themselves when they're

14:36 new.

14:36 All right.

14:37 But I think one of the biggest skills someone can be providing on a team is actually anytime

14:43 someone gets stuck, they kind of float over to that group and help them keep that part of

14:47 the group and help that person keep moving.

14:49 They can like almost write no code, but sort of float around and make sure nobody's stuck

14:54 and everyone's productive.

14:55 And that could be a massive boost to the team.

14:57 Some of the things like bringing up a couple more items, estimating how long something's going

15:02 to take or how much work it is.

15:04 That's hard.

15:05 It's always hard.

15:06 You know, as you grow in your career, you realize that you can't avoid it.

15:09 So you just need to embrace it and be okay with giving ballpark estimates to people and

15:14 then trying to get better at it.

15:16 Keep track of what you did and try to improve.

15:18 The other one that's difficult is trade-offs.

15:20 You know that you can't get all the information before you start working.

15:24 So making good trade-offs between information and just risks and benefits and making good judgments.

15:31 Those are things you just have to practice.

15:33 Another thing that I wanted to bring up is the...

15:36 There's a list of cognitive biases.

15:37 So just knowing that when you're making a decision, there are biases in yourself that you will...

15:43 You need to be aware of and try to, you know, make sure those are there and compensate somehow.

15:49 Yeah.

15:49 These are all really good.

15:50 I like to understand that not all projects are filled with rock star on the stage work, right?

15:55 Like you got to become a finisher.

15:56 And a lot of times there's a bunch of just little details nobody likes that are part of that.

16:00 Probably the first half of my career, most of my job usually was I would be taking on the things that everybody else didn't want to work on and forgot about and just wrapping them up and finishing.

16:11 But then it looked like I was getting off and getting credit for finishing projects that most of the work was done by other people.

16:17 They just didn't finish it.

16:18 Yeah.

16:19 Become a finisher.

16:19 It's good stuff.

16:20 And then there's finally the 10 commandments of egoless programming.

16:23 Want to take us quickly through them?

16:25 Understand and accept that you will make mistakes.

16:27 You are not your code.

16:29 Each of these could be episodes in themselves.

16:32 No matter how much karate you know, someone else always knows more.

16:35 Don't rewrite code without consultation.

16:39 Oh, I'm guilty of that.

16:40 Definitely.

16:41 Treat people who know less than you with respect, deference, and patience.

16:45 The only constant in the world is change.

16:48 The only authority stems from knowledge, not from position.

16:51 Fight for what you believe, but gracefully accept defeat.

16:54 Don't be the coder in the corner.

16:56 And critique code instead of people.

17:00 Be kind to the coder, but not to the code.

17:02 These are good.

17:03 Yeah, these are all great.

17:04 So yeah, I really connect with a lot of these.

17:06 These are great.

17:07 So I'm glad you brought this up.

17:08 And nowhere in here it says don't make fun of PHP.

17:11 No, you could totally make fun of PHP all day long.

17:13 Yeah.

17:13 But you could also kind of make fun of Python, unfortunately, a little bit for its lack.

17:18 UI frameworks.

17:19 Like we talked about that last week, right?

17:21 How that's kind of one of its last remaining places where it really quite a bit could be

17:27 better.

17:27 One of the things that we didn't read above was no empty complaints.

17:31 And one of my rules is I don't like criticizing stuff without offering a better alternative

17:38 or at least some kind of constructive thing.

17:39 So here's my attempt to backfill my criticism of the lack of UI frameworks with, well, here's

17:45 where we are.

17:45 And here's some things that seem like they're working and a little bit of constructiveness.

17:49 So I went through and I said, all right, well, what are the options for Python UI frameworks,

17:53 right?

17:53 Like I'm not loving what's available right now in a lot of different angles.

17:58 But what can we do?

18:00 We talked TK Enter, not super amazing.

18:02 Like I have a link to an example.

18:05 Brian, pull it up and tell me how you feel about that application.

18:07 Does that look like a classy new app from 2018?

18:13 No, it looks like it's running on Windows 95.

18:15 Yeah, it's like Battleship Gray.

18:17 Looks like it could have easily come off of.

18:19 It looks like whoever built the original Minesweeper in Windows 3.1.1, they built this, right?

18:26 And so that's kind of what the current UI looks like in Python, which is, it's a problem,

18:31 right?

18:31 So what are the options?

18:32 So we have that, but we have PySide and Qt.

18:35 And actually Qt is pretty nice in terms of how it looks like.

18:39 It really looks like it belongs on the platforms.

18:41 There is a designer for it, a draggy droppy designer type thing, which is pretty good.

18:45 But the license is a little weird.

18:47 It's like PyCute is GPL.

18:51 PySide is like an LGPL way to try to go around it.

18:55 But PySide, I think, is slightly out of version with Qt.

18:58 There's just like real messiness around it, but it's still a pretty nice thing.

19:02 There's Kivy, which, and there's PyGame.

19:05 There's also PyOpenGL.

19:06 But to me, if you go to like Kivy and you look at the showcase or the examples of people,

19:13 what they built, they all look like AR type things or games or other types of simulation

19:19 stuff.

19:19 And I know you said people have created a few things that are not quite like that, but it's

19:22 not really the focus of the platform, right?

19:24 One that actually looks pretty good is WX Python.

19:27 Click on the example for that.

19:28 That's one that I've used before.

19:30 Yeah, this looks pretty good, right?

19:31 This could certainly be at least circa 2005 on Windows.

19:35 It looks pretty different.

19:37 And it has like a GUI, a couple of GUI designers.

19:41 It has these widgets.

19:42 That's pretty cool.

19:42 Yeah.

19:43 And then there's all the Electron.js stuff.

19:45 So Electron.js is basically Node.js plus hosted headless Chrome plus JavaScript and HTML for

19:53 the front end.

19:53 So there's actually a couple of attempts to create a, what I really think should be called

19:58 Proton.py instead of Electron.js.

20:01 A Python equivalent where the running back end is Python.

20:05 And, you know, you could use like Sculpt or something to actually make the JavaScript side

20:09 also Python, like the Anvil guys are doing, which is pretty cool.

20:12 So there's this thing called Eel, which is a library for making Electron-like offline HTML

20:18 JavaScript GUI apps.

20:20 And like I said, you could put Python in the front end with that Sculpt bit.

20:24 And that looks pretty interesting.

20:25 So you get basically like a local app packaged up as a .app or .exe, but, you know, it's more

20:31 or less like a web front end that can do like local machine Python capabilities.

20:36 So that's pretty cool.

20:37 And then there's a bigger version called CEF Python.

20:41 That name is not amazing, but it's like Chrome embedded framework Python.

20:45 And it's the same type of thing.

20:47 It's a little more advanced, but it's also a little less obvious how to get started with

20:51 it, I think.

20:52 But both of those look like interesting ElectronJS-like things.

20:56 These are people trying to solve the problem we have.

20:58 So that's good.

20:58 Yeah, it's good to see them do it.

21:00 And yeah, last week we talked about possibly having an open session at PyCon in Cleveland

21:06 in May about this.

21:07 So it'd be really fun for people to get together and join us there.

21:10 I think Paul Hilderman is kicking that off, but we can all go and be part of it.

21:15 Definitely.

21:15 Awesome.

21:16 Well, anyway, here's my research bit I did on Python UI frameworks.

21:20 None of them are like, yes, this is the answer, but maybe they'll spark some thinking and get

21:24 us going in the right direction.

21:25 All right.

21:26 Well, that's all I got for you this week, Brian.

21:27 Anything else from you?

21:28 No, that was a fast six items.

21:30 It was because they're so fun.

21:32 I really like the one that you got about the senior developer stuff.

21:35 It's not totally from 2018, but it's that evergreen material that just lasts, right?

21:40 We talked about quite a bit of those, but every single item that we talked about has

21:44 a paragraph or more of discussion on the article.

21:47 So please go read that, everybody.

21:49 It'll make you a better person.

21:50 Absolutely.

21:51 Very cool.

21:51 All right.

21:51 Well, Brian, thanks for sharing this with everyone.

21:53 Thank you.

21:54 Bye.

21:54 Yep.

21:54 Bye.

21:55 Bye.

21:57 Bye.

21:58 Bye.

21:59 Bye.

22:00 Bye.

22:01 Bye.

22:02 Bye.

22:03 Bye.

22:04 Bye.

22:05 Bye.

22:06 Bye.

22:06 Bye.

22:06 Bye.

22:08 Bye.

22:10 Bye.

22:11 Bye.

22:12 Bye.

22:12 Bye.

22:12 Bye.

22:12 Bye.

22:12 Bye.

22:12 we're always on the lookout for sharing something cool. On behalf of myself and Brian Okken,

22:17 this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and