Transcript #61: On Being a Senior Engineer

00:00 Hello and welcome to Python Bytes where we deliver Python news and headlines directly to your earbuds.

00:04 This is episode 61, recorded January 10th, 2018.

00:10 I'm Michael Kennedy.

00:11 And I'm Brian Okken.

00:12 And we have, again, some really cool stuff we want to share with you, the best of what we found this week in the Python space.

00:18 Before we get to it though, I want to say thank you to DigitalOcean, this podcast, these mp3 files, everything is coming to you directly out of DigitalOcean servers.

00:27 Tell you more about that later.

00:29 Right now i want to talk about dictionaries yeah i want to talk about dictionaries to so thanks to reamon this is a i'm gonna reference brandon roads actually for later on but.

00:38 Raymond head and head and your head injure is probably the person that i have to thank for telling me how cool dictionaries are in the first place cuz he's talked about a minute different icons.

00:48 In the past and basically i mean tons of python is built on top of dictionaries.

00:54 But one of the things that I did not know was how awesome key sharing is.

01:01 And we had one of our listeners on Twitter, Ned Letcher, keyed us in on this and said, "Hey, if you haven't checked it out, you should go look at Brandon Rhodes' 'The Dictionary, Even Mightier' talk from PyCon 2017." And so I did, and yeah, the key sharing thing is like something I didn't know about, which is totally cool.

01:23 It's kind of funky to think that it's not that way until you really think about how it works, right?

01:29 So when you create a class, and you create an instance of it, an object, every one of those has a dunder dict, right?

01:35 And what goes in the keys are the names of the fields of the class and what goes for the values are, well, the values, right?

01:42 So when you say in dunder init, you say self.name equals this, self.id equals that.

01:47 Those are putting entries in that dictionary, right?

01:50 Yeah.

01:50 But the thing is, if I create a million of those, I have a million times the string name as the key and a million times the string ID as the ID.

02:01 So basically, every instance you create stores a separate copy of the names, not just the values, but the names of the fields, which can be super problematic, right?

02:11 The names of the fields are, and in Brandon's talk, he talks about, since it's a dictionary, there can be hash collisions for figuring out where to put it.

02:20 And so each of that happens for every single object has to figure that out.

02:24 I think it was what Python 3.3, 3.4, where they changed it to where.

02:31 So here's the trick.

02:31 As long as you put all of the fields that you're going to use for your class objects, you initialize them with something in your init function, then all of your objects can share the same dictionary for the key lookup.

02:47 Yeah, that's really cool.

02:48 So I think that's quite awesome and should really help with performance as well, especially memory wise.

02:52 Yeah, it's a little bit of a little weird thing of like you have to kind of think about how this is implemented.

02:57 But Brandon does a really good job of walking you through it and so that it's obvious how the savings is.

03:04 And apparently from the PEP 412 that this was started with, for object oriented programs, the memory reduction is like 10 to 20% less using quite a bit less memory.

03:16 So that's definitely worth it.

03:17 Oh yeah, that's always a nice benefit there.

03:19 So I have an interesting project recommended by a listener as well called Python Hunter.

03:27 Have you heard of this?

03:28 Well, just because it was recommended to us, but I haven't looked at it yet.

03:31 Yeah, so it's pretty interesting. The idea is it's a little bit like code coverage.

03:36 And it's a little bit like a debugger, in that you like enable it on your code.

03:41 And then as you interact with the code, you get a report. But what you actually get here is it will give you, basically, as your program runs, it'll show you the internals of basically any module or package you want to point it at as it executes.

04:01 So there's an example where they turn it on and they call os.path.join, and it shows you basically all the internal implementation running, like printed into the console intermixed with your output.

04:14 Oh, interesting.

04:15 Yeah, so if you were like, I would like to explore what this does when I call this function, and I don't want to root through all the source code and try to figure out what piece this is referring to, you can just turn this on, call the function, and it'll show you the path through the external code it took.

04:29 - Okay. - Yeah.

04:30 So pretty interesting. It has some integration with debuggers and things like that.

04:36 So you can also overwrite it.

04:38 And it doesn't happen for every bit of code you call.

04:40 You point it at a module, like say, I'm trying to work on, let's say, POSIX path.

04:46 I want to understand my interaction with that module.

04:49 Then you call os.path.join, and off it goes.

04:51 So, yeah, pretty sweet, actually.

04:53 Like, for instance, if I want to mock out my request interaction to external databases or external sites, I could point it at requests, for instance, to find out exactly what my application is doing with requests.

05:06 Yes, exactly. And you would say, "Oh, this is the part where it goes to the network.

05:09 That's the one thing I have to mock out.

05:12 The rest I can just ignore." That would be a great tool for, especially for people that are trying to isolate bits of their application.

05:18 Yeah, that's cool. So this was recommended to us by Even Page.

05:20 So thank you for that. And it's also kind of interesting that it is built on Cython.

05:25 So it's nice high performance.

05:27 It's always cool to see Cython making an appearance.

05:30 Nice. So you know, I always feel like when I'm working with Bash, I could do a better job.

05:36 And in fact, I don't use Bash that much these days. I use OhMyZSH.

05:40 But same thing, there's all this stuff about the shell I could do better.

05:43 You use ZSH?

05:44 I use OhMyZSH. I love it.

05:46 OhMy?

05:46 OhMy.

05:47 Oh, okay.

05:47 Yeah, it's really got some nice integration with Git, the history and the search and the autocomplete, like a tabbed selection of various things.

05:56 If you have like, you type a few characters and hit tab, it'll show you the list of all the things that you've done involving that.

06:02 It's kind of cool.

06:02 Okay. I'll have to check that out sometime.

06:04 But I think more people use Bash, actually.

06:06 So I went through a phase where I started out learning KSH when in college and in my first job and then i tried bash and i went to z shell for a while but it had support issues on whatever operating system i was using at the time so bash is just everywhere so we can just use it as is the place where you have to jump through a few hoops is on windows so i wanted to bring up before we get into this article there's an article called 10 things i wish i'd known about bash which is a really great little quick start on things that might might speed up. It's just some fun things. There's a somebody trying to sell a book also in it, but it's only like five bucks. So go for it, dude. But kind of things that I'd always wanted to know, like what the difference is between back ticks or the dollar sign per end replacing bits in your command. Just if it is, is they're identical. It's just one's a lot easier to read if you have a whole bunch of them. Yeah, okay, very cool. And then, you know, like globbing versus regular expressions. That's something that confused me at first.

07:08 So that's a good thing.

07:10 Dealing with exit codes.

07:11 The number four is the one I really like the most because that one confused me is when you're using if statements, what's the difference between bracket and bracket bracket?

07:20 And I'm not gonna talk about it here, but these are some good things.

07:24 I also wanted to bring up, I spend a lot of time trying to hunt for a good Bash application for Windows.

07:32 And right now what I'm using the most is Git for Windows.

07:36 and it has Git Bash is embedded in it.

07:39 I think that's great, even for people that are not using Git, because it's just a good bundle of all of your Bash tools, and it just always works.

07:48 So I'm pointing everybody at that now, even if they're not using Git.

07:52 We did have a recommendation from one of our listeners a long time ago about a tool called Commander, I think.

07:58 - Yeah, I'd say Commander, yeah.

07:59 - Yeah, C-M-D-E-R dot net.

08:02 And I think there's a reference on here that I think it's related to Git for Windows somehow.

08:07 So not sure, but if you don't really use all the, if you don't want to try Git for Windows, but you still want some emulator working on Windows, check out Commander also.

08:18 >>Yeah, and it has the Monokai theme and actually looks really nice.

08:22 Commander looks quite cool.

08:23 I saw it on my Windows 10 machine.

08:25 One more thing I'll throw on your list here while you're talking about these is, I don't even remember what version of Windows 10 has this.

08:32 It's not the original, but maybe the one last fall.

08:36 I can't remember what came out, but they've added the Ubuntu subsystem for Windows, which is pretty cool.

08:43 But you have to turn that on in the turn Windows features on and off.

08:46 It's off by default.

08:47 - And I'm glad you brought that up 'cause I want to try to revisit that.

08:50 I did try those and I had issues with being able to launch external applications.

08:56 - Yeah, they're really isolated.

08:58 Like the Ubuntu part and the Windows part, they don't really communicate very well.

09:03 And so it's not the same, I don't think.

09:05 - It's not like you have to do that a lot, but I often just want to open up my text editor from the command line.

09:11 - Yeah, exactly.

09:12 - Things like that.

09:13 - Cool, all right, yeah, those are all nice things to check out, especially on Windows.

09:16 It needs the most help with the shell, actually.

09:19 Cool, so before we get to the next one, which is a bunch of pretty pictures that I hope will help people understand the definition of micro frameworks versus batteries included frameworks, things like that.

09:30 I just want to tell people about DigitalOcean.

09:32 Those guys are doing awesome work over at do.co/python.

09:37 So check that out there.

09:40 This is where I have all my servers.

09:41 I probably have eight servers for the various things all working together now, failover and database servers and whatnot.

09:48 But yeah, it's great, super easy, 30 seconds.

09:51 You got a Buntu machine or what other, there's a bunch of different distributions and pre-configured servers.

09:57 Go over there, set them up, check it out.

09:59 They're doing a bunch of great stuff.

10:00 and it's definitely affordable and reliable and fast.

10:02 - Awesome.

10:03 - Yeah, cool.

10:04 So there's this thing called Snakefood, which is a tool written by Martin Blaise, and it creates Python dependency graphs.

10:14 So I've got one package or one module or one class that depends upon some other thing, and it'll put them in a graph and draw lines between them.

10:23 So that's pretty cool.

10:24 I haven't done a whole lot with it.

10:26 You can combine that with GraphViz, and it'll create these nice visualizations of Python code bases.

10:32 - Hmm, nice.

10:33 - So that's pretty cool.

10:34 And this is not a brand new item, but I had seen it a while ago.

10:37 It's from the Grok code guys.

10:39 I think I had them on Talk Python, like episode eight or something.

10:43 So it's been a while.

10:44 It wasn't about this, but they took this snake food plus graph viz, and they turned it on a variety of the frameworks that we know and love.

10:53 And it's really interesting to pull up the pictures and just look at the relative complexity.

10:59 So if somebody tells you, well, bottle is fast because it's simple, but it doesn't do very much for you, so you gotta bring a lot of pieces in.

11:06 Django is much more full-featured and more complex, but it has a lot of things built in.

11:12 And Flask is somewhere in the middle, and Pyramid, just a little more than that.

11:15 You can look at these pictures and get a sense.

11:17 Do you have the pictures pulled up?

11:18 - Yeah, and definitely, you can totally see that that's the case.

11:22 - Bottle is like incredibly simple.

11:24 I'm actually blown away that it is as simple as it is.

11:27 But you can just tell straight away by looking at it.

11:29 >> It's so elegant. There's no cycles.

11:31 >> Yeah, there's no cycle.

11:33 It's an acyclic directed craft.

11:35 That's quite interesting, isn't it?

11:36 >> Yeah.

11:36 >> Cool. Then Django, how would you describe Django?

11:40 >> It's like if you clean out a comb after it's been used for a year, that's what it looks like.

11:46 >> Yeah, it's a little bit like one of those inkblot tests.

11:49 >> Yeah.

11:49 >> Flask and Pyramid, they're on the same scale.

11:53 Pyramid's got a little bit more going on, but they're definitely not simple, but they're not insane, right?

11:58 They're not like Django level of interaction.

12:01 Then they go and they do this for Celery and RQ as well.

12:04 And you could tell Celery is way more complicated.

12:07 It's more complicated than Pyramid or Flask, by the way.

12:10 But it's also more complicated by a ways than RQ.

12:14 Let's see, what else do they cover?

12:15 They cover Twisted, and Twisted is pretty insane.

12:18 It's just, it's probably more tied together than Django.

12:23 I would say.

12:24 Mercurial, quite interesting as well.

12:26 Request is in here.

12:27 Request has got some pretty nice look, actually.

12:30 - Yeah, it's oddly elegant and pleasing.

12:32 - It is.

12:33 IPython, not so pleasing.

12:34 - No.

12:35 - Anyway, I recommend people pull up this snakefooding link and just have a quick glance through the pictures.

12:40 I think it's quite insightful.

12:41 And, you know, it gives you the technique for applying this to other codebases as well.

12:45 - Yeah, these would be fun things to just print on your wall and tell people that you're using it as a reference.

12:51 - Yeah, absolutely.

12:52 - Pretty cool.

12:54 Yeah, I just follow the graph when I need to know how things work.

12:57 - Yeah.

12:58 - Your next item is about being a good senior engineer.

13:04 - The new year, it's 2018 now.

13:06 There was somebody that posted on Twitter that I'm terrible about references, said, "Hey, it's the new year, you should go read this again." So this is a 2012 article called On Being a Senior Engineer.

13:18 It's from a website called Kitchen Soap, which is funny.

13:22 But anyway, I think there's just a lot of really good advice.

13:25 And I just went through and pulled out a bunch of the headlines, but it's some things that I've been thinking about lately as I'm hiring people and thinking about, you know, always with evaluations and stuff like that.

13:40 What does it mean?

13:40 What does an engineer versus a senior engineer mean?

13:44 And these, a lot of traits, I think, fit well.

13:47 - I think these are all really, really good.

13:49 I mean, maybe you don't, I should cover them all, but like don't practice C-Y-A-E.

13:54 - Yeah, cover your ass engineering.

13:56 - Exactly. - Oh, sorry.

13:57 - No, no, exactly, that's good.

13:59 So I mean, I think that's a really good point.

14:01 Like a lot of people who, when they're starting out, they're trying to like make sure they're never perceived as not knowing everything, right?

14:08 - Right.

14:09 - By the time you're a senior engineer, you've been beaten with the fact that you don't know stuff and new stuff is coming along that you also don't know continuously, right?

14:15 - One of the top ones is seek out constructive criticism of your designs.

14:19 And yeah, you're just trying to find other perspectives to make sure you haven't missed something.

14:25 - One of the I really like is lift the skills and expertise of those around you.

14:29 - Yeah, well, you and I are definitely trying to do that.

14:31 So that's good.

14:32 - I think a lot of people also try to like keep the recognition for themselves when they're new.

14:36 All right.

14:37 But I think one of the biggest skills someone can be providing on a team is actually, anytime someone's get stuck, they kind of float over to that group and help them keep that part of the group and help that person keep moving.

14:49 They can almost write no code, but sort of float around and make sure nobody's stuck and everyone's productive, and that could be a massive boost to the team.

14:57 - Some of the things, like bringing up a couple more items, estimating how long something's gonna take or how much work it is, that's hard.

15:05 It's always hard.

15:06 You know, as you grow in your career, you realize that you can't avoid it, so you just need to embrace it and be okay with giving ballpark estimates to people.

15:14 And then trying to get better at it, keep track of what you did and try to improve.

15:18 The other one that's difficult is trade-offs.

15:21 You know that you can't get all the information before you start working, so making good trade-offs between information and just risks and benefits and making good judgments, those are things you just have to practice.

15:33 Another thing that I wanted to bring up is the, there's a list of cognitive biases, so just knowing that when you're making a decision, there are biases in yourself that you will, you need to be aware of and try to, you know, make sure those are there and compensate somehow.

15:49 - Yeah, these are all really good.

15:50 I like to understand that not all projects are filled with rock star on the stage work, right?

15:55 Like you gotta become a finisher, and a lot of times there's a bunch of just little details nobody likes that are part of that.

16:00 - Probably the first half of my career, most of my job usually was, I would be taking on the things that everybody else didn't wanna work on and forgot about, and just, and wrapping 'em up and finishing.

16:11 But then it looked like I was getting, often getting credit for finishing projects that most of the work was done by other people, they just didn't finish it.

16:18 - Yeah, become a finisher, it's good stuff.

16:20 And then there's finally the 10 commandments of ego-less programming.

16:24 Wanna take us quickly through 'em?

16:25 - Understand and accept that you will make mistakes.

16:28 You are not your code.

16:30 Each of these could be episodes in themselves.

16:32 No matter how much karate you know, someone else always knows more.

16:36 Don't rewrite code without consultation.

16:39 Oh, I'm guilty of that, definitely.

16:41 Treat people who know less than you with respect, deference, and patience.

16:46 The only constant in the world is change.

16:48 The only authority stems from knowledge, not from position.

16:52 Fight for what you believe, but gracefully accept defeat.

16:55 Don't be the coder in the corner, and critique code instead of people.

17:01 Be kind to the coder, but not to the code.

17:03 These are good.

17:04 - Yeah, these are all great.

17:05 So yeah, I really connect with a lot of these.

17:06 These are great, so I'm glad you brought this up.

17:08 - And nowhere in here it says, don't make fun of PHP.

17:11 - No, you could totally make fun of PHP all day long.

17:13 - Yeah. (laughs)

17:14 - But you could also kind of make fun of Python, unfortunately, a little bit for its lack of UI frameworks.

17:19 Like we talked about that last week, right?

17:21 How that's kind of one of its last remaining places where it really quite a bit could be better.

17:28 One of the things that we didn't read above was no empty complaints.

17:32 And one of my rules is I don't like criticizing stuff without offering a better alternative or at least some kind of constructive thing.

17:39 So here's my attempt to backfill my criticism of the lack of UI frameworks with, well, here's where we are, and here's some things that seem like they're working, and a little bit of constructiveness.

17:49 So I went through and I said, all right, well, what are the options for Python UI frameworks, right?

17:53 Like, I'm not loving what's available right now in a lot of different angles, but what can we do?

18:00 We talked TKInter, not super amazing.

18:03 Like, I have a link to an example.

18:05 Brian, pull that up and tell me how you feel about that application.

18:08 But that like, does that look like a classy new app from 2018?

18:14 No, it looks like it's running on Windows 95.

18:15 Yeah, it's like Battleship Gray.

18:17 Looks like it could have easily come off of it.

18:19 It looks like whoever built the original Minesweeper in Windows 3.1.1, they built this.

18:25 Right? And so that's kind of what the current UI looks like in Python, which is, it's a problem, right?

18:31 So what are the options? So we have that, but we have PySide and Qt.

18:35 And actually Qt is pretty nice in terms of how it looks like.

18:39 It really looks like it belongs on the platforms.

18:41 There's a designer for it, a draggy droppy designer type thing, which is pretty good.

18:46 But the license is a little weird. It's like, PyQt is GPL.

18:51 PySide is like an LGPL way to try to go around it.

18:55 But PySide I think is slightly out of version with Qt.

18:58 There's just like real messiness around it.

19:00 But it's still a pretty nice thing.

19:02 There's Kivy, and there's PyGame.

19:05 There's also pyopengl.

19:07 But to me, if you go to like Kivy and you look at the showcase or the examples of people, what they built, they all look like AR type things or games or other types of simulation stuff.

19:19 And I know you said people have created a few things that are not quite like that.

19:22 But it's not really the focus of the platform, right?

19:24 One that actually looks pretty good is wxpython.

19:27 Click on the example for that.

19:28 That's one that I've used before.

19:30 Yeah, this looks pretty good, right?

19:31 This could certainly be at least circa 2005 on Windows.

19:35 It looks pretty different.

19:38 And it has a couple of GUI designers.

19:41 It has these widgets.

19:42 That's pretty cool.

19:43 Yeah, and then there's all the ElectronJS stuff.

19:45 So ElectronJS is basically Node.js plus hosted headless Chrome plus JavaScript and HTML for the front end.

19:54 So there's actually a couple of attempts to create a, what I really think should be called proton.py instead of electron.js.

20:01 a Python equivalent, where the running back end is Python.

20:05 And you could use like Sculpt or something to actually make the JavaScript side also Python, like the Anvil guys are doing, which is pretty cool.

20:12 So there's this thing called Eel, which is a library for making electron-like offline HTML, JavaScript, GUI apps.

20:20 And like I said, you could put Python in the front end with that Sculpt bit.

20:24 And that looks pretty interesting.

20:25 So you get basically like a local app packaged up as a .app or .exe.

20:30 but it's more or less like a web front end that can do local machine Python capabilities.

20:36 So that's pretty cool.

20:37 And then there's a bigger version called CEF Python.

20:41 That name is not amazing, but it's like Chrome Embedded Framework Python.

20:46 And it's the same type of thing. It's a little more advanced, but it's also a little less obvious how to get started with it, I think.

20:52 But both of those look like interesting ElectronJS-like things.

20:56 These are people trying to solve the problem we have, so that's good.

20:59 Yeah, it's good to see them, see them do it.

21:01 And, yeah, last week we talked about possibly having an open session at PyCon in Cleveland in May about this.

21:07 So it'd be really fun for people to get together and join us there.

21:10 I think Paul Hilderbrand is kicking that off, but we can all go and be part of it.

21:15 - Definitely. - Awesome. Well, anyway, here's my research bit I did on Python UI frameworks.

21:20 None of them are like, "Yes, this is the answer," but maybe they'll spark some thinking and get us going in the right direction.

21:25 All right, well, that's all I got for you this week, Brian.

21:27 Anything else from you?

21:28 No, that was a fast six items.

21:30 It was, it's because they're so fun.

21:32 I really like the one that you got about the senior developer stuff.

21:35 It's not totally from 2018, but it's that evergreen material that just lasts, right?

21:40 We talked about quite a bit of those, but every single item that we talked about has a paragraph or more of discussion on the article.

21:47 So please go read that, everybody.

21:49 It'll make you a better person.

21:50 Absolutely. Very cool.

21:51 All right, well, Brian, thanks for sharing this with everyone.

21:53 Thank you. Bye.

21:54 Yep, bye.

21:56 Thank you for listening to Python Bytes. Follow the show on Twitter via @pythonbytes.

22:01 That's Python Bytes as in B-Y-T-E-S. And get the full show notes at pythonbytes.fm.

22:07 If you have a news item you want featured, just visit pythonbytes.fm and send it our way.

22:12 We're always on the lookout for sharing something cool. On behalf of myself and Brian Okken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.