« Return to show page
Transcript for Episode #49:
Your technical skills are obsolete: now what?
Michael KENNEDY: Hello and welcome to Python Bytes, where we bring Python news and headlines directly to your earbuds. This is Episode #49, recorded October 24th, 2017. I’m Michael Kennedy.
Brian OKKEN: And I’m Brian OKKEN.
KENNEDY: We have yet another six more amazing news items, actually tons of really cool stuff to share with you. But before we get to them, I want to say thanks to DigitialOcean for creating Spaces and sponsoring this podcast to tell you about Spaces. Spaces are like S3 but way, way better. Check them out at do.co/python. I’ll tell you more about that later. Right now, I want to go watch some videos.
OKKE: Yeah. There’s a lot of conferences that go on around the world. Anybody that’s doing a Python conference, I have no problem with trying to highlight those so let us know when they’re going on and when the videos are up. That’s what I want to talk about right now. There’s a couple conferences who have videos available now. DjangoCon and PyGotham both happened recently and both of their videos are available, so we’ve got links to both. PyGotham, I don’t know if it’s up on YouTube or not, but it’s up on pyvideo.org. DjangoCon 2017 is up on YouTube and I’ve got links to both those playlists.
I was excited to look at a lot of these and one of the videos I started off with was a video from DjangoCon called, “Django Vs. Flask” and it’s by David “DB” Baumgold.
KENNEDY: That sounds fun.
OKKEN: It actually was interesting because it’s somebody that respects both Django and Flask. It was probably one of the best Flask videos I’ve watched, talking about how to deal, how to do some of the things in you can do in Django right out of the box, but how to do it in Flask, and comparing those. It’s cool.
KENNEDY: That sounds like a really cool video. It’s one thing to go study Django, it’s one thing to learn Flask, but to see the features side by side sounds like a really nice way. If you’re new and trying to decide, ‘Should I go with Flask or should I go with Django?’ This is like, half an hour, 45 minutes. You’ll have a really good idea rather than spending hours learning each and then deciding, right?
OKKEN: It sounds like he likes the flexibility of Flask, however, it’s obvious that he has respect for both and he’s talking at a Django conference, of course. There’s some things I didn’t know about, which is cool. I didn’t know about Flask Blueprints and I’ve definitely never heard of Flask Marshmallow for APIs before.
KENNEDY: (Laughs) Marshmallow. Yeah, that’s great. And it covers things like SQLAlchemy, MongoEngine, Peewee, all the great ODM (Object Document Mapper)/ORM (Object Relational Mapper) data access layers. It’s cool.
OKKEN: Yeah, but there’s so many other great videos up from those two conferences and I really appreciate that part of the Python culture that we try to make the conference videos available for people.
KENNEDY: It’s great. I love that these conferences are so quick about getting the videos up. The videos are really high-quality usually. The sound is good, everything’s in focus. A lot of times it’s picture-in-picture. Well done. It’s nice.
OKKEN: What do you have first, Michael?
KENNEDY: So, it’s pretty cool. There was a tweet from one of the guys there, @llanga, I think he works there. I’m pretty sure he works there from the meaning of the tweet. He was talking about Facebook and said, ‘Hey, Python 3.6.3, which released on Tuesday, and I think last Tuesday you covered that, which is pretty awesome.’ Maybe this was two weeks ago. When it was released though, they said this tweet was three days after that. It said, ‘All the machines at Facebook are already running Python 3.6.3.’
OKKEN: That’s cool.
KENNEDY: In three days. And they said that 36.3% of their Python apps are based on 3.6 already. How about that?
OKKEN: That’s great. Man, I love 3.6 now.
KENNEDY: I do, too. The f-strings are just so delightful. So, this is super interesting. What’s more interesting is, I would say – I’m forgetting the numbers – I’ll have to just send you guys to the video and let you pull the numbers out. But a couple years previously, Facebook was not friendly to Python 3. Basically, they had just gotten access to Python 2.7 from 2.6. That was the big upgrade. And all the recommendations were to use Python 2, the onboarding processes said, ‘Hey, when you get here, you’re expected to write in Python 2, et cetera, et cetera.’ And this guy named Jason Fried – not of 37signals fame but a different Jason Fried – took over the process of trying to shift the organizational culture to valuing Python 3 as the default, and making people who do Python 2 look outdated and funky.
So, there's this really great presentation called, “Rules for Radicals: Changing the Culture of Python at Facebook.” And it’s a step-by-step playbook basically of how he transformed the culture to where we’re in this place where you say, ‘Python 3.6.3 was released on Tuesday, Facebook machines are now running it.’ And that tweet comes out on Friday.
This is not, ‘Well, Facebook's a startup it can always do like…’ No, it was this huge organization transformed through clear, simple, repeatable steps. So, I think this is just a really cool evidence of Jason Fried’s work and well done, Jason and everyone at Facebook.
OKKEN: Yeah, I can’t wait to watch this video. Thanks for linking to it.
KENNEDY: Yeah, it’s super cool. It was at PyOhio a couple years ago.
And in more modern Python news, Ubuntu 17.10 just came out and I upgraded one of my servers recently to it. It says:
“Python 2 is no longer installed by default. Python 3 has been updated to 3.6.”
How about that?
OKKEN: That’s great.
KENNEDY: Another step forward. Alright, so modern Python is on the March.
OKKEN: And 3.3 is End Of Life probably by the time this comes out.
KENNEDY: It’s already End Of Life so, if you have something on 3.3 it's time to move up. And if you’re going to move up, don’t go to 3.4, go to 3.6.
OKKEN: Speaking of new technologies, there was a pretty good article, not Python specific, called, “Your Technical Skills are Obsolete: Now What?” It’s actually a pretty fun article. Since you and I are big proponents of people keeping their skills up to date and learning new things, I thought it fit in nice. This article is focused on how to do that while at work, to not try to eat into your home life to bed up your skills.
KENNEDY: I think there’s a lot of people who feel kind of in a tough situation because they see all these new technologies coming out and they hear presentations or people saying, ‘Oh, the stuff that you do is so old and crappy. You should be doing whatever whiz bang new thing it is.’These recommendations and this excitement, probably rightly so, comes from people who are young, maybe don’t have kids, or they’re single and have a lot of free time. To spend an extra two hours after work on some project is no big deal if you’re young and got lots of free time and energy. But if you’re getting dinner for the kids and you’ve been up early driving them to school and have to work with homework, it’s a whole other level, right?
OKKEN: I think it’s a great idea to try to fit that into your work if you can. One of the things is that I really like that he starts off by talking about the reasons, sometimes good reasons why to stay with an old technology. However, if Facebook can change in a few days, what’s your company’s excuse?
KENNEDY: (Laughs) That’s right.
OKKEN: Seriously, though. Sometimes there’s good reasons to not jump on the bandwagon right away. He (Itamar Turner-Trauring) goes through some steps of identifying obsolete and problematic technologies and trying to do some research on potential replacements and then trying to get buy-in from your management, hopefully to try to get some time for yourself, or maybe for you and some other people to do a pilot project exploring this process.
KENNEDY: The main message of this was, ‘Look, if you want to find a way to learn new skills, you need to fit it into work often, unless you just want to put in a lot of energy to learn something outside of work. But if you’re going to try to fit it in work, go and identify a problematic area of your technology stack and create a plan on how to solve that problem with the new technology by you, thereby you learn the new technology by solving the problem. So, you’re actually adding value, not just saying, ‘Please send me training, you haven't sent me in six months.’
OKKEN: He does talk about how to compose those questions and talk to your manager about making the change. But he kind of ends with, if you’re boss says no, then you at least have some extra skills that you can apply to your next job. I’d like to shim in there and say, I think you should use it to start a conversation with your boss or your management chain about that you care about the company, you care about increasing the skills of both yourself and the others on your team and you can go about doing that. And, ‘Why the no?’ Is it bad timing, or you just have a bad proposal or too much time spent? I think that you should try not to throw your company under the bus right away and try to work a little harder.
I think actually, the video you linked to with the “Rules for Radicals” probably fits right in here too, of trying to change your company.
KENNEDY: Yeah, absolutely. Jason didn’t just go, ‘Well, they said Python 2, so I’m going to go find a startup. Forget this.’ He actually changed Facebook, which is even more impressive.
OKKEN: And then a book, it’s a little tiny read, a book that I read a couple years ago called Team Geek that has some great advice on this. It says things like changing your process, you probably ought to limit it to under 50% of your time. I would say stronger, I would say 10 to 20% of your time at most, trying to fix things.
And then the other thing is there’s a nice section on ‘Sometimes it’s easier to ask for forgiveness than permission.’ You can try to get your manager buy-in, but also, if you’re not spending much time on it, you might be able to just change some things and make it better. Like one of those disclaimers, ‘My advice cannot be taken as legal advice and you should seek your own legal counsel.’
KENNEDY: (Laughs) Or ‘Your conditions may vary.’ I think it’s really good advice and this article is really practical, especially for people who are not trying to spend an extra hour a day outside of work learning something.
That's really cool. I feel like there’s a lot of slack in big organizations already. You get put into a lot of meetings. There’s time between the actual work that seems to get done and if you can just wall off like a little bit of that for learning and working on a project. It could be a Skunk Works thing, ‘I’m trying to prototype something different or just learning something.’ But if you could say, find an extra half hour a day to be studying something in the context of your work that you’re trying to learn, that actually could make a huge difference over a couple of months. That’s usually pretty good to do.
OKKEN: I completely value in my employees somebody who really wants to increase their skill set and also cares about saving the company money and making things better. So, at the very least your management chain should know that you care about that, too. They should care about that.
KENNEDY: People who care about this stuff and are actively working on it, those are the type of people you want. Ones that don’t complain, that are just going to work on that 20-year old technology. And that’s fine if they’re great employees and doing that, but that’s not going to push you guys forward. So, I think people should see this as a positive. Awesome.
Speaking of making things better, DigitalOcean has made online cloud storage for apps much, much better with a thing called Spaces. Check out Spaces at do.co/python. The idea is that it’s a similar service to Amazon S3. You have a cloud storage and have either private or public things stored up there, huge amounts of data. If your app say, processes a bunch of pictures and you’ve got to store the pictures somewhere, instead of jamming them in the database, put the gigs of pictures up in Spaces.
It’s really easy to adopt because it uses the same APIs as S3, so tools like the Transmit app I use on my Mac, I just point at the URL for Spaces and I built an S3 thing and it just works. So, that’s super cool and easy to do. The pricing is so much better than S3, $5 a month flat for 250 gigs of storage, a terabyte of outbound bandwidth, which would be $90 alone on S3. And then it’s just one cent per gigabyte instead of nine. Definitely check it out. I’m using for this podcast and many other things. The audio you’re listening to came over it, so how about that. It’s super easy to use and it’s a great deal and very predictable. So, do.co/python and check out DigitalOcean Spaces.
Let’s talk about visualizing things, as well here. So, last week you brought up that deep dive inside of garbage collection (GC). That was a super good article and I really appreciated getting a look at how the reference counting GC worked. About how the actual make sweep garbage collector that would solve the cycle problem worked, how Python uses a combination of them. A lot of the really nice details about, ‘At this number is behaves differently if it’s bigger or smaller.’ These little cut-offs are always fun to know.
There are many different garbage collection algorithms, of which reference counting and market sweep are just one. So, this guy named Ken Fox – it’s not a brand new article or anything, but it’s really a cool follow-up – wrote this thing called, “Visualizing Garbage Collection Algorithms.” And if you liked the last article you should definitely check out this one.
OKKEN: The little videos on visualizing this are kind of mesmerizing. I kind of want to leave one up on my screen or something. They’re great.
KENNEDY: Exactly. If you could have a screensaver of visualized GCs, that would be beautiful. (Laughs) We need that company that made the toasters, the flying toasters. We need them to make that, right.
OKKEN: Even though I covered it last week, I’m trying to remember. Python is both a reference counting collector and a mark sweep collector, right?
KENNEDY: Right. By default, Python tries to use reference counting. So, every time you have a pointer to an object, it increments the counter and once that counter goes back down to zero, then it just deletes it. The problem with all reference counting collectors are if you have any form of cycle, that’s straight up a memory leak right there. Parent-child, child-parent, boom. Never goes to zero, never deleted. That can be a huge problem. So, there’s ways to solve that. Usually that’s some other kind of managed garbage collector, right, and that’s why they bring in the market sweep one. That one goes and finds these cycles and gets rid of them.
OKKEN: One of the questions I have, I don’t expect you to have the answer, but I’m about of fragmentation becomes a problem within Python or not. I think it does. You can actually see it in the picture.
Let’s go through the five visualized here. There’s actually five different types of GC. There’s No GC, which means we’re just going to allocate and not clean up stuff. Hopefully, we won’t spend too much memory and it will be fine. That’s actually a super efficient way as long as you have enough memory. It might sound outrageous but Apache web server actually creates small pools of memory per request and then just throws them away and doesn’t try to manage that memory. I guess in small doses that could work. There’s the reference counting one, and that’s Python’s first pass, as we talked about. There’s a mark and sweep collector, which goes and says, ‘I’m going to start at all the known pointers that I have. Follow them to the things they point to.’ Follow all that sort of transitive closure pointers and indicate, or mark everything that is not garbage. It’s like a not-garbage Finder and whatever it doesn’t find, well, that’s garbage. And then there’s some stuff more like Java and .Net and those types of things that are mark and compact collectors.
So, you asked about the fragmentation, as far as I know, it doesn’t move the memory around and change where the pointers point. After it does a collection, it just cleans up the cycles. So, if you had like a 10-byte object in between two things that got cleaned up, that 10 bytes is just a whole. Maybe you can fill it, maybe you can’t. But the compacting ones, those will squish the memory back together. And what’s interesting about the compacting ones, allocation in a compacting collector is insanely fast. Probably faster than C++ or C, mainly because you don’t have to traverse that list of open slots where you’re memory might fit, you just always allocate at the end because you’ve compacted it down. Allocate means increment the pointer. You want 10 bytes? You increment the pointer by 10. Done. (Laughs) It’s pretty interesting.
There’s a couple of kinds and the list goes through and visualizes all of these and you can go through and actually see the effect and see the fragmentation that you were talking about and the reference counting one. But it’s pretty cool this set of visualizations.
KENNEDY: You know, I feel like I should be using pathlib more than I do, and I just do os.path.join and that sort of stuff, one of the older styles of working with paths in Python. And that’s worked for me, but this article by Doug Hellman that you pulled out here, this is pretty cool. It makes a solid case.
OKKEN: Yeah. Actually, when did pathlib come in? Supposedly, it was introduced in Python 3.4, so it’s a Python 3 new thing. It’s an object-oriented system to work with paths, paths as objects. We’re pointing to Doug Hellman’s module of the week, the Python 3 version, and it’s a very quick read. You can skim over all this stuff you can do.
Yeah, I learned things just the other day. I’ve been thinking, oh, my gosh, I’ve been using OS too much. I should use this more.
KENNEDY: Yeah, for sure. And I was really blown away at the way you the division to hack the visual. The way you use the divider / (forward slash), so you can have one path/the other path and it just joins them, as if it was this thing/that thing in an actual string, right?
OKKEN: And I use that division operator with either path objects or just some strings and characters in quotes to add things to you path. I didn’t know that it had .glob() in there, so that’s pretty cool. And .iterdir()s, .iterdir() for iterating over files in a directory. That’s pretty nice.
KENNEDY: Right. One of the things that’s annoying about os.path is, if you want to actually create directories or delete them or stuff, there’s all these unrelated ways. Like, you go out to other areas, like os.mkdir, rather than OS.path.mkdir. Just funky inconsistencies and this brings it all together. You create and delete directories and things like that.
OKKEN: And then there’s some top level stuff for concrete paths that are short cuts, like path.home and path.cwd that just work whether you’re on Windows or a Linux-based POSIX system.
KENNEDY: Yep, that’s really nice. Cool, yeah. Speaking of learning new things, here’s something people can learn. This is great.
So, we talked about visualization before but can computer’s see?
OKKEN: Yeah, sure.
KENNEDY: I think they can. I mean cars, we are surprisingly close to a world where cars can drive on their own. Are you ready?
OKKEN: Yeah, totally ready.
KENNEDY: We’ve got a whole bunch of deep learning and stuff happening. NVIDIA made a car, I think we talked about this before, where they just had it observe humans and let it go drive. They don’t know how it knows how to drive, but it can. They didn’t teach it to drive, it just learned.
So, I want to highlight this thing called LUMINOTH: An Open Source Computer Vision Toolkit. This is a new computer vision toolkit for Python and it used deep learning to understand computer vision. Basically, its main job right now is to work on object detection and classification, but they’re trying to do a lot more. That’s the first problem they solved.
It’s built with Python, it used Tensor Flow and Sonnet, which is Google’s deep learning framework and DeepMind graph library. So, these are really cool. This is the same thing that Google is using, very likely for things like its AI that just beat GO, or it beat GO before but they just announced that there’s a new AI that beat GO without anyone teaching it how to play GO. That’s the news this week as well, by the way. I don’t have a link to the article.
You can super easily train this thing. There’s like a couple of lines of code and you can start asking it to find stuff in objects. It will say, ‘That’s a bike.’ ‘That’s a hand.’ ‘That’s a person over here. No, that’s three people.’ Things like that. It’s really cool and it uses this modern, state-of-the-art algorithm called Region-based Convolutional Neural Networks, R-CNNs. It even comes with GPGPU (General Purpose Computing on Graphics Processing Units) support and it’s cool. They’re even going to send pre-trained checkpoints on objects that are already pre-classified with some really large datasets from other challenges.
OKKEN: Oh, wow. That’s great.
KENNEDY: Isn’t that cool? So, if you want to go and create a project that does some sort of image classification or identifies objects in images, check this out because if it comes with pre-trained algorithms already and you have a similar type of data as what they already trained it on, you might be able to just pick this up and use it.
OKKEN: One of the examples it shows is detecting dinosaurs in a picture.
KENNEDY: That’s awesome. So, you can build like a home security?
OKKEN: Yeah, it’s useful so you can find out if there’s a dinosaur at the door.
KENNEDY: (Laughs) ‘Don’t open it. No, it’s a dinosaur.’
For some reason, I’m fixated on doorbells. It’d be great if you could come up with a doorbell that rings differently based on what it detects.
OKKEN: Oh, nice.
KENNEDY: If it detects a kid, like one of your kids’ friends is at the door, it could make one kind of noise. If it’s a UPS guy, it could make a package noise. Wouldn’t that really be a cool project?
OKKEN: Yeah, like a big, happy sound if it’s a pizza delivery guy.
KENNEDY: Yes. That sounds like a very good business. Let’s do that. Anyway, if you guys are into image recognition, check this out. It’s a cool project, LUMINOTH. It’s fairly new.
You’ve got a bonus for us, right?
OKKEN: Yeah, I was going to put this as one of my picks. Anyway, there’s a bonus article called, “The Cleaning Hand of Pytest.” It’s somebody else’s experience of what it was like to work in unitest and knows after using pytest. At the end, he links to my book, which is good. Check that out.
KENNEDY: That’s nice. That’s some nice validation. “The Cleaning Hand of Pytest.” That sounds really fun.
OKKEN: How about you? What’s going on with you?
KENNEDY: Not much. I wanted to highlight two things. You mentioned the various PyCons going on and PyTennesee 2018 is coming up. So, if you’re anywhere near Nashville and you want to go do a PyCon and you don’t want to wait until May to go to the big PyCon in Cincinnati – I think it’s somewhere in Ohio this year – you can go February 10th and 11th to the one in Nashville, PyTN. So, check that out. That’s really awesome.
OKKEN: That’d be fun.
KENNEDY: Yeah, I was thinking of trying to go but it turns out my wife is already out of town that week and with children, we can’t both be out of town the same week. So, that squashed that.
The other thing I want to point out is a movie that’s not super new, it’s a couple years old, but I think the people in this audience would really appreciate it. Are you into gaming, Brian?
KENNEDY: I’m not either so I’m kind of oblivious to this world and what has happened to it in the last five years, but there’s this movie called All Work and All Play and it's available on Netflix. I linked to the trailer. It is a really interesting look inside what is going on in the world of professional gaming. The World Championship of Gaming they had in Poland, and there’s some big upsets and surprises and I think they said 100,000 people attended in person. It’s insane. So, if you haven’t been paying attention to this, this movie is pretty eye-opening.
OKKEN: So, this is people playing video games?
KENNEDY: This is a competition of League of Legends players. I think there were six teams, no eight teams, I think. And these are people watching the games being played, live in person.
OKKEN: You’re not a gamer wither? We found two nerds talking to each other and both of them are not gamers? What are the odds?
KENNEDY: I know, it’s pretty low I guess. My pastime is to work on software projects, that’s my game, solving software problems not made up game problems.
OKKEN: Well, cool. Thanks, a lot.
KENNEDY: You bet. I’ll catch you next week. Thanks everyone for listening.
Thank you for listening to Python Bytes. Follow the show on Twitter via @pythonbytes. Get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way. We’re always on the lookout for sharing something cool. On behalf of myself and Brian Okken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.