Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.
This is episode 103, recorded November 6, 2018. I'm Michael Kennedy.
And I'm Brian Okken.
Hey, Brian. How you doing?
I am great today.
Yeah, it's another wonderful day. A lot of cool news in the Python space.
I have something that I think you all will really like. I'm looking forward to sharing that.
Cool.
Yeah, and I know you got some good ones, so I'm looking forward to talking about those as well.
Before we do, let's say thank you to DigitalOcean. Check them out at pythonbytes.fm/DigitalOcean.
Get $100 free credit for new users. I'll tell you more about them later.
Right now, you've got kind of this magical C++ Python combination thing queued up for us, right?
I do. And I just, we found out about this because a listener, I think it's Sebastian Srebart.
Wait, Sebastian Srizard. Srizard. Sorry about your last name.
Thank you, Sebastian, for sending that in.
Yeah.
Brian, if anyone else wants to send in some news, you can also butcher their name in honor of them sending that in, right?
Yeah, it's a tradition.
As I would as well. I do a lot too. It's becoming a tradition. Thank you, Sebastian. All right, tell us about this thing.
Well, I think because of the flames in the logo, they actually intended to be pronounced Phoenix.
It's F-E-N-I-C-S. It's a project. I'm glad that Sebastian sort of translated this for us.
It's an open source computing platform for solving partial differential equations.
This is actually really cool. And I'm going to quote right from their site.
Phoenix enables users to quickly translate scientific models into efficient finite element code
with the high-level Python and C++ interfaces to help you get started.
It's got powerful capabilities for experienced programmers, but it's easy to get started.
And it runs on multiple platforms from laptops to high-performance clusters.
And it actually looks really pretty cool for anybody dealing with partial differential equations.
It's a num-focused-backed project.
So there's money behind it, which is cool.
Num-focused seems like it's everywhere these days.
Yeah, well, especially in...
And it's good to highlight that, to say that this is a backed project,
because it's a...
These sorts of things you wouldn't want to depend on and then have them go away.
Right. Some open source projects, depending upon them, are kind of like getting a puppy, right?
They're cute and fun, but then you've got to walk them and stuff like that if they get dropped
or don't work the way you want.
Yeah. And Sebastian said right off the bat, it looks cool, but there's some features
features inside that you might not know about right off the bat.
So I'm going to quote an email that he sent us.
He said,
Phoenix is, in fact, a C++ project with a full-featured Python interface.
The library itself generates C++ code on the fly and be called on the fly from Python.
It's almost magical.
Under the hood, it uses SWIG and recently moved to PyBind 11.
I guess the architecture that was set up to achieve this level of automation might be useful
for other situations.
Yeah. So that's crazy.
You write Python code, this thing writes C++ code, and then calls it all dynamically at runtime.
Yeah. That's amazing.
And from the project website, being able to develop the algorithm locally on whatever computer you're on, even a laptop or desktop, and then deploying the same code to run in parallel on thousands of processes, that's just awesome.
Yeah. I think there's a lot of cool stuff happening here.
I mean, not everyone is solving using finite element methods to solve PDs, right?
I understand that's a limited group, but there's a lot of projects that may find what this project is doing interesting from a performance perspective and a dynamic meets compiled language perspective.
Yeah. And it's also one more example of problems being solved in Python that you wouldn't have thought you could solve with Python because they're just to take too much high performance computing.
That's a super interesting point.
So often I hear people who are not that familiar with Python say, well, Python's slow.
I'm like, hold on.
What do you mean that Python is slow?
Like you have to say doing this operation in Python is slow because there's so many variations.
I mean, it can be, well, Python is slow and CPython, so you can use PyPy, but it could be way more interesting.
Like, well, yes, but you would actually use a, say some kind of library that has C level compilation elements like SQLAlchemy or NumPy or something.
And so when you actually talk about that, like you're doing C, you're not doing Python at the hotspots.
And then you get, you know, way out there with things like Dask and like this and so on.
And it's pretty awesome.
Yeah, definitely.
It's neat to see.
You're kind of a fan of regular expressions, aren't you?
Yes.
I like regular expressions.
Are you a fan of cursive language, like the fancy calligraphy type stuff?
Like, do you write that way often?
Well, I like it.
I don't write that way, but.
I don't write it that way either.
This next project I want to talk about is its goal, I think, primarily is to make regular expressions more easy to indicate their intent and easier to maintain.
I like to joke that regular expressions are kind of a write-only language.
You know, you write them, but then you can't read them anymore.
But they're magic and they do their thing, right?
So we got this project called cursive underscore RE.
And RE is, of course, the regular expression module in Python, right?
Yeah.
This comes from Chris Patty of Podcasting at Fame.
And it's actually created by Bogdan Popa.
My turn to hopefully take a shot at getting the name not too wrong.
But this is a library for doing regular expressions in modern Python.
So 3, 6 and above.
And the idea is instead of writing in Python,
the string symbols like you normally would, you know, bracket 9, 0-9 dot, you know, close bracket dot, that sort of thing,
you write in this higher-level language of combinators, they call them.
And then that overrides the operators in Python to generate a regular expression.
So this is like all sounds kind of wonky and crazy, but if you've seen an example, it's super clear.
So you can go and define, say, like a hex color.
And the way you define it is you say, I would like to create a, you say, beginning of line plus this hash symbol plus group repeated hex digit
or repeated other hex digit exactly three times plus end of line.
And you write it in these things that are symbolic of what regular expressions do.
And they can be, you know, sort of orred and and together and added together and so on.
And then if you call the string representation of them, you get the actual regular expression.
Okay.
So it's a library to build regular expressions.
Yeah.
It's a library to build regular expressions.
The way Bogdan describes it is he says, it's a tiny Python library made up of these combinators to help you write regular expressions that you can read and modify six months down the line.
Yeah.
Definitely one of the problems with regular expressions is they're they're terse and that's good and bad.
You they're too terse sometimes.
So, yeah, sometimes.
And then the other problem is there's a lot of special symbol.
Like regular expressions are basically nothing but symbols.
Right.
They're like a symbol like exploded and its guts came out all over this text.
Right.
Yeah.
But the problem is like some of those symbols are have to be escaped if you actually want to search for them.
But it's always kind of hard to know.
Well, which ones do I have to escape?
So it also does things like if you tell it I'm looking for the text of like square bracket, square bracket, it'll escape properly in regular expression format.
The the text representation of that.
Right.
Because bracket normally means something else.
Like it's a set of characters or something like that.
Yeah.
So I really like that it is sort of a safe way as well.
Like it's a more you talk about what you want and if it has to be escaped or whatever it does.
So, yeah, it's I like this.
This is great.
Yeah.
I do, too.
I can certainly see myself using this if I'm writing regular expressions.
It's great.
Speaking of seeing yourself.
Seeing yourself.
Yes.
Actually, for a long time, I've been following and paying attention to what Adrian Rosebrock.
This is even an easy one.
And I even mass record this.
Adrian Rosebrock.
He has a site called Py Image Search.
And essentially, Adrian is teaching people about OpenCV and Python and actually a lot of AI stuff and doing some really cool things with Python and like cameras and webcams and stuff.
And even on Raspberry Pis and stuff and doing lots of neat things.
And I don't have a particular article to point to.
We just haven't covered it before, I don't think.
And people should know about it.
So Adrian has both paid and free resources to teach people all about computer vision.
And I think he's doing a cool job.
Yeah, he's doing a real cool job.
And there's so many great examples over there.
I think OpenCV is great.
And this is probably the best resource for OpenCV and Python intersected, right?
I get emails from him every now and then with boxes detecting things running around on videos or something like that.
It's great.
So if you've got to do anything with computer vision and you want to use Python to do it and why wouldn't you want that, then this is a cool place, right?
Yeah.
And he has some like tons of cool projects that he's done over the years of, you know, hooking up a Raspberry Pi with a camera to detect people coming to your door and stuff like that.
And it's some cool stuff.
And then you brought up that he had one of the most successful Kickstarters ever.
Yeah, he did a Kickstarter called Deep Learning for Computer Vision with Python.
And yeah, it's definitely one of the most successful Kickstarters ever.
So if you want to check out that book, that's really good.
I think he has some videos that are coming along with it, but I linked to the Kickstarter as well.
Yeah, it did okay.
Okay.
Speaking of doing very, very well, I want to tell everyone about DigitalOcean before we move on.
And I decided it's time to think about this a little bit differently.
You think about DigitalOcean and you're hosting and stuff a little bit differently.
So, you know, most of us, I think you brought this up.
You're not Netflix.
You're not Google.
You're not Facebook, right?
Remember that?
You're not LinkedIn.
Yeah.
You don't need these crazy architectures.
And yet some of the most popular hosting platforms out there, you know, like AWS or Azure,
they are built with thousands and thousands of knobs so that you could be Netflix with 50,000 servers running continuous chaos experiments and all that crazy stuff, right?
But if you are actually just building what 99.9% of us are, more standard applications, then all that stuff is just overhead and complexity and cost.
So you can join companies like Slack, WeWork, Docker, GitLab, and of course, us over at DigitalOcean and pay like five times less than what you would for AWS or Azure, right?
So, for example, bandwidth is one cent instead of nine per gigabyte.
Servers are five bucks instead of, you know, 50 or whatever a reserved instance is over at AWS.
It's not just about price.
Keep it simple.
Use what you need.
Don't have all these crazy, crazy features that you probably don't actually need because you're not building Netflix or Facebook or LinkedIn.
Anyway, try them out at pythonbytes.fm/digital ocean.
Get $100 credit for new users and see why we all love it for their infrastructure.
Awesome.
Thanks.
Indeed.
So, Brian, this next one, have you watched this video here that I'm about to talk about?
I don't think I have.
Maybe I have.
All right.
So, while we're talking, click this and open it up.
Okay.
And I'll tell people about it.
Maybe mute your YouTube that it's going.
Okay.
So, this thing that I want to talk about is a visualization of Python development from original origin way back in the day up until 2012.
And it's using this underlying system called GORCE, G-O-U-R-C-E, GORCE, I'm guessing.
And the idea is GORCE is a visualization library that visualizes trees and stuff like this, like graph trees, not real trees.
And what you can do is you can point it at a source repository.
It could be SVN.
It even supports CVS, but Git and Mercurial.
And you can point it there, and it will do an animated over time visualization of that source library.
So, not just Python.
Any of these repositories you can point it at.
And it will have the files as they grow, the size of the repository, and these little animated characters come in and start editing files and interacting with it.
So, if you play this video that we're linking to, this is a GORCE animation of Python development up until 2012.
And you see it starts out, and Gito is cruising around, adding little bits.
Then a few more people come.
And then if you start skipping further and further ahead, it gives just madness at the end.
People are swirling all over the place.
And it's just a really great way to, like, see the growth of Python visually through animation, at least in terms of who is participating in building it.
What do you think?
Yeah, it is cool.
And one of the fun things to see is that there's sometimes some people that just sort of sit around one area, which makes sense.
And then other people that fly around and edit all sorts of stuff.
Yeah, it's really interesting, right?
And you can see people appear, and then they'll fade away.
They'll come and make some contributions, and then they'll, like, leave the scene.
So, anyway, I think this is really cool.
Yeah.
And it's, you know, there's not a lot to take away from it other than it's just nice to appreciate it.
I would say watch the first minute and then just, like, skip minute by minute and watch a little bit.
Because it's 14 minutes.
You don't really want to watch the whole thing.
I think I do.
But you could just leave it running for the rest of the show.
Yeah.
So, here's what I think.
The call out to the audience.
One, who wants to build this for 2012 to present on Python, you know, again?
And I think this would make an amazing lightning talk if you built that video, and then you, like, went up there and just did a four-minute animation.
Wouldn't that be cool?
It'd be cool to just have that going on in the background while you did some other talk.
Right, yeah.
Or run this at, like, between sessions at a conference.
I think it'd be great.
Anyway, I think people will appreciate checking it out.
Cool.
So, one of the things, Brian, that you hear a lot in software development is that there's often a wide range of skills and productivity between developers.
And I've done a lot of training and interacted with, you know, literally thousands of people in person.
And I think it's something of a myth, but I think largely there's a lot to this.
Yeah.
Some people that just fly and they're just focused and others that just kind of bounce around the keyboard randomly.
What do you think?
So, I know that it's a bit controversial.
There is this notion of a 10x developer.
And often there's a backlash against it also.
But I think people think of it as, like, somebody that's really 10 times better than the average good developer.
And I don't think that's it at all.
I think that it's just a notion that there is sometimes orders of magnitude between the most effective person in an organization and the least effective.
And I don't know how you argue against that.
And if you've ever been at large organizations, it just is.
At least maybe it's not 10 times, but there's definitely – there's a lot to it.
It's a multiplicative factor in there, I would say, for sure.
Yeah.
And so, this is – regardless of what you want to take away, there's some good advice in this article.
There's an article I'm linking to that's what any developer can learn from the best.
And I think these are good things.
So, one of the things – the idea around it is just this isn't magical and it isn't something that is – it isn't just about skills and hard skills.
It's other stuff too.
And there's a clear path to excellence.
People are not born great developers.
They get there through focused, deliberate practice.
And here's a few traits.
They just listed some traits of things that they see in good developers versus not so great.
So, great developers are – a few of the traits are problem solving.
They're skilled at what they're doing.
They're mentors and teachers.
They're excellent learners and passionate about stuff.
So, the problem solver bit I think is really interesting because often the 10x or the multiplier doesn't come in from that they do the same work faster.
It's sometimes they can just look at things differently because of experience, because of playing around with lots of different things and say, oh, let's just solve this problem differently.
And it just gets done faster because they take that – take a different approach.
One of the – I was talking with somebody about databases recently.
And there's some – for instance, there's some problems that can be solved with graph databases easily that are almost impossible with a relational database.
It's just using the right tool for the right job sometimes.
Yeah.
And I think a lot of that is the – it's not that knowing that means you're 10x smarter or 10x better.
That means you are curious enough to keep looking and to continuously be on the lookout for, well, okay, I now totally know all about relational databases.
What else can I learn?
How do I compare it?
And I think that's one of the biggest traits that I've noticed among people that have some number of multiplier of efficiency or skills is not that they are necessarily more skilled or have more natural talent, but they're just continuously learning and really passionate about it.
And they're just – they're just always picking up these little things that help you at each step, right?
Yeah.
And there's an honesty of just like – this isn't just about developers.
I think it's in every field.
For instance, somebody that's working with tools and stuff, somebody that needs to hammer in a lot of nails might be smart enough to go, hey, I should go get a nail gun.
I could do this a lot faster with a nail gun.
You know what that makes me think of?
Like you could say I need to have the rings on my engine or my transmission fixed on my car and you take it to a regular place.
You're like, great.
We'll have it back in three weeks.
You go watch something like IndyCar or Formula One.
They'll pull in and they'll like change the transmission in two laps.
Yeah.
Right?
Those are both mechanics.
Yeah.
Those are not the same.
Right.
Exactly.
The deliberate practice is important.
And I think some people forget that the difference between being an average or a below average developer and being an above average developer is mostly just deciding that you're going to do that.
And setting aside some of your time in your life to pick something you want to improve and then go and do it.
Figure out what you want to learn and go do it.
So.
Yeah.
Anyway.
I guess what I would say is to take away from this article is it's really interesting.
It's numerically based.
Right?
There's a lot of it's based on a survey the guy did with like a thousand folks or something.
And most importantly, it's about a growth mindset.
Right?
It's not to say, well, there's these people and they're just smarter than you or they're smarter than other people or whatever.
It's here's how those people got that way and you can do it too.
And I think that's the right message.
Right.
And the person that wrote the article said this is from a lot of he's taught a lot of people and there's a lot of people that want to be better, but they don't know how to.
So this is just sort of some direction on what things to work on.
So.
Yeah.
It's a good article.
Yep.
And a nice find.
Let's close this out with a bit of chaos.
Sure.
Why change tradition?
Right?
That's right.
So you've heard of, I'm sure you've heard, I'm pretty sure you've been out on your show.
If I remember correctly, you've talked about the chaos monkey and things like that, right?
Yeah.
Tell people what the, yeah, he's from Netflix, right?
He used to be.
Yes.
Okay.
So I think the idea of the chaos monkey originated there.
Tell us what the chaos monkey is.
Well, I'm probably going to get this wrong, but there's this notion of taking parts of your system and intentionally breaking parts or shutting down, especially in a distributed system, taking some nodes and just killing them every once in a while and seeing how your system recovers from it.
Yeah, exactly.
And then in production, it does it.
And then, you know, you just have to build.
So, cause you know, the chaos monkey is coming and it means like, well, a standard failure is like nothing.
Cause the chaos monkey is a madman, right?
He's running around all the time.
So this is not a bad philosophy for large organizations or large bits of software.
However, you're not Netflix, probably people listening.
So how are you going to build that up?
Right.
How are you going to create these things?
So there's this cool thing I found called chaos toolkit.
So chaos toolkit is a library built in Python that will help create these chaos monkey like things.
Cool.
So chaos engineering is what they call is the discipline of experimenting on distributed systems in order to build confidence in the system's capability to withstand turbulent conditions in production.
All right.
So we talked about the chaos monkey and the friends, right?
There's other types of chaos things that Netflix, but here's a way that you can easily build those types of experiments and systems.
And it integrates with Kubernetes, AWS, Google cloud, Microsoft Azure, some other, other things like that.
Right.
So just to give you a sense of what it can do, like if you look at the AWS API, it'll say, you can do things like go to AWS Lambda and call a delete function concurrency that removes the concurrency limit on any specific Lambda.
Or you can go just call stop instance on an EC2 instance or, you know, whatever you want.
Right.
And presumably it's going to put that back.
I'm not entirely sure.
But I guess you probably got to call start instance again on it or set, you know, add function concurrency or set, something like that.
All right.
But there's this infrastructure to help you change these types of settings and these types of things around your cloud providers and, you know, make sure your system can take it.
Nice.
Yeah.
You cannot plan for the best.
So you plan for what you can for.
That's right.
Yeah.
So you build it so that it doesn't have to be perfect.
And then you're in pretty good shape.
The stuff that the folks at Netflix are doing is insane, though.
Like they take it to another level.
Well, yeah.
One of the things I remember talking with them about is the reason why they test and do all this in production is because their system is so large.
You can't.
It's essentially the world.
You cannot.
You can't have a test bed that's similar enough to the real world environment.
So they don't have that luxury.
Right.
So they just test it in production.
And I've not seen Netflix really go down.
So I'm saying they're doing it.
Yeah.
All right.
That's it for our main news.
I have a couple of quick ones.
You got anything you want to share, Brian?
Yeah.
Just on the Python bytes episode 100.
We one of the things we talked about was pyproject.toml.
And I wanted to take a deep dive.
So the last episode of testing code is me talking to Brett Cannon and talking about that for almost an hour or so.
Oh, nice.
I definitely want to check that out.
That's a good one.
So how about you?
I have two other things that I don't think justify a whole segment, but I just want to throw them out there.
Remember a while back we talked about that there was potentially some experiments or something where some college researcher had put some malicious, potentially malicious libraries on PyPI?
Yeah.
Well, it turns out recently someone actually put malicious libraries on PyPI.
So this is October 27, 2018.
And apparently 12 packages were discovered with various levels of vulnerabilities and stuff.
So I'm linking to that.
People should check that out.
If you don't know how to spell Django, you're going to have a bad time, by the way.
Yeah.
So it's a lot of this typo squatting like, oh, I forgot the J, so now I have a virus.
Something like that.
Yeah.
That's sort of lame.
Yeah.
It's upsetting.
At least they've done some work over at PyPI to block properly spelled things that are not actually packages, like RE, for example, right?
The built-in stuff.
Well, yeah.
That's one of the things that's, yeah, was an issue is people can try to pip install things that are in their standard library.
You don't have to do that.
Yes, exactly.
Exactly.
Don't do that.
Then the other one was just a quick little Twitter message that someone sent my way.
And it's pretty cool.
This guy, not the one who sent it to me.
I'm sorry.
I don't remember who sent it.
I should have written it down.
But this person called Xtrek, something to that effect, on Twitter, decided to go scanning the standard library source code for interesting things like, what is the longest class name?
What is the longest function name?
And so on.
So they found out that actually in CPython, the longest class name is 200 characters, just the letter A 200 times, which is some kind of test case.
But for the real ones, there's one called test, mutually exclusive optionals and positions, mixed parent, as the longest class name is there.
In the longest function name, they believe it's test underscore parser underscore regression underscore special underscore character underscore in parameter column of doctrine first line, which is 84 characters long.
And then there's other examples.
Someone says there's actually a test C types that has 33 million characters in it.
Anyway, it's an interesting thread if you're just wondering what the really long names are.
Yeah.
It's almost competitive.
It almost could compete with Java names.
Yeah, exactly.
Standard types right there.
Awesome.
All right.
Well, definitely fun to share all this news with you, Brian.
As always, thanks for doing it.
All right.
Thank you.
Bye.
You bet.
Bye.
Thank you for listening to Python Bytes.
Follow the show on Twitter via at Python Bytes.
That's Python Bytes as in B-Y-T-E-S.
And get the full show notes at Python Bytes.fm.
If you have a news item you want featured, just visit Python Bytes.fm and send it our way.
We're always on the lookout for sharing something cool.
On behalf of myself and Brian Okken, this is Michael Kennedy.
Thank you for listening and sharing this podcast with your friends and colleagues.