Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.
This is episode 96, recorded September 19th, 2018. I'm Michael Kennedy.
And I'm Brian Okken.
Hey, Brian. How are you doing?
I'm doing really good. How are you?
I'm doing great. There's always something to talk about in the Python space. It never gets boring.
Yeah, no.
Definitely doesn't. And we're here bringing this to you today in part because of DigitalOcean sponsoring this episode.
So check them out at pythonbytes.fm/DigitalOcean and get $100 credit for some server goodies.
Tell you more about that later.
Brian, you found an interesting one. I don't even know what to make of it here. Tell us about it.
I can't even remember where I came across this. I guess it doesn't matter.
There's a Python library called Plumbum, and that's spelled P-L-U-M-B-U-M.
I have no idea where they came up with that name.
But it's a toolbox of goodies to do shell-like things in Python.
It says shell combinators and more. I'm not sure what a combinator is.
Oh, there was that Y combinator on Back to the Future.
Other than that, I don't know what a combinator is.
Is it like the pipes between things?
Maybe.
Perhaps.
Anyway, the motto of the library, and this is a direct quote, is never write shell scripts again.
And thus, it attempts to mimic the shell syntax.
Oh, shell combinators is shell syntax, where it makes sense while keeping all the Pythonic and cross-platform goodiness of Python.
And it's kind of fun.
And it's one of those things, it's kind of cheating, actually, I think.
But it's cool.
We have an example in the show notes, but there's a bunch of examples on their website, on the readme and stuff.
You can do things like chaining commands with pipes.
And there's a built-in grep and word count and head counts within this library.
So if you've got a bash shell script and somebody says, hey, you should convert that to Python, this would take you like five minutes to convert it to Python using this.
And that's not necessarily a bad thing.
If you know kind of how to transform something here, you can convert it right away.
And then maybe if you want to go figure out how to do that with other ways, you could.
But for a lot of quick and dirty things, why not do something like this?
It looks really cool.
Like, certainly writing bash stuff, the more tests you have to do and the, like, looping over things.
It's just like, how do you do that in bash again?
You know, it's just, it's not so nice.
But there's certainly things you can do in the shell that are, like, painful in Python.
Like, show me the files here.
It's like, okay, well, fine.
We're going to import OS.
And we're going to, like, do a lister and loop over that.
Right?
Like, LS.
Right?
That's super easy.
And this kind of brings those things together.
It's really nice.
Yeah.
So I've got a couple things I'm going to throw this out right away is we're converting from on, in Jenkins, you can do shell commands within, like, the little command steps of Jenkins.
But if you're doing pipelines on Windows, you can't do bash for some reason.
Or I can't figure out a way to do bash.
But I can do Python.
So I could write a little Python script to do the thing that I wanted to do in bash and do it right there.
So I'll try that.
Yeah, nice.
So this is a really cool project.
And certainly if you're thinking about some kind of bash script or something like that, this is a really nice option.
Maybe I should just try to use the Linux subsystem on Windows 10 instead of...
Yeah.
So if you have a problem with getting bash to work, it doesn't work on Windows, right?
But you can definitely bring in the Windows 10 Linux subsystem.
And obviously that's Windows 10.
I'm sure that it's available on the Windows server equivalent of Windows 10, which I honestly don't even know what version that is.
But there's probably a server equivalent that has the Linux subsystem as well.
People can throw that in the show comments if they want.
But there's an article by Marcus Sherman called the Windows 10 Linux subsystem for Python developers.
First of all, have you heard of this Windows 10 Linux subsystem?
Do you know what this is?
Yeah, I've been trying to keep an eye on it.
And as they make fixes to it and add features, I've been trying some of these things out.
Yeah, it's interesting.
So it's basically Linux embedded inside of Windows.
I mean, you've always been able to run in Windows like a VMware virtual machine or something and run, say, Ubuntu.
But that's not what this is.
It's not like a separate, entirely separate running VM.
But it's like a little tiny micro VM type thing that is the command line shell for Ubuntu.
So you can basically have Ubuntu commands in an Ubuntu terminal in your Windows.
The idea is somebody says, hey, here is some library you want to use.
And the way you get it is you compile it from source and run it.
Or, you know, you need to have this other weird utility or type these commands, right?
Like, how often have you seen a tutorial that says, okay, Brian, the next step is to type Python 3-mvnv.env.
And then you say source.env bin activate.
That's awesome, right?
It works fine on your Mac.
It works fine on my Mac.
It works fine on my Linux server.
Does it work fine on Windows?
No, you have to use the scripts directory.
Scripts directory.
There is no Python 3.
There is no source.
Like, what is all this stuff, right?
It's like, why is it different?
And so the motivation that Marcus had for writing this is like, hey, if you're going to learn Python and you're on Windows, you might consider using the Ubuntu subsystem.
Because then all those commands that everybody tells you to do, they just work.
Oh, that's a good idea.
You could use, say, Anaconda, and you get all the tools.
But you still can't type Python 3.
You still can't type source.
Like, all these things don't work unless you run Ubuntu for Windows.
So it's a really nice step-by-step way to go through it specifically for setting up for Python development on Windows.
Okay, cool.
Neat.
Yeah, pretty sweet, right?
So you might say it's a little cheat sheet for getting Ubuntu work in there, right?
Yeah.
Hey, we're doing really good on the whole transition thing.
That's working.
Yeah.
We've got a cheat sheet on my next item.
I've been trying to do type hints a little bit more, especially, so we've got a lot of the people at work and myself are using PyCharm.
And the little pop-up dial, I don't know what that's called.
Autocomplete, drop-down, and type dot.
Yeah, all that stuff works really great.
And it also works better if you have type hints in your API.
It's just two things.
So I've been trying to do, at least for the external functions, I've been trying to do type hints in there.
But I forget some things.
So I ran across this type hints cheat sheet, and it's actually on the mypy documentation site.
But the things that, I mean, I remember some of the easy things, like how do you represent string and int and stuff like that.
Colon int.
What's the problem?
But what about duct type stuff?
Like if I have a thing that acts like an array, but it might be a tuple or it might be a list or it might be something else that just,
I just know that I need to be able to use the bracket index operator and find the length.
Well, that's a sequence for type hints.
But I don't always remember that if that's an iterator, iterable or sequence or things like that.
So those duct type things are what I'm using it for the most.
But I have other people that I work with that are just sort of learning this type hint thing.
So all of it's useful.
Yeah, and it's not that long.
You know, a scroll bar estimation is probably seven or eight pages.
Flip through that once and you pretty much have mastered the type hints.
There's a couple interesting things in here that I thought was nice.
Talking about adding type hints for asynchronous functions.
So async def.
The one that I had never seen was class level variables.
So class var looks pretty interesting.
Like it's not on the instance, but it's on the type.
Oh, interesting.
You can use class var annotation to declare a class variable.
I've never done that.
I'm just, you know, that's pretty interesting.
And then also specifying the type for, say, a generator or something that yields some kind of iterator.
It's pretty interesting.
There's also a generator, actual generator type decoration that takes three arguments that's not covered in here.
So that's also potentially worth noting.
But yeah, very cool stuff.
And I think this is a nice find.
Cool.
Nice.
Now, before we get on to the next thing, I want to tell you quickly about DigitalOcean.
Like I said, they're sponsoring the show.
And they have lots of good reasons to go check them out.
Something that they just announced that's coming soon and is available for early access is Kubernetes.
So if you have some kind of site that you would like to run on a Kubernetes cluster, you can now go first visit pythonbytes.fm/DigitalOcean.
Create an account so you get the free $100 credit.
And then go to DigitalOcean.com slash products slash Kubernetes.
And you can sign up for early access to check that out.
So then you can submit containers to run on the Kubernetes cluster and have simple zero downtime deployments.
That's actually very, very cool.
Yeah.
I'm pretty excited about that coming along.
That's going to be fun.
So every now and then we cover these sort of higher level topics.
A little softer.
Here's one that tries to look a little bit farther in the future.
And I'm just going to put these out there and then I'll let you give me your, you react to this article.
Okay.
So the overall theme here is that Python is driving the creation and foundation of new languages.
Okay.
So this was an article and it says, here are five predictions for what programming will look like in 10 years.
Programming will be, okay.
So number one, programming will be more abstract.
When I first read that, I'm like, oh, please don't tell me it's drag and drop.
UML again or something like this, right?
But no, they say that trends like serverless technology containers, we just talked about Kubernetes,
and low code platforms suggest that developers may be able to work at higher level of abstractions in the future,
not thinking about servers, for example.
And that sounds reasonable.
Yeah, to some extent.
Yeah.
Potentially.
AI will become a part of every developer's toolkit, but it won't replace them.
What do you think?
I don't buy that actually, but okay.
You have some comment I could say to some of the voice assistants,
but I'm pretty sure I'll set off like thousands of them, so I won't do it.
All right.
A universal programming language will arise.
Well, Python.
It's already arisen.
So, yeah.
So, the reason this whole article was pretty interesting is it has a pretty strong Python influence.
So, for example, at the beginning it says, look, just in the last few years,
some languages have risen and some have fallen.
And like the have risen links to an interesting article about Python,
the have fallen links to an interesting one about Ruby, for example.
And then under this section, there's a nice quote that says,
Python may be remembered as being the great, great, great grandmother of languages,
of the languages of the future, which underneath the hood may look like English,
but are actually far easier to use.
Okay.
Yeah.
Okay.
You ready for my reaction yet, or you got more?
I'm ready.
You can bring it.
I think some of the things are, one of the jokes about Python sometimes is that it's runnable pseudocode.
I don't like it.
I don't actually want it to get closer to English because English has so many subtleties and weird things that you can do that have implied meaning
that the closer we get to the actual English language, actually the harder it is to actually debug something.
So having it not be that close, I think is better because it's less up for interpretation.
Right.
Please read that sentence.
The people over there have already read the red sign, you know, things like that.
Yeah.
I'm not so sure that a universal language will arise either.
There's always something you may want to take most advantage of, and by doing so, you make it less good for other things.
For example, look at like, say, the data science tool set.
It's better working with sets of data than individual things.
I don't know.
There's just like, it focuses on one thing, and so then other things become less easy with it.
All right.
So, I don't know.
One of the changes, which is nice, is that the languages are not being developed in the vacuum that they used to be.
They're learning from each other.
Yeah, that's for sure.
And we're getting the mega cool feature in one language that we would like to have in another.
We're already seeing that.
Python's taking some features from other languages and pulling it in.
Other features are taking cool languages.
Now, I've heard from a lot of people that hardly any other languages have good testing tools like pytest.
Specifically, they don't have pytest for other languages.
And that would be cool if we could have some better testing tools in other languages.
Yeah.
I mean, there's certainly examples of that.
Like Swift, Apple's Swift is very much Python inspired.
You see like the optional, the null coalescing operator being moved towards Python and 3A potentially.
Things like that.
So, I agree.
I definitely see it as a blending.
I don't know if there'll be a single one.
So, all right.
The final two.
Every developer will need to work with data.
I think that they're doing that now, right?
Yeah, I think that's already.
We always have.
Yeah.
And then programming will become a core tenant of the education system, like reading and writing.
I believe that.
I do too.
It isn't necessarily like reading and writing.
I think it's more like learning biology and basic chemistry.
It's not that you actually need it every day, but it's good that most people kind of understand how it sort of works.
Yeah, I'm totally with you.
And the thing I think it will eject from the education system is geometry.
I feel like the reason that geometry is taught in middle school and high school is so that critical thinking and proof type of logical thought can be taught.
Like you don't really need to prove this thing about isosceles triangles.
Like that doesn't help you in life.
But the thought process does.
And I feel like programming, you have the same type of thought processes, but you get a skill, not just a grade.
Anyway, that's my prediction.
That's interesting.
Yeah.
We'll see about that.
All right.
So another one on async.io, something I've been digging into a ton lately.
Yeah.
I wasn't even aware that this was a thing going on.
But apparently the standard library async.io documentation was completely rewritten from scratch recently.
That's awesome.
That's big news.
Yeah.
It even says it in the in the tweet announcement.
This is big news with like two flames on the side.
So I haven't actually read through this a whole bunch, but it does look like there's quite a bit of documentation there.
So that's neat.
That's really cool.
And it says they're not done.
So there's a comment that there's this is just the beginning that there's they're going to add some tutorials for high level concepts and APIs.
And then they're also going to keep going because apparently there's some new things planned for 3.8.
Yeah, that's awesome.
Yeah.
You highlight some of the things coming for async.io itself, like the library and the capabilities async and await are getting better, right?
Yeah.
And I think this sort of talks to the importance of async in the future.
If not only are people still working on making improving the async story within Python, but they're also trying to improve the documentation to make sure that it's usable, which I'm glad to see that sort of documentation going side by side with new concepts.
That's a good thing.
Yeah.
But that kind of highlights the importance of async going forward.
Yeah, I think async is just becoming increasingly important.
And I'm really glad to see it really gaining some momentum because if you look at apps written with and without async in a library that can take advantage of it, it's like night and day.
It's ridiculous.
And I think that, like you said, again, it's a different concept sort of thing.
You can think in higher level concepts and break problems up into the more logical breaking points.
Right.
You don't have to think about threads and processes and coordination and all that.
You just say, I'm going to do this part, then I'll wait, then I'll do that part, then I'll wait, then I'll do that part.
And then, you know, that's sort of that higher level abstraction.
Yeah.
You want to get caught up on all the places that Python is headed, all the directions, all the threads.
Yeah, definitely.
So there was the 2018 Python Language Summit.
Okay.
That's where all the core developers get together and they talk about what they're doing.
They do proposals to each other like, hey, I think we should focus on this or we need to address that and so on.
And there's some really interesting items on lwn.net, which is a pretty interesting site.
They've got a lot of great stuff, mostly focused on Linux.
It's kind of like the Craigslist of Linux news or something.
I don't know.
It's got an interesting design.
But basically, this is a meta article about the Python Language Summit this year.
And it lists off a whole bunch of sessions, each of which are covered in detail in another article.
So what I'm going to do is cover the high level bits of each one of these main sessions.
Okay.
So the first one is, could we have sub-interpreter support for Python?
So we were just talking about asyncio and stuff.
One of the main problems, I don't know if I'd call it really a problem because there's a lot of ways around it, actually.
But conceptions is that Python's not great for threading, specifically around taking advantage of multiple cores for CPU stuff because of the GIL.
Right?
It could have 100 threads and only one will run at a time if it's pure Python because of the GIL.
So the suggestion is, could we take this feature, which is apparently already in Python, called sub-interpreter support to create like semi, like lightweight sub-processes that run within the same process?
So sub-interpreters will allow multiple Python interpreters per process.
And there's potential for zero copy data sharing between them and things like that, which is pretty cool.
That's interesting.
It might be interesting even for versioning, right?
Like this library requires this version where that library requires that version.
Well, you can run like maybe multiple sub-interpreters.
I don't know.
That's a whole different angle.
And it wasn't part of this presentation.
But they say the problem is that sub-interpreters currently share the GIL.
So the potential work is, could we make them not share the GIL so they actually can take advantage of multiple cores?
Yeah.
Right?
Like it's becoming a thing.
So like my MacBook, my new MacBook here has 12 cores as far as the OS sees it.
And if I write Python code and it takes full advantage of all the processor that it can in single thread, it's only 8.3% of the capacity of the CPU.
That's a lot less than, you know, 100%.
So having features like this to let you do more computation would be great.
Yeah.
All right.
So I don't want to go too deep in these because it'll like make this an hour-long show.
So I'll try to go a little quicker.
So the next one is modifying the Python object model.
Looking at CPython's structure.
So Carl Shapiro from Instagram talked about how they might rework the way the interpreter interacts with classes for better performance and memory and stuff.
Because Instagram is all about that.
It's one of the interesting things in this article is that there was some controversy around like the presentation and some other stuff.
So I'll let you read about it.
I won't bring it up here, but it's worth looking at.
We just talked about the GIL for parallelism.
And Larry Hastings has been on and off working on this thing called the gillectomy, the removal of the GIL.
And the GIL basically is a thread protection around allocation in Python, which means you don't have to like take a bunch of fine-grained locks when you're allocating new objects.
But because of that, it means only one instruction can run at a time.
So could we take that out and then have a Python that does like true parallelism?
Basically, the update is not much has happened.
So for better or worse, Marietta talked about using GitHub issues for Python.
I didn't realize this wasn't already happening, but apparently it's not.
Apparently there's another site that's currently being used and trying to move over to GitHub issues.
That's pretty cool.
Lucas talked about shortening the Python release cycle.
So right now it's on an 18-month release cycle.
So in 18 months from whenever 3.7 was out, 3.8 will be out.
Can we make that yearly?
It's possibly the conversation.
So you can read in that.
One that you might find interesting is unplugging old batteries.
So unplugging old batteries, there's like encoding libraries for like formats that like predate MIME and other weird type of things that are in there that nobody likely uses anymore.
So what is the process for taking stuff out of the standard library?
There's a process for it getting in, but how do you get it out?
Oh, that's interesting.
Yeah, a lot of things could probably be not just completely eliminated, but moved to like just a pip package or something.
Exactly.
Like this is now out, but if you need it back, you can pip install it separately.
But your job is to maintain it.
Like we're not shipping it anymore.
Something to that effect.
It's kind of cool.
Yeah.
They talked about Linux distributions and Python 2.
With Python 2 coming to end of life, what is the process for like basically getting those out of the Linux systems that they ship with?
And how do we do that?
You talked about Python's static typing.
So there's an update on that.
Mostly focused on stub files.
Have you heard of stub files?
Yeah.
It's like the dot, dot, dot, right?
I've looked at them and they basically are the method signature and then dot, dot, dot.
And they're not actually the implementation.
They're just saying here is the actual shape of this method.
And so what you do is they get imported alongside of your code by things like mypy.
So they're kept in this thing called the type shed, which is a place for third-party libraries to be in there.
So you can look through there and see things like what type of stuff, like the context variable and what is its type definition, for example.
So there's a bunch of things with dot, dot, dot, dot.
They're just like empty, not implemented sort of structures.
And so this is where a lot of the third-party stuff and the standard of the library goes.
But they're saying, look, a lot of packages are not putting things here, like namely things like Django and SQLAlchemy.
So there's a proposal of finding a way to bind the type information to the package and not to the central location so that package maintainers can ship those stubs along with their libraries.
Yeah, that would make more sense.
Or at least it would make some sense.
Yeah, like it'd be easier.
Because then you don't have to, say, like do a PR to some other repo.
Like you just put it along with your library and it's in pip.
Yeah.
It's in PyPI.
Okay.
So Python virtual environments, there's some debate about whether they're useful and what we might do instead.
Primarily around, like if you're trying to teach Python, you shouldn't have to start out with a 20-minute conversation of how to do virtual environments both on Linux, Windows, and macOS.
And then you can actually talk about Hello World, but just how do we just do that?
So an interesting debate there.
There was some conversation around the whole new decision-making process now that Guido stepped down.
Another one about getting along in the Python community, trying to make things like the dev mailer list be more interesting and welcoming as well.
Also, finally, mentoring and diversity for Python with Marietta.
And Victor Stinner as well, two separate presentations in that general area.
So there's every single one of these things I've talked about, there's an article about it and more references.
And so if you want to dive into it, it's like it'll keep you busy for an afternoon.
Okay.
One of the things that you wrote down but we didn't talk about was TK Enter?
Oh, so the one about getting along and being more civil on dev was somebody complaining that TK Enter is so badly out of date that it should be removed from Python, but it's shipped with Python, which kind of brings us back to how do we unplug old batteries?
Because if people don't maintain it and it doesn't work for anyone, then why would TK Enter be there?
So we might get back on another gooey kick if we go down that too far.
Okay.
Anyway, that's what that was about.
The sort of rudeness on the mailing list was focused around TK Enter.
Okay.
And whether it needs to be upgraded, removed, whatever.
All right.
Well, that is it for the Python language summit.
But I thought that was super interesting.
And there's just so many looks at where people are putting their effort.
I mean, this is where the core developers are focusing their time.
So I think it's definitely worth paying attention to.
Yeah, definitely.
Cool.
All right.
You got it?
Got anything else you want to share with folks while we're here?
Yeah.
One thing.
I noticed this from – I'm going to get his name wrong.
Anthony Saudeel?
Anthony Saudeel.
Code with Anthony on Twitter.
I mentioned that pre-commit had a huge jump in downloads on July 30th.
And I first assumed that it was because we covered it or something.
But apparently it didn't line up with us covering it.
You should have covered it earlier.
Could have gotten credit, but no.
But I looked at other packages and there is a weird jump in almost all package downloads starting at the end of July.
I don't have any news around that.
Just that's weird.
And I wonder what's going on.
You know, I wonder if that has to do with when the new PyPI launched.
Oh, yeah.
I wonder.
And it's almost coincident with the pip 18 as well.
The new PyPI launched in April.
So, yeah, it could be with pip 18.
Who knows?
I bet it's some low-level infrastructure thing.
Like caching got expired and everything that had cached it everywhere got a new one or something.
Yeah.
Maybe they changed the way they were caching it or something.
I do remember seeing that from pip that every now and then, like, if you run a sufficiently old pip,
it'll say that it's, like, its offline cache is not compatible and it'll redownload.
So, I'm wondering if in the reverse direction when a new pip came out, that may happen.
So, I bring that up so that if anybody knows what's going on, they can get a hold of us and we can find out.
Yeah.
Throw in the show notes.
Pythonbytes.fm slash 96.
Yeah.
And that's one of the things that I love about this is because we can talk about things that we don't quite know about and people will correct us.
They're generally pretty nice about it, too.
So, thank you for that.
Yeah.
So, one of the things that somebody mentioned, we've talked about Pi Cuda.
Cudpie.
Did I get that right?
I think it was Cudpie.
Cudpie.
Yeah.
Cudpie.
Yeah, Cudpie.
Cudpie.
We were guessing that maybe if you had a video card on your laptop, maybe you could use it.
But we were corrected that it would have to be an NVIDIA card because Cuda only runs on NVIDIA.
Yeah.
So, apparently Cuda is an NVIDIA thing, which I didn't realize.
I thought it was just a graphics card standard.
But, no, it's an NVIDIA thing.
So, this is all triggered by your question.
Could I do this on my MacBook or on my laptop?
If you have, like, a Windows or Linux laptop that has an NVIDIA card, yes.
Otherwise, no.
Apparently, it's the answer.
So, a couple people wrote us and communicated that to us.
So, thank you all for that.
I guess if you're on a MacBook, you could get an external.
On the new ones, you can do external GPUs.
And you could get an NVIDIA one there.
But even the one sold by Apple is actually a Radeon, so it doesn't help you.
Okay.
So, anyway, just thanks for the clarification.
Yeah.
Do you have anything else to add?
Nothing else.
Not right now.
I have some good news next week I'm going to share.
But right now, nothing to say.
All right.
Well, thanks a lot.
Just working away on projects.
Yeah, thank you.
Great to chat with you, as always.
Yeah.
Thank you for listening to Python Bytes.
Follow the show on Twitter via at Python Bytes.
That's Python Bytes as in B-Y-T-E-S.
And get the full show notes at pythonbytes.fm.
If you have a news item you want featured, just visit pythonbytes.fm and send it our way.
We're always on the lookout for sharing something cool.
On behalf of myself and Brian Okken, this is Michael Kennedy.
Thank you for listening and sharing this podcast with your friends and colleagues.