Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.
This is episode 176, recorded April 1st, 2020.
April Fool's.
So, I'm Brian Okken.
I'm Michael Kennedy.
And this episode is brought to you by DigitalOcean.
We'll talk more about them later.
But, yeah, let's just catch up a little bit.
What's going on?
Oh, you know, this is one of the few bits of human interaction outside of my house that I get.
It's to talk to you here on Skype video, right?
Like, it's a weird time, isn't it?
It's totally weird.
We normally don't record video, but we do see each other.
And we've noticed as we contacted us or connected today that both of us have new haircuts, but they're homegrown haircuts.
I never cut my own hair.
And it was getting bigger and bigger.
And my hair doesn't get long.
It just gets big and poofy.
And it's more like a Q-tip on top of my head.
And eventually, I just break down.
Like, I can't do this.
So, at first, I looked at my daughter who's 19.
I'm like, here's a shaver.
You got to cut my hair.
And she's like, no, no.
And so, she did an okay job.
And that was really good.
And then I decided to go back.
And, like, it was still quite a bit, actually, longer than I wanted.
So, I gave it a second haircutting as well.
Yeah.
And, you know, honestly, it's about as good as the haircuts I pay for.
So, I'm feeling like this might be a trend that I was forced into.
And you're looking good yourself.
Yeah.
So, I normally have, like, a short goatee thing that I keep.
And I just have, when I bought the trimmer for it, I just only needed one guard because,
I mean, I didn't think ahead.
And I just, like, I don't know where all the rest of the guards were.
So, I only have that, like, little short guard for the trimmer.
And my wife was willing to cut my hair.
But it's a little shorter than I was expecting.
But I think it's all right.
I don't have to worry about it anymore.
My kids, like, say it's like a peach.
So, they come by and rub my head all the time.
It looks good.
And that's the weird world we find ourselves in.
You know, I'm here drinking coffee out of this Pittsburgh mug.
But there will be no Pittsburgh, right?
PyCon is now officially canceled.
Yeah.
That's a bummer.
This might be the first recording since it was actually fully, fully canceled.
At least the first time we spoke about it, possibly.
We might have mentioned it last time.
But there's certainly a tightening up on getting together for conferences and that kind of thing.
And, you know, we spoke a little bit about what's going to stick.
These changes that we have to have that are going to stick.
Like, a lot of meetings could just be emails.
Or they could be remote or whatever, right?
And I wonder what that's going to do to the conference space.
Yeah.
I wonder, too.
One of the reactions I heard from somebody was, even if it was to move towards a virtual,
which they were talking about that a little bit, maybe doing a virtual conference.
But that wasn't the thing.
It's essentially canceled.
Although some people are sending in videos, they're going to have some people be able to
send in their presentations as videos.
So that's neat.
But it's the meeting people.
We interact with people online all year long.
And it's one chance to get together.
So at least with PyCon, I don't think that'll go away.
No, I don't think so.
For me, it's not about the talks.
I go to talks and they're interesting.
But it's about the stuff that happens in between it.
It's about hanging out with you, hanging out with people from other places I don't normally
see, making those connections, and just stumbling onto things that are like, oh, I had no idea
that this was happening.
Let's talk about that.
That's the value.
And that's really tough to replace.
Yeah, it really is tough to replace.
Maybe more video chats will help.
Absolutely.
All right.
Well, I guess everyone be safe out there.
Yeah.
And follow our lead.
Stay home, even if you got to cut your own hair.
Well, let's move on to some topics.
So first off, I was hoping to talk about PyProject.toml.
I think we've, I'm sure we've covered it before.
But the Brett Cannon is, is he still part of the core or did he get voted out?
I think he's still part of the core at Honchos.
I think he's on the steering council.
Yeah.
He wrote an article just released recently called, What the Heck is PyProject.toml?
And one of the reasons why he wrote it, and I think this is an interesting side effect,
is so if you're just, if you're just joining us, the PyProject.toml came out of the efforts
with PEP 517 and 518 to define what the file looks like.
And it kind of defines what tools are needed to build a project and, and how to build it.
And that used to be mostly just setup tools, but now there's lots of other tools like Flit
and Poetry, and you can make your own if you want to.
But these are, this file, one of the side effects is people started adding other, their tool
configuration to this file, such as like coverage and talks.
Now you can, you can put the, those configurations into PyProject.toml for those tools, even,
even though it has nothing to do with building and it just saves you from having an extra file,
but even though those tools have their own configuration files.
And then Black came around and Black uses, only has a few configurations available.
So instead of creating their own configuration file, Black uses the PyProject.toml for looking
up configuration if you want to configure like line length or something.
And so there's, there's been projects that have added PyProject.toml files for the reason
because they were trying to, they're using Black also.
And now their, their builds don't work.
And the reason is if there's a PyProject.toml, even if you're using setup tools, pip will,
Pip will look for the build stuff around that within the file.
So Brett is, has gone and added, showed the, the few lines of code you need to add to the
toml file to specify how to build with setup tools.
That's the main contribution for this, but it's also, I think that people ought to, ought to
go read the article because it's a good, a good summary of where we're at right now.
Yeah.
The PyProject.toml is great.
And it's super cool that you can specify things like here's how you can say, build the package,
right?
It requires a wheel and whatnot.
Yeah.
It's, it's cool.
I really like it.
I honestly need to start working with it.
I'm still on the requirements.txt side of the fence, just waiting for things to, you know,
shake out, but it certainly seems like there's a lot of energy here.
Yeah, definitely.
Yeah.
So definitely good pick there.
You know, sometimes we talk about awesome things on Python bytes, right?
Yeah.
And we've even talked about awesome lists, which are lists organizing various other things,
projects, websites, tools, whatever around something.
But last time in passing, we mentioned that there was a cool article by Jack McHugh, where
he blogged about the awesome Python bytes, awesome list.
Yeah.
That was cool, right?
Yeah.
So now Jack has made this a proper GitHub repository where it has a clear way to contribute to it.
You can do PRs and all sorts of stuff.
And he said he'll be adding personally to the repo whenever he hears about awesome things like
this.
Maybe he'll add his own list to it, which would be very meta, right?
It would just link back to itself.
No, just kidding.
But that's really awesome.
And there's already five PRs from listeners accepted on this list.
Oh, wow.
That's great.
Isn't that cool?
People seem to be liking it.
After we announced it, then we had other people saying, and I saw some commentary about there
was some missing that he forgot about.
Yeah.
Well, it's just the stuff that really stood out to Jack in the beginning, right?
So if you want to go back and kind of look through what some people thought of a year and a half
ago that we talked about that was awesome, you can go there.
And if you look at it, it comes with graphics.
Like all good things that present things, if you can have graphics, guess what?
It's like a thousand words, isn't it?
Oh, yeah.
Isn't that cool?
This is great.
Yeah.
It's really, really polished.
And you can just flip through and like, we talked about D type, the visualization for
Panda's data frames.
And, you know, there's like some nice examples and pictures there.
Really, really good stuff.
So I think it's a great way to explore some of the stuff that we talk about.
A couple of fun projects I saw on there that I just remembered.
Oh, yeah, we did talk about that.
It was cool.
It was great expectations for validating, documenting, and profiling data.
Panda's vet, which is a flake eight, a linter for opinionated pandas code.
Geoalchemy, which is like geospatial databases on top of SQLAlchemy.
And Vue.py, which provides in-browser Python runtimes interacting with Vue.js.
So just, you know, all sorts of cool stuff there.
And I guess related to this, although not technically anything to do with that, is we
have a super cool search over at Python by set of M slash search.
And it's really fast.
It even searches our spoken words.
So if you want to just find something that maybe we talked about a long time ago or see
if it's covered, throw it in there and it'll pop up.
That's really neat.
And I love this because I'm like flipping through this going, oh, I want to go back and
check that out again.
Exactly.
Like that was like a year and a half and we only spent like, you know, half an hour
playing with it before we talked about it.
Then maybe we didn't use it again.
But yeah, it was cool.
So I legitimately enjoyed going through the list here and I thought listeners might as well
if they heard it before or even if they didn't.
Yeah.
That's awesome.
Yeah.
Well, I guess you were going to continue a theme a little bit with your, from the pyproject.toml
onward, huh?
I've been playing with a project for a while off and on associated with testing code called
the cards project.
And it's a little to-do app thing.
It's just a little sample thing to mostly to play with all of the stuff around Python and
distribution and testing and, and all that.
And, somebody, oh gosh, it was months ago and I should have their name here.
But one of the, somebody else contributed a pull request to add a GitHub actions.
So GitHub actions are a way to, you know, for, you can use that for like CI, CD workflows
with Python project with all, all sorts of stuff, but with, especially with Python projects,
because you know that the building pipeline is kind of short, with, with Python, it's
not as complicated as other things.
So often you don't really, it's, it might be overkill to use another CI system.
Um, and so GitHub actions are really pretty cool for that.
And so I incorporated that and then was looking into the last step was how do I publish to
the PyPI?
I'd really like to add that.
So there's a, I was looking it up and the PyPI, the Python packaging authority actually
has a article written called publishing pack, publishing packets, distribution release releases
using GitHub actions with CI, CD workflows.
And so this was like, I'm following through this.
I'm actually ran into some hiccups, but I wanted to cover it anyway.
My promise is that by the time this goes live, the cards project will actually be complete
because it's, it's almost there.
There's a few hiccups I've ran into.
PyPI requires you to have an email associated with package, which is kind of annoying because,
split and pip don't anymore, but PyPI still does.
So I had to do that.
And there's a few, a few other hiccups.
I don't know if I'll write it.
If there's just these, I'll probably just fix them and have it ready for people to look at.
But, if it's too many more, I'll have to write a new blog post and I'll associate that
with this as well.
Yeah, this looks great.
And I had no idea about the GitHub secrets.
So one of the challenges is you want to have your personal PyPI account where you are the maintainer admin of that package on pypi.org.
You might not want to contribute that into the source code, especially for an open source project, right?
Yeah, probably a bad idea.
Yeah, that's going to result in some bad things happening there.
Eventually it talks about how to use the secrets settings in GitHub to store those there and then how to like pull those out as replacements in your GitHub actions, which I had never done.
That's pretty cool.
Yeah, and the workflow that they're suggesting, which is, I think, a great idea, is to go ahead and have all, every pull request or merge to your, some master branch or to anywhere, any branch, go through the entire thing and try to publish to the test pypi server.
And only ones where you change the tag, which is where you would change, you'd change a tag if you change your version number.
That's when it would push to a pypi, the real one.
But to have that workflow even going through the test pypi every time you push something, I think that's a great idea.
Yeah, it is.
But somehow you've got to change the version number or it's not going to let you publish even to test, right?
Yeah.
So I'm not sure.
I still am figuring out how to get around that.
So we'll see.
Yeah.
Cool.
One of the things I love, I love really great tutorials.
The internet, I think, is working because of the great tutorials around.
And one of the groups that have put out some awesome tutorials is DigitalOcean.
So DigitalOcean is sponsoring this episode.
And one of the things that they're offering, of course, is they offer a way to get started with, they offer Kubernetes clusters.
And you can do that with DigitalOcean.
But to get started with hosting and running Linux servers with Kubernetes clusters, that can be a little tricky.
And that's why we want to highlight that DigitalOcean has launched their new support center.
And this makes it easier to find answers to your questions, help you get up to speed, get what you need.
And you can search across product documentation and community tutorials and forums all in one place.
And especially with something tricky like a Kubernetes cluster, really, even if you're new at any of this, their support center is awesome.
Got great tutorials.
So visit pythonbytes.fm/doc support to see their tutorials.
And of course, use pythonbytes.fm/DigitalOcean for $100 credit for new users.
Yeah, absolutely.
Their tutorials are so good that you can even select different operating systems and different versions so that the steps exactly line up.
If you're on, say, Ubuntu 16, you don't have to try to patch that back.
The tutorial steps would be different, which is, that's taken to the next level.
Very good stuff.
I've heard some from somebody once before where they were actually using a different host and using DigitalOcean tutorials to help them set it up.
And then they finally realized, why am I giving money to somebody else?
Let's use DigitalOcean because they helped me out.
That's where our infrastructure is.
Very good.
Now, this next one, Brian, this one is best seen through pictures.
So make sure you open this up and we talk through it.
So like I said, all good presentations, especially stuff that's graphical, has pictures, people.
This is amazing.
So this thing that I want to talk about is called rich.
Rich text for terminals.
So we've talked a lot about how it's great to have GUIs and web apps and stuff like that that's very visual for the presentations, which are still true.
But a lot of times you just want something in a little terminal app, command CLI app.
And it would be nice if it wasn't just all one plain color or just, you know, all text left aligned.
So there's this cool project called rich.
And rich lets you have up to 16.7 million colors for your terminal colors, not just like eight or whatever it is, you know, red and light red, like 16.8 million colors.
It supports bold, dim, italics, underlined, strikethrough, and even, please don't use it, the blink tag.
You can put the blink tag out there.
You can have text that's left aligned, centered, right aligned, justified, supports like Chinese and Korean, has emojis.
Like you can put colon, apple, colon, and an apple shows up.
You can, as part of a string, put little escapes.
So you can say this word is bold magenta.
And then here's the rest of the words and just print that thing out.
So you have like inline styling.
Tables.
You have beautiful tables like in the terminal.
Those are really nice.
Syntax highlighting for code.
So you can print out Python code with line numbers and it'll highlight it.
And it even has a markdown support.
So you could write markdown and it renders it as its own version of rich text.
Not quite HTML or something like that, but it has, you know, bulleted lists and like titles and whatnot.
Even as progress bars and logger support and all kinds of stuff.
Isn't that cool?
Yeah.
The logger handling is actually pretty great.
Yeah.
So if you're working on a terminal app and you're like, we're just going to keep it this way, but I want it to look nicer.
I want it to look a little more professional.
This is like kind of your all in one, do a bunch of cool stuff here.
You can even have multiple progress bars, all updating in parallel on the screen.
So if you've got a bunch of downloads, you can indicate them all happening or something like that.
A bunch of jobs running.
Sweet.
So yeah, this is a cool project.
Might have to add this to cards.
That would be cool.
That would be awesome.
Yeah, it definitely would.
It even has all of that support for Windows.
If you use the new Windows console or terminal, I guess is the word that they're now using because console, I guess, is the old school thing.
And Microsoft's like, no, no, we all have terminals now.
And you can go get that from the, you probably don't have that on Windows.
It's in preview, but you can get it on the Windows store for free.
And I link to that as well.
So if you're on Windows and you want a better terminal in general that supports this kind of stuff, check that out too.
Hmm.
Interesting.
Second, do you know if that'll roll into the normal, just all releases of Windows at some point?
I would hope so.
Maybe.
It's still in beta.
I guess we'll have to see, but it would be nice, right?
It'd be one step closer to parity across all these platforms, which, you know, it's always good.
Yeah.
I just use the bash terminal for bash for or get for Windows comes with a bash terminal.
Right.
Nice.
Speaking of Windows, I guess it's any operating system, but I'm actually kind of surprised.
This has been on our to-do list for a long time.
A library called psutil.
It's a cross-platform library for process and system monitoring in Python.
And I'm actually surprised that we haven't really covered it.
We must have covered it in passing a couple of times.
But I wanted to highlight it because it is an amazing little tool.
It's not something I really like love when I have to use it.
But things like there's time zone control and multiple computers or services running on a different machine with Windows.
Like I said, you can use it with it.
It's a cross-platform thing.
But you can use this to grab CPU utilization, memory utilization, disk usage, network, what ports are being used.
And you can even see all of this information based on which process.
So you can get like a per process list of CPU usage and everything within that.
And then around processes, you can suspend and kill and signal different processes.
So if you're one of the things that we use this for is monitoring our build servers.
I know there's other ways to do it, but this is a pretty easy way to go over to another server and grab the CPU usage and memory usage.
So we can keep track of all of our, make sure all of our build servers are working and not overloaded.
So we use this.
It's a pretty cool little tool.
Yeah, this looks really great.
I think we also talked about it last week briefly when we covered pytest Monitor.
And that was built on top of psutil amongst other things.
We didn't cover psutil, but like it's that kind of tools that this enables, right?
Yeah, pytest Monitor used that also.
I'm like, man, I think we covered this recently.
But yeah, it was last week we covered that.
By itself, it's still a really great thing to use right itself.
And the readme is huge, but it shows you a lot of different examples for how to use it.
Yeah, it looks super useful.
If you're going to do any automation and sort of admin sort of work, it looks great.
One of the great things that I love is the cross-platform part because this sort of stuff you can do on,
I mean, you can do directly with Windows and with Unix, but, you know, it's different on everything else, on everything.
And so having a cross, I'm pretty impressed that it can even exist as a cross-platform thing, but it can.
Yeah, no kidding.
Apparently it can, and it's awesome.
Definitely a good one.
So let me ask you this question.
If I'm going to store some numbers, let's say up to 100,000, how big of an integer do I need to make in Python?
I don't have to make an integer.
They're just there.
It's beautiful, right?
But I know that you've done a lot of C and C++, and I have as well back in the day, and you used to have to really think about that, right?
Like if you saw a negative 32,000 for a number and you thought you were adding to it, you're like, oh, it was a short and it overflowed.
Well, I guess it's maybe it should be a U short, unsigned short.
It could hold up to 65,000, right?
Like this is something you always had to think about.
And in Python, you don't.
And how that happens is pretty interesting.
I didn't really know the internals.
I kind of guessed maybe something like what's happening, what's happening, but I didn't know.
And so there's a cool article by Arpiz Bayani, and he wrote something called How Python Implements Superlong Integers.
So, for example, if you tried to take two and raise it to the power of 20,000 in C, you would get infinity.
Right, because it's, well, it's bigger than we can hold it, so it must be infinity as far as we're concerned.
But in Python, it's fine.
It just gives you a 6,021-digit integer, and you don't have to declare, like, I'm working with really big numbers or anything like that.
And so Python is pretty cool in how it's transparent with these, right?
Yeah.
So this article digs into the CPython source code.
It talks about the algorithms and the data structures that make this happen.
So basically, the numbers in Python are represented as what's called a pyvar object.
So pyobject, that's the core type of things in Python.
But this is a variable length one, right?
And so there's a couple of different types that are like this.
We've got lists.
We've got tuples.
But we also have numbers.
And that indicates that they can be of different size, and they can basically grow as they need to.
Right?
So Python's numbers ultimately are represented by this thing called a long object.
And that has a pyobject base.
But then it also has a size and a digit.
And these digits are, I think they're four or eight bytes long.
I can't remember.
It doesn't say here.
It's a macro that would expand at four or eight, something like that.
But basically, it uses a list of digits.
And it, you know, initially, it just uses one of those.
But then when it gets full, it adds another and another and another.
What's interesting is they're base 230, not 10, not 16, 230.
That's weird.
Okay.
Yeah, because apparently it can most efficiently use that's exactly that space.
Okay.
Of its four bytes or whatever as individual elements in a base.
So it's pretty interesting.
Like there's this ginormous number.
And if you were trying to represent it in base 230, it's 100.
I'm not going to read it off because it's really, really long.
But it's pretty interesting how it uses this.
But then when you get into operations, right, if I'm going to add two numbers and they're
base 230, like what algorithm do you do?
It's not like base 10 where you normally do the thing or division or something.
So that's also interesting to think about.
If you look at arithmetic, it's pretty straightforward.
You just add within the digits.
And if you overflow 230, you do a carryover like you learned in elementary school.
Subtraction is like you do the borrow.
So it's like reverse.
But then multiplication, in order to keep things efficient, uses an algorithm called the
Karastabu algorithm, which is an interesting way to multiply two in-digit numbers in different
bases and stuff.
And yeah, so it's if you've ever wondered how come you don't have to worry about numbers
overflowing in Python.
Here's a cool look inside that at the CPython source code and some of the algorithms.
That's amazing.
Yeah.
Actually, that's pretty cool.
It is pretty cool.
I'm so glad I don't have to worry about it, right?
It's just digits happens.
Yeah.
One of the definite benefits of Python is this notion of developer time is way more valuable
than computer time.
So let's figure all this stuff out for everybody else.
And then we can stop thinking about it so much.
Yeah, for sure.
I had a project many years ago where I had to work with an FPGA and the clock system was
such that the timing had to set the current time, had to divide that out for a multi-radix
number.
So it wasn't like base 10 or base 230.
It was each digit had its own base.
Oh my goodness.
That's crazy.
It was kind of a beast.
It was cool.
A cool algorithm, but it was a fun thing.
Yeah.
Love this sort of stuff.
Yeah.
This is really cool.
A cool glimpse behind the curtain.
A little bit of the red pill.
Go down inside, see what's happening.
One final comment here is there's some funny little tricks you can play on people to ask.
You can like create the number 10 and then create the number 10 somewhere else and ask
if those are the same number, like with the word is, it's true for small numbers, but it's
false for large numbers.
And that's because Python pre-allocates the numbers negative five to 256.
Oh wow.
Okay.
When you have a hundred in your program, it's the same hundred everywhere.
But if you have 1,226,411, that was made on the spot.
Cause these are pointers.
These are not just like four bytes on the stack.
These are getting allocated in complicated ways each time.
And so they said, look, negative five to 256.
We use these all the time.
Let's just make them when Python starts.
Why minus five?
Well, because who uses minus six?
Come on.
I don't know.
I have no idea.
I can see minus one and zero to 256.
But beyond that, I don't really know what they must have some reason.
Yeah.
I don't know why.
They probably started with negative one.
Somebody said, well, let's do minus two also.
Oh, let's go hog wild and go all the way down to minus five.
Exactly.
Anyway.
Hey, that's our six.
You got anything, any news for us or any extras?
I have two really quick things.
You know how Microsoft bought GitHub?
That was kind of interesting news a while ago.
And actually people were contemplating the effects that would have on open source and all.
Yeah.
Well, NPM, the pip equivalent of JavaScript, is now was acquired by GitHub as well.
So just interesting moving and shaking over there behind the scenes.
That's really interesting.
Unlike PyPA and PIP, NPM was like a commercial venture or something to that effect to try to
bring that order to the JavaScript side.
And now, so since it was commercial, it's been acquired.
So NPM is the JavaScript thing?
NPM is how JavaScript spells PIP.
Yeah.
Okay.
Quick other one is we're going to try to set up a YouTube channel where people can see both
of us talking, the silly stuff that we do in a fairly uncut, unedited way.
But we're going to try to put a video for each topic.
So we just talked about the Python number thing.
Like just a single video on that.
It'll be easy to share with friends.
It'll be something we put up there.
So we'll have more details with you soon on the YouTube channel.
But I'm looking forward to it.
I think people, it'll give people a new look.
I think it'll be fun.
Yeah.
And then people will be able to recognize our faces when we are not walking around because
everybody should stay home.
Exactly.
Assuming that we continue to just give ourselves haircuts.
So we look the same.
Yeah.
All right.
Anything you want to share with folks?
Nope.
Just working from home.
All right.
Well, I had this joke about C and like numbers overflowing and like, oh, why is it negative
32,000?
Oh, because it's 37,000 positive or something like that.
Right.
Well, here's another one.
And this one is just to kind of make you feel good about yourself as a Python developer.
Right.
Yeah.
So this one's visual as a lot of them seem to be lately, but I'll go ahead and we'll do
our best to describe it.
Right.
I'm going to let you be the developer.
So you do the first three.
You got to give a little description though.
Okay.
So the dude's 830 in the morning.
I'm staring at the screen and I comment stupid bug.
And then seven hours later and this guy apparently grows facial hair really fast because he's
already got stubble at seven, seven hours.
Oh, it must be Linux.
And then the next day his face is red and he's even got more hair and he says, JavaScript's
broken.
Okay.
Oh yeah.
Bob comes in.
Our other guy comes in and looks at his, oh, hey Bob.
It looks like you forgot a semicolon.
Ah, fix.
Oh yeah.
Oh man.
Yeah.
So that'll be in the show notes.
You could check out the little comic.
It's fun.
It's by Eric Burke.
Nice one.
Yeah.
Don't miss semicolons.
No.
Oh, there's a lot of things I don't miss.
Although I'm doing a lot more C now or C++.
I got to do it again.
Well, it just makes you appreciate when you get to do Python.
Yeah, it sure does.
So, and appreciate my time with you because this is so special.
Absolutely.
Thank you.
It's great to get together and chat about this and share with everyone.
Thanks.
Thank you.
Yep.
Bye.
Thank you for listening to Python Bytes.
Follow the show on Twitter at Python Bytes.
That's Python Bytes as in B-Y-T-E-S.
And get the full show notes at Python Bytes.fm.
If you have a news item you want featured, just visit Python Bytes.fm and send it our way.
We're always on the lookout for sharing something cool.
This is Brian Okken and on behalf of myself and Michael Kennedy, thank you for listening and
sharing this podcast with your friends and colleagues.