Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.
This is episode 260, recorded November 19th, 2021.
I'm Michael Kennedy.
And I'm Brian Okken.
I'm Chris Petty.
Chris, welcome to the podcast.
You're a podcast veteran, but the first time here on Python Bytes.
Happy to have you here.
Thank you very much.
It's a pleasure to be here.
It's good to be back.
We're going to jump into a bunch of the news, but before we do, you know, tell people a bit about yourself.
Absolutely.
I've been in the tech business for about 35 years now, something like that.
And I've been a self-identifying Pythonista for about the last five or six years.
I work for these days.
I work for just recently changed jobs within Amazon.
I used to work for the Web Services Elastic File System team.
And I now work.
I switched gears from, you know, living in the clouds to living in the Amazon Fulfillment
centers.
We're now a network.
So does that mean you're doing what?
Like programming in COBOL or are we using now?
Yeah.
Nothing but COBOL and Fortran.
Just all the worst, all the most possible.
You know, not at all.
We do, my team, OpsTech IT, we do network automation for infrastructure.
So it's a lot of fun, actually.
It's really kind of neat going from building microservices and things like that, being infrastructure
oriented at that level to like working with being responsible for automating things like,
you know, network protocols, DNS, DHCP, fun stuff like that at scale in the fulfillment centers.
It's a really cool gig.
Yeah.
That sounds really interesting.
And you get an interface with physical things that go physical places, which is interesting.
It really is.
Thank you.
I only ship beds.
I don't know.
Brian, you also, you also work on physical things.
Yeah.
I like working with actual, we make things.
It's nice.
Yeah, absolutely.
It's a nice change of pace.
Yeah.
It's all I've known.
Out in the audience, Ned Batchelder.
Hey, Ned.
Thanks for being here.
Does Chris Boston represent?
Hey, Ned.
Yeah.
Ned is amazing.
He does so much work with Boston Python and I can't wait till we can get back to actually
like being in a room together with other people again.
I know.
Absolutely.
So, Brian, I used to feel like a cog in the machine, but now I work at a pretty small
company where I make a lot of the choices, so I don't have to feel like that.
But, you know, I think we're going to talk about something that I think is awesome because
Ned is here.
Yeah, it's kind of nice that Ned's here.
We picked this just for you, Ned.
But the, so Ned, one of the, Ned does a lot of stuff for Python.
And one of the things that he's done is this cool tool called Cog, which I've been watching
for a while, but I haven't had a really good reason to use.
But what cog does is you point it, you can have these cog, like these, it's like three
brackets and the cog word and then some code.
And then, then another magic incantation at the end.
It's pretty simple, but it's hard to explain.
But you have some code.
And when you point cog at it, it will run the code and then put the output of the code
somewhere else where you put it to.
So, seems neat.
I wanted to use it.
And, but I didn't know what a use case I would use it for.
Well, Simon Willison wrote this article called Using Cog to Update Help, the help output for
a markdown readme file.
And I think this is brilliant.
So, this is super awesome use case.
So, Simon, he even has it going up on his CSVS to SQLite project.
And the idea is you've got at the end, the end of the project, he actually lists on the
readme, he lists the help output.
Now, I actually have a project that the help output is out of date because the way I deal
with it is I run the help and then copy and paste.
But this is, Simon's right, this is a great use for cog to just point cog at it.
So, so he wrote a little snippet of code that goes in his readme.
And since cog ignores comments, you can put it in an HTML comment block.
And it, so cog will run that file and took the output and stick it out.
And then Simon didn't stop there.
He said, I want to make sure that this gets run when it needs to get run.
So, he wrote a test in pytest, of course, to check to see if cog needs to be written.
So, this is just, this is really pretty cool.
I like it.
This is the kind of thing that's, it's so cool.
Pardon me, Mike.
This is the kind of thing that's so cool because anytime you can eliminate a DIY, right?
Anytime you can make one thing, the one true source of whatever piece of the puzzle you're
looking to, to, to, to build you with.
And this is a great example of that.
Yeah.
Out in the audience, a little light, a real-time feedback.
Ned says, I also just updated the coverage and cog docs to do the same thing and to get help output into the docs.
Nice.
Yeah.
Very cool.
I love the inception of it.
Yeah.
This is definitely a cool project.
I want to take you all on a journey.
Come with me into the world of bank Python.
My drink.
This, this is an interesting article here.
It's one of these things that says, I can't exactly tell you precisely where I worked or what we did.
And I have to make up names for the projects that are not the actual names.
It's by Cal Patterson.
And it says the strange world of Python as used by big investment banks.
So this is quite the read.
If you go through it, if you click the little reader bit here, it says, you know, 30 minutes of reading time to read it.
So it's, it is a non-trivial bit of history, but let me see if I can give you some of, some of the highlights that I took from this.
And you can go read it if you really feel like you want to dive into it.
So one of the things they talk about is the database and all these things have interesting names.
I'm sure that they have actual interesting names as well that are not these names.
So for example, like the database like system is called Barbara.
And I guess the place to start here with this oral history that Cal talks about is that bank Python implementations are basically forks of the entire Python ecosystem.
Not just a fork of CPython, but like the libraries that it uses and everything, which are used at many, but not all of the biggest investment banks.
So it says, let's talk about this system called Minerva.
Minerva.
And it's a global database of Python objects.
And Barbara is this way to talk to it.
Okay.
So a lot of interesting keywords to keep it hidden.
Right.
But one of the things that's wild, this Barbara thing is it's like a hierarchical database that stores basically zipped pickle objects.
Wait, wait, what?
So it's in, when I say hierarchical, I mean, it's in what they call rings.
So you would talk to a database and there might be like a nearby replica that you talk to that might have a record or it might not.
And so then there's like a bigger replica, a sort of fallback to go look in this place.
And if it's not there, go back and look at another one.
They call these rings.
And then you would ask for something by its key and then you get it back.
So first of all, what do you guys think of just pickling all of your data objects?
It's really, I, that, that bit made me chuckle because it is exactly this, the kind of like at the time, pragmatic engineering trade-off that somebody made right years ago before we all saw pickles as the, the, the, the, the, the great villain of the Python ecosystem.
Before we knew that there could be downsides to pickling all the things and they're like, well, it's perfect.
Pickle idea.
What more could you want?
And now we're all just kind of looking at it.
It's so simple.
It's small.
It's fast.
Yeah.
How do you index something?
Actual database.
Just curious.
Yeah.
This is a fine question.
They do.
If you read it, they do also use databases sometimes.
Okay.
So, but the primary way seems to be this, this system called Barbara.
So one of the things that comes to mind for me here is, oh my gosh, what if you upgrade something?
What if you change the file, the, the structure in code of something that you put in the database five years ago?
Whoops.
Yeah.
That used to be, you know, signed up for the mailing list.
Yes or no.
And now when you change the order that yes or no became yes or no is admin.
Oh, oops.
Or it just doesn't match anymore and it crashes and you can't read your data because it's incompatible.
So this would happen with Python, the runtime, but also the libraries you're using, it seems to me.
And it's just incredibly unstable.
How much you want to make a bet they also have like some kind of really highly evolved over many years validation system that ensures that the, that's good.
That the pickles that you think are being used in the way that you think they're being used are actually still used that way.
Yeah.
Yeah, exactly.
There's got to be layers.
Sam out in the live, in the audience live stream says, good luck upgrading a new Python.
Yeah, exactly.
Yeah.
Like as in never.
And that's probably why part of it is like, yeah, we just forked it and froze it because that's crazy.
Let me show you something else.
I will, try to jump around the article by a search.
Cause I took notes.
Applications also commonly store their internal state in Barbara, writing data classes straight into and out of Barbara with very simple locking and transactions.
If not, and one of the things they decided is there's no file system available to these, to this platform and the little bits of data that the script picks gets picked up and outputs.
It all just goes back into this key value store pickled database thing called Barbara.
That's pretty wild, right?
Yeah.
Okay.
Um, they also talk about having these overlay features, which, is pretty wild.
This has to do with the, the different rings that I talked about.
So when you connect, you can say, I want to connect to the middle office ring and then the FICC ring and then the default ring.
So then when you do a database query, which by the way, it looks like a dictionary index, just bracket key.
It first is going to look for it in the middle office database.
And then it's going to look in the FICC database and then the default database.
So there's like these layers of settings and these layers of specificity.
And they, they even do things that are to some effect of like monkey patching.
Like if you, if you need to change it, you can have like your dev ring and then the middle office ring and then the outer ring.
And you could change the value and kind of like for mocking or something.
Um, so yeah, to your point, Brian, they say there are some good reasons.
There are some good reasons not to use Barbara.
If your data, let me count the ways.
If your data set is a large, it may be a good idea to look elsewhere, perhaps in a traditional SQL database or a KDB plus, which I have no idea what that is.
Uh, there's a limit on the size of Barbara objects, which is compressed at when it's zipped 16 megs or, smaller, which is pretty interesting.
And then paired with this is this direct, this thing called dagger, which is a directed acyclic graph that allows you to set up relationships between instruments.
Like, okay, I've got a bond and then I've got a credit default swap on that bond.
And then I've got like a fund of default swaps or, you know, like this whole layer of stuff.
And all of these things are like linked objects through these pickled things that you can ask questions about and like, get sort of like graph database type things.
That's pretty interesting.
Um, they talk a lot about how you model these things, like class, like instruments and swaps and stuff and Python.
And that's not super interesting, but, I wanted to highlight this section and see what y'all thought.
It says, if you understand, if you understand, Excel, you'll be starting to recognize how a lot of the stuff here works in Excel spreadsheets their cells and they have dependencies, which is like an acyclic graph, like this dagger hierarchy, graph database thing has.
Um, and it has a lot.
So they basically modeled so much of the code around how Excel works.
So you model the data first and then the behavior rather.
Usually we write code and then we have data that comes out of it.
So it's, it's pretty interesting.
There's also this thing called a wall pole, which is a database runner.
So when you get to this section, it says it's time to drop a bit of a bombshell.
Let's talk about deployment, versioning, et cetera.
The source code of the source code of this entire system itself, even including a Barbara is stored in Barbara, not on disk.
And it remains composed.
And it's kept in a special Barbara ring called source code.
All right.
What do you all think of that?
My hidden release engineer has left the building.
Oh man.
That's frightening.
Yeah.
So, not keeping the source code in the file system breaks some assumptions.
Like how does the program run?
Yeah.
So this wall pole thing runs, it's built to basically run source code that originates from a database.
And you, it's all one huge database.
So like, if you think mono repo, it's like that because you import from database and it's just like, whatever you need is there.
So versioning, it's like the anti microservice, basically the entire whole thing is literally there.
And then the way that you deploy it, let's see there, they talked about vouch.
So the way that you deploy to it is really interesting in that you, it reminds me of the, it works on my machine certification program.
So the way you deploy to the system is you just commit it to the database and boom, it's deployed.
It's running, right?
Because there's only one place the code lives.
It's in the database.
So once you do a commit, that's a full on deployment.
Now, how does it get into there?
There's a vouch system where you can get somebody who can code review what you've done and say, yeah, I vouch for this.
You can commit it unless you have a high reputation for not breaking the system.
In which case you can directly commit to it without question.
So I guess the culture of code review is not like a thing there.
Uh, I, I guess.
Um, so another interesting knock on that comes from storing your code in a database is how do you edit it?
Right?
Like what editor to use?
VI won't work on it.
PyCharm won't work on it.
VS Code won't work on it because those work on files.
So, this Minerva system has its own IDE because most IDEs think you have files, not databases that contain the data.
Does that make sense?
It is.
That makes sense.
It really is.
The guy, Cal says, what I can't understand is why does it contain its own web framework?
Investment banks have a, a one way approach to open source software.
Some of it can come in, but none of it can go out.
Well, I, I've seen, I've seen projects that like, like, people building IDEs within a company.
And I always shake my head.
It's like, why you, this isn't your business.
Why are you making an IDE within your company?
There are a couple of things.
There are a couple of things that strike me from this article.
The first is as I read it, cause I, I really enjoyed it.
Did you read the whole thing?
I, most, it's long, right?
It's the way through.
It is.
It's crazy, right?
It really is.
The thing that struck me is this is a perfect example of large organization, complex, code base and engineering system.
And tie.
Cause this really just smells like something that just got like built up, like, you know, the layers of rings around a tree over time engineer after engineer year after year making decisions.
And I'm sure like in the modern day, everyone who walks to the door, they probably have to plan for the period of time where the new engineer just freaks out when they tell them all this dumb as they adjust to the new reality that they're being exposed to.
Yeah.
And that is in some sense, this is a particular wacky case, but it's that way in kind of every large corporate environment in a, in a way.
That's true.
Yeah, that's totally true.
He actually, Cal actually discusses this a little bit here and says, this fact that you can't do very much of open source, you're stuck on some version of Python.
That's never going to see, never going to go back to sync with the real Python.
Cause how do you even do a database migration to that thing?
Says it could be the biggest disadvantage for you as your profession.
Every year you spend in the Minerva monoculture, the skills you need to interact with normal software atrophy.
By the time I left, I had pretty much forgotten how to wrestle pip in virtual EMV into shape essential skills for normal Python.
When everything is in the same repo and all the code is just an import away, software packaging just does not come up.
So yeah, there's a lot of, that's one way to deal with retention is to have a system where people, need to be skilled to use it.
And, it removes all of their other skills so they can't get another job.
Oh, I was just about to say, this seems like a retention at it pattern to me, but, you know,
It depends on which side of, of that, leave stay story you're on.
Because if your job is like, I want job security and I don't want to have to learn new things.
Like welcome to Minerva, whatever this is, right?
You're, you're home.
You can stay here forever.
They won't replace you.
On the other hand, if you want to use like FastAPI, you're like, I got to get out of here.
We can't be on, you know, Python two, three this long.
I'm out.
Off.
So I did want to mention that both Minerva and Barbara are things that you can pip install, but those aren't things we're talking about.
No, these are like secret code names.
I suspect the Minerva you installed doesn't have its own ID.
I'm just guessing.
Right.
Yeah.
Um, there's a lot of, a lot of interesting conversation here about this.
Um, just a quick little note.
I actually ended up, I'm like, it was late last night.
I'm like, I don't want to read this like 30 minute thing, but I do want to talk about it tomorrow.
So I actually use this thing called natural reader.
You can just throw this in here and it'll just like read it back.
Super nice.
That's so cool.
So I was, it was like 1130 last night when I started on this.
I'm like, Oh, I'm a little too tired to focus on this much stuff.
But, yeah, it's pretty nice.
You may have just made my day because I actually, you know, being part.
Yeah.
Because being partially blind, I, I, I, I feel like, you know,
like, you know how some people who have like lupus and other kinds, they talk about spoons.
Like they only have so much energy.
My eyes only have so much like ice cream spoons.
Yeah.
In the course of any one day that I can, I can do it.
There are times when I'm like, I cannot stare at a screen for even a moment longer.
And I really love things like audio books or the rare services like pocket that will actually
read you articles.
And so this sounds like an awesome tool for those things that don't have that feature
already.
Nice.
It's, it's really neat.
It has different, different modes.
I don't know how to make this thing come up without creating an account because I don't
want to know what my login is here, but it has like, if, if you're dyslexic, it has like
a dyslexic font you can turn on and it'll highlight as you write.
It's, it's, it's pretty neat.
That's so, yeah, cool.
Not directly related, but I've found it to be useful in doing this research.
I was too tired.
A little bit of audience interaction here.
Um, Brandon says run, run the source code from a database.
That's a one heck of an injection attack.
Yeah.
I mean, little Bobby tables would be just all over this thing, right?
It's one big injection risk.
Although on the other hand, I feel there might be some big safety here as well because everything
is pickled.
you've got to know what version of Python they forked it from and any variations.
They may be changed to the object model to even get it, to parse it back.
So it might be naturally safe as it's just like an opaque, it's almost like encryption.
Really, really, really weak encryption.
Yeah.
And then let's see.
Yeah.
David says absolutely.
Everyone in the live audience has a furrowed brow right now.
Uh, yes, David, or, or some like.
Nervous laughter.
Yeah.
There is one serious, serious aspect of this though.
We're talking about like retention and, and the article talks about like, as he worked
up in this, in this private ecosystem, he felt his external skills atrophying.
That is a common problem that I have heard.
So many people who work in these large kind of like pocket universes, whether it's an Amazon
or a Google or a Microsoft or an IBM or whatever people, people, people can kind of feel very
disconnected from where the field is currently going at the hot leading edge in the outside
world.
And, and I think it's a real issue because you can't always justify pulling in the latest,
greatest.
Sometimes there are engineering or earing implications to that, that would be really problematic to your
businesses bottom line.
And so I say, and I realized time is, is finite and it's, this is hard, but this is where I
say even dabbling in having side projects, even if it's just the kind of thing you do on weekend,
every once in a while, when you have time, it's a great way to sort of like feel unconstrained
by the requirements of your business and to go sort of play on your own terms.
I mean, write helper scripts for yourself, even for your job in, in modern Python.
Bingo.
Yeah, absolutely.
Good, good advice.
Indeed.
All right.
Over to you, Chris.
Okay.
Thank you.
So I decided I was thinking about this and thinking about the fact that, you know,
most of the people who come in and our guests here on this podcast are really sort of accomplished
hardcore Python programmers and they're familiar with the latest and greatest tools.
There is no way that I'm going to scoop you all.
So I decided to go a completely different direction and go in the direction.
That's right.
Exactly.
Of, of the whimsical side of computing, because I think we all need to stay in touch with that.
I know I had a, an issue a number of months ago, before I changed jobs where I was feeling
very burned out and I felt like I just, I need to sort of get back in touch with like, why am I
passionate about, about computing and programming in this space?
And one of the things that really was ball into my, you know, soothed by my burnt out soul was
these retro consoles up.
I used one called take 80, which was not Python based.
It's it's Lua based.
But the idea here is you can build like basically retro style video games very, very, very easily
with very few lines of code because these things, they do have some constraints, but the constraints
are completely artificial, right?
Like in this case, this one is fixed 16 color palette, 256 by 256.
Um, you know, like the constraints, the world that you're building in is uncomplicated and small.
And so as a result, you can sort of like step in and wrap your head around this thing so quickly and come up to speed.
And suddenly you're like, you know, Oh, I, I, I made a flying saucer fly across the screen.
I built a spray on making music.
I'm coding, you know, the mean loop of my game and you can do it in a very comparatively speaking,
short period of time with just a few lines of code.
I'm just skipping down a little bit here.
Um, like let's take an example.
They have the sort of like, you know, sneak eating dots kind of game.
And it's comparatively speaking, like when you compare it with like one of the modern,
you know, game, framework code bases.
Like unreal engine or something.
Unreal engine or do you think we were talking about like, here's my 60 pages of,
of coordinate transforms.
And you just look at it and you're like, I can't, I can't.
And my a hundred gigs of, textures that we're going to export.
Yeah.
Exactly.
Yeah.
So this project is called pixel.
I'm guessing that's how you P Y X E L.
And it allows you to create extremely simple 2d animated games in Python, right?
Yes.
I apologize.
I shouldn't have said that.
Uh, yes, exactly.
Um, and so, back when I was using the other one, I kept thinking the entire time,
like, wow.
Wow.
You know, I'm really having fun with this.
And I'm really, I actually really enjoyed learning Lua and it's a really fun language.
And it has a lot of, a lot of, you know, characteristics that are wonderful.
But I kept thinking like, wow.
I kind of wish I didn't have to get my brain to switch gears, which is not very good at
doing always.
I wish I could be coding this in Python.
And so this framework gives you the opportunity to do just that.
Um, and so you can use these.
It almost feels like for those of us who are old enough to remember, it almost feels like
coding in basic on the microcomputers that some of us grew up with in the eighties.
Right.
Like it's that sort of kind of like back, back when you could type it in, you're like,
I saw it printed and I typed it in and I ran it.
Exactly.
You know, it's like, yeah, it really, really is.
And so I really, this one, what's, what's kind of interesting about this one is like,
this is a great example that the sort of like sounding music section of this snake game here
is, this one is actually a little bit more Pythonic, which I think is actually really
kind of cool for Python folks like us, because the other ones that I've seen kind of like
drifted more back towards the 1980s, like, you know, a little bit lower level of abstraction
kind of thing.
Whereas this one really takes advantage of some of the beautiful data structures like
stats and dictionaries and the like that Python has to offer.
So you can use this really kind of nice express and tax to, to write you games.
And I am just super excited.
I, I, I played with it a bit.
I wrote a few lines of it and I, I, you know, kick the tires just to see how it worked, but
I really have not had the opportunity yet to, to dive deep and kind of rewrite my side
project in it.
But I totally intend to, and I'm super psyched about it and I'm really, really glad that it
exists.
Yeah.
I'd love to do this.
I mean, I got my start typing in games from the back on a TRS 80 from the back of magazines
and, and have an example.
So one of the things, something like some engine like this, one of the things I love to do with
somebody is, is take an existing game and say, let's look at the code and see what,
what happens if we modify this, this field.
There's, there's any way we can get the character to run faster.
Uh, does that make it more fun to play the game or less fun?
Things like that.
Absolutely.
And that's, and that kind of thing, that kind of like, well, let's take a look at this and
tweak it a little bit is so simple with this kind of thing, because it's like, everything
is so like, uncomplicated and straightforward.
Like chances are like the, here is the number of frames that we skip as we transition our
sprite character across the screen.
Oh, let's just tweak that.
Now you've done your sort of like, let's make him run a little faster kind of thing.
It's all so satisfyingly in a world where I don't know about you guys, but my day job,
especially the old one that I did was like, you know, make a code change or make a configuration
change.
And then wait, you know, eight hours for it to deploy through the pipeline.
Like having something that's immediate instant feedback like this was just a delight.
Sorry.
I'm just shocked at the eight hour turnaround time.
I'm so sorry.
That was it.
That was just a really bad example there that, you know, a couple of cases.
It's not, that's not my day-to-day life.
I actually love my day-to-day life.
I'm very happy, but that was just, you know what I mean?
We all have these like long build times.
Yep.
Will McGugan.
Hey, Will.
Says, this makes me nostalgic for my ZX Spectrum in the 80s.
Exactly.
There's an Atari 800 XL behind my head that I enjoy as well.
So I feel that.
Who's not surprised that Will's been coding since the 80s?
So anyway.
Now, now.
And then also a few more follow-up items for your comment on side projects.
Sam in the audience says, 100% agree, Chris.
Side projects are a great way to keep yourself sharp.
And Jose says, plus one for the side projects to try out the latest.
Indeed.
So before we move on though, Brian, I want to tell everyone about our sponsor.
This episode is sponsored by Shortcut.
Again, thank you Shortcut for sponsoring and supporting Python Bytes.
So there's a lot of project management tools out there, but they often suffer from either
being too simple for the engineering teams or too complex.
And they end up looking like Bugzilla or something insane.
And you're like, I really don't want to go here.
So you want something nice and simple, but that actually scales with you.
Something that's good both for managers and for engineers.
So that's what Shortcut is.
It's built to cater to both of these groups.
So it's built for software teams based on making workflows easy.
Big scale objectives, goals, roadmaps can be easily tied into a team or individual work
sprints.
Progress on those are easily flow back to higher level reports.
And of course, as all things should be, it's keyboard friendly.
You don't have to take your hand off and do a bunch of mousing around.
You can use the UI if you like, but there's a bunch of hotkeys, which is fantastic, as I
think they all should be.
So it makes you work faster and just keep going.
Good version control integration.
Works with Git, GitHub, GitLab, and so on.
A lot of the places that you probably already know.
So you can update your task project just by doing a commit or a PR, which is fantastic.
The last thing you want to do is update Git and GitHub and then go update your system, right?
As iteration planning, you can see your team activity when you want to, or just focus on
what you're doing.
And you can even let it schedule your project for you.
So nice burn down charge, cycle time, all that kind of stuff.
So finally, you can have a project management tool that's both for engineers and managers.
Give it a try at pythonbytes.fm/shortcut.
It's a cool project and it helps support the show.
So thank you for supporting us.
Brian, what do you got next?
What are we ditching now?
We're ditching CodeCov.
Harsh crowd.
So, okay.
So CodeCov has been great for a lot of people who have used it.
It's a third-party service that you can attach to your repo or something.
And it makes sure that you, and this has been helpful for merge requests and stuff.
So if somebody does a PR, it makes sure that the coverage is 100.
If you've had 100% coverage, it makes sure that they don't degrade it.
The code they add makes the coverage stay at a high level.
But so it's not without problems, however.
And Hinnick mentions that on some of his projects, it's been sometimes a CI pipeline is blocked just because it timed out with CodeCov.
And that's not a good reason to fail a build.
So anyway, Hinnick has this project where he said, wait a second.
I'm just trying to make sure everything goes to 100%.
Coverage.py.
Hey, Ned.
Second thing here for you.
But coverage.py from Ned Batchelder, maintaining it, of course, has a thing that it says fail if under 100%, under a percent.
So you can do 100%, but you can do 80% or whatever your current percentage is.
But anyway, if you want that, something like CodeCov, you can get that with coverage.py.
But there's some tricks around it.
So if you've got, one of the things coverage does is let's say you've got a project that has some code that is run,
like you're trying to take advantage of some of the newer things, but you also, newer Python things,
but you want to run on older Python too.
So there's some if statements for Python version, for example.
But it might be different hardware as well, like Windows versus Mac.
But there might be code that's run in one test suite and not in the other.
So in order to do that, you kind of have to run in parallel mode and then combine the coverage reports at the end.
And it sounds complicated, but mostly we just copy somebody else's work.
So, and Hinnick has the examples here.
So Hinnick goes through using GitHub and coverage to do everything that you used to be able, you were doing with CodeCov.
So this is a nice article with examples.
So we've got, for each project, we're downloading the coverage report and then combining it at the end with a report with a fail under 100.
Really cool.
It's, if this is something you want to do, it's basically, it's a short article with examples.
He even shows, he's got in his struct log, he shows his main workflow for GitHub pipelines that does all of this.
So just kind of read his code and make it work.
And I'm, I was actually thinking of doing this on a couple of projects I've got.
I wanted to make sure that they were up at 100% and do something like this.
But I actually, for some reason, I was dragging my feet and I didn't know why, but now I don't need to.
I'll go ahead and use Hinnick's suggestion.
Now you don't have to invent it.
Just do it.
Just set up that job on GitHub.
This is actually something I've been, I've been kind of struggling with.
I shouldn't say struggling with.
I should say this is a journey that I have been on in that testing and code coverage and that whole class of, of proof of correctness is my weakest suit.
And one of the things that I've really been sort of try to wrap my head around is I see a lot of like unit test code that seems like it was just, it's just there to keep the coverage percentage up.
And as far as I can tell, it doesn't actually test anything like we're creating these mocks and like in everything and, but they don't actually like enforce any contracts or prove anything out.
And I, I, I, I've trying to figure out how we can use these tools in ways that actually sort of raise the bar on code quality without encouraging this kind of like, I don't know, T crossing and I dotting behavior.
Well, we're kind of getting on a tangent, but I would say that code reviews on tests are important because that's, that's just, I mean, one of the easiest ways to get a hundred percent coverage with no tip failing tests is don't put any asserts in your tests.
Just saying.
Good.
That's it.
And that makes an excellent point unto itself.
Yes.
Thank you.
Yes.
Nice.
All right.
Well, let's talk about the tippy top, the tip top.
So I'm sure you all are familiar with top.
It's like this somewhat dreadful way to tell what's happening on a Unix system, on Linux to see like what's happening.
Is it, why is it busy?
Is it not busy?
I say dreadful because it's so simple and there's better tools.
One of them being this tip top thing for, so for a long time, I use glances.
I didn't know.
I still do use glances.
I'm not sure if you're all are familiar with the glances, but the UI that it creates is fantastic.
So you just type glances and you get this sort of progress bars that show in the CPU, the memory, all the processes.
It has like hotkeys.
You can sort by CPU or memory, filter by process name or ID, all kinds of stuff going on here.
And this, this is what I use now.
But Zach Villers sent in tip top as a competing thing.
And by the way, glances is Python and tip top is Python.
So very good candidates for pip X because no project needs them.
You just want to run them as commands, but they're Python, right?
So if you look at the same picture over on the tip top page, you can see that you get similar information, but you get a little bit better historical understanding of how your CPU is doing, how your memory is doing.
It even breaks it down per thread or per core.
It shows you your memory broken down by what's being used, what's being cached, what's free.
There's a graph of that over time.
Something that made me sad is it shows the network over time as well, the in and out flow.
But what happened to me is one of my servers, I ran this and it only would show the output from like a, some kind of, there were multiple network adapters and would only show the one that's not being used, not the one that's being used.
So I'm like, well, that's, that part is useless, unfortunately here.
But, and then it shows all the, the programs, like the CPU usage, the user it's running as the arguments that were passed to it, how much memory it has and stuff.
So I don't think I'm going to give up using glances because I really love that, but this feels like a, another tool to get a view of that sort into it.
What do y'all think?
I like the visual design on the face of it.
It, it like, especially comparing it to the glances screenshot that you showed, it feels like it might, it's a little prettier and it might, to me at least it looks a little bit less like dense and maybe a little bit easier to sort of like, maybe it's just the bounding boxes.
But I also feel like it's the way the screen is laid out a way to visually, an easier way to visually parse the information that's presenting.
That's always something I look for in the tool that I use.
Absolutely.
Brian?
Why do you need some, I don't use anything like this.
So.
All right.
So for example, if you log into, let's say you logged into a web server that was running a search job and it said, you're running a little bit low on memory as a, over the last few minutes.
And you've got five different web apps set up running there, web processes with like a bunch of their sub processes running.
And you wouldn't answer the question.
Okay.
Which one is it?
Which one do I have to restart?
Okay.
Yeah.
Things like that.
It's easy to see where you might not encounter this or have a need for this when you're working in the world of things, right?
Like the world of not servers, but embedded devices that might run Python or C or whatever.
And, you know, this is like managing infrastructure.
So you don't use this on your desktop?
No, I don't use this on my Mac, but I use this on all of my, not necessarily tip top, but glances on all of my Linux servers, including the Python bytes one.
So it's, it's nice to go in there and see what's going on in the servers with all I have is SSH.
I'm more of a high level person.
I'd rather pay somebody else to do this for me.
Well, let's like, but it looks pretty.
Let's see if I can pull up a picture of top.
On behalf of my employer, Brian, thank you.
Thank you.
Yes, we're happy to take me to do that for you.
No problem.
Like this, this is the thing it's replacing, by the way, is it's just a list of processes and it moves around all the time and the stuff shoots off the screen.
It's like, why is it doing this?
You know, it's.
I mean, to be fair, it was also written probably in like 1968 or something.
Yeah, yeah, absolutely.
Yeah.
I'm not knocking on it.
I'm just saying I would rather use, you know, something like this that gives me way more real time.
Like graphs, not just snapshot.
Now it is.
But like, here's, here's what's happening as the things are running and doing their work.
Right.
So that, that's super helpful to me to see.
To be fair.
I also just, just installed Tiptop just for the heck of it on my Mac and a really quick install.
And it does look pretty.
I just don't know what to do with it.
But yeah.
And by the way, let's just tie it back to the audience again.
So Tiptop uses textual for layout, which is based on, which is Wilma Googan's thing also based on rich.
So it's quite modern.
I think if we had Ned and Wil in the audience, we would always touch on something they were doing.
Every year.
I'll tell you what, Brian, here's a way you could use this totally in your day-to-day life.
Your Mac is running slow, right?
And you say, why is my Mac running slow?
You could totally use this and at a glance say, what's hoovering all my RAM?
What is throttling my disk?
What is using up all my CP, et cetera, et cetera.
Okay.
Yeah.
Like, for example, you can run activity monitor, which will show you the process thing.
And you go to the memory thing, which will show you just a single graph.
But like, why is it using all that memory?
Is that actually being used or is that cache?
Because you loaded a bunch of files and now it's kind of could be free, but it's not.
What's happening on the network?
You can't see very well on macOS.
I don't know why activity monitor is adverse to showing you network traffic, but you could
see like, am I downloading something or not right now?
I can be taught new things.
Okay.
Thanks.
Yeah.
Anyway, I'm not excited for that use case.
I'm excited for using it on the servers to understand them better.
Yeah.
All right.
Let's bring it home with some more retro, Chris.
Back in the world of Wibzy.
I've always been into, well, you know, always, ever since they weren't retro and they were,
they were just 8-bit machines.
I have, I grew up, started with, with 8-bit computer.
I was an Atari kid.
Never owned a C64, but someone has actually written a Commodore 64 emulator called PyC64
in pure Python.
And the reason that I picked this is-
That's wild.
It really is, isn't it?
I think it is so cool because not only do you get this Commodore 64 emulator, it's not
100% coverage, like he has openly emulated the display to the point where you can display
Petsiki characters.
But the bottom line is, it is other than that, a largely full machine emulator.
It runs on the Pi65 library, which is a 6502 CPU chip emulator also written in pure Python.
Oh my gosh.
That's hard work.
It really is.
And, and it's so neat.
I was able to pull this thing down and run it.
And I was actually writing like Commodore 64 basic that poked bits into screen memory and,
you know, drew pictures off the screen.
And, and one of the really cool things is this thing even concludes, you can sort of like pop
out of your emulator 64 with the gopy command to get a Python repl.
Skype is actually doing that, like, quality quality, the emulated Commodore 64's memory
by tweaking the Python data structures inside the repl.
It is just, it is.
That's pretty deep.
Thank you.
It is, it is really cool.
And, and it is also from my perspective, one of the reasons I picked it, it's a testament
to this programming language that we all use and love for our work.
And, you know, whether it's webby stuff or testing devices or in my case, automating network
infrastructure, it can do so much.
You can reach out into all these other really kind of, you know, cool niche areas, whatever
you're interested in and, and do neat things with Python on your computer.
So I just thought this was incredibly cool.
I had enjoyed playing with it and I definitely intend to enjoy playing with it some more.
It might be some interesting ways to go back and study operating systems back when they were
simple.
If you were like a computer science student as well.
Yeah, for sure.
I think there's a lot to learn there.
Um, you know, even if you're not necessarily someone who is keenly interested in writing your
own operating system, I think there's a lot to learn just in terms of the programming paradigm,
right?
Like when, when you wanted to solve a problem and you want it to display something on the
screen and a perfectly reasonable answer was I'm going to poke the right bits into screen
memory because that's a more efficient way to display this thing as opposed to using whatever
my high level language provides.
It is, it's such a different way of thinking the reality that most of us have it day to day.
It's just a fun, refresh, refreshing place to visit, at least from my perspective.
Yeah.
Very cool.
It's a good one.
Thank you.
Brian, it's time for extra, extra, extra here all about it.
How many extras do you have?
I have zero today.
You have no extras.
You're not adding to the number of extras.
Chris, anything else you want to throw out there that people know about?
Sure.
Absolutely.
My, my team at Amazon is hiring.
Hopefully we can put a link in the show notes.
I work for OpsTech IT.
We work with the fulfillment centers and I, the, I cannot say enough how much I love working
with this team.
People are incredible and they're sort of like, at least the folks I'm working directly with
are people who started out in the fulfillment centers and were interested in technology, right?
And they were able to prove that they could actually walk the walk.
And now they're being promoted into full on engineering positions.
And I just find that so incredibly refreshing.
Yeah.
That's fantastic.
Yeah.
These people just have a very different perspective.
You know, they're not the jaded.
Like I've been in the tech industry for 20 years types like me.
It's, it's really inspiring to work with them.
Yeah.
Not, not too many debates of like, why don't we just switch to a functional language?
Come on.
More practical.
We'll just get stuff done.
I love it.
Exactly.
Thank you.
Before we move.
Yeah.
So we'll put that link in the show notes.
Before we move on to my extras, which are not many.
Sam says, it's so helpful to understand how a computer works when you're writing code.
These kinds of things are a good way to learn that.
Talking about the Commodore 64 emulator.
Awesome.
All right.
So I just have one and that's, I'm going to be speaking at FlaskCon 2021.
And so FlaskCon is the first week of December, which has a remote attendance.
I'm going to be talking about HTMX and Flask once again, because I'm super psyched about
all that stuff.
And so if you want to see some really cool apps get built using HTMX and Flask, drop in over
there.
There's a whole bunch of other talks going on as well.
So I'll put the link to that in the show notes.
All right.
Shall we finish it off with a joke?
Yeah.
Levity is the soul of wit.
Absolutely.
It is.
So we just recently went through the whole daylight savings unraveling and went back
to normal time.
And I don't know what it's like for you all, but when I look at the weather and stuff here,
I suspect Chris, it's similar in Boston.
I mean, I know it's similar for you, Brian.
Yes.
It's like sunset is at 438 PM and we're headed towards the winter solstice.
So it's only getting earlier in the day.
It's like, ah, that, that, that doesn't make me, it doesn't spark joy in my, in my heart
that it's already dark at 430.
I like it.
Yeah.
I like the dark.
I don't mind that it gets dark.
I just wish it wouldn't.
I wish we could just move it.
So there's a little time at the end of the day where there's still sun.
I remember.
I'm with you, Michael.
I'm solo powered.
So I enjoy the sunlight.
I don't like it getting dark earlier either.
Yeah, absolutely.
I don't hate the winter.
I just don't wish it shifted around a little bit.
So this is a technical take on that.
Our joke as this comes from the New Yorker.
I mean, we're pretty highfalutin, right?
We're not doing like dev jokes.io or whatever it was anymore.
So this is by Matt Reuter and it's, it's a little cartoon of two people clearly in kind
of a darker space, a woman sitting in her chair with a laptop and you could just like
the beaming laptop screen is like a lightener up.
Right.
And the guy here on his tablet, same thing glowing on his face.
And underneath it, it says, I hate how the screens get bright so early this time of year.
Nice.
This is so much about our modern world.
It's both, it's both hilarious and kind of sad.
All it wouldn't go.
Absolutely.
Yeah.
So welcome to daylight savings or standard time.
I guess everyone, the screens do get bright this time of year.
That's funny.
Nice.
And Chris, thanks for being on the show.
Brian, thanks as always.
Yeah.
Thanks a lot.
Thank you so much.
You know, I really appreciate everything that you guys do for the community.
It's, it's really, you guys are great examples of why Python is such an amazing place to
live.
Thank you.
Thanks, Chris.