Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.
This is episode 240, recorded July 1st, 2021, How Time Does Fly.
I'm Michael Kennedy.
And I'm Brian Okken.
And I'm Chris Moffitt.
Hey, Chris. Welcome to the show.
Thank you. Great to be here.
Yeah, it's great to have you here.
We've had you talking about the missteps of Excel and how the Python data tools chain can make that better over on Talk Python a few times.
This is your first time on Python Bytes, right?
It is. Yes.
Yeah. Exciting to have you here.
Definitely. Definitely.
But maybe you want to go ahead and kick us off.
I want to talk about subclassing today.
But Hinnick wrote an article called Subclassing in Python.
And, you know, dealing with classes is just everywhere in Python.
Even if you're not using classes, Python itself has all sorts of classes and objects that you're using all the time, whether you know it or not.
But when you start getting into larger design, there is a question around, you know, composition versus inheritance and stuff.
So I really like this article that Hinnick put together because I think people should think about the ramifications more.
So the general gist is he prefers composition over inheritance and I do, too.
And but then goes through if you have to do inheritance when sometimes you do in Python, for instance, the greatest the greatest example I know of is when you're having exception hierarchies.
And it's really it's really easy to build up exception hierarchies in Python.
And it's there's like nothing there is except for like the class definitions and their inheritance.
And that's the easiest class you've ever created.
Class exception name path.
Yeah.
And but it's useful to do that.
But then if you want to go further, there's other there's other design patterns and stuff, especially from the C world, C++ world where people might be thinking, well, I want to do something similar in Python and stuff.
And so this is this is actually kind of a really great article to discussion about it.
It's pretty long.
I don't want to summarize it too much, but I'll jump into the three types.
So he talks about three types of subclassing that's often happens subclassing for code sharing.
And the short answer is it's just people are trying to do the drive principle and try to share code.
And it's just it ends up being a bad idea, essentially.
And there's a bunch of references for it.
And if you don't, I think if you don't believe me or him, read, read this article and read a bunch.
He's got a whole bunch of linked articles to that discuss it.
But but I kind of agree.
The the second type is inherited in is abstract data types or interfaces in a lot of languages are called the interfaces.
And this is kind of a neat use of it.
And there's a bunch of things.
But it's it's and I thought, OK, yeah, you definitely will use.
It's a little bit of inheritance and composition for for data type stuff.
But there's in his discussion, he talks about some of the cool things that Python has that allow you to to have these sort of hierarchies without actually doing subclassing.
So there's there's some cool features of Python, like the protocol syntax that came in recently in in typing dot protocol.
Yeah, protocols like formal duck typing, which is an odd thing to combine.
But yes.
Yeah.
But it's it's kind of really cool how it's put together in Python.
So I like that.
And then lastly is a specialization.
And that's where kind of the exception hierarchies come in.
But also he's got a great discussion about structuring data classes that have common elements.
And and I think that's an interesting discussion, too.
And I I think I already said this.
The summary really it's really hard to summarize this article other than it's good to think about your design, especially if you're going to try to bring subclassing into it.
So let's do that.
Yeah.
Awesome.
I haven't had a chance to dive into this article, but I do want to read it and explore it.
You know, it touches on a couple of things like it touches on namespaces and modules, which I think is pretty interesting.
So many people coming from C++, C#, Java, et cetera, like all these really strongly OOP, especially C# and Java, where everything has to be a class.
They you'll see people creating classes just for things like static variables and so on.
Right.
Or static functions.
If you just have a bunch of static functions, you know what works really well for that?
A module that has functions in it.
Right.
That's the same thing.
You import module.
Then you say module dot function name is the same as from module import class, class dot static function name.
Right.
Like it's just a layer that doesn't really need to be there.
So the article touches on that, which I think is neat.
Like sometimes you just don't need those.
And then also the composition over inheritance.
I think composition over inheritance is a really important thing to think about, because so often people say, well, you can't use OOP because it's horrible in all these ways.
And you end up with like a robotic duck that can't quack.
Right.
Like you end up with these weird situations.
If you like derive too many things and you put a weird specialty on the end, you're like a duck is an animal.
And then like, but it has wings.
But wait, now it's a robot.
Now, why does it eat water?
You know, it's like what happened to it?
Right.
But the composition allows for you to keep things much more tight and small in the inheritance stack, but still put them together in meaningful ways.
So anyway, yeah, I want to see more about this.
This looks great.
And I'm coming from a standpoint of I'm a C++ person as well.
And I've done both extremes.
I've like gone way down the inheritance hierarchy thing and had like seven deep in hierarchy, maybe not seven, but like five deep.
And it gets to be a nightmare.
So then I got went to the other direction and didn't do any inheritance at all in a design.
But there's there's problems there, too.
So thinking about it and doing it smartly is you just need to.
Yeah.
So it's often like salt.
I could see it's really good when you have some.
You try to go like salt.
It's great.
I'm going to have that for dinner.
Like, no, you shouldn't do that.
I think the other thing that's really important about this is depending on how long you've been working in Python, sometimes you kind of get stuck in a rut.
And you're always doing the same thing.
And the language has evolved and grown over time.
And so I think articles like this kind of force you to take a step back and see if you're using all the new features in a way that maybe aren't idiomatic.
Also, quick comment out there from the live stream.
Paul says, first time watching the live stream.
Hey, Paul.
Weird seeing everyone when they say the intro.
Indeed, it is kind of weird.
But I want to highlight this one to say hi to Paul.
Thanks for being here.
But also, if you're listening, you're like, hey, I'd kind of like to see what's on the screen while you're all talking about this.
You'll follow us on YouTube.
There's like a live stream menu right on Pythonbytes.fm.
So it's easy to sign up for that.
And also, Sam out there in the live stream following up on that says, currently maintaining a library with a deep templated class and hierarchies.
It's very hard to keep track of it all.
Yeah, I hear you that.
That's for sure.
All right.
Let's switch over to the next one.
Now, I tried.
I tried to resist this, Brian.
I promise I did.
But I've ended up with an extra extra with seven more extras.
Hear all about it.
And it just had to become a main item because otherwise we'd be here for hours.
It's not the idea of the show.
So we've got an extra extra.
Hear all about it.
Nine extras.
Let's pull them up.
Action number one.
We've talked about Pyodide.
I had a whole talk Python episode on Pyodide, which is an interesting thing.
It is this project by Mozilla where you take Python and you run it in the browser.
And then you take many of the data science packages like NumPy and Matplotlib and stuff and compile them into the browser.
And then you basically have client side Python data science, which is really interesting.
This project is being spun out as its own topic, as its own project.
It's no longer under Mozilla.
Usually that doesn't sound good to me.
It kind of sounds like it's been orphaned.
I have no idea what the status of Pyodide is.
People can check that out.
But it's no longer under Mozilla.
It's its own separate thing, as they say.
So it's cruising out there.
And also, it didn't get compiled to JavaScript.
It got compiled to WebAssembly, which is interesting because that's faster.
All right.
That's number one.
Number two, I just, as in a couple hours ago, released a brand new course, Python powered chat
apps with Wilio and SendGrid.
So the idea is if you want to have some kind of chat bot, but a lot of that conversation
has to involve your database and your data and verifying things.
Like the app that we built here is a tech savvy bakery where you can order cakes by sending
it a WhatsApp message.
And then it'll say, hey, you want a cake?
Well, here's the menu.
And it actually gets the menu from our Flask app.
And then they pick something off the menu.
And once they pick all the details, they said like this.
Okay, great.
We send it back to our website and figure out how much that's going to cost.
They order it.
It goes back.
We send them a, if once they accept it, they get like a customized pretty email.
It goes back to the backend.
The bakers bake it.
It sends them another message to let them know.
So if you want to build kind of like that workflow, if Wilio and SendGrid, check it out.
This course is super fun.
It's six hours and it's 100% free.
So people can check that out.
I think if you're trying to build that kind of thing, that'll be a lot of fun.
So links in the show notes there.
I had something.
Yes.
If I can't afford free, can I get a discount code?
I will give anyone listening 50% off that.
So I have this really cool tweet and Twitter is broken from what I can tell for everything
that's not the homepage.
So something went wrong, but let me describe it.
So when you look at it in the show notes, you'll be able to see there's a really cool tweet
from Nick Maul, who was on the guest on the show last week.
Oh, you got it.
How can you get this to work?
You've got some sort of magic.
All right.
Well, so thanks for putting on your screen.
So here we have Will McGuigan showing an animation of basically this really cool, like collapsible
sidebar and like scrolling within sub windows inside of textual.
Right.
We talked about textual as well.
And it's just such a cool graphic that says like, wow, you can build some pretty amazing
applications there.
What do you think?
Will's just knocking it out of the park with this.
It's fun to watch him go so fast.
Absolutely.
Well done there, Will.
I'll switch back to mine for a moment.
Okay.
Ars Technica works on my computer.
So remember we did an episode and I titled it something like Flock No or something like
that.
So Flock, Federated Learning of Cohorts, is something that Google was trying to do so that they can
replace third-party cookies.
Why?
Because people are running ad blockers or like I am right now, a VPN that at the network
level blocks all the ad tracking and third-party cookies.
So they're just basically not working very well anymore.
So they need to, they're going to cancel third-party cookies from, which means they're
canceling for the net, the internet.
And, but because they're Google and they're based on ad revenue primarily, they can't just
go and we canceled tracking.
Hooray.
We're all winning on privacy, right?
It has to be replaced with some other form of tracking, which they call this Federated Learning
of Cohorts.
But the Federated Learning of Cohorts has all these almost more negative consequences.
And I don't want to go too much into that because we went into quite a lot of detail.
But for example, you can say, I would like to target, you know, lesbians who just got divorced.
You run an ad on that.
People show up on your site.
They sign up.
You have an email.
And now guess what?
Not only do you know what their email is, you know that they're in this group.
And maybe this is the very first time you've ever met them, right?
So really weird, creepy stuff that you could like pull out with this.
Anyway, the big news is Google delays the rollout till 2023 because you know what?
People don't like it.
They're not super keen about it.
So there's a whole bunch of people who are against that.
But you're saying they're just delaying it, not stopping it?
Yes, for now.
Like, let me, this is a great article that people should check out.
Like, let me read the first sentence or two.
Google's plan to up in web advertising and user tracking by dropping third party cookie
support in Chrome has been delayed.
Most browsers block third party tracking cookies now as the VPN, like I mentioned.
But Google, the world's largest advertising company, it wasn't going to follow suit without
protecting its business model first.
But there's a lot of challenges with this.
A lot of people have come out against it.
And yeah, it's not going to work out super well.
So they decided to delay it.
That's what they said.
Stage two starts mid 2023.
Google says it's received substantial feedback.
Including from us.
And other companies out there are like, we kind of want to keep tracking too, but we're not
really excited about this.
So we're just going to not say anything.
Like Apple, Opera, Mozilla, Microsoft.
Yeah, they're like, ah, what is I sure about this?
Anyway, yeah, they've received substantial feedback.
So hooray, I think for now.
One thing that we don't talk very often about in Python is what if you want to ship your code
to somebody and it has sensitive algorithms in it, right?
It's not that common, but you could get Py2.exe or Py2.app bundle up your code and give it to somebody.
For example, Dropbox does something to this effect, right?
Got your Python code running up in your little menu bar.
There's other ones as well.
But you might want to encrypt how that works or protect how that works.
So people can't just open the PY files and look around.
So there's this thing called Source Defender.
I'll be clear, this is a paid commercial product.
I have no affiliation, but they pointed out, they sent me a message.
Hey, we're doing this thing.
What do you think?
It looks kind of interesting.
I think it's going to be a pretty limited set of people who actually care about this.
Like if you're running on Docker, you're running on the server, you probably don't care.
Maybe you do, but probably not.
But if you'd like to be able to encrypt your source code so it's much harder to see and then
ship that to somebody, you can use this thing as part of their paid service.
So that's kind of cool.
People can check that out.
Let's see, oh, there's a play noise.
I don't want that.
So I was recently interviewed on a day in the life and a work from home Pythonista, which
is a cool series being done by the folks in the Philippines.
If you want a tour of the behind the scenes studio and all the work from home stuff, people
can check that out.
Python 396 was just released.
We can check out the change log and see what's happening there.
There's a security HTTP client about what I think is like a denial of service.
It sounds like it avoids an infinite loop sort of thing.
So that might matter to people.
Probably not, but maybe it does.
Then a bunch of changes that are happening here, including platform specific ones.
So if you're running Python 39, and why wouldn't you be?
Update that.
Because you're running 310.
Yes, that's right.
You're already ahead of the world.
You're in the future.
So also, we had Calvin on from six feet up a while ago, and we talked about the conference
that he was putting together.
Well, the videos from that conference are out as a YouTube playlist, so people can check
that out.
I don't remember how many videos there are.
Let's click on it and see.
There are 61 videos, including one on the Python memory deep dive talk that I gave.
So if you want to check that out, they can.
Let's see.
Oh, this one.
Check this out, Brian.
Have you seen this?
Did you know you can pip install Python bytes?
Yeah.
You can literally pip install Python bytes because of Scott Stoltzman created this for us as a
joke.
He was listening to one of our episodes.
I can't remember what we talked about.
This was episode 239, but we must have talked about packaging and pip and things like that.
So he created a package called Python bytes.
And what it does is basically you give it a number, like 240, and it would download this
version as an MP3 file and put it right next to whatever the working directory is.
So if you want to pip install Python bytes and then python bytes.downloadepisode instead
of using a podcast player, we're all for that.
You can check that out.
Yeah.
And that's it.
That's extra, extra, extra, extra.
Well, many, many extras.
Yeah.
So the Python bytes package is just sort of, it was for fun, but it also, it's really
small.
And one of the things I like about it is it's just a really cool example of like with Python,
you got something that downloads MP3 files off of a feed somewhere.
It's that easy.
It's just, that's pretty cool.
Yeah.
That's fantastic.
Absolutely.
All right.
Let's see.
A couple of things from the live stream.
Sam says, things have happened with Mozilla the last two years that really shook my confidence
with them.
I am still a big fan of Firefox and I support their mission, but yeah, it's, I want to see
them succeed.
Let's see.
Another one from the live stream.
Antonio said, Hey guys, have you mentioned Kivy before?
Hey, GUIs and Kivy.
There you go.
I watched a video about this week.
It's a GUI that's compatible with many things, including the mobile devices.
I do.
My feeling is that Kivy is, is a lot about, it's more about building almost game-like interactions.
Whereas a lot of GUIs people want, they want like, here's a text box.
I type in the text box.
Here's a button I drop in, you know?
So, but yeah, pretty cool.
Well, let's see.
Kim Venwick says, as an aside, shipping a Docker image won't obfuscate the Python.
The image can be taken apart in files like that.
That's true.
They absolutely can.
I was just thinking like, you're probably just running on like a container service.
But yeah, if you're shipping it to someone, it's the same.
Nick Harvey out in the live stream says, could just send the PYC files with no PY.
It's not foolproof, but it does require more work.
You're right.
You'd basically be down to like this dot this and like reading the byte code.
Yeah, for sure.
Let's see.
Final one.
Rahan says, if it ends up running code on your machine, you can read it.
It's about putting enough barriers that people won't bother.
Yeah, that's definitely true.
I mean, you think of C++ and things like that being completely opaque and yet people take
that apart all the time.
But there is also a difference from I'm literally shipping you the source files here to, you know,
because then you could go in like, oh, here's where the license check is.
Let's just, you know, command slash comment that out.
All right.
Now we're ready to run.
Yeah.
Right.
You want to make it a little bit of a challenge, at least I suspect.
Anyway, thanks for all the feedback out there, everyone.
That's the everything extra, extra nine times.
All right, Chris, what's your first one here?
All right.
So the first one is from Andreas Kanz, I think is how you pronounce it.
And it's a library called Clib, I believe.
I wasn't sure if it's K-Lib or Clib, but I think it's Clib.
And it's for automated cleaning of Pandas data frames.
I guess I should even say it's a little bit more than just cleaning.
It's automated analysis.
And, you know, I'll be the first to say I'm a little skeptical about some things that try
and automate the process.
But I was playing around with it.
And there's some pretty cool things that it does.
The documentation, probably the best way to learn about it is the Towards Data Science article
that he wrote, which gives a pretty nice overview of what it does.
It has some, as I mentioned, some pretty nice cleaning features as well as analysis features.
So I was going to kind of go through a couple of the, describe a couple of things.
The first one that I thought was really interesting is the, there's this function called data cleaning.
And it essentially does, you can control what it does.
So it can clean the column names.
It can convert data types.
It can drop missing.
So one of the things that pandas does is it's not really aggressive about the size or the data types that it uses.
So when you read in data, it will just kind of assign it maybe to a float or, you know, an object.
But if you want, you can get in there.
And if it's a value, if it's a column, let's say that has only values from, you know, less than 100.
If you convert it to an integer, it saves memory.
If you save enough memory, then you can actually speed up your code.
And so this goes behind the scenes and takes your data frame and converts it essentially to the smallest value, NumPy value that it can store.
And then, you know, I took a random data set and sure enough, it did reduce the memory footprint quite a bit, which I thought was pretty interesting because it's one of those things that is very tedious to do on your own by hand.
Does it do like if you have the same string?
Does it just create a pointer to one copy instead of having that many times, stuff like that?
It can do that by converting it to a category type.
That's essentially what pandas is doing when you create a category.
It does that to kind of string to like a list conversion.
And it's, you know, it's pretty effective.
And yeah, I've used the category piece before, but I haven't actually gone in and tried to, you know, shorten up the numeric columns, which is really useful.
The other thing...
You just convert them to all the integers and then it'll just be shorter.
So you don't have to worry about the size.
I'm just teasing.
Yeah.
But I mean, it does even do, it's like, it can do even like int 16s or int 32s or int 8.
Oh yeah, interesting.
Like it'll shrink to the size that'll like, oh, these are all under 256.
So we'll go to like one byte.
Exactly.
You know, and I haven't looked at the code to see, you know, how it actually figures it out, but I had a fairly large data frame and it was pretty quick.
The other one that was interesting is the clean column names.
So I think there are some other libraries out there that will like strip spaces or special characters from column names.
But what this one will actually do is actually, if you have a column name that has, let's say camel case, it'll convert it to all underscore.
Or it will just essentially normalize all of your column names, which, you know, you could have a debate about whether you want to do that.
But when you have a data frame that has a lot of columns and you're just looking at it the first time, that can really be helpful.
And then the other function that it does that works pretty well is for cleaning duplicate data or empty data.
So if you have a lot of columns that have no values in it or just, you know, maybe 90% of the values are empty, you can set thresholds and just clean that all out.
So I was playing around with it and I was pretty impressed.
And I kind of wanted to call it out because the documentation right now is mostly around the Jupyter notebooks that he has.
So I think, you know, it would be nice if we could get some more docs in there and some more examples.
But overall, I was really impressed with the library.
And I think people should kind of take a look at it and see if it's something they want to use for some of their own processes.
Yeah.
Some of them sound interesting, even if you don't have to trust it, right?
Like the shrink to the smallest data set, data type, for example, or normalized column names.
Those don't seem as risky as, you know, clean it up.
Exactly.
Find the wrong data.
Exactly.
And then I forgot to mention, it also has some nice correlation plots.
And some of these things you can already do with Seaborn or Matplotlib, but I found that it gives you a little more control and it's just a little bit easier to do it.
There are certainly other tools out there that do this as well.
So, oh, and then the categorical data plots I thought was a nice summary of the data and gives you some nice graphs and it helps you understand where you've got some missing values.
But yeah.
Yeah.
The visualizing the missing data is a really interesting feature.
Yeah.
And there is another Pandas data frame called MissingNo that does this and does it well.
But I think this is a unique combination, especially some of the data, the memory saving features that it has are pretty neat.
Yeah.
So the cleaning features, though, have a lot of, there's a lot of parameters to it.
So it looks like you have a lot of control.
And one of the, I mean, again, this is open source, so it isn't that magical.
You can just look at the source and see what it's doing.
So it looks.
Exactly.
Yeah.
And that was one of the things I was looking at is like data cleaning, I think is kind of the top level.
And you can just run that wide open and it'll do everything.
And it actually prints out a pretty nice summary of what it does.
But you can also go in there and specify parameters, like you said, to control it so that maybe it doesn't rename the columns.
Or drop some of the missing data.
The other thing that I tried to play with that seemed really interesting is this pool duplicate subsets.
And essentially what it tries to do, and I had a little bit of trouble with this because I think I put too much data at it.
But it tries to, maybe if you have 10 columns of data, it says, well, you know what, four or five of them are very heavily correlated.
So we're going to drop them and just give you the four or five that are actually most useful.
And so I think that's some interesting tools to use when you get some data that maybe you haven't worked with before.
Yeah.
Very nice.
What a good find.
Brian, you got the next one?
Sure.
Yeah.
Just a second.
So I wanted to remind people to every once in a while, look at functools.
Because I've experienced functools as kind of an interesting library that's built in.
That is, it kind of grows with you.
So if you're new to Python and you look at it, it's going to be confusing.
It's like all intermediate stuff in there.
But as you learn and experience more Python programming, come back to it every once in a while because there's stuff in there that you'll use that you didn't think about before.
So I'm going to go through a few things.
And actually, I wanted to call out, there was an article by Martin Hines that I read that kind of reminded me to go through and look this.
So I want to shout out to him.
Thanks.
We've talked about some of this stuff before.
We talked about function overloading and using single dispatch as one of the ways you can do function overloading in Python, which is cool.
And that's part of functools.
And hopefully people are familiar with wraps.
Wraps is a way to create decorators that act like the thing that you decorated.
And so if you're writing decorators, make sure you check out wraps.
And then caching as well.
I think, I'm sure we've talked about LRU cache.
I'm sure we have, yeah.
Yeah.
So that's in functools, the caching.
And new in 3.9, there's just a simple cache.
You don't have to say LRU cache.
And it's just a convenience wrapper around LRU cache.
But it also, there's no max size.
So you don't want to do that for things that you actually want to throw items away.
But caching is super cool.
So check that out.
And then I didn't.
When I first saw the LRU cache, I'm like, whoa, I got to go figure out what this LRU is.
And it's not like, rather than just like cache the response.
Yeah.
I guess the other question, though, you might be is like, well, what if you pass two variable or two arguments or sets of arguments?
How do those?
Yeah.
So either way, it's kind of not 100% totally obvious what's going to happen.
Yeah, it's very cool.
Yeah.
So there's a bunch of caching stuff in there, like the LRU cache.
But then you can also cache a property.
And actually, the property one I hadn't used before.
But I was playing with it this morning.
And it's really cool.
So like, for instance, if you've got a data class or any class that has a bunch of stuff, and some of the things, you have an expensive read on one of those because you have to calculate the value.
You can throw a cache property on it, and it looks pretty cool.
One of the neat things about it is, so it only reads it once, and then it caches the value of the property.
And if you need it to refresh, you call delete on it, which is kind of a weird but kind of cool also.
But it's odd to call delete on something that you want to still be there, and it'll just reread it next time.
So that's how that works.
That is weird.
That's definitely weird.
Total ordering, I didn't realize, was there.
So if you have some data type that you want to be able to compare, you can use total ordering to define equal and one other operator.
And then you get all of the comparison operators to show up.
You can use that.
And then the last one I wanted to highlight is partial method, which partial and partial method, which these are kind of neat in that, like, let's say you've got a function that takes a whole bunch of arguments.
But you want to pre-fill some of those in and create a new function that has some of the arguments pre-filled in.
You can do that with this, and it's pretty neat.
Yeah.
Okay, interesting.
I see you partially supply some of the arguments, but not all of them.
Yeah.
So, yeah, just a shout out to this, that, yeah, these are intermediate or advanced topics.
But, but there, as, so as you learn more Python, come back to this every once in a while and you, you might use, find it useful.
Yep.
Indeed.
I was like, how did I miss this hashed property thing?
Like, surely I would have paid attention to that because what, so often these properties that are like computed things, but they, you know, often don't change.
Right.
You get something back from the database.
You want to, it has time sorted in seconds.
You want to know how many days it is.
So if something happens, you might have a day's property.
Right.
But that's probably not good.
So having that hashed is cool.
If you're sure it's not going to change.
But I'm like, how did I miss it?
It's new in 3.8.
Yeah.
So it's not, it's not super old.
And like Chris said, one of the reasons to revisit a lot of these things and pay attention to the news on Python is because the language changes like this.
So, yeah.
Yeah, for sure.
Kim out there in live streams says, also worth looking at intertools from time to time.
Definitely.
Great.
Indeed.
It's in the same level of complexity, but for collection.
It's kind of like, you wouldn't first go there, but eventually like, oh yeah, this is what I wanted.
I just didn't know it.
Speaking of things you didn't know it, let me scare you all a little, make you all delighted.
I don't know.
You tell me how you take to this.
So let me set the stage.
GitHub has a little bit of source code.
Much of it actually public, right?
Like it's public repos and whatnot.
So it can be analyzed and talked about and shared or used to train an artificial intelligence, which is pretty crazy.
And if you look at the artificial intelligence around text, there's the GPT-3 stuff, which is like scary, good text-based AI.
Well, they decided, what if, you know, our parent company also makes this editor?
What if we hit an AI based on understanding the source code from GitHub, like all the source code from GitHub and put it into VS Code and then it did stuff?
Have you all seen this?
It's called GitHub Copilot.
Yeah.
I haven't done it yet.
I was going to put the link in there and you beat me to it.
Oh yeah.
I was on top of it.
So if you go over here, there actually works for TypeScript, Go, Ruby, Python, a couple other languages.
It says it works for many languages, but it's best on those, of course.
But if you just look like at their homepage, the copilot.github.com, they've got this little animation and it says, I'm going to write a function that says parse expenses.
And it takes some kind of text.
And you put a doc string, literally a doc string in Python.
It says, parse the list of expenses and return the list of tuples, date, value, currency.
Ignore a line starting with hash.
Parse using date time.
Here's some examples.
Tab.
And then it writes the code that does that.
And let's see, what is it going to do?
It says, it's in the middle of animation.
It creates a list of expenses.
It goes through each line on split.
It says, if the line starts with hash, this is all Python code.
Continue on your loop.
Otherwise, date, value, currency equals split it.
And then it knows how to parse the date one.
Convert the value to a float and then store the currency as a string.
And it's not just that sometimes it'll do this.
You can actually get alternate implementations by tabbing through its recommended solution, which is,
pretty crazy.
So this is powered by OpenAIs.
It's called Codec or something like that.
I don't see it right here right now.
Anyway, I'll probably run across it in a second.
That's what it's powered by.
And it says things like, you're the pilot.
So with GitHub Copilot, you're always in charge.
You can cycle through alternative suggestions and choose which to accept or reject and then manually edit the suggested code.
Oh, yeah.
And it learns from you.
So I don't know.
This is wild, y'all.
This is pretty wild stuff here.
What do you think?
I think it's really impressive.
I mean, it will be interesting to see what it's like when you use it in real life.
And I think that there could certainly be limitations.
But I don't know about you.
But whenever I'm programming, there's always these things I just need to go and look at the documentation or look at Stack Overflow to refresh my memory.
You've got to connect to SQLAlchemy.
And I totally forgot how to do those three steps for that connection string sequence.
Right?
Exactly.
Yeah.
And I've seen, I saw on Twitter where someone was throwing a little shade at that example that you're walking through because they said, well, why are you storing the currency as a float?
It should be a decimal because if you store a currency as a float, you're going to have all the rounding issues.
So...
That's how Superman makes all his money or the evil villain in Superman.
What was it?
One of those shows.
Yeah.
Richard Pryor in one of the original Superman.
Yeah.
And it's not just based on the doc string.
Like the example I first spoke about was you wrote complex doc string and then say, do that thing.
But you can do it based just on function name.
You can just type a meaningful function name.
What was the example they used?
I can't remember.
But yeah, so you basically just write a doc string, a comment, a function name, or even some code to give more context to it.
And then off it goes.
So yeah, pretty neat.
Codex, that's the name of the AI system behind it.
So basically, this is a plugin for VS Code, but a really nice one.
So here's some examples we'll all be familiar with.
So fetch tweets.
And the example here is you literally write def fetch underscore tweets underscore from underscore user have.
And then what it auto completes with is, oh, you're going to need to pass the username in.
And then here's how you authorize with Tweepy, set up the API credentials.
And then here's the code you write.
Oh, yeah.
And here's your return.
Or I want to do a scatter plot.
And you write import, import matplotlib dot pyplot as plot, draw scatter plot, have.
And then boom, there it is.
Or memoization.
I wanted to point this one out because of what you're covering, Brian.
It says, oh, here's how you memoize a function, which is to, if it's past a set of arguments, it's always going to return the same answer.
So just give that answer.
Like, remember, these arguments equal this return value once it's run.
And it shows how to create a complex decorator that is going to have a function that remembers the values using caching.
It could just go at funktools dot cache.
You know what I mean?
Like, so there's things like that that is missing, right?
Because you could achieve the exact same outcome with funktools cache decorators, right?
Instead of trying to write a bunch of code that reimplements that.
But anyway, pretty, pretty wild thing.
I don't know really how to feel about that.
I've been thinking about this today.
It's kind of freaking me out, but it's also kind of cool.
Yeah.
I wanted to point out a comic that people have been pointing out with relation to this is, is the, you know, I wish we could have just specify what we wanted to do, the computer to do.
And, and it just does it.
And we already have that.
It's called code.
So anyway.
Yeah.
I, I don't think this, you know, people often say things like.
I remember hearing this 20 years ago.
Oh, this low code thing where you create these little boxes that do stuff and you drag and drop between them.
We're not going to need programmers anymore.
We're all just going to become drag or droppies.
And then like you programmers won't be needed.
The business people will just drag you drop either way the future.
And that never, ever happened.
Right.
Because people got to put them in production.
They've got to debug them.
They've got to scale them.
And so on.
Yeah.
I think the same thing here.
Like sure.
It wrote it once, but you can't have a write only experience.
For your code.
You have to understand your code and be able to evolve your code and work with.
So this might power you into a solution faster.
But I don't think it escapes the need of people doing meaningful software work.
The person that pointed out.
And several people pointed out the example of using money.
Floats and money.
That does highlight one of the problems with something like this, though, that everybody
needs to be careful of is the code that's generated.
Now you have to like, you were already creating, carefully thinking about it when you were creating
it.
But if something else creates it, you've got to scrutinize that to make sure that's really
doing the right thing.
And so you're code reviewing some AI code while you're coding your own stuff.
That's just a different part of your brain.
You've got to make sure that you're really paying attention.
Yeah.
And even I was looking at that matplotlib example.
And I would even argue that's not really the way you should do a scatterplot in matplotlib.
Because you should use the object-oriented interface in matplotlib.
I mean, the code will work.
But I wouldn't advocate that you use that code.
And so to your point, I think it will be interesting to see if it does learn on your own coding style.
So does it start to recognize those things that you're always, you know, like you said, connecting
to a database or fetching a file or doing a certain pandas function?
Will it start to learn that?
And then that would be really interesting.
I think there's something about it adapting to you and it learning from what you're doing.
But I have no idea what that actually means.
Yeah.
Hopefully it's paying attention.
So if it generates something and you change it to the different method and everybody else
is doing that also, maybe they'll stop suggesting the old one and start suggesting the new one.
Yeah.
I'll, you know, Chris, your point about having to, maybe it was you, Brian, sorry.
Whoever said about, you've got to like criticize this and you didn't write it.
So you basically have to study it and then, then understand or understand it and study it
to make sure it's doing the right thing.
You know, I, a couple of years ago, I don't know, a while ago, I was river floating and
broke my hand on some rocks, broke my finger in a bunch of places.
And like my fingers were completely wrapped up all the way to the very tips.
There was no like, oh, little pecking typing while my hand healed.
It was like, nope, no one handed, really slow.
So to keep things going, I used voice to text to try to like, at least keep email flowing
for a month or something, you know?
And what I found was I could write pretty decent emails.
It's hard to like stop and think in whole sentences the way the little tools like it to
work, but you can get it to work pretty well.
But the mistakes it makes that are phonetically correct, but actually what you mean wrong, like
they and they, or, or something that sounds like what you said, but is actually not
what you mean to say is incredibly hard.
It's much harder to understand and edit than you would think.
And so things like this, like, well, I wanted it to do that and I hit tab and I, okay, it's
doing, I feel like there's going to be a lot of blind spots.
Yeah.
Well, it did what it says.
It didn't.
I typed the thing and it seems right.
And like, how do you really, really know?
I, it just seems like in the same type of situation, it's going to be harder than normal code to
check.
Cause you didn't have to think through it to create it.
You know?
Yeah.
A couple of comments from the live stream.
Ray Han, don't, don't give him ideas.
It says, greeting Dr. Falcon.
Gosh, where have you been?
Let's play thermonuclear war as a dock string.
And Nick says, I can't help but think of Microsoft pay, which Microsoft pay was this really cool
bot that was like super good at adapting this stuff and they put it on Twitter, but people
decided to like be mean to, to, to it instead of teach it.
I think in like Japanese Twitter, it became a very kind and intelligent bot, but on like English
Twitter, it got turned into like a racist, horrible creature like right away.
And they actually had to cancel the project.
So yeah.
And then Arthur says, next April fool, April, April fool's day prank.
Everyone start writing terrible code that influences the AI.
And this, this is why English day went down the tubes.
Let's see.
And then Sam, for goodness sakes, don't trade it on GitHub code.
It'll arbitrarily turn on debug mode.
Yeah, perhaps.
Yeah.
Kim thinks this is both very impressive and vaguely unsettling.
And that captures what I was thinking.
Rahen, will it go and talk to the marketing people for me?
No.
I'm good with people.
That's what I do.
Yeah.
Okay.
Another thing that's not mentioned here explicitly, but I think is interesting is this code is coming
from GitHub.
Yeah.
When I go and I'm saying like, I'm working on super secret commercial project for large organization
that has lots of people trying to scrutinize it.
And I hit memoize tab.
It's going to write some amazing code.
Oh, by the way, was that GPL?
Where did that code come from?
Right?
Like what's the license of the code that was on GitHub?
Did I just now all of a sudden grab something that turned, you know, like if I was doing this
on Windows and I hit tab is Windows now open source.
I don't know.
That's a really interesting point.
And you would think if it was a small startup, someone will probably sue them.
But, you know, this is Microsoft now.
Yeah.
Exactly.
Yeah.
Anyway, so I agree with Kim.
This is both very impressive.
If this is the start, like where will it go?
It would be very amazing.
But it's also vaguely unsettling at the same time.
And I don't know how I feel about it.
Other than I wish it was in PyToram so I could play with it more often.
All right.
Chris, you got the last one?
I do.
So this is another library called Cats.
And it's a time series analysis library.
And it's made by the same.
Well, it's from Facebook.
And a lot of people may have heard of Profit, which is a library for time series forecasting.
And one of the things that's interesting to me about Profit and Cats is I think time series
forecasting is something that's really common in the business world.
I mean, you think about trying to forecast sales or maybe inventory movements or stock prices,
a whole bunch of different use cases for it.
And I think in general, most organizations don't have a group of PhDs that are really sophisticated
in their analysis.
So people use Excel and kind of come up with their own approaches.
And that's why I thought Profit was interesting.
And I think this is interesting because it does come from Facebook.
And you have to assume that they've got a lot of smart people that are doing a lot of forecasting.
And they've taken some of the things that Profit was good at and added some additional tools.
So before I go too much into Cats, one thing I wanted to mention is I did write an article
about Profit.
But I think other people, this gentleman, Peter Cotton, wrote an article about Profit and essentially
questioning how good it was.
And this is a really long, really well thought out article.
And some of the math and some of the concepts are way over my head.
But I do encourage people, if you're looking at time series forecasting, take a look at this.
But what Cats does is instead of just doing forecasting with Profit, it has a couple of different models
that you can use.
You can also do some more just basic time series analysis with it to detect seasonality patterns
and change points and other trends.
There's also, if you want to incorporate this in some of your other machine learning algorithms
to pull out features from your time series data, you can do that with this library as well.
And there's a whole bunch of other libraries or utilities to build like ensemble models and
other approaches for time series forecasting.
This is another one where it is relatively new.
So there's not a whole lot of documentation, but it's a whole bunch of different Python notebooks,
Jupyter notebooks, I mean.
And like one of the things I think is interesting is from a forecasting perspective, you can use Profit,
but use the same API and use Cerema, I think, Cerema and Holt Winners, as well as some other
ensemble models.
You can backtest, you can tune your hyperparameters.
And then you can also, it's got several of these other algorithms for change point detection.
And, you know, a lot of this, like I said, is I'm not an expert on the math, but I am interested in how you
figure out how to take these tools and apply them to those real world business problems.
And so I think it's really great when we have some of these libraries out there that are developed by
really smart people that do understand the state of the art, that can maybe make it a little simpler
for others to apply to their own unique challenges.
Yeah, this looks really nice to bundle these all together.
What's a type of problem you might answer with this?
I think so.
One example could be help me figure out what my blog or my website traffic is going to look like in six months
from now.
So I need to figure out, do I need to, you know, resize my servers or upgrade my disk space or...
What's my AWS bandwidth bill going to be?
Exactly.
You know, the other one that I think it's probably used a lot in inventory.
So trying to figure out, okay, what do I think sales is going to look like?
What do I need to reorder so that I actually have enough product so that we don't stock out?
I think those are some pretty common use cases.
A lot of the examples here are the airline flight data.
So anything that you have that's over a period of time, typically kind of on a daily basis over multiple years,
you can then start to forecast out what the future, what those future numbers would look like.
Then you have this magic prediction power for the executives.
Exactly.
And I think what's interesting about it is most of these, I think most times when people do prediction in Excel,
it's kind of you put the numbers in there and kind of do your linear line.
But these tend to give you more error bars so you can give a range.
So I think a prediction like this is much more valuable when you say it could be between, you know, 100 and 110 versus it's going to be 101.5.
And when you do that, it conveys a lot more precision than is really there.
Yeah, that makes a lot of sense.
Comment from the live stream, Sam Morley says,
when I was experimenting with time series data, I managed to get better results with a fairly simple, naive Dharma model than I did using profit.
And I think that's exactly what this article, I don't know if he's read this article, but this, the article from Dr. Cotton,
that's essentially what he says is some of the more simple models did outperform profit.
Yeah. Interesting.
Well, cool.
All right.
Brian, is that it for all of our items?
It is.
Got any extra stuff you want to throw out there?
Oh, I just had a, just a quick one.
Somebody on Twitter last week asked, why did I write a second edition of the book?
And so I thought, well, that's a reasonable question.
So at pytestBook.com, you can go and I've added a, why a second edition section.
So I think I'll read that.
New built-in fixtures, new flags, package scope features, f-strings, types, all sorts of good things are available that weren't available then, right?
Yep.
Yeah.
There's all sorts of reasons.
Always good to see Pathlib there.
I love Pathlib.
It makes my life so much easier when dealing with files.
Yeah.
I finally made the move.
I've put down OS.path and I'm now all about the Pathlib.
Loving it.
That's good.
Yeah.
Chris, anything else you want to throw out there?
I was going to throw out one other.
I was doing some research for working with units of measure.
UNYT.
That allows you to do things like convert kilometers to miles.
it works with numpy it works with all the scientific stack and that was um i i hadn't
heard of that one i thought it was kind of interesting and want to put that out there
next time you need to actually do something with units and convert uh back and forth might want to
consider that and then it looks like there's a lot of like physics and chemistry type things like
the mass of the earth the radius of the earth as constants probably pi and e and all those things
yes exactly and i think it's when you start getting into it there's probably temptation just
to code it all yourself just put those constants in there but uh when it starts to get more
complicated i think something like this could be really uh useful and then there's another approach
called pint which also works there we go uh which also works with units and it has a little bit
different uh approach and so i i think it's good to take a look at both of them and if you have a need
then you can decide which api is going to work best for your unique situation yeah these are cool i
haven't looked at unit but i love pint and i think the name is so good yes uh because my wife asked me
like how many else's are in a pound or how many pints are in a liter i'll be like or even in i don't know
a court or vice versa i'm like i have no idea i just these are such messed up volume measures and so
it's like here's the thing that takes the thing you don't really know about and allows you to convert it
to the others in a safe way it's good exactly i got one more quick extra to throw out as well for
you chris there i forgot to mention that you were the author of the move to from excel to python
with pandas course over at talk python trainings which is a really popular course basically it's
a intro to pandas course disguised as solving problems you might with excel right exactly yeah
yeah no thanks and uh i've had a lot of good feedback from folks so hopefully it's uh
interesting to the listeners that haven't had a chance to check it out i might have to buy that for
my boss maybe you can get that discount code yeah yeah the discount code all right you ready for
some jokes my twitter came back so i can show the twitter joke now yeah all right so dean who is
often but i don't see him this day today on the live stream sent a joke over and said do you know how
they say async in italian async yo or async io which i thought was a pretty good one async io i think you
love italian all right um you guys got another one out there i saw one in the notes another joke but i
didn't see who put it there i i've got one um so does anyone know why cryptocurrency engineers
allowed to vote no i don't know because they're minors oh that's a good dad joke yeah it is
absolutely well on that high note let's uh let's call a show what do you what do you say
yep right thanks as always chris thanks for joining us this time thank you very much really appreciate
it yeah bye everyone thank you for listening to python bytes follow the show on twitter via at
python bytes that's python bytes as in b-y-t-e-s and get the full show notes at pythonbytes.fm if you
have a news item you want featured just visit pythonbytes.fm and send it our way we're always on the lookout for
sharing something cool on behalf of myself and brian aukin this is michael kennedy thank you for
listening and sharing this podcast with your friends and colleagues