Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.
This is episode 320, recorded January 24th, 2023.
I'm Michael Kennedy.
And I'm Brian Okken.
And this episode is sponsored by us over at Talk Python Training and Testing Code and Patreon supporters.
We'll talk more about all that a little bit further into the show.
And also, yes, for those of you listening live, you already know this.
But for most people, they may not know that we're live streaming this on YouTube.
And if you just go to pythonbytes.fm/stream slash live, then you'll be able to follow over there.
Usually on Tuesdays at 11 Pacific Time, which it is right now, you can be part of the show.
That's how some of the folks are in the audience.
So hopefully you go over there and subscribe on YouTube and press the notify so you know when we're posting live streams.
And with that, let's get going, Brian.
Let's mark it down.
I'm going to talk about markdown a little bit.
While you're queuing up your next item.
We do pay attention to Rich and Will McCuggan in the show.
And I'm not going to talk about them too much, except for that.
I noticed that Rich updated.
We got a release note that they updated their markdown parser.
And from something called CommonMark, which I was familiar with, to markdown it pi.
And I was curious about this.
So I went and took a look.
So the CommonMark library, if we go take a look at that, that's up on PyPI, CommonMark, no big deal.
But if we go look at the source in GitHub, it says this repository has been archived by the owner on November of 2022.
Oops.
And if we look down a little bit further, it says, warning, it's deprecated.
We recommend using Markdownit pi.
And okay, I haven't used it.
So I wanted to go take a look.
So Markdownit, I think I'm saying that right, is it's built on top of another Markdownit tool.
So Markdownit is its own tool.
And the Markdownit pi is a Python wrapper around it.
So great.
So it's what do we have to worry about this one going out of a fashion as well?
And I think we're probably safe because the project, this is a part of executable books.
And executable books is a project to try to build more collaboration with Jupyter Notebooks.
So I think we're safe now to use Markdownit pi for a while.
And I just did a quick, there's a live demo thing.
There's converting Markdown.
It's kind of a nice package.
You can do it by itself.
You can convert from Markdown to HTML on the command line.
Or you can use it within your code.
So you can use, and it's got some nice features for in-code,
pulling in different plugins and stuff if you have different specialties.
And if we go back to the original release,
one of the reasons why they did it wasn't just why,
the reasons why Rich started using it isn't just because it's been,
the common mark's been deprecated.
But they say it will allow us to implement a number of additional Markdown features in the future.
So this is pretty exciting.
And yeah, I don't know a lot about Markdownit, but I am going to check it out.
So this is...
Well, if it's used by executable books and it's used by Rich,
I think it's...
Those are some pretty solid endorsements for it.
I saw that it got installed as well, and I wasn't sure why,
but now I see that it's from Rich.
Yeah.
So...
I use Markdown 2, and I have no idea what the status of it is,
but there is so much of our website that runs on Markdown.
Like, every episode page you visit, that's a Markdown page.
There's a bunch on Talk Python, a whole bunch of Talk Python training.
So anything that's better for Markdown, I'd definitely be wanting to give that a look.
Well, and I don't want everybody using the same thing,
because we do want some different tools to build on each other and stuff.
Indeed.
Yep.
All right.
Should I sketch out the next idea for you?
Yeah.
Let's take a look.
Oh, my God.
This is fantastic.
So this was sent over by Jake Furman, and Jake said,
have you seen Sketch?
So we've seen a lot of AI coding assistants.
We've talked about the potential bordering on violating license agreements
and stuff of GitHub Copilot.
Everyone has surely heard about ChatGPT.
I have friends who are not even a programmer.
They come, Michael, I'm working on some programming thing because of ChatGPT.
I'm like, all right, well, interesting.
We'll see where all that stuff goes.
But this thing called Sketch is like a simple view,
a simple way to ask natural language questions of your data that's contained in a panda's data frame.
Ooh.
Yeah.
Okay.
So the way it works is you go and it's not built into the editor.
It's part of monkey patches, I believe, pandas.
So it adds a function to pandas called, I think, ask.
Let's see.
What is the way we...
Yeah, it's .sketch.ask or .sketch.howto or sketch.apply.
So these are the different kinds of prompts that you can give it.
Okay.
So you just pip install this.
And once it's pip installed and you import it, that's all you got to do.
And it lets you ask data cataloging questions like, is there any personally identifiable information, PII, within here?
Like, is there a phone number or is there an email address or is there a social security number?
This could be some big, large data set and you don't want to save necessarily that or share it.
You can ask it and it'll ask you how to...
It'll basically tell you how to get rid of it as well.
Also, give me descriptions of this data set, okay?
Okay.
Which is nuts.
It also does data cleaning and masking to, for example, removing the PII.
And it'll extract some features and just lets you visualize stuff.
So the best way to do this is to watch this one and a half minute video here.
And so what you do, you know, what it shows you is says, here's a Pandas data frame read from a CSV.
And it's got like an order ID and a price and a date and an address.
And they just say data frame dot sketch dot, is there any personalizing information or any personal information?
And it says, yes, these fields.
And it says, how can I remove it?
And so it writes out the code.
You just copy and paste that.
Boom.
Now it's removed.
And then it says, give me a friendly...
See if I can pause this going quick.
It says, can you give me friendly names?
Stop moving.
Can you give me friendly names in a single sentence description of each column?
Format output as an HTML list.
And boom, it says index, the numerical index, the da-da-da-da-da-da.
Product, the product purchase in each order.
Order ID, a unique identifier for each order.
Purchase address.
The address of the customer who placed the order.
And so on and so on.
Nice.
Isn't that neat?
Yeah.
So when it spits out the answer, it puts a little copy tag so you can copy.
You can ask it, how do I do this?
You know, how to extract city, state, and zip from the address, which is a single string.
And it writes the code to like do the split statements to pull it apart.
I'm telling you this thing is, you know, it's not going to be perfect, but it's going to be
pretty good.
It's pretty interesting.
I just jumped in.
What do you think of this, Brian?
I think it's great.
Hey, Sketch, how do I remove the data that disproves my hypothesis?
Exactly.
I'm going to need the data that shows a downward trend here.
Can you extract just those rows?
So anyway, people can check this out.
I think it's pretty neat.
It says, in the future, we plan to update the prompt at this endpoint with our own custom
foundation model built to answer questions more accurately than GPT-3 can with its minimal
data context.
So because it doesn't know a lot about your data.
You can also directly call OpenAI directly, it says, and not use their endpoint by using
your own API key.
So there's that.
And anyway, it talks about how it works and what it sends over and stuff.
This is pretty powerful, though, because there's so much data for people using and different
things.
We kind of expect it now.
I was working on a project just recently where somebody, things like test result analysis
and a question of like, can we just put natural language strings in there to query it?
Why do we have to?
And so people are kind of expecting this all over the place now.
I just want to highlight just the last one there.
It says, go to the data frame, sketch on how to, using Plotly, plot a map of the total sales
in each state.
Boom.
You get a perfectly, perfect interactive map, a geographical map of the United States with,
that's pretty neat.
Yeah.
To be able to just jump in and do that.
Like, sure, you could go search and look that up.
But, you know, instead of just doing df.head, df.tail, and so on, like, how to?
I could think I'm ready to hang up a shingle for like a data science side job.
I'm pretty sure that you and I could pass as at least junior data scientists now.
Just with this.
Yeah.
Anyway, I think this is pretty cool.
I'm pretty psyched about it.
It's pretty cool.
Yeah.
People should check it out.
From what I can tell, it doesn't cost anything.
So just play with it, which is really nice.
And also, before we move on, I just want to tell people that this episode is brought to you
by us, right?
So there's a whole bunch of Python courses that if you get inspired over at Talk Python Training,
we've got the Python 3.11 guided tour, which is a couple hours of hands-on, showing why
that's awesome.
Python data visualization.
So if the sketch stuff was interesting, like, there's a bunch of that there.
And, oh, there's also something on pytest, isn't there?
Yeah.
Highly recommend that course.
It's great.
I do as well.
All right.
So if you want to support us, the best way you can do it is to support our other work as
well as share the podcast with your friends.
So thank you.
Thank you.
Thank you for that.
Yes.
I also want to thank the Patreon people because they're great.
We don't really talk about it much, but you can support us through Patreon as well.
And we very much appreciate it.
Absolutely.
All right.
What you got for our next one, Brian?
Well, I want to go in circles a little bit, actually going back to the year 2021.
No.
In 2021, we covered an article by Hinnick called Subclassing in Python Redux.
And it's a great article to talk about, like, just how to subclassing and dealing with classes
in Python and just to get your head around it.
So I did read it and I enjoyed it.
And then this year I came across the problem and I'm like, I think I can solve it with something
I read in this article.
So I'm going to scroll now to the thing specifically.
And it's called, so we have nominal subtyping with abstract based classes.
I tried that, but I decided to go with structural subtyping.
And with structural subtyping, basically, my understanding is it's like duck typing, but
with like, it's like strict duck typing.
And it says here, as you can see, if you've got something calling two things, two classes
talking to each other, one of them doesn't have to know about the other.
So in this example, we've got a reader and a foo reader.
And, you know, it's just a reader protocol.
We just need to know that there's a reader protocol here.
So I kind of like that.
I just knew this was here and I'm like, how do I apply this?
So I went ahead and wrote up an article called fixing circular imports with in Python with protocol.
And this is directly from this.
So the it's just a zoom in on one special case.
I really have like if I've got a class director and actor and they talk to each other.
So director tells an actor to do an action.
No big deal.
It has to import the actor.
Right.
So the actor gets a director.
So I'm going to it's gets data from the director.
And so it has to be able to call it.
Well, I could just pass it in self.
And since we have duck typing, it all works fine.
That we just call director get data.
So far, we have no types that the problem comes in when we have types.
If we add types to the whole mix, then when we add types to the director, it's no big deal.
We just add types to everything.
But to add types to the actor, suddenly the actor needs to know what type the director is.
And that's where we get the problem.
And that's exactly the kind of problem that I ran into is I wanted to add typing to this to a similar problem.
And it blows up.
It actually blows up really bad.
You can't you Python just says you can't do that.
It's circular important.
So the protocol solution is really slick.
And I'm going to scroll down to the answer.
The answer is just I just need a little bit of a snippet of code that says I've got this class that is derived from protocol.
But it's just got a get data function.
And that's all I know about it.
I know what it returns.
And that's all I know.
And I can say that my director really is that type.
And that's it.
The actor has to change to say what type it is.
And the rest of the code just works.
And I don't have to change the director.
I don't have to change the test code.
It's just this one file that I'm changing.
And I like this because I'm adding typing.
I'm not changing the code at all.
And the only thing is that, yeah, anyway, it's a slick way to use it.
I think so, too.
It's a little bit like interfaces, but less, you know, in the traditional Java or C#, I innumerable or whatever type of interfaces.
Yeah, the first.
And then Hinnick goes through abstract base classes also.
And that's one thing you could do is you could have a base class that has this get data function.
And both the director and the actor know about the base class.
But you don't really need that.
Nobody else needs to know that except for this one file.
So protocol is a nice hack.
Interesting.
Yeah.
Jonathan, the audience is asking, why not just use abstract base classes?
They both work.
So it's just a personal preference thing.
So, yeah.
And Kim in the audience is asking, as a question of style, do you prefer an ellipsis over the keyword pass to indicate an empty function?
Well, I'm warming to the ellipsis a lot because I write a lot of test code and I write a lot of example test code.
And if I say an empty function that is a test function and I use dot, dot, dot, it's clear that I'm not finishing it.
Whereas if I say pass, I've actually had some people get confused and think that the pass keyword is how you pass a test.
It's not.
It can be.
Well, you're not going to raise an error in that test method if you go past in there.
I mean, it's sort of a way to pass it.
Yeah.
I kind of like it too to say, look, I don't want anything here.
I don't intend to come back and fill it out.
Just, this is it.
And it kind of matches, the ellipses match how we do, like, type stubs as well.
I think I'll use ellipses, but I don't know if they have to.
I think they do as well.
Like, if you go and look at type shed and you look at the stubs in there, like, let me just grab one.
Rando.
Yeah.
All those have dot, dot, dots.
Dot, dots for their default value, dot, dot, dot for the implementation.
Kind of gnarly looking if you open them up, but there they are.
So, yeah, to me, I kind of feel like I'm matching the stub definition.
Like, I don't intend anything to be here.
Yeah.
Exactly.
Right.
All right.
Ready for the next one?
Yes.
This is a simple and I think will be useful to many, many folks.
This one comes to us via Rud Vanderham.
Rud Vanderham, thank you.
Sent it over and recommended, also created it.
So, we've all seen, like, code samples that, I don't know why, but I guess it's a decent way to explore it.
But it's always, I've always found it a little bit difficult to consume.
Basically, I want to show you how this bit of code works in the REPL.
Here's what happens.
Copy this and use it, right?
And when you see that here, you have the triple arrow and you write some code.
The triple, you know, right arrow, greater than, greater than, greater than.
Right?
You might do a for loop, which then has the indents and it does a triple dot dot dot to show you that, well, now you're still in the same command, you know.
And then eventually a closing dot and you run it.
And if you print out, you just say a variable or a response, then that gets printed without any of the dots or the greater than or anything.
It's just straight up as if you said print that, right?
So if I do, like, copy this and I want to explore it, I've got it.
Okay.
You know, remove all the triple dot dot dots, unindent it correctly, remove the arrows, common, you know, it's just like, yuck, right?
It's just a hassle.
So enter unREPL, the inverse of REPL, where you say, take this code and make it runnable for me.
Isn't that cool?
Yeah.
Super simple.
Like, even if you're in the REPL, you can't take what you're copying from someone else's example and put it in the REPL because then you'll have triple greater thans.
And so you just take this code and the way it works is you copy it to the clipboard, your code sample that you got from this REPL thing.
And then you just call unREPL as a CLI and it replaces the clipboard with contents of exactly what you would want.
And it even comments out the output.
So what wasn't code is now a code comment.
Oh, wow.
Nice.
That's perfect.
Yeah.
Really perfect, right?
Yeah.
So I can certainly see that this is a handy thing that people may want to install.
So when you see the instructions here, it says you can just use unREPL.py.
And the way that you run it, it says up here somewhere.
Okay.
What you do.
So what you do is you copy it and then you run.
Okay.
He's updated.
It just says unREPL.
It used to say run unREPL.py, but it's also on PyPI, which means that you can pip install it.
But I would not pip install it.
This is not the thing that goes into a virtual environment to be part of a program.
This is a CLI tool that you just want to have available.
So I would replace pip with pipx.
And then you just globally have this command on your computer.
Just happens to be implemented with Python.
But you copy it, you know, just type unREPL.
And then, you know, whatever was in the clipboard is now better.
It doesn't say use pipx, but I think that's the way.
That's certainly how I would do it if I'm going to install and use this thing.
There's a way to set up an auto hotkey on Windows only.
I have no idea about this.
So y'all are on your own there.
Okay.
So how do you use it on the command line?
You just type the word unREPL once you've installed it in the path, which pipx, of course, does.
And then you paste your code or use it to.
Yeah.
So you copy your code, type unREPL on the command line.
It looks at your clipboard.
Oh, okay.
UnREPLs it and then it replaces the clipboard, like basically with a new copy.
Oh, that's neat.
Yeah.
That's great.
Yeah.
So really easy.
It also says you can import it into a program and use it for some reason.
You feel like that's the thing you want.
And it has special handling of the underscore, which has special meaning in the REPL.
Like it knows about underscore and treats it accordingly.
Well, like for instance, like somebody could like maybe change Bpython or something like
that to allow you to paste right in.
So.
Yep.
Neat.
Yeah.
People out there think it's nice.
This is a nifty little tool, says Kim.
See, it uses TK Enter presumably for clipboard access.
I believe so as well.
It should use Paperclip, I believe.
Paperclip rather.
Piperclip.
If I were saying it, that's like a super small, simple little thing, cross-platform copy and paste.
But, you know, it doesn't matter really what it uses to accomplish, copy and paste.
But TK Enter is like built in now, or is just normally.
Yeah, that's true.
So I guess there's no dependency.
That's right.
There's no dependency.
So, okay, maybe I'll take that back.
Maybe that is better.
Anyway, I love Paperclip.
But this is cool.
If you're, I don't know, if you're working on a book or if you're learning a lot where you're
grabbing stuff off of tutorials, I kind of sort of spoke tiny, a tiny bit disparagingly of this
because the style of showing the REPL output, because I'd rather show an example that has
executable code and a print statement.
You know, I kind of already write this output like, here's the code, you know, comment, here's
the output, right?
So I don't know.
It just, it seems a little more reasonable to me, but whatever.
People can write them in any style they want and this un-REPL will roll it from one to the other.
Yeah, you can also use a doc test to check your REPL docs.
Yes, exactly.
And while out there, it's giving a challenge to the community.
I think someone should write a PyCharm and VS Code plugins using this so it becomes part
of the editors as well, which would be pretty neat.
Paste as, paste from, right?
Paste from REPL.
Yeah, that'd be cool.
Yeah, but it's already really handy to have it just, you know, just pop over the terminal
and run this.
Yeah.
All right.
Awesome.
I guess that's it for all of our items, isn't it?
I think it is.
Yeah.
We have, we've come to the end of all of the main things.
What do you got for our extras today?
Well, I was going to mention that I wrote an article, but it was, it was my item.
So.
Correct.
So that's it.
There you go.
How about you?
There you go.
I got a couple of quick ones here.
So I think this happened since our last show, but pretty much right after it shipped.
If you have Git, you should update it.
So there's remote code execution level vulnerabilities in the Git client that is sitting on your machine.
If you have less than 2.39, maybe even less than 2.39.1.
I can't remember the exact version, but until last week, there was a problem.
I'll link to this over here, but you don't really want that.
Yeah.
It affects 2.39 and older.
Yeah.
No.
So careful.
I mean, if you're only pulling stuff from your own repo and only you can put stuff in your repo, you're safe.
But if you're checking out someone else's repo or PRs or things, you might not like it.
Okay.
Another one here.
Over on Aquathon training, we, as much as I don't like to, had to add some kind of validation.
It used to be that dreaded reCAPTCHA thing from Google.
And it would say, find all of the stoplights.
And you're like, well, is it the pole of the stoplight?
Also the stoplight.
I mean, technically it is, but it's not the light part.
And it'll be like, try again.
You're like, no.
Right.
But because it's the internet and we can't have nice things, unfortunately.
I had to put something there to slow people down because they were just pounding away on it in various ways that were not ideal.
We had stuff to mitigate it.
But once you start encountering botnets, then you kind of get yourself into a bad place.
So I had to put the reCAPTCHA stuff there to keep people from messing with it.
And I hate reCAPTCHA, but I did it because I had to.
So then when Cloudflare came out with Turnstile, I'm like, yes, this is so much better.
It doesn't ask you about chimneys or fire trucks.
It just does a little like run some code on your browser to prove that you're not just requesting and posting this from some bot.
But it's a real browser.
Off it goes.
Right.
So that's great.
Eli Cobbler said, hey, this is awesome that you got this working.
I can't get it working how to use it.
So I ended up posting a gist, which has an implementation of this using like a faux web framework.
It didn't really, I didn't want to tie it to Flask.
It just says like, here's your thing that handles the forum posts.
Do this.
It shows the HTML bit, which also Cloudflare would show, but they don't have the Python version.
And then it has the, identic based API validation stuff here, which is, this is the gnarly bit.
So he was like, I can't get it working.
Could you post this?
Boom.
And apparently he got it working as well, but I'm sure there are other people who equally hate find all the haystacks, find all the fire trucks and would rather use Turnstile.
But it's kind of tricky to use.
So they can also use that gist if they so wish.
Yeah.
That's funny.
Yeah.
David Poole says Google's, Google's captchas made me doubt my own humanity.
So I always thought like, that'd be fun to do like trees instead or like plant identification.
Like, yeah.
Like find the azaleas.
Find the azaleas.
What?
Let's see.
So on, when I talked about getting a bunch of stickers on my computer the other day, remember that?
Yeah.
Well, one of the stickers I put is, I'm not a robot.
And it's like a validated recapture.
So I just thought that was fun too.
But that's cool.
Yeah.
Anyway, so if people are interested and they want to try on Turnstile, which is way more privacy oriented, also nicer because you don't have to interact with it.
It's just a little hard to work with.
Here's some code that they can try that at least one other person has been successful using.
So that's a tepid but somewhat good testimonial.
How's that?
That's nice.
All right.
Thank goodness it wasn't in the JavaScript, Brian.
Yeah.
Because if it's in the JavaScript, I don't want to have to go debug that.
Do you want to debug it?
No, I don't want to debug it in the JavaScript.
It might make you so crazy as if to go and like literally write a song about how much you don't want to go into the JavaScript.
Yeah.
Now, I am very unskilled at music, but Dylan Betty is actually quite skilled and has a whole bunch of really funny songs here like The Big Rewrite based on American Pie.
An eight and a half minute long song, I'll point out.
And he wrote a programming song against it.
Awesome.
Your API is a hall of shame.
You give rest a bad name based on the Bon Jovi song, but neither of those are the one that I'm referring to.
No, I'm referring to the song called Bug in the JavaScript based on Piano Man by Billy Joel.
This is good stuff, right?
Yeah.
Now, it's six and a half minutes, so there's no way I can play it.
But it starts out like you've checked all of your database indexes.
You've tuned all your API hooks.
You're starting to think that you might need a drink because there's only one place left to look.
There must be a bug in the JavaScript because everything else is built properly, but the front end's a pile of crap.
It's so fun.
I like it.
I'm starting to think that I might need a drink because there's only one place left to look.
It's really good.
I recommend people put this on.
And if you're working with Angular or Vue or some front end framework, you can play it loud in the office just to share a little bit of the feeling.
Yeah.
Anyway, that's what I got for you this week.
That's good.
People can go watch the YouTube video.
Thanks.
Yeah, you bet.
And thanks for being here.
Thanks, everyone, for listening, as always.
Thank you.
Bye.