Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.
This is episode 164, recorded January 9, 2020.
And I'm Brian Okken.
And I'm Michael Kennedy.
And this episode is brought to you by Datadog.
We'll talk more about them later.
But speaking of data, data-driven journalism.
Yeah, let's start there.
So there's this project called CJ Workbench, or just Workbench, something like this.
And this recommendation comes to us from Michael Palowski.
Sorry if I kind of mispronounced your name there, but thank you, Michael.
And it's data-driven journalism as a service, as a platform, along with training on how to use it.
Not training on journalism, but training on basically how to do data journalism.
So it's a pretty cool idea.
That is neat.
Yeah.
And so what you do is you can go and say, point it at, I don't know, some website.
And say, I want you to scrape this from the website.
And then you go and you fill in, like, this UI.
You fill in and say, okay, now I want to filter out the empty columns.
And now I want to do this and that transform and other transforms and so on.
And so it basically lets you do kind of what you might do with a Jupyter notebook.
But there's no code for the journalist, which is pretty cool.
And so you might wonder, well, if there's no code, why are we talking about it?
There's all sorts of software that does interesting stuff that we don't talk about.
Well, one of the things you can do is if you find that the built-in features are not enough,
then you write a few lines of Python and you start extending it for the journalist using Python,
which is a pretty cool way.
So instead of you worrying about the UI and all the basic stuff like loading CSVs
and working with data frames and all that, you just say, use this.
And if we got to write a little Python code, we will.
You can use their system and just add your own little code snippets to it?
Yeah, exactly.
So if you look at the features, it has modules to scrape, clean, analyze, and visualize data.
It's got built-in training, like it'll teach you how to use the feature that you've got built in.
It connects to things like social and Google Drive and APIs, like JSON endpoints, I think.
It has recorded workflows, so it's really repeatable.
Versioning.
But finally, you can write custom modules in Python and then add them to the little drop-down module library that you can use.
Oh, that's neat.
Yeah.
Plus, this is all in GitHub, so that if you really want to hack the back end, you can also.
Yeah, exactly.
It's open source.
It's on GitHub.
You can go grab it, download it, and run with it.
But they also have it online as a service.
So you can decide, do you want to run your own server?
Do you want to use their public server that's in beta?
Or whatnot.
So I covered this partly because I think it's really cool, and I also covered it because, boy, I think the world needs journalism right now.
Yes.
And, yeah, more people saying, I don't know if I believe your analysis.
I want to do my own analysis on it.
Yeah, and here you don't have to commit to a huge amount of work.
It's cool you pointed all these live sources.
Like, you can point it at some website, and it can, like, real-time refresh the live data that it's scraping for your article so it can evolve over time and whatnot.
It's pretty cool.
I might have to try this.
Yeah.
Yeah, nice.
Have we covered GUIs yet on this podcast?
I think I do remember way back when at some point we talked about GUIs, right?
Yeah, at least once.
Yeah.
So...
What do you got for us this time?
Well, so here's another one, another GUI.
This one's called Remy.
The Remy comes from Remote Interface Library.
I'm not sure why it's called that.
All GUIs are essentially...
Why is it not Remy?
Remy, I don't know.
No, no, it's cool.
But Remy, a platform-independent Python GUI library for your applications.
And yes, we've covered things like this a lot of, like, okay, I want to just scrap the idea of a native user interface, and we'll just use web user interfaces.
The thing that I liked about this, it's so small.
There's no dependencies.
I'm not sure why they're pointing you to do a pip install from their GitHub repo, but I'm guessing they're not really that great about...
updating the PyPI version, because it's there also.
But anyway, I did this morning.
Grabbed a...
Just did a pip install of Remy, but from their Git repo.
And then I tried their little sample basic app, and I'm like, is it really this easy?
And yeah, this is like 30 lines of code or less to just write.
And if you run it, just Python bloop.
Oh, first of all, when you pip install it, it installs the package, and then that's it.
It doesn't install anything else.
Because it's only dependent on the standard library, which is cool.
And then you start it up, and it runs a little server on your thing, and it's really easy to throw buttons together and stuff.
It looks pretty good enough, and the code is simple enough that I thought I'd include it, because there's a lot of cases where, like in an internal team, where you have...
If you want to throw something together where people have access to some Python data or some Python analysis with a user interface, and something like this would be cool.
And it's neat.
Yeah, it's pretty neat.
It's like a Python-only electron.js-ish type of thing without the HTML part.
So you define your app, and you programmatically define, like, I want a button here, and I want a label there.
And when I click the button, I want it to call this other Python function.
So you just write pure Python, and then it turns the GUI definition into HTML, shows that in a server, and then calls back to your code, something like that, which is pretty cool, right?
Yeah, and it feels similar to working with TK back in the day.
Within the Python code, you create a label object and a button object, and then you put them in a container.
There's no packing so far that I can see, but it has different types of containers that you append them, and then just throw those in a box.
So, yeah, like you said, you don't have to write any of the HTML.
It's just all Python code.
So, yeah, for what it's worth, it's kind of simple.
Yeah, it's a nice, simple little thing, and it looks pretty cool, and yeah, I kind of like it.
One of the things I was thinking about doing was, sorry, was we have a switch box that's controlled through.
We've got a Python thing that controls the switches, and sometimes it'd be nice to be able to just see what the current setup is and just have a web page that just shows that.
It'd be really easy to write in Python, kind of hard to do.
I don't want to deal with a whole user interface sort of thing, but having that, I think I could probably throw that together with this tool in like half an hour.
Yeah, and it's a proper local application, so it can look around on the system and do what it needs to do.
It needs to be really a web server.
Yeah.
Although you can expose, I saw there's a way to tell it to listen on the network, broader interface, not just local host.
So you can't expose this thing to sort of the world, for what that's worth.
I don't know if it's a good idea or not, but you could.
Like I said, we would do it within our internal network, so there's no malicious code flying around internally.
Yeah, yeah.
Yeah, cool.
Anyway.
You know, I'm starting to get excited again about Qt after Ogie Moore's presentation at the last PDX Python West meetup.
Like that looked pretty compelling, actually.
Yeah.
What a slick program that was.
And zippy.
Very zippy.
Nice.
Speaking of slick, Datadog.
Yeah.
Python Bytes today is sponsored by Datadog, a cloud-scale monitoring analytics platform that unifies metrics, logs, and distributed traces from your Python application.
The Datadog tracing client auto instruments popular frameworks and libraries such as Django, Flask, Postgres, and AsyncIO even, so you can quickly get deep visibility into your application.
Trace requests across services boundaries with flame graphs.
I like flame graphs.
Correlate traces with logs and metrics and plot your application architecture with the service map.
Sign up for free for a free trial at pythonbytes.fm/Datadog and you'll receive a free t-shirt.
Nice.
Yeah, they're doing good stuff.
Thanks, Datadog.
This next one, Brian, you wanted it, but I got it.
Yeah, you got it first.
It's cool.
I'm excited about it.
Yeah, so there's a couple of cool libraries for building CLIs.
Click is probably the most prevalent one.
We've got argpars.
But people who do FastAPI, the cool new API framework, these folks have now gone, let me just double check that it's actually the same organization.
Yep.
Yes, good.
Yeah, same work.
Perfect.
So they've gone and created a CLI definition library like Click.
And I believe it's actually based on Click.
Yeah, it's based on Click.
So you might wonder, well, what does it do?
The name gives you a little bit of a hint.
It's based on Python type hints.
So Typer looks at the various hints.
It can, you can say like, I have a function.
It takes a name, which is, you know, name colon stir.
It takes a string.
And then you just use it.
And when you go run it, it'll say, hey, this thing that we ran, in order to call it, you have to specify a name and the name has to be a string and so on.
So it just looks at the arguments and the types of the methods you're trying to run, in the simple case at least, and it uses that.
That's pretty cool, huh?
It's super cool.
Yeah.
And then you can grow in more complexity.
Like you can have sub commands.
You can have help on all the commands.
You can actually get autocomplete.
You can install autocomplete in your shell, all kinds of cool stuff.
So you can create a little app.
You say, here's a command and here's a command.
And then, of course, the commands are functions.
Those functions take arguments.
The arguments have types.
They can have default values, all that kind of stuff.
So you can try to run it.
It'll give you help pretty much automatically, which, yeah, this really speaks to me the way this works.
I could definitely see myself using it.
Yeah.
So a really good, clean command line interface that has, if you're writing a tool with a command line interface,
you're going to want help associated with it.
You're going to want description of the different methods and what they do.
But also, with as little boilerplate as possible, you don't want to clutter up your code.
And as far as I'm concerned, right now, typer wins.
There's hardly any extra overhead that you have to add to make this work other than type hits.
And if you're not using a main or something, you have to put a little decorator on the different entry points in your program.
But really clean.
I'm totally going to start using this.
It's really nice.
And because it can leverage so much of the natural program itself, right?
The argument names, the fact that it has arguments, the argument types.
That's really helpful for it, yeah.
So you can also integrate things like Colorama and click, which is based on, will automatically start using color for its output.
If you just have Colorama around, you can also use click completion.
And typer, this is the part I was talking about, will automatically configure it to provide completion for all the shells.
That's really cool, too.
So the one thing I was concerned about is it's based on click.
And one of the things I love about click is that it has a good testing interface.
You can write tests against your command line app as if you're writing from the command line,
but you don't actually have to spawn a new subpros to test things.
And yes, typer has the same sort of model where you can write your tests against the interface and not have to do a subprocess.
Oh, that's really cool.
Yeah, nice.
Because, yeah, you don't have to start it basically shell out and see what happens, right?
Or like issue the command line directly to the shell to see what happens.
Yeah, really good job.
Yeah, the other thing I like about this is you could say, well, we don't want the API of click.
So we're going to start from zero and rebuild our version of what this whole world looks like.
But what they did is said that we're going to build a new interface for writing against this, but on top of click.
So things like Colorama and the completion just plug in seamlessly.
I think that's a cool lesson.
Yeah, definitely.
Also, it's a smaller project for them to have to maintain, too.
Yeah, exactly.
Yeah.
Exactly.
Cool.
All right.
This next one that you got here is from Chris Moffitt, right?
Yes.
And this is actually...
So I'm hoping this is completely still accurate.
I imagine it is.
The interface for Matplotlib hasn't changed too awfully much.
I know they've had some cool releases, but I want to talk about an article called Effectively Using Matplotlib.
And this is a 2017 article.
But in the middle of the introduction, he says, at first, I think I was a little premature in dismissing Matplotlib.
To be honest, I did not quite understand it and how to use it effectively in my workflow.
And I think that matches my understanding of Matplotlib as well.
I mean, when I first needed to put charts or graphs or something, of course, you try Matplotlib.
And then you're...
You kind of hit in the face with the complexity of the interface.
And you know you can do the simple thing simple, but how to do something a little bit more less than simple or more than, you know, more complex is a little daunting.
And then I went out and looked for other alternatives.
But now I think the alternatives have their issues too.
So I'm ready to go back and take a serious look at Matplotlib.
This article is a really good look at kind of understanding it and at least Chris's workflow, how to use it.
And one of the things I like is the description that one of the reasons why Matplotlib's interface might be a little confusing is because it's coming from...
It's really got two interfaces.
It does a Matlab-like state-based interface.
And I'm not a Matlab user, but I know that a lot of people have come from Matlab to Matplotlib.
And so that sort of interface is important to have that transition be easy for a lot of people.
But then there's also an object-based interface that makes a lot more sense.
And that's Chris's recommendation is to use the object-based interface instead.
And so now after he's gone through it a few times and he has some recommendations.
And his recommendations are to first learn the basic Matplotlib terminology, specifically what is figure and axes and around those.
And what's difficult for me is that those are just normal words that I think I already understand them.
But I need to make sure I understand what Matplotlib thinks those things are.
Yeah, sometimes it's just hard to work with Matplotlib unless you find an example, right?
A lot of times what I...
My goal is to hunt the internet for something that looks like I want.
Figure out if I can find the Python code that created it.
And then I'm good.
Yeah.
He does have an example in this, which is kind of walks through at least one example.
But if you really want to kind of get your head around it to start with using pandas with Matplotlib.
So yeah, you start your visualization with basic pandas plotting and then add, if you want it more complex, try Seaborn.
And then if you want to customize it a little bit more, he said use Matplotlib native stuff to customize the visualization.
And I wouldn't have thought to use that workflow.
So that's kind of nice.
It also includes this really kind of cool handy reference that there's a little graphic that he created that has like all the different...
What all the axes are and if you want to customize a certain part of a graph, what that name is within Matplotlib so you know how to look it up.
And that's often...
The secret of programming anymore is to know what specific word to Google.
So I think that's a good thing.
Nice.
And also I want to remind folks, long-time listeners of PyLustrator.
I'm not really sure about the pronunciation like Illustrator, but Py for the first two words.
I'll put a link in the show notes.
And what it is, is it lets you take a basic Matplotlib plot.
It launches a GUI where you customize the look and feel and then it'll output the code that would have done that for you.
Oh, cool.
Yeah.
We talked about that in episode 137, which is like a year ago or something.
But anyway, still good stuff.
So I'll put a link in there for that.
I'm also including a link in a almost related section.
Maybe there's an article about this, but I haven't found it.
There's a Stack Overflow answer that has a little...
The code you need to stick a map...
Generate in a Matplotlib graph and put it in a Flask file on the fly without having to save it to a file.
Oh, yeah.
Like a live dashboard type thing.
That's cool.
Yeah.
Nice.
Speaking about doing stuff on the web, if you're using Django, there's a cool little library that'll let you do stuff in the background.
Maybe you want to do some query and update some kind of report or you want to send out a bunch of emails or recompute some indexes or recompute some search results or something like that.
And it's going to take a little while and you don't want to just have it block on your main request.
There's a bunch of different things you could do.
You could set up a separate server with like Celery or RabbitMQ or something like that.
But that's a whole other set of servers and it just gets way more complicated.
Sometimes you want to just run it in the background just in the same process, right?
It's easy to share data because you just pass the pointers.
So there's this thing called Django Simple Task.
And it allows you to run background tasks in Django 3 without requiring any services or workers.
Oh, cool.
Yeah, basically the reason it has to be Django 3 is it's based on AsyncIO.
And Django 3 is the first version that uses ASGI.
And so you actually need to run it in ASGI server like UV, Acorn or something like that.
So be really aware that you have to run it in its async way for this to work.
But basically when it starts, when the web app starts, there's a queue created and some workers to listen to the queue.
And then you just go to any function you want to run and you just call defer on it and that'll kick it off in the background.
And you could do that now.
Like you could create a thread and just launch it as a background thread.
But one, you don't get like a thread pooling.
But the other is when it's time for the worker process to shut down, like you're doing a new deploy or something, it'll actually wait until its background work gets done before it allows you to restart it, at least in a friendly way.
So that's cool.
And you can either give it a asyncio coroutine or just a function and it delegates it, either runs it or delegates it to a thread pool if it's not async, things like that.
So it's pretty cool and it's really easy to use, like ridiculously easy.
Oh, that's neat.
Yeah.
So there's a simple little example I put in the show notes where it just imports the deferred operation method.
It has two tasks, one of them which uses time.sleep that requires threading.
And one uses asyncio because it's an async method.
So it would run on the async event loop.
And in the view, you can just say defer one, defer method one, defer method two, and then do your regular work.
And that kicks it off but doesn't wait for the response.
But does it in a slightly more durable, better way than just like launching threads or kicking it off on its own little background thing to go run unmonitored.
So in this example that nobody can see, but they will if they look at the show notes, will the HTTP response happen and the view return before the defer tasks get run?
They might get started, but because in both of them they sleep for a second to kind of, that's their work, right?
They're sleeping for a second.
So certainly the response would be sent back to the user before those are done.
And this is a cool interface because the two tasks, one of them is async and one is not.
And the defer call doesn't, you can't tell.
Right.
You don't have to treat it differently.
But if you can do async, great.
It'll do that on the async event loop, I think.
Otherwise, it just puts it into a thread pool, which is pretty cool that you don't have to worry.
You can just kind of go, here's some background stuff.
If it's IO driven, we'll do it this way.
Otherwise, just do it that way, which is, I like it.
It's nice.
Yeah, really nice.
Cool.
All right.
What's this last one you got for us?
Well, so something happened recently.
I've got a handful of, I don't open source, work on open source projects really as much
as I'd like to.
I'd like to work, do some more work on open source projects.
But I have a few that I maintain myself.
But project is in air quotes I'm doing because they're really pretty small projects.
And a couple of them, cards and submark are things that I just created to help with teaching
people how testing works.
They're not really intended to be useful projects.
And then another one was a plugin for pytest.
Actually, I do use it and it's useful for me.
But I started getting, like in the last handful of months, I started getting pull requests and
issues and stuff.
And I didn't know anybody was using it at all.
So this was surprising to me.
But I wanted to highlight, I don't know if, I can't believe we haven't covered this or maybe
we already have.
There's a website called PyPI stats.org.
Yeah, that's news to me.
I hadn't heard of it.
It's just a simple interface.
Now, the backend might be complicated.
But in the interface, it just has like a little box and you plug in a Python package and it
tells you what the download stats, actually quite a bit of history too, of different packages.
And so I use that for all of my different projects.
And sure enough, cards was downloaded like, I don't know, less than 400 times in the last
month, submark less than 70 times in the last month.
And then pytest-check, which is a little app or a plugin that lets you have multiple failures
per test, but keep going during the test.
That was downloaded just a little less than 20,000 times in the last month.
So clearly that one's more.
And that's the reason why I'm having more people try and tell me what's wrong with it.
So I guess I have to start maintaining.
Yeah, this is a super cool interface.
You get all kinds of graphs and ability to slice it by dates and whatnot.
And yeah, I like it.
And there's also a top 20.
So you can look at like really what are the top 20 packages on PyPI.
And that's why I say that mine are small peanuts because at the most, it's like 19,000 a month.
But the top 20 all get over 1.3 million downloads per day.
Wow.
Yeah.
And you can look at them, look at the trends.
That's pretty interesting.
There's probably some cool stuff to go through here.
And I guess if you were trying to do predictive stuff or analytics, there's probably a lot of
things to be done here.
Yeah, definitely.
Also, just occasionally, I'm just the top 20.
I like because I was occasionally I'm curious about that of like, what are the top packages
on PyPI?
What do people use it?
Yeah.
They weren't what I expected.
I mean, some of the ones I obviously expected were, but I think right now URLlib was the top
top.
And I didn't expect that.
So yeah.
Yeah.
URLlib 3 specifically.
But yeah.
It must be used by a whole bunch of other stuff.
Like 6.
I don't think people are specifically downloading 6.
It's the 2 or 3.
Like I think Request uses both of those and Certify and that like hits the top 4.
So Python.
DateUtil is a good one up there.
Boto Core.
Pretty interesting.
I use Boto 3, which depends on Boto 4, but it kind of makes me frustrated because Boto 3 won't
use the latest version of Boto Core, but it's all made by the same people.
Anyway, there's a bunch of interesting stuff up here.
Yeah.
I think I'm going to have to look at this some more.
Pip is in the top 20.
Nice.
Got to get started somewhere.
Yeah.
All right.
Well, that's it for all of our main items.
What you got for the extras?
Well, I wanted to actually just ask for some help.
The last episode we put out was a live recording of the January Python PDX West meetup.
Actually, not the entire meetup, but just a-
Flash in case people haven't been paying attention.
They're near Portland.
There's a second Python meetup, which runs out in Hillsborough.
Much better for Westside folks.
So yeah, you put that together.
Good job.
Yeah.
And one of the things that we wanted to do with this one was to, I got this from a lot
of people.
And in the tech community, well, the Westside has a lot of high-tech stuff.
There's a lot of people coming from other languages to Python.
And so they might not be beginner programmers, but they're beginners at Python.
We've had this before, people saying some beginner-friendly content per at least one topic per meetup so
that people aren't lost with everything.
And yeah, we kind of forgot this last time, but I enjoyed the topics.
I'd like to hear from other people as to what sort of topics would be good at a meetup for
Python beginner-friendly.
And just let us know.
I think that'd be cool.
Yeah.
Well, it's easy to get super into focusing on what's the most latest, coolest, hottest
thing.
But there's a lot of folks who really just appreciate setting the ground, a good foundation.
And so, yeah, I think that's a good topic.
Yeah.
And one of the things I'd like to do is possibly get a list of those sorts of things so that
we can try to ask people to put together something.
Like one of the meetups I did, like just describing how pip and virtual environment work and how
to use those.
And there's a lot of basic stuff that people don't even think about maybe that they should
talk about.
And it'd be good to hear more.
I also want to try to get other people to do it because you and I could probably do
those sorts of beginner-friendly ones.
Sure.
It'd be fun to have other speakers too.
Yeah, absolutely.
It would be very fun.
If people are willing to get to Portland on a Tuesday evening, then they can come give a
presentation.
Yeah.
One more thing.
I want to do a shout out to PyCascades 2020.
I can't remember when.
It's in February coming up.
February, I believe.
I believe it's like 8th and 9th.
Let me double check.
It is February 8th and 9th in Portland, Eastside.
Wonderful.
So people should get tickets so they could come.
And if you haven't bought your ticket yet, check the show notes because we've got a discount
code for 10% off.
Yeah, absolutely.
Yeah, it's right there at the bottom.
I don't really have anything this week for extras to share.
But one thing that did come out and caught my attention yesterday that I think I might as
well just throw out that's kind of cool is Firefox 72 is out and it has come with anti-fingerprinting,
which is pretty exciting.
So I don't know how many people are aware.
I don't know if you're aware, Brian.
That even with ad blockers and blocking like tracking cookies and stuff, there's all these
weird things that you can do to figure out.
Actually, it's that same person, right?
Like you can ask, what are all the installed fonts and what version of OS do you have?
And here's a canvas thing.
Let's run some stuff on there and like try to detect weird CPU cycles and other kinds of
there's all these like weird things you can do in browsers and pretty much uniquely identify
everyone running a web browser.
Oh, that's lame.
Even without cookies.
Yeah.
So Firefox is trying to fight back against that.
So I think that's pretty awesome and worth checking out.
I'll actually link to ours Technica article about it.
Okay, cool.
Yep.
Well, I guess that leaves us with a joke, doesn't it?
Yeah.
Last time we did a joke about describing different software programming jobs.
This one, you're going to have to imagine that people who are fans of various programming
languages are in like a literature course and they have to write an essay.
Okay?
Okay.
Have you pulled up the picture?
Yes.
I'm already laughing.
All right.
Cool.
So basically there's professor looking person with glasses trying to read like a paper, a
physical paper and a student looking at it, getting the feedback from the professor.
This is important because at the end it gets interesting with the pictures.
So for Python, it says, this is plagiarism.
You can't just import essay.
For Java, it says, I'm two pages in and I have no idea what you're saying.
Assembling.
Pretty appropriate.
Yeah.
Assembling.
Do you really have to redefine every word in the English language?
With C, it says, this is great, but you forgot to add a null terminator.
Now I'm just reading garbage.
I don't really know if I understand the C++ one, but it says, I asked for one copy, not
400 and the professor's buried in books.
Oh, I don't get that either, but.
Yeah.
Okay.
Unix Shell.
This one's pretty good.
I don't have permission to read this.
LaTeX, which you often use to describe scientific papers and super elaborate mathematical formulas,
says, your paper makes no sense to me, but it's the most beautiful thing I've ever laid
eyes on.
And the final one is the HTML programmer.
And instead of holding a paper, the professor's holding a pot and they just say, this is a flower
pot.
What is it?
I don't get it, but it's funny.
Yeah.
I don't either.
All right.
So anyway, I thought that was pretty fun.
You all can check it out.
The comic is linked in there, but it's pretty fun.
You can't just import essay.
This is plagiarism.
All right.
Well, thanks a lot, Michael.
Yeah, you bet.
Thanks, Brian.
Good to be here.
Thank you for listening to Python Bytes.
Follow the show on Twitter at Python Bytes.
That's Python Bytes as in B-Y-T-E-S.
And get the full show notes at Pythonbytes.fm.
If you have a news item you want featured, just visit Pythonbytes.fm and send it our way,
where I was on the lookout for sharing something cool.
This is Brian Okken, and on behalf of myself and Michael Kennedy, thank you for listening
and sharing this podcast with your friends and colleagues.