Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to
your earbuds. This is episode 124, recorded April 3rd, 2019. I'm Michael Kennedy. And I'm Brian
Akin. And this episode is brought to you by DigitalOcean. Check them out at pythonbytes.fm
slash DigitalOcean. Get $100 credit for new users. Tell you more about why that's awesome later.
All right now, Brian, I feel like we're going to go to your happy place.
Yes, I'm definitely one of the testing nerds that's around. And I'm excited about the pytest 4.4.0
released. There's a lot of stuff in the 4.4 release. And that's pretty cool. There's a lot of actually
really neat things. One of the things, I'm going to save the big one for last. But the one of the
cool things is there's a test paths variable that you can put in your configuration, any file,
it tells the pytest where to look for tests. And it kind of short circuits some of the
searching everything under the current directory.
Just what, like two subdirectories or something, but not everything?
Yeah, like for instance, this is very common if you've got like a structure where you get tests
in one directory, your source in another directory, and then maybe some documents or something.
And it's a large project. If you just launch pytest from the top, it'll look at everything.
And it really should just look in the test directory.
Right, exactly. It makes sense.
So that's pretty cool. It's been around for a while, but it's kind of hidden. Like it just
worked that way. And what they've done now is they've added in the output, it tells you which
tests paths have been set. So that's nice.
Yeah, that's cool.
And then there's a whole bunch of stuff that's relevant to especially to plugin writers.
And a little bit for test writers. There's actually quite a few things. I encourage people
if they're nerdy about pytest to check out the changed list. The thing that I'm super excited
about is there were a couple of internal changes that made it so that there's a new plugin called
the pytest Subtests that works.
Oh, nice.
So pytest has always been able to run unit test code. But one of the things that happened is if
unit tests had subtests in them, and subtests are a way to have kind of multiple checks within a test,
if you had those in there and one of them failed, it's supposed to keep going and check all of them.
But pytest would stop at the first failure. Now it doesn't. Now it continues on with running all of them.
That's cool. Like, so if I've got, say, a list of five things, and I want to loop over those and
test each one of them without some kind of parameterized test or something, I just want to loop over them
all and write one test and do that. Like, I would do that with a subtest.
Yeah, or at least you can.
Yeah.
And it works pretty good. So it really was to the plugin and the changes were really to fix this
little corner case of the unit test support. And so I'm really glad that they did that.
But they also threw in a new fixture called subtests. So even if you're not using unit test,
you can use this new fixture. And it has a subtest.test context manager that works just like
the unit test version, but you can use it in pytest tests.
Yeah, that's cool.
That's neat.
Another one that I think is nice is the async. So you have the async, you know, mark as async,
decorator you can use. But previously, if you did that, but for some reason, like the right
plugin to run the async, say like paytest-async.io or paytest.trio wasn't there, they would just not
run, but they were still marked as passed. And now at least you get a warning and they're not run or
something like this. That's cool.
Yeah, that's a oops.
That's one way to make your test pass. Like, let's just not install that async runner.
We'll be cool.
Yeah. Yeah.
Well, keeping with the async theme, I want to roll right into requests. So I literally just
hung up recording or stopped recording with Kenneth Wrights. We're doing an episode on Talk Python and
we're talking to him about requests a little bit, but mostly other things. Anyway, this next item
follows right onto that is request-async. So currently without this library or something like it,
if I want to use async and await and I'm doing web requests like microservices or calling like a
Stripe API or something, I either do it synchronously with requests or I can do it asynchronously, but I
have to switch out my calling package. I have to use like aiohttp.client session or something totally
different. Rewrite my code. There's a package called request-async, which is pretty cool. And it's
actually a different thing you import, but if you import it as requests, then you can do await
request.get, await request.post and all of that. So it seamlessly fits into your async and await methods.
Maybe you're testing with that pytest thing I was talking about. And it's basically the same API,
but now it's async friendly, which is cool.
That's very neat. Yeah.
And like I said, there's other stuff you could use. You could use aiohttp, which is nice,
but it means you rewrite your stuff, right? And a lot of people depend on like changing DNS resolvers
and all sorts of crazy stuff deep down inside a request. So this will let you do that.
Theoretically, at least I haven't tried all the edge cases, but let you do that without rewriting
your code, which I think is pretty cool. Now I did say, I was just talking to Kenneth and
one of the things he said is they are working on native async and await support for requests.
So that's pretty cool, which would make this unnecessary. So why do I bring it up? Well,
he said it's probably like not done for a while. It's probably like a year away because they're
doing like major rework inside of requests. And, you know, think how many libraries and people
depend upon requests. So, and how hard that must make it to change at all. Right.
Yeah. It needs to not break back backwards compatibility and it's not even just breaking it. It can't
slow things down either. Yeah, exactly. So they're talking about making a new package called request
three. So you import request three and it has a slightly different API, just barely. So anyway,
here's an interim way to get async and await working with request. And another thing, there's a cool bit
of testing going on here, Brian, I thought you might like, so I can mock out my endpoint. Like let's
suppose I'm trying to call an API or something and I don't really want it to go to the real server.
I want to give it a test one. I can create a flask app or court or starlet or whatever
and create a mock like connection pool or something to that effect and give it my real website
implemented in Python. And that's the mock behavior of the, this thing.
Okay. Well, that's neat. Is there more information about that somewhere?
Sure. It's covered in the GitHub repo that I linked to. And if you go to the bottom down
there, it talks about it. I don't totally know what's going on exactly. But for example, like,
you know, you want to call, I don't know, your API and you could have like a little local one running
on SQL, SQLite or something. And you could like literally give it the app. It won't even go to
the network. It'll just like directly interact with the app as if it was the web server.
Oh, that's really cool.
Yeah. It's pretty wild. So anyway, this is...
I'll play with that.
Yeah. Yeah. It's a pretty cool thing to be playing with. And it has some other cool
async testing support. So it's got a cool testing angle as well.
Nice. Cool.
Speaking of services, what do you got next?
I did notice that there was an article in the register about a bunch of, I don't know how a
bunch, but some layoffs atnpm.
Right. npm is the commercial equivalent of pip, sort of. And PyPA, Python Package Authority,
like mushed into one, right?
Yeah. I didn't know that it was a for-profit like company, but apparently it is.
Yeah. That's interesting all by itself right there, isn't it?
Yeah. So Dustin Ingram wrote an article talking about, and Dustin is one of the people that
works with PyPI and the Packaging Authority. And he wrote an article called PyPI as a Service.
And essentially there's a couple of interesting comparisons between npm and PyPI. Firstly,
NPM has been around for longer, I guess, since 2014. PyPI in 2002. Oh no, we're older.
So PyPI is older. Math is easy. Anyway, they have like $10 million in funding. And whereas
in 60, around 60 people working there and the PyPI has like less than half a million in grants.
And then just like, there's only half of a fraction of a single employee and then some volunteers.
So there's a lot less people running PyPI and a lot less money. But what if they changed it?
Apparently they've had a lot of people ask for an as a service version. So they could have a,
like a private thing. So you could have your, you could use something like PyPI, but with your own
stuff that's internal or whatever. And generally just talked about some of the problems with that.
One of the problems is it might jeopardize the nonprofit aspect of the Python Software Foundation.
That would be bad. Also, there's a whole bunch of people that, companies that donate services and
infrastructure to make PyPI run at the tune of about a million dollars a year.
And I don't know if, and he mentions that they might not be thrilled about donating that service
to a for-profit company.
I'm sure they wouldn't.
I agree. There's also some of the ecosystem. There's, there's other options out there. However,
I've kind of looked into these and I'm not really thrilled with the other options available. So
I actually think there's room for somebody else to create something like PyPI for private
use that might as a service that might be an opportunity for somebody.
Yeah, for sure. You know, one of them that comes to mind is pydist.com. That's in beta right now,
but it's basically that. It's basically the service that people are asking for. Honestly,
I totally see the problem here, right? We've got this careful balance of this actually really expensive
thing to run, PyPI, especially the traffic and the website and whatnot. And then we've got these
donations taking care of that. There probably is a business model to be had here, but there's going
to be like a wicked dip to zero, near zero. And it has to, somehow you have to be able to climb back
high enough to cover just the expenses, right? Like that's pretty risky to do that.
Right. And also, what do you do with the people that are now volunteering? Do you start paying them?
What if they only volunteer for a couple hours a week or a couple hours a month? Do you pay them for that?
Yeah. And then also, they support some of these alternate services right now, PyPI does. And if they
had an incentive to not support the other services and only support their own profit, that's not good. So anyway,
it's an interesting topic.
I honestly think that there's actually a huge potential. I've talked to people in the community
about this, but I don't know that I've talked about on the podcast before around pypi.org.
If you look at how much traffic is there, there's a huge opportunity for non-invasive ethical ads and
other types of promoted stuff to be put on there. And I'm sure they could do more than a million dollars
with the amount of traffic they have, but there would be that bad dip and they might not make it through
and so on. But I certainly see this as totally possible that it could go that way. And it may be,
you know, given that they're basically saying, hey, we need to receive a donation of $40,000,
$45,000 a month of bandwidth. And if we don't, we're done. Right? That's a really scary situation
to be in as well. They're dependent just on one company. So I don't know. It's super interesting.
The layoffs at NPM, that doesn't make me sound, it makes me really want to encourage PyPI to go that way.
Right. It was interesting at the end, the conclusion wasn't that it's never going to happen. It's just
right now, it looks like it's not worth it. It would be kind of a pain and it might backfire. And
so right now they're not looking into it, but it's not a never thing.
That makes sense. All right. Before we get on to the next one, let me just tell you all about
some cool features at DigitalOcean. Something they just announced is the DigitalOcean Marketplace.
So it's like one click apps and server configuration for all sorts of tools and whatnot.
So maybe you want a discourse server, a GitLab enterprise server, a MongoDB server, or even
Django. You can go up to there, create an account, go into the marketplace and just click the button
and boom, you have an infrastructure all set up, pre-configured to run whatever app it is that
you want there. So there's a bunch there. And I think you can even create more and add them
if you want your project in there. So check them out at pythonbytes.fm/DigitalOcean.
Get $100 free credit for new users and definitely get a play with that marketplace. It's quite cool.
Brian, if I was looking to run, say, some data science Python in the cloud, and I maybe wanted
to do that without paying any money, maybe I was a college professor and I wouldn't have
my students do it or even high school, or I just didn't want to pay money or whatever.
There's a bunch of these services now that'll run basically Jupyter in the cloud, right?
Okay.
So there's a couple of big ones. We've like Azure and we have Google, but there's some smaller ones
as well. And a friend of ours, Kevin Markham, put together a cool article called six easy ways to run
your Jupyter notebook in the cloud. So basically he went through and compared six different services,
assuming they all have, you know, they got into this list. If they had all the characteristics
in that they don't require you to install anything on a machine, they're either completely free or they
have a completely free plan. They give you access to something like a Jupyter notebook environment.
You can import and export real Jupyter notebooks through the IPYNB format, and they support Python
language, maybe others as well. Out of all the ones he looked through, there's six that were decent that
matched those criteria. So we have binder. Maybe you've gone to GitHub and you've seen like a GitHub repo
that has some IPython notebooks in it and a little binder, like a run in binder button. So if you click
that, you can basically use the binder service and run any Jupyter notebook that lives in a public GitHub
repo on binder. So you just click a button and say, run this repo. That's kind of cool, right?
Yeah, very cool.
So that's nice. What I like about Kevin's article is he goes through and says, these are the pros,
these are the cons. So like pros, obviously this is free and easy. If you already work on GitHub,
it's just right there. If you don't, or what you have is a private repo, then binder is not such a
big help. Another thing that's big in data science is Kaggle, right? So they have these Kaggle competitions,
which are like, here's a bunch of data, try to solve a problem by, you know, getting the data to tell you
or training a neural network or something like that. So Kaggle is known for that. But they also have this
thing called kernels, like a free service called kernel. So Kaggle kernels. And these are kind of like
Jupyter Notebooks, like super simplified ones. So you can run your stuff there. That's cool.
Maybe one of the bigger ones is Google Collaboratory or Google Colab. And long as you have a Google account,
it's like Google Docs, but for Jupyter Notebooks, you just log in, go. What I really like about this one is
it's like Google Docs in that like you and I could be working on a problem. And we could just both be typing
at the same time and working right alongside each other. So the Collaboratory bit is like super big there.
Oh, that's nice.
So that's pretty good. And then also you can run your code regularly on a server through Google
Colab, but you can also run it on a GPU or a TPU, TensorFlow unit, processing unit, which is pretty
awesome for something. I don't know if you have to pay for the GPU option, but still pretty cool.
Yeah.
Because a lot of people don't have, certainly don't have TensorFlow chips laying around probably.
Maybe comparable to the Google one is the Azure Notebooks. So, you know, this is a big part of
what Python and Microsoft are doing. You know, what Python is sort of showing up in, in the Microsoft
space is over on Azure and Azure Notebooks. So they've got something that's more of like a project,
right? It's not just a notebook, but it's like multiple notebooks, markdown files, data sets. So
you kind of create these like project folder type things that you can run on Azure Notebooks. And
that's cool. Also free. All of them are free. There's something I had not heard of cocalc for
collaborative calculation. Also super collaboratory, like the Google collaboratory one. Yeah, it's
pretty cool. It lets you do all sorts of stuff like Jupyter notebooks, but also Sage worksheets
and other things, which is good. And finally from JetBrains, we have data lore and it's not exactly
Jupyter notebooks, but it's like a re-imagining of a Jupyter notebook, which is cool, but you can
import and export Jupyter notebooks from it. So that's pretty cool. Like all the features and
auto-complete and cool stuff that you get in PyCharm, but like in a notebook, Jupyter-like notebook is sort of
data lore. Also real-time collaboration there. So those are all, yeah, those are all pretty cool. So if people
are out there thinking like, Hey, I want to do some Jupyter in the cloud, maybe for a course or for
collaboration or something, these are all good options. Yeah. Very cool. Yeah. Nice article, Kevin. And
hopefully it helps some folks out there. Speaking of Jupyter, what do you got next?
Yeah. Did you have this down and I just didn't see it beforehand or did we just coincidence?
Just a coincidence.
Okay. So my next item is Jupyter notebook tutorials. So let's say you have some people you're trying to
collaborate or not. And you have some people that aren't quite familiar with Jupyter notebooks and you
want to get them up to speed really fast. I'm actually in this situation right now, not for me,
but with some people I'm trying to ramp up. And I found there's a ton of tutorials about Jupyter
notebooks, of course, out there, but I really liked, there's two tutorials from a data quest.
The first is Jupyter notebook for beginners, a tutorial. And the second is tutorial advanced
Jupyter notebooks. And it's by the same author, which I should have written down, but I didn't.
It starts out incredibly gentle, but it's also useful and concise and quick. You can get through
quite a bit right away. And so it even starts with like, if you want to install Jupyter on your
computer, how to install it, how to get started up. You don't even, if you aren't even familiar with how
these work, how to get going, a little bit of a discussion about the file type, the IPYNB,
and then a run through of the interface and how, how it works to work with a notebook.
And then even into data. So like loading data and some important things like plotting data right
off the bat, and then even how to share and then using save and checkpoint frequently to make sure
that you save parts of your notebook. It's like this saving often is something we take for granted with
an editor, but it's kind of nice to have this within a Jupyter notebooks also.
Yeah, for sure.
And I kind of like that he says, he stops the beginning one. It's long enough to get people
started right at there, but then the advanced one, once people play with it a little bit,
getting into some advanced things, which like the magics, keywords, debugging,
shell commands, logging, using Seaborn, do using macros and all sorts of stuff in the advanced
one, which are not, it's nice, but it's not, they're not too long. You could probably take an
hour or so and go through it and learn what you need and skip what you don't need yet.
Yeah, that's cool.
I've also been playing with using a PyCharm to edit iPython notebooks. It works really good.
I would recommend if people are debugging notebooks, maybe not use a write in Jupyter. I would probably
use PyCharm for debugging. Yeah. PyCharm is pretty sweet for debugging.
Anyway, so that's what I got. Yeah, it's nice. I think those definitely go well together. So
people can check out those articles and then try them online. They don't even have to install anything.
All right. Final one, Brian comes from friend of the show, Trey Hunter, and, it's called
unique sentinel values. Identity checks and when to use object instead of none. And I think this is
just a really nice reminder. Maybe something a lot of folks don't know, or haven't really thought
about often we have to clear a variable and it may or may not have a value, right? Think of like
computing the minimum of something. So you're gonna, the algorithm is you have the minimum value and you've
got to set that to something and then you're gonna loop through all the items and you're gonna set it to the
lowest. But how do you kind of initialize that, right? Do you set it to none? Do you set it to maybe negative one?
But well, what if the values could be negative? Like what, what is that value, right? So often none becomes this thing
that is the, it's not yet set value, but if none is a valid value, or maybe in this minimum example,
like none could actually be a value that was passed to you in the list that you're trying to find the
minimum of, but you need to treat it differently than actually the not having, you know, like it gets
really weird, right? Yeah. So Trey makes a case for this, for something different in this article.
I really like this. It's like, instead of sending a value to none, you can set it to object,
parenthesis, parenthesis, and that allocates an object, much like none is an object, ironically,
singleton, and things like that. But object is going to be allocated on a heap and we'll never,
no one will ever pass that thing to you, right? There's no other use case for allocating an object,
object, right? You always have something derived from object, right? I don't know, a string,
a number, a customer, but not object itself, like the base class. So just allocating one of those is
like the perfect sentinel value. And there's all kinds of examples where he goes through default
values for function, like default parameters for functions, how you deal with that, how do you deal
with like this minimum example, what is the first value before you've looked at the list. And so you
end up with really nice checks, like that set minimum equals to object. And then if minimum is not initial,
right, or is not this, you know, you set the initial to like this new object, right, then you go work
with it. So super simple adaptation that solves a certain class of bugs and makes it really nice.
That's interesting.
Yeah, it's super simple, but it's like, it's kind of a good reminder. And there's the example,
he re-implements the min function that's a built-in, but in Python, and goes through all the cases to
make it work and shows how this pattern is much better than using none.
I'll have to hit up Trey and ask him how to deal with that with type hints.
Yeah.
Because I'm used to...
Oh, yeah, I know, I know. You can't just say optional. I guess you got to do like union of
object, but that's also not a good answer.
Because that would be everything.
Yeah.
Yeah, right. Yeah, it's not so good.
Anyway, interesting. But it's a cool idea.
Yeah, it's a great idea. It is a little bizarre with the, I don't know, with the type hints,
you're right, but that's okay.
I think you can get around it or just put like a, you know, ignore this one line type
of thing.
Yeah, well, I mean, and it only really will come into place if it's part of your interface
to something.
I mean, you'd probably do have to be careful with like mypy and mypyC and those tools,
so they'll probably freak out if you don't take it into account. So, but yeah, I love the
pattern in general, how it fits with type hints. That's like a different, different aside.
Yeah, sorry, Trey.
Yeah, Trey, there's your follow-up article, man.
Brian, that's it for our official six topics as usual. You got anything extra you want
to share with anyone?
I do. I forgot to write it down, but you reminded me because you're talking about Sentinel values
and Sentinels were part of the Matrix, right?
Oh yeah, they were scary. Those are the big metal things that went through like the underground
tubes and they would like grab onto the spaceship with the good guys in it. Yeah.
Yeah, yeah. But there's something that's not scary is the cool computer screens that would
like go downwards, right? Right.
So one of the things I found, which is, I brought it up because I brought up pytest earlier,
was a plugin called pytest-neo. And it runs all your pytest, like normally if you just run pytest by
default, it'll print out like the test file name and then a whole bunch of dots and the dots mean
everything passed or like an F for fail. The pytest Neo plugin, if you have that installed, that happens,
but it goes downwards and looks like the screen, like a Matrix screen, but actually it looks cool.
But it's actually informative. It has all the same information, just in the wrong, a different direction.
Whoa. I need to learn unit testing. Chunk. Yeah, that's awesome. I love it.
Brings me back to the 90s. That's cool. Very, very cool.
How about you?
I've got a couple of things. Let's start with this one. So I recently released a free course. So
people can check that out over at training.talkpython.fm or just click the link. I think we've talked about
Kenneth Wright's new web framework called Responder, right?
I think so.
I think so. Yeah, I'm pretty sure we have. So I decided to create a super short little mini course on it.
So I created a almost one hour long course on Responder and the framework and it actually uses, goes and builds that we go and build out like a cool API using Responder and then actually consume it with Vue.js as well.
So there's a cool little Vue.js front end on top of that. So if people want to get some quick exposure to Responder, maybe see some of the cool features that it can do.
Kind of Flask-like, but it does a bunch of other awesome stuff as well. It's not just Flask. Check that out. So check out the Responder course. Like I said, it's free. So no problem. You can just go check that out.
And the other one is really cool, but I have to really remind myself to use it. So imagine this. I'm working in an editor. This is for PyTarm or any of the IntelliJ platforms like WebStorm or whatever.
And I've got like a big monitor up, a whole bunch of code on the screen, and I have my fingers on the keyboard and I want to go up to that section where I'm doing like open something, something as fin or whatever.
Right. But I don't want to take my mouse and go over that. I don't want to arrow up like 17 lines and then go to the right control, whatever.
So there's this thing called Ace Jump. Have you heard of this?
No.
Ace Jump lets you hit a hotkey. I think it's on Mac. It's command semicolon.
And then if you type, let's say F-I, it'll go and put like a single character on every space that F-I is a substring of on the screen.
Maybe the one I'm looking for has like a J byte. So it's F-I and hit J. And it'll take you straight to it.
So it basically turns your screen, your code into like, you do a little bit of a search and then like one character keystroke to jump to that section of code.
Oh, that's cool.
It's super cool.
It's super hard to remember to do that. Not just arrow, arrow, arrow, arrow or mouse or whatever. Right. But it's if I can remind myself to do this, it's going to be great.
Okay. Well, I mean, I don't use arrows, man.
Sometimes.
There's a V-I mode.
True. True, true.
Anyway. So, no, that's cool. I'm going to have to try that out.
Yeah. So it's just basically like type as you search and the whole thing becomes like sort of quick jump around your editor.
All right. Are you ready for a pie joke?
I am.
I got to hit it a few times.
What's the object oriented way to become wealthy?
I don't know. How?
Inheritance.
Okay.
That's bad, right?
A good programmer is someone who always looks both ways before crossing a one-way street.
Child.
Dad.
Why does the sun rise in the east and set in the west?
Dad.
Sun is working.
Don't touch it.
All right.
I'll do one more for you because it's about Python 2.7.
Remfinally, Beth removed Python 2.7 from her server in 2020.
Finally, she said with glee, only to see the announcement for Python 4.4.
Yeah.
All right.
That's pretty cool.
Well, I guess we'll leave it there, huh?
Yep.
That's good.
All right, Brian.
Thanks for being here.
Thanks for finding all these items.
Thanks.
Bye.
Thank you for listening to Python Bytes.
Follow the show on Twitter via at Python Bytes.
That's Python Bytes as in B-Y-T-E-S.
And get the full show notes at pythonbytes.fm.
If you have a news item you want featured, just visit pythonbytes.fm and send it our way.
We're always on the lookout for sharing something cool.
On behalf of myself and Brian Okken, this is Michael Kennedy.
Thank you for listening and sharing this podcast with your friends and colleagues.