Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to earbuds.
This is episode 147, recorded September 4th, 2019. I'm Michael Kennedy.
I am Brian Okken.
And this episode is brought to you by DigitalOcean. Tell you more about some cool stuff they got coming up later.
Right now, I want to talk about tables.
Okay. And really quick, what's 147 in hex? No, just I couldn't do that. I don't expect you.
Anyway, no, but one of the things I do need to do occasionally, I need to do really fast is generate tables.
And I know it's not something that everybody needs to do, but if I've got a whole bunch of data, like in this case, some dictionaries list, and I want to be able to just print that stuff out.
They're all the same type of dictionary. They just hold some data. I want to print them out.
How do I do that quickly? And for test reports or whatever, I found this called rapid tables.
And it's really kind of awesome.
Oh, that is so cool. Wow.
Yeah. So you just basically, you've got a list of dictionaries, and then you want to print them out, and you want it to look like a good ASCII table.
It shows you how to do it, and it also shows you how to do it with a thing called term color.colored.
I didn't know this was around. So you can do colored tables if you want. I probably won't do that.
It also converts stuff to markdown and restructured text. That's kind of neat. I might use that, but right away, I just need the just really easy tables.
And it says it's super fast in speed, like speed-wise. It goes fast.
But what I like is that it's not a lot of extra code gunk in your code. You just kind of say, I want this table, and you can print it out easily.
Yeah, that's super cool, because a lot of times, like this kind of formatting, it's a lot of Z fill and pad these strings and L pad.
Like just like trying to get the spacing, or you're putting tabs, but every now and then the tab, like the word is too big, and so the tab will shoot it to the next column.
It's a lot of work, and this is not you doing that work.
Right. So for all the columns, you have to go through each of the elements to find the longest one to figure out how wide you need to make the column.
And yeah, there's other libraries to do this, but this one I think is pretty cool.
Yeah, it's very cool. I also like the markdown and restructured text output option. Yeah, very cool.
So if you think about what is like one of the most popular packages, what would you say? Take some guesses.
Requests.
It's got to be requests. I don't know if it's the most used one, but it's certainly the one that has people's attention and imagination.
So what would you think if I told you here is a competitor to requests called HTTPX?
Ooh, I like the X.
It's extra cool. Absolutely.
What's really interesting about HTTPX, though, is it's not a from scratch, like throw away the ideas of requests.
It's compatible with the request API.
So what you can do with requests, you can do with this.
So you can kind of drop it in.
But it has all these other cool features.
And in fact, it comes from this project called ENCODE, E-N-C-O-D-E, ENCODE.
And over there, they have a ton of cool things, including Django, REST Framework, Starlet, UVicorn, HTTPX, and also some other async stuff.
And a lot of the things happening around here, UVicorn databases, ORM, HTTPX, is all about adding the asynchronous capabilities to these various things.
And that's one of the shortcomings of requests is it doesn't support an async option.
But this HTTPX, it does a bunch of cool things around the async space to kind of let you gradually go up the stack of complexity.
So you can just be straight synchronous stuff doing what request does.
It has this concept of parallel requests.
So you can issue a bunch of requests in parallel, but you don't have to think about actually creating the async event loop and stuff.
You just say, I want to run these things all in parallel.
And you could even ask it, say, like, I want to process the responses as they come in.
So I can say, like, while parallel.haspending responses, the response is parallel.nextresponse.
So I can kick off, like, 20 requests and just process them as they come back in, like, four lines of code or something.
Oh, that's so cool.
I like it.
Yeah.
Or if you really want, if you're in something like Starlit, where it has actual async support for the method, you do want that to be async.
So you can actually flip into, like, an async mode where those happen the same way I described, but also on the asyncio event loop.
So it's really cool.
Also, some other neat features.
So, like I said, a request-compatible API.
It does HTTP2, which, like, I could do a request for a page.
And as I pull down, like, say, five CS style sheets and a JavaScript and an image, you can go get those all in one request.
So it has support for that, which is cool.
It has a standard synchronous interface, but also async and await if you want it.
And, Brian, you'll like this one.
It allows you to make direct requests to WSGI or ASGI, the asynchronous variant of that, apps.
So imagine I'm creating a Flask app.
I don't want to test it.
I can actually, inside of my Flask app, provide HTTPX the app, right?
The Flask app object.
And then I can issue requests that don't go through the network layer.
They just go through this test app instance that I've created.
Oh, that's cool.
Isn't that cool?
So if I want to, like, mock out some behavior or, like, interact with it as if I were externally, I don't have to, like, start a server and talk to it.
I'd literally just, like, use the same API here but, like, directly skipping all the infrastructure.
Oh, that sounds fun.
Yeah.
In, like, you know, a nerdy kind of fun.
Yeah, that sounds, like, really fun.
But, yeah.
It is also fully type annotated.
So it's very, like, Python 3 friendly.
Has 100% test coverage.
And, yeah, it has support for mocking out things.
A lot of cool, nice features.
And the fact it's built, all this is built on top of requests means if you're already using requests, it's, like, you can adopt it pretty easily.
Because you don't have to change anything.
But if you want to bring in some of these cool new features, it's just, you know, you start using them.
Nice.
Now, one caveat.
I does say that this is in some form of alpha.
It's, like, pretty early days for this project.
But, yeah, it says this project should be considered an alpha release.
It's substantially API complete, but some areas need to work.
Well, you know, people start using it.
People start contributing to it.
All of a sudden, it becomes not alpha.
So, anyway, it's out there for people to use.
This is sort of the same umbrella area as Tom Christie in the Django REST framework area.
So, I think it's a pretty good place.
Cool.
Okay.
Neat.
Speaking of stuff under the encode GitHub project.
So, Starlette is part of that also?
Yeah, yeah.
Okay.
So, interesting that we did not plan this.
No.
The universe planned it to line it up for us.
Oh, my gosh.
We're wearing the same color shirt also.
Oh, my gosh.
It's black.
So, that's not that weird of a coincidence.
Anyway, so, the article I wanted to highlight was a quick and dirty mock services with Starlet.
This is by a friend of the show, Matt Lehman.
Hey, Matt.
It's kind of a neat thing.
So, one of the ideas, I don't know if we've talked about this on the show, but the idea of testing.
So, if you've got something that you're part of your application or the thing you want to test talks to a third-party service, sometimes you might not actually want to call that third-party service.
You can mock or stub out the calls to the service.
Or, one of the ideas is to create a fake service to talk to.
And so, that's what this article is about is how to create a fake service so that you can use it for testing.
And Matt writes a really quick one using Starlet.
And I think you could use other frameworks, too.
But I like how simple this is.
One of the needs he had was that he didn't want the service to, like, just come back right away.
He wanted some delay because there were some of the service calls that were taking up to a minute or longer.
And he wanted to be able to simulate that within his fake service so that you can test the code under test to make sure it is working correct with a big delay.
Anyway, like, it was a really easy-to-read write-up, easy way to do that.
And I know that that's a recommended practice for a lot of instances.
But there aren't any very many write-ups of how to do it.
And so, congrats, Matt.
I like it.
Yeah, that's really cool.
When you think of dependencies, you think of, well, my database layer or Stripe API or something.
But time is one of those dependencies that's really tricky.
Most of the time, one of the reasons why we try to do a Maka service is to eliminate that time.
You know, you want to make sure that the flow of the data goes quickly, but you don't want to wait for the service during your test.
But at the same time, you kind of want to make sure that you're – and in most test cases, maybe that's the right thing.
But you also want to make sure that your application deals with long latencies if that's the normal situation.
Yeah, for sure.
You might want to shrink it from five minutes to five seconds or something so the tests don't run super long.
But yeah.
Or even like I want to start something and then see in an hour if it's done.
You could like, you know, have to fake out like, you know, datetime.now.
So the next time you ask it, it's like two hours from now, but it's one millisecond, you know?
Yeah.
Well, and, you know, make sure your parameterization of that test doesn't make it so that you're running 50 different versions of the hour-long wait.
Yeah, for sure.
All right.
Now, before we get to the next item, let me tell you about some new services from DigitalOcean.
So they've traditionally had recently added Postgres as a service.
So now hosted database, which is really cool.
But maybe you don't use Postgres.
Maybe you want to use Redis.
We just talked about queuing and stuff.
So Redis is now a service that you can get over at DigitalOcean.
Redis as a service.
And also MySQL as a service.
So check that out over at pythonbytes.fm/DigitalOcean.
Get $50 credit for a new user.
They've got all sorts of good stuff coming your way.
And, of course, all of our infrastructure runs on it as well, which is fun and very good.
All right.
The next thing I want to talk about is that other larger cloud provider.
Have you heard of something?
Some people started using this, Brian.
It's called AWS.
Have you heard of this?
I think so, yeah.
Yeah.
I think like a quarter of the internet at least runs on it.
So the Python API, the Python package that used to talk to things like S3 and EC2 and AMI and all the stuff over at AWS is called Bodo.
Yeah.
So there's a project brought to our attention by Giuseppe Consolo.
Thank you for that, Giuseppe.
Called Mocking Out AWS APIs with Modo.
So if you're going to mock out Bodo, what would you call it?
Modo.
Oh, nice.
Now, it lets you do all sorts of stuff.
Like if you, say, want to mock out an S3 bucket, right?
So S3 is like file blob storage.
You can mock it out and then like save a file to it and then see if it's there.
Preload it with data and then interact with it.
And it's got all sorts of interesting stuff.
But if you start to dig through this library, it turns out it's got both some really interesting ways to test stuff and it's also super comprehensive.
So, for example, if I want to test working with S3, I can say from Modo import mock S3.
And then I just put an at mock S3 on my method, on my test method.
And now all of my true S3 calls with Bodo actually don't go to Bodo.
They go to this Modo mock version.
Oh, that's neat.
Yeah.
But apparently it doesn't really end there because if I wanted to work with like EC2, I could import mock EC2 and put that on there and say Bodo dot working with, you know, do all the Bodo client EC2, whatever.
It's like, I don't know why Bodo is so terribly complicated.
It's like super overloaded.
But anyway, that's a different time for a different story.
You can go and like ask for your reserved instances and it'll use your mocked out ones and things like that.
So, just like if you were to go flip through all the stuff that it supports, ACM, API gateway, auto scaling, cloud formation, cloud watch, cloud watch events, cognito identity, data pipeline, DynamoDB, et cetera, et cetera.
And we're only into the Ds, right?
It just goes on and on and on of all the different AWS things that it mocks out.
So, if you got to test AWS, this is pretty killer.
Like you can mock out Glacier, IoT, Kinesis, whatever, you name it.
One of the notes you added was that it can run as a standalone server mode so that you can even test non-Python code.
That's right.
So, apparently I can, yeah, there's a way to run it as a server and then talk to it as if it were some of these services as well, which is pretty wild.
It is incredible.
And I guess good that Bodo is all of the interface so that it's one point to mock it.
Yeah, yeah, Bodo, the reason I don't like it is you say Bodo and you create a client and then you pass it a string.
The string could be like EC2 or S3.
And then how you interact with it has to do with what factory that string drove it to to create a different, like it's really hard to discover what can I do with this thing?
Because it's like a generic blob that can be converted into something more real.
I don't know.
It's funky.
But yes, in this regard, it's definitely very nice.
Apparently, your life is more complicated than mine.
I guess I've worked with S3 and Elastic Transcoder and all that stuff too much.
Yeah.
Okay, cool.
What's next?
I was just thinking that maybe we should have done a testing code episode with all this testing stuff.
Yeah.
But one of the things that we have, I think we've talked about Mongo Engine before.
And I know that you've used Mongo Engine in some of your courses, right?
Yeah, for sure.
And on the sites.
Yeah.
And it's like a mapper's...
They don't say ORM because object relational mapping because it's not really the R.
So they say ODM for object document mapper.
That's like the...
Okay.
The adapted acronym.
So I didn't know that there were...
So I knew about Mongo Engine because of you, but I didn't know about others.
And here's a project called MicroMongo or Mew Mongo.
Not sure.
Yeah.
I think MicroMongo is the way you would say it.
I think that's the Greek.
Yeah.
Mew, yeah.
It's a Python MongoDB ODM.
And it comes because the people that made it had two needs.
They wanted an asynchronous ODM.
And for some reason, they felt like it was difficult to do serialization and un-serialization of documents
with existing ODMs.
And so this one doesn't.
So it's an asynchronous sort of thing.
It works directly with some of the popular Mongo drivers like PyMongo, TX Mongo, that's
for twisted, Motor AsyncIO, and MongoMock.
And I'm kind of excited about MongoMock.
Yeah, for sure.
I got to say, this is probably the most interesting aspect of this because something like Mongo
Engine, it just works with PyMongo.
But this one says, you know what?
We actually can work with all these different foundations, which gives it like fundamentally
different behaviors and features, right?
If you pointed at the twisted one, it can now integrate with twisted's async-ers model.
Motor AsyncIO now lets it plug into true AsyncIO stuff.
Like you could plug it into Scarlet like you discussed before and those async methods there.
And then the mocking as well.
Like we're just going to take the foundation and make it a mock version.
All those are super cool.
Yeah.
And then also because of this, I learned about MongoMock, which is a cool way to mock out
your Mongo database.
So kind of neat.
Definitely want to try this.
Yeah.
Sometime.
Yeah.
It definitely looks interesting.
I don't remember exactly when this got created.
Do you know what GitHub needs?
Is it needs a, this project was created on this day, right?
Because you can go back to the commits.
But if there's thousands of commits, you've got to like infinite scroll your way back to
the origins.
But anyway, yeah, it's really cool.
I think if I had known about this at first, I might've even chosen this over MongoEngine.
But yeah, it's really nice.
It's nice.
It has a 90% plus test coverage.
I really like the different foundations, which means you learn like one ODM style of programming,
but then you can use it in these different situations.
Yeah.
Super cool.
I like it.
And also I didn't know about MongoMock either, but now I'm excited about it.
That looks cool.
Nice.
All right.
So let me hit you with an opinionated piece here and you can give me some feedback.
Let me know what you think about it.
Some parts of it.
Tomatoes ready.
Yeah.
Perfect.
Get ready to throw them.
I'll get ready to dodge them.
So this article was sent over by Tyler Madison.
Listener, thanks for sending that over.
Now it's called the single responsibility principle in Python.
And the single responsibility principle or SRP comes out of this larger group of design principles,
coding principles, architecture principles, something like that called solid.
So S is a single responsibility principle.
O is the open close principle.
L is Liskov substitution principle.
And these even come in really interesting demotivator style posters.
I don't know if you're all familiar with demotivators.
You're probably familiar with motivators.
You will see this probably in the entryway, like kind of near the CEO office.
It's like probably got a giant eagle soaring above clouds.
Like together we can fly higher than you ever dreamed or something stupid like that.
Well, the demotivators take pictures like that but put like negative captions and connotations.
So these are really interesting things.
For example, the Liskov substitution one has a duck and a rubber duck in the water.
And it says, if it looks like a duck and quacks like a duck but it needs batteries, you probably have the wrong abstraction.
Right?
So there's all these really cool demotivator style posters around the solid principles.
And I think the solid principles are pretty neat.
Like you can go overboard with them.
But they are pretty powerful.
So this article focuses specifically on the single responsibility principle.
And I think talking about it is really interesting.
I think the solution is unexpected.
And it made me think.
And at first I'm like, oh, I don't like it.
But after a while I'm like, but it is pretty creative.
And things that make me think I like to share with people.
So I guess that's why I chose it.
And it starts out by saying this article will guide you through the complex process of writing simple code.
And it talks about stuff that you would expect.
Right?
So there's some kind of super long function.
So this is terrible.
You can't test it.
You can't reason about it because it's doing five things.
How do you know whether you're doing too much?
You can look at measures like the cyclometric complexity, the Halstead complexity thing, the number of arguments, the number of statements, body length, all that kind of stuff.
And that will help you condense down into, say, reasonable classes or functions.
And this is really mostly focusing on functions.
So all right.
So let's think of another simple example.
I want to calculate the price of a bunch of products given a list of products.
And it seems all well and good except for that it also happens to use some kind of logger object and log out the price.
Well, now this thing is doing two things.
It's one, calculating it and logging it out.
What if you want to test it or work with it with its computation bits but not the logging bit or something like that?
So how do you go around dealing with these types of dependencies?
Right?
This kind of goes back to the testing side of things, Brian.
Yeah.
How do you deal with things like files and database connections and or database connection objects and, you know, loggers and whatnot?
So this guy takes us through this whole sort of thought process, which is really good, and says, you know what we're going to do?
We're going to come up with callable classes.
Oh, dear.
Well, the callable classes will be given the dependencies, and then you call them like functions, allowing you to pass different dependencies or other dependencies.
And now that seems like a really bad idea, but they're using a lot of – my first thought was, okay, well, that's, you know, that's like a curvy-shaped weird hammer that I guess you could hit it with.
But it's using a lot of interesting Python features.
So if you create a class that has – it's a data class.
You set frozen equal to true, slots equal to true.
It means all of its variables become internal and private only.
You set it to be private or final, rather, so it can't be derived from.
And you give it a call function.
It turns out to be pretty interesting.
And then they talk about some dependency injection frameworks that work with these types of classes.
In the end, it comes up with a pretty interesting way to, you know, create these things and then call them.
I'm not sure that I love it, but it definitely made me think, and I was a little bit surprised by it.
So I thought you all might enjoy it.
Yeah.
It's an entertaining read.
I'll give you that.
Yeah.
Look at the very last code fragment in that article, Brian.
Okay.
The last code fragment that says, like, send postcards equals container dot resolve.
Send to these postcards.
Yes, exactly.
And a little bit above it, yeah.
Yeah, yeah.
So it is powerful in that you can create these objects.
You can pass in mock objects without actually using mock dot patch and stuff.
It's interesting.
I don't think that I would go write software using this today, but I might use some of the concepts of it, maybe.
My opinion is that it is possible to do this, and it might make sure that you don't get fired because nobody else can edit your code.
Yeah.
Okay.
So I do like the single responsibility part of solid.
I am not a fan of the Luskov substitution principle or the open closed or dependency injection is something that frightens me.
These are things that were put in place to work around limitations of other languages like Java and C.
And I'm going to get tomatoes thrown at me for coming up with saying this, but some problems are complex.
And you can remove the complexity out of one part of your code, but it moves to somewhere else.
And maybe it makes your unit tests better, but your system tests are just the same.
Or it makes writing regular code super hard because you're like, where did this database connection come from?
I have no idea how I got here.
Right?
Yeah.
It is a thing.
It's an interesting article.
I'm not a fan.
Yeah.
I submit this to everyone as a thought piece, not necessarily as guidance.
Yeah.
But I'm not willing to say that it's wrong.
It's just my opinion.
All right.
Well, I think we should leave it there for our main topics.
Got anything you want to share with folks this week?
I've got one quick one.
Oh, you share it with us then.
So do you know Bob and Julian over at PyBytes?
Of course, right?
Well, they recently have their code challenges platform, which is at codechallenge.is, right?
The .es extension, right?
Just code challenges, put the period between G and ES.
And that's a place where you can go do practice exercises.
And it's a cool platform they've created.
And they just came out with Byte220, where they walk people through analyzing the Python Byte's RSS feed.
Oh, cool.
Yeah.
So you can go in there, and it shows you how to use Feed Parser, which is a really cool way to parse RSS feeds.
Go in there and actually answer questions.
What were they looking at?
It was things like, what is the average duration of an episode in seconds?
What are the number of episodes with special guests?
Like Trey was on the show last week, for example.
What are the most common domain names we mentioned?
Like GitHub probably is right up near the top and things like that.
So it's pretty cool that there's like this guided challenge to get through our platform, at least our platform's data.
Yeah, I'll have to come up with ways in our show notes to break their challenge.
No.
Yeah, that sounds good.
So that's all fun.
People can check that out if they're interested.
You ready for some jokes?
Definitely.
These come from Web Boss's Dad Jokes GitHub repo.
So they're pretty short and sweet, but they're in the dad joke style, which is kind of bad jokes that are hopefully funny.
So what do you get when you cross a computer and a lifeguard?
I don't know.
What?
A screensaver.
What do you get when you cross a computer with an elephant?
Lots of memory.
Okay.
Nice.
So I'm going to share this because I made it up and I thought it was funny, but it's an anti-joke.
So a Python developer, a PHP developer, a C# developer, and a Go developer all went to lunch together.
Oh my gosh, what happened?
They had a nice lunch and all got along fine.
Perfect.
Of course they should.
They're all developers.
Anyway, thanks.
Awesome.
Yeah, well, thanks as always, Brian.
See you later.
Thank you for listening to Python Bytes.
Follow the show on Twitter via at Python Bytes.
That's Python Bytes as in B-Y-T-E-S.
And get the full show notes at pythonbytes.fm.
If you have a news item you want featured, just visit pythonbytes.fm and send it our way.
We're always on the lookout for sharing something cool.
On behalf of myself and Brian Okken, this is Michael Kennedy.
Thank you for listening and sharing this podcast with your friends and colleagues.