Transcript #30: You are not Google and other ruminations

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly

00:04 to your earbuds. This is episode 30, recorded on June 13, 2017. I'm Michael Kennedy.

00:11 And I'm Brian Okken.

00:12 And we have tons of good stuff to share with you today. I'm very excited about all of our items.

00:16 But first, we have a brand new sponsor. I want to say thank you.

00:20 That's exciting.

00:21 It's super exciting. So I want to welcome Datadog to Python Bytes as a sponsor. And Datadog has

00:27 this cool thing where if you do a little test integration with them, you actually get a free

00:32 shirt. So I'll tell people how to get a free shirt later.

00:34 Wonderful. Cool.

00:35 I heard that not everybody writes software like at the scale of Google.

00:38 Yeah, I think that's probably true, especially of me. But there's a couple articles that I want

00:45 to talk about today. Actually, mostly one. There's an article by, I think it's Ozan One, which is a

00:53 pretty cool name, called You Are Not Google. And also he goes on to say that you're also not

00:58 Amazon and you're also not LinkedIn. But it isn't to say that the Google, Amazon, LinkedIn all have

01:06 applications that might look similar to normal folks' applications. But the scale is definitely

01:12 different. So just be aware of that. And I guess it's a reaction to people chasing a lot of the

01:18 shiny new technologies like asynchronous IO and other things. And I'm guilty of that as well. But

01:26 he presents, when looking at solutions, he presents a model called UNPAT, I think, or maybe it's UNFAT.

01:34 I'm not sure. U-N-P-H-A-T, which is try to understand the problem that you're first. Enumerate multiple

01:42 possible candidate solutions. Read papers or articles about the solution that you're going to try. Look at the

01:51 historical context of why the solution came to be in the first place. List out the advantages and disadvantages.

01:58 And don't forget to think. Make sure that that solution really fits your problem. And anyway, I think it's a good

02:04 discussion about how a lot of the architectures and stuff that people write about and how Google and

02:11 others do things might not apply to you. So it's just a heads up. I think it's a super interesting article.

02:16 And it's really, it's a good reminder. I mean, on one hand, it's really interesting to think about

02:23 computation at like a massive, massive scale, like some of these data center things and the incredible

02:29 failover and possibly like global redundancy that some of these companies like Google operate at.

02:37 And people read these things are like, oh, we're going to do our startup and it's going to have like,

02:42 if we're lucky, we'll have a thousand users after like three months. Well, maybe you don't need to

02:48 architect around the same things that say Google or LinkedIn or Amazon are architecting around.

02:53 Maybe you need to link architect around. Let's get this thing working as fast as possible.

02:57 And then we'll deal with the scale later. So there's a quote that says, and here is pretty nice

03:02 says the thing is there's like five companies in the world that run jobs as big as Google for everybody

03:08 else. You're doing all this IO and fault tolerance and things you don't really need. Yeah. It's really

03:12 just this idea of like, it's cool to study in these patterns, but these patterns were created within

03:18 the context of a problem. Do you have that same problem as like the recurring theme? I really

03:22 like it. Yeah. There's also, even if you are huge, even if your problems are big, they still might not

03:28 be the same. And one of the things he talks about is maybe the, like your data store, maybe the number

03:34 of rights is more important than, than how often it's read or vice versa. So even at large scales,

03:41 the problem might be different than somebody else's. So yeah. Yeah. For example, Amazon optimized

03:47 for right tolerance on the database that backs their shopping cart. But is that, is that your

03:55 primary concern? If it is, maybe do what they're doing. If it's not, then maybe that's not the best

03:59 database for a general database, right? For example. Yeah. Yeah. Anyway, it's good read. Speaking of

04:04 databases. Oh, do you, let's talk about your microservice one as well that ties into this.

04:08 Oh, just along the same line, I came, I ran across another article that just is called

04:14 enough with microservices. And it's a similar type of article. It's also well-written. It's by Adam

04:20 Drake and we'll have a link up. Mostly it's similar sort of talk of discussion about microservices and

04:26 dependencies and, and how that's a, that's a complexity that is, it adds to your cost. So make sure that

04:34 you're aware of that before you try to jump into it. Yeah. I recently had Miguel,

04:38 Miguel Grinberg on Talk Python. I haven't released that episode yet because I have like three months

04:44 backlog of stuff recorded that's got to get out, but it's, it's on its way. And it's a really good

04:47 episode around microservices. So if you're really thinking about microservices, check that out. It's,

04:53 it's really enlightening. Actually, I learned a lot from talking to him, but I think one of the takeaways

04:58 is that switching to microservices makes your application simpler than it otherwise would be.

05:04 Instead of having one complex application, you now have like say six very simple applications,

05:08 but your DevOps and deployment and coordination and your infrastructure story gets way more complicated.

05:14 And so you're kind of pushing the complexity of your application around. And does it make sense to push

05:21 programming complexity into infrastructure DevOps complexity? I think it depends on your organization,

05:28 how many people are on your team, how complex is your app? But certainly a lot of small apps probably

05:33 shouldn't be microservices. It's interesting that you bring that up with what your organization looks

05:37 like, because there's a lot of small startups and small organizations and sometimes just individual

05:43 people that you really pay attention to what you like to spend your time on and what your skills are.

05:48 Because like when you just said that, I was like, wow, I better be very careful about that because

05:54 DevOps is not my strong suit. So exactly. I'd rather have the complexity in my app and carefully factor

06:00 that thing rather than push it to a bunch of servers that coordinate. Right?

06:04 Yeah. Yep. Yep. Pretty, pretty interesting. So speaking of databases and things that are complex

06:09 and things that scale super high, let's actually not talk about a database with this thing called

06:13 NoDB. Okay. I haven't heard of this at all. So tell me about NoDB. So NoDB is a Pythonic object store

06:20 that uses Amazon S3 as the backend. So it, as a programming interface, it looks like

06:28 a simple NoSQL database. But what it's actually doing instead of running a server or something is

06:35 it's talking to S3 and storing your objects there. So you can like insert into the database and you

06:43 configure that you're like your connection string, if you will, for the database is here's my S3 account.

06:48 And here's the bucket. I want to store it into a folder, think folder. And then you just like insert,

06:54 query, update and delete from this database. But what actually happens is it stores it over there.

07:01 And I believe the default is actually to pickle the Python object. So you even get like type

07:07 preservation, like you could insert a customer and then boom, out comes a customer with its like functions

07:12 Okay. Interesting. Yeah. Interesting. Right. So this was done by Rich Jones. He released it in April.

07:20 And it's, it sort of ties in with some of this serverless architecture, right? Like, this is the guy that works on Zappa. We talked about Zappa last time, which lets you run web applications on Amazon Lambda, which is already pretty interesting. So this is like another, it's like, you don't have a web server. So maybe you get away with not having a database.

07:41 And it can handle a decent amount of load, but it's not like a full on super database. It's more like for prototyping and things like that.

07:48 Okay, cool.

07:49 Yeah. So some of the examples of he said it might be good for prototyping, like I said, but also storing API event responses for like replay. So if you are doing microservices, you want to store all the traffic that goes back and forth, you could just do that here really easily capturing logs, simple data like here, add me your email to this, this thing.

08:10 One of the more interesting things is if you're doing Lambda, AWS Lambda, you can have triggers that call the function based on S3 events, this file was changed, this bucket had a new thing added. And what that means is you could insert into the database and it would call an AWS Lambda function as a result of that. So you could like insert this thing and the act of storing it also kicks off some action.

08:38 Oh, neat. That's actually pretty cool.

08:39 Yeah, it's pretty neat.

08:40 So, plus the article has a nice picture of a fish skeleton. I like it.

08:46 Yes. Yeah. Pictures are important. The one that you're talking about next also has a cool picture. Cool logo.

08:53 Yeah. I've heard of Elizabeth a few times. Yeah. I do have to admit that the logo did bring me into this a little bit. So.

09:02 Yeah. We talked about Faker before, which would let you create like test data that looked real, like give me an address, give me an email, things like that. This is like a competitor to Faker, huh?

09:12 Yes. And it's, so, if you haven't listened to Faker, that was on episode 25. I looked it up this time. It's definitely a competitor to Faker. They even, have some comparisons and it looks like on their, their project page.

09:27 One of the main features that they're going for, for Elizabeth is performance. Apparently it's faster than Faker.

09:34 Yeah. Faker's kind of slow. I mean, Faker's really nice, but it is, I tried to generate a database that had like a couple million entries with Faker and it was a little, it took a while. Let's just say.

09:43 Oh, yeah. Yeah. I haven't tried anything huge. I wonder, I'm curious to how Elizabeth compares, but it, it definitely is a similar space, but I think it, it's just another project. Maybe it fits better for your project. It's, and there's the, the articles were really well, well written.

10:00 So there's, we're linking up two part medium articles and, there's also a, it looks like the same person wrote a pie test plugin so that you can, and the pie test plugin is actually pretty darn cool. It, it, allows you to within a test to be able to, as a fixture, you can bring in, different parts of the, fake data.

10:20 So, yeah, that's really cool. Yeah. It definitely looks nice. I'm, I feel like it's in some ways complimentary to Faker. I'm not sure you would use both in the same thing,

10:30 but you can get, kind of get slightly different data. So depending on what you're after one or the other, maybe better.

10:34 It's a slightly different model of how to pull the data out. So I think it's good for people to try both and see which style works best for them. It also does a different, localization as well.

10:44 So yeah, the localization, all these is pretty impressive to me actually.

10:48 Yeah. I wouldn't want to try to do that project myself, but I'm glad it's around.

10:52 I'm glad it exists.

10:54 Yes.

10:54 Are you ready to hear about how to get a free t-shirt?

10:57 I really want a free t-shirt actually.

10:59 Actually. Yeah. So, Datadog, those guys came along and said, Hey, we'd love to sponsor and support the show and get the word out about, out about our, our project that we got here.

11:09 So Datadog is, we've talked about Rollbar before, right?

11:13 And Rollbar monitors your application for errors.

11:16 Well, Datadog kind of does something a little bit similar on a grand scale.

11:21 So Datadog will look at your application and all the layers of infrastructure on it.

11:29 Let's suppose we have say a Flask app.

11:31 We could integrate Datadog and it will give us metrics about that Flask app, but it'll also tell you about like the engine X web server and your database and the Linux machine that it's running on.

11:43 And basically the entire stack of your application from the servers, the database servers, the web server, all those things, and put all that stuff together.

11:53 So you can have a really holistic view of what you're doing.

11:57 And you can even integrate it with all these different things.

12:00 It'll integrate with things like AWS.

12:02 It integrates with Rollbar if you use those guys.

12:05 It integrates with many, many different things that you might already be using.

12:08 So it's super powerful.

12:10 It integrates with Postgres, with MongoDB, and so on.

12:12 So very, very cool.

12:14 Companies like Zendesk and Salesforce and even PagerDuty use it.

12:18 If you haven't heard of Datadog, if you haven't tried it, go to pythonbytes.fm/Datadog.

12:24 And they've got this little thing you tried out and you get a free t-shirt.

12:28 So pythonbytes.fm/Datadog.

12:30 Support the show and get a shirt.

12:31 I think this shirt's cute also.

12:33 Yeah.

12:33 Yeah, it's nice.

12:34 So thanks, Datadog.

12:35 And you know what?

12:36 Let's talk about what's coming in Python.

12:39 I feel like my next two items actually are both sort of future-looking Python things.

12:45 So I feel like we just talked about Python 3.6, didn't we?

12:49 We've been talking about it since the beginning, yeah.

12:51 Yeah, it's been out for, I guess it's been out for a while now.

12:54 And so they're starting to talk about what's coming in Python 3.7.

12:58 Okay.

12:58 I haven't looked at all, so I'm interested.

13:01 I kind of wanted to highlight that.

13:02 There's a whole bunch of things that I put here that are interesting.

13:05 Two that I think are really worth, like, super interesting.

13:09 And I'll just touch on the other ones.

13:10 The first one is an optimization.

13:13 Okay.

13:14 So Python works by having a bunch of opcodes and then interpreting those opcodes in this,

13:20 like, giant switch method in this file called cval.c.

13:25 And it basically is a loop and a switch method.

13:28 And it looks the opcodes and it figures out what to do.

13:30 So they've added two new opcodes, load method and call method.

13:34 And it allows them to skip some instantiation of a few objects.

13:39 And it results in potentially methods in Python 3.7 being 20% faster than Python 3.6.

13:46 Oh, cool.

13:47 So one of the big sort of trade-offs that you make in Python is function calls are relatively expensive compared to other operations.

13:56 And we obviously want to write smaller functions and break our code apart for usability and readability.

14:03 But that can make things slow.

14:06 So having faster functions can actually make a really big difference in Python.

14:09 Okay.

14:10 Neat.

14:10 So 3.6 to optimize dictionaries a lot.

14:13 And we might optimize function calls in 3.7.

14:15 Yeah, absolutely.

14:17 Absolutely.

14:17 So there's some new modules.

14:19 Like, there's a new remainder function in math, the dis function, which is a disassembly function.

14:25 If you've ever, if you haven't done this, it's pretty cool.

14:27 You can say import dis.

14:28 I think it's dis.dis, module.disassemble.

14:33 And you give it like a function or a class or something, and it'll show you the opcodes, kind of like that load method call method I was talking about.

14:39 Yeah.

14:40 Another really interesting thing that's coming in 3.7 is async context manager.

14:45 So a context manager is a thing you can use in a with block, right?

14:50 Like a file handle, database transaction, those types of things.

14:54 Well, you can have asynchronous context blocks, and this async context manager lets you basically make the instantiation step in those context managers asynchronous, which is pretty cool.

15:07 Oh, that's cool.

15:08 Yeah, one more that's kind of for the crazy book is now functions can have more than 255 arguments.

15:16 Apparently, that was a limit that was bothering someone, and they said, well, let's make it possible for functions to have more than, you know, like 300 arguments because 250 wasn't enough.

15:27 Yeah, I run into that all the time.

15:29 I do, too.

15:30 It's really frustrating.

15:31 Why would you need that?

15:34 I have no idea, especially when you got star args and star star kiddovi args.

15:38 So anyway, it's now a thing, or it's going to be a thing in 3.7.

15:42 Yeah, interesting.

15:44 It looks like you wrote down bytes from hex and bitery from hex.

15:50 Yeah, so those are conversion functions that'll parse hexadecimal strings into bytes.

15:58 And the change is that it used to have an error if there was like white space on the beginning or end, which really didn't affect what the thing was, but it wouldn't accept them.

16:08 So now they basically strip off all the white space for you.

16:12 And so it's a little more tolerant of inputs.

16:14 Okay, cool.

16:14 That'll matter for some people.

16:16 More tolerance is always good, in my opinion.

16:20 Yeah.

16:21 I would love it if there was like an army of people or things that could go test my code and find all my errors for me.

16:27 Yeah.

16:27 Well, I was really glad.

16:30 So there's an article called Unleash the Test Army.

16:33 It is about a hypothesis.

16:35 And I'm glad this came around because since I talk about testing a lot, I get questions about hypothesis a lot.

16:44 And I have never used it.

16:46 I know that you've had...

16:48 Dave McKeever?

16:48 Yeah.

16:49 I think you've had him on the show.

16:51 Yep, I have.

16:52 On Talk Python episode 67.

16:54 Oh, you're ready too.

16:55 Did you look that up?

16:56 No, I was talking about it last night, actually.

16:58 It's somebody's experience with working with Hypothesis.

17:02 It's a good introductory article to kind of tell you what it is.

17:06 So Hypothesis is a testing framework that will really just come up with a lot of different ways to throw...

17:13 You set it up so that it throws different data at your code.

17:18 And it's more of a unit test type thing, I think.

17:21 You have to define the input and output of your functions and whatnot to make it work.

17:26 It's really pretty quick about being abusive and getting at where the problem areas might be.

17:32 This is the first article that I've read that kind of explained how to get into it quickly.

17:36 Because Hypothesis doesn't look like something that you can really just pick up right off the bat.

17:41 But this is a short introduction.

17:44 One of the things I like is at the end he talks about his conclusions with working with it.

17:49 And one of the conclusions he came up with is that it forced him to pin down his function specifications and really to consider special cases.

17:58 So really think about the interface to the function you're going to test.

18:01 What are the good parameters?

18:03 What is the expected behavior?

18:05 And what are the bad outputs?

18:07 And what do those look like?

18:08 Making you think about your interfaces is a good thing.

18:11 So if Hypothesis helps people think about interfaces, great.

18:14 Yeah, I think it's really...

18:16 Hypothesis is interesting.

18:17 I haven't had a chance to do a ton with it.

18:19 But basically, instead of choosing examples like, well, let's see what's an edge case if the register value is false and the email address looks like this and the price looks like that.

18:30 That seems like a good example.

18:32 Let's pass that to my test and see what happens, right?

18:34 So instead of doing that, you can go to Hypothesis.

18:37 Just write a regular test, but then add on to it this decorator that says, okay, that thing is like an email address.

18:43 That thing is a Boolean, and these are some numbers, here's their range.

18:47 Go after it.

18:49 And it'll just do a bunch of different examples and record which examples worked and which ones failed and things like that and store that little file.

18:56 And it's pretty cool.

18:58 It can find those edge cases and other things you might forget about.

19:01 And this example of kind of do it in an interactive way, like you're not really sure how you should test your...

19:07 I mean, you've written some tests, but you're not really sure what inputs to throw at it, which test cases.

19:13 And making you think about where the edges are and the different corner cases.

19:17 I think that's a good thing.

19:19 That is a good thing.

19:20 The edges and corner cases are a super important part of unit testing, I think.

19:23 Yeah, I'm still trying to figure out exactly what level of the development process and what level of testing this makes the most sense at.

19:32 But there's definitely algorithmic pieces in your code that might be a little confusing.

19:37 I don't think this would make sense to throw at every unit test in your system.

19:41 But there's definitely places where this would make sense.

19:44 Yeah, well, it's cool.

19:45 People should check it out.

19:46 And it's an approachable article for sure.

19:48 The last thing is one of these Python versus legacy Python things.

19:53 And chalk up one more win for Python.

19:56 So most people have heard of Heroku.

19:58 Heroku is a platform as a service cloud provider.

20:01 Kenneth Wrights works there, for example.

20:03 So his unofficial title is something to the effect of like Python overlord at Heroku.

20:09 That's like on his business card or something.

20:13 And so anyway, he and the crew there basically make it so you can say, here's my app.

20:19 And here's my requirements.txt.

20:21 Run this, please.

20:23 And until recently, the default has been when you say run this Python app.

20:29 It's like, cool, you mean 2.7, right?

20:30 And you could run it on Python 3.

20:33 But you had to like configure it explicitly.

20:36 If you said nothing, it ran on Python 2.

20:38 The big news is on, what's that, June 20th, 2017, Heroku is switching the default to Python 3.6.1.

20:48 Wow.

20:48 So hooray for Python 3.

20:50 So now if you go to Heroku and you say run this, it's going to be like, awesome, Python 3, right?

20:55 That's what you wanted.

20:55 And so this thing that I'm linking to basically links over or displays their blog post.

21:01 And their blog post is super short.

21:03 It talks about it.

21:04 It just says basically what I said.

21:06 Inactive Tuesday, the default runtime is now Python 3.6.1.

21:10 Yeah.

21:11 So if you've already got a job running there, it won't switch, right?

21:14 Exactly.

21:14 No.

21:15 It is only for new projects.

21:16 So in the Reddit thing, there's a few interesting quotes.

21:19 Somebody said lots of new projects start out on Heroku all the time.

21:23 So this is really great news for Python 3 adoption.

21:25 Someone else said Python 3 is really happening.

21:27 Yay.

21:28 I was actually a little worried about the future of Python for a while.

21:31 But I feel like it's all downhill from here.

21:33 Yeah.

21:34 Apparently people that don't listen to our podcast.

21:36 That's right.

21:36 Our listeners know better.

21:38 I mean, there's a lot of these examples, right?

21:40 We've got all the new frameworks that are exciting.

21:42 But we also have like Django 2 dropping support for Python 2.

21:46 And ironically, those numbers match up.

21:49 But the newest version of Django is only going to be Python 3 only and things like that.

21:53 It's really starting to pick up speed.

21:56 Yeah.

21:56 One of the comment there was interesting is that a lot of new projects start out on Heroku.

22:00 So it must be people starting out a project and then later grabbing different server solutions or something.

22:07 I haven't done a lot with Heroku, to be honest.

22:09 But I think it's really simple to basically just wire up a Git repository, do a push to it, and it'll just start running your app magically.

22:18 So it's really, really easy to get started.

22:20 And then maybe as you grow, maybe like costs become a concern or you just want more control or whatever.

22:25 But it's super easy to get started.

22:27 Okay.

22:28 And however you get started on whichever version of Python is probably where you're going to stay.

22:32 So that's good news.

22:34 Yeah.

22:34 Great.

22:35 Well, cool.

22:36 Yeah.

22:36 Yeah.

22:36 Very cool.

22:36 And that's it for the news, Brian.

22:38 You got anything else you want to share?

22:39 No.

22:40 So, wow, number 30 in the can almost.

22:42 30, yeah.

22:44 That's awesome.

22:44 I'm finishing up the last chapter this week, chapter 7 for Python testing.

22:50 So that's going to be done soon.

22:52 Yeah, yeah.

22:53 Very, very cool.

22:53 One of these days, the book will be a thing that you've done in the past instead of a constant job of yours.

23:01 Yeah, yeah.

23:02 And hopefully, I can't wait until it's an actual physical copy.

23:05 So it'll be good to have a stack of copies with that.

23:08 Yeah, that's awesome to hear.

23:09 You're making progress.

23:10 And so thanks for coming to the news with me.

23:12 How about you?

23:13 Do you have like now four months of podcasts ready?

23:16 I have about three months of podcasts that I've recorded.

23:20 I'm going to go on vacation for a while in the later half of the summer.

23:23 So I'm trying to make sure that everything is going to be smooth, no interruptions.

23:28 And so I have, I think, 13 or 14 episodes of Talk Python already recorded.

23:33 There's tons of interesting stuff I'm really looking forward to sharing.

23:36 I don't want to hold it back, but I've got to dole them out week over week or it won't solve the problem.

23:41 How about this?

23:42 And as for this podcast, if we, we haven't really decided yet, but if we do a break, we'll definitely let people know before that happens so that they're not just hanging out there waiting.

23:52 Yeah, absolutely.

23:53 We'll try to keep it rolling, but we might miss a week or two with some trips there.

23:59 Okay.

23:59 It is summer.

23:59 All right.

24:01 Well, thanks for sharing your news with everyone.

24:03 And thank you to Datadog.

24:05 Get your t-shirt, pythonbytes.fm/datadog.

24:08 Thanks, Brian.

24:09 See you next week.

24:10 Thank you.

24:10 Yep.

24:11 See you.

24:11 Thank you for listening to Python Bytes.

24:14 Follow the show on Twitter via at Python Bytes.

24:17 That's Python Bytes as in B-Y-T-E-S.

24:20 And get the full show notes at pythonbytes.fm.

24:23 If you have a news item you want featured, just visit pythonbytes.fm and send it our way.

24:27 We're always on the lookout for sharing something cool.

24:30 On behalf of myself and Brian Okken, this is Michael Kennedy.

24:34 Thank you for listening and sharing this podcast with your friends and colleagues.