Brought to you by Michael and Brian - take a Talk Python course or get Brian's pytest book


« Return to show page

Transcript for Episode #97:
Java goes paid

Recorded on Wednesday, Sep 26, 2018.

00:00 KENNEDY: Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is Episode 97, recorded September 26, 2018. I'm Michael Kennedy.

00:11 OKKEN: And I'm Brian Okken.

00:12 KENNEDY: Hey Brian, how you doing today?

00:13 OKKEN: Hey, I'm doing great.

00:13 KENNEDY: Awesome, awesome, me too. Lot's of big news, so much news that I can't even keep up with it.

00:19 OKKEN: Yeah it's a good thing there's like this weekly podcast that keeps us up on Python news.

00:23 KENNEDY: I know, where's the one that we get to listen to? That's what we need, we need a meta podcast, okay.

00:29 OKKEN: No, that's okay, other people don't, don't do that.

00:32 KENNEDY: Yeah, that's right. So I also want to say thank you to DataDog for sponsoring this episode. Really happy to have them on board. Check them out at pytonbytes.fm/datadog. A cool T-shirt lives at that URL. More to say about that later. PyPI just keeps getting better and better, doesn't it?

00:48 OKKEN: Yeah, PyPI keeps getting better and the tools around it, and a recent release, just a couple days ago, of Twine, so, Twine has been updated. For people that don't know, Twine is the thing that pushes your code from your laptop or your computer or your sever or wherever, to PyPI, and Twine has a check function now. Actually, there's quite few things in the new Twine upgrade, but this one's neat because it, once you've packaged your distribution up and everything in your wheels, and whatever, it usually shows up in a dist folder, and then you say check dist/* and then everything in it, and it checks to make sure that one of the things it does, I think it does a lot of the things, but at the very least, one of the things it does is it makes sure your readme is rendering properly, supposedly.

01:42 KENNEDY: Yeah, that's really cool, that it even checks that, right? Cause that's hard to test, right? How do you know if your readme's not rendering properly?

01:50 OKKEN: Yeah, and we're linking to an article that's from, on the PyPA, that is, how to make a PyPI friendly readme. So there's actually, it's not really difficult, but it's one of those things that, it's one of the last things you do before you, well, it shouldn't be the last thing, but when you're getting a package together, you're like, hey, I have this really cool idea I want to push it up on PyPI. You got to make sure the readmes all work, the readme works, and this is one of those steps, and there's just a little short article on how to do that right. Although, like magic, it's not difficult, but the stuff you have to put in your setup.py and everything.

02:27 KENNEDY: Yeah, that's a good find. Anyone who is either maintaining or thinking of creating a PyPI package should definitely check it out.

02:34 OKKEN: Yeah, and I think in your build tool chain or your talks tool chain to add a check step to make sure, go ahead and check before pushing it up is, that would be a good thing to do.

02:44 KENNEDY: That can be like a pre-commit hook or something.

02:46 OKKEN: Wow!

02:46 KENNEDY: I don't know. It definitely, put some part of your continuous integration, anyway.

02:50 OKKEN: Yeah, definitely part of your CI.

02:51 KENNEDY: The first item I have is pretty interesting, and, when you think of the languages and technologies that you can use, most of the time, we think of programming languages and their subsequent runtimes, basically being free, right? I mean, obviously Python, like, you don't think, well, what does CPython cost, but even from places where you might not necessarily expect it, right? Like, C++, it's free, Go is free, Microsoft .NET is free, right, all these things. You know what's not free these days?

03:25 OKKEN: What?

03:25 KENNEDY: Java.

03:26 OKKEN: Really?

03:26 KENNEDY: So, apparently, Sun has made the Java runtime for, you're going to have to just check out the article on the register to make sure, hey, I get this exactly right, but roughly, Java is making the runtime a subscription, and the subscription costs $25 per processor per month for commercial entities.

03:48 OKKEN: Wow!

03:51 KENNEDY: Is that crazy? There's definitely some cases, like, as students and whatnot where you can just use Java for free, but a lot of commercial organizations that depend upon Java need to read this article and the subsequently linked 330-page document that describes this process, over at Oracle. So that's pretty interesting, right?

04:11 OKKEN: Yeah, it's the runtime's even, so you have to, like, if somebody else makes a Java application, it's not the person making it, it's the person running it?

04:20 KENNEDY: Yeah, so it says, Oracle's put a price on the Java SE and its subsequent support, so, $25 per processor per month, plus a $2.50 per user per month on the desktop, unless you buy a whole bunch.

04:34 OKKEN: Yeah, so that looks like, maybe, $2.50 a, wow.

04:37 KENNEDY: Yeah, it's kind of all convoluted, but it's, yeah, it's quite interesting. They say that the older runtimes, you can still use them for free, but, like, who wants to use a runtime that has no bug fixes, no security patches, or things like that, right?

04:53 OKKEN: Yeah, I guess maybe they're trying to make everybody move to Python?

04:58 KENNEDY: Yeah, that's exactly, that was my first thought is like, yeah, Java's been pretty high in the language rankings, maybe it's going to plummet now. I mean, seriously, this is pretty crazy. So, yeah, it says also, there's a little bit of a stick to go with the carrot, yeah, come January 2019, Java SE 8 on the desktop won't be updated anymore unless you buy a subscription. And yeah, just, the short version is, every commercial enterprise needs to look at their Java Standard Edition usage to see if they have something to do with licensing now. So this was sent in by one of our listeners and I am super sorry I didn't write down the name in the show notes, but thank you for sending it in. This is interesting. I know it's not directly a Python thing, but it's just, you know, here's a thing, I kind of always sort of thought of Java as more or less open source, and I guess it's just, you know, that Oracle oversaw, but I guess that's not the case.

05:47 OKKEN: Yeah, weird, okay.

05:49 KENNEDY: Yeah, anyway, there's that.

05:51 OKKEN: Yeah, well, this next topic is really really important.

05:54 KENNEDY: For importing?

05:54 OKKEN: Importing, yes.

05:57 KENNEDY: It's very very importing. Tell us about it.

06:00 OKKEN: Import, yeah. Absolute versus relative imports, in Python. This is an article on the Real Python, I think it's on Real Python, realpython.com. There's a lot of great articles there.

06:11 KENNEDY: Yeah, doing a great job over there.

06:12 OKKEN: I like what they're doing.

06:12 KENNEDY: Mm-hm, me too.

06:14 OKKEN: The relative, and I lost my tab, but the, most of your imports are going to be, the first part of the article is talking about normal kinds of imports, like, just, you know, you import a package, it's the import statement, or you do from package import, something, or from package dot some sub-package, there's a lot of versions of that, so they're, it's covering that, but there's, like, for instance, there's relative imports, which includes dots, so, like from dot import something, or, import dot something, and I never, I kind of never remember how all that works, so I'm glad that somebody wrote it up. I know I could probably find this other places, but this is a nice writeup of what all this means, so in general, people say, the recommendation is, don't do this, don't do relative imports, but sometimes you have to, and there's a lot of good, good times, so for instance, if you're a package maintainer, or you're developing a package, the modules within the package should be able to use functions from other modules, that's why you have them, but how do you import that? You don't just, and one of the ways is, you import from dot neighbor module, import something, and that's what the single dot is, is, import a module from the same package you were already at, relatively.

07:33 KENNEDY: Yeah, that's cool. You know, the other thing that bugs me about when you have a package and sub-packages that all interact and you use the full or external import style, if you want to rename that package, you're like, oh, I kind of like would rather call it this. All the sudden, there's like lots of pain and suffering that happens.

07:51 OKKEN: Yeah. Well, even within it, if you import it from the top, yeah, you have to go through and change it, I don't know.

07:58 KENNEDY: Yeah, exactly. It's not something I see very much, but I think it's a great article to have it all written up.

08:02 OKKEN: Yeah, well, I use the dot dot, I guess it's a parent package thing. I don't really use that, but the single dot, I use all the time to just say, the import something from the same package I'm in, to keep, that way, like you said, you don't have to, if you decide on a different name for your package, you don't need to change that.

08:21 KENNEDY: Yeah absolutely, very nice.

08:21 OKKEN: So, this is a good writeup.

08:23 KENNEDY: Yeah, so, before we get to the next one, which is kind of retro and fun, let me tell you quickly about DataDog. If you're not familiar with them, you should check them out. They're especially good for distributed systems, microservices, things like that, so they're a monitoring platform that brings together metrics, logs, request traces, all in one place. DataDog has a trace as search and analytics feature that lets you break down your, actually your Python app performance by user, by URL, all sorts of cool stuff like that, so you can quickly, easily troubleshoot and optimize your Python app, so try that out today with a free trial, and DataDog will send you a free, cool, dog T-shirt. It's very nice, check them out at pythonbytes.fm/datadog to get started.

09:06 OKKEN: The user ID thing, searching on user is kind of cool. You could maybe line that up, if like that particular user had a complaint or something, you could maybe use that.

09:16 KENNEDY: Exactly, I get this all the time, like, oh, this thing is not working super well on your platform. I'm like, it seems to be working pretty well. I just don't understand, but if I could actually dig into it by user ID, that'd be awesome, so definitely, yeah. So I promised you something retro, right?

09:33 OKKEN: Yeah, I love retro, being that I'm retro.

09:35 KENNEDY: Quite retro. So, you should check out, if you feel like making a game, you know, a lot of people get into programming cause they were interested in making games, and that's why they got into programming. That was not me, actually, like, I enjoyed playing games when I was younger. I sometimes still do, but I wasn't, like, oh, I have to go into computers so I can make a game. I know a lot of people did, but the reality of make a game now is, get 100 of your highly-skilled developers and artist friends together and work for a year, which doesn't make a lot of sense, right? So you might make like a cool 2D scroller retro thing, and this thing called Pyxel, P Y X E L, like pixel, Python, pixel, sort of thing, is a retro game engine for Python.

10:15 OKKEN: How fun!

10:15 KENNEDY: So the idea is, you only get 16 colors, you get four sounds that can be played at the same time. It's a Python 3 run, game platform. It runs on all the major platforms, Windows, macOS, and Linux. It even comes with a bunch of cool examples, so if you pip install it, then you can just run install Pyxel examples and then you can go and explore them, but what do you think, looking at some of the screenshots there?

10:39 OKKEN: It looks kind of fun. I got to try to check this out.

10:43 KENNEDY: Yeah, you don't have to be like massively skilled to create the graphics, cause you only get like a few colors, you only get like a certain size. It's not, you know, you don't have to be crazy artistic, game studio type place to make this.

10:56 OKKEN: Yeah, these are the types of video games that I grew up on, so, that's nice.

11:00 KENNEDY: Yeah, I think people might enjoy this, and like I said, I think it makes it pretty accessible. It's an interesting way to bring UI capabilities to Python for people who might want to, like, if you were doing some sort of programming class or you're doing like a school club or an hour of code type of thing, an hour of code's probably too short, but you know, those types of projects, where you want to let, especially kids, who are impatient and don't care that much for terminal apps and are not ready for web apps, like, they could build a fun little thing if you gave them this as a foundation.

11:31 OKKEN: Yeah, neat.

11:31 KENNEDY: Yeah, so, cool, yeah. If people are into retro games, check this out. You can create your own with Python.

11:37 OKKEN: What if I don't want to create a game, I just want to create a command line interface?

11:42 KENNEDY: You mean like most of the apps, yeah. So, that's, you can even make your game have command line interfaces. Yeah, so there's a couple of options, right? You could use sys.argv, that'd be pretty awesome. That's pretty accessible. You could also use higher level things, like argparse or Click, right?

12:00 OKKEN: Or Click, yeah, so.

12:00 KENNEDY: Yeah, what's up with Click?

12:02 OKKEN: I know there's a lot of great options, and sometimes you just reach for what you know, and since I'd written a few command line interfaces from the sys args and not enjoyed it, I have enjoyed working with Click, and so, Click's been around for a while, but, the news, really, I wanted to highlight it because it's cool that the Click version 7 was released recently, and there's a huge list of, in their change log, of all the stuff they fixed, and normally, I mean, normally we don't just cover a version update of something just for the heck of it, cause things update all the time, except for Click doesn't update all the time. The last version of Click was released January of 2017, so, that's neat that we have a new one, so there's all sorts of fun stuff in here. I don't know what I'm really excited about. There's Z shell auto-completion support that was added, that's kind of neat.

12:55 KENNEDY: That is quite cool. I use Oh My Zsh, which I love, and this is really exciting for me, cause it's based on Z shell.

13:03 OKKEN: One of the other things is, Click interfaces have, they include a help option, you can do --help, and it tells you what the options are for your command line interface, and how to call it, you have to build that sort of stuff in, but it's pretty easy to write up. But one of the things they've added is that, if you do it wrong, it doesn't tell you, it just says, this is wrong, or something. It doesn't tell you, it doesn't point to the help, but now it does, so in this version, any usage errors will now hint at using the help option, which is cool.

13:31 KENNEDY: Yeah, yeah, I like it.

13:33 OKKEN: And actually, there's pages, and pages and pages of updates. There's a lot of stuff that changed, so, good job, folks at, and I didn't realize it was part of the Pallets project, which is also Flask, the people that worked on Flask.

13:45 KENNEDY: Yeah, yeah, it's part of Pallets, and you know, you talked about how it hasn't been updated in a while and this is a major release, like, Flask just went 1.0, and we talked about that a little while ago, as well, that's pretty awesome.

13:55 OKKEN: Yeah, and I know that the Pallets people are, they take donations for if you want to, if you're utilizing this and want to help out, they would love to have your money.

14:04 KENNEDY: They would, they would. They have a special arrangement with the PSF, so it's sort of a non-profit thing as well somehow I think.

14:11 OKKEN: Yeah, and nice people, too.

14:11 KENNEDY: Yep, absolutely. So it's cool, I'm glad you highlighted it. I want to understand the Zsh auto-complete a little bit more, that's sweet. So, Brian, do you use the cloud much?

14:22 OKKEN: I think so, well, we live in Oregon, so we get lots of clouds.

14:25 KENNEDY: Well, yeah, we, like, definitely use the cloud at least half the year. No, so.

14:31 OKKEN: That was a bad, really bad joke. They don't get much better than that, so.

14:34 KENNEDY: It's a pretty good one, yeah, I like it. So, one of the challenges of, especially like, like platform as a service and things that are not infrastructure but other stuff, database as a service and other higher order things, is that you often pay, like, as you use them, right, and like, if I use a virtual machine, I turn on the virtual machine, I know what it costs, right, like, I go to Digital Ocean, fire that thing up, I know like the server that runs Python Bytes, plus the database server, they cost $15 a month together, right, I know it's $15, pretty much, other than bandwidth, which is mostly covered. If you use something like, say, Firebase, or AWS Lambda or other things, like, you pay as you use it, and generally, that's to your advantage.

15:18 OKKEN: Right, but the you isn't really you. The you is your users are using it.

15:22 KENNEDY: Right, and you don't necessarily know how much your users are going to use it. I mean, it's a good problem to have, like, I have lots of users, they're using my app a lot, so I'm using my cloud service a lot, but it's super, super scaling, and it turns out that if you program badly, you may have a higher bill than you otherwise expected. So here's an example which I think is just maybe a cautionary tale for everyone using clouds and it doesn't really too much depend on, like, what cloud you're using, anything that, where the price depends upon user consumption, I have a real world example from myself, I'll tell you about at the end. But the article I want to cover here is, the title is, how we spent $30,000 in Firebase in less than three days.

16:05 OKKEN: Wow!

16:08 KENNEDY: And Firebase is like a database added service exposed over like a HTTP endpoint for Javascript, type of thing, like, that's pretty much my understanding of it. I don't really use it, but, it's like if I'm doing, say, an Angular JS or some kind of front end Javascript app, I can like just connect to Firebase straight from my client-side code, and then, then we're good. It can do the various things that it needs to do, right? So there's this place called Vaki, Vaki, something like that in Colombia, and it's like, kind of like Kickstarter, but Colombian, and they had just launched and it's turned out that they had this project on there that became the largest crowdfunding campaign in Colombia by like three, collecting three times as much money as anyone ever had, in the first two days, so they collected tons and tons of money. They had like two million sessions, like, 20 million page views, 15,000 backings or whatever you call that, sounds good, right? Well, it turns out the site was running really slow, and the folks who built it, they were somewhat new developers, they were like, ah, I don't know what to do to make this work, I know, let's just upgrade all the frameworks, we're using Angular 4, let's upgrade to Angular 6 or something to this effect, it wasn't Angular, maybe it was, I don't know, whatever framework they have, they're just like, we're going to upgrade these things. Nope, that didn't fix it, right? So they spent like a day like poking around, and it was still running slow and it kept slowing down more and more, and they couldn't figure out what's going on and they logged into their Firebase account and they saw their bill was like $30,000. It was costing $600 an hour more every hour.

17:36 OKKEN: Oh, wow, yikes.

17:37 KENNEDY: They're like, what is going on? So it was like, you know, like Firebase, you pay on like the number of calls and type of thing, so it turned out that they had this little part of the site just like on Kickstarter, in the right, it says there's this many contributions and currently this much money has been pledged towards this account, and I think what happened was, their implementation of showing that number, it didn't, like, sum it in the database, it called, it like, sucked down all the payments table, and then like added it in Javascript, and it kept doing that to like keep it live or something.

18:07 OKKEN: Yeah, every time somebody goes and looks, oh yeah.

18:09 KENNEDY: Yeah, exactly, they made like 40 billion requests, billion with a B, to their, like, database, and that got them like a $30,000 bill.

18:19 OKKEN: Yikes.

18:20 KENNEDY: So, so this, it's pretty interesting, worth checking out, this article. The comments are super interesting. Some of them are super supportive, and others are kind of mean. You probably wouldn't imagine this, but there's a theme of testing, they should have tested better before they rolled out the site.

18:34 OKKEN: Well, yeah, but you can't test everything and foresee.

18:37 KENNEDY: Yeah, you can't foresee everything, that's true.

18:39 OKKEN: All right.

18:39 KENNEDY: And I'm sure it seemed fine when you tested it with like the 10 purchases they had in their little test project, but, you know, 15,000, all of a sudden, it starts to behave badly.

18:48 OKKEN: Yeah, and I heard about somebody doing some things, similar, not a success story, really, just somebody saying that they would run, kind of some abuse testing against a local database, or local mock service, and then, set it all up, and they were going to just see how it did for abuse testing over the weekend, got back and realized that they were not testing, abuse testing in the lab, they were abuse testing the live service, and, yeah, similar issue, a big huge AWS bill because of a test running all the time.

19:23 KENNEDY: Auto scale is amazing, it just takes care of all all the traffic you have. Right, what? How much does this cost? So, I was thinking about this article a few weeks ago and thinking we might cover it on this episode, and I thought, ah, good thing this doesn't happen to me. Never do this. Today, today, today, remember, I don't know if I told the audience this, but I know I told you this, when we were hanging out at PyCon last year, like, for my courses, I set up a whole bunch of different videos, servers throughout the world, so I've got one in Mumbai, one in Tokyo, one in Sydney, one in Frankfurt, and so on, and I got tired of synchronizing the files, so I wrote this system that would automatically sync them across the world to these different locations. Well, it turned out, there's a small find and replace that has to be done to adjust the location from the source to the destination servers, and I logged into that thing today, and it was still synchronizing, just continuously. I'm like, oh no. This server's been continuously downloading and uploading to S3 as fast as it can, for two weeks.

20:25 OKKEN: Oh no!

20:27 KENNEDY: Like, what is my AWS bill? And I logged in, and it wasn't so bad. It was like $300 more than normal, which is okay, but, I'm just like, yeah, that pretty much is exactly, just a less bad version of what happened to these guys. And it was a bad file replacement. Basically, sync, the word sync was being replaced in some circumstances, and I have a new course called Async, and it was taking out the sync part and so it just kept going oh, file's not there, we got to upload it, file's not there, we got to upload it, you know, cause they just, they mismatched. Yeah, that was bad news.

21:00 OKKEN: Oh yeah.

21:00 KENNEDY: Anyway. So, it can happen to anyone. It happened to me this morning. I fixed it, by the way.

21:08 OKKEN: Well, yeah, if you're paying for any of those services, check once in a while to make sure that.

21:12 KENNEDY: Yeah, and do you know what, I actually, I have, that's kind of one of the takeaways, both from the article and from why I wanted to say that, is I have an alarm set at AWS, and it says, if my bill goes over this amount, send me an email. If my bill goes over this amount, send me an email, and I woke up and I saw that alarm as well. I'm like, oh, that's not good. So, anyway, it's all fine, but check it out. It's definitely worth thinking about it, if you do anything that is in the cloud, that is based on consumption, by users.

21:39 OKKEN: Yeah, definitely.

21:39 KENNEDY: All right, well, that's our main items, I have a really exciting one I wanted to get in, but actually, I just learned about it 15 minutes before we started recording, so I didn't have time to dig into it, so the news is, Dropbox, Dropbox, one of the biggest users of Python, as we know, and they have been stuck on Python 2 for what seems like eternity, which is so ironic cause Guido van Rossum works there and some of the other core developers that work there, it's like, yeah, Python 2 all day, you know, got to love it, right? So the big news is, as of yesterday, Dropbox has now upgraded to Python 3, and they have like a nice article writeup on it. We'll cover it in more depth next week, but I'm going to link to it now since it's timely and important.

22:19 OKKEN: That's so cool, though, yeah, very neat.

22:21 KENNEDY: Yeah, I know, what about conferences?

22:23 OKKEN: Well, we just got an announcement, I think I just saw this today, that the call for proposals for the 2019 PyCon is now open.

22:30 KENNEDY: Oh, really? That's pretty awesome.

22:31 OKKEN: So you can get started on your proposals for next year.

22:37 KENNEDY: Yeah, I'm thinking about those for sure. I definitely have some stuff I want to propose, so thanks. See, I learn about stuff right here on the show. All I got to do is just sit here.

22:44 OKKEN: Yeah, and the PyCascades is still open, too, so.

22:47 KENNEDY: Yeah, til October.

22:48 OKKEN: Yeah, so it's proposal time, I guess.

22:50 KENNEDY: It is definitely proposal time. Also, PyTennessee, I can't go, but I know that they have their call for proposals open as well, so there's lots of conferences. I, you know, conferences are a spring-summer type thing a lot of times, and so the call for proposals are sort of now.

23:03 OKKEN: Yeah, I mean, you know, why not spend your holiday season coming up with things to...

23:09 KENNEDY: Yeah, that's what I'm doing on Thanksgiving for sure. Maybe Halloween as well, my kids will love it. You know what's scary? This talk's going to be scary. No, Dad, I want to get candy! This is even better than candy.

23:19 OKKEN: No, if you wait til Halloween, you'll miss the PyCascades. You got to do that before.

23:22 KENNEDY: That's true, yeah, it's true, I can't do that, all right. No, I won't wait.

23:26 OKKEN: You got one more for us?

23:28 KENNEDY: I got one more, so, I just released, like literally an hour and 15 minutes ago or so, released a new course called Async Techniques and Examples in Python, and this one is something I'm super excited about, I've been wanting, I've been wanting this course to exist ever since I started in Python, and I've decided that I'm going to put in the time to create it, and so I just launched a five-hour course on all of the async things in Python.

23:52 OKKEN: Oh, that's very exciting, cause I want to learn all the async things in Python, so I'll have to check that out.

23:57 KENNEDY: I have a resource to recommend, yeah, so I put the link in the show notes. People can check it out at training.talkpython.fm. Should be in the course list.

23:57 OKKEN: Nice, cool.

23:57 KENNEDY: Yep, definitely. All right, Brian, thanks for being here and sharing this with everyone.

23:57 OKKEN: Yeah, thank you.

23:57 KENNEDY: You bet, bye.

23:57 OKKEN: Bye.

23:57 KENNEDY: Thank you for listening to Python Bytes. Follow the show on Twitter via @PythonBytes, that's Python Bytes as in B Y T E S, and get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Okken, this is Michael Kennedy, thank you for listening and sharing this podcast with your friends and colleagues.

Back to show page