« Return to show page
Transcript for Episode #137:
Advanced Python testing and big-time diffs
00:00 KENNEDY: Hello, and welcome to Python Bytes where we deliver Python news and headlines directly to your earbuds. This is Episode 137, recorded June 26th, 2019. I'm Michael Kennedy.
00:11 OKKEN: And I'm Brian Okken.
00:11 KENNEDY: And this episode is brought to you by Rollbar. I'll tell you all more about them later. For now Brian, I always wonder about, you hear that Python is an efficient and expressive language and if you write code in C++ it'll be a lot longer but how can you quantify that?
00:25 OKKEN: Well, you can set up a whole bunch of people to write the same thing in a whole bunch of languages.
00:31 KENNEDY: Well, that's awesome that people did that. It seems like a lot of work, but yeah, I guess that's cool. Tell us about it. Like this is your first time, right?
00:38 OKKEN: So this an article called Comparing the Same Project in Rust, Haskell, C++, Python, Scala, and, how do you pronounce that? Oh camel? O-C-A-M-L.
00:49 KENNEDY: OCaml I think, yeah.
00:50 OKKEN: So this was written up by Christian or Kristen Hum, or Hume, and this is about a university project, which is kind of a neat project. Basically, they had to implement like a big chunk of Java so it's a Java to x86 compiler, as part of a compiler class, and they were basically had to get up teams of people to do it and they could pick any language they wanted, which is kind of cool because, you know, people will be better at different languages to let them use what they're good at.
01:22 KENNEDY: Yeah, let them use what they're good at 'cause then they'll do it properly. Not just try to cram it. They have the most efficient use of that language, for sure.
01:30 OKKEN: Up to three people on a team and it was a multi-month project. And then also, tests ran. So this is kind of a neat part of the process, which I think is an awesome way to teach people is have some published tests of, you're going to have to run these test cases and they have to pass. But then also have some secret ones where people don't, they don't know what tests are going to be tested against it, which is kind of nice because people will have to be able to make sure their implementation's robust without knowing the test cases. It's kind of neat.
02:02 KENNEDY: Yeah, that's cool. I do love it that there's unknown tests. Like, these are the specifications, you can kind of get close with these tests, but to pass, you actually have to just work.
02:12 OKKEN: That's totally like real life, you know? Be allowed to write down some specifications and there's some specifications that are not written down, they're just supposed to be known. And then there's other things that people once they see the implementation, they go, "Oh yeah, I wish it did this also." So I think that's a cool idea. And they weren't shooting for lines of code or anything complex. They were just tryin' to finish the project. So this analysis was done after. They had a baseline implementation written by two people that were familiar with Rust and then they compared everything against that. So there was another Rust team that chose different design decisions and they had like three times the code so these are all just comparing lines of code. The Haskell implementation was about equal, but depending on how you measured, one to 1.6 times the code. Same for the OCaml. C++ was bigger by about 1.4 times the baseline. Scala was a little bit less with about 70% lines of code. The big outlier was Python, which had a lot of standouts. Python implementation was at half the size approximately, plus written by one person, and had extra features, passed all the secret tests plus others. Somebody excellent at programming, of course, used some of the metaprogramming techniques, and anyway, kind of a fun article. One of these, I forgot to mention, the hindrances was they were only supposed to use standard libraries, no extra, and then not any parsing libraries, even if they were part of the standard library. So even the parsing had to be kind of built up from scratch.
03:59 KENNEDY: Yeah, how interesting. I wonder if that would make things like Python even better possibly. I don't know about Rust, maybe as well. But C++ doesn't have parsing libraries built in that I know of. Things like that, right?
04:09 OKKEN: There's a lot of mini-language parsing libraries around Python, so it'd be interesting to do that with, you know, go wild and use whatever's available.
04:18 KENNEDY: Right, like maybe take this project and then go, "All right, well what if we hit it with all of the pip install-able things?" What happens then, right?
04:26 OKKEN: Yeah, exactly.
04:27 KENNEDY: Yeah it sounds like a super intense project though, right? Like deep, deep into the language, right? I mean, on one, you're writing the compiler, you're understanding Java, you're compiling to x86. You're doing metaprogramming. Like there's a lot of stuff going on here.
04:39 OKKEN: It's a pretty cool article.
04:40 KENNEDY: Cool. If that last one really connected with like your deep geek out, let's go like really hard into language, this next one is going to connect with you, or I just wanted to work really quick and easy. So this one is really nice. So if I was a data scientist, I might use Matplotlib, or just any kind of person who wanted some visualization of data. I might use Matplotlib for that and that's great, except for, at least I personally can't make Matplotlib look super good, right? Like if I used Excel, I could put the data in there and I highlight the stuff and I would say, "OK, insert chart." and I would pick the kind then I would go and I would right-click and edit the chart and I would maybe drag it around to size it correctly. Double-click on the axes to change the axes. But in Matplotlib, you just write code and the picture comes out, right?
05:24 OKKEN: Yeah, and I know you can do all this stuff, but it's not obvious and you have to look every little thing up and tweak it.
05:31 KENNEDY: Yeah, so there's this project that we heard about from one of our listeners. I can't remember. I'm trying to remember who it was. Oh, here it is. This is from Lee Wagner. So thank you, Lee, for sending this in because this is killer. So there's this project called Pylustrator for styling your Matplotlib plot. So you just do your Matplotlib code, but you import Pylustrator, and you say, "Start at the beginning." And what it does is it pops up when you show your plot an interactive thing much like Excel where you can drag and drop and arrange your different plots. You can go to the properties and edit the axes and the colors and just all the kind of stuff that you might do. It even has like the cool design layout stuff where like it'll help you equally space stuff between each other. It will put those little bars to say right there. If you drag and drop it, it will be equally spaced or the lines on the tops and the side.
06:28 OKKEN: Yeah, and with that, the start thing, you can even fill it with some of your data to begin with. So if you kind of know the data you want to plot, because that's going to affect how you're going to design it. So if you pre-fill it then drag it around and design it. It's just totally cool.
06:45 KENNEDY: It's totally cool.
06:46 OKKEN: I'm glad they have. So the link we're going to show has a little embedded video. I mean, talking about it, you're like, "Yeah, I think this might be useful." But you watch this video and you're like, "Oh, my God, I need to use this right away."
06:58 KENNEDY: Yes, I had the exact same experience. I'm like, "Eh, kind of interesting. Oh, look at the video. Oh, my God, it's amazing." So this is super cool. And obviously, you don't save your changes to like an Excel workbook. What you do is you save your changes and you can actually call 'save' in Pylustrator. What it'll do is it'll put the configuration in Python back into the file that ran it. So that's pretty wild actually.
07:22 OKKEN: Yeah, then you un-comment the Pylustrator. You don't have to import it later because it's not a dependency on your project afterwards.
07:29 KENNEDY: Right, it's just a little design tool. So, it's super cool. If anyone's doing Matplotlib and they want to have it styled, especially if you're doing more than one plot and you want to put them side-by-side, this is super cool, so check that out. Definitely a fan. Another thing I'm a fan of, Brian, MongoDB. Love it.
07:44 OKKEN: Since you and I are paying attention to a lot of projects, there's a lot of different release cycles and we kind of decided early on that we weren't going to try to track everybody's releases because that might get boring to people. However, we covered MongoDB 4 because it came out with...
08:00 KENNEDY: Transactions.
08:01 OKKEN: Transactions, which was
08:01 KENNEDY: Among other things.
08:02 OKKEN: A big thing, and 4.2 is out, and I'm kind of excited about couple features that it came out with. So the transactions are there, but now they're multi-doc, they're distributed transactions, so they're transactions that cross sharded clusters and replica sets, and that's just really cool.
08:23 KENNEDY: Yeah, that's super cool. Yeah, I mean, you could use that cool transactional stuff before, but you were kind of limited. And now it's like, no matter what crazy cluster, or scaling and charting, or replication you have setup, and you just do a transaction and it's all good. Pretty cool.
08:36 OKKEN: They're a good idea anyway, but with testing, you can setup a complex database full of stuff, and then at the beginning of your test start a transaction and then after your test, and you roll this into a fixture. You can just roll back and your next test has the same data, saves times, so that's cool.
08:54 KENNEDY: Yeah, and it's probably got isolation if for some reason they ran in parallel or whatever. Yeah it's really cool.
08:59 OKKEN: Yeah. The other feature that's pretty amazing is the field-level encryption, and this is encryption done on per-field basis on the client-side, so the server doesn't even have, it's not doing it on the server, so system administration can be done without having to make sure everybody signs NDAs and all that stuff, that you can just manage your database without even being exposed any of the secret stuff.
09:29 KENNEDY: Yeah, that's awesome. Most databases, most of what's in them is not sensitive, but there's often a little bit that is that's really you don't want anyone to get access to, and yeah, this is really cool, because like you said it's done in the library that talks to MongoDB, so in PyMongo, for the Python folks, and you just set the encryption, or decryption key, over there, and the server cannot decrypt it, so if somebody breaks into the server, or you lose it, or it's like you set it up on a cloud for testing, and you forget that it's there. All the kind of random stuff that happens to databases, it doesn't matter in terms of this encrypted stuff, because the database doesn't know how to read it, it's the drivers, on the client side, that have the keys.
10:13 OKKEN: So with GDPR stuff, if the customer says, "Hey, delete my stuff." that's always been an issue with databases, it might in a whole bunch of tables, but if you destroy the customer key, the data might still be there, but it's unreadable to anybody, so it may as well be garbage.
10:31 KENNEDY: Absolutely, and it's gets to be really tricky, because even if you set up the right code to delete all the customer data out of your database, what about the backup? That somebody made when an older admin was hired, and they stored that in the S3 buckets, so it was offsite. How do you delete the data out of there? You know what I mean? But if it's encrypted, then you can just throw the encryption key, and then it's just gobbledygook.
10:59 OKKEN: Yeah, cool.
10:59 KENNEDY: Pretty cool. I like it. Speaking of cool, Rollbar, happy to have them come along and sponsor the show. We use Rollbar on pythonbytes.fm, among other things, so if anything goes wrong, and it's kind of fortuitous, I guess. I woke up this morning with a ton of Rollbar messages, because there's a datacenter failure that caused some connectivity between MongoDB and pythonbytes.fm. How 'about that for a funny thing? So, some network card broke, right? The site couldn't talk to the database server, so it was freaking out. How do I know? Nobody complained to me, although they probably should. Like, "Michael, your site's down, what's goin' on? It's really messed up." But I just open up my email, I'm like, "Whoa, there's a lot of Rollbar stuff goin' on here." So if you want to be notified right away, even when users don't tell you, check them out. They have a free tier, they have some great paid tiers. Visit pythonbytes.fm/rollbar, super easy to integrate into Python, into the web frameworks. They've just got one or two lines you enter, maybe a little configuration. A few settings, off you go. It's really, really nice, so check them out, pythonbytes.fm/rollbar.
12:08 OKKEN: Nice
12:08 KENNEDY: So, kind of like Pylustrator, that's sounds kind of useful and interesting. This next one also sounds useful and interesting, but like Pylustrator, it's as you into it, you're like, "Whoa, this thing does a lot, man, look at it go." So, there's this project that was recommended by Francois Leblanc. Thank you for that, Francois. And it's called Deep Difference. That's a lie, it's just called DeepDiff. And so it does deep differences and search of any Python object graph. So I've got an object which holds a list. That list points to a bunch of objects. Those have other pointers, I want to know is this thing somehow referenced by that. Let me do a search on it. Where is it? Is this giant, crazy data structure same or different than other giant, crazy data structure? And you could compare them, so that's pretty cool. So there's DeepDiff, there's DeepSearch, and then it also has DeepHash. So if I've got some giant, crazy data structure, you would like to know that if the data is the same across two of those, that the hash result is identical, and if any part of the data changes that the hash then changes.
13:15 OKKEN: Oh yeah.
13:15 KENNEDY: Possibly, right? So, it will do that on object graphs that are not even hashable themselves.
13:20 OKKEN: Really? Wow.
13:21 KENNEDY: Yeah. So that's pretty wild. Ah yeah, just a lot of nice touches in here that kind of made me realize, like, "Wow, this is wild." So, for example, it'll get me the differences in a list, ignoring order and duplicates, right? Just, what is the essence
13:37 OKKEN: Oh wow.
13:38 KENNEDY: of this data? Or you can say "Is any data repeated in this list, or in this dictionary?" or something like that. You can exclude certain types. Maybe I want to know the data's the same, but they're both using a Thread object, and the Thread object is different, so of course they're going to different, but say "Don't check on the Thread object. Just check the other stuff." So you can explicitly opt-in or out data types that you might use. You can say "I'd like to compare these things", but only to four significant digits, because I computed them slightly differently, and maybe I can't get them to their decimal accuracy to be exactly the same, just the way they're done. You can exclude parts of your object tree that you got for comparing, I mean, this is insane.
14:22 OKKEN: They being able to do significant digits in a deep data structure, that's amazing, that's really cool for the all the stuff I work with.
14:31 KENNEDY: Yeah, I can imagine exactly. And you know what? I bet this would be really good to mix in with testing. You could create your test, and you deep-diff it against the result.
14:39 OKKEN: Yeah, exactly, 'cause there may be noise in the system and you know some of the signals are noisy, so yeah, this is awesome. Cool.
14:47 KENNEDY: It's super simple, but yeah it's pretty cool. So if that sounds like problems you're tryin' to solve, it sounds like you are Brian, I think it's definitely worth having a look at it.
14:54 OKKEN: Yeah, thanks.
14:55 KENNEDY: Yep, you bet. See, we just do this podcast to help each other out, and people can listen in. Speaking of testing.
15:02 OKKEN: Josh Peak is somebody that we I'm sure we met him before at a previous PyCon, but he stopped by at PyCon this last year, and met us, and really great guy. He wrote this great article called Advanced Python Testing, and it's kind of incredible. He goes through his, he got it in a situation at work, where he was asked to do complex task, where he had to he knew that testing and making sure that he was doing things properly would endue good coding practices, would help the entire process and make it go smoothly. So this is sort of a start-to-finish summary of it, but it's not that long of a read, but he talks about his learning journey, which he includes some great podcasts, including ours. Also an awesome book on testing, and I know the author for that one. Not just pluggin' our own stuff, he's got some great stuff in here. He talks about, he's starts off with just a basic, for people new to testing, what a basic test function looks like, and having good structure. But then he talks about, he wanted to ensure, you know, do static analysis and code style, so he uses Black within his testing, and when he was talking about using PyLint, I don't use PyLint every day, so I didn't know there was, it's a very comprehensive check, but it takes some time for large codebases, I didn't know that, but has a cool hack that he puts in place to only lint for check-in tests, only lint modified files.
16:35 KENNEDY: Oh, that's cool. Yeah, 'cause of course, if they're unmodified, then would they...
16:38 OKKEN: Yeah, they wouldn't be different
16:39 KENNEDY: have a different outcome.
16:40 OKKEN: Right, and he uses, incorporating Flake8 to do docstring testing, to make sure that people are using consistent docstring styles. He covers all of his tox.ini configurations changes, he was trying to increase his code covered, so it includes coverage.py, but then also has a 'cov-fail-under' flag that he adds for testing to make sure that if code coverage drops below a certain point, it fails the test, but then just generally gradually ratchet that up, so that it increase, his target was 75%, so it talks, even goes into fixtures and mocks and spies and stubs, and then even a cool tool called pytest-vcr, which records your network interactions, and then replays those for future test runs, and he saw 10x speed-up in that.
17:35 KENNEDY: That's really cool.
17:36 OKKEN: There's so much cool stuff in here.
17:37 KENNEDY: pytest-vcr, that's really cool. I think the only problem with it is maybe a lot of folks using it have no idea what VCR means.
17:44 OKKEN: Oh yeah. That's true. I mean, even, yeah so.
17:48 KENNEDY: Yeah, but no, it's awesome that you just record the network interactions, and don't have to depend on anything at all. I love it.
17:54 OKKEN: And the recordings are done based on a per-test basis, so if you re-run an individual test, it only plays back the recording for that portion. It doesn't have order dependency built-in, which is cool.
18:07 KENNEDY: Yeah, super cool. I love it. Yeah, that's a really nice article, Josh. Well done. The last one I want to talk about was sent over by Kevin Buchs. Now, we've covered a few of the language, sort of the language-level learning things recently. We talked about the CPython byte compiler either last time or the time before that, how it doesn't really optimize stuff, and maybe there's some opportunities there, but more just understand what's going on. So, Kevin sent in a message, said, "Hey, I'm basically a C/C++ guy, and I saw the del keyword in Python, and it threw me for a loop, 'cause del seems like delete in C++, which means free memory, but it doesn't necessarily mean that in Python." So, even seems like some of the books out there are kind of being a little misleadin'. At least, according to Kevin's reading of them. So, I thought I'd just pull up an article that he sent over, and talk a little bit over some of the uses for del.
19:05 OKKEN: Great. I don't use it, so this should be good.
19:07 KENNEDY: Yeah, so the context where I know del is I want to get something out of a list. Or I want to get something out of a dictionary, right?
19:16 OKKEN: OK.
19:17 KENNEDY: And it's a little bit weird. It's like 'in' keyword, right? A lot of times, I would expect some operator to be on the object I'm modifying. Like, 'list' or, you know, 'string.in' or something, and you give it the value, right? But you say 'string in the', the variable, right? So, it's a little bit funky that you apply it, not on the object, but as a keyword in the language. And del's like that, right? So if I have a dictionary, and want to remove a key, not set it to nothing, but make it not be in the keys collection, you could say, 'del dictionary of bracket', like as if you're accessing that value but puttin' the del there takes it out.
19:52 OKKEN: Oh, OK.
19:54 KENNEDY: Yeah, and you also do that for lists, so I can go in and remove it, remove something from a list, if I want, there's a remove function on the list, but somewhat confusingly, potentially? It's by value, right? So I could say remove Jeff from the list and Jeff will no longer be in that list, wherever he appeared. But if I want to say, remove the third thing, there's no remove at or anything like that, right? I can't pass 2, 'cause that's not a value, right? So, del will let me remove that. You can also use pop for that, I believe, on a list, but del's a little more general-purpose, and you can also delete slices. So I could say, "Go to the list and take out everything from 2 to 5." You know, 2:5, like that. So these are all pretty interesting. Now I'm linking over to the official docs that talk about it, and this article that kind of talks through some of these examples, and shows you how to use it. You can also delete a variable out of a local or global namespace. So, if there's a variable that's been defined, and you want it to not be defined, I can say del variable name, and now it's as if I didn't do that line that defined it, that created it.
20:58 OKKEN: Does it remove it from the namespace?
21:00 KENNEDY: Yeah, it doesn't free the memory necessarily, but it takes out as a global variable.
21:06 OKKEN: OK, interesting.
21:06 KENNEDY: Or a local one. So does it actually free any memory? It depends. So, if I have it in the global, let's say it's a global. It has obviously the thing that has a value at that variable. It's taking up some memory. If nothing else is pointing at it. It's still going to be around, 'cause that global variable's pointing at it, but if you call 'del' that variable, you'll de-reference that one reference to it, putting the reference count to zero, and freeing it up. So theoretically, you could free up memory using del. Similarly, if it's in a list, and the only place that points to it, has a reference to it, is that list itself, and it delete out of there, or out of the dictionary, it goes away, memory-wise. But if something else is pointin' at it, then obviously it's not going to got away. We also talked about how the CPython bytecode compiler is dumb. Dumb, as in not super-optimizing, maybe on purpose, and I think you could also, you know, if you're really dealing with memory issues, and you're like, "I really wish this thing would just go away sooner, in this one little edge case." you could probably use del to put in some of the optimizations that you might hope that the compiler itself might do, but doesn't. De-reference a thing as soon as it's used within a function before you can get to the end or things like this.
22:21 OKKEN: Yeah, OK.
22:22 KENNEDY: So, is it for memory? Sort of. Not really. But may be as a side-effect.
22:27 OKKEN: Yeah, this has been a long time, but I do remember it tripping me up, because I was like, it seems a lot like delete, which should have a matching 'new' to it.
22:37 KENNEDY: Yeah, exactly. Exactly, we've both done the C++ thing, where's the new that goes with del? I have never seen a 'new'. Anyway, it's pretty cool, there's a couple links here, there's official documentation, there's the article Understanding Python's del, and then there's the reference to that bytecode compiler people can check out.
22:53 OKKEN: Yeah. In C++, I don't there's a way to remove a name from a namespace.
22:58 KENNEDY: Yeah, I don't think so either. You can make it point at null, but that's about it. But I mean, you got to think about classes. You could delete a field out of a class, because it's just a dictionary. So much of Python is built on dictionaries, like the variables, or the variable names are the keys in the dictionary, and their values are their value, and so you just take it out the global dictionary, effectively.
23:21 OKKEN: Yeah, OK, cool.
23:21 KENNEDY: Pretty sweet. So those are our main items for today. You got anything else you want to chat about, Brian?
23:28 OKKEN: I'm just, I'm glad it's summer, it's startin' to feel nice, feels like summer. But other than that, not much. How 'about you?
23:34 KENNEDY: Summer's awesome, but makes programming hard, 'cause programming's indoors, although some of my friends and I, who work from home, we try to get out and program at like a coffee shop or a cafe by a lake or something, and periodically, we have the weird experience of getting a sunburn while writing code, and yeah, we've dubbed it a codeburn, and it's kind of a badge of honor.
23:55 OKKEN: That's funny. Cool.
23:56 KENNEDY: Yeah, yeah. So there's actually of things I want to throw out here. We recently had Max Sklar for The Local Maximum podcast, and afterwards he's had me onto his podcast, so I'll be on episode 73, which should be out not yet, but thanks to time-shifting, when this episode comes out, it should already be out. I'll put a link to that. Josh Thurston sent over a cool video of the popularity of languages on StackOverflow over time, as a bar-chart race. I didn't know about bar-chart races, but these are basically animated bar charts over time, and you just watch the bars grow and shrink, and it's really cool. Python kind of a little, tiny consideration at the bottom, and obviously we know that Python is crushing it on popularity on StackOverflow and all those things, so it's like a minute and a half video. I think everything'll appreciate watchin' it, and if just got a minute to kill.
24:46 OKKEN: Nah, it's a fun video, and one of the things I enjoy about it is early on, you see the Java bar going up and down based on the time of year, because it's used, it was used in education a lot, that totally made sense.
25:00 KENNEDY: Exactly, you're like, "Oh, there's a huge spike in September, I wonder why." Maybe a bunch of people got a job. No, like CS 101 is now back in session. Exactly. Then the last one I want to throw out is this thing called Pynsource. So what this does, this comes to us from Anders Klint. It's basically a UML diagram creation tool for Python code. So you give it some Python files, it will generate a UML diagram, that shows the relationship of all the classes in there.
25:30 OKKEN: Oh, that's cool.
25:32 KENNEDY: Yeah, it's pretty cool. There's a free, maybe even open-source version, and there's also a paid version, so you can buy it and I'm actually not a huge fan of UML, but if you have Python code, and you think a UML diagram would help describin' it, this thing's pretty cool, actually. And it's a little GUI app. There's a bunch of screenshots, you can check it out, and see if it'll help you, but looks pretty neat, and it does proper UML, not just sort of visualization of classes, so that's kind of nice.
26:00 OKKEN: My favorite use of these kinds of diagram is to print them out, and pin them to your wall, your cubicle wall, so that other programmers think that you're smarter than they are.
26:09 KENNEDY: Absolutely. Put some little cryptic notes on them, as if you're marking them up. Yeah, absolutely. Yeah, love it.
26:16 OKKEN: Definitely.
26:17 KENNEDY: Yes, you can do this with your project. Yes, this huge thing of a project. Anyway, it's pretty cool, and there's a free version, like I said, so maybe it'll some folks out there. All right, are you ready for some jokes, Brian?
26:27 OKKEN: Yes, definitely.
26:28 KENNEDY: Alright, I have a, you've heard about the glass being half-full and half-empty, and like, "Oh, I'm a half-empty sort of person, I sort of see the world as slightly negative."
26:35 OKKEN: Yes.
26:36 KENNEDY: So here's the developer version, we have an optimist who says the glass is half-full, we have the pessimist who says the glass is half-empty, and we have the programmer who says the glass is twice as large as it necessarily.
26:47 OKKEN: Yes, definitely. So I wanted to extend that with the pragmatist, that says I'm just allowing enough room for requirements oversight, scope creep and schedule overrun.
26:59 KENNEDY: That's right, it's perfect. I love it, and then this other one about software start-ups?
27:04 OKKEN: Yeah, I man, it's not really any start-up, but I watched The Upside with Kevin Hart last night, and there was a joke that I couldn't help but sharin', I can't remember the characters, but Kevin's character said, "Would you invest in my business idea?" And the other guy says, "That seems too niche." Kevin: "What's niche means?" "Oh, it's the girl version of nephew." It's a charity.
27:31 KENNEDY: I love it, that's bad. If you got asked, that's a pretty good answer. Cool, cool. All right, well thanks for putting all the cool stuff together as always, and bein' here.
27:40 OKKEN: Yeah, thank you.
27:41 KENNEDY: Thank for listening to Python Bytes. Follow the show on Twitter via @pythonbytes, that's Python Bytes as in B-Y-T-E-S, and get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Okken, this is Michael Kennedy. Thank for listening and sharing this podcast with your friends and colleagues.