Brought to you by Michael and Brian - take a Talk Python course or get Brian's pytest book


« Return to show page

Transcript for Episode #60:
Don't dismiss SQLite as just a starter DB

Recorded on Wednesday, Jan 10, 2018.

00:00 KENNEDY: Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is Episode #60, recorded January 10th, 2018. I’m Michael Kennedy.

00:00 OKKEN: And I’m Brian Okken.

00:00 And we’ve got a bunch of stuff lined up for you as always. Before we get to it, I want to say thank you to Datadog. Datadog is sponsoring the show at pythonbytes.fm/Datadog. More to say about that later.

00:00 I want to know who’s at nine?

00:00 (Laughs) Who’s at nine? Yes, who is at nine?

00:00 Wouldn't it be awesome if you could just utter this question into the air and have it automatically answered for you?

00:00 Yeah, well, that’s the promise of Alexa and Google Home and all these home assistants, right?

00:00 That’s right.

00:00 This came up, what was it, Episode #33 of Python Bytes? You brought up building an Alexa skill. Somebody saw that and we have a link to a Twitter thread.

00:00 Yeah, it was Greg Quinlan who posted that on Twitter. That was cool.

00:00 Okay. Well, he ran with it and looked into programming both skills for Alexa and Google Home and he came up with this “Who’s at nine?” video that we have a link to and I think it’s hilarious. It’s basically the old “Who’s on First?” routine, but done with an Alexa and Google Home together. It’s funny.

00:00 Yeah, he’s got like a little Dot and a Google Home. He activates them both and they talk to each other.

00:00 It cracked me up. It’s good. We should have had a clip for it or something for the show.

00:00 Yeah, we should just play it. That would be fun. One of the things that strikes me – first of all, nice work, Greg, that’s awesome – Flask-Ask is pretty interesting to work with. Flask-Assistant is the Google equivalent of Flask-Ask; it’s just opted to that API. What’s really interesting to me, the API here effectively is Flask and I’m noticing this in lots and lots of places. Someone just sent us something that was almost a web API with a command line. I’m noticing Flask being like the de facto API for so many things these days.

00:00 Yeah, it must just be that it’s easier to set up a small Flask application for something.

00:00 I guess. Pay attention to #4 of our pics as well; we’ll come back to this.

00:00 There’s something coming up related to the “Who’s at nine?” that’s on Talk Python, right?

00:00 Yeah, so I have three guys, a panel discussion, who are all involved in building Echo/Assistant-type things. Some of the people behind Flask-Ask and some of the people behind Flask-Assistant – those are sometimes the same people – are on Talk Python this week. It should come out about the same time this episode comes out, which is pretty awesome. If you really like this and you want to dig into it, check out talkpython.fm/146. There should be a whole lot there.

00:00 That's cool. I’ll check that out.

00:00 Sounds good.

00:00 I came across an article that I think is pretty interesting. You and I have been singing the praises of Python and how it’s really popular. It’s becoming quite the important language in so many ways, right? We talked about the incredible growth of Python, things like that. But I guess this struck a chord with me because I feel like one of the places where Python really falls down is the intersection of packaging for delivering an application to non-developers or non-servers, especially in that area around UIs, like native UIs.

00:00 Yeah, definitely. We don’t have that, really. It’s not as obvious what to do.

00:00 It’s not obvious. There’s a few half-solutions that sometimes work, except for when they don’t. The thing that I ran across that made me bring this up again, it’s not that I think Python completely sucks because it doesn’t have it – obviously, I don’t think that – but I see it as a major weakness and if we can solve this challenge, I feel like it would just make Python even stronger and reach even deeper into big enterprises that have to have desktop UIs, and things like that.

00:00 there’s this article I found called, “Retiring Python as a Teaching Language.” This is by James Hague and it basically says, ‘Through the last ten years, my advice for someone getting started at programming is to say start with Python. It’s awesome.’ But then he wrote this article to say, ‘Actually, that’s not my belief anymore.’ That’s kind of sad, right?

00:00 Yeah, it is. Okay, so his big ‘beef’ really is part of not being able to do these interfaces, like desktop applications, easily.

00:00 Yeah, absolutely. And mobile as well; these are all sort of cousins. He says, ‘It’s all great. People work at Python when they’re getting started. But then one day the student will innocently ask, Instead of writing a poker simulator with the command line, can I just put it in a window with a button and some cards?’ Yeah, okay. So, what do you do then?

00:00 there’s TKInter, but I don’t know, that looks like it’s from the early ‘90s, late ‘80s. It definitely doesn’t look fresh and exciting like modern applications do. We have PyGame, but that’s only if you want to game, right? But suppose if say, I work at a stock trading place and I want an app that is across three monitors and showing me the status of all my trades running real time, like a websocket back to some server, as fast as it can be, like completely show the traders what’s going on. What’s the answer for Python there?

00:00 I don’t want to dis PyGame too much because I know some people who have done some non-game things with it, but I don’t really have an answer to that either.

00:00 I think that’s a big shortcoming an if we had a few good answers for that like some of the other frameworks. Even JavaScript has really good answers for that, which is kind of bizarre. Visual Studio Code, Atom, those are all Node.js plus JavaScript front end apps, right? That’s Electron.js. I would like to see a modern sort of thing for Python and I think this is just worth considering. Actually, the Twitter thread – we’ll link to the Twitter thread, it’s pretty interesting – if you open up my initial tweet, I don’t know if you want to call it a complaint or warning to the community, you can see all the conversations. Then Guido (van Rossum) gets int here a little bit to talk about some of the history, which is pretty interesting. Thanks for that. I don’t know that there’s a great answer right now, but I’m trying to inspire people.

00:00 I’m going to play devil’s advocate a little here. I know that you’ve said that you’ve written a lot of desktop applications, but to be honest, I have written a couple little tiny things that I've needed for utilities at work. There’s a whole bunch of programming that isn’t writing desktop applications.

00:00 Oh, absolutely. And a lot of it’s services, a lot of it’s on the web or it’s just command line utilities, like you said. But this is why I think we’re underestimating the importance of this. If you could open up your editor and it was just, ‘Here’s a few of the widgets, I just dropped them in here.’ And I could put a .app or a .exe somewhere and my entire company could pick it up and run it locally. How have you not even entertained that idea because you’re like, ‘That doesn’t exist so why even consider this weird idea, right?’

00:00 There’s a lot of internal projects that are set up as web applications, internal web applications, that could definitely be desktop applications with a shared database back end. We don't do it that way because it’s not easy to do that.

00:00 Exactly, it’s super hard. But it would be really nice to have some sort of ability to take advantage of the local application and local OS. Maybe you talk to a web service for shared data, maybe you just use SQLite locally, that might be interesting. But I don’t think we do this because it’s so not working. Even if you got the UI working, it's like, ‘Now what do we do for packaging?’ I know there are answers, but they’re kind of always tricky, right?

00:00 if that was entirely obvious, the tooling was like, ‘Yeah, of course you do this.’ How many apps in the Mac App Store would be Python apps? Right now, I bet it’s like zero percent. Basically, right? (Laughs) Once you rounded it down to the significant digits, right? They don’t exist. And these things can just open up, if we could somehow as a community get really nice cross-platform UI setup.

00:00 I agree. I think some of the stumbling clocks on that are that little thing that you throw in that’s cross-platform. I know that that’s cool, but I don’t know if it’s necessarily a requirement.

00:00 That’s true actually. I would say, if you had a killer way to build MacOS apps stop, that would be a huge benefit for Python. If you had a killer way to build Windows apps, there’s still the majority of business people sit down and login on a Windows machine. If you could do that, that would be an amazing advancement for Python. So, you’re right, it doesn’t necessarily have to be. One model is what the people at Zamran are doing. They basically have cross-platform stuff but every platform has its own UI bit. They have these kind of designer things that help with that. I don’t know. I just wanted to put that out there and let people know about this conversation and think about it. I think we’re having an open session at PyCon on it.

00:00 That will be good. I’m glad we’re bringing it up because we do need to hammer on this until it’s fixed.

00:00 If I look across all of Python, this is really the place where there’s still a weakness. It would be nice to just fortify that wall and make it a perfect place to be.

00:00 Definitely.

00:00 And like I said, SQLite as a little database, you don’t even a server. That might be a good choice, right?

00:00 (Laughs) Yeah, definitely. So, I came across – well, this is an older interview but somebody recommended to us that we go back and listen – Episode #201 of The Changelog. It’s another podcast but there’s an interview with Richard Hipp. He calls it “S-Q-L-ite,” which I never heard it pronounced that way.

00:00 As in a meteorite? A very interesting way of pronunciation, none of which I would have guessed.

00:00 Right. So, the rest of us know this is SQLite (pronounced Sequel-ite) I think.

00:00 That’s how I know it. That’s what I’m calling it.

00:00 This is the database, if you just say, ‘SQLite3 import’ in Python, you get a little single file database.

00:00 This is beautiful because it’s built into Python. Every instance, everybody who has U.S. Python has this so you can just count on it being there and it runs in process, so there’s no configuration, right?

00:00 Yeah, definitely. I didn’t realize how dang cool the thing is until I listened to this episode, this Changelog episode. One of the comments from the interviewers was they just assumed it was a starter database that you always eventually have to move over to something else and that’s not true. As long as your application is not a web application with extreme client server concurrency, you probably can get away with a lot of desktop applications that are small. Or if you have a client-side part of your application that could just be a SQLite database and it will be fine.

00:00 Yeah, that would be awesome. If you had some sort of client-side thing that you periodically wanted to sync with a real database, you could store it all in SQLite locally, and just synchronize that data across some kind of service with your real data and that would be a real nice way to take the load off the server, to have a local offline version, all sorts of stuff.

00:00 Some things I didn’t know about SQLite before I listened to this was that it started its life as a TCL application.

00:00 (Laughs) Yeah, which is bizarre, but that’s what happened.

00:00 (Laughs) Very bizarre. But I guess that’s where TK started, in the TCL world. But also, one of the things you have to watch out for is that it’s not as typesafe – it’s “type-flexible” – when you are changing over to another database and your other database might be a little pickier than SQLite, so be careful of that.

00:00 So, an example is you might have a column, which is in the DDL, designated as an integer. If you try to assign “7 (quote seven) to it, it would just convert that to 7. Things like that. It’s very funky in that regard; it’s kind of script-y like that.

00:00 He comes from a background of interpreting languages and weakly typed, so that makes sense.

00:00 It’s kind of cool. I thought that the interview was really interesting. Definitely a lot of respect for what he’s doing. He’s go some very different opinions and perspectives on Open Source, on version control and editors. I’m not sure I agree with all of them but they’re certainly interesting to think about.

00:00 A couple more things. There’s a page of how SQLite is tested and I think it’s really cool that they put that out there, like what their checklist is for testing it.

00:00 The amount of tests are huge, right?

00:00 Yeah. And then also, I didn’t realize it was in everything. Every Android phone is using it and things like that.

00:00 I think iOS as well, yeah.

00:00 Yeah, so it’s quite ubiquitous.

00:00 It’s definitely ubiquitous. It’s very cool and I think it’s definitely a good place to start. A lot of times you might no need more than that, which is one fewer server to configure, one fewer thing to patch. One fewer thing to make sure the firewalls are alright. It’s really nice to just have that simple thing. Nice.

00:00 speaking of servers, let’s talk about Datadog real quick. Datadog is a monitoring solution that provides deep visibility inside of not just your app but the infrastructure your app works with. So, within a couple minutes you can investigate a bottleneck in your code by checking out some flame graphs on dashboard. You can visualize your performance. If you go and do their free trial, you’ll get a free Datadog t-shirt which has got a cool little dog on it. So, check it out at pythonbytes.fm/Datadog and let them know thanks for sponsoring our show.

00:00 Yeah, thank you.

00:00 So, I told you about Flask being this sort of like API that is standing in for all these other places. So, the next thing I wanted to cover is server-less programming.

00:00 this joke that I actually don’t agree with. I kind of disagree with it philosophically but it’s making fun of Cloud computing saying, ‘Well, that’s just another person’s computer.’ (Laughs) There is no cloud, it’s just another person’s computer that you’re borrowing.

00:00 Isn't that true?

00:00 It’s technically true but my disagreement is the Cloud computing stuff really has to do with hardware that you can basically create and control through programming. So, if there’s an API, you can say instantly, ‘I want a machine. I want to scale my machine. I want to replicate my machine.’ That’s not the same as co-location or a rented server. Because you can’t just go and rent a server and then hit an API and have another server. Anyway, that’s my disagreement. But to some degree that’s true right?

00:00 we’ve moved beyond that joke to a farther world of potential ridiculousness, where we have server-based programs that have no servers, right? This whole AWS Lambda, Azure Functions, serverless programming, it’s kind of funny, right?

00:00 Yeah.

00:00 Yes. So, basically this idea of the serverless programming, whether it’s AWS Lambda or Azure Functions is you right a single function – preferably in Python but they support JavaScript , C#, things like that – and that single function is the entry point into some piece of functionality and you’re going to host that function on the Cloud. And it’s up to the infrastructure to figure out what docker container to create, run that on, make that happen, scale it, etc. You just say, ‘Here’s literally a Python function and its dependencies. Run that.’ So, that’s pretty interesting,

00:00 main reason here that people go for this one is that there’s no server at all to maintain, not even a virtual one. The scalability if basically infinite and the price is insanely cheap because you only pay while your function’s literally executing.

00:00 Okay. There’s till servers, you just don’t have to set it up.

00:00 Yeah, and your code is not stuck to one of them. Every function call basically is a new docker container. I’m not entirely sure about the internals but basically every call is like an isolated execution and then it goes away.

00:00 That’s part of the appeal is that you don’t really have to care about it.

00:00 That’s totally the appeal.

00:00 The infrastructure just deals with it.

00:00 You do pay for a small latency on requests because it’s got to spin off this bit. But that’s still, for a lot of things that aren’t performance-critical at least in the latency, it would be fine.

00:00 I wanted to talk about this week is this thing called, “Chalice: Python Serverless Microframework for AWS.’ So, basically, this is a Flask API.

00:00 Another Flask API?

00:00 Another Flask API. So, you implement these functions in Chalice, and you decorate them with this Flask API but instead of running a Flask app, you create a Chalice app. Then you could even say, ‘Chaline deploy’ and run it and then you just go and basically request your thing. What you effectively create is something that looks like a Flask app with all the different pieces that are related and so on, but then they’re hosted as individual function calls on the serverless infrastructure. So, it’s pretty cool, right?

00:00 Wow, okay. That’s cool.

00:00 So, I think this is really interesting. This is just like, ‘We’re going to take Flask and put it onto AWS Lambda’ which is pretty cool. Maybe the other counterpart that I know pretty well is Zappa. So, Zappa is, I believe, it’s own framework. I don’t think it has Flask variation. I could be wrong, I haven’t done that much with it. Zappa also lets you basically create this API against this website equivalent, scaled out across these different functions. So, pretty cool thing to be playing with. You don’t even need Cloud anymore, you don’t even need servers. It’s all serverless.

00:00 Of course, you still have to set up an AWS account and get your little token or whatever to hook into this, I assume.

00:00 Yeah, exactly. You’ve got to get your own token and all those things. You probably are talking to some sort of hosted database or S3 or some combination thereof to actually do your work, right?

00:00 Right. And actually, I’ve never worked with AWS but these sorts of things make me kind of want to try because it doesn’t look scary at all.

00:00 Absolutely. So, let’s wrap it back to Item #1. When you create your Flask skill, your Ask-Flask stuff and when you work with Alexa, one of the main ways to work with Alexa is to actually tie those to Lambda functions, these serverless function, so actually you can put it all together like that.

00:00 Neat.

00:00 Cool. So, I think you can probably help some people with job interviews coming up here. First, how do I reverse a string? Second, what’s the fastest way to uniquify a list in Python?

00:00 Reverse a string? This is a short little thing but there’s a blog article called, “The Fastest Way to Uniquify a List in Python Greater Than or Equal to 3.6.” Basically, 3.6. (Laughs) Unless you’re running 3.7 already. I just appreciated, instead of having an article that says, ‘Okay, here’s the answer,’ there is a discussion of the ways you might do it and the code for how he timed it. So, if you don’t believe it, you can reproduce all the code and get your results on your machine that’s probably similar. The short answer is, if you want to keep the order in place, you simply do a list of a dict from keys in your sequence/list(dict.fromkeys(seq)). And we’ll have this in the show notes, of course, and a link to the article.

00:00 if you don’t care about the order, then just convert it to a set and then a list. So, that’s it. I’ve used the list set before and it makes sense that fromkeys works; I’ve never really used that before.

00:00 Yeah, and it’s going to get interesting with the promises of dictionaries being ordered. You could actually get some of these to be a little quicker but still preserving the order.

00:00 Well, isn’t that the way it is now?

00:00 Not all of them. There’s like two graphs that says, ‘not order preserving functions’ and then ‘order preserving functions.’ And right now they have lists of sets of lists, not order preserving.

00:00 Right, because set isn’t an order preserving thing.

00:00 It’s pretty cool though, pretty cool. Definitely worth thinking about all the different ways and seeing the trade offs and so on.

00:00 I want to wrap up this episode with leaving some people with homework. They’ve got some research and some chilling on the couch, watching some Python videos.

00:00 Okay.

00:00 So, there’s two sets of conference videos that came out, and I’m sure there’s some that I missed, but these are the two that stood out to me. PyTexas and PyCon Australia. Both have their videos up and I linked to the playlists on YouTube for both of them, which is pretty cool. There’s, like always, 40 or more videos. There’s tons of really interesting stuff. I kind of want to touch on some and these are not necessarily endorsements. The just look really interesting to me and I’d like to go watch them. I haven’t spent all this time watching all these videos on the list right now.

00:00 PyTexas, there’s a talk on Micropython, which Micropython is really cool, right? You can run your Python on a five dollar microchip and hook your Lambda functions, your Python ones, up to hardware interrupts. That’s just the coolest.

00:00 then there’s so much talk about machine learning and AI and all that stuff. So, there was a talk given there called, “What is Machine Learning Anyway?” Or something like that. A foundational background on machine learning, which is cool.

00:00 one is, “C for Yourself.” Trying to explore the internals of C Python.

00:00 then there’s one on Python and .NET, for this thing called Python.NET, which is not the IronPython that’s been around for a long time, but this newer way to try to integrate this things. All the people who work on the big enterprise apps and C#, they can now plug in some sweet Python. Or vice versa.

00:00 Yeah, okay. Those all look good to me, too, especially the Micropython.

00:00 Yeah, Micropython is great.

00:00 that’s Texas. And then in Australia, those guys were killing it with names down there. They just did a really good job. There’s “Gradual Typing.” We’ve talked about that, the type hints and adding them and stuff and I think this is probably a really interesting survey of that.

00:00 Reloading Python Web Servers at Scale.”

00:00 Oooo, that sounds nice.

00:00 Yeah, that sounds like a lot of cool internals.

00:00 Python Microservices in Production” and “Secrets of a WSGI Master,” like understanding a base HC protocol. And then another one, “Python 3 for People Who Have Not Been Paying Attention.”

00:00 (Laughs) So, that’s not anybody that’s been listening to this.

00:00 (Laughs) Exactly. They know about Legacy Python.

00:00 “Identity 2.0: The What, Why and How of Social and Federated Login.” So, if you’ve got to do anything with identity across providers, that’s so painful. This will be cool. And then here’s sort of a Tesla reference, I’m guessing.”Python: Ludicrous Mode with Django.” So, ‘ludicrous mode’ in Tesla is the insane amount of power, give your neck an ache if you hit the gas too hard, sort of thing. Or something to that effect.

00:00 then finally, “Scaling Down: Running Large Sites Locally.” We often talk about scale up and all these complexity you have to deal with. You even talked about how you’re not Google, you’re not Facebook, or any of these things awhile back, which was cool. But this is if you do have one of these large sites that’s got a complicated architecture, say a Reddis back in message queue type thing, and a database and a whole bunch of other stuff. Some of those are hosted in the Cloud. How do you actually develop that locally?

00:00 That’s cool. There’s definitely a need for everyone, I think.

00:00 I’m sure there's a lot of good lessons in there. So, the links to both of those playlists are up on the show notes, so that should be fun.

00:00 Alright, cool.

00:00 How about you? Any other news?

00:00 We’re out. That was six. Cool.

00:00 actually been crushing it lately.

00:00 I’ve been so busy, it’s been so fun though. So, last week I released my “Everything Bundle” where people can get all the courses. And so, I shipped my first course for that “Everything Bundle” called “Mastering PyCharm. Which yes, it covers PyCharm, but it’s also basically like a little mini-course on 16 different parts of software development. There’s like a little, tiny 20-minute thing on refracturing, a 15-minute thing on unit testing and pytest, 30-minute thing on Git. So, it’s like all those pieces put together through the lens of working with PyCharm. I’ll link to that and people can check that out if they want.

00:00 also doing a webcast with the JetBrains people, but on MongoDB. If people want to drop in on that on January 30th, I’m doing a live event with JetBrains and look for that registration. It’s free. You can come check it out.

00:00 Cool. I’ll come watch.

00:00 Alright, awesome. Yeah, that’s all I’ve been doing this week.

00:00 Nice.

00:00 Well, thank you, Brian.

00:00 Thank you.

00:00 There was a lot of good stuff this week and looking forward to more next.

00:00 Talk to you next week.

00:00 Alright, bye.

00:00 you for listening to Python Bytes. Follow the show on Twitter via @pythonbytes and get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way. We’re always on the lookout for sharing something cool. On behalf of myself and Brian Okken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.

Back to show page