Transcript #204: Take the PSF survey and Will & Carlton drop by

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 204, recorded October 14th, 2020. I'm Brian Okken.

00:10 I'm Michael Kennedy.

00:11 And we have a couple of guests, Will Vincent and Carlton Gibson.

00:14 Hello.

00:14 Hello.

00:15 Hey guys.

00:15 Thanks for having us on. It's really quite exciting.

00:18 Before we move on to the first topic, people may already know you. You guys are Django famous, I hear. Tell people about your podcast real quick.

00:26 Yeah, so I'm the Django fellow, help maintain the framework there. And with Will, we run Django chat podcast. So and we'll do some other things. What do you do? Well, Yeah, well, when we started the podcast, I was just a book author, but I'm Django Software Foundation board member now. So I have a hand in coordinating with Carlton in official capacity. But mainly I teach Django through books and learn django.com website.

00:50 Oh, and I have a Django news newsletter as well. So I keep piling things on. We don't have two podcasts like both of you, but maybe we'll get there. We haven't got time for two pokos.

00:58 You start with one and then you just get another one. That's how it goes, you guys.

01:01 Okay, like children.

01:02 Brian's going to start a third one, I heard. I'm going to start a Rema for him.

01:05 I'm going to start a third.

01:06 Yeah, oh yeah, why not?

01:07 I just should jump to four. I mean, binary, right?

01:11 Exactly. Hey, Brian, we talked about Jupyter a bunch last couple times, like, don't let us down, like, keep it rolling.

01:17 It's a great thing to provide a tool for people. And then we get a whole bunch of people calling or getting a hold of us and saying, "Hey, there's more stuff you should know about." And this is the case this time. So, Marco Garelli, I think his last name is, sent us a... Also, I have to say, he said he was a long-time listener and he's a Patreon supporter.

01:35 So, thank you, Marco, for being a Patreon supporter.

01:37 Thank you, Marco.

01:38 Very cool. So, he said, "You guys should know about MBQA." So, MBQA is Quality Assurance for Jupyter Notebooks. So, it also can run black. So, you can... One of the things... If you just want to run it like a black thing to run black. One of the benefits of using it to run black is that you can run it on an entire directory, not just a single notebook, but a whole directory full.

02:00 There's some modifications with its use of black so that it keeps diffs fairly minimal for the diff set. And then there's black will take off trailing semicolons because in Python they don't really mean anything and they look ugly. But in Jupyter notebooks apparently they mean something.

02:18 they mean to suppress the output of the notebook or suppress output. So that's the black version or the NBQA version of black. We'll turn that but leave those in place. And also supposedly supports standard magic commands and magics are kind of a big part of Jupyter thing. So another thing I want to mention it doesn't just run black, you can also use NBQA to run isort and mypy and flake8 and and even pi upgrade and doc test.

02:48 So that's pretty neat.

02:49 - Yeah, this is really cool.

02:50 I think it brings so much of that code formatting, code analysis, cleanup to notebooks, which I think have been really lacking.

02:59 - Yeah, some of the standard practices that a lot of people are using now, as well as the configurations all in a pyproject.toml file.

03:07 And you can hook it up with pre-commit so that you can have all these things running when you check stuff in, whatever.

03:13 - Yeah, and you can even run it against a whole directory, not just one notebook, which is sweet.

03:16 - I'm definitely gonna check this out.

03:17 It looks really fun.

03:18 Oh, one thing I wanted to mention, I checked out the source code for it and it's 100% covered and covered by pytest, of course.

03:27 So, nice.

03:27 - Woo-hoo!

03:28 - Nice.

03:29 - Little chip.

03:30 - Well, Carlton, what do you guys think about this?

03:31 - Well, I was thinking I use all those tools, but I don't use them in the notebook format.

03:36 I have to say, I sort of bottled with notebooks, but I'm not a big user there, so it sounds super.

03:41 But I recommend all of those.

03:42 - Yeah, ditto. - Yeah, absolutely.

03:44 - Whenever I think of Jupyter Notebooks, I'm always reminded of, I think the finest tech talk I've ever seen is, I hate notebooks, that Joel gave a couple years back.

03:53 Yeah, Joel Grus, which is, but I mean, he's not just slamming out the whole time, but it's a very educated talk, and I think that's a high bar for sort of complimenting and pointing out issues on a framework or on a project that can be improved.

04:07 - Yeah, well, I think that actually, a lot of the complaints are starting to get addressed with things like this, right?

04:14 It's starting to get a little bit better.

04:16 There's also some other cool ones called JupyterLab-LSP for language server protocol or provider or something like that, which also does a bunch of things that make it a little bit better.

04:27 So yeah, it's getting there, it's pretty cool.

04:29 - Just one thing, the ultimate for general web developers is if you could take a Jupyter notebook and just snap your fingers and have it be a Django site, you can't quite do that.

04:37 But if I take off my technical head and just what would change things, like I'm surrounded by a lot of scientists, to turn a Jupyter thing into a website with standard CRUD, I feel like it's possible one day, but we're not quite there.

04:50 - That would be really fantastic.

04:51 And we're gonna hear like five ways we could do it that we didn't know about from listeners, which is great.

04:56 One thing I would really love to see in Jupyter Notebook and maybe someone out there knows about it is I would like to see collapsible sections.

05:04 So I've got like a report and it's got like some markdown and then some code, then a graph it's generated, and then some more, like maybe a picture, and then some more markdown.

05:14 And the code in there is really awesome to have, but if you're going through it as a report, you don't necessarily wanna see the code unless you wanna like dig into it.

05:20 So it'd be great if you could say these columns or these cells are collapsed.

05:24 I really would love to see that, but I don't know about that yet.

05:26 That's opposite direction of what you're looking for.

05:28 That's making the notebook more of a article, less of a website.

05:32 All right, so the next thing I wanna cover is the PSF yearly survey.

05:37 Have you guys taken your yearly survey for 2020?

05:40 - I've done it, sir, I've done it.

05:41 Already got my homework in.

05:42 - Well done, Carlton, well done.

05:44 - I have not, but I've done it past years and I will.

05:46 And we actually, in the Django world, we're inspired.

05:48 We had our own survey this year, the first time in five years, 'cause I don't think Python does as well, but Django doesn't track anything.

05:55 So we actually don't know.

05:56 - And no installs or usage.

05:58 - Right, okay.

05:58 - And that's obviously helpful to fellows and technical team.

06:01 - Of course, and I think basically the extent to which they track that is the analytics that come out of PIP.

06:07 like pip was run on this operating system.

06:09 This package was installed this many times.

06:11 It was this version of Python that did it.

06:13 Beyond that, I don't think there's much tracking in the Python world either.

06:17 The broader--

06:18 - We have PyPI, but that's not completely accurate in terms of popularity.

06:22 - All the Docker rebuild stuff that's happening all the time, like that counts, but it's not legit and so on.

06:28 So yeah, for sure.

06:29 So if you haven't taken the PSF survey, I put a link in here.

06:32 It takes about 10 minutes.

06:33 You should go do it.

06:35 This is the fourth time they're doing this developer survey.

06:38 And it's the major, a major, the major, I'm not sure, a major source for sure about the current state of the Python community.

06:45 So what editors do people use?

06:47 What web frameworks are people using?

06:49 Are you a data scientist or are you a web developer?

06:53 Et cetera, et cetera.

06:54 Are you just getting into Python?

06:55 And if you haven't seen the analysis of this before, I linked to the 2019 results, which were put together by JetBrains.

07:05 and they did a really nice job of making a compelling story to be told out of it, right?

07:09 Yeah, no, it's really nicely done and presented.

07:11 You're like, "Oh, wow, yeah, it's super." That's why, you know, it's worth putting in too, because the production value at the end is great, and so it's a valuable resource.

07:19 I felt bad for a second that we didn't have that on Django, but instead I just...

07:23 We're not JetBrains.

07:24 Yeah, we're not JetBrains.

07:25 That is the gold standard. I was like, "Oh, it'd be nice to have that." And I was like, "Or I could just make it a Google form." Well, you guys should reach out to the JetBrains team and see if they want to partner up.

07:35 Yeah, well, they've probably got the resource in place, you know, they've got the infrastructure.

07:39 Now, have they always done prizes?

07:41 I don't remember that they have, but they have them now, right?

07:44 Yeah, so that's cool.

07:45 Yeah, so they announced that 100 winners, completely random, if you've completed the survey, will receive the amazing Python surprise gift pack, which I have no idea what it is because that would ruin the surprise.

07:56 I saw some good Python socks on Twitter today. I hope it's got Python socks in it.

08:00 Oh my god, I love socks.

08:01 Like that's half the reason I go to conferences.

08:03 Let's be honest.

08:04 Got my T-shirt with socks.

08:06 I got all my different socks.

08:07 I got my MongoDB socks.

08:08 I've used to go just for T-shirts, but now I kind of like the socks more.

08:12 I do too.

08:12 We've added an official Django merchandise store and there's some items on there and that's been helpful with the virtual conferences, but we don't have socks.

08:19 So there's a lot of inspiration we can take for having better official gear out there.

08:24 Absolutely, get your sock game on.

08:26 Yeah.

08:26 And then stickers, right?

08:27 It goes T-shirts, stock, socks, stickers.

08:29 I think in the hierarchy of swag.

08:32 That's right.

08:32 That's right.

08:33 Okay.

08:33 I have to up my game.

08:35 I'm just giving out stickers.

08:36 Usually I'm in the enamel pin man myself.

08:38 Oh, enamel pins.

08:40 Oh yeah.

08:40 I should mention JetBrains, which is doing that.

08:44 That survey is a big sponsor of Django.

08:47 We do a couple of week long thing every year and they're a major corporate sponsor of Django, so shout out to JetBrains.

08:54 Very nice.

08:54 Yeah.

08:54 Very nice.

08:55 And one of the things I'm really excited about is we have a new sponsor and it's another podcast.

08:59 Yeah, so that's pretty cool.

09:01 So this episode is brought to you by Tech Meme Ride Home podcast.

09:05 For more than two years and nearly 700 episodes, that's amazing.

09:09 The Tech Meme Ride Home has been a Silicon Valley favorite tech news podcast.

09:14 The Tech Meme Ride Home is a daily podcast, only 15 to 20 minutes long, and every day by 5 p.m. Eastern, it's all the latest tech news.

09:24 But it's more than just headlines.

09:25 You could get a robot to read you headlines, but the TechMeme Ride Home is all the context around the latest news of the day.

09:32 It's all the top stories, the top posts, the tweets and conversations of those stories, as well as behind the scenes analysis.

09:38 It's like a TLDR as a service.

09:41 The folks at TechMeme are online all day reading everything so they can catch everything up for you.

09:47 Search your podcast app right now for Ride Home and subscribe to the TechMeme Ride Home podcast or just visit pythonbytes.fm/ride to subscribe.

09:57 Yeah, it's like Python bytes, but just for general tech.

09:59 Every day though, these guys don't mess around.

10:01 That's incredible.

10:02 Yes. All of us who are podcasters, we're like, "Oh my gosh, that's insane." Yeah, we all wince a little bit hearing 700.

10:09 - Every day. - Promise by 5pm.

10:11 Sounds like a burnout algorithm.

10:13 It's well done. But yeah, cool.

10:16 Thank you guys for sponsoring the show.

10:17 Will, what's your item?

10:18 Oh, so my item is from prototype to production in Django.

10:22 So this is a common thing where you get a little familiar with Django and you say, well, what's the, there's this big chasm basically from building a CRUD app locally and deploying it properly without being hackable.

10:35 - What do I do?

10:35 I run it as a root, I leave the debug setting on.

10:38 What else do I do?

10:39 - That's important.

10:40 - Yeah, well, it's sort of like, you don't know what you don't know.

10:45 And then the older you get, the more scared you get because you've seen it all go bad.

10:48 But when you're starting out, you're like, what could go wrong?

10:50 - Yeah, as soon as it works, it works.

10:51 - Yeah, so specific to Django, I think like most web frameworks, it has to, it focuses on local production.

10:57 So when you run, you run a start project command and it creates some scaffolding for you.

11:02 And then specifically it has a settings.py file.

11:04 That's kind of the global config and that's set for local development.

11:09 So works great locally, but if you just dump it into production, you're gonna be wildly insecure and easily hacked.

11:15 And so it's a quick list of things you wanna change.

11:17 And Carlton, please jump in here as the Django fellow if I missed something.

11:21 But debug is a setting that is, you want to switch to false, that provides a very nice error message, but it also is a roadmap to hacking your site if it's left on.

11:30 Right, these are in settings.py generally.

11:32 In settings.py, so it's all about settings.py basically.

11:34 There's a secret key that Django, that is a 50 character long string randomly generated, you want that to be secret, because it's used as a hash throughout Django.

11:42 And of course, what happens is you do one git commit, and then it's out there.

11:45 Yeah.

11:46 So you need to change that, or really put it into an environment variable, which I'll get to in a sec.

11:50 Are you familiar with shgit?

11:52 No, I'm not.

11:52 SSH Git.

11:54 So this is super scary.

11:58 This is a...

11:59 Oh my gosh, it's live right now.

12:01 I can't believe it.

12:02 So I think it's at SSH Git.com, but there's also the open source version that you can get.

12:08 You can see it on GitHub.

12:09 Let me just read you this title just to like point out how seriously this should be taken.

12:14 Shigit finds committed secrets and sensitive files across GitHub, Gist, GitLab, Fitbucket and your local repos in real time.

12:21 It does this by subscribing to the commit stream on GitHub and instantly posts the secrets like AWS secret keys and stuff.

12:30 You can see if you go there like...

12:31 Yeah, I see six for Django right now.

12:33 - And it's all the configuration. - Yeah, I just got two more.

12:35 Yeah, I just got two more, three more, five.

12:37 I just got five more. I mean, it is insane.

12:39 - Yeah. - Huh.

12:40 If you think my repo is not so popular, it will be fine.

12:44 It may not be so fine.

12:45 But this is the thing with security, right?

12:47 is that it doesn't matter how small you are, because the people who are attacking you, they're using automated scripts.

12:52 So they're checking every port on every addressable server with every known weakness.

12:56 It's not if you'll be hacked, it's when.

12:58 Now setting the stage how frightening this is.

13:00 Carry on why we shouldn't put that in.

13:02 Well, I think it just reinforces that the settings.py file is where most of the action is in Django, and you want to be careful with it.

13:11 I mean, I remember GitHub back in the day, you could just global search for AWS keys and Stripe and everything.

13:16 now at least you can't global search for that stuff and they'll even ping you. So for me, like I have some secret, I have some projects in my books that are on GitHub. And there's a secret key there and they bug me all the time saying, Hey, you have a secret key exposed.

13:28 I'm like, Yeah, I know I do. I don't, it doesn't matter. So it's gotten better. But yeah, it's still all out there. So secret key, keep that secret. Allowed host is probably the last big one. These are the hosts that can come in. It Django will prompt you to change that.

13:41 So if you're using Heroku, and it's myapp.heroku.com, you want to set that host to be only that host, not all hosts can come in.

13:49 Database is the next one. So by default, Django has a SQLite database, file based, really easy to use.

13:55 Fantastic for production, large scale, Facebook uses it.

13:58 You know, it can be.

13:59 No, I just do it.

14:00 If your workload is read only, so say you're running a content site, and it's just you editing it.

14:04 SQLite will go all the way with you.

14:06 But as soon as there are more than one editor, and...

14:09 It's incredibly fast. It's in memory, right?

14:11 Yeah, you can have it in memory, we can have it on the file, but on read-only workloads, it will go you know, right out there.

14:17 Sorry, I meant in process, in process, not in memory, but it's not like a separate server, right?

14:20 Exactly. It's just a file next to all your other files and it can hold you know, terabytes of data without a problem.

14:26 But as soon as you've got multiple users logging in at the same time or that kind of thing, then you need, you know, something like Postgres or MySQL can handle that kind of concurrency.

14:35 Yeah, so you probably want to get ignore your SQLite file. But also, you definitely want to use whatever using production locally. So Postgres, MySQL, MariaDB, and Oracle are the supported databases. What else almost done here, you configure your static and media files. So static would be images, JavaScript, media refers to anything that's user uploaded, you definitely want to be careful whenever you have anything from users can't trust them.

15:00 You want to use Django forms, you probably want to use bleach to sanitize and you want to have that on a CDN, not on your server. Two more to finish up. So the admin Django has a famous admin that's very powerful, which is at slash admin, you should change that away from dot admin because to Carlton's point, there are bots searching for Django sites at slash admin and they will come in and hack away at your site. There are a number of fun technical things you can do to honeypot it or this and that, but you should just change it away from dash admin. I'm tempted to know there's a very famous Django site that still has slash admin, but I won't mention it. Carlton and I both use it though for our work. And then the last thing is user registration. Django comes with login, logout, reset, but it doesn't have a signup. So you can roll your own or most people would use a third party package called Django all off. That's fantastic that has social support. So Django has very robust third party ecosystem that over time, the most popular ones are the strongest ones are rolled into Django. But there's also separation because it's too much for Django maintain Django all off is not is a third party package, but it's, in my view, effectively part of Django for most people. So those are the big ones are the key things I mentioned their environment variables, it used to be with the settings file that back five years ago, you'd have multiple settings files, you'd have a base settings file, Carlton still doing that, I still have multiple settings files go with those folks. I mean, you have environment variables to but multiple settings files for the week. He's a Django fellow. He doesn't know he's talking about you need to use environment variables for this, because then you have one settings file and you loaded into local staging or production. It's much easier.

16:35 Spaces.

16:35 But it works for Chrome.

16:36 Spaces.

16:36 There's a number of third-party packages that will help you with that. I like environs, which will be linked there, which also has dj-database-url, which is a nice feature on environment variables for databases. It just means you have a single settings file, and you switch with environment variables. There's also Django has a handy deployment checklist, which I think a lot of people don't know about. We have a link to that. You can run python check --deploy. And it will check that all the things I mentioned plus a number of HTTPS things are actually configured properly. So you don't want to deploy your site. If you don't pass most of those, if not all those. Yeah. And that's awesome. You know, there's testing, logging, performance security, we can go on and on. I wrote a whole book Django for professionals on this.

17:17 But those are the highlights. And there's a check there's a good pet doc on the Django docs for deployment checklist, which, you know, you should open that every time - Yeah, the hard thing is there's like a couple of must-haves, like the ones I listed there.

17:30 And then there's a lot of it-depends-nice-to-haves.

17:33 And that's where it's harder to make generalizations.

17:36 - The stuff is so rewarding when you get it right.

17:38 To see your site up and running, you know, 99% plus uptime and people using it, so fantastic, highly responsive.

17:46 But soon as you see like something go wrong, it just, your heart sinks.

17:50 And so most of those things are because, like you both have hinted at, there's some kind of bot that's looking for some vulnerability and like a known thing, and just make sure you don't put those known things in front of the internet.

18:02 - Yeah, and I should say actually, there used to be a site called Pony Checkup that you could put in your URL and would automatically test a lot of this for you.

18:11 It's actually, someone has taken that over from the original maintainer.

18:14 So it's now djcheckup/pony.

18:17 So you can type in your URL and check.

18:20 That's kind of a good way if you're a beginner, if you're not sure.

18:22 There's nothing like going to a webpage and seeing security issues in your site or others to kind of scare you into doing something.

18:28 - Yeah, yeah, for sure.

18:30 Awesome.

18:31 Well, I'm glad you covered that.

18:32 And Karl, it sounds like the one that you got is a bit of a--

18:35 - Similar topic, actually.

18:36 So I've been thinking about it, but I think it must be Django chat.

18:39 Every week, it seems, we have a guest on and we end up talking about deployment and it's massively complicated.

18:44 And Will's just gone through a whole list of things and that's only the tip of the iceberg kind of thing.

18:50 And the thing that I got thinking about was that there is this deployment gap.

18:53 So I imagine someone finishing the Django tutorial, finishing the REST framework tutorial, or finishing Will's Django for Beginners book, and then how on earth do they get their app online?

19:02 And it's like, you know, unless you're going to dedicate a week or two weeks full-time researching and trying and following tutorials online, it's like this chasm. We can't do it.

19:12 And so platforms as a service like Heroku or App Service or App Engine or DigitalOcean have got their new one.

19:18 They look like a great starting point because they're kind of easy, but in a way, for me, they're a kind of cul-de-sac.

19:22 You go into them, you get to the end, and then you kind of have to go back out again when you want to do something more advanced.

19:28 But then on the other hand, you've got this do-it-yourself option of provisioning servers and setting up firewalls and virtual private clouds, and ah, it's just, it's way too much.

19:37 It's scary, right?

19:39 And then you read some blog posts and it's saying, well, you've got to do it with microservices, or you've got to do it with this container orchestration platform, No, no, no, it's too much for me.

19:48 So I've been thinking about this and trying to come up with a story of my own for it, and which I'm launching next year.

19:54 It's going to be called Button.

19:55 It's going to be a little tool that just wraps it up and tries to take some of the fuss out of deployment.

20:00 So that's not ready to go yet, but I wanted to mention it because it ties into what Will was talking about.

20:05 And you can sign up for early updates on btn.dev, button.dev, btn.dev.

20:10 So that's my topic.

20:12 That's a great topic.

20:13 I think I personally struggled through this.

20:16 Right? I started out trying to run my websites in Python on some pass place that was very simple and easy to get started, but it's just there's a ton of downtime and things weren't working the way that I was really exact, you know, hoping and I ended up having to do a lot of things anyway. And so I finally bit the bullet and figured out how do you run microWSGI safely? How do you keep these things up? How do you do zero downtime deployment? How do you do continuous? There's just so many And how do you keep updated, right? How is it?

20:44 So, okay, you get it set up and it's fine, but then six months later, there's a security patch, which you don't quite know how to apply without bringing your whole site down and rebuilding a server.

20:53 It's like, how do you deal with those problems?

20:55 They're not something you can learn quickly or easily.

20:58 Yeah, absolutely.

20:59 Brian, you got to deploy things?

21:01 I have before and that's why I don't anymore.

21:03 - I mean, very wise. - Yeah, last time we talked.

21:06 Yeah, last time we talked about DigitalOcean's new pass service, and you're like, I'm all about this.

21:11 Yeah, like for instance, so you did your own like Python bytes and talk Python, you did those applications, right?

21:19 Talk Python, talk Python chain and like 10 other like little APIs and stuff that I'm running, all those, yeah.

21:24 Right, so I've done, I mean, I've built websites before, like in the way past, like decades ago.

21:30 And then when I wanted to do a podcast, I did a Python, the testing code started out as a, as just part of my blog, and it was like on WordPress or something like that.

21:40 Now I will go with a fireside thing.

21:43 And I don't, I mean, fireside's a good service.

21:46 But it's not, I mean, it's not ideal.

21:48 It doesn't, it isn't perfect.

21:50 But I don't have to think about it.

21:51 What I want to do is podcasting.

21:53 I don't want to maintain a website.

21:55 So there's a lot of things where we do need to build these custom websites.

21:58 And I'm glad that there's some attention given to this, because, yes, I can learn how to do a Django website.

22:04 But going from there to a live site is horrifying.

22:08 So, yeah.

22:10 I literally just spent an hour this morning, maybe hour and a half before, like up to 10 minutes before we started recording, deploying my first FastAPI endpoint through gunicorn, uveicorn, behind-engine X and all that stuff.

22:25 And a lot of it was the same, but that's the first time I've done it with uveicorn and the settings are a little bit different than say, microWZGY.

22:31 So I can run it with some basic stuff.

22:32 And I kind of lived in that world of like, I got to, what is the config key to make sure, or the config settings that make sure that it runs is a different user and not root again in this server?

22:41 And just like you just go through all these things and having that like really automated would be great.

22:45 - Yeah, I mean, the other day, my site went down for half an hour and it turned out to be the DNS in the end, but I never thought the DNS would go down.

22:52 How often does that happen?

22:54 So I log into the server and I'm checking the application server, that's fine.

22:57 So I check the local NGINX instance, that's fine.

23:00 I check the load balance, that's fine.

23:02 And I'm like, it is, it's the DNS.

23:03 And then by the time I've worked this out, the DNS comes back up and the site's back up.

23:06 And it's, ah!

23:07 And what I want to do is pull that fully automate, you know, so I just run the diagnostics. It goes green, green, green, green, green, red. There's the problem. Okay, fine.

23:16 That's awesome. Let me know when you got that. I'm excited.

23:18 21 btn.dev.

23:20 Okay. Cool.

23:21 Yeah. Well, and Carlton, I think your cul-de-sac analogy, which I haven't heard you say before. That's exactly right. Yeah, because it sounds like, oh, this will solve my problems. And then you learn everything. And then you can come back with a different problem. I mean, part of the problem with doing server stuff, I think is that it's, it feels good to an engineering mind, right? It feels good to be like, you know, drive manual and tweak things, but then you've lost weeks of time and your app looks the same.

23:46 And, you know, so there is a danger there in terms of, I feel like you kind of need to do it once or twice and then be like, okay, I'll just have Carlton handle it for me.

23:54 Like, I trust him more than...

23:56 This is the via media, right? This is between the sort of platform of service which perhaps doesn't cover all your need cases, and then the over-engineered, you know, I'm using container orchestration for a Microsoft thing and I've only got one server and a thousand hits a month, right? It's you know Yeah, there's a middle way between that AWS will never be bothered to make it friendly to Small people because they don't care. I mean they have this, you know Big clients. So yeah. Yeah. Yeah, very cool. I also like the cul-de-sac thing. See you think it's an on-ramp, but it's really a cul-de-sac Yeah You're just an infinite loop of But that's the thing, especially somebody who's in that deployment gap scenario that I talked about.

24:35 They go down the platform and service route because that's the only thing they can do.

24:38 But at the end, they're like trapped there.

24:40 And I want to do more, but I've got to go all the way back down here to learn this other stuff, which is so hard, and so scary, and so overwhelming.

24:48 Yeah, well, I want to talk about something that's easy.

24:50 And well, it's sort of easy.

24:52 It's contributing to other people's open source projects.

24:55 That's not easy.

24:56 You're just petting Carlton here.

25:00 No, I'm really excited about this topic.

25:02 So I ran across a thing called all contributors and actually we talked about in NBQA before and that's where I got this idea because they use this tool called or this service called all contributors.

25:14 And it's sort of a service but also just a spec. So I'm just going to read it. It says this all contributors is a specification for recognizing contributors to an open source project in a way that rewards each and every contribution, not just code.

25:30 The basic idea is to use the project readme to recognize the contributors of members of a project community.

25:38 The idea is like there's a lot more stuff going on than just code. There's things like documentation, design, writing examples, doing maintenance, writing plugins for things, doing podcasts, giving talks and all that stuff, and it'd be cool to recognize all these people.

25:53 So they've got this spec for kind of how to do that.

25:56 But then there's even more. There's a cool emoji key.

25:59 I love the emoji key part. It's so friendly.

26:03 Yeah, so it has recommended emojis to use for your contributors list that includes things like a little laptop computer for code, a little thing for documentation, design, examples, all the sort of things you'd want with like fairly good, you don't have to think them up, they thought them up for you.

26:20 So it's nice. They also have like this bot that you can attach to your GitHub repo.

26:25 So you can just add comments into somebody's pull request or something to say, "Hey, all contributors, please add this person to the contributors list or something." And it just does it.

26:35 Okay, that's super cool. We're looking into that with Django. A few weeks ago, I was reading, Will was reading, other people were reading the Working in Public book. For the life of me, I can't remember the author's name this moment.

26:46 It's Nina, I think.

26:47 Okay.

26:48 Agbar.

26:49 But it mentions this all contributors thing.

26:51 And one thing we've got with Django, we've got a massive contributor base, but we kind of only historically recognized the sort of 30 or so people that were in Django core.

27:00 And then over the course of the last couple of years, we've kind of tried to restructure the governance, and we've managed to do all of that.

27:06 And now we're in the position where we want to try and recognize all the other people.

27:09 We translate the docs into however many languages.

27:12 That translation team gets virtually no recognition.

27:15 Let's recognize them.

27:17 Yes, there's code commits, But there's also all the people that help triage and review the tickets and review the pull requests.

27:23 Those people need recognition.

27:25 There's the people who organize all the Django cons.

27:27 Those people need recognition.

27:28 We really want to try and show that Django isn't just, "I committed to Django," it's the whole ecosystem.

27:35 So I think this all-contributors thing is a great tool.

27:38 You said you were reading the Working in Public book by...

27:41 Would you recommend it? Is it good?

27:42 Yeah, it was amazing.

27:45 just the first few chapters, it's just like describing Django to a T.

27:48 It's like, "Oh, yeah, this is the challenge we face every single day." Like a couple of years ago at DjangoCon in San Diego, I gave a talk about your web framework needs you, and I put up a graph of contributors.

27:58 And in the first chapter, it's the exact same graph.

28:01 I mean, it's got different numbers and is for a different repo, but it's the same power law shape.

28:05 It's the same problem. It's like, this isn't just Django.

28:08 It's every open source project out there.

28:10 It's got the same issues, and it's the same dynamic.

28:13 Do you grow in on yourself and get smaller and more enclosed?

28:17 Or do you open out to the community and welcome contributions and find a way of doing that?

28:21 And if you can, you can survive and flourish.

28:24 And if you can't, well, you'll weather and die.

28:26 Yeah, I mean, I literally took screenshots of the book because I was like, "Carlton, you should read this." He's like, "Uh..." And he's like, "Okay, okay. It's exactly my thing." And this all contributors is so relevant because the most Carlton mentioned at Django, we're changing around what Django core refers to.

28:41 And I literally have a huge spreadsheet with all the various people we're trying to categorize that this would fit in perfectly for.

28:47 So I'm going to potentially use this.

28:50 And it has a GitHub bot, which is fantastic.

28:52 Yeah, what can be done manually and, you know, five minutes can be automated in an hour.

28:55 So that's right.

28:56 Well, Michael, what are you going to finish us up with?

29:00 Well, you know, I want to keep us on the move.

29:03 Don't sit still, you know, rolling stone gathers no moss sort of thing.

29:07 So pandas is a super popular library in data science, right?

29:12 And there's a bunch of visualizations.

29:14 One way to work with geospatial data is with geopandas, which is cool.

29:19 So there's a library called moving pandas.

29:22 And the idea is if you give locations plus time, you can map all sorts of interesting things and analyze all sorts of interesting things.

29:32 It's cool, right?

29:33 And this project, as it should, has a bunch of animated GIFs.

29:36 - Yeah, that's what we need to see. - You can tell exactly what it's about, right?

29:40 Get in there.

29:41 So it provides trajectory data structures and functions for analysis and visualization.

29:47 It started out as this QGIS plugin, but they decided it made more sense to just be its own thing.

29:52 So it's its own thing.

29:53 So basically, it takes GeoPanda's geo data frames with timestamp points, and it converts them into moving pandas trajectory collections.

30:04 And properties such as speed and direction.

30:07 You can turn continuous observations into trips like I was here for a really long time and then I went to the store and I was there for a while and then I came back.

30:16 That kind of stuff, I think.

30:18 It'll aggregate them into flow maps.

30:20 So instead of, I went exactly from here to here, you can say here are the nodes where I spend time and the paths I take between them.

30:28 Almost like graph theory type of stuff.

30:30 And work with it is super straightforward.

30:32 So you can just go create a pandas data frame, pass a geometry and a time, and you convert it to a geo data frame, and then you just say, give me the trajectory and you can plot it.

30:44 And that's it. Like, incredibly simple.

30:46 - That sounds super.

30:46 Their website's really good as well.

30:48 I was just clicking on it because it's to see the animated GIFs and whatnot.

30:51 - Well, it seems like that would overlap with...

30:54 So, I mean, Django is a big framework.

30:56 There's a whole geo Django area, which Carlton and I have discussed.

30:59 I mean, that and the ORM are the two parts of Django.

31:02 I kind of don't really know, to be honest, but they're very powerful and people use them.

31:07 - Cool.

31:08 You can also get like graphs of kind of derived data.

31:11 So like speed over time, rather than just position, you can get these other analysis in there.

31:19 And I can see lots of interesting places.

31:21 I had Ken Replical on Python to talk about how they're using Python on the race teams for simulations and stuff.

31:29 And like those types of analysis, This seems so perfect.

31:33 Go spend a day to track, collect a bunch of data, throw into these types of things and look at the curves and whatnot.

31:38 - Yeah, and pandas is almost like the kind of data transfer format now.

31:42 I mean, you know.

31:43 - Yeah, absolutely.

31:44 - So to be able to integrate this sounds super useful.

31:46 - Yeah, it looks cool.

31:47 - Awesome.

31:48 - All right, well, that's the last item.

31:49 Brian, you got any extra stuff you wanna share with people?

31:51 - No, just working and plugging along.

31:53 - Yeah. - How about you?

31:54 - I do have a few things.

31:55 First of all, I was talking to Hugo Anderson from Coiled And he asks, "Hey, when is the transcript "from our research show gonna be out?" I'm like, "Eh, don't really have transcripts at the moment.

32:08 "The company I was working with to generate 'em "stopped generating, stopped doing that kind of stuff, "and I haven't figured out what to do." He's like, "Oh, you should check out otter.ai." I'm like, "Yeah, but isn't that like for your phone "and like you can have conversation, that's cool." But what I realized is I can upload files to that, our old episodes, it'll convert it to mostly correct transcripts, like pretty good actually.

32:30 you know, it'll get like AWS, right?

32:32 And things like that.

32:33 And-- - We use that.

32:34 - Then I just wrote, yeah, I just wrote some automation to turn that into transcripts.

32:37 So I added like half a year worth of transcripts back, which means that feeds our search engine.

32:42 So search should be better as well and stuff like that.

32:44 - Yeah, I think it's the top one.

32:46 I think Wes Boss was also asking something and I was tweeting with him saying, "Yeah, Otter, I think when I checked "is by far the best one." It's not really designed for transcripts.

32:54 It's designed, I think, for like meetings and groups, but it works.

32:57 We've been using it for a year.

33:00 - Yeah. - That's awesome.

33:01 And you guys like it?

33:02 It's been good?

33:02 - Yeah, as you say, I mean, it's the most accurate out there and usually it gets almost everything.

33:07 You can kind of have custom things like AWS, if it gets it wrong.

33:11 And yeah, I mean, usually I run it through and give it a quick scan.

33:15 Maybe there's a couple things to switch, but yeah, it's a no brainer.

33:18 - Yeah, it's got like a nice editor that like plays and highlights the words if you were actually gonna stop and edit them.

33:24 I also have automation, like for my courses, I have automation through AWS Elastic Transcribe or just transcribe, whatever it's called, to generate those and then hand them off to people.

33:33 But Otter's looking nice.

33:34 I'm not sure if I'd switch the courses over.

33:36 But anyway, we have a bunch of transcripts, right?

33:38 - Yeah, so I've never really done, I started doing testing code transcripts, but I was actually just paying somebody to do them and it was getting expensive.

33:47 So yeah, let's check this out too.

33:49 - Yeah, it's worth checking out.

33:50 - It's the SEO that really matters, I think.

33:52 That's like the killer feature.

33:53 - Exactly.

33:55 That's why I first created them.

33:56 And I thought, okay, I'll make them searchable so people can also get some value out of it.

34:00 But my original reason for doing it was like, instead of having three paragraphs of content for an hour conversation, let's have the real conversation, right?

34:08 - Yeah. - But then, you know, someone will find that and that will be useful.

34:11 But, and you know, they'll be like, ah, this is, you know, even if it's badly transcribed, this is roughly what I'm looking for, I'll listen to the episode.

34:17 - Right, right, yeah.

34:18 Let me listen, here's the timestamp and they can get some value out of it.

34:21 So hopefully, yeah, that's the idea.

34:22 - Nice. - All right.

34:23 Also, I'd switch from Google to try to live in DuckDuckGo land just using DuckDuckGo.

34:27 - Oh, join us, yeah.

34:29 - Yeah, are you guys doing it?

34:30 Are you liking it?

34:31 - I've been there for three years exclusively.

34:33 - Oh, God, I'm still on Google, but I did install the PiHole this week, so, you know.

34:38 - Oh, yeah, yeah, yeah.

34:40 - Swings and roundabouts.

34:41 - Carlton's like, I see your DuckDuckGo usage, and I gotta--

34:44 - I'll raise your PiHole.

34:46 - This is what I have to deal with, guys.

34:47 (laughing)

34:49 - So far, I'm liking it.

34:50 I mean, I've been using Firefox with all sorts of privacy stuff for a long time, but I figured just one more thing, And I just want to point out, if you're trying to be like slightly less connected to Google, they have Google Takeout, or you just want to back up, right?

35:01 If you use Google Drive, and you sync your Google stuff, it'll give you just a link to the spreadsheet or whatever on Google.

35:07 Excuse me, if you use Google Takeout, it'll actually convert like, say, your spreadsheets to Excel.

35:13 So you actually have them.

35:14 Anyway, so that's part of that.

35:15 And like I said, I got to deploy my first FastAPI app today, basically, and I'm just, I'm really enjoying it.

35:22 And I feel like it's bringing in a lot of these ideas.

35:24 I'm hoping, maybe you guys can just comment super quick on this.

35:28 It brings in so many of these new ideas into the web space, like the async and await stuff feels super natural.

35:33 You don't have to do anything to make it work.

35:37 The type annotations mean things.

35:39 I just feel like there's a lot of interesting sort of modern Python stuff coming together here.

35:44 Like, what's your Django perspective?

35:45 Well, so FastAPI is built on top of Starlet, which is by Tom Christie, who's Django REST framework creator.

35:50 So from the async side, that's what we're trying to build into Django now.

35:54 And we have async views in 3.1.

35:56 And, you know, we're working on the ORM next.

35:58 And then from there, it will spread out.

36:00 So there's a PR came in this week about the cache layer.

36:02 So there'll be async, third-party async cache backends for Django soonish.

36:08 The, an async free, fully async framework like Starlet, it's always going to be out there.

36:13 You know, it's going to be ahead of where Django is, but we'll wrap it up and give it that nice Django feel where you define your class-based views and all these things.

36:20 We're not there yet, but that's what we're aiming for.

36:22 Then the other thing that FastAPI brings out, which is quite exciting, is Pydantic, which is the type hinting used for the serializers and for the validation.

36:31 That's kind of really cool.

36:32 And at the moment, we don't have a story there with Django.

36:35 We've got Django forms, we've got REST framework serializers doing the same kind of thing.

36:39 But we've got our eye on that and we'll see how it goes.

36:41 - I know you guys are definitely thinking about these things.

36:43 It's very exciting.

36:44 - Yeah, I mean, but what's really nice about the current, particularly the ASCII world, where all the kind of ASCII is the standard, is there's an amount of interoperability, in that you can kind of nest your ASCII apps inside each other and wrap winnaways around them, and it's just another ASCII app.

36:58 And so actually, there's a lot of interop things.

37:00 So it's really rich and fertile time for Django web frameworks.

37:04 Yeah, awesome.

37:04 I don't have anything to add, Carlton. I defer to Carlton.

37:06 Other than it's sort of wild that, you know, I mean, from Tom, we've known about Starlet, that FastAPI is better known than Starlet.

37:14 It's a little strange to me, but it makes sense, because Tom's been busy rebuilding everything in async the last couple years, kind of on the side.

37:20 Also, the thing is that people touch FastAPI.

37:23 They just live on top of it, like they live on the shoulders of Starlet, but they touch FastAPI, right?

37:28 Yeah, no, exactly.

37:29 Yeah.

37:30 All right, Brian, I put in two jokes that we can offer up today.

37:34 Let's shout out to some of the stuff that Will and Carlton are doing first.

37:37 Yeah, absolutely.

37:38 Okay, well, we just, I'd say, listen to DjangoChat, which is at djangochat.com.

37:42 That's our podcast. It's fortnightly now.

37:44 That's a fancy British word there for you.

37:47 Check out wills tutorials and books at learn django.com and then yes sign up for the early updates on button Which is at btn.dev just did super welcome aboard. You'll be subscriber number three, I think Well, actually I had to tell carlton I was like get up a page before we go on the podcast because he's been telling me about button for a year and Yeah, it looks good. Well, it looks like a sign up for him right now. Well, yeah, that's what it is right now But yeah 2021 it's coming nice. Yeah. Okay now a joke. Thank you Thank you for getting us back on track, Brian.

38:18 So you've heard about give a person a fish versus teach him to fish.

38:22 There's a programmer version, did you know that?

38:24 Yeah.

38:25 If you give a person a program, you can frustrate them for a day.

38:27 But if you teach them a program, you can frustrate them for a lifetime.

38:31 Yes, definitely.

38:33 Unless what, Brian?

38:35 Unless you teach them to test at the same time.

38:37 Ah, very good.

38:39 Exactly.

38:40 And speaking of FastAPI, here's a joke that I just saw that's relevant, sort of similar to one put out by Sebastian Ramirez from FastAPI.

38:52 So somebody just failed a job interview and the verdict was delivered like this.

38:57 I'm sorry, we're looking for someone aged 22 to 26 with 30 years of experience with Flask or Django.

39:03 (laughing)

39:05 - Well, didn't he tweet about someone who was looking for five years of FastAPI?

39:08 And he was like, even I don't have that.

39:09 - Yes, exactly, exactly.

39:11 He's like, well, as the creator of FastAPI, I would not qualify you for your job having done only 1.5 years of it.

39:17 Yeah.

39:17 Yeah.

39:18 Well, I don't think people realize that it's so new.

39:20 I mean, it's kind of taken over very quickly, but it hasn't been around for very long.

39:25 Yeah.

39:26 So yeah, it's pretty interesting.

39:27 Anyway, guys.

39:28 Yeah.

39:29 Thanks for joining us.

39:30 Yeah.

39:31 No, thank you.

39:31 It's been really cool chat.

39:32 Really enjoyed it.

39:33 Yeah.

39:33 Thanks.

39:33 Yeah, it's been fun to have you here, Carlton.

39:35 And we'll.

39:35 All right.

39:36 Bye.

39:36 Bye bye.

39:38 Thank you for listening to Python Bytes.

39:39 Follow the show on Twitter at Python Bytes.

39:42 That's Python Bytes, as in B-Y-T-E-S.

39:45 And get the full show notes at pythonbytes.fm.

39:48 If you have a news item you want featured, just visit pythonbytes.fm and send it our way.

39:53 We're always on the lookout for sharing something cool.

39:55 This is Brian Okken, and on behalf of myself and Michael Kennedy, thank you for listening and sharing this podcast with your friends and colleagues.