Transcript #204: Take the PSF survey and Will & Carlton drop by

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to

00:04 your earbuds. This is episode 204, recorded October 14th, 2020. I'm Brian Okken. I'm Michael

00:10 Kennedy. And we have a couple of guests, Will Vincent and Carlton Gibson. Hello. Hello. Hey,

00:15 guys. Happy to have you here. No, thanks for having us on. It's really quite exciting.

00:19 Before we move on to the first topic, people may already know you. You guys are Django famous,

00:23 I hear. Tell people about your podcast real quick. Yeah, so I'm the Django fellow,

00:28 help maintain a framework there. And with Will, we run a Django chat podcast. So and Will does some

00:34 other things. What do you do, Will? Yeah, well, when we started the podcast,

00:37 I was just a book author, but I'm a Django Software Foundation board member now. So I have a hand in

00:42 coordinating with Carlton in an official capacity. But mainly I teach Django through books and

00:48 learndjango.com website. Oh, and I have a Django news newsletter as well. So I keep piling things on.

00:53 We don't have two podcasts like both of you, but maybe we'll get there.

00:56 We haven't got time for two podcasts.

00:58 You start with one and then you just get another one. That's how it goes, you guys.

01:01 Okay. Like children.

01:02 Brian's going to start a third one, I heard. I'm going to start a Rima for him.

01:05 I'm going to start a third.

01:06 Yeah. Oh, yeah. Why not?

01:07 I just should jump to four. I mean, binary, right?

01:11 Exactly. Hey, Brian, we talked about Jupyter a bunch the last couple of times. Like,

01:15 don't let us down. Like, keep it rolling.

01:17 It's a great thing to provide a tool for people. And then we get a whole bunch of people calling

01:21 or getting ahold of us and saying, Hey, there's, there's more stuff you should know about. And

01:25 this is the case this time. So Marco Gorelli, I think his last name is sent us a, also I have

01:31 to say, he said he was a long time listener and he's a Patreon supporter. So thank you,

01:36 Marco, for being a Patreon supporter.

01:37 Thank you, Marco.

01:37 Very cool.

01:38 So he said, you guys should know about MBQA. So MBQA is quality assurance for Jupyter notebooks.

01:45 So it also can run black. So you can, one of the things, if you just want to run it like a black

01:50 thing to run black, one of the benefits of using it to run black is that you can run it on an entire

01:57 directory, not just a single notebook, but a whole directory full. There's some modifications with its,

02:02 its use of black so that it keeps diffs fairly minimal for the diff set. And then there's,

02:08 black will take off trailing semicolons because in Python, they don't really mean anything

02:13 and they look ugly, but in Jupyter notebooks, apparently they mean something they mean to

02:18 suppress the output of the notebook or suppress output. So that's the black version or the MBQA

02:26 version of black will turn that, but leave those in place. And also supposedly supports a standard

02:32 magic commands and magics are kind of a big part of Jupyter thing. So another thing I want

02:38 to mention, it doesn't just run black. You can also use MBQA to run ISORT and mypye and

02:43 FlakeAid and Pylent and even PiUpgrade and Doctest. So that's pretty neat.

02:49 Yeah, this is really cool. I think it brings so much of that code formatting, code analysis,

02:54 cleanup to notebooks, which I think have been really lacking.

02:59 Yeah, some of the standard practices that a lot of people are using now,

03:02 as well as the configurations all in a pyproject.toml file. And you can hook it up with pre-commit so

03:09 that you can have all these things running when you check stuff in, whatever.

03:12 Yeah. And you can even run it against a whole directory, not just one notebook, which is sweet.

03:16 I'm definitely going to check this out. It looks really fun.

03:18 Now, one thing I wanted to mention, I checked out the source code for it and it's 100% covered and

03:25 covered by pytest, of course. So nice.

03:27 Nice.

03:28 Little chip.

03:29 Will Carlton, what do you guys think about this?

03:30 Well, I was thinking I use all those tools, but I don't use them in the notebook format. I have to

03:36 say I sort of potter with notebooks, but I'm not a big user there. So it sounds super, but I recommend

03:42 all of those.

03:42 Yeah, absolutely.

03:43 Ditto. And I would, whenever I think of Jupyter notebooks, I'm always reminded of, I think the

03:47 finest tech talk I've ever seen is, I don't, was it I hate notebooks?

03:50 Yeah, by Joel Bruce.

03:53 Yeah, Joel Bruce, which is, but I mean, he's not just slamming out the whole time, but it's

03:57 a very educated talk. And I think that's a high bar for sort of complimenting and pointing

04:03 out issues on a framework or on a project that can be improved.

04:07 Yeah. Well, I think that actually a lot of the complaints are starting to get addressed with

04:13 things like this, right? It's starting to get a little bit better. There's also some other

04:17 cool ones called JupyterLab-LSP for language server protocol or provider or something like

04:25 that, which also does a bunch of things that make it a little bit better. So yeah, it's

04:28 getting there. It's pretty cool.

04:29 Just one thing, the ultimate for general web developers is if you could take a Jupyter notebook

04:33 and just snap your fingers and have it be a Django site, you know, you can't quite do that.

04:37 But if I take off my technical head and just what would change things, like I'm surrounded

04:42 by a lot of scientists that turned a Jupyter thing into a website with standard crud, I feel

04:47 like it's possible one day, but we're not quite there.

04:50 That would be really fantastic. And we're going to hear like five ways we could do it that

04:53 we didn't know about from listeners, which is great. The one thing I would really love

04:58 to see in Jupyter notebook and maybe someone out there knows about it is I would like to

05:02 see collapsible sections. So I've got like a report and it's got like some markdown and

05:07 then some code and then a graph it's generated and then some more like maybe a picture and

05:12 then some more markdown. And the code in there is really awesome to have. But if you're

05:16 going through it as a report, you don't necessarily want to see the code unless you want to like

05:20 dig into it. So it'd be great if you could say these column or these cells are collapsed.

05:24 I really would love to see that, but I don't know about that yet.

05:26 That's opposite direction of what you're looking for. That's making the notebook more of an

05:30 article, less of a website. All right. So the next thing I want to cover is the PSF yearly

05:36 survey. Have you guys taken your yearly survey for 2020?

05:39 I've done it, sir. I've done it. I already got my homework.

05:42 Well done, Carlton. Well done.

05:43 I have not, but I've done it past years and I will. And we actually in the Django world,

05:47 we're inspired. We had our own survey this year, the first time in five years, because

05:51 I don't think Python does as well, but Django doesn't track anything. So we actually don't know.

05:56 No installs or usage.

05:57 Right. Okay.

05:58 And that's obviously helpful to fellows and technical team.

06:01 Of course. And I think basically the extent to which they track that is the analytics that come

06:06 out of PIP. Like pip was run on this operating system. This package was installed this many

06:11 times. It was this version of Python that did it. Beyond that, I don't think there's much

06:14 tracking in the Python world either. The broader, broader.

06:17 We have PIPI, but that's not completely accurate in terms of popularity.

06:22 All the Docker rebuild stuff that's happening all the time, like that counts, but it's not legit

06:27 and so on. So yeah, for sure. So if you haven't taken the PSF survey, I put a link in here. It takes

06:32 about 10 minutes. You should go do it. This is the fourth time they're doing this developer

06:37 survey and it's the major, a major, the major, I'm not sure, a major source for sure about the current

06:43 state of the Python community. So what editors do people use? What web frameworks are people using?

06:49 Are you a data scientist or are you a web developer, et cetera, et cetera? Are you just getting into Python?

06:55 And if you haven't seen the analysis of this before, I linked to the 2019 results, which were put

07:03 together by JetBrains and they did a really nice job of like making a compelling story to be told out

07:08 of it, right? Yeah, no, it's really nicely done and presented. You're like, oh, wow. Yeah, it's super.

07:13 That's why, you know, it's worth putting into because it's the production value at the end is great.

07:17 And so it's a valuable resource. I felt bad for a second that we didn't have that on Django, but

07:22 instead I just, we're not JetBrains. Yeah, we're not JetBrains. That is the gold standard. I was like,

07:27 oh, it'd be nice to have that. And then I was like, or I could just make it a Google form.

07:31 Well, you guys should reach out to the JetBrains team and see if they want to partner up.

07:35 Yeah. Well, they've probably got the resource in place. You know, they've got the infrastructure.

07:38 So maybe now have they always done prizes? I don't remember that they have,

07:43 but they have them now, right? Yeah. So that's cool.

07:45 Yeah. So they announced that a hundred winners, completely random. If you've completed the

07:50 survey, we'll receive the amazing Python surprise gift pack, which I have no idea what it is because

07:55 that would ruin the surprise. I saw some good Python socks on Twitter today. I hope it's got Python

07:59 socks in it. Oh my God. I love socks. Like that's half the reason I go to conferences. Let's be honest.

08:04 Got my Twitter socks. I got all my different socks. I got my MongoDB socks.

08:08 I've used to go just for t-shirts, but now I kind of like the socks more.

08:12 I do too. We've added a official Django merchandise store, and there's some items on there and that's been helpful with the virtual conferences,

08:18 but we don't have socks. So the, you know, there's a lot of inspiration we can take for

08:22 having better official gear out there.

08:24 Absolutely. Get your sock game on.

08:26 Yeah. And then stickers, right? It goes t-shirts, stock, socks, stickers,

08:29 I think in the hierarchy of swag.

08:32 That's right. That's right.

08:33 Okay. I have to up my game. I'm just giving out stickers usually.

08:36 I'm an enamel pin man myself.

08:38 Oh, enamel pins.

08:40 Oh yeah.

08:40 I should mention JetBrains, which is doing that. That survey is a big sponsor of Django. We do a

08:48 couple of week long thing every year and they're a major corporate sponsor of Django. So shout out to

08:53 JetBrains.

08:53 Very nice. Yeah. Very nice.

08:55 And one of the things I'm really excited about is we have a new sponsor and it's another podcast.

08:59 Yeah. So that's pretty cool. So this episode is brought to you by tech meme,

09:03 right? Home podcast for more than two years and nearly 700 episodes. That's amazing.

09:09 The tech meme right home has been a Silicon Valley favorite tech news podcast. The tech

09:14 meme right home is a daily podcast, only 15 to 20 minutes long. And every day by 5 PM Eastern,

09:21 it's all the latest tech news, but it's more than just headlines. You could get a robot to

09:26 read you headlines, but the tech meme right home is all the context around the latest news of the day.

09:32 It's all the top stories, the top posts, the tweets and conversations of those stories,

09:36 as well as behind the scenes analysis. It's like a TLDR as a service. The folks at tech meme are

09:43 online all day reading everything so they can catch everything up for you. Search your podcast app right

09:49 now for right home and subscribe to the tech meme right home podcast, or just visit pythonbytes.fm

09:54 slash ride to subscribe. Yeah, it's like Python bytes, but just for general tech. Every day,

10:00 though, these guys don't mess around. That's incredible. Yes. All of us who are podcasting,

10:05 we're like, Oh my gosh, that's insane. Yeah, we all wince a little bit hearing 700

10:09 promised by 5 PM. Sounds like a burnout algorithm.

10:13 It's well done. But yeah, cool. Thank you guys for sponsoring the show. Will, what's your item?

10:19 Oh, so my item is from prototype to production in Django. So this is a common thing where you get a

10:25 little familiar with Django and you say, well, what's the there's this big chasm basically from

10:30 building a CRUD app locally and deploying it properly without being hackable.

10:34 What do I do? I run it as a root. I leave the debug setting on what else do I do?

10:39 That's important. Yeah. Well, it's sort of like you don't know what you don't know. And then the

10:45 older you get, the more scared you get because you've seen it all go bad. But when you're starting

10:48 out, you're like, what could go wrong? Yeah. But as soon as it works, it works.

10:51 Yeah. So specific to Django, I think like most web frameworks, it has to, it focuses on local

10:57 production. So when you run, you run a start project command and it creates some scaffolding for you.

11:01 And then specifically, it has a settings.py file. That's kind of the global config. And that's set

11:07 for local development. So works great locally. But if you just dump it into production, you're going to

11:12 be wildly insecure and easily hacked. And so it's a quick list of things you want to change. And Carlton,

11:18 please jump in here as the Django fellow if I missed something. But debug is a setting that is you want to

11:23 switch to false that provides very nice error message. But it also has a roadmap to hacking your

11:28 site if it's left on. Right. These are in settings.py generally. In settings.py. So it's all about

11:33 settings.py. Basically, there's a secret key that Django that is 50 character long string randomly

11:38 generated. You want that to be secret because it uses a hash throughout Django. And of course,

11:42 what happens is you do one git commit and then it's out there. So you need to change that or really put

11:48 it into an environment variable, which I'll get to in a sec. Are you familiar with shgit? No, I'm not.

11:52 shgit. So this is super scary. This is a, oh my gosh, it's live right now. I can't believe it.

12:02 So I think it's at shgit.com. But there's also the open source version that you can get. You can see

12:08 it on GitHub. Let me just read you this title just to like point out how seriously this should be taken.

12:14 Shgit finds committed secrets and sensitive files across GitHub, GIST, GitLab, Bitbucket,

12:19 and your local repos in real time. It does this by subscribing to the like commit stream on GitHub

12:25 and instantly post the secrets like AWS secret keys and stuff. You can see if you go there like,

12:31 yeah, I see six for Django right now. And it's all just got two more. Yeah, I just got two more,

12:36 three more, five. I just got five more. I mean, it is insane. If you think my repo is not so popular,

12:42 it will be fine. It may not be so fine.

12:45 But this is the thing with security, right? Is that it doesn't matter how small you are,

12:49 because the people who are attacking you, they're using automated scripts. So they're checking every

12:53 port on every addressable server with every known weakness. It's not if you'll be hacked, it's when.

12:58 Now setting the stage how frightening this is. Carry on why we shouldn't put that in.

13:02 Well, I think it just reinforces that the settings up PY file is where most of the action is in Django.

13:09 And you want to be careful with it. I mean, I remember GitHub back in the day,

13:13 you could just global search for AWS keys and stripe and everything. Now, at least you can't

13:17 global search for that stuff. And they'll even ping you. So for me, like I have some secret,

13:21 I have some projects in my books that are on GitHub. And there's a secret key there. And they

13:25 bug me all the time saying, Hey, you have a secret key exposed. I'm like, Yeah, I know I do. I don't,

13:29 it doesn't matter. So it's gotten better. But yeah, it's still all out there. So secret key,

13:34 keep that secret. Allowed host is probably the last big one. These are the hosts that can come in.

13:38 Django will prompt you to change that. So if you're using Heroku, and it's myapp.heroku.com,

13:45 you want to set that host to be only that host, not all hosts can come in. Database is the next one.

13:50 So by default, Django has a SQLite database, file based really easy to use.

13:55 Fantastic for production, large scale, Facebook uses it.

13:58 You know, it can be.

13:58 No, I just do.

14:00 If your workload is read only. So say you're running a content site, and it's just you editing

14:04 it. SQLite will go all the way with you. But as soon as there are more than one editor in it.

14:09 It's incredibly fast. It's in memory, right?

14:11 Yeah, you can have it in memory, we can have it on the file. But on read only workloads, it will go,

14:15 you know, right out there.

14:17 Sorry, I meant in process, in process, not in memory, but it's not like a separate server, right?

14:20 Exactly. It's just a file next to all your other files. And it can it can hold,

14:24 you know, terabytes of data without a problem. But as soon as you've got multiple users logging in at the

14:29 same time, or that kind of thing, then you need, you know, something like Postgres or MySQL can

14:33 handle that kind of concurrency.

14:35 Yeah, so you probably want to .gitignore your SQLite file. But also, you definitely want to use

14:41 whatever using in production locally. So Postgres, MySQL, MariaDB, and Oracle are the supported

14:47 databases. What else almost done here, you configure your static and media files. So static would be

14:53 images, JavaScript, media refers to anything that's user uploaded, you definitely want to be careful

14:57 whenever you have anything from users can't trust them, you want to use Django forms, you probably

15:02 want to use bleach to sanitize and you want to have that on a CDN, not on your server. Two more to finish

15:08 up. So the admin Django has a famous admin that's very powerful, which is at slash admin, you should

15:15 change that away from dot admin, because to Carlton's point, there are bots searching for Django sites at

15:20 slash admin, and they will come in and hack away at your site. There are a number of fun technical things

15:25 you can do to honeypot it or this and that, but you should just change it away from dash admin.

15:31 I'm tempted to know there's a very famous Django site that still has slash admin, but I won't mention it

15:36 publicly. Carlton and I both use it though, for our work. And then the last thing is user registration. Django

15:42 comes with login, logout, reset, but it doesn't have a signup. So you can roll your own or most people would use a

15:49 third party package called Django all off that's fantastic that has social support. So Django has very robust third party

15:55 ecosystem that over time, the most popular ones, or the strongest ones are rolled into Django. But there's also

16:00 separation because it's too much for Django to maintain. Django all off is not is a third party package, but

16:06 it's, in my view, effectively part of Django for most people. So those are the big ones are the key things

16:11 I mentioned there environment variables, it used to be with the settings file that back five years ago,

16:16 you'd have multiple settings files, you have a base settings file. Carlton still doing that?

16:20 I still have multiple settings files go with those files. I mean, you have environment variables too,

16:24 but multiple settings files for the win.

16:26 He's a Django fellow, he doesn't know what he's talking about. You need to use environment variables

16:29 for this, because then you have one settings file and you load it into, you know, local staging or

16:33 production. I've much easier.

16:35 Spaces.

16:35 But it works for Carlton.

16:36 Spaces.

16:36 And there's a number of third party packages that will help you with that. I like environs,

16:39 which will be linked there, which also has DJ database URL, which is a nice feature on environment

16:45 variables for databases. It just means you have single settings file, and you switch

16:49 with environment variables. There's also Django has a handy deployment checklist, which I think a lot

16:54 of people don't know about. We have a link to that you can run Python check --deploy.

16:59 And it will check that all the things I mentioned, plus a number of HTTPS things are actually

17:04 configured properly. So you don't want to deploy your site. If you don't pass most of those,

17:09 if not all those. Yeah. And that's, you know, there's testing, logging, performance security,

17:14 you can go on and on and on. I wrote a whole book Django for professionals on this. But those are the

17:18 highlights. And there's a check, there's a good doc on the Django docs for deployment checklist,

17:23 which, you know, you should open that every time you review.

17:26 Yeah, the hard thing is, there's like a couple of must haves, like the ones I listed there. And then

17:31 there's a lot of, it depends, nice to haves. And that's where it's harder to make generalizations.

17:36 This stuff is so rewarding when you get it right to see your site up and running, you know, 99% plus

17:42 uptime and people using it. So fantastic, highly responsive. But soon as you see like something go

17:48 wrong, it just your heart sinks. And so most of those things are because like you both have hinted

17:54 out there's some kind of bot that's looking for some vulnerability and like a known thing.

17:58 And just make sure you don't put those known things in front of the internet.

18:02 Yeah. And I should say, actually, there used to be a site called Pony Checkup that has,

18:07 that you could put in your URL and would automatically just test a lot of this for you.

18:11 It's actually someone has taken that over from the original maintainer. So it's now DJ Checkup

18:15 slash Pony. So you can type in your URL and check. That's kind of a good way if you're a beginner,

18:21 if you're not sure. There's nothing like going to a webpage and seeing security issues in your site

18:26 or others to kind of scare you into doing something.

18:28 Yeah. Yeah, for sure. Awesome. Well, I'm glad you covered that. And Carlton,

18:33 it sounds like the one that you got is a bit of a...

18:35 Similar topic, actually.

18:36 Similar, yeah.

18:37 So I've been thinking about, but I think it must be Django chat. Every week we,

18:40 it seems, we have a guest on and we end up talking about deployment and it's massively complicated.

18:44 And Will's just gone through a whole list of things and that's only the tip of the iceberg kind of thing.

18:50 And the thing that I got thinking about was that there is this deployment gap. So I imagine someone

18:54 finishing the Django tutorial, finishing the REST framework tutorial, or finishing Will's Django

18:58 for beginners book. And then how on earth do they get their app online? And it's like, you know,

19:03 unless you're going to dedicate a week or two weeks full-time researching and trying and following

19:08 tutorials online, it's like this chasm. We can't do it. And so platforms as a service like Aroko or

19:14 App Service or App Engine or DigitalOcean have got their new one. They look like a great starting point because they're

19:19 kind of easy. But in a way, for me, they're a kind of cul-de-sac. You go into them, you get to the end, and then you

19:24 kind of have to go back out again when you want to do something more advanced. But then on the other hand, you've got this

19:29 do-it-yourself option of provisioning servers and setting up firewalls and virtual private clouds.

19:34 It's just way too much. It's scary, right? And then you read some blog posts and it's saying, well, you've got to do it

19:41 with microservices or you've got to do it with this container orchestration platform. And no, no, no, it's too

19:47 much for me. So I've been thinking about this and trying to come up with a story of my own for it,

19:52 and which I'm launching next year. It's going to be called Butter, and it's going to be a little tool that just

19:57 wraps it up and tries to take some of the fuss out of deployment. So that's not ready to go yet,

20:02 but I wanted to mention it because it ties into what Will was talking about. And you can sign up

20:06 for early updates on btn.dev, button.dev, btn.dev. So that's my topic.

20:12 That's a great topic. I think I personally struggled through this, right? I started out trying to run my

20:18 websites in Python on some pass place that was very simple and easy to get started, but there's just,

20:25 there's a ton of downtime and things weren't working the way that I was really exact, you know, hoping.

20:30 And I ended up having to do a lot of things anyway. And so I finally bit the ball and figured

20:35 out how do you run micro WSGI safely? How do you keep these things up? How do you do zero downtime

20:40 deployment? How do you do continuous? So there's just so many.

20:42 And how do you keep updated, right? How is it? So, okay, you get it set up and it's fine. But then

20:47 six months later, there's a security patch, which you don't quite know how to apply without bringing

20:51 your whole site down and rebuilding your server. It's like, how do you deal with those problems?

20:55 They're not something you can learn quickly or easily. Yeah, absolutely. Brian, you got to deploy

21:00 things? I have before. And that's why I don't anymore. I mean, yeah, last time we talked about

21:07 DigitalOcean's new pass service and you're like, I'm all about this. Like, yeah, yeah. Like for instance,

21:12 for me. So you did your own like Python bytes and talk Python, you did those applications, right?

21:19 Talk Python, Talk Python Training and like 10 other like little APIs and stuff that I'm running all

21:23 those. Yeah, right. So I've done I mean, I've built websites before, like in the way past like decades

21:29 ago. And then when I wanted to do a podcast, I did a Python. The testing code started out as a,

21:36 as just part of my blog. And it was like on WordPress or something like that. Now I will go with a

21:42 fireside thing. And I don't, I mean, fireside is a good service, but it's not, I mean, it's not ideal.

21:48 It doesn't, it isn't perfect, but I don't have to think about it. What I want to do is podcasting.

21:53 I don't want to maintain a website. So there's a lot of things where we do need to build these custom

21:58 websites. And I'm glad that there's some attention given to this because yes, I can learn how to do a

22:03 Jenga website, but going from there to a live site is horrifying. So yeah.

22:10 I literally just spent an hour this morning, maybe hour and a half before like up to 10 minutes before

22:16 we started recording, deploying my first FastAPI endpoint through G unicorn, UV a core and behind

22:24 engine X and all that stuff. And a lot of it was the same, but that's the first time I've done it with

22:28 UV a corn and the settings are a little bit different than say micro WSGI. So I can run it with some basic

22:32 stuff. And I'm, I kind of lived in that world of like, I got a, what is the config key to make sure,

22:37 or the config settings that make sure that it runs as a different user, not root again in this server.

22:41 And just like, you just go through all these things and having that like really automated would be

22:45 great.

22:45 Oh yeah. I mean the other day, my site went down for half an hour and it turned out to be the DNS in

22:50 the end, but I never thought the DNS would go down. How often does that happen? So I log into the server

22:55 and I'm checking the application server. That's fine. So I checked the local engine X instance. That's

23:00 fine. I checked the load balance. So that's fine. And I'm like, it is, it's the DNS. And then by the time I've

23:04 worked this out, the DNS comes back up and the site's back up and it's, ah,

23:07 and what I want to do is put all that fully automated, you know? So I just run the diagnostics

23:12 and it goes green, green, green, green, green, red. Ah, there's the problem. Okay, fine.

23:16 That's awesome. Let me know when you got that. I'm excited.

23:18 21 btn.dev.

23:20 Okay, cool.

23:21 Yeah. Well, and Carlton, I think your cul-de-sac analogy, which I haven't heard you say before,

23:25 that's exactly right.

23:26 Yeah.

23:26 Because it sounds, you're like, oh, this will solve my problems. And then you learn

23:30 everything and then you kind of come back with a different problem. I mean, part of the problem

23:33 with doing server stuff, I think is that it's, it feels good to an engineering mind, right? It feels

23:39 good to be like, you know, drive manual and tweak things, but then you've lost weeks of time and

23:44 your app looks the same. And, you know, so there is a danger there in terms of, I feel like you kind

23:49 of need to do it once or twice and then be like, okay, okay. I'll just have, like, I'll just have

23:53 Carlton handle it for me. Like I trust him more than.

23:55 This is the via media, right? This is between the sort of the platform and service, which perhaps

24:01 doesn't cover all your need cases. And then the over-engineered, you know, I'm using container

24:06 orchestration for a Microsoft thing and I've only got one server and a thousand hits a month, right?

24:10 It's, you know, there's a middle way between that.

24:13 AWS will never be bothered to make it friendly to small people because they don't care. I mean,

24:18 they have this, you know, big clients. So yeah. Yeah. Yeah. Very cool.

24:22 I also like the cul-de-sac thing. So you think it's an on-ramp, but it's really a cul-de-sac.

24:27 Yeah. You're just in an infinite loop of...

24:31 But that's the thing, especially somebody who's in that deployment gap scenario that I talked about,

24:35 they go down the platform and service route because that's the only thing they can do. But

24:39 at the end, they're like trapped there. And I want to do more, but I've got to go all the way back

24:42 down here to learn this other stuff, which is so hard and so scary and so overwhelming.

24:48 Yeah. Well, I want to talk about something that's easy and well, it's sort of easy. It's

24:52 contributing to other people's open source projects.

24:55 That's not easy.

24:56 You're just petting Carlton's beer.

24:59 No, I'm really excited about this topic.

25:02 So I ran across a thing called all contributors. And actually, we talked about NBQA before.

25:08 And that's where I got this idea because they use this tool called or the service called all

25:14 contributors. And it's sort of a service, but also just a spec. So I'm just going to read it. It says

25:20 this all contributors is a specification for recognizing contributors to an open source project

25:25 in a way that rewards each and every contribution, not just code.

25:30 The basic idea is to use the project read me to recognize the contributors of members of a project

25:37 community. The idea is like, there's a lot more stuff going on than just code. There's things like

25:42 documentation, design, writing examples, doing maintenance, writing plugins for things, doing

25:47 podcasts, giving talks and all that stuff. And it'd be cool to recognize all these people.

25:53 So they've got this spec for kind of how to do that. But then there's even more. There's a cool

25:58 emoji key, which I love this.

26:00 I love the emoji key part. It's so friendly.

26:02 Yeah. So it has recommended emojis to use for your contributors list that includes things like a

26:10 little laptop computer for code, a little thing for documentation, design, examples, all the sort of

26:16 things you'd want with like fairly good. You don't have to think them up. They thought them up for you.

26:20 So it's nice. Yeah. They also have like this bot that you can attach to your GitHub repo.

26:25 So you can just add comments into somebody's pull request or something to say, hey,

26:30 all contributors, please add this person to the contributors list or something. And it just does

26:35 it. So.

26:35 Okay. That's super cool.

26:37 We're looking into that with Django. A few weeks ago, I've been reading, I was reading,

26:40 Will was reading, other people were reading the working in public book. I, for the life of me,

26:44 I can't remember the author's name this moment, but.

26:46 It's Nina, I think.

26:47 Okay.

26:48 Eggbar.

26:49 But it mentions this all contributors thing. And one thing we've got with Django, we've got a

26:53 massive contributor base, but we kind of only historically recognized that the sort of 30 or

26:58 so people that were in Django core. And then over the course of the last couple of years,

27:02 we've kind of tried to restructure the governance and we've managed to do all of that. And now we're

27:07 in the position where we want to try and recognize all the other people that we translate the docs

27:10 into however many languages that that translation team gets virtually no recognition. Let's recognize

27:16 them. There's yes, there's code commits, but there's also all the people that help triage

27:20 and review the tickets and review the pull requests. And, you know, those people need recognition.

27:25 There's the people who organize all the Django cons. Those people need recognition. We really want to

27:29 try and show that like Django isn't just, you know, I committed to Django Django. It's the whole ecosystem.

27:35 So I think this all contributors thing is great tool.

27:38 You said you're reading the working in public book by, would you recommend it? Is it good?

27:42 Yeah, it was amazing book. Like just the first few chapters, it's just like describing Django to a T. It's like, oh yeah, this is the challenge we face every single day. Like a couple of years ago at Django con in San Diego, I gave a talk about your web framework needs you. And I put up a graph of contributors. And in the first chapter, it's this exact same graph. I mean, it's got different numbers and it's for a different repo, but it's the same power law shape. It's the same problem. It's like, this isn't just Django. It's every open source project out there. It's got the same issues. And it's the same time.

28:12 Do you grow in on yourself and get smaller and more enclosed? Or do you open out to the community and welcome contributions and find a way of doing that? And if you can, you can survive and flourish. And if you can't, well, you'll weather and die.

28:26 Yeah. I mean, I literally took screenshots of the book because I was like, Carlton, you should read this. He's like, and he's like, okay, okay. It's exactly my thing. And this all contributors is so relevant because the most Carlton mentioned at Django, we're changing around what Django core refers to. And I literally have a,

28:42 huge spreadsheet with all the various people we're trying to categorize that this would fit in perfectly for. So I'm going to potentially use this.

28:50 And it has a GitHub bot, which is fantastic.

28:52 Yeah. What can be done manually and you know, five minutes can be automated in an hour. So that's right.

28:56 Well, Michael, what are you going to finish this up with?

29:00 Well, you know, I want to keep us on the move. Don't sit still, you know, rolling stone gathers, no moss sort of thing. So pandas is a super popular library in data science, right? And there's a bunch of visualizations. One way to work with geospatial data is with geo pandas, which is cool. So there's a library called moving pandas. And the idea is if you give locations plus time, you can map all sorts of interesting things.

29:29 things and analyze all sorts of interesting things. It's cool, right? And this project, as it should, has a bunch of animated GIFs.

29:36 So yeah, that's what we need to tell exactly what it's about, right? Get in there. So it provides trajectory data structures and functions for analysis and visualization. It started out as this QGIS plugin, but they decided it made more sense to just be its own thing. So it's its own thing.

29:54 Basically, it takes a geo pandas geo data frames with timestamp points, and it converts them into moving pandas trajectory collections.

30:04 And you know, properties such as speed and direction, you can turn continuous observations into trips, like, I was here for a really long time, and then I went to the store, and I was there for a while, and then I came back, right? That kind of stuff, I think.

30:18 It'll aggregate them into flow maps. So instead of, I went exactly from here to here, you can say, here are the nodes where I spend time and the paths I take between them.

30:28 Almost like graph theory type of stuff. And work with it is super straightforward. So you can just go create a pandas data frame, pass up a geometry, and a time, and you convert it to a geo data frame.

30:40 And then you just say, give me the trajectory, and you can plot it. And that's it. Like, incredibly simple.

30:46 That sounds super. Their website's really good as well. I was just clicking on it to see the animated GIFs and whatnot.

30:51 Well, it seems like that would overlap with... So, I mean, Django is a big framework. There's a whole geo Django area, which Carlton and I have discussed.

30:59 I mean, that and the ORM are the two parts of Django. I kind of don't really know, to be honest.

31:05 But they're very powerful, and people use them, so...

31:07 Cool. You can also get, like, graphs of kind of derived data. So, like, speed over time, rather than just position. You can get these other analysis in there.

31:19 And, you know, I can see lots of interesting places. You know, I had Kane Replical on Talk Python to talk about how they're using Python on the race teams for simulations and stuff.

31:29 And, like, those types of analysis, this seems so perfect. Go spend a day on the track, collect a bunch of data, throw into these types of things, and look at the curves and whatnot.

31:37 Yeah, and Pandas is almost like the kind of data transfer format now. I mean, you know...

31:43 Yeah, absolutely.

31:43 So, to be able to integrate there sounds super useful.

31:46 Yeah, that's cool.

31:47 Awesome.

31:47 All right, well, that's the last item. Brian, you got any extra stuff you want to share with people?

31:51 No, just working and plugging along.

31:53 Yeah.

31:53 How about you?

31:54 I do have a few things. First of all, I was talking to Hugo Bowne-Anderson from Coiled, and he asked,

32:02 hey, when is the transcript from our recent show going to be out? I'm like, eh, I don't really have transcripts at the moment.

32:08 The company I was working with to generate them stopped generating, stopped doing that kind of stuff, and I haven't figured out what to do.

32:14 He's like, oh, you should check out otter.ai. I'm like, yeah, but isn't that, like, for your phone?

32:18 And, like, you're going to have conversations. It's cool.

32:20 But what I realized is I can upload files to that, our old episodes. It'll convert it to mostly correct transcripts.

32:28 Like, pretty good, actually. You know, it'll get, like, AWS, right, and things like that.

32:32 And then just wrote, yeah, I just wrote some automation to turn that into transcripts.

32:37 So, I added, like, half a year worth of transcripts back, which means that feeds our search engine, so search should be better as well, and stuff like that.

32:44 Yeah, I think it's the top one. I think Wes Boss was also asking something, and I was tweeting with him saying, yeah, otter, I think, when I checked, is by far the best one.

32:52 It's not really designed for transcripts. It's designed, I think, for, like, meetings and groups, but it works. We've been using it for a year.

32:59 Yeah, that's awesome. And you guys like it? It's been good?

33:02 Yeah, as you say, it's, I mean, it's the most accurate out there, and usually it gets almost everything. You can kind of have custom things, like AWS, if it gets it wrong.

33:10 And, yeah, I mean, usually I run it through and give it a quick scan. Maybe there's a couple things to switch, but, yeah, it's a no-brainer.

33:18 Yeah, it's got, like, a nice editor that, like, plays and highlights the words if you were actually going to stop and edit them.

33:23 I also have automation, like, for my courses, I have automation through AWS Elastic Transcribe, or just Transcribe, whatever it's called, to generate those and then hand them off to people.

33:33 But otter's looking nice. I'm not sure if I'd switch the courses over. But, anyway, maybe we have a bunch of transcripts, Brian.

33:38 Yeah, so I've never really done, I started doing testing code transcripts, but I was actually just paying somebody to do them, and it was getting expensive. So, yeah, let's check this out, too.

33:49 Yeah, it's worth checking out.

33:50 It's the SEO that really matters, I think. That's, like, the killer feature.

33:53 Yes, exactly. That's why I first created them, and I thought, okay, I'll make them searchable so people can also get some value out of it.

34:00 But my original reason for doing it was, like, instead of having three paragraphs of content for an hour conversation, let's have the real conversation.

34:08 Right?

34:08 Yeah.

34:08 But then, you know, someone will find that, and that will be useful. And, you know, they'll be like, ah, this is, you know, even if it's badly transcribed, this is roughly what I'm looking for, I'll listen to the episode.

34:17 Right, right. Yeah, let me listen, here's the timestamp, and they can get some value out of it. So, hopefully, yeah, that's the idea.

34:22 Nice.

34:23 All right. Also, I'd switch from Google to try to live in DuckDuckGo land, just using DuckDuckGo.

34:27 Oh, join us, yeah.

34:28 Yeah, are you guys doing it? Are you liking it?

34:30 I've been there for three years exclusively.

34:32 Oh, God, I'm still on Google, but I did install a pie hole this week, so, you know.

34:38 Oh, yeah, yeah, yeah.

34:39 Swings and roundabouts.

34:41 Carlton's like, I see your DuckDuckGo usage, and I got to...

34:44 I'll raise you a pie hole.

34:45 This is what I have to deal with, guys.

34:47 So far, I'm liking it. I mean, I've been using Firefox with all sorts of privacy stuff for a long time, but I figured just one more thing.

34:55 And I just want to point out, if you're trying to be, like, slightly less connected to Google, they have Google Takeout, or you just want to back up, right?

35:01 If you use Google Drive and you sync your Google stuff, it'll give you just a link to the spreadsheet or whatever on Google.

35:07 Well, excuse me, if you use Google Takeout, it'll actually convert, like, say, your spreadsheets to Excel, so you actually have them.

35:13 Anyway, so that's part of that.

35:15 And, like I said, I got to deploy my first FastAPI app today, basically.

35:19 And I'm just... I'm really enjoying it.

35:21 I feel like it's bringing in a lot of these ideas.

35:24 I'm hoping maybe you guys can just comment super quick on this.

35:27 It brings in so many of these new ideas into the web space, like the async and await stuff feels super natural.

35:33 You don't have to do anything to make it work.

35:36 The type annotations mean things.

35:39 I just feel like there's a lot of interesting sort of modern Python stuff coming together here.

35:43 Like, what's your Django perspective?

35:44 Well, so FastAPI is built on top of Starlet, which is by Tom Christie, who's Django REST framework creator.

35:50 So from the async side, that's what we're trying to build into Django now.

35:54 And we have async views in 3.1, and, you know, we're working on the ORM next.

35:58 And then from there, it will spread out.

36:00 So there's a PR came in this week about the cache layer.

36:02 So there'll be async, third-party async cache backends for Django soonish.

36:07 An async, fully async framework like Starlet, it's always going to be out there.

36:13 You know, it's going to be ahead of where Django is.

36:14 But we'll wrap it up and give it that nice Django feel where you define your class-based views and all these things.

36:20 We're not there yet, but that's what we're aiming for.

36:22 Then the other thing that FastAPI brings out, which is quite exciting, is Pydantic, which is the type hinting used for the serializers and for the validation.

36:31 That's kind of really cool.

36:32 And at the moment, we don't have a story there with Django.

36:35 We've got Django Forms.

36:36 We've got REST framework serializers doing the same kind of thing.

36:38 But we've got our eye on that, and we'll see how it goes.

36:41 I know you guys are definitely thinking about these things.

36:43 It's very exciting.

36:44 Yeah.

36:44 I mean, but what's really nice about the current, particularly the ASGII world, where all the kind of ASGII is the standard, is there's an amount of interoperability in that you can kind of nest your ASGII apps inside each other and wrap window bears around them.

36:56 And it's just another ASGII app.

36:58 And so, actually, there's a lot of interop things.

37:00 So it's a really rich and fertile time for Django web frameworks.

37:04 Yeah, awesome.

37:04 I don't have anything to add, Carlton.

37:05 I defer to Carlton.

37:06 Other than it's sort of wild that, you know, I mean, from Tom, we've known about Starlette that FastAPI is better known than Starlet.

37:14 It's a little strange to me, but it makes sense.

37:16 Because Tom's been busy rebuilding everything in async the last couple of years, kind of on the side.

37:20 Well, the thing also, the thing is that people touch FastAPI.

37:23 They just live on top of it.

37:25 Like, they live on the shoulders of Starlet, but they touch FastAPI, right?

37:28 Yeah.

37:28 No, exactly.

37:29 Yeah.

37:29 Yeah.

37:29 Yeah.

37:29 All right, Brian, I put in two jokes that we can offer up today.

37:34 Let's shout out to some of the stuff that Will and Carlton are doing, though, first.

37:37 Yeah, yeah, absolutely.

37:38 Okay.

37:39 Well, we just, I'd say, listen to Django Chat, which is at DjangoChat.com.

37:42 That's our podcast.

37:43 It's fortnightly now.

37:44 That's a fancy British word there for you.

37:46 Check out Will's tutorials and books at LearnDjango.com.

37:50 And then, yeah, sign up for the early updates on Button, which is at BTN.dev.

37:54 Just did.

37:54 Super.

37:54 Welcome aboard.

37:55 You'll be subscriber number three, I think.

37:57 Well, actually, I had to tell Carlton, I was like, get up a page before we go on the podcast,

38:03 because he's been telling me about Button for a year.

38:05 Yeah, it looks good.

38:07 Well, it looks like a sign up form right now.

38:09 Well, yeah, that's what it is right now.

38:11 But yeah, 2021, it's coming.

38:13 Nice.

38:13 Yeah, great.

38:14 Okay, now a joke.

38:15 Thank you.

38:16 Thank you for getting us back on track, Brian.

38:18 So you've heard about give a person a fish versus teach them to fish.

38:21 There's a programmer version.

38:23 Did you know that?

38:24 No.

38:24 Yeah.

38:24 If you give a person a program, you can frustrate them for a day.

38:27 But if you teach them to program, you can frustrate them for a lifetime.

38:30 Yeah.

38:30 Yes, definitely.

38:33 Unless what, Brian?

38:34 Unless you teach them to test at the same time.

38:37 Ah, very good.

38:38 Exactly.

38:40 And speaking of FastAPI, here's a joke that I just saw that's relevant.

38:46 That's sort of similar to one put out by Sebastian Ramirez from FastAPI.

38:51 So somebody just failed a job interview and the verdict was delivered like this.

38:57 I'm sorry.

38:58 We're looking for someone aged 22 to 26 with 30 years of experience with Flask or Janko.

39:03 Yay.

39:04 Well, didn't he tweet about someone was looking for five years of FastAPI?

39:08 And he was like, even I don't have that.

39:10 He's like, well, as the creator of FastAPI, I would not qualify you for your job having done only 1.5 years of it.

39:17 Yeah.

39:17 Yeah.

39:18 Well, I don't think people realize that it's so new.

39:20 I mean, it's kind of taken over very quickly, but it hasn't been around for very long.

39:25 Yeah.

39:26 Yeah.

39:26 It's pretty interesting.

39:27 Anyway.

39:28 All right, guys.

39:28 Yeah.

39:29 Thanks for joining us.

39:30 Yeah.

39:31 No, thank you.

39:31 It's been a really cool chat.

39:32 I really enjoyed it.

39:33 Yeah.

39:33 Thanks for having us.

39:34 Yeah, it's been fun to have you here, Carlton.

39:35 Well, all right.

39:36 Bye.

39:36 Okay.

39:37 Bye-bye.

39:37 Bye-bye.

39:37 Thank you for listening to Python Bytes.

39:39 Follow the show on Twitter at Python Bytes.

39:42 That's Python Bytes as in B-Y-T-E-S.

39:45 And get the full show notes at Pythonbytes.fm.

39:48 If you have a news item you want featured, just visit Pythonbytes.fm and send it our way.

39:53 We're always on the lookout for sharing something cool.

39:55 This is Brian Okken, and on behalf of myself and Michael Kennedy, thank you for listening

39:59 and sharing this podcast with your friends and colleagues.