Transcript #213: Uh oh, Vulcans have infiltrated Flask

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 213, recorded December 23rd, or Anthony, Anthony Shaw here.

00:12 Is this December 23rd or 24th? You tell me.

00:14 It's the 24th, it's Christmas Eve.

00:16 Yeah, of course.

00:16 Yeah.

00:17 In Australia, awesome.

00:18 In the future.

00:19 Yeah, 23rd for us, Brian and me, here in the US, 2020.

00:25 And yeah, this episode is brought to you by us, so we'll talk more about that later.

00:30 And I'm Michael Kennedy.

00:31 And I am Brian Arkin.

00:32 And Brian, we got a special guest here, friend of the show, Anthony Shaw.

00:36 Welcome, Anthony.

00:36 Hi there. Great to be on.

00:38 Yeah, it's good to have you here. Thanks for taking time out of your holiday.

00:42 Yeah, no, it's a pleasure. It started last night, so the Christmas holiday has now started.

00:47 I'm off for two weeks. It should be lovely.

00:48 Yeah, it should be very lovely.

00:50 So, you've got, I think everyone on Twitter is jealous of all the pictures that you post

00:56 when it's like cold and gray.

00:58 And you're like, oh yeah, there's this beautiful sunny beach here in the summer in Australia

01:01 that I happen to be suffering through.

01:03 Yeah, it is.

01:05 Yeah, it's the middle of summer here and it's, yeah, beautiful weather and five minute walk

01:10 to the beach.

01:10 So, yeah.

01:11 Yeah, yeah, yeah.

01:12 Not too bad.

01:13 Not too bad.

01:13 Cool.

01:14 Well, welcome to the show.

01:15 Happy to have you here.

01:16 I want to kick us off with this project called Django Ledger.

01:21 And Django Ledger is, have you guys heard of this?

01:23 No.

01:24 Yeah, so Django Ledger, you've probably heard of QuickBooks or FreshBooks or some of this accounting

01:30 software, right, that you've got to work with.

01:32 You know, it allows you to do either it's a desktop app or some kind of online thing where

01:37 you create purchase orders, you keep track of who your customers are, when they owe you money,

01:44 you accept payments, all that kind of stuff, right?

01:46 So Django Ledger is something like that built in Django, which I think is really cool because

01:52 it could serve multiple purposes.

01:53 One, you could take it and just run it for yourself or for your company and then customize

01:58 it.

01:58 Or you could actually use it to extend something, you know, build something on top of it, or maybe

02:04 even offer services in it, right?

02:05 If you're like Stripe, maybe it makes sense to integrate some sort of plug in here because

02:10 then you get 3% of everything that company makes basically, that's where credit cards

02:14 go, right?

02:14 So it's a bookkeeping and financial analysis engine for the Django framework, which pretty

02:19 sweet.

02:20 It's open source.

02:20 And if you look through its features, it has a chart of accounts and basically customers,

02:26 financial statements, has multi-tenancy support, has stuff for operations, for investing, has

02:33 bills and invoices and bank accounts, all that kind of stuff.

02:37 And yeah, pretty awesome, right?

02:39 Wait, multi-tenancy?

02:40 So is there like North Tennessee and South Tennessee or?

02:43 That would be Dakota.

02:44 I think Tennessee, there's only just one tenant.

02:46 No.

02:46 Yeah.

02:48 I think if you wanted to run this as like a platform as a service type of thing and you

02:52 wanted to offer up, if you wanted to basically create your version of FreshBooks, I feel like

02:56 that's what it would be.

02:57 If you're an accountant that serviced multiple clients, you'd probably do it that way, I

03:03 guess.

03:03 It looks pretty cool.

03:04 I thought there's a project called Ledger, which is open source, but it's not a, I don't

03:09 think it's a Python project.

03:10 I think it's written in something completely different.

03:12 So yeah, this is really cool and great, I think, for businesses who maybe have someone

03:18 who can set this up.

03:19 I'm guessing it's not available as a service.

03:21 You have to kind of...

03:23 I don't think so.

03:23 Especially, yeah, yeah.

03:25 If you look at it, it says this project is under active development.

03:28 It's not quite ready for production.

03:30 So this comes to us from Miguel Sanda.

03:34 I believe that's his project.

03:35 And I would love to see what the roadmap for stability is and when this is ready, when it's

03:41 coming out.

03:42 But it's basically all Python and HTML and just a tiny bit of TypeScript, like a salt

03:47 level.

03:48 Stuff that's coming is like inventory management, cash flow, taxes, all the fun stuff.

03:54 And Brian, even BDD is coming.

03:56 I don't understand.

03:57 Behavior driven...

03:59 Oh, behavior driven development tests.

04:00 Yeah, yeah, yeah.

04:01 And so they're also actively looking for contributors, especially anyone with financial or accounting

04:07 experience.

04:08 So if you're looking for some project to contribute to, you know, that'd be great.

04:12 You could contribute to this one.

04:13 It's not super well known yet, but I thought I would shine a little bit of a light on it because

04:17 it seems like it's a cool idea.

04:19 Well, it's even got invoicing and stuff too.

04:21 So I think this is pretty neat.

04:24 Yeah, absolutely.

04:25 So what's up next for you, Brian?

04:27 What do you got for us?

04:28 Next up?

04:29 Oh, so another web sort of thing.

04:31 But this ran across Flask Meld.

04:34 Flask Meld?

04:36 I've never heard of this.

04:36 So it's like...

04:38 Is it like a Vulcan thing from Star Trek?

04:41 No, the idea is like melding the front end and the back end.

04:44 Okay.

04:45 And the...

04:47 It's a pretty cool...

04:48 There's a...

04:49 You should click on the example article.

04:51 But yeah, that first link.

04:54 Right there.

04:54 Oh, right here?

04:55 Okay.

04:56 Yeah.

04:56 There's a...

04:58 It's basically...

04:59 It's fairly simple interactive stuff that JavaScript gives you.

05:04 But it's super fast.

05:07 And it just...

05:09 Like, you don't have to write the JavaScript code.

05:12 So the article in this article talks about how...

05:16 This is from Michael Abraham...

05:18 I think it's Abrahamson.

05:19 But he wanted to avoid writing JavaScript.

05:23 So he wrote a whole bunch of JavaScript to make this plugin.

05:28 So it's a Flask...

05:29 His first Flask extension.

05:31 But you kind of modify the Flask templates to insert these elements.

05:37 And then the elements just show up as like objects in Python.

05:41 And you can program them like that.

05:43 So all these elements are all implemented in Python.

05:46 Oh, cool.

05:47 And there's a little video on there.

05:49 But there's a demo as well.

05:51 So there's a working example.

05:53 And then the code for the example is up on GitHub as well.

05:57 So you can play with it.

05:58 One of the fun parts in one of his working examples is the drop-down for like a search.

06:06 So if you start like predictive search sort of thing.

06:09 So if you start...

06:09 The example shows states.

06:11 If you start typing states.

06:12 It'll like start filling in this list of states that you might mean.

06:17 And it's like super fast.

06:19 Nice.

06:20 Yeah.

06:20 Apparently it says right here that it utilizes web sockets and morphed them to create server-side rendered HTML

06:27 and swap out DOM elements without refreshing the page.

06:30 That's pretty killer actually.

06:31 Yeah.

06:32 And he's nice enough.

06:34 I think this is cool.

06:35 He's saying that he basically stole all the ideas from other people and put them together for this.

06:40 Or standing on the shoulders of giants, so to speak.

06:44 And there's a similar project called Django Unicorn for Django people.

06:48 I played with that a little bit.

06:50 The Django Unicorn is...

06:54 The documentation site's amazing.

06:56 It's got a bunch of examples.

06:58 But it seemed...

06:59 I don't know if it's the server that it's hosted on or what, but the Django example seemed a little bit slower.

07:06 It seemed like there was this round-trip thing going on, whereas the Flask example, Flask meld, was zippy enough that that would be completely sufficient for a lot of the applications.

07:16 I'm thinking of throwing in some interactive stuff.

07:18 Yeah.

07:19 Yeah.

07:20 This is really neat.

07:21 This idea of components is pretty cool here.

07:24 Anthony, what do you think?

07:25 Yeah, it's interesting.

07:26 I don't really use Flask, but in the Django world, quite a lot of components and stuff that I've pulled in and plugins to do interactive elements.

07:34 And each one is implemented completely differently, which is really frustrating.

07:40 Whether it's like a search box or a type ahead or like a multi-select field, just something that I need, which is not available in the standard forms model.

07:50 And each one tends to have its own JavaScript and its own collection of bugs.

07:54 And yeah, it is quite frustrating, to be honest, because you kind of think, oh, I don't need to write this from scratch in JavaScript.

08:01 I'll use an extension that already exists out there.

08:04 And then you kind of get stuck in, does it work in Django 3?

08:08 Right.

08:09 You know, what state is the JavaScript in?

08:11 And then you read through the source code and you stumble across security issues.

08:15 And it's like, okay, I wasn't, you know, there's a line between it's easier to write it myself.

08:20 And you end up, I end up just maintaining forks of all these extensions and then trying to get pull requests back into them to kind of fix things up.

08:27 But yeah, it's interesting to see, I guess, a different model for doing it, which, you know, could be applied to Django.

08:34 This thing brings me back to like the year 2001 web with ASP.NET web forms, which are like such a weird way to build web apps.

08:45 They kind of like tried to mirror desktop apps.

08:47 But you could put a little Ajax tag onto like a section and just that part of the page would just become like would automatically refresh and interact differently.

08:56 This gives me like kind of that feel, but like not old school web, but more modern web, which is nice.

09:01 Yeah.

09:01 I mean, like I was thinking of it and now a lot of people that are comfortable with like jumping into some JavaScript.

09:07 This seems silly, I'm sure.

09:09 But for instance, like me, I never touched JavaScript.

09:12 So if I've got a little and I've got like a little flask app that pulls up like test result data.

09:18 And it'd be great to just have a be able to get a little form there that says, hey, here's here's the version I want to see the results for and be able to pull that up.

09:26 And it it doesn't have to be pretty.

09:28 But this way I could implement it without having to go in and learn JavaScript.

09:33 So, yeah.

09:34 Yeah.

09:35 And I think this is super neat.

09:37 I'm with you, Anthony, on the tradeoff of grabbing some of these cool plugins, Flask, Django, whatever.

09:43 OK, this is just now adding functionality.

09:45 But then you've kind of got to understand its assumptions.

09:48 When does it work?

09:49 Like, why does it not quite work for what you're doing?

09:52 It's always a tradeoff.

09:53 I usually go for the vanilla version of the web and just build it myself until it's like really clear that there's a big benefit.

10:00 Yeah.

10:00 And you bring up an interesting point with the security concerns, especially anytime you've got dealing with input fields and stuff.

10:07 You've got to be careful with that.

10:09 Yeah.

10:09 Because they normally run queries in a database.

10:12 So you've got to kind of audit these things to make sure that they're not using raw queries and that they're not using weird templates and stuff like that.

10:21 All right.

10:21 They better be parameterized queries and not little Bobby table type queries.

10:25 Exactly.

10:26 Yeah.

10:26 Yeah.

10:27 All right.

10:27 Wrong one.

10:29 This one.

10:30 So what's next?

10:31 Maybe they're using even Bitwise operators in there, Anthony.

10:35 Possibly.

10:35 Yeah.

10:36 So my next link is Bitwise operators in Python by Bartosz Zacznicki.

10:43 And this one's really cool.

10:46 Actually, I think if you've ever used Bitwise operators or seen them in the Python language.

10:52 So this is where you'd use the pipe symbol to less than, less than, which kind of looks like two arrows or right, right, right, which is greater than greater than.

11:03 There's a XOR.

11:05 Is there another way to say XOR?

11:07 I just use XOR.

11:09 Oh, yeah.

11:09 Yeah.

11:09 That's what I say as well.

11:10 You didn't say XOR or something.

11:12 No.

11:12 Which is the hat sign.

11:16 I can't remember what that symbol is called.

11:18 The tilde is for not.

11:19 So, yeah, these are basically used for specific types in Python, which support Bitwise operators.

11:26 I like this article because they're rarely used in Python because you typically use types where you'd use a method to do a lot of these things.

11:36 Most of the time, you wouldn't necessarily need to work with data, which is binary.

11:42 So you wouldn't necessarily need to do these things.

11:45 But if you are working with binary data, they're super useful.

11:49 And it takes a bit of time to get your head around.

11:52 So I've got a couple of examples.

11:54 But, yeah, I love this tutorial because it actually doesn't assume you know anything about this topic.

12:00 It explains what the binary system is.

12:03 And it uses these icons of hands.

12:06 It uses emojis, kind of like a mini sign language to explain the kind of the bytes and stuff like that, which is really cool.

12:14 So, yeah, I really like it for that reason.

12:18 And it's really illustrative and kind of takes you through a few concepts.

12:22 And then how you can use these special operators for some of the built-in types, Python integers, bit strings.

12:30 You can also use them for byte arrays, which is really helpful.

12:35 And then it goes into things like bit masks and stuff like that.

12:38 So if you're ever working with any lower level data, this is super helpful to understand.

12:44 And also a little trick that I discovered a while ago was if you've ever used the enum built-ins.

12:54 So enum is in the standard library.

12:59 And if you wanted to describe an enum, you basically create a class and inherit from an enum type, which is in the enum package.

13:06 And you can do a whole bunch of things and just represent things as enums in Python instead of other weird ways of doing it.

13:14 I don't know.

13:15 Introduced in Python 3.7, I think it was.

13:19 But there's a little known feature in the enums, which is there's an int flag and a flag type.

13:28 An int flag is an enum, which you can represent as a flag, which means you can combine multiple of them.

13:38 So, for example, if you had a color enum and you had red, blue, and green, you could represent white as being red, blue, and green by combining the colors together.

13:51 Nice. Wow. That's cool.

13:52 Yeah, you probably wouldn't make a paint mixer in Python, but there's lots of things.

13:57 Flags, for example, like if you're representing system flags or compiler flags or any other kind of flags,

14:05 this is really useful because it actually implements all the binary operators.

14:08 So, yeah, flag and int flag are super helpful types, which are built into the standard library.

14:16 So, yeah, if you ever find yourself doing something like this or using a package,

14:21 which has implemented its own weird version of this feature, which is built in,

14:25 then you can upgrade it to this new syntax and then you're done.

14:29 Yeah. Nice.

14:30 Tell me, what does this auto do?

14:32 Oh, so in enums, if you're going to say, you know, you need to make a representation of the enum name.

14:42 And if it's an int enum or an int flag, you'd say, you know, one, two, three, four, five.

14:49 Like, so if I save it to a file or pickle it or something, then it's, this is the number that it represents.

14:54 So I can convert.

14:55 So if you converted the number one to your enum, it would know that red was one.

14:59 So, you know, you could basically use it for storage or something.

15:02 It's also really helpful.

15:04 Like if you're reading from XML files or JSON files or something, and there's like, there's a field which is stored as an integer,

15:13 but you know that it actually represents something a bit more logical, like enumeration, then you can, you can do it that way.

15:19 However, for flags, then because they support bitwise operators, they should follow the binary sequence.

15:26 So one, two, four, eight, 16, 32, 64, et cetera.

15:29 And once you get past 1024, most people start to forget.

15:36 So instead of working that all out in your head, you can just use auto, which is a function built into the enum.

15:44 And it will basically just work out what that value should be for you.

15:47 So you don't have to work it out in your head.

15:50 It'll let you like reorder stuff and not have to go, oh, now I want to just want to go one, two, three.

15:55 Or I want to add another one, but in the middle and it accidentally messed it up or whatever.

15:59 Yeah.

15:59 I think I might use those anyway, just to, just to indicate that the actual number isn't important.

16:04 It's just that they're, they're unique.

16:07 Exactly.

16:07 Exactly.

16:08 I think if you, if you don't assign it, then in enums, if you don't assign it, then it works.

16:12 But with flags, you need to assign it to something.

16:15 Yeah.

16:15 Yeah.

16:15 Just for people listening who don't necessarily see the code, the show notes, the idea is you create an enum class and you say like category equals just lowercase

16:24 auto open close.

16:25 That's the way that you sort of invoke this behavior.

16:28 Also a couple of questions from the listeners who are in the live stream.

16:31 Anthony Lister says useful for MicroPython or hardware, maybe.

16:34 What do you think?

16:35 Yeah, absolutely.

16:36 If you're, I mean, if you're working with like embedded systems or anything, I mean, Brian,

16:40 you're the expert.

16:42 Yeah.

16:43 You have to work with binary a lot.

16:44 So yeah, this is super helpful because you can represent stuff, which is different states

16:50 in the system.

16:50 Or if you want to read, read inputs and stuff from multiple channels, you're going to need

16:55 to use bitwise operators.

16:57 So both the bitwise operators are really helpful as well as the flags enums.

17:02 Yeah.

17:02 And Brandon Rainer says bitwise always confuses me.

17:06 And when I see it, I tend to roll my eyes back and ask why.

17:09 I'm sure there's reason.

17:10 But well, I mean, one of the things that comes to mind for me is, you know, you're thinking

17:13 about storing stuff in memory.

17:15 You know, if I wanted to store, say, a number or something like that, or, you know, you put

17:19 that into a Python number, that's like 28 bytes.

17:22 But if you create an array of bytes and you know the size of them are going to be, you know,

17:25 packed into little bits there, you know, like one to 10 or something, a whole bunch, you

17:29 could be way more efficient by, you know, creating little smaller containers and then bitwise

17:34 boring them together and whatnot.

17:35 Yeah.

17:36 Like bitfields, for instance, are a really cool thing, but even just straight numbers with bitwise

17:42 operators are important for hardware because you're like a lot of times you just have register

17:47 access to something or you have memory mapped registers and you, you know, just kind of read

17:52 those out and there may be a whole bunch of data.

17:54 So each bit might represent completely wildly different things.

17:58 So you can't really just check for equality.

18:00 You have to check, is this bit on or off?

18:03 Or I need to set this bit and leave all the rest alone and things like that.

18:07 Yeah.

18:07 Yeah, absolutely.

18:08 Absolutely.

18:08 All right.

18:09 Before we get to our next item, let me just let you know.

18:11 this episode is brought to you by us, things that Brian and I are doing.

18:15 And so one of the things we just launched over at Talk By Theen Training is our brand

18:19 new FastAPI course, which is a super cool new API framework that I think brings together

18:25 a lot of the things that we've been proponents of, you know, things like Pydantic,

18:29 Typense, Async and Await, all those, they all come together really nice over there.

18:33 So if you want to learn that, check that out.

18:34 Brian has some book on pytest.

18:36 So yeah, you can check that as well.

18:38 Links in the show notes.

18:40 The next thing I want to talk about, though, is why you should use an ORM.

18:45 Here.

18:45 There we go.

18:46 So Anthony, Brian, what do you guys think?

18:49 Raw SQL ORMs?

18:51 What are your thoughts here?

18:52 Well, I've been using like document databases lately and I don't really need an ORM.

18:57 So.

18:57 I'm with you.

18:58 So over there, maybe the R is a D, at least if it's a document database, it's an object

19:03 document mapper maybe.

19:04 But yeah, I find being able to work with classes like the way I think of it in Python

19:09 to be that's how I want my data to be.

19:11 And just something else can figure out how the database has to break that apart into relationships

19:15 and stuff.

19:15 Super neat.

19:16 You know, like SQL can go and say, create like if I had a user and the user had orders,

19:22 like they might have an orders list on the user class to create a new order.

19:27 You can just go to the user.orders.append the new thing and hit save, commit changes,

19:32 and then boom, you know, it like figures out that has to be inserted and the relationship

19:35 has to be said and all that.

19:36 Anthony, what do you think?

19:37 I use the Django ORM quite a lot and really like it.

19:42 Learning all the edge cases where it creates queries which are not super efficient.

19:48 But so I guess there's pros and cons.

19:50 Like if you were to write raw SQL, you know, sometimes if you know SQL really well, then

19:55 you can write more efficient queries.

19:56 However, there are typically ways around that.

20:00 The N plus one problem in Django.

20:02 The N plus one is the biggest problem for sure.

20:06 You want to describe the N plus one problem or take a shot at that?

20:09 Yeah.

20:09 So if you sort of have a foreign key and you reference an entity which might be a many

20:15 to many relationship and then you reference a property of it in the query,

20:20 not actually not in the query, but actually in the view, you mentioned something or you

20:25 look up a field which is part of a mapping to another table.

20:28 Kind of like I described, like if you gave a user to the view and the view wanted to know

20:32 about its orders.

20:33 Yeah.

20:34 More importantly, if you gave a list of users and it wanted to know about the orders for

20:38 each of them, right?

20:39 That's the real bad case.

20:40 Yeah.

20:40 So you had it for like, you know, which group people are in or like which team they're in

20:45 or something.

20:45 And it's just, it's a team ID.

20:48 And then you actually want to show it on the table as the team name.

20:51 So what ends up happening is that for every row in the table, it looks, it does another

20:56 query to look up the team name, even though they're the same across the board.

21:00 Whereas if you're writing that in SQL, I'd hope you'd do that as an outer join.

21:06 But yeah, so, sorry, not an outer join.

21:09 So yeah, OOMs are really useful in that sort of thing, but they do have edge cases.

21:15 For N plus ones, pretty sure you covered this one before, but there is an awesome tool called

21:19 N plus one, which you can run in your test suite.

21:22 So when you do all your integration tests with Django, it fails the test if it detects

21:27 an N plus one query, which I use quite extensively.

21:31 I think Adam.

21:33 Oh, I had not heard of this.

21:36 I mean, this is not such a problem for me because I work like Brian and document databases.

21:41 But to the extent that I work with like SQLAlchemy and stuff, like this is really, this

21:45 is the one you're talking about?

21:46 Yeah, yeah.

21:47 That's it.

21:47 Yeah.

21:47 P-L-U-S-O-N-E.

21:50 Yeah.

21:50 That one.

21:51 Yeah.

21:51 So I add it, I only add it in the test suite.

21:54 So you don't, you don't really need to put this into production.

21:57 So you just load it into Django.

21:59 You're assuming that you have tests, Anthony.

22:00 Yeah.

22:01 Well, that's probably a starting point.

22:02 So, or if you're a developer in an environment, you can just turn it on, but it would.

22:10 Yeah, it basically, it would print a warning and stuff like that.

22:13 However, there is a config option that says that it can raise an exception if it detects

22:18 one, which is awesome because when you run all your tests, it will fire off and fail the

22:25 test if it detects this type of query.

22:27 And the workaround in Django is actually really simple.

22:30 You just add another function to the chain, the query command, basically, which indicates

22:36 that I'm going to use this field in this other table.

22:39 So it kind of pre-looks up.

22:40 Right, right.

22:41 Please do the join or sub-query load or whatever it needs to do.

22:44 Yeah.

22:44 Yeah.

22:44 And the other thing about IRMs that I really like is the migration ability.

22:50 Like, it's fine when you initially design a system, you could, yeah, fine.

22:54 You can write your own custom SQL queries and stuff.

22:56 But, you know, within a week or two, you've added like five extra fields.

23:00 And actually, that one field you added now needs to be a different type or, you know, databases

23:05 are not static things.

23:07 Like, database structures change all the time in a real application.

23:12 So that becomes an absolute nightmare if you've handcrafted all your SQL, especially if you

23:18 don't have tests.

23:20 So, yeah.

23:22 I mean, for different database engines, there's different ways to do this.

23:25 I mean, SQL Server like has DAX.

23:28 So you can do DAX packs and DAX pack migrations.

23:31 But most of the other database systems don't have that.

23:34 Whereas if you're using the IRM, then, you know, Django and Flask and stuff would create

23:39 the migration for you.

23:41 And you're kind of pretty sure that you can add fields as you wish without necessarily

23:48 breaking things.

23:49 Or you can add types and you can describe what to do with the old ones.

23:54 So I think it saves so much time in just working with a live system where you're actually constantly

24:00 making modifications to the table structures and stuff.

24:03 And maybe you have different developers at different stages or you want to go back in the brand.

24:08 You want to say, I'm going to go back here and then work on this version of the app at this

24:12 stage.

24:13 You want to be able to go to just run the migrations and get to the right structure.

24:16 Yeah, I find it to be super, super helpful.

24:19 I mean, there are times where, you know, you want to get 100,000 records back from the

24:24 database and an ORM doesn't make sense.

24:25 But to me, I feel like, you know, you should 80, 90% of the time use an ORM.

24:30 And then there's that little edge case where something slightly different maybe needs to

24:33 happen, but you shouldn't start there because there's that 10% or 5% chance.

24:37 Yeah, most ORMs give you the ability to write a raw query.

24:41 Yeah.

24:41 And yeah, you should also check there's raw queries for SQL injection.

24:46 And I do have a tool for that.

24:48 Yeah, if you want to, you can run it through the Python security thing that I've built,

24:53 which will scan your code and spot where you've used the raw SQL functions in the Flask ORM and

25:01 the Django ORM.

25:02 And it will see if there's anything in them, which is pretty likely to leave you subject to

25:07 SQL injection.

25:08 Yeah.

25:08 Is that built into your PyCharm security plugin?

25:12 Yes.

25:13 Yeah.

25:13 Maybe.

25:14 Yeah.

25:14 I want to just mention this come like this conversation, although not exactly.

25:18 We didn't pull out much out of it, but comes from an article.

25:21 I wanted to highlight called why should I, why should you use an ORM object relational mapper

25:25 written by Kareem?

25:25 I'll leave that in the show notes.

25:27 There's a bunch of details that walks you through it.

25:29 So I wanted to cover this.

25:31 So people, if they're having this debate on their team or they're not sure which way to

25:35 go, or they're, they're new and they're like, what are all these funky acronyms about?

25:38 You know, they can check that out.

25:39 There's, there's the obligatory little Bobby tables joke in there.

25:44 I got to, got to go with that.

25:46 This is what your, your plugin is supposed to detect is this cool XKCD.

25:50 Cool.

25:51 You're not on the receiving end.

25:52 One of the things I'd like to bring up around ORMs is that like a lot of the ORM tutorials

25:58 assume that, you know, SQL.

26:00 And I'd like to see more tutorials on how to use an ORM and how to use it correctly

26:05 without any SQL examples.

26:07 Yeah.

26:08 Yeah.

26:09 You know, classes in Python.

26:10 Yeah.

26:11 Here's your way.

26:12 You don't have to know SQL.

26:13 Yeah.

26:13 I mean, I thought that was the point is that you can think in objects instead of in SQL.

26:19 So yeah.

26:20 Why do I got to learn SQL in order to understand the tool?

26:23 Sure.

26:24 Sure.

26:24 Another thing that you can turn on that's helpful for the N plus one problem, at least with SQL

26:29 Alchemy, you can go to the engine when you create it and you can say echo, echo equals

26:34 true.

26:34 And it'll echo every single underlying SQL command sent to the database.

26:39 You can do something in SQLAlchemy and it'll say, here's what the actual thing is.

26:42 And if you've written your code well, and you've done the right join stuff, you'll just see like

26:47 a couple of entries for each page or interaction you're having.

26:51 You've done it wrong.

26:52 You'll see your, your output just scream by full of these things.

26:55 Like, ah, there's one of these problems.

26:56 So that's, that's a easy way as well.

26:58 Is there like a test way to like, like for instance, to check to see how many for

27:03 certain test sequence, how many, database interactions you've done?

27:07 Yeah, there is.

27:08 Yeah.

27:09 there is in, there is in, Django, you can kind of hack a bit of middleware, which,

27:13 kind of catches SQL queries and stuff like that.

27:17 it's not built in, but you can write it.

27:20 I think I've got code sample somewhere that does that.

27:23 and just say how many queries this page executed.

27:26 Obviously it depends on, that's why it's important to see the database first with test data, because,

27:32 you know, if you just run it on an empty database, then typically, or you've got like one row or

27:38 something, then it's, it's going to be like, yeah, it's fine.

27:40 But then when you deploy it to production, like it's actually running thousands of queries

27:43 for every page.

27:44 Yeah.

27:44 Did you end up with what you got in the United States when we tried to roll out the, the healthcare.gov

27:50 and the entire system went down and yeah.

27:53 Like, like the, yeah, whatever.

27:55 They knew it was going to be the entire country using it and they didn't load test it.

27:59 Exactly.

28:00 Exactly.

28:01 one more thing, a comment from Mateus, it's also useful, when using Django to

28:07 add the Django debug toolbar or Django silk to be aware of what queries are going on.

28:11 And also I would add for pyramid, there's a pyramid debug toolbar and you can actually see the

28:16 underlying queries and how many of them there are and the timing.

28:19 It's, it's really nice.

28:20 Those things.

28:21 All right.

28:21 All right.

28:22 Well, let's move on to the next one.

28:24 Sticking with the database stuff, Brian.

28:26 Definitely sticking with databases, and SQL.

28:29 So, this one's from Simon Wilson and, he's got a tool called, library.

28:36 Well, yeah, sure.

28:37 Dataset, but, SQLite utils.

28:40 this is, it's developed as part of data, the dataset project, but it's, it's usable

28:47 by anybody that uses, SQLite, SQLite.

28:50 And, it's a couple of things.

28:52 It's a command line utility, which, a lot of databases do have a command line way to enter

28:58 to query the database.

28:59 But I don't know if SQLite does.

29:01 but this is a pretty cool command line.

29:04 You can interrogate, analyze tables and dump things and do all sorts of stuff.

29:09 and even search, it's pretty useful and pretty simple examples.

29:15 the, one of the things I really liked though, is the, the API that he has.

29:20 So there's a Python API to, I mean, you can use, like SQLAlchemy for instance,

29:26 to interact with SQLite.

29:28 But, another way to do it is to use the SQLite utils as a, as an API for, for SQLite.

29:35 And it's a pretty clean, simple, interaction.

29:39 And I think that's, it's something I'm, I want to try because it looks like a, I do need to

29:44 get back into some SQL, SQL database work.

29:47 And, I think this would help a lot.

29:50 Yeah.

29:50 This comes from the dataset found, it's like one of the foundational tools from dataset,

29:54 this project that he's working on, which is really interesting.

29:57 It's like empowering data driven journalism and data exploration across all these different

30:03 data sources by converting them all down to SQLite databases.

30:07 And then once they're there, you can explore them in interesting ways.

30:10 So there's like Twitter to SQLite, various other things to SQLite, Gmail to SQLite.

30:16 And then you can like explore all these different things, like geo JSON to SQLite.

30:21 what else have we got here?

30:23 There's just tons and tons of these things that plug into other stuff and then gets into this

30:27 common format.

30:28 And then he also has this cool tool called dog sheep, which builds on top of those databases

30:33 that creates like a personal search engine for your life.

30:36 So you could like plug in Twitter, you could plug in, your iPhotos library.

30:40 You could plug in your Gmail and then there's a search engine that just says search that.

30:44 So it's like the, one of the foundational building blocks of like, how do I get everything from

30:48 its original source into SQLite?

30:50 Pretty cool.

30:51 Yeah.

30:51 Yeah.

30:52 So Anthony, Anthony, what do you think?

30:54 I'm actually really interested about the next, link on his blog, which says I commissioned

30:59 an oil painting of Barbra Streisand's cloned dogs.

31:05 It's really nice.

31:06 It's really nice.

31:06 Actually.

31:07 I kind of like it.

31:08 I thought it was a joke.

31:09 You actually did.

31:10 For gazing the tombstone of the dog that they are.

31:12 yeah.

31:13 Okay.

31:13 Interesting.

31:14 Yeah.

31:15 I'm not sure what it has to do with SQLite, but no, it's pretty cool.

31:17 I think, yeah, more tools for, journalists and stuff as I start to work with data and

31:23 data becomes more readily available or at least there's more of it.

31:26 So, yeah.

31:27 And people aren't necessarily, yeah.

31:29 And don't necessarily have the technical skills to work with massive data sets and stuff like

31:34 that.

31:34 So yeah, it'd be really interesting to, yeah, use some, some of these tools.

31:39 For sure.

31:39 Simon Wilson was one of the co-founders of Django.

31:43 I had him on Talk Python recently, but it's not published yet.

31:46 And we talked about data set and dog sheep and there's, there's just a lot of interesting

31:50 stuff that one of the stories that he tells, so people can listen to the episode, but he

31:54 has a dog, which may be this, commissioned thing.

31:56 He's like really got this dog that has a Twitter.

31:58 The dog has its own Twitter account.

32:00 So for example, some of the interesting stuff that he did was his dog in the Twitter account

32:05 will tweet when it goes to the veterinarian, like how much it weighs and stuff.

32:08 And then it will tweet pictures when it's on a walk.

32:10 And he was able to do things like we create a graph over time of his dog's weight by just

32:17 doing a SQL query against the dog's Twitter account that got pulled in like all sorts

32:22 of weird connections of like pulling data together that you just couldn't imagine.

32:25 So, yeah.

32:27 Anyway, really, really neat stuff there.

32:28 Okay.

32:30 Yeah.

32:31 All right.

32:31 People will check that out.

32:32 and then you want to wrap it up with a, a talk that is very, work from home ish,

32:39 a conference that's very work from home.

32:41 I should talk from there maybe.

32:42 Yeah.

32:43 Yeah.

32:43 So, my next one is from the Pyjamas conference, which happened a couple of

32:49 weeks ago.

32:49 and this is, yeah, lots of online conferences happening at the moment.

32:54 which I kind of sign up for and look at the, you know, the talk list and think, oh,

32:59 great.

33:00 That'd be really interesting.

33:01 And then never actually get around to watching any of them.

33:03 Exactly.

33:05 yeah, I'm just kind of struggling with this online conferencing thing.

33:08 I like the ideas are really cool.

33:10 The concept is cool, but the practicalities of it, yeah, especially like, I don't know

33:15 if it's different in our house, but, the, the conference run over a weekend and I just

33:20 wanted, you know, maybe an hour or so to watch some of the talks and things, but just sitting

33:24 down on the couch and turning the TV on, putting it on YouTube and then sitting and watching

33:28 talks.

33:29 And it was like, the kids isn't making so much noise.

33:31 My wife's like, why are you watching TV?

33:33 There's loads of stuff to do and I'm like, no, but I'm, this isn't.

33:36 And then is this work?

33:38 It actually is work.

33:39 I'm sure it is.

33:40 Yeah.

33:41 Yeah.

33:41 Sure.

33:41 It looks like work.

33:42 Yeah.

33:42 so yeah.

33:44 I see pajamas.

33:44 There's pajamas up there.

33:46 This doesn't make it.

33:46 This doesn't feel like work.

33:47 Yeah.

33:48 And I definitely wasn't in my pajamas.

33:49 I had to get dressed because it was 10 in the morning.

33:51 so yeah, I don't know.

33:54 I know I'm kind of talked about this and a couple of other people said they're having similar

33:58 challenges with these online conferences.

34:00 Just like, where do you fit them in?

34:02 and if you actually go to a conference, it's like, you've got, you've got an excuse

34:07 that this is more, this is like, my time is now set out.

34:10 Whereas if I, if I was to say, I'm going away for a few days to a conference, that's fine.

34:14 But if I try to attend that conference at home, it's like, that's not, yeah.

34:18 Yeah.

34:18 That's completely different.

34:19 I'm going to lock myself in my room and watch TV for two days.

34:21 I'll be back.

34:22 Yeah.

34:23 Yeah.

34:23 Exactly.

34:24 It's not the same story.

34:25 Yeah.

34:26 Someone at work actually suggested that we have like a, a work hotel sort of thing where

34:30 people can attend virtual conferences.

34:32 where, you know, especially if you're on, if, if they're late at night or they're in

34:38 weird time zones and stuff.

34:40 So even if it's just in your local city, you can go and stay there.

34:43 So you can at least dedicate the time.

34:46 anyway, the, so the pajamas, which I really wanted to attend, I managed to get through half

34:51 a talk.

34:52 and then I watched the other half like a week later.

34:55 so that was not very successful.

34:58 however, the talk was brilliant.

35:01 So I wanted to share a link.

35:03 It was called what the struct, by Zachary Anglin.

35:07 And it's talking about the, the struct library, which is built into, the standard

35:13 library.

35:14 I didn't really realize there was a struct library, right?

35:17 Like there is an array library that says this will hold floats contiguously.

35:21 Same thing for structs, right?

35:22 Yeah, exactly.

35:23 And I think, you know, Brian was talking about data classes before we went live actually,

35:28 but, you know, data classes are really cool.

35:30 Like if you wanted to represent, interesting structures and stuff like that, ORMs are

35:35 really important for this sort of thing.

35:37 But if you're working with binary data, reading the binary data and then converting

35:42 into native Python types, often, you know, you do C for that sort of thing.

35:47 And in C you just declare a struct and just say, there's these filled.

35:51 And you can just read and write that into binary, structures, which is kind of built

35:56 in, but sometimes you need to do that in Python.

35:58 If you're working with, actually, I've got some examples, but, so this explains what

36:03 the structure library is.

36:04 And it also has this macro language for describing what the underlying type is.

36:09 and then you can kind of like pack and unpack it basically.

36:12 So you can say, here's my, here's my class or whatever, and here's the fields and the underlying

36:18 binary structure is a, you know, a float, a 32 bit integer or binary, Boolean, or,

36:25 you know, it may be a, a, a ASCII string or something.

36:28 so yeah, there's basically all these like little, characters, for packing and

36:33 unpacking data.

36:34 so yeah, it was really helpful.

36:36 if you're getting into this topic of working with binary data structures or something that

36:41 you've wanted to do, I recommend if you're on a Mac using a tool called Sinalize

36:48 it, or spice analysis, which is basically a GUI for working with binary data and it has,

36:56 a database of grammars.

36:58 so you can basically like open a compiled executable or, you know, something like that.

37:05 and it would be like, Oh, I know what this format is.

37:07 And it would then just represent it in actually something more understandable and you can edit

37:12 fields and hack around with stuff.

37:14 it's great if you're doing capture the flags.

37:17 or if you're doing some hacking for good, it's also really cool if you want to,

37:26 edit files, which are not in a human readable format, if they're in a binary format.

37:31 and in the database, they've got a whole bunch of examples, including a lot of save game

37:37 formats.

37:37 So this is actually how I got into this topic.

37:40 originally it was when I was a lot younger playing games and stuff.

37:43 And I'm like, what is this file?

37:45 And then, you know, trying to echo it on the shell and it would be like, Oh, that's all

37:49 gobbled.

37:50 And then realizing it's all in binary.

37:52 And then, you know, trying to figure out if you open it up in a hex editor or something,

37:56 what it is.

37:57 And then, Oh, if I change this field, then I can give myself more gold or I can make my

38:02 character invincible or.

38:04 So basically like hacking the game by modifying the save game.

38:07 This end boss would be so much easier if I had a hundred thousand hit points or whatever.

38:11 So let's just increment that number.

38:12 Yeah, exactly.

38:13 I think modern games are a bit, they kind of put protections around the save game because

38:17 there's a lot more online aspect, but definitely older games.

38:20 You can, you can hack the save games and most of them give yourself whatever it is that you

38:25 feel like.

38:26 so yeah, understanding binary structures is really helpful on the struct.

38:30 you can even automate it using the struct module.

38:33 Yeah, nice.

38:34 Yeah.

38:35 There's also some good comparisons between like the values of structs and then tuples and name

38:41 tuples and data classes and Pydantic.

38:43 So there's sort of a spectrum that gets covered there as well.

38:45 Yeah.

38:46 Interesting.

38:47 Goes online well with the, the bit manipulation topic you were talking about.

38:52 It's like, for instance, very structured, structs are, are very important for like message, systems like communication systems.

39:03 we have very defined, fields within each, with, within the, you know, we've

39:09 got 128 bit sequence or a thousand bits in there.

39:12 Each bit is a specifically assigned to different things.

39:17 and those are, yeah, we, you definitely don't want to just like hope that it's right.

39:22 You have to put it exactly where you need it.

39:24 Yeah.

39:24 Yeah, exactly.

39:25 Like little network packet headers and things like that.

39:28 Yeah.

39:28 Yeah.

39:28 Yeah.

39:29 I'm using this at the moment cause I'm writing a disassembler.

39:32 So it's quite helpful.

39:34 yeah.

39:36 Awesome.

39:36 Yeah.

39:37 That you're, you're definitely doing some low level stuff these days.

39:39 so I guess that's our, our main items for today.

39:43 Brian, you got some, some extras you want to share.

39:46 Yeah.

39:46 We got, poked by the Python software foundation that we should probably plan some events.

39:51 because we're, we're, we've got a meetup, the, Python PDX West meetup, but I can't

39:59 access the building that we normally held it in and nobody would come anyway.

40:02 Right.

40:02 So, we're doing virtual.

40:04 So we've got, January 14th, we're going to do, we're going to start seeing what

40:09 it's like to do a virtual thing.

40:10 And I thought, you know, normally when we did the meetups before it was in the evening,

40:14 I don't really want to hang out in the, in the evening doing this, but a lunchtime would

40:18 be fine.

40:19 So I thought like a lunch, a lunch and learn thing.

40:22 So we're going to try that and see what it's like.

40:24 Yeah.

40:24 It's perfect.

40:24 Being virtual, you could just do it during lunch, right?

40:27 You don't have to go, okay, well I'm going to drive out, you know, fight the traffic or

40:31 whatever.

40:31 Right.

40:32 Yeah.

40:32 And I don't have to convince my boss to buy a bunch of pizzas.

40:34 so yeah, exactly.

40:37 Exactly.

40:37 All right.

40:38 I got a couple of extras as well.

40:40 I ended up doing a FastAPI webcast with the folks over at JetBrains.

40:44 So people can check that out.

40:46 They want to go learn more about that.

40:48 And then also let me close my, our notes here, but I'll get them back.

40:54 Brian, you go next.

40:55 I got to pull this up somewhere else.

40:56 I just went.

40:57 So let's, sorry.

40:58 Yeah.

40:59 You got it.

40:59 Anthony, yeah.

41:00 Anthony, what are you up to these days?

41:01 What extra you got to share with folks?

41:03 yeah.

41:04 So, I'm going to be starting at Microsoft in February, which is big news for me.

41:09 Yeah.

41:09 Congratulations.

41:10 That's awesome.

41:11 Yeah.

41:11 Yeah.

41:11 I'm really excited about it.

41:13 so yeah, I'm going to be the Python developer advocate working with, Nina Zakarenko.

41:17 So Nina's the, the current Python developer advocate and I'm going to be, yeah, helping

41:22 out.

41:22 so yeah, really looking forward to it and working with a bunch of smart people over at

41:27 Microsoft.

41:28 So yeah, all things Python.

41:30 do you know any particular projects you're working on?

41:33 So I'm not going to be, I'm not going to be, I'm not on the engineering team.

41:37 and I'm definitely not working on Pidgin.

41:41 This has got nothing to do with Pidgin.

41:42 contrary to the rumor, I guess.

41:46 you were hired to work on Pidgin.

41:47 That's what I heard.

41:48 Come on.

41:48 Yeah.

41:49 I, yeah, I'm definitely not.

41:51 so yeah, this is kind of a side thing.

41:53 but yeah, it's kind of, definitely the, Azure platform and like using Python

42:01 and Python working with Azure and on Azure, VS Code and the Python extension and

42:07 just the tooling and around that, as well as the Python ecosystem with Microsoft products

42:12 in general or Microsoft technologies, and Python in general.

42:16 So, all things Python and yeah, really looking forward to starting on the 1st of Feb.

42:21 and yeah, I have a long list of things that I want to, want to get stuck into.

42:26 Yeah.

42:27 I'm guessing that's remote, not just now, but permanently, right?

42:30 Yes.

42:31 Yeah.

42:31 Yeah.

42:31 That's super, that's super exciting.

42:33 but that does mean Brian that he'll be somewhat in our neighborhood more often.

42:39 I suspect you'll come up to Seattle and Redmond sometimes when that happens again, when the

42:43 world is let loose.

42:44 Yeah.

42:45 I expect so when, we're not even allowed to lead the country at the moment.

42:49 Yeah.

42:50 Well, we'll see you in the Northwest once in a while.

42:53 Yeah.

42:54 Yeah.

42:55 When, when, maybe 2022.

42:57 Yeah.

42:58 What 2030?

42:59 We'll definitely see you.

43:00 2030.

43:00 No problem.

43:01 Also, from, from, the YouTube stream, piling says, congrats Anthony on the new job.

43:06 Thanks, piling.

43:07 Yeah.

43:07 Yeah.

43:08 So we should have like a, like a webpage with like a, graphic to show like how many,

43:13 like well-known Python people there are and the, like maybe a death star Microsoft sucking

43:20 all the Python people into Microsoft.

43:22 Yeah.

43:23 They're pretty evenly distributed.

43:26 Yeah.

43:26 Yeah.

43:28 No, it's all good.

43:30 All right.

43:30 Well, I feel like, might be time to wrap this up with a joke.

43:34 You think?

43:34 Sure.

43:35 Yeah.

43:36 So, this one came to us over Twitter from Lars and there was a, apparently a question.

43:41 Someone named Kate Maddox asks, what's the best programming language for coding your own

43:47 therapist?

43:47 You know, like we're all stuck at home.

43:49 People are depressed, have social anxiety or just anxiety about the world in general.

43:54 And Wes comes along with a, an appropriate answer.

43:57 What do you guys think here?

43:58 Python, of course, is the language.

44:00 So you can call it thera, I, E-H-E-R-A dot P-Y.

44:04 Yeah.

44:05 I think it's pretty much on par for our standard jokes though.

44:09 Yeah.

44:12 might be better than average actually.

44:15 Yeah.

44:15 Yeah.

44:16 It could be better than that.

44:17 You need a little drum kit in the background, Brian, that you can.

44:19 Yeah.

44:20 Some symbols.

44:22 We'll set you up with like a little.

44:23 It's a business expense.

44:24 I'll totally get a drum kit.

44:25 Yeah.

44:26 Fantastic.

44:27 All right.

44:28 Well, Brian, thanks as always.

44:30 And Anthony, thanks for joining us.

44:31 Thank you.

44:32 Yeah.

44:32 Always a pleasure.

44:33 Yeah.

44:33 You bet.

44:34 And everyone listening.

44:35 See you next time.

44:36 Thank you for listening to Python Bytes.

44:37 Follow the show on Twitter via at Python Bytes.

44:40 That's Python Bytes as in B-Y-T-E-S.

44:43 And get the full show notes at Pythonbytes.fm.

44:46 If you have a news item you want featured, just visit Pythonbytes.fm and send it our way.

44:50 We're always on the lookout for sharing something cool.

44:53 On behalf of myself and Brian Okken, this is Michael Kennedy.

44:56 Thank you for listening and sharing this podcast with your friends and colleagues.