Transcript #201: Understand git by rebuilding it in Python

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to

00:04 your earbuds. This is episode 201, recorded September 24th. I'm Michael Kennedy, and Brian

00:10 is not here this week. Instead, my friend and special guest, Cecil Phillip. Welcome, Cecil.

00:14 Hey, hey, everybody. I've been a long fan of the show. And so, you know, Brian, I'm sorry that

00:19 you're not here, but I promise I'm going to do a good job and do you justice. So I'm going to do

00:23 my best Brian impression today on the podcast. Beautiful, beautiful, beautiful. So Cecil,

00:29 you've been on the show before. You're a developer advocate at Microsoft. Is that right?

00:33 Yep, yep. Still developer advocate. And as folks can imagine, we're all at home and not traveling

00:39 as much as we used to anymore, which, you know, is usually something that's attached to the job. But

00:42 now we're just trying to find different ways to kind of reach out to the community and work with

00:48 folks and make sure that they have what they need to, you know, just be successful and productive.

00:52 Yeah, absolutely. And you're doing a lot of live streaming stuff these days rather than

00:56 appearing in conferences.

00:56 Yeah, we're doing a lot of live streaming, a lot of live events and even, you know,

01:00 live conferences too. I think a lot of conferences now, and you've seen this a lot as well with

01:04 PyCon and tons of the other, tons of the other online conferences is just because we can't do it

01:10 physically together like we're used to, you know, now a lot of folks just doing it online. So,

01:14 you know, I'm sure if you'd look online, you see tons of, you know, register for free. Here's my

01:19 conference on Tuesday and Wednesday and Thursday. And, you know, everyone's just trying to find a

01:23 way to connect.

01:23 Yeah, it's got some interesting implications for accessibility, right? Like, I couldn't necessarily

01:28 go to EuroPython, but I could attend the virtual one just as easily as long as I'm willing to get

01:34 up a bit early. That's kind of cool, actually.

01:36 Yeah, that's one of the things I like about it the most is that, kind of like what you said, like

01:40 before, I didn't want to go buy a plane ticket to go to Australia or, you know, somewhere that's maybe

01:45 halfway around the world to me. But now I can have maybe not the same experience, but I could still engage

01:50 with that community of folks virtually by just signing into their YouTube or their Twitch stream or

01:56 whatever the case is. And I can get the content the same way. And it's great.

01:59 Yeah, absolutely. So let me kick this off with our first item, where we go from Python to speak

02:04 another language, you know, a little different cultural here. So we're going to talk about

02:09 talking from Python to C++. Maybe not what you thought when I said that, but really C++ is one

02:15 of these or C really the more low level is one of the reasons I think Python is doing as well as it

02:22 is these days, right? Like a lot of what you do in Python might be a little bit slow,

02:25 but you know what? You implement part of that package and see, and then all of a sudden,

02:30 it's a lot faster than things you might compare it to. And that's a lot of the magic. A lot of the

02:34 data science works that way and whatnot, right? Right. Yeah. So there's this article that I'd like to

02:39 point folks to that basically walks you through what it's like to call C and C++ code from Python.

02:46 So there's a lot of things to cover. That's pretty interesting. It talks about,

02:51 first, what do you get when you capacity? Obviously, you get something that's executable.

02:57 What it is, is actually different depending on the machine you're on, right? So if you're on Linux,

03:01 you'll get an executable and linked format, which is ELF. On Windows, you get a portable executable.

03:06 On macOS, you'll get a mock object, a mock.o, and so on. So it compiles to whatever machine

03:14 instructions you need. And then depending on what platform you're on, again, you can put that in some

03:19 central location so that compiled C library can be used by Python. So on Linux, it'd be user slash lib.

03:27 On Windows, what do you think? Windows slash system, system 32, something like that, Cecil?

03:32 Sys32 or something crazy like that. Yeah.

03:34 Yeah, exactly. Something there. But once you put it there, then your program can find it and you can

03:40 just import it. So there's different ways you can do this, right? Like you can use the C types library,

03:45 which is simple, sort of, sometimes simple. And you can just go and import C types and then just say,

03:51 here's a C DLL and point literally at the file, the output from compiling some C code and import it and call it.

03:59 That's pretty simple. Coming from Microsoft World made this DLL import type of equivalent.

04:06 That's why I'm not entirely sure. But there's some interesting things that you run into. Like,

04:09 so you get this library back, but it doesn't necessarily know what the return types are,

04:14 right? Like, oh, you get a function, you call it returns a float. Maybe it doesn't know what to do

04:19 with that. So you've got to like, as you import it, tell it, oh, you return a float here and here are

04:24 your functions that you can call and so on. So there's a lot of interesting things you can do.

04:29 If you remember your C, you have to set the calling convention, which can be really tricky.

04:33 Like extern C, oh my gosh. Is it standard call? Is it a C decal? I mean, there's so many variations.

04:42 Do you remember this? Like, did you ever do some C and just go, why won't this library work with that

04:46 other one?

04:47 I know now, like I'm looking through this document as you're talking about it and I'm like, oh my gosh,

04:51 this is bringing back so many memories.

04:53 Yeah. I mean, you remember like how hard these things were to get them right. And it reminds you

05:00 how nice it is to work with like more modern languages like Python. So there's actually an

05:05 interesting library called libffi. So this is a C library. So if you're going to write the C code,

05:12 you can use libffi. And when you compile it, as you compile it for different platforms and stuff,

05:17 it'll automatically figure out the calling convention for the functions at runtime.

05:22 Oh, nice.

05:22 So, so that's pretty sweet. So you can, I mean, this didn't exist to my knowledge when I was

05:26 trying to do these things. That's interesting. Another thing you can do is you can add an entry

05:31 point function to your code. And then Python will ask, basically you can call that and say,

05:39 you know, what functions do you have and what are their signatures? So there's a way

05:43 to have your library kind of introduce itself over.

05:46 It's kind of like providing metadata, I'm guessing.

05:48 Yeah, exactly.

05:49 Here's the structure of my stuff and I'm going to give it to you.

05:51 And now you can kind of understand it.

05:53 Exactly. Cause you don't have a header file or anything like that, that defines it. It's just,

05:57 here's the executable compiled functions go right. And so it gives you that metadata. Exactly.

06:04 Of course you find you, the goal is to have C++ code that you can call from Python.

06:10 And there's a Python specific, a Python friendly way to do this as well. If you're willing to work with a

06:17 Python type. So like PI object pointers and so on. And the way you would do that is you create this

06:22 entry point, but it's the function name is PI in it underscore module name. So if it was requested,

06:29 it'd be PI in it underscore request. Exactly. And so that returns all this metadata that says,

06:35 here's all the C stuff and the signatures and whatnot. So after going through all this stuff,

06:42 how you write the C code and you do it, it says, you know what, maybe you just want to skip that.

06:45 Yeah. Can we please?

06:47 Can we please? Yeah. So what you could do is you could write it in Cython,

06:51 which compiles to C, which compiles to machine instructions, or there's also boost.python.

06:59 I wasn't aware of boost.python. That's pretty cool. And pybind11, which allows you to bind,

07:05 I believe to C++ 11, which is pretty cool as well. Yeah. So really nice integration stuff there.

07:11 And some examples of libraries that people might know of that are written in Cython,

07:16 but basically natively compiled and do awesome stuff is AIO HTTP.

07:22 Oh, I didn't know that. That's cool.

07:24 Yeah. Yeah. Use the Cython for the HTTP parsing. So that string stuff is super fast. If you do async,

07:29 you can, one of the things that's interesting in Python about Python's async as a wait,

07:34 and a wait is you can change the implementation of the underlying of the event loop. Right. And so

07:41 one way you can do that is you can replace it with UV loop, which is based on libuv, which I always

07:49 thought of as a C library, but it's actually fully written in Cython.

07:52 Oh, I didn't know that either. Really? Yeah. That's pretty cool. Then finally,

07:55 HTTP tools, bindings to Node.js HTTP parser is fully written in Python, along with Sanic,

08:03 UV core and a bunch of other stuff that use this library. So if your goal is to try to get a little

08:08 bit lower level, maybe work with some C, make your code go faster. Here's a really cool write up of how

08:14 to do all those things. That's really cool. You know, I'm a big fan of language interoperability in

08:18 general. Yeah. And you know, one thing I could tell you, it feels like every language knows how to work

08:23 with C, at least. Like C is like the common denominator that every language knows how to like

08:29 talk back to. And I really wish there was a way for us to, I mean, I love C. I used to love C. But I

08:36 really wish there was a way for us to like, have like a very standard way to like interact with each other a little

08:41 bit better. And I remember, I think the last time that I was on the podcast, I talked about

08:46 Wazi. So I think it was a WebAssembly. Yes. It's like a WebAssembly module that you could use.

08:51 And now all of your code compiles down to WebAssembly and it exposes different functions and

08:56 whatnot. But now that that'll be like a portable runtime, but essentially I can, I can import it in

09:02 Python and I can import it in Java or JavaScript or .NET or whatever the case is, because they all have

09:08 that, you know, that knowledge of how to work with that WebAssembly binary, right? That portable

09:13 WebAssembly thing. Yeah. I should look that up. I don't know where they are with that, but I know

09:17 that was a thing that folks were getting really excited about. Not just from the fact of, hey, I can run my

09:22 Python code or my language X code in the browser, but also too, I can consume it from other languages

09:29 that would make it. Right. If somehow you can package your code in whatever language you use in a WebAssembly

09:35 format, all of a sudden it becomes interoperable with other WebAssembly library, anything else that

09:41 can be packaged into WebAssembly basically, right? Right. And what's good about that is it's, it's

09:45 relatively a standards compliant way where, because WebAssembly is a standards based thing. It's not

09:49 owned by a company or one particular runtime. All of us run and play in the web and use browsers and use

09:56 browser technologies for one thing or the other. So it's definitely something that everyone can kind of

10:00 get involved in one way. Yeah, for sure. I'm super excited about that as well.

10:04 All right. Moving on to the next one, your topic here. You've got something very small that's

10:07 actually quite cool. Yeah. And so just kind of talk about interoperability a little bit. One of the

10:12 things that we all use a lot as developers is Git, right? Like I use Git a lot and particularly in the

10:18 GitHub space. Actually, I don't think I know a developer today that does not have a GitHub account,

10:22 but you know, maybe that happens. Do you remember when people used to debate about what kind of

10:26 version control they use? Like, oh, we use Perforce. Oh, we use Subversion. Oh, we're still

10:31 on CVS. We use a WinC. I use Tortoise CVS and I got a right click. That used to be a debate.

10:36 That debate is gone, it seems like.

10:38 I remember, man. I remember there was a debate back when Git first started too. There was a debate

10:43 between Git and Mercurial because those were the two distributed version control systems. And people

10:49 were like, oh, I'm going to use this one. I'm going to use that one and trying to figure out like

10:52 which one that was going to be the thing to use. But I think it's pretty safe to say that

10:56 in GitHub have won. Yeah. Pretty much. Yeah. GitHub too. Yeah. GitHub's awesome. Yeah. I think

11:02 they've pretty much just won. So now this article, why I think this is really interesting is, you know,

11:07 as a developer, since we, again, Git is pretty much like the standard of source control for all of our

11:12 projects. It's important for us to understand how Git works, or maybe you might be curious about what the

11:17 underlying internals of that looks like. And so what this is, is a really cool tutorial.

11:23 It's an interactive tutorial that you could do kind of in your browser as well, where you could

11:28 play around and implement Git yourself, your own version of Git in Python. And I'm like, oh, that

11:33 sounds cool. Like, I definitely want to do that. Because one, I think, hey, you're just giving me

11:38 another project that I could get better at writing Python. But two, I can also really try and understand

11:43 what Git does. I know a lot of folks, you may know a few Git commands, but oh, man, like, I really don't

11:48 understand what rebase does. And I don't know, what's the difference between Git branch and Git switch? And

11:54 you know, like some of those types of things. And so I'm hoping like we could go through this tutorial. And by

11:59 implementing it, we could really understand like how the underlying data structures are different and how that

12:04 Git database is different.

12:05 That's so interesting. And thinking about understanding Git by re-implementing it from scratch is pretty

12:12 interesting. Yeah. And I think it's worth pointing out just the format of this article tutorial thing.

12:19 It's quite unique. So if you open it up on the left hand, the browser split 5050, the left hand

12:26 column is the description and message. And on the right, it's basically got a diff of the code that

12:34 you're writing at each step. So it doesn't look like you can execute or anything, but it's a little bit

12:39 like replet or something like that. Yeah, I'm looking at this. And I think I was trying to see

12:44 if you could click on this thing, if it'll take you to the source. But I think this is in addition to

12:48 just being a really cool topic. I think it's an interesting... Oh, it does. As you click around it,

12:52 it'll, it'll reload the side to like take you to different parts in the code. Yeah. Oh, does it?

12:56 Okay. Yeah. Yeah. So there it does. There you go. There it does it. But I think this is a really

12:59 interesting experience just for learning in general, right? It's very interactive. So I could read the text,

13:04 the verbiage on the left hand side, and I could look at the code and I can see how it changes on

13:08 the right side. So it's not just like, Hey, I'm just reading the super complex book about like get

13:13 internals. Like I can actually see the code, which, you know, for folks that like to learn by seeing,

13:18 it's a really good experience to be able to understand like what exactly is going on.

13:22 I might have to go through this. This looks really interesting. And I might learn a couple of things by doing it for sure.

13:28 Yeah. Yeah. This definitely looks really cool. But now speaking in terms about learning,

13:32 like you had a topic you want to talk about, right? That has to do about learning, right?

13:36 I would say you and I probably these days we qualify as senior software engineers.

13:41 I don't know. I don't really want that title on me anymore. Cause I don't know. It sounds like I go to

13:46 an office. That's my title, but nonetheless, we've been around software for a while. Right.

13:51 And there's this cool article by Neil Kakar who wrote it. It is an in-depth write-up of things I

13:59 learned to become a senior software engineer. I love these things where people sort of look back on

14:05 what they've done in their career, how it's, you know, what went right, what went wrong, and just

14:10 trying to help other people short, shorten that cycle to get farther, faster, quicker, that kind of

14:15 thing. So there's a lot of lessons he talked about. And I just wanted to cover some of them.

14:20 I said it's really long. And so I'm not sure I want to go into all of it there, but so the first one was

14:26 he advanced his career by growing his skills and experience using different ladders of abstraction.

14:33 That's how I put it. So it looks, the first year or so he had basically learned the ins and out of

14:39 software. Like he knew his language, his compiler was working and I believe he's doing Python. So maybe not

14:44 his compiler, but all the stuff that he's doing to work there is pretty much figure out the software side

14:49 of things. But his software development cycle was like a small cog within a bigger cog and so on.

14:57 So he said, look, there's really the life cycle of the product. There's a life cycle of the

15:02 infrastructure and the business and like working your way and sort of out of that loop to understand the

15:09 bigger pieces really helped him do more than just like write code, but like really be a bigger part of

15:15 the team.

15:16 This is really interesting. And I'm kind of scrolling to the article again, as you're talking

15:20 about it. And it brings up to mind, like a discussion that I get a lot from some of the students that we

15:25 talk to, you know, with the different engagements we do. And that's always their question. It's either

15:31 one, how do I become a good software developer or how do I level up in my career? Like what are the

15:36 things that I need to do to step forward in my career? And sometimes I feel there's certain things that for sure you could

15:43 definitely do. But then I think that's only like 50% of it. And I think there's another 50% that's could be a

15:49 little subjective based on the area that you're in and the region that you're in. And like, what are the

15:55 things that you really want to do? Like, what are the things that you really care about? Like, I could

15:59 tell you when I first started, honestly, I was just, I was just trying to get a job. I wanted to leave

16:05 school. I wanted to work because being an international student that I had to worry about

16:10 like visas and, you know, green cards and all that type of stuff. And I'm like, the first person that

16:16 hires me, I'm gone. Like, that's it. Right. But obviously that's not something you base longevity on

16:21 when you think about a career, right? You want to think about, okay, now I'm working and, you know,

16:26 I, maybe I've spent a year or a year or two inside of the environment and you, you know, you know,

16:31 you know how to code, you know, how your deployments go. You understand your ticketing

16:35 system and all the scrum meetings that you have to sit in on, but now you got to understand, well,

16:39 well, how do I get better? And one of the things I always tell folks is, well, it's kind of like being

16:46 a specialist, right? Like what are the things that you want to get better at? Because there's so many

16:50 different things to look at. What are some of the key things that really interest you? So do I want to be

16:56 really good at doing security? Do I want to be really good at doing performance? Do I really want to

17:01 get good at maybe microservices and distributed systems is interesting? Right?

17:05 Right. You get pigeonholed and people will just leave you there if you don't try to stretch.

17:10 Right. But you have to want that though, right? Like you have to want to get better. And then after

17:14 you have that want, you got to figure out where exactly do you want to go? And then for me,

17:18 then the next step after that is really trying to understand, well, what are the fundamental pieces

17:23 that I need to understand? Like what are the fundamental algorithms and data structures I need to

17:28 understand? And I'm not talking about like link lists and bubble sorts and that type of stuff,

17:32 but like within that space, right? Like, so in, when we talk about security, like what are the things I

17:37 need to know about cryptography? Right. When I talk about like distributed systems, what do I need to

17:41 know about networking and how networks work? You know what I mean? Like how do I get the knowledge to

17:47 understand that? So now I could continue to move forward and really become like very knowledgeable and

17:52 successful in that particular aspect of software development.

17:55 Yeah. I think that's a really big part of it. And that's definitely some of the stuff they were touching

18:00 on. I think one of the big levers you have to pull is to try to get yourself to do stuff that is

18:06 uncomfortable, right? Like I don't really know about continuous deployment, but we don't have a great

18:11 answer for that. So I'm going to force myself to learn that and I'll volunteer to be the person to make that

18:16 happen, even though I'm not totally there yet. Right. But that'll get you there if you take those steps.

18:20 Another one that Neil pointed out was learning what people around him are doing. So not just what are

18:27 the developers doing, but what are the project managers, the salespeople, the analysts and so on.

18:31 You know, for you working in a company like Microsoft, there's layers and layers and layers of that, right?

18:36 Yeah, for sure. And you know, there's good and bad things to that. One of the good things I like about

18:42 that is that there's always people that you could ask a question to. And sometimes you might be in

18:47 environment that that's not really the case. Sometimes you're in a company and everyone

18:50 is at the same level that you are. Yeah. You can't talk to other people, at least,

18:55 or maybe you don't have access to talk to other people. But when you're in an environment that

18:59 there's a lot of folks that you can ask questions to, one of the best things that you could do for

19:03 your career is make sure that you spend time with them, however that works, to really understand like,

19:09 how does their world kind of move? Particularly if it's something that you're interested in.

19:13 So again, like, hey, I don't know a lot about security and I have a security feature I need

19:18 to work on. And it's something I really want to dive deep into. Let me go and find the PM that does

19:22 security for this thing. Or let me go and find the person that works under Docker integration for this

19:27 thing or the Python integration for this thing and really have like a deep conversation with them.

19:32 And the good part about it, if that person is willing to sit down with you and give you the time,

19:37 that is like gold. Like it is the best thing ever to be able to have someone pass on knowledge to you.

19:42 I say that too, because we move forward. And as we become quote unquote, senior engineers,

19:47 we have to remember too, that the folks that are coming that are not so senior,

19:50 they're going to want to do the same thing for us, right? And we have to have that mindset of being

19:54 willing to share that information, not try and keep it inside and take it with us when we leave the

19:59 company kind of thing.

20:00 Yeah. Well, the other flip side is, you know, the best way to learn something is to teach it often.

20:04 And that's your opportunity. Yeah.

20:06 Yeah, exactly. Exactly.

20:07 Another interesting lesson was strategies for making the day to day more efficient,

20:12 which I suppose the larger the company, the more important this can be.

20:16 And Neil said some good habits. Notice was never leave a meeting without making a decision or having

20:21 a next action to do. Meetings drive me crazy. And it's so often you just have a meeting because

20:26 we always meet on Mondays for half an hour, even if we don't have to like, no, please not again.

20:31 Decide who's getting a thing done because if no, it's not on, you know, somebody's responsibility

20:36 to get it done is probably not getting done. And then document design decisions made during a project.

20:42 So that was pretty cool. Those are good ones. Yeah. Acquiring new tools for thought and mental models.

20:48 So one example in the software space would be like, it was recently struggling with the domain of a lot

20:55 of complex business logic and all these weird edge cases and stuff. And so found the domain driven design book

21:03 by Eric Evans and that whole space and said, oh, there's a whole world over here around like modeling objects

21:09 and relationships that I didn't understand or didn't know about. And diving into that stuff's pretty interesting.

21:15 Here's my favorite. And this has driven me crazy about so many different things.

21:21 Protect your Slack. So not your Slack, the not competitor, not your channel, but Slack as in,

21:29 I've got a little bit of time in my day-to-day schedule that is not clearly assigned, right? Not like

21:35 I'm spending the next two hours on this feature because I agreed to it in the sprint and then I'm done and I

21:42 have to move on to the very next thing, right? If it's so tightly packed that you don't have the

21:47 the freedom to explore, like that didn't quite work the way I expected it. Let me dig into that. Or

21:55 what if we re-implemented it this way? Let me just try that idea. Or I don't know anything about domain driven design.

22:00 So let me learn. Let me spend the next hour reading about that and see if that'll help us later.

22:05 If all those opportunities are taken away and your time's like squished just back to back to back, you end up just doing 20 years of the same thing instead of actually growing and learning new stuff.

22:16 I'm a big fan of companies that allow their employees, developers or not, time to sit down and experiment and try out new things and whatever.

22:26 You know, prior to coming to Microsoft, that was in a company that they used to have, like every time there's a release, right after the release, they had this thing.

22:35 Oh man, what was it called? I don't remember what it was called, but it was like in events.

22:39 It was like a whole company event where for like the next week, all they could do was just work on side projects.

22:45 Like it was a whole week, right?

22:46 Oh wow.

22:47 Obviously the service folks had to be on service, but for everyone else, like the PMs, the engineers, the developers, you know, they spent this entire week just hacking on stuff.

22:56 And then the next week, you know, there was a day where they gave presentations and everyone talked about what they did.

23:01 And it was a whole showcase, but it was only internal.

23:03 It was just for the folks inside of the company.

23:05 And not only did you have time to do that and learn and try new things out, but you know, there were prizes, right?

23:11 Like they were, whether it was stock or money or like just, you know, there was stuff that you got, which was really cool.

23:17 So now it shows like how much the company like really believed in you.

23:20 And so again, I bring it up because I think it's an important one.

23:23 Not only do you give your employees, if anyone is an employer that's listening, that you give your employers, employees, the ability and space to learn, but also incentivize them to do it, right?

23:33 Like really encourage them and let them know that you're supporting it.

23:36 And regardless of whether that means it's something that's going to go in your product or not, like the fact is like they're becoming better, right?

23:43 And so if you, all of your employees are becoming better, like that means that they're going to be able to do better work for you at the end of the day.

23:49 And so I think that's always a really cool thing to do.

23:51 Yeah, absolutely.

23:52 I think this concept of Slack is super important.

23:55 And so much people looking from the outside, they try to squash it and compress people's time down.

24:00 And I think it's in the end, it's not so good.

24:02 So there's a ton of other stuff.

24:04 I want to just throw out really quick.

24:05 One last thing in the sake of time.

24:07 So we don't spend too much time on this.

24:08 The last was this idea of building up superpowers.

24:11 So are you getting into the source code of some project, some package and the documentation isn't enough?

24:18 All right, you have a quest.

24:19 That's to go read the open source code and spend some time like learning from the code, not from the docs.

24:25 You want to quickly build a mental model for code you're looking at?

24:28 Reading open source code again.

24:29 Embracing fear.

24:31 You're something you're not competent or feeling comfortable with.

24:35 Do it as a side project.

24:36 That's your quest.

24:37 Confidence to express ignorance or not knowing.

24:40 Like, fine, right?

24:41 Just we all got to ask questions we don't know and get over it.

24:45 And think you practice with that.

24:46 There's just a lot of good takeaways from here.

24:49 And like, we really just scratched the surface on it.

24:51 Sure.

24:51 And kind of going back to that point about reading the open source code.

24:54 I think that's one of the reasons, too, why GitHub is so popular.

24:57 Because it made searching source code so easy.

25:01 And I'm not going to lie.

25:02 There's a lot of times throughout my day that I'm thinking, hey, the documentation is not the best for whatever it is that I'm looking at.

25:10 I'm going to search for like the class or the method.

25:13 I literally just look for the class name and the method and filter by language.

25:16 Just to see, well, I just want to see how are other people using it.

25:19 And, you know, what are the types of issues that are created around it?

25:22 And, hey, should I wrap this in a try catch?

25:25 Or, you know, like what kind of parameters do you pass in?

25:27 Like, how do you deal with these things?

25:28 And being able to kind of look and see how other folks do it gives you some ideas now.

25:33 Okay, well, this is how this thing was meant to use.

25:35 Now, the flip side of that is that, you know, you might run into a bad example.

25:39 But, again, that's a learning experience, right?

25:41 Like, you'll run it and you'll figure out and then, you know, we're working out.

25:44 Yeah, exactly.

25:45 That's great.

25:46 So, speaking of learning, you guys have a pretty cool project.

25:50 A data source, I guess, to learn from or context to learn from?

25:53 Yeah, yeah.

25:54 So, what's happened over the past, oh, man, over months and months and months.

25:59 So, Microsoft actually started a project where they're working along with NASA to create these learning modules on Microsoft Learn.

26:07 And we can kind of go in and learn how to do, like, machine learning with Python.

26:12 But you could do it within the context of exploring space, which is really cool.

26:16 So, you could do different things like classifying space rocks and predicting weather.

26:21 It'll help you, like, predict, you know, rocket launch delays based on the weather.

26:25 Like, tons of great stuff.

26:26 And so, there's a...

26:29 I'm trying to think about how many are there.

26:30 There's a one, two, three.

26:31 Yeah, there's at least six learning modules, again, all based on Python and AI with machine learning and NASA, where you can kind of go in and learn all these different things.

26:40 So, definitely recommend that folks go in and check it out.

26:42 It's 100% free.

26:43 So, you could, you know, just click on the link and kind of go through them yourselves.

26:47 You don't have to sign up for anything, which I love.

26:49 Like, I love free learning resources, right?

26:51 But, yeah, I thought this was really cool.

26:53 And if anyone's a space buff at all, like, this would be a great adventure for you to kind of go through and kind of play around with some data from NASA.

27:01 Yeah.

27:01 To learn how to upset space.

27:03 Yeah.

27:03 And so much of being successful, especially when you're learning early, is about having context that inspires you.

27:10 Yeah.

27:11 Because we all get stuck or things get frustrating.

27:13 You're like, why won't it?

27:14 Why is this function not doing what it's supposed to do?

27:16 But if you're like, and if I get it to work, I get to understand a space rock or something like that, all of a sudden, it's like, yeah, that's not so frustrating.

27:23 Right, right.

27:23 And I think NASA also has, for folks that want to get a little bit more into NASA, NASA also has, like, I think it's called NASA Socials, which is like a national social club where you could, again, work along with other folks in the community.

27:36 Back before social distancing, you could actually go to a social event and you just learn different things about the work that they're doing with NASA.

27:43 So that's also really cool, too.

27:44 I think you could do those social events online.

27:46 So if you're interested, probably like, hey, maybe you learned a little bit in doing some of these courses and you want to engage with other like-minded folks that are into space exploration, but they don't necessarily work at NASA.

27:57 You know, that might be another thing to definitely check out.

27:59 Yeah, absolutely.

28:00 The next one I want to cover has to do with performance.

28:03 And I don't know if you've ever had this experience, but I have definitely had it where the code is going slower than I expect.

28:10 And I'm like, oh, it's got to be this part of the code that is so slow.

28:13 Yeah.

28:13 And then I profile it or I try to fix it.

28:16 And it's absolutely not, it has nothing to do with it.

28:18 That's like 5%.

28:19 It's somewhere else.

28:19 It's completely somewhere else.

28:22 Have you had this experience?

28:23 Yeah.

28:23 And you know what happens?

28:24 It's always a database.

28:26 Have you heard that?

28:28 There are no indexes.

28:29 Yeah.

28:29 People always say that.

28:30 There's like, oh, you have a performance problem?

28:33 It's always a database.

28:34 I know it's not always a database, but I totally get what you're saying.

28:37 There's definitely cases where, because who else would know your code better than you would?

28:41 And so if you're thinking, oh man, if this is where the problem is, this has to be where it's going.

28:45 Yeah.

28:46 Well, we just have such a, often we have a bad conceptualization of what is fast and what is slow.

28:52 Yeah.

28:52 Because I don't know, computers are so fast.

28:54 It's a little bit crazy.

28:55 Yeah.

28:55 But profiling is usually the first step to figure out what that answer is.

29:00 And then if you're going to change it, you can see, did it get better?

29:02 Did it get worse?

29:03 Where did it get better and worse?

29:04 And so on.

29:04 Right.

29:04 So if you're doing Django, I want to talk about profiling requests there, because there's a couple of interesting options.

29:11 So there's one library that will let you plug in basically C profile, which is the low level C profiler from Python that chips with it right into your request.

29:24 So if you plug it in, you can just do a question or an ampersand prof or question mark prof at the end of URL and you'll get a profiling output.

29:33 Oh, nice.

29:33 It's pretty cool.

29:34 So there's a thing called Django dash C profile dash middleware.

29:37 You just install that.

29:38 And then if it's installed, you can put basically prof in the query string and you'll get that, which is great.

29:44 And that gives you an output there.

29:45 Or you can put ampersand download and it'll download it.

29:50 You can visualize it graphically with snakebiz.

29:52 So that's even better.

29:54 So you get like this download profiling report for your request.

29:57 All that's pretty good.

29:59 But what would be a lot better is to be able to like visually dive in to the request and do things like understand, you talked about the database being slow, like understanding like the time you're spending in the database versus other places.

30:11 So there's this thing called Django Silk.

30:14 Okay.

30:14 And you also add that as a middleware, but then it keeps track of all the requests on your site.

30:19 Hopefully you're doing this in development, not production.

30:22 You keep track of all of those.

30:24 And then it'll show you like, this is the general time for this URL.

30:27 Click on it.

30:28 It'll show you things like these are the database queries that were run actually as SQL, even though using RM, it'll show you the SQL that was run.

30:36 It'll show you how much time the database is spending, how much time other parts of your code are spending.

30:41 So yeah, a really cool way to explore the performance from like all the tiers of your Django app.

30:48 Yeah, that's really cool.

30:48 And I always find database interceptors to be a really powerful concept, like across all languages, right?

30:55 Like being able to see like, like, let's say I'm using SQLAlchemy or I'm using, you know, Mongo Engine or something like that.

31:01 Right.

31:02 Like I'd really want to know what's the query that I'm sending over.

31:05 Like what does it think I'm asking for versus what I wrote in code, right?

31:09 And what am I actually getting back?

31:11 And being able to see, oh man, like this thing is doing like four or five joins.

31:15 Like I don't need to do that.

31:16 Like I could try and optimize this query a little bit more to one, make it more performant from a database perspective.

31:22 And then two, obviously passing on that, that performance over to my application.

31:26 Or I thought I did one database request.

31:28 Why are there 51?

31:29 Right, right.

31:31 Exactly.

31:31 Like, oh, that lazy property that, that we forgot about every single time through that loop, we're hitting the database again.

31:37 Like such a bad idea.

31:38 Maybe we should do a subjoin, like eager, eager load sort of thing.

31:42 Yeah, for sure.

31:43 Yeah.

31:43 Yeah.

31:44 So I think this is really nice.

31:45 Yeah.

31:45 Yeah, it is really nice.

31:46 And this is one of the scenarios too, where I think it's interesting to get like a log collector.

31:51 So, you know, there's tons of services that do this, but like something like Datadog or Rollbar or something like that, that not only capture your logs, but allow you to kind of go in and search through them, search and filter your logs.

32:02 So you could say, okay, well, hey, I'm having performance problems, but it only happens at two o'clock in the afternoon for some reason.

32:09 Or, you know, my memory always spikes within these hours of the day.

32:13 And being able to have a history of logs and be able to query them and then get that rich information.

32:19 And I don't know whether you plot it or turn it into a report or something like that.

32:23 Being able to do that, I think is hugely important.

32:25 Yeah, absolutely.

32:26 I totally agree.

32:27 And being able to visualize it, right?

32:29 So often having the picture is what you need.

32:31 Sometimes, though, it's good to just get a text.

32:34 Yeah, that's true.

32:36 Sometimes it's good to just get a message or get like an alert.

32:38 Hey, Michael, the website is going down.

32:40 You need to get here ASAP.

32:42 Exactly.

32:43 And so that kind of leads into the next one.

32:45 That was a good segue.

32:45 I like that.

32:46 Thank you.

32:46 So that can lead into the next topic I did want to talk about.

32:50 At the time that we're recording this, this week is actually Microsoft Ignite, which is, you know, one of the conferences that we do.

32:56 And it's all online.

32:57 We announce products and tons of stuff like that.

33:00 The last time that we got to actually see each other in person and hang out a little bit was at Microsoft Ignite in Florida.

33:07 And I was blown away at how ginormous that conference is.

33:10 Like you were on Talk Python then and Python.

33:12 Right, right.

33:13 That's 30,000 people.

33:14 So I guess it's 30,000 video streams now.

33:18 Yeah, I remember that last time we were all together.

33:22 It was in Orlando.

33:23 And Dan Bader was there.

33:24 I remember.

33:25 Yeah, that's right.

33:25 Episodes with Dan.

33:27 I don't even know what the numbers are this year.

33:30 That would be interesting to find out.

33:31 I should look that up.

33:32 Yeah, for sure.

33:32 See like what those numbers kind of seem like.

33:34 So it's been a year and you guys have a new announcement.

33:37 Yes, it's been a year.

33:38 And so there's a new service.

33:39 And so this is Azure Communication Services, which is cool.

33:43 One, because, you know, we have a new service.

33:45 But two, like I had no idea this was coming out.

33:46 I kind of found out like everyone else did on social media.

33:50 So this is Azure Communication Services.

33:51 And essentially, it's a way for you to add, well, communication services to your application.

33:56 So it does a lot of cool things.

33:58 Like it does voice, it does video, it does text messaging.

34:02 And you can obviously receive and send phone calls and things like that, which is really cool.

34:06 But what's even better about this, more specifically for the folks of this show, is that there's a Python SDK on day one.

34:13 I guess we could call it day zero since it's not officially out yet.

34:16 But, you know, on today, you know, in preview, while the service is in preview, there's a Python SDK, there's a JavaScript SDK, and there's also a .NET SDK.

34:26 And I looked at the code for the Python SDK, and there, I almost want to say like, there's nothing there.

34:31 I mean, obviously, there's something there, right?

34:33 But most of the code is like, create a directory, create a virtual environment, install pip, SMS client, that send message.

34:40 I'm like, wait, that's it?

34:41 Yeah, it's like, literally, the meaningful bit is like five lines of code.

34:46 Right, exactly.

34:46 And so it looks like a really easy thing to kind of get started with.

34:50 And so I haven't played with it yet.

34:51 Again, I just literally discovered this yesterday, kind of like everyone else did.

34:55 So I'm really interested in plugging it into some of my demos and seeing like what the experience is really like.

35:00 Yeah, this looks really cool.

35:01 If you got to send, if you want to add text integration, like text alerts to your app, seems like a really good choice.

35:06 Right.

35:06 And so then we could plug in, imagine if we could plug that into the Django profiler.

35:11 And then as things are happening, it sends us text message alerts.

35:14 And so now we can know, oh, there's a problem with the website now.

35:17 That's right.

35:18 It's running slow.

35:19 The memory's too high.

35:20 Exactly.

35:20 Check it out.

35:21 Yeah, that's awesome.

35:22 Nice one.

35:23 People can check that out.

35:25 I guess that's it for our items for the week.

35:27 Quick shout out.

35:28 I just say I was on, you mentioned Dan Bader.

35:31 I was on Real Pythons podcast with Christopher, the host there last week.

35:35 So I'll be sure to link to that.

35:37 It was a really fun look back on the trends and stuff I've seen over the last five years of podcasting in the Python space.

35:44 So if you want to hear about that chat, follow up on that show.

35:47 And then how about you?

35:49 Anything else you want to share?

35:49 One of the things that our team does is they work with students from different universities across the world.

35:55 And, you know, they join a program that we have that's called the Microsoft Student Ambassador Program.

35:59 Now, if you're familiar with like Microsoft MVPs or Docker captains or what is it called?

36:06 Google Developer Experts is something very similar to that.

36:10 But it's just this time we're talking about students.

36:12 Right.

36:13 And so, you know, these are students that are going out and they're creating content and doing sessions and workshops and, you know, writing blog posts and different types of things within their school and within their communities.

36:22 And so, you know, we have them join, you know, this program where we're able to support them a little more and give them Azure credits and Visual Studio things and LinkedIn learning things and some private events.

36:34 Yeah.

36:35 Yeah.

36:35 They have little private events and mentorship sessions and things like that.

36:38 But, you know, it's a really great way for us to kind of engage with the students.

36:41 Right. And I really just understand, like, what are they doing?

36:43 What do they care about?

36:44 What are some of the ways that we can affect their learning?

36:46 And so what I wanted to do was share the link to the Microsoft Student Ambassador website.

36:51 And you can actually kind of go here and you can search by region, by language, by area of focus.

36:56 And you can kind of see what the students are doing, which I think is really great.

36:59 And if anyone that's listening is a student as well, a university student that's interested in joining the program, you know, you go ahead and hit that big blue apply now button.

37:09 And, you know, go ahead and submit an application for the next for the next cohort.

37:12 I think that'd be a great opportunity for students.

37:14 So people should check that out.

37:16 All right.

37:16 Let's finish this off with a joke.

37:18 And we're better to go than XKCD.

37:20 I love SKCD.

37:21 I know they got some good stuff.

37:23 So this one is entitled Dependency.

37:27 And I think it just captures the whole bring together different languages and technologies and especially a bunch of open source libraries.

37:35 Do you want to describe this picture for the listeners?

37:38 You can always grab graphical stuff for our jokes.

37:41 Sure.

37:41 So this reminds me of a game of Jenga.

37:43 That's what I'm thinking.

37:45 Like a super Jenga.

37:47 This is like recursive Jenga.

37:49 Yes.

37:50 That's what this looks like.

37:51 And so there's some big blocks and there's some little blocks.

37:54 And it looks like as the closer you get to the top, they start to become a lot more specific with like the type of blocks that are up there.

38:01 And then at the bottom, it's much bigger at the bottom.

38:04 And, you know, that looks like that could be trouble.

38:06 Oh.

38:07 And then on the right side, there's one little part that's holding everything up.

38:11 And it's really, really small and fragile.

38:14 At the top, it says all modern digital infrastructure.

38:18 And at the bottom, it says the little tiny stick is a project some random person in Nebraska has been thanklessly maintaining since 2003.

38:27 That's amazing.

38:29 I love this.

38:30 In the hover over messages, someday image magic will finally break for good and we'll all have a long period of scrambling as we try to reassemble civilization from the rubble.

38:41 To me, I think of this as OpenSSL.

38:45 That little stick, that's OpenSSL down there.

38:49 There was like one person who maintained it and everything seemed to be built upon it.

38:55 And when there was that bug that needed to fix them, it was a huge problem.

38:59 I find anytime I try to home, so I use a Mac.

39:03 And every time I homebrew install OpenSSL and it updates the version that's on my system, like everything just stops working.

39:11 Like I can't install any pip packages.

39:13 Like nothing works.

39:15 I'm like, what happened to my virtual environment?

39:17 Like what's going on here?

39:18 Why can't I install anything?

39:19 Then I'm like, oh, OpenSSL.

39:21 Yeah, exactly.

39:22 That's the stick that can bring it all down.

39:25 Well, thank you for being on the show, Cecil.

39:27 It was great to catch up with you.

39:29 You had a lot of good stuff to share.

39:30 Hey, thank you for having me on.

39:31 And Brian, I know you're not here, but I hope I did you service and hope you come back soon.

39:37 And also too, I didn't get to say this in the beginning.

39:40 Congratulations on 200 episodes.

39:42 Well, over 200 at this point.

39:44 Remember, I listened to episode 199.

39:47 I'm like, wow, has it been like that many episodes already?

39:50 Like it's crazy.

39:51 Yeah, that's coming up on like four years almost.

39:53 It's insane.

39:53 Wow.

39:54 That's awesome.

39:55 Well, congratulations to you both, man.

39:56 That's a great achievement for podcasting.

39:58 Yeah, thanks a bunch.

39:59 And take care.

40:00 Take care.

40:01 Thank you for listening to Python Bytes.

40:03 Follow the show on Twitter via at Python Bytes.

40:05 That's Python Bytes as in B-Y-T-E-S.

40:08 And get the full show notes at Pythonbytes.fm.

40:11 If you have a news item you want featured, just visit Pythonbytes.fm and send it our way.

40:16 We're always on the lookout for sharing something cool.

40:18 On behalf of myself and Brian Okken, this is Michael Kennedy.

40:21 Thank you for listening and sharing this podcast with your friends and colleagues.