Transcript #201: Understand git by rebuilding it in Python

00:00 Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:04 This is episode 201, recorded September 24th.

00:08 I'm Michael Kennedy and Brian is not here this week.

00:11 Instead, my friend and special guest, Cecil Philip. Welcome, Cecil.

00:14 Hey, hey everybody. I've been a long fan of the show.

00:18 And so, you know, Brian, I'm sorry that you're not here, but I promise I'm going to do a good job and do you justice.

00:23 So I'm going to do my best Brian impression today on the podcast.

00:27 Beautiful, beautiful.

00:28 So Cecil, you've been on the show before.

00:30 You're a developer advocate at Microsoft. Is that right?

00:33 Yep, still a developer advocate.

00:35 And as folks can imagine, we're all at home and not traveling as much as we used to anymore, which is usually something that's attached to the job.

00:42 But now we're just trying to find different ways to kind of reach out to the community and work with folks and make sure that they have what they need to just be successful and productive.

00:52 Yeah, absolutely. And you're doing a lot of live streaming stuff these days rather than appearing in conferences.

00:57 Yeah, we're doing a lot of live streaming, a lot of live events, and even live conferences too.

01:01 I think a lot of conferences now, and you've seen this a lot as well with PyCon and tons of the other tons of other online conferences is just because we can't do it physically together like we're used to, now a lot of folks are just doing it online.

01:14 So I'm sure if you look online, you see tons of register for free, here's my conference on Tuesday and Wednesday and Thursday, and everyone's just trying to find a way to connect.

01:23 Yeah, it's got some interesting implications for accessibility, right?

01:27 I could necessarily go to EuroPython, but I could attend the virtual one just as easily, as long as I'm willing to get up a bit early.

01:35 That's kind of cool, actually.

01:36 Yeah, that's one of the things I like about it the most, is that, kind of like what you said, like before, I didn't want to go buy a plane ticket to go to Australia, or, you know, somewhere that's maybe halfway around the world to me.

01:47 But now I can have, maybe not the same experience, but I could still engage with that community of folks virtually by just signing into their YouTube, or their Twitch stream or whatever the case is.

01:57 And I can get the content the same way.

01:59 And it's great.

01:59 Yeah, absolutely.

02:00 So let me kick this off with our first item, where we go from Python to speak another language.

02:05 A little different cultural here.

02:08 So we're going to talk about talking from Python to C++.

02:12 Maybe not what you thought when I said that.

02:13 Really?

02:14 C++ is one of these, or C really, the more low level, is one of the reasons I think Python is doing as well as it is these days, right?

02:23 Like a lot of what you do in Python might be a little bit slow, but you know what?

02:27 You implement part of that package in C, and then all of a sudden, it's a lot faster than things you might compare it to.

02:33 And that's a lot of the magic.

02:34 A lot of the data science works that way and whatnot, right?

02:36 - Right. - Yeah.

02:37 So there's this article that I'd like to point folks to that basically walks you through what it's like to call C and C++ code from Python.

02:47 So there's a lot of things to cover that's pretty interesting.

02:51 It talks about, First, what do you get when you compile C? Obviously, you get something that's executable.

02:57 What it is actually different depending on the machine you're on, right?

03:00 So if you're on Linux, you'll get an executable in linked format, which is ELF.

03:04 On Windows, you get a portable executable.

03:07 On macOS, you'll get a mock object, a mock.o, and so on.

03:13 So it compiles to whatever machine instructions you need.

03:15 And then, depending on what platform you're on, again, you can put that in some central location so that compiled C library can be used by Python.

03:24 So on Linux, it'd be user/lib.

03:27 On Windows, what do you think?

03:29 Windows/system32, something like that.

03:32 - System32 or something crazy like that, yeah.

03:34 - Yeah, exactly. Something there.

03:36 But once you put it there, then your program can find it and you can just import it.

03:41 So there's different ways you can do this, right?

03:43 Like you can use the C types library, which is simple, sort of, sometimes simple.

03:48 And you can just go and import C types and then just say, here's a CDLL and point literally at the file, the output from compiling some C code, and import it and call it.

03:59 That's pretty simple.

04:01 Coming from Microsoft Word made this DLL import type of equivalent.

04:06 That's why I'm not entirely sure.

04:07 But there's some interesting things that you run into, like, you get this library back, but it doesn't necessarily know what the return types are, right?

04:15 Like, you get a function, you call it, it returns a float.

04:17 Maybe it doesn't know what to do with that.

04:19 So you've got to like, as you import it, tell it, Oh, you return a float here.

04:23 And here are your functions that you can call and so on.

04:26 So there's a lot of interesting things you can do.

04:29 If you remember your C, you have to set the calling convention, which can be really tricky, like extern C.

04:35 Oh my gosh.

04:36 Oh, is it standard call?

04:38 Is it a C decal?

04:40 I mean, there's so many variations.

04:42 Do you remember this?

04:43 That's like, did you ever do some C and just go, why won't this library work with that other one?

04:47 I know now, like, I'm looking through this document as you're talking about it.

04:50 And I'm like, Oh my gosh, this is bringing back so many memories.

04:53 Oh, yeah.

04:54 I mean, you remember, like, how hard these things were to get them right.

04:59 And it reminds you how nice it is to work with like more modern languages like Python.

05:04 So there's actually an interesting library called libffi.

05:09 So this is a C library.

05:11 So if you're going to write the C code, you can use libffi.

05:14 And when you compile it, as you compile it for different platforms and stuff, it'll automatically figure out the calling convention for the functions at runtime.

05:21 Oh, nice.

05:22 So that's pretty sweet.

05:23 So you can, I mean, this didn't exist to my knowledge when I was trying to do these things.

05:27 That's interesting.

05:28 Another thing you can do is you can add an entry point function to your code.

05:35 And then Python will ask, basically, it can call that and say, what functions do you have?

05:41 And what are their signatures?

05:42 So there's a way to have your library kind of introduce itself over.

05:46 It's kind of like providing metadata, I'm guessing.

05:48 Yeah, exactly.

05:49 Here's the structure of my stuff, and I'm going to give it to you.

05:51 And now you can kind of understand it.

05:53 Exactly, because you don't have a header file or anything like that, that defines it.

05:57 It's just, here's the executable compiled functions, go, right?

06:01 And so it gives you that metadata, exactly.

06:04 Of course, the goal is to have C++ code that you can call from Python.

06:09 And there's a Python-specific, a Python-friendly way to do this as well if you're willing to work with a Python type, so like Py object pointers and so on.

06:20 And the way you would do that is you create this entry point, but it's the function name is py_init_module_name.

06:28 So if it was request, it'd be py_init_request.

06:31 Exactly.

06:32 And so it returns all this metadata that says, here's all the C stuff, and the signatures and whatnot.

06:40 So after going through all this stuff, how you write the C code and you do it, it says, you know what, maybe you just want to skip that.

06:45 Yeah, can we please?

06:47 Can we please? Yeah, so what you could do is you could write it in Cython, which compiles to C, which compiles to machine instructions.

06:55 Or there's also boost.python.

06:59 I wasn't aware of boost.python, that's pretty cool.

07:01 And pybind11, which allows you to bind, I believe, to C++ 11, which is pretty cool as well.

07:09 Yeah, so really nice integration stuff there.

07:12 And some examples of libraries that people might know of that are written in Cython, but basically natively compiled and do awesome stuff is AIO HTTP.

07:22 Oh, I didn't know that. That's cool.

07:24 Yeah, uses Cython for the HTTP parsing.

07:26 So that string stuff is super fast.

07:28 If you do async, you can...

07:30 One of the things that's interesting in Python about Python's async as a weight and a way is you can change the implementation of the event loop, right?

07:41 And so one way you can do that is you can replace it with UV loop, which is based on libuv, which I always thought of as a C library, but it's actually fully written in Cython.

07:52 Oh, I didn't know that either.

07:53 -Really? -Yeah, that's pretty cool.

07:55 Then finally, HTTP tools, bindings to node.js HTTP parser is fully written in Cython, along with Sanic, UVCore, and a bunch of other stuff.

08:04 that use this library. So if your goal is to try to get a little bit lower level, maybe work with some C, make your code go faster, here's a really cool write-up of how to do all those things.

08:15 That's really cool. You know, I'm a big fan of language interoperability in general.

08:19 And you know, one thing I could tell you, it feels like every language knows how to work with C, at least.

08:25 Like C is like the common denominator that every language knows how to talk back to.

08:30 And I really wish there was a way for us to... I mean, I love C.

08:34 I used to love C.

08:36 But I really wish there was a way for us to have a very standard way to interact with each other a little bit better.

08:42 And I remember, I think the last time that I was on the podcast, I talked about Wazzy.

08:47 So I think it was WebAssembly.

08:49 Yes.

08:49 It's like a WebAssembly module that you could use.

08:51 And now all of your code compiles down to WebAssembly.

08:55 And it exposes different functions and whatnot.

08:57 But now that'll be like a portable runtime that essentially, I can import it in Python, and I can import it in Java, or JavaScript or .NET or whatever the case is, because they all have that, you know, that knowledge of how to work with that WebAssembly binary, right, that portable WebAssembly thing.

09:14 I should look that up. I don't know where they are with that.

09:17 But I know that was the thing that folks were getting really excited about.

09:20 Not just from the fact of, hey, I can run my Python code or my language Xcode in the browser, but also too, I can consume it from other languages.

09:29 - That would make it convenient. - Right. If somehow you can package your code in whatever language you use in a WebAssembly format, all of a sudden it becomes interoperable with other WebAssembly libraries.

09:40 Anything else that can be packaged into WebAssembly, basically, right?

09:43 Right. And what's good about that is it's relatively a standards compliant way, because WebAssembly is a standards-based thing. It's not owned by a company or one particular runtime.

09:53 All of us run and play in the web and use browsers and use browser technologies for one thing or the other.

09:58 It's definitely something that everyone can kind of get involved in one way.

10:01 Yeah, for sure. I'm super excited about that as well.

10:03 Alright, moving on to the next one, your topic here.

10:06 You've got something very small that's actually quite cool.

10:08 Yeah, and so just kind of talking about interoperability a little bit.

10:11 One of the things that we all use a lot as developers is Git.

10:15 I use Git a lot and particularly in the GitHub space.

10:19 Actually, I don't think I know a developer today that does not have a GitHub account.

10:22 But you know, maybe that happens.

10:23 Do you remember when people used to debate about what kind of version control they use?

10:27 like, Oh, we use Perforce. Oh, we use Subversion. Oh, we're still on CVS. We use a wind. See, I use tortoise CVS and I got a right click. That used to be a debate. That debate is gone.

10:38 It seems like I remember, man. I remember there was a debate back when Git first started to, there was a debate between Git and Mercurial because those were the two distributed version control systems. And people were like, Oh, I'm going to use this one. I'm going to use that one and trying to figure out like which one that was going to be the thing to use.

10:55 And I think it's pretty safe to say that in GitHub of one.

10:58 - Yeah. - Pretty much.

10:59 - Yeah.

11:00 - You know?

11:01 - GitHub too.

11:02 Yeah, GitHub's awesome.

11:03 - Yeah.

11:04 I think they're pretty much just one.

11:05 So now this article, why I think this is really interesting is, you know, as a developer, since we, again, Git is pretty much like the standard of source control for all of our projects, it's important for us to understand how Git works, or maybe you might be curious about what the underlying internals of that looks like.

11:20 And so what this is, is a really cool tutorial.

11:23 interactive tutorial that you could do kind of in your browser as well where you could play around and implement git yourself your own version of git in python and i'm like oh that sounds cool like i definitely want to do that because one i think hey you're just giving me another project that i could get better at writing python but two i can also really try and understand what git does i know a lot of folks you may know a few git commands but oh man like i really don't understand what rebase does. And I don't know, what's the difference between git branch and git switch?

11:53 And you know, like some of those types of things. And so I'm hoping like we could go through this tutorial. And by implementing it, we could really understand like how the underlying data structures are different and how that git database is different.

12:05 That's so interesting. And thinking about understanding git by re implementing it from scratch is pretty interesting. Yeah. And I think it's worth pointing out just the format of this article tutorial thing, it's quite unique.

12:20 So if you open it up on the left hand, the browser split 50/50, the left hand column is the description and message.

12:30 And on the right, it's basically got a diff of the code that you're writing at each step.

12:35 So it's, it doesn't look like you can execute or anything, but it's a little bit like replit or something like that.

12:40 Yeah.

12:41 I'm looking at this and I think I was trying to see if you could click on this thing, if it'll take you to the source.

12:46 But I think this is in addition to just being a really cool topic.

12:49 I think it's an Oh, it does.

12:51 As you click around it, it'll, it'll reload the side to like take you to different parts in the code.

12:55 Yeah.

12:55 Oh, does it?

12:56 Okay.

12:56 Yeah.

12:56 Yeah.

12:56 So there we go.

12:57 There you go.

12:57 There it does it.

12:58 But I think this is a really interesting experience just for learning in general.

13:01 Right.

13:01 It's very interactive.

13:03 So I could read the text, the verbiage on the left hand side, and I could look at the code and I can see how it changes on the right side.

13:09 So it's not just like, Hey, I'm just reading the super complex book about like like, I can actually see the code, which, you know, for folks that like to learn by seeing, it's a really good experience to be able to understand like what exactly is going on.

13:22 I might have to go through this. This looks really interesting. And I might learn a couple of things by doing it for sure.

13:28 Yeah, this definitely looks really cool. But now speaking in terms of about learning, like you had a topic you want to talk about, right? That has to do with learning, right?

13:36 I would say you and I, probably these days we qualify as senior software engineers.

13:41 I don't know.

13:42 I don't really want that title on me anymore because I don't know.

13:45 Sounds like I go to an office.

13:46 That's my title.

13:47 But nonetheless, we've been around software for a while, right?

13:51 And there's this cool article by Neil Hickar who wrote it.

13:57 It is an in-depth write-up of things I learned to become a senior software engineer.

14:02 I love these things where people sort of look back on what they've done in their career, how it's, you know, what went right, what went wrong, and just trying to help other people shorten that cycle to get farther, faster, quicker, that kind of thing.

14:16 So there's a lot of lessons he talked about.

14:18 And I just wanted to cover some of them.

14:19 I said it's really long, and so I'm not sure I want to go into all of it there.

14:23 But so the first one was he advanced his career by growing his skills and experience using different ladders of abstraction, is how I put it.

14:34 So the first year or so, he had basically learned the ins and outs of software.

14:39 like he knew his language, his compiler was working, and I believe he's doing Python, so maybe not his compiler, but all the stuff that he's doing to work there is pretty much figure out the software side of things.

14:50 But his software development lifecycle was like a small cog within a bigger cog and so on.

14:57 So he said, "Look, there's really the the lifecycle of the product.

15:01 There's the lifecycle of the infrastructure and the business, and like, working your way and sort of out of that loop to understand the bigger pieces, really helped him do more than just write code, but really be a bigger part of the team, I guess.

15:16 This is really interesting, and I'm kind of scrolling through the article again as you're talking about it.

15:20 And it brings up to mind a discussion that I get a lot from some of the students that we talk to, with the different engagements we do.

15:28 And that's always their question.

15:30 It's either, one, how do I become a good software developer, or how do I level up in my career? What are the that I need to do to step forward in my career.

15:39 And sometimes I feel there's certain things that for sure you could definitely do, but then I think that's only like 50% of it.

15:47 And I think there's another 50% that could be a little subjective based on the area that you're in and the region that you're in.

15:54 And what are the things that you really want to do?

15:57 What are the things that you really care about?

15:59 Like I could tell you when I first started, honestly, I was just trying to get a job to be honest with you.

16:05 I wanted to leave school, I wanted to work because being an international student, then I had to worry about like visas and green cards and all that type of stuff. And I'm like, the first person that hires me, I'm gone. That's it. But obviously that's not something you base longevity on when you think about a career. You want to think about, okay, now I'm working.

16:25 And maybe I've spent a year or a year or two inside of the environment. You know how to code, you know how your deployments go, you understand your ticketing system and all the scrum meetings that you have to sit in on.

16:38 But now you got to understand, well, how do I get better?

16:41 And one of the things I always tell folks is, well, it's kind of like being a specialist, right?

16:47 Like, what are the things that you want to get better at?

16:49 Because there's so many different things to look at.

16:51 What are some of the key things that really interest you?

16:55 So do I want to be really good at doing security?

16:57 Do I want to be really good at doing performance?

17:00 Do I really want to get good at maybe microservices and distributed systems is interesting, right?

17:05 Right, you get pigeonholed and people will just leave you there if you don't try to stretch.

17:10 Right, but you have to want that though, right? Like you have to want to get better and then after you have that want, you got to figure out where exactly do you want to go.

17:17 And for me, then the next step after that is really trying to understand, well, what are the fundamental pieces that I need to understand?

17:25 Like, what are the fundamental algorithms and data structures I need to understand?

17:29 And I'm not talking about like link lists and bubble sorts, that type of stuff.

17:32 But like within that space, right?

17:34 Like, so in when we talk about security, like what are the things I need to know about cryptography?

17:38 Right? When I talk about like distributed systems, what do I need to know about networking and how networks work?

17:44 You know what I mean? Like, how do I get the knowledge to understand that so now I could continue to move forward and really become like very knowledgeable and successful in that particular aspect of software development?

17:55 Yeah, I think that's a really big part of it.

17:57 And that's definitely some of the stuff they were touching on.

18:01 I think one of the big levers you have to pull is to try to get yourself to do stuff that is uncomfortable.

18:08 Like, I don't really know about continuous deployment, but we don't have a great answer for that.

18:12 So I'm going to force myself to learn that and I'll volunteer to be the person to make that happen, even though I'm not totally there yet, right?

18:18 But that'll get you there if you take those steps.

18:21 Another one that Neil pointed out was learning what people around him are doing.

18:26 So not just what are the developers doing, but what are the project managers, the salespeople, the analysts, and so on.

18:32 You know, for you working in a company like Microsoft, there's layers and layers and layers of that, right?

18:36 Yeah, for sure.

18:37 And you know, there's good and bad things to that.

18:40 One of the good things I like about that is that there's always people that you could ask a question to.

18:46 And sometimes you might be in an environment that that's not really the case.

18:49 Sometimes you're in a company and everyone is at the same level that you are.

18:53 Yeah.

18:53 you can't talk to other people, at least, or maybe you don't have access to talk to other people.

18:57 But when you're in an environment that there's a lot of folks that you can ask questions to, one of the best things that you could do for your career is make sure that you spend time with them, however that works, to really understand like, how does their world kind of move, particularly with something that you're interested in. So again, like, hey, I don't know a lot about security, and I have a security feature I need to work on. And it's something I really want to dive deep into, let me go and find the PM that does security for this thing. Or let me go and find the person that works on the Docker integration for this thing or the Python integration for this thing and really have like a deep conversation with them. And the good part about it, if that person is willing to sit down with you and give you the time, that is like gold. Like it is the best thing ever to be able to have someone pass on knowledge to you. I say that too, because we move forward and as we become quote unquote senior engineers, we have to remember too that the folks that are coming that are not so senior, they're going to want to do the same thing for us, right?

19:52 And we have to have that mindset of being willing to share that information, not try and keep it inside and take it with us when we leave the company kind of thing.

20:00 Yeah. Well, the other flip side is, you know, the best way to learn something is to teach it often.

20:04 And that's your opportunity. Yeah.

20:06 Yeah, exactly. Exactly.

20:07 Another interesting lesson was strategies for making the day-to-day more efficient, which I suppose the larger the company, the more important this can be.

20:16 And Neil said some good habits.

20:18 Notice was never leave a meeting without making a decision or having a next action to do.

20:23 Meetings drive me crazy and it's so often you just have a meeting because we always meet on Mondays for half an hour even if we don't have to.

20:29 No, please, not again.

20:31 Decide who's getting a thing done because if it's not on somebody's responsibility to get it done, it's probably not getting done.

20:39 And then document design decisions made during a project.

20:42 So that was pretty cool. Those are good ones.

20:45 Yeah, acquiring new tools for thought and mental models.

20:48 So one example in the software space would be like, I was recently struggling with the domain of a lot of complex business logic, and all these weird edge cases and stuff.

21:00 And so I found the Domain Driven Design book by Eric Evans and that whole space and said, oh, there's a whole world over here around like modeling objects and relationships that I didn't understand, or didn't know about and dive into that stuff.

21:15 Pretty interesting.

21:16 Here's my favorite.

21:17 And this has driven me crazy about so many different things.

21:21 Protect your Slack.

21:22 So not your Slack, the not competitor, not your channel, but Slack as in, I've got a little bit of time in my day-to-day schedule that is not clearly assigned, right?

21:35 Not like I'm spending the next two hours on this feature because I agreed to it in the sprint and then I'm done and I have to move on to the very next thing, right? If it's so tightly packed that you don't have the the freedom to explore, like, that didn't quite work the way I expected it. Let me dig into that.

21:54 Or what if we re implemented it this way? Let me just try that idea. Or I don't know anything about domain driven design. So let me learn, let me spend the next hour reading about that, and see if that'll help us later. Right? If all those opportunities are taken away and your time's like squished just back to back to back, you end up just doing 20 years of the same thing instead of actually growing and learning.

22:16 I'm a big fan of companies that allow their employees, developers or not, time to sit down and experiment and try out new things and, and whatever, you know, prior to coming to Microsoft, I was in a company that they used to have, like, every time there's a release, right after the release, they had this thing. Oh, man, what was it called? I don't remember what it was called. But It was like an event. It was like a whole company event where for like the next week all they could do was just work on side projects.

22:45 Like it was a whole week.

22:46 - Oh wow. - Obviously the service folks had to be on service.

22:50 But for everyone else, like the PMs, the engineers, the developers, they spent this entire week just hacking on stuff.

22:56 And then the next week, there was a day where they gave presentations and everyone talked about what they did.

23:01 And it was a whole showcase, but it was only internal.

23:03 It was just for the folks inside of the company.

23:05 And not only did you have time to do that and learn and try new things out but you know there were prizes right? Like there were whether it was stock or money or like just you know there was stuff that you got which was really cool.

23:17 So now it shows like how much the company like really believed in you.

23:20 And so again I bring it up because I think it's an important one not only do you give your employees, if anyone is an employer that's listening, that you give your employers, employees the ability and space to learn but also incentivize them to do it, right? Like really encourage them and let them know that you're supporting it.

23:35 Regardless of whether that means it's something that's going to go in your product or not, like the fact is like they're becoming better, right? And so if you, all of your employees are becoming better, like that means that they're going to be able to do better work for you at the end of the day.

23:49 And so I think that's always a really cool thing to do.

23:51 Yeah, absolutely.

23:52 I think this concept of Slack is super important and so much people looking from the outside, they try to squash it and compress people's time down and I think it's in the end, it's not so good.

24:02 So there's a ton of other stuff.

24:04 I want to just throw out really quick one last thing, take a time.

24:07 So we don't spend too much time on this.

24:08 The last was this idea of building up superpowers.

24:12 So are you getting into the source code of some project, some package and the documentation isn't enough.

24:18 All right.

24:19 You have a quest that's to go read the open source code and spend some time like learning from the code, not from the docs, you want to quickly build a mental model for code you're looking at.

24:28 reading open source code again, embracing fear, you're something you're not confident or feeling comfortable with, do it as a side project. That's your quest.

24:37 Confidence to express ignorance or not knowing, like, fine, right? We all got to ask questions we don't know and get over it and practice with that.

24:47 There's just a lot of good takeaways from here.

24:49 Like we really just scratched the surface on this.

24:51 Sure. And kind of going back to that point about reading the open source code, I think that's one of the reasons too why GitHub is so popular.

24:58 because it made searching source codes so easy.

25:01 And I'm not gonna lie, there's a lot of times throughout my day that I'm thinking, hey, the documentation is not the best for whatever it is that I'm looking at.

25:10 I'm going to search for like the class or the method.

25:13 I literally just look for the class name and the method and filter by language.

25:16 Just to see, well, I just want to see how are other people using it.

25:19 And, you know, what are the types of issues that are created around it?

25:22 And hey, should I wrap this in a try catch or you know, like, what kind of parameters do you pass in? Like, how do you deal with these things?

25:29 And being able to kind of look and see how other folks do it gives you some ideas now, okay, well, this is how this thing was meant to use.

25:35 Now, the flip side of that is that, you know, you might run into a bad example.

25:39 But again, that's a learning experience, right? Like, you run it and you'll figure out, and then, you know, we're working out. - Yeah, exactly. That's great.

25:46 So, speaking of learning, you guys have a pretty cool project, a data source, I guess, to learn from, or context to learn from.

25:53 Yeah, yeah. So what's happened over the past, oh man, over months and months and months.

25:59 So Microsoft actually started a project where they're working along with NASA to create these learning modules on Microsoft Learn where you can kind of go in and learn how to do like machine learning with Python, but you could do it within the context of exploring space, which is really cool. So you could do different things like classifying space rocks and predicting weather, it'll help you like predict, you know, rocket launch delays based on the weather, like tons of great stuff.

26:27 And so there's a, I'm trying to think about how many of them there's a 123.

26:31 Yeah, there's at least six learning modules, again, all based on Python and AI, with machine learning and NASA, where you can kind of kind of go in and learn all these different things.

26:40 So definitely recommend that folks go in and check it out.

26:42 It's 100% free.

26:44 So you could, you know, just click on the link and kind of go through them yourselves, you don't have to sign up for anything, which I love.

26:49 Like I love free learning resources, right?

26:52 But yeah, I thought this was really cool.

26:53 And so if anyone's a space buff at all, like this is be a great adventure for you to kind of go through and kind of play around with some data from NASA.

27:01 - Yeah, I think that's super cool.

27:02 - To learn how to unset space.

27:03 - Yeah, and so much of being successful, especially when you're learning early, is about having context that inspires you.

27:11 - Yeah.

27:11 - 'Cause we all get stuck or things get frustrating.

27:13 You're like, why won't it, why is this function not doing what it's supposed to do?

27:16 But if you're like, and if I get it to work, I get to understand a space rock or something like that.

27:21 All of a sudden it's like, yeah, that's not so frustrating.

27:23 - Right, right.

27:24 And I think NASA also has, for folks that want to get a little bit more into NASA, NASA also has like, I think it's called NASA Socials, which is like a national social club where you could, again, work along with other folks in a community.

27:36 Back before social distancing, you could actually go to a social events and you just learn different things about the work that they're doing with NASA.

27:43 So that's also really cool too.

27:44 I think you could do those social events online.

27:47 So if you're interested, probably like, hey, maybe you learned a little bit in doing some of these courses and you wanna engage with other like-minded folks that are into space exploration, but they don't necessarily work at NASA, you know, that might be another thing to definitely check out.

27:59 - Yeah, absolutely.

28:00 The next one I wanna cover has to do with performance.

28:03 And I don't know if you've ever had this experience, but I have definitely had it where the code is going slower than I expect.

28:10 And I'm like, oh, it's got to be this part of the code that is so slow.

28:13 And then I profile it or I try to fix it.

28:16 It's absolutely not, it has nothing to do with that.

28:18 That's like 5%. It's somewhere else.

28:19 It's completely somewhere else.

28:22 Have you had this experience?

28:23 Yeah, and you know what happens?

28:24 It's always the database.

28:26 [laughter]

28:27 -Have you heard that? -Why are there no indexes?

28:29 People always say that.

28:30 There's like, "Oh, you have a performance problem?" It's always the database.

28:34 I know it's not always the database, but I totally get what you're saying.

28:37 There's definitely cases where...

28:39 Because who else would know your code better than you would?

28:42 And so if you're thinking, "Oh, man, if this is where the problem is, - This has to be where it's going. - Yeah.

28:46 Well, we just have such a...

28:47 Often we have a bad conceptualization of what is fast and what is slow.

28:52 - Yeah. - Because, I don't know, computers are so fast.

28:54 - It's a little bit crazy. - Yeah.

28:55 But profiling is usually the first step to figure out what that answer is.

29:00 And then if you're going to change it, you can see, did it get better?

29:02 Did it get worse? Where did it get better and worse? And so on.

29:04 - Right. - So if you're doing Django, I want to talk about profiling requests there because there's a couple of interesting options.

29:10 So there's one library that will let you plug in basically cProfile, which is the low-level cProfiler from Python that ships with it right into your request.

29:24 So if you plug it in, you can just do a, what is it?

29:27 Question or an ampersand prof, a question mark prof at the end of URL, and you'll get a profiling output.

29:33 - Oh, nice. - It's pretty cool.

29:34 So there's a thing called django-cProfile-middleware.

29:37 You just install that.

29:38 And then if it's installed, you can put basically prof in the query string and you'll get that, which is great, and that gives you an output there.

29:45 Or you can put ampersand download and it'll download, you can visualize it graphically with snake viz.

29:53 So that's even better. So you get like this download profiling report for your request.

29:58 All that's pretty good.

29:59 But what would be a lot better is to be able to like visually dive into the request and do things like understand, talked about the database being slow, like understanding the time you're spending in the database versus other places.

30:11 So there's this thing called Django Silk.

30:14 And you also add that as a middleware.

30:16 But then it keeps track of all the requests on your site.

30:19 Hopefully you're doing this in development, not production.

30:21 It keeps track of all of those.

30:24 And then it'll show you, like, this is the general time for this URL.

30:27 Click on it, it'll show you things like, these are the database queries that were run, actually, as SQL, even though using RM, It'll show you the SQL that was run, and it'll show you how much time the database is spending, how much time other parts of your code are spending.

30:40 So, yeah, a really cool way to explore the performance from all the tiers of your - Django app. - Yeah, that's really cool.

30:49 And I always find database interceptors to be a really powerful concept, like across all languages, right?

30:55 Like being able to see, like, let's say I'm using SQLAlchemy, or I'm using Mongo engine or something like that.

31:01 I'd really want to know what's the query that I'm sending over, like, what is it that I'm asking for?

31:07 And versus what I wrote in code, right?

31:09 And what am I actually getting back?

31:11 And being able to see, oh, man, like, this thing is doing like, four or five joins, like, I don't need to do that.

31:16 Like, I could try and optimize this query a little bit more to one, make it more performant from a database perspective, and two, obviously, passing on that performance over to my application.

31:26 Or I thought I did one database request.

31:28 Why are there 51?

31:30 Right, right, exactly.

31:32 that lazy property that we forgot about every single time through that loop, we're hitting the database again.

31:37 Such a bad idea.

31:38 Maybe we should do a sub join, like eager load sort of thing.

31:42 Yeah, for sure.

31:43 Yeah, so I think this is really nice.

31:45 Yeah, it is really nice.

31:46 And this is one of the scenarios too where I think it's interesting to get like a log collector.

31:51 So, you know, there's tons of services that do this, but like something like Datadog or Rollbar or something like that, that not only capture your logs, but allow you to kind of go in and search through them, search and filter your logs. So you could say, okay, well, hey, I'm having performance problems, but it only happens at two o'clock in the afternoon, for some reason, or, you know, my memory always spikes within these hours of the day, and being able to have a history of logs and be able to query them, and then get that rich information. And I don't know whether you plot it or turn into a report or something like that.

32:23 Being able to do that, I think is hugely important.

32:25 Yeah, absolutely. I totally agree. And being able to visualize it, right? So often having the picture is what you need.

32:31 Sometimes though, it's good to just get a text.

32:34 Yeah, that's true. Sometimes it's good to just get a message or get like an alert.

32:39 Hey, Michael, the website is going down, you need to get here ASAP.

32:42 Exactly.

32:43 And so that kind of leads into the next one. That was a good segue. I like that.

32:46 Thank you.

32:46 So that can lead into the next topic I did want to talk about.

32:50 At the time that we're recording this, this week is actually Microsoft Ignite, which is, you know, one of the conferences that we do and It's all online, we announce products and tons of stuff like that.

33:00 Well, last time that we got to actually see each other in person and hang out a little bit was that Microsoft Ignite in Florida.

33:07 And I was blown away at how ginormous that conference is.

33:10 Like you were on Talk Python then.

33:11 And Python. That's 30,000 people.

33:14 So I guess it's 30,000 video streams now.

33:18 Yeah, I remember that last time we were all together, it was in Orlando.

33:23 And Dan Bader was there, remember?

33:25 - Yeah, that's right. - Yeah, episodes with Dan.

33:27 I don't even know what the numbers are this year.

33:30 That would be interesting to find out. I should look that up.

33:32 - Yeah, for sure. - See like, what those numbers kind of seem like.

33:34 So it's been a year and you guys have a new announcement?

33:37 Yes, it's been a year and so there's a new service.

33:39 And so this is Azure Communication Services, which is cool.

33:43 One, because you know, we have a new service, but two, like, I had no idea this was coming out.

33:46 I kind of found out like everyone else did on social media.

33:49 So this is Azure Communication Services.

33:52 And essentially, it's a way for you to add, well, communication services to your application.

33:57 It does a lot of cool things like it does voice, it does video, it does text messaging, and you can obviously receive and send phone calls and things like that, which is really cool.

34:06 But what's even better about this, more specifically for the folks of this show, is that there's a Python SDK on day one.

34:14 I guess we could call it day zero since it's not officially out yet.

34:16 But on today, in preview, while the service is in preview, there's a Python SDK, there's a JavaScript SDK, and there's also a .NET SDK.

34:26 And I looked at the code for the Python SDK, and there, I almost want to say like there's nothing there.

34:31 I mean, obviously, there's something there, right?

34:33 But most of the code is like, create a directory, create a virtual environment, install pip, SMS client that send message.

34:40 I'm like, wait, that's it?

34:42 Yeah, it's like literally the meaningful bit is like five lines of code.

34:46 Right, exactly. And so it looks like a really easy thing to kind of get started with. And so I haven't played with it yet.

34:51 Again, I just literally discovered this yesterday, kind of like everyone else did.

34:55 So I'm really interested in plugging it into some of my demos and seeing what the experience is really like.

35:00 Yeah, this looks really cool.

35:01 If you want to add text integration, like text alerts to your app, it seems like a really good choice.

35:06 Right, and so then we could plug in.

35:08 Imagine if we could plug that into the Django profiler.

35:11 And then as things are happening, it sends us text message alerts.

35:14 And so now we can know, "Uh-oh, there's a problem with the website now." That's right. It's running slow.

35:19 - The memory is too high. - Exactly.

35:20 - Go check it out. - Exactly.

35:21 Yeah, this is awesome.

35:23 Nice one. People can check that out.

35:25 I guess that's it for our items for the week.

35:27 Quick shout out. I just say I was on, you mentioned Dan Bader, I was on RealPythons podcast with Christopher, the host there, last week.

35:35 So I'll be sure to link to that. It was a really fun look back on the trends and stuff I've seen over the last five years of podcasting in the Python space. So if you want to hear about that chat, follow up on that show.

35:47 And then, how about you? Anything else you want to share?

35:49 One of the things that our team does is they work with students from different universities across the world.

35:55 And they join a program that we have that's called the Microsoft Student Ambassador Program.

36:00 Now if you're familiar with like Microsoft MVPs or Docker Captains or what is it called GDEs, Google Developer Experts, it's something very similar to that, but it's just this time we're talking about students, right?

36:13 And so, you know, these are students that are going out and they're creating content and doing sessions and workshops and writing blog posts and different types of things within their school and within their communities.

36:23 And so we have them join this program where we're able to support them a little more and give them Azure credits and Visual Studio things and LinkedIn learning things and some private events.

36:34 Yeah, they have little private events and mentorship sessions and things like that.

36:38 But it's a really great way for us to kind of engage with the students, right?

36:41 And really just understand like, what are they doing? What do they care about?

36:44 What are some of the ways that we can affect their learning?

36:46 And so what I wanted to do was share the link to the Microsoft Student Ambassador website.

36:51 And you can actually kind of go here and you can search by region, by language, by area of focus, and you can kind of see what the students are doing, which I think is really great.

37:00 And if anyone that's listening is a student as well, a university student that's interested in joining the program, you know, you go ahead and hit that big blue Apply Now button.

37:09 And you know, go ahead and submit an application for the next for the next cohort.

37:12 I think that'd be a great opportunity for students. So people should check that out.

37:16 All right, let's finish this off with a joke.

37:18 And we're better to go than XKCD.

37:20 I love XKCD.

37:21 I know, they got some good stuff.

37:23 So this one is entitled Dependency.

37:26 And I think it just captures the whole bring together different languages and technologies and especially a bunch of open source libraries.

37:35 Do you want to describe this picture for the listeners?

37:38 You can always grab graphical stuff for our jokes.

37:41 Sure. So this reminds me of a game of Jenga.

37:44 That's what I'm thinking.

37:45 Like a super Jenga.

37:47 This is like recursive Jenga.

37:49 Yes.

37:50 That's what this looks like.

37:52 And so there's some big blocks and there's some little blocks.

37:54 And it looks like as the closer you get to the top, they start to become a lot more specific with like the type of blocks that are up there.

38:01 And then at the bottom, it's much bigger at the bottom.

38:04 And you know, that looks like that could be trouble.

38:07 Oh, and on the right side, there's one little part that's holding everything up.

38:11 And it's really, really small and fragile.

38:14 At the top it says all modern digital infrastructure.

38:19 And at the bottom, it says the little tiny stick is a project some random person in Nebraska has been thanklessly maintaining since 2003.

38:28 That's amazing.

38:29 I love this.

38:30 And the hover over message is someday image magic will finally break for good and we'll all have a long period of scrambling as we try to reassemble civilization from the rubble.

38:43 To me, I think of this as OpenSSL.

38:45 That little stick, that's OpenSSL down there.

38:50 There was like one person who maintained it and everything seemed to be built upon it.

38:55 And when there was that bug that needed to fix, it was a huge problem.

38:59 I find anytime I try to home...

39:02 So I use a Mac and every time I homebrew install OpenSSL and it updates the version that's on my system, like everything just stops working.

39:11 I can't install any pip packages.

39:13 You're like, "No, don't do it." Nothing works. I'm like, "What happened to my virtual environment?

39:17 What's going on here? Why can't I install anything?" Then I'm like, "Oh, open SSL." Yeah, exactly. That's the stick that can bring it all down.

39:25 Well, thank you for being on the show, Cecil.

39:27 It was great to catch up with you and you had a lot of good stuff to share.

39:30 Hey, thank you for having me on.

39:31 And Brian, I know you're not here, but I hope I did you a service.

39:35 And I hope you come back soon.

39:37 And also, too, I didn't get to say this in the beginning.

39:40 Congratulations on 200 episodes.

39:42 Well, over 200 at this point.

39:44 Remember, I listened to episode 199.

39:47 I'm like, wow, has it been that many episodes already?

39:50 Like, it's crazy.

39:51 Yeah, that's coming up on like four years almost.

39:53 It's insane.

39:54 Wow.

39:54 That's awesome.

39:55 Well, congratulations to you both, man.

39:56 That's a great achievement for podcasting.

39:58 Yeah, thanks a bunch.

39:59 And take care.

40:00 Take care.

40:01 Thank you for listening to Python Bytes.

40:03 Follow the show on Twitter via @pythonbytes.

40:05 That's Python Bytes as in B-Y-T-E-S.

40:08 and get the full show notes at pythonbytes.fm.

40:11 If you have a news item you want featured, just visit pythonbytes.fm and send it our way.

40:16 We're always on the lookout for sharing something cool.

40:18 On behalf of myself and Brian Okken, this is Michael Kennedy.

40:22 Thank you for listening and sharing this podcast with your friends and colleagues.