Transcript #214: Python year in review (2020 edition)

00:00 Hey everyone, 2020 has been quite the year, hasn't it?

00:03 Definitely one of those for the history books.

00:05 So over on Talk Python to me, I did a year in review show, and it actually had me and Brian on there, along with a bunch of other really great guests who have been on this show and on Talk Python before.

00:18 So I thought what we could do for our year in review here is to bring you that same show, to bring you that same conversation.

00:24 In fact, it was more in the style of Python Bytes than it was Talk Python, to be honest.

00:28 So I think it belongs here.

00:30 So without further ado, let's join that conversation where we look back on 2020 with Cecil Phillip, Ines Montani, Jay Miller, Paul Everett, Reuben Lerner, Matt Harrison, Brian Okken, and me.

00:42 Enjoy the show and thank you so much for listening to Python Bytes this year.

00:46 We'll see you in 2021.

00:48 So let's go ahead and just kick off this episode.

00:51 This one is a little bit different than the standard show.

00:53 Everyone here has been a guest and a great guest, and some of you even repeat guests.

00:58 Some of you are repeat guests and scheduled for more episodes, even hint and Ines, but let's go ahead and kick it off.

01:04 So this is going to be a year in review.

01:05 There's a bunch of us here, seven or eight of us.

01:08 And I'll just do a quick introduction.

01:10 Let me just welcome you individually on.

01:12 You could say, you know, a couple of sentences about who you are and whatnot.

01:15 So people know, and then we're just going to get into what you thought was the most interesting, impactful thing of the year.

01:21 And I have some software to help me do this.

01:23 So let me run it here.

01:24 Darn it.

01:24 It's me.

01:25 All right.

01:25 We'll go back to that.

01:26 Well, let me welcome you all.

01:28 Then we'll go through this.

01:29 So Jay, Jay Miller, welcome to talk by the name again.

01:32 Happy to be here.

01:33 I haven't been a guest in like three years.

01:35 So I know, but your episode was so popular, like over a hundred thousand people listened to your episode.

01:40 Can you believe that?

01:40 That's so insane.

01:42 I'm shutting down my podcast.

01:43 I don't get those kinds of lessons.

01:44 Tell people about yourself.

01:47 Sure.

01:48 Jay Miller.

01:49 I'm a podcaster developer now developer advocate for elastic, you you know, the people that do search.

01:55 And we are database shots again, Michael.

01:59 But yeah, I'm just happy to be here.

02:02 Yeah, that's it.

02:03 - Awesome, welcome.

02:04 Reuven Lerner, welcome back.

02:06 Good to see you, man.

02:07 - Hey there, nice to be here.

02:08 - How early is it?

02:09 - It's not that early, it's only 5.30.

02:11 - Oh, okay, well thank you so much for waking up early.

02:14 Yeah, we have people from basically every corner of the world, so yeah.

02:18 Tell people about yourself, again, if they haven't heard the episodes.

02:20 - Sure, so I'm a Python trainer.

02:23 So most days I'm in different companies doing on-site training.

02:27 Nowadays a little less on-site for some strange reason.

02:29 And so I normally do corporate training, everything from intro to Python for non-programmers all the way up to advanced Python workshops.

02:36 And I also sell a whole bunch of courses online to individuals.

02:39 Oh, and I have my book, my book, Python Workout.

02:42 Yay, Python Workout.

02:42 - Yeah, and we're streaming live with video, so you can hold it up, yeah, that's awesome.

02:46 - Oh, I can hold it up.

02:48 - Python Workout, awesome.

02:49 - And this is a picture of me on the cover, of course.

02:51 - Of course, yeah.

02:51 - That was when you were working out, preparing for the Olympics?

02:54 - Precisely, precisely.

02:54 - Yeah, exactly.

02:55 So we'll be sure to link to your book and some of your courses as well.

02:58 Ines, welcome from the other side of the world.

03:02 - Yeah, I'm currently in Australia, escaped the EU lockdowns and now I'm here.

03:07 Yeah, happy to be back.

03:08 I think I've been on a few of your podcasts, but it's always my favorite podcasts.

03:11 - Oh, thank you.

03:12 Yeah, it's awesome to have you here.

03:14 And I have definitely dreamed about, well, my daughter goes to school online.

03:19 I always work online.

03:20 Maybe we should just go somewhere by a beach, away from people.

03:23 How could we do that where it's warm, right?

03:25 And it seems like you pulled it off.

03:26 Beautiful.

03:27 All right, Paul Everett, welcome.

03:28 - I just saw Anthony Shaw talking, so if you want to talk about converting your life to work at the beach, just subscribe to his damn Twitter account.

03:37 - Welcome, Anthony Shaw.

03:38 - Anthony, I'm raging against you.

03:39 - Oh my gosh, also from Australia.

03:41 Hey, Anthony.

03:42 And Andrew from PyCharm is here, a bunch of folks.

03:45 Mark, awesome.

03:46 Paul, introduce yourself.

03:47 - Sure.

03:48 I'm an advocate at JetBrains for Python stuff and web stuff.

03:51 Most known for giving the Python 1994 talk.

03:55 My purpose on this panel is to increase the median age so that we all qualify for the discount at the buffet.

04:01 Hey man, I'm right there with you.

04:03 Right there with you, awesome.

04:04 Nice to have you, Paul.

04:05 Brian, we've met before, right?

04:07 Yeah.

04:08 Brian Okken, he and I co-host Python Bytes Together.

04:11 And you've been on the show quite a few times as well.

04:13 Last time, I think we covered 30 pytest plugins.

04:15 That was quite the whirlwind tour.

04:17 Yeah, so I highly recommend this book to anybody that wants to learn pytest.

04:20 Awesome. Matt Harrison, welcome back.

04:22 Thanks. Good to be here. I guess I'm similar to Reuben. I do corporate training and consulting in Python and data science. And I guess since we're flaunting books, I'll flaunt my latest book.

04:33 Wait, wait, the tiny Python 3 notebook.

04:35 Tiny Python notebook. It's very small. Anyway.

04:38 It is tiny. Well, what I was going to say, Matt, is like, you're going to need to be like, basically an octopus to hold up all your books. Like multiple books on each tentacle.

04:46 I've got a few here. So yeah, it's good to be here. Thanks for having me on.

04:49 Yeah, you were just recently on about learning how to learn.

04:52 Yeah.

04:52 Does it count if I have everyone else's book?

04:55 Yes, it does. But it will also be hard to hold them all up, especially if you got all of them.

04:58 Cecil, hello, man. Hello. Good to see you, friend.

05:02 Hey, thanks for having me on. I don't have any books to share.

05:05 But because I hate writing, so I don't write anything. I do do some video.

05:08 So if you're interested, I do do a weekly Python stream on Wednesdays.

05:13 So if you folks want to check it out, it's on Twitch on the Microsoft developer account.

05:17 And we do very beginner friendly conversations about Python and stuff like that.

05:22 And Michael, you were just on the other day too, right?

05:24 - Yeah, it was so fun to be on there with you.

05:25 We got to play around with FastAPI and man, that's a cool framework.

05:28 - So it's Coding From The Beach.

05:29 What's the name of the show?

05:31 - That's a good name, actually.

05:32 Maybe I should coin that one.

05:33 No, so we do PyWeekly.

05:36 So it's me and my co-worker, Brian Clark.

05:39 And we kind of approach Python from the perspective of beginners, but also from two people that don't necessarily know Python very well.

05:47 So, you know, Brian is a JavaScript guy, I'm a .NET guy, but we're both, I like to call it Py-curious, you know what I mean?

05:53 Like we're Python curious.

05:54 And, you know, we definitely want to show people like the different perspectives of learning like new languages.

06:00 - Yeah, it's super interesting.

06:01 And it is really low-key.

06:02 It's like you said, it's this Twitch sort of exploration style of code, not polished presentation style.

06:09 So you just guys sit down, take feedback from the audiences, and go with it.

06:13 - Yeah, I love this show, 'cause I'm never prepared.

06:15 Like, I kinda just show up and talk about stuff, and people love it, and it's great.

06:21 I don't have to do notes or anything like that.

06:23 We just kinda be like, "Hey, what do we wanna learn about today?" Right, well, let's go ahead and do it.

06:26 - Yeah, fantastic.

06:28 All right, well, given how many of us there are, let's go ahead and just jump in, and we're gonna do our year in review.

06:33 And so I've written a small piece of Python software here in the REPL.

06:37 It's very advanced.

06:37 It's at least four lines of code, I gotta tell you.

06:39 And what it's gonna do is it's gonna pick one of you, And then we're going to have you talk about what you felt was the most significant or impactful thing that you saw happening in the Python space.

06:49 So round one, Cecil, you're on screen and it pulled you up.

06:53 Go for it.

06:54 Wow, I got lucky.

06:54 Okay.

06:55 So the most exciting thing that, that I think happened in the world of Python on one that I have a little bit of a close relationship to is the fact that Guido van Rossum joined Microsoft just a couple of weeks ago.

07:08 So I didn't mention this before, but I do work at Microsoft as a developer advocate. And I think for me, it's been a really interesting to kind of see how the company has, you know, been embracing open source and working with a lot of the creators and now seeing a lot of the leaders of the industry kind of joined the company.

07:25 You know what I mean? It kind of, you know, it's a kind of, it's like a different world.

07:29 I mean, I don't know how else to explain it or, or talk about it. It's been really great to have him here. I had the opportunity actually to interview him. And we did like an internal interview just talking about like, well, what made you decide to come on a retirement?

07:42 That's awesome. Was this on channel nine? Is it public or is it?

07:46 No, it was internal. It was an internal interview. I don't know what his, his, I know I'm sure he will do some other ones, but it was an internal one.

07:53 Did you make him whiteboard?

07:54 No, we didn't do him any whiteboarding. We did ask him what some of his favorite shows were. So that was, that was pretty interesting. We asked him like the Star Trek versus Star Wars conversation too.

08:03 I couldn't, I couldn't handle that pressure.

08:05 It's hard, man. It's a tough crowd.

08:08 I think it's definitely amazing to be able to see folks from the industry embracing these different companies and us really supporting open source in a very authentic way.

08:19 Yeah, it's such a big change.

08:20 Reuven, you were just talking before we started streaming, I think, about 10 years ago, how this was really different.

08:26 Oh, yeah. I mean, even more. But yeah, even 10 years ago, the notion that Microsoft would be so involved in the open source space, and specifically the Python space was basically unthinkable. I mean, I think it was Steve Ballmer who said that, you know, Microsoft, not Microsoft, like the Linux and open source were a cancer on the industry.

08:43 And they were definitely seen, you know, Microsoft was seen as an enemy of the open source world.

08:49 And it's been truly amazing to see them just embrace it. I mean, you see the booth at Python, you see the VS Code investment.

08:55 It's very, very impressive. And it demonstrates, and he says, we'll say this earlier, like, how much leadership at the top really can change the direction and the attitude of a company, and that filters through to even the lower level people and the decisions made there.

09:07 Yeah, that's a lot of hard work, right? Because if you think about it, we've had different leadership for a very long time. And so for some of us, some of this change that people are still recognizing, I mean, we've been doing it for a very long time.

09:19 You know what I mean? Like we didn't just in 2020 decide, oh, hey, let's adopt and support open source.

09:24 It's been one of those things that have been building very much over time.

09:29 But the fact of the matter is, you still have those old stigmas that we still have to shake, and we still have a lot of folks that we have to help convince that we're not the Death Star, and we won't destroy the world kind of thing.

09:40 But I think it's a good responsibility to take on, because there's also a lot to fuck up.

09:47 And if you commit to doing something, like GitHub acquisition, and say, "Okay, we're doing that and we're running it." You know, we're good sign forward because every misstep can have like, you know, a really big impact.

09:59 - Yeah.

10:00 - You know, if Microsoft starts doing bad things again, people are not gonna be, you know, they forgiving at all.

10:03 So.

10:04 - I think that was the most sensitive, one of them all is the GitHub acquisition because what they do to Azure is kind of their story.

10:10 What they do to Windows is their story, but what they do to GitHub, that's everyone's story.

10:14 And all of a sudden, that was a big deal.

10:16 - Yeah.

10:17 - And on top of that, you had the ability to give GitHub a little bit more power in the space.

10:22 Like, I mean, we've all been kind of watching what's happening with like YouTube DL and stuff like that.

10:27 And just seeing the support, it's not just GitHub anymore now.

10:31 It's like, OK, what does one of the the wealthiest companies in the world, you know, say about this particular position that YouTubers have a problem with, that people in tech have a problem with, like a lot of abusive behavior?

10:45 And on top of that, you're providing support for things like NPM.

10:48 It's one of those questions.

10:50 We all want to contribute to open source, but none of us have the financial ability to contribute like a company like Microsoft can.

10:58 So I'm not just saying we're going to be a part of it, but saying, no, we're really going to be a part of it is like a huge deal.

11:03 Yeah.

11:04 The million dollar defense fund that they just announced is a big deal.

11:07 Yeah, for sure.

11:08 And when you think about GitHub, right?

11:10 Like you think about like developer culture, that's what GitHub kind of represents to me.

11:15 is the new, you know, the future of developer culture.

11:18 Like, how do we collaborate? How do we work with each other?

11:21 How do we share not just source code, but like, experiences and knowledge, right, through, like, something that we can all connect together with?

11:27 That's code, right?

11:29 And then kind of circling it back around to Guido, being able to, again, see these leaders of these really large open source projects put their faith in the company, and even considering what the past of the company was to what it is today, I think hopefully like the rest of the open source world would start to realize that we are serious about being good open source citizens and supporting the culture in the community.

11:51 I mean, me personally, I'm just really excited to see what he's going to do what he's going to work on.

11:55 The future is bright, man.

11:56 Yeah, absolutely.

11:57 I think someone on on Twitter had said, so now we actually hire folks that contributes to, you know, obviously C# and F sharp and dotnet stuff, but also folks that contribute to Java, Rust, C++, and now obviously Python, right? So we have a whole host of language folks underneath the umbrella of the company. And so now being able to have that space of collaboration, like underneath the umbrella of one company, I think is going to be really interesting to see what the next couple of years looks like. Yeah, you have to come back in six months and give us an update on all the Python changes. I'll give you a six month update on it.

12:30 This episode is brought to you by Linode. Simplify your infrastructure and cut your cloud builds in half with Linode's Linux virtual machines. Develop, deploy, and scale your modern applications faster and easier. Whether you're developing a personal project or managing larger workloads, you deserve simple, affordable, and accessible cloud computing solutions.

12:50 As listeners of Python Bytes, you'll get $100 free credit. You can find all the details at pythonbytes.fm/linode. Linode has data centers all around the world with the same simple and and consistent pricing regardless of location.

13:05 Choose the data center that's nearest to you and your users.

13:08 You'll receive 24/7/365 human support with no tiers or handoffs regardless of your plan size.

13:15 You can choose shared or dedicated compute instances, or you can use your $100 in credit on the S3 compatible object storage, managed Kubernetes clusters, and more.

13:25 If it runs on Linux, it runs on Linode.

13:28 Visit pythonbytes.fm/linode and click that create free account button to get started.

13:33 All right, I'm going to run the chooser.

13:36 Paul, it has chosen you.

13:38 Not sure about your chooser algorithm here.

13:40 It doesn't seem to be...

13:42 Random dot choice, I'm telling you.

13:45 This time it's not a good time to talk about election strategies.

13:50 And it's seen as how it chose you first.

13:52 It could be flawed.

13:54 Are you changing the random seat on every...

13:57 Exactly.

14:00 Yeah, I should be. I'm not.

14:02 My topic is "Things change, nothing changes." And I think I'm maybe stealing someone else's topic, so I appreciate your random algorithm choosing me first.

14:10 And it's kind of along the lines, I'm kind of interested in open source communities over the years and decades and why one works, why one doesn't.

14:17 In particular, a lot of open source projects that get successful kind of founder when the founder leaves.

14:23 And over the last couple of years, you've seen Guido transition out, Stepping out as BDFL and kind of a new language governance model that he didn't even decide on.

14:33 He let it emerge.

14:34 And then new release managers, new PSF directors, all happening this year.

14:39 A pandemic, which kind of destroyed the big fundraiser, an organizing event called PyCon, Python 2.7 end of life, which I guess a lot of us thought would never effing happen, and it effing happened.

14:53 And it threw all of this change for a project that was like reaching mega escape velocity and popularity Nothing changed. Everyone's like who cares python just kept on doing the doing its business releases kept cranking out peps kept cranking out Elections well python elections kept cranking out Michael and brian kept cranking out podcasts Hallelujah something good happened in 2020 And so it's interesting to me, healthy projects are those with a healthy bench, kind of a middle management.

15:24 Python has a hell of a middle management now. It's awesome.

15:28 And it's like middle management in every vector now.

15:30 It's got language leadership, middle management, foundation leadership, middle management, conferences, all the regional conferences are kicking ass, even online.

15:41 So it's really good for me to see a balance between some of the old heroes like Brett Cannon who gave this awesome keynote two or three or four years ago at PyCon about open source.

15:51 And then some of the new heroes like Marlene Mangami.

15:54 When I give the Python 1994 talk, sometimes I give it with Barry.

15:58 And it's like, "Well, did you ever know back then that it was gonna..." Hell no, we didn't know.

16:03 And so seeing all of this happen, seeing generation after generation come in, do their thing, raise the next generation, is pretty cool to see.

16:10 Yeah, it's fantastic.

16:11 And I know there was a lot of uncertainty when Guido stepped down about, oh my gosh, what's going to happen?

16:17 But it's kind of been a non-issue, which is good.

16:20 Non-issue in a good way.

16:21 - Indeed.

16:22 - Yeah.

16:22 - Yeah, I think it's really nice to also have, you know, some stability, even, you know, especially nowadays that's quite rare.

16:28 And just to see, you know, stuff just works.

16:31 Nothing goes wrong, no drama, no, you know, nothing terrible, just drums, almost boring.

16:36 - Yeah, boring's good these days.

16:38 One of the things that really surprises me, Paul, as you mentioned, the escape velocity.

16:42 What's weird to me is how Python went like 20 years on medium, like simmer popularity, and then it hit the afterburners and took off.

16:51 Right? It seems just, I think there's going to be some interesting studies done about why or how did a language that had kind of been around for 20 years all of a sudden really dramatically improve in popularity.

17:01 I have some ideas, but I just think that that's an interesting aspect of what you're talking about here.

17:05 Yeah, it's also the whole aspect of the, you know, the previous like Microsoft and Java and sort of, I guess, big money behind those sorts of languages and like the early 2000s pushing them.

17:17 And then I think community and to the point of like developer gave developers what they wanted, rather than maybe say enterprise-y languages.

17:28 And then I think there's a whole confluence of other things that happened as well.

17:32 But it is kind of crazy, right? That like one of the top, if not the top languages right now, They're almost 30 years old.

17:40 And it's got issues, it's got warts and whatnot, but it's a thriving language.

17:45 And without significant big companies behind it, that it's competing with companies that have thrown millions of dollars into development of these things.

17:54 - Yeah, absolutely.

17:55 And Matt and Ines, you two are on the data science side.

17:58 And I think the data science and computational folks are the ones that poured the gas on the fire and really made it take off.

18:05 And that's not where the growth necessarily completely is, but I think that gave it a serious kick.

18:09 - Yeah, no, actually, because I get asked about this a lot, like, oh, what do you think made Python the language for machine learning data science?

18:16 And my theory is actually that, A, Python, of course, was lucky that it was in the right place at the right time, but I think actually it won because it's a general purpose language.

18:25 People can come from other fields of Python into machine learning data science, for example.

18:30 Like, what did people do before?

18:31 If you look at kind of these surveys of what to use Python for, before it's a lot of web stuff.

18:35 You can build kind of anything in Python.

18:38 And I think that's what makes it so appealing and much more appealing than, let's say, like an AI language.

18:43 And that's also, in my opinion, why a lot of these attempts at making an AI language haven't taken off.

18:48 And people are like, "No, Python's great." It's fast enough.

18:52 It's usable enough.

18:53 But I think the general purpose is really what kind of got it over the edge there.

18:57 And a lot of people that I'm teaching or training are people who don't want to be programmers.

19:03 They want to use Python as a tool.

19:05 It's like Excel to them.

19:06 They don't want to be programmers.

19:07 And so something that they can pick up and it just sort of works and they can go to Stack Overflow and copy something.

19:14 And as long as they get the indentation right, it sort of works as long as they're using the right version or whatever.

19:20 There's no compiling.

19:21 There's no linking.

19:22 Yeah.

19:23 Weirdness.

19:24 And then you have things like Google Colab or whatever, where you have an environment that's all set up and you can just sort of go right.

19:30 And so that is really empowering for people who don't want to be programmers, but view Python as a tool, a necessary tool that they need to leverage.

19:39 And in addition, there's over a quarter of a million packages on PyPI, right?

19:45 And that's like literally standing on the shoulders of giants.

19:48 You can go there and pick out something that you want to do, and you're a pip install away from it.

19:55 And that's super powerful, right?

19:57 You don't have to set up a huge environment.

20:00 Yeah, it just really simplifies life for a lot of these people who don't want to be programmers but want to leverage these really cool, bleeding-edge...

20:08 But at the same time, you do have very strong support for C extensions.

20:12 So if you're looking at the top packages people download from PyPy, most of them are written in C, Cython, whatever.

20:20 That's what makes them so powerful.

20:22 So we have that while also being able to provide a pretty good developer experience.

20:26 Yeah, absolutely.

20:27 I think it was Seymour Papper who talked about a good language having, he was talking about Logo, but it's even more appropriate for Python, having low floors and high ceiling.

20:36 That's really easy to get in and do simple stuff, but when you want to do complex stuff, you're not limited.

20:42 And so just yesterday I was doing a workshop with people, and someone who had just started with Python, I said, "Oh, but look what we can do if we extend it several layers." And he was amazed how complex you can get and how sophisticated you can get just in a few lines of code.

20:57 It's really a testament to the elegance of the design of the language, I think.

21:00 Andre Vlasovsky in the chat makes a good point that many folks say Python is their second favorite language.

21:07 And there's something to be said for that.

21:10 Dierks, sorry, should be there first.

21:12 I'm with you.

21:14 Well, to that point, I mean, it's also been said that Python is the second best language for doing things right.

21:19 So you might have a language that's better.

21:21 That's actually his quote, I got it right.

21:23 or you might have a language that's better for web apps, you might have a language that's better for system admin, but there's probably not a language that's going to get you all three of those, right?

21:33 And Python is sort of that Venn diagram that gets you in the middle and is good enough to do a lot of what you want to do.

21:40 Absolutely. Random by choice, chooses Brian Okken.

21:43 Well, hey, cool. Let me pull up my notes.

21:46 It was interesting, we were talking about Microsoft, and I didn't choose this, but I'm going to mention it anyway.

21:52 tons of projects switched over to GitHub Actions in 2020.

21:55 And I mean, because it just kind of hit the area right like at the end of 2019.

22:02 So I saw a lot of projects migrating to GitHub Actions.

22:05 Anyway, I wanted to talk about packaging, which is sort of related.

22:08 There's a couple things, I know I was supposed to pick one.

22:11 The exciting thing happened today, pip20.3 was released and the dependency resolver is now the default.

22:19 The dependencies resolver is already angry at me for trying to deploy some dependabot changes.

22:23 I'm excited about this and it's bit me already too. I've had requirements.txt files that have broken dependencies. So I have to go look at those and see which, you know, I have to pay attention to it.

22:35 So there's a little bit of pain for some people, but it was such a long struggle.

22:41 I look back and the request for a dependency resolver came in 2013.

22:46 So, 7 years later.

22:48 But something magical happened in 2019, and that was some money from Mozilla came through and they threw some of that money at the dependency resolver.

22:56 That was pretty cool.

22:57 Yeah, that's a big deal that money was there's been a couple of these experiences where money was given to Python specifically to hire a team to improve something. PyPI, the rewrite of pypi.org was one and this is another.

23:10 Yeah, and they rolled it out in several versions. So we had our version early early in the year.

23:15 And then around July, we had another version that had a different flag that you could try it out with to try to get some of the early adopters to try it out to make sure it's working.

23:23 And then today, it's rolled out for good. And I'm pretty excited about it.

23:26 I kind of want to tie this in with the packaging. I've seen a lot of progress on and a lot of adoption of the PyProject.toml, kind of taking over more packages and more uses.

23:38 So we saw flit used to have its own any file now only uses PyProject.toml.

23:43 There's been additions to the metadata definition that hopefully will take hold.

23:50 And even pytest now, you can throw your configuration in a pyproject.toml.

23:55 And I've seen more and more projects saying, getting pressure from their users to say, "I don't want another any file. I want to put my settings in the pyproject.toml." So that's a good thing.

24:05 Yeah, absolutely.

24:06 You know, a couple of things on that. You talked about two things when you talked about packaging, you talked about GitHub. I mean, the fact that now you can actually do some package work within GitHub itself, I built a lot of small little tools, not things that I would want to go on PyPI. No, we don't need to clutter that up anymore with my garbage. But it's great when I can just say like, hey, here's a link. And if you're really, you know, concerned on how to add it, you can just hip install but point to this GitHub, you know, and that that works.

24:36 Yeah, and also the stuff about finding security vulnerabilities automatically and doing PRs for you that GitHub does against like your various package definitions.

24:46 I think that's great as well.

24:47 Hey, has anybody ever put a JavaScript package on the GitHub package.json?

24:53 Probably, but not me.

24:55 I have.

24:57 You know this is a Python podcast, right?

24:59 It's Gatsby, so I basically put 500,000 JavaScript packages into my repo.

25:06 Yeah.

25:07 And I'm saying, that is something I appreciate about Python a lot, that like, I actually, I never thought I'd say this, but like, I do prefer also that you don't get...

25:14 I can't resolve the Dependabot security mega warnings fast enough.

25:20 It's just incredible.

25:21 I know.

25:22 And I just turn them off, right?

25:23 Like, you just don't.

25:25 That totally fixes it.

25:27 - That is one way to do it, yes.

25:30 - It's like, I've always said that's a problem with writing tests, right?

25:34 Like when I started programming, I was all like, kind of hate writing these tests because they keep pointing out shit that's kind of wrong and then I have to fix that.

25:41 But I don't have tests.

25:42 It all works fine and then I'm happy.

25:44 Same with, I don't know, TypeScript.

25:45 - Exactly.

25:46 - Or actually even TypePens to some, fuck yeah, I know that's kind of wrong.

25:49 So now I actually have to fix it.

25:51 So, you know, just don't.

25:53 - So we should not let you speak at our high school.

25:56 It's the see no evil, hear no evil.

25:58 I do think that the Dependabot stuff, there is something about turning it off.

26:04 It drives me crazy. I really value it.

26:06 But at the same time, the reason I had this conflict with the dependency resolver is it told me I should, it gave me a PR to upgrade one thing, but not one of the dependencies for which the newer version was required or it would break the resolver.

26:18 It's just like, "Ah, what is this thing doing?" Alright, very good, Brian.

26:22 The chooser is all choosing.

26:24 - You need a batch dependency resolver.

26:26 - Yes.

26:27 - One that resolves everything at once and then makes sure you've got a working state at the end.

26:31 - Well, the thing is, the PIMP is fine.

26:33 It was right.

26:34 It's that Dependabot, if you have 100, not 100, I probably had 20 dependencies in this thing, it gives you one PR for each line.

26:42 It sometimes just like lags a day on the other one.

26:44 And if it gets it out of order, then the resolver, anyway, long story, frustrating.

26:49 - I mean, to be fair, speaking of dependency resolving from the perspective of a library developer.

26:54 It is hard, and you do want to use versioning properly.

26:57 And I do feel like there's this eternal conflict between, how do we manage dependencies properly for users, and how can we make sure that we can define how our packages depend on each other in a way that's reasonable?

27:10 Because not-- and use semantic versioning.

27:12 Not everything is a major release.

27:14 Not everything is a breaking release.

27:15 But stuff just depends on each other, and you don't want to lock people in.

27:18 There's always this-- the idea is, you should be able to pick up your project a year later with its requirements and it should run.

27:25 And that's really, really, really hard.

27:28 - Yeah, and if it gets wrong, then you probably get a lot of email or a lot of GitHub issues.

27:32 - Hey, Brian, on your topic of PyProject TOML, what's the latest, greatest, oneditable installs?

27:37 - I don't think it's completely working for all the same.

27:40 So I'd like to be able to say pip install -e, is it -e?

27:44 And just a project.

27:46 You can say pip install just a directory, no matter what kind of packager it has.

27:52 So that works now.

27:53 So you can, even if you use flit or something, you can't do -e, but flit itself, for instance, has its own editable install thing.

28:02 And I don't know what poetry does.

28:03 I don't use poetry.

28:04 So it's a different command.

28:06 I'm annoyed that it doesn't just convert it.

28:08 I wish it would plug it in, but I think that'll be coming in maybe in 2020 or 20, probably not this year, maybe next year, hopefully.

28:16 - Yeah.

28:17 Bad Wolf says in the chats, what is a quick solution for getting pip version updated?

28:22 The first thing you create a virtual environment.

28:23 This is another thing that drives me crazy about pip and if that could get changed, I would love it.

28:28 When you create a new virtual environment, basically for about a week, you have the latest PIP.

28:32 And then afterwards, soon as you pip install something, it says, warning, your pip is out of date.

28:37 Soon as you create a virtual environment, it's like, well, why didn't you just update it?

28:39 I want, if you're gonna tell me it's wrong, just fix it.

28:42 - You can use virtual environment.

28:43 Instead of VNV, use virtual environment because then you can update your pip and you can update the global one and install it in all your environments.

28:51 - Interesting.

28:52 Yeah, I just wrote an alias that will create the virtual environment, upgrade pip and set up tools, and then activate it.

28:57 Other way around, activate it, then upgrade.

29:00 - Yeah, it's funny, I have a shortcut like that too.

29:02 - Yeah, awesome.

29:03 - And then you can also have it install everything you need, you know, in the same command, automatically create once, which I shared my script somewhere, I think on Twitter, if anyone's interested.

29:12 - Nice, yeah, mine's just v and v.

29:13 I just type v and v and magic happens.

29:15 I don't think of it anymore.

29:16 - Well, when I finally switched to virtual Linv, it's like, I make so many virtual environments, it's so much faster.

29:23 And just that, even just a few seconds, several times a day, it's worth it for me.

29:27 So, I like it.

29:29 - Awesome.

29:30 All right, I've chosen next, and there's a dependency in the chooser, so it's worked correctly.

29:35 So it's chosen me actually, to talk about the next thing.

29:37 And Paul touched on this, but I think this is a really important one.

29:41 Like this is the year that Python 2 actually went away.

29:45 And it's, I just spent so long having to justify why Python 3 is the right thing, or more often teaching people, no, no, no, you can't type Python, please don't do that.

29:57 Unless you're on Windows, then you have to do that.

29:58 But if you're not on Windows, then you better not do that.

30:00 So you gotta, you know, it's just like this drag, it always felt like things like Django or Flask and whatnot were being held back.

30:07 All these libraries were being held back by Python 2 support.

30:11 And last year they started dropping it, and then Python 2 itself actually went away.

30:15 Brian and I tried to create the right nomenclature around this, I guess.

30:19 I was calling it legacy Python and modern Python rather than two and three, and that was fun.

30:24 People started to quote it and use that terminology, which was kind of cool.

30:27 I don't know how well it worked, but it's really nice that Python 2 is finally officially gone.

30:33 And I thought there'd be a bunch of people would freak out for Python and start creating alternate old versions.

30:39 And I haven't, besides some commercial support stuff, I really seen that happening.

30:43 So it's pretty interesting, I think.

30:45 - Yeah, I think we should have brought like horns and stuff and like, you know, waving.

30:50 - Some of those pop, like bottle-looking - Yeah, yeah.

30:54 - stuff on New Year's, absolutely.

30:56 Yeah, I mean, we really didn't get the party we were hoping for, right?

30:59 Like PyCon was supposed to be this big send off for Python 2, but no, there was not so much of a PyCon.

31:05 Not really, I mean, there's online, but it wasn't the same.

31:07 - Now we just gotta wait until we have all of the articles about the companies that finally decided to switch and the steps that they took that they avoided for the last decade to make this a reality.

31:22 One of my biggest corporate training clients, they have been using Python 2 for years and years and they do all sorts of testing of hardware using Windows with all sorts of custom C extensions.

31:35 So I've been saying to them for years, every time I do training, "Look, you should really think Python 3 and you should realize you should move in that direction." And their attitude was basically, "No, no, no. We've got so much invested in what we've done. We're not really going to upgrade." And finally, about, I guess, a year ago, they realized, "Huh, we're going to be stuck on our own maintaining everything, including all these packages of the API. Maybe it's time to start talking about it." And so I'm actually doing some training for them now. And I asked a group, "Should we do Python 2 or should we do Python 3?" And the majority actually said, "It's okay can use three because our group has switched.

32:05 So if they are moving, it's clear that even sort of the dragiest of the draggiest companies are doing it.

32:12 - Yeah, I was just gonna echo what Ruben said that, you know, when you have success and you get enterprises using your stuff and enterprises are slow to move.

32:22 And I don't wanna like criticize Python three too much, but I think if they would have put f-strings in 3.0, then they wouldn't have had this problem.

32:29 But because they waited to put f-strings in, then they had people dragging their feet.

32:33 It's so weird how that drag, that was one of the most popular features for sure.

32:37 So you're saying the walrus operator was what finally did it?

32:40 [laughter]

32:42 I'm not gonna go there.

32:44 You know, I was not super excited about the walrus operator.

32:46 I've found myself using it all the time, I like it.

32:48 It's neat.

32:49 Oh, it's cool, I think it definitely solves a problem.

32:51 I just, wait, that's 3.8, right? The walrus?

32:54 I believe so.

32:55 Yeah, I'm still, I mean, I feel like I'm now back at like, okay, we finally dropped support for 2 from all of our libraries that we're developing, and now we're on 3.6, because--

33:03 - Yeah, yeah. - No, no, no.

33:05 3.9 is current, 3.8 is now the new legacy Python.

33:09 (laughing)

33:11 - Ines, how was it for you, like, as a library developer?

33:15 - Yeah, I mean, for a lot, I feel like I only really got to use Python 3 this year.

33:19 I mean, of course, I've been using it privately for much longer, but I would say, in mid this year, I was finally like, cool, I just use type hints, I use fstrings across all of our projects, because before, our dependencies had to support people on two.

33:31 And our old version of spaCy had to support people who wanted to use two.

33:34 So that kind of becomes the baseline.

33:36 And now we've moved that up to 3.6.

33:38 And so until 3.6 is dead, there's still always new features that we can't just ship.

33:45 - Yeah, but that step from two to 3.6 is gonna be way bigger than 3.6 to seven, right?

33:51 That's a very small step, the 3.6 to 3.7 or 3.7 to 3.8.

33:54 So you've done the big step.

33:56 - Yeah, and some of the new features, It's like, yeah, whatever I can, you know, live without writing those in my code base and people can still use them in their code bases.

34:04 But yeah, all the new versions we've got to publish like spacey three that's coming soon.

34:08 We'll use Python three in the code base and we'll only work with Python three.

34:13 We want distribute wheels for Python two, which we might have not even done for a while because it's really, really a pain for some of our more abstract and complex dependencies that we're also building.

34:23 So far people haven't really complained.

34:25 Like I was, I was expecting people to complain a bit more, but like, no.

34:28 - Well, if you look at the PSF JetBrains survey, the data science folks that side of the house have definitely moved to Python 3 quicker and bigger than the web team.

34:37 - I'm actually surprised how quickly people move up.

34:40 Like, you know, I don't know, even after 3.9 came out, we've had people questing like, "Oh my God, why does your annotation tool not work on 3.9?

34:47 "Or why does this not work?" And it's like, "Nothing works on 3.9." Or it's like, "LCI doesn't do 3.9 yet.

34:53 "Like, I can't build anything.

34:55 "None of our other packages run.

34:57 Like, sorry.

34:58 I felt like a big person developer for like the first time ever when I actually got my first issue with running the latest and greatest Python, it was because I was running GCP and I kept running into an error, running into an error and I was like, you know what?

35:11 It used to work.

35:13 Let me just try it on three, eight, five, and then sure enough, it worked perfectly and I was like, I've done it.

35:17 I've reached the new achievement.

35:19 Nice.

35:21 Yeah.

35:21 I've taken down the talk by then training website by shipping the the walrus operator accidentally.

35:26 At the time, Ubuntu didn't support it.

35:28 A quick question or comment from Zach in the comments about when we can actually say legacy Python's gone when the OS vendors stop supporting it. Well, I think Python 2 is no longer shipping with the new macOS and Canonical is starting to make Python 3.8 the default.

35:44 So I think we're getting there.

35:46 I thought they were removing Python in general from macOS and just making you install it yourself.

35:52 I think there might be some internal system thing.

35:54 I saw people talking about the system Python 3.8, even though I agree I heard the same thing you said.

35:59 But I don't have a fresh install.

36:00 - I think it's not actually such a bad idea.

36:02 Like I think, you know, for most things you do, you have to install some extra stuff anyway.

36:06 So making it a more explicit install from a user's perspective, I think, isn't even so terrible.

36:12 - Yeah.

36:12 - Because otherwise, you know, if you just type in Python, you just run into all kinds of potential issues anyway.

36:17 So yeah.

36:18 - Yeah, especially when it's super out of date.

36:20 All right.

36:20 I end up telling people in my courses, oh, you're using a Mac?

36:24 Yeah, ignore the Python that's installed.

36:26 I know this is a Python course.

36:28 Ignore the ones installed.

36:29 And here are ways that you can install it on your own so it'll be up to date.

36:32 And here's how you can make sure it doesn't conflict with what's already installed.

36:35 So maybe removing it all together is not such a terrible thing.

36:39 - Yeah, for sure.

36:39 Homebrew, homebrew.

36:41 All right, Ines, the voting system has chosen you next.

36:44 - Unmute myself and look at my notes.

36:45 Yeah, so I think I've already mentioned the topic I was going to talk about about five times so far, but yeah, I think in 2020, we really saw the ecosystem around type hints, static type checking, types in general mature and like really be widely adopted.

37:02 And I do think a part of that is the more widespread adoption of Python 3 because before that, to be fair, you could use type hints before Python 3 just for completeness, but like that's not what people did and it looked very, very different.

37:15 So yeah, now, and actually even I couldn't really Imagine writing code without type-ins anymore.

37:20 Like if I write something and I'm not adding any types, it feels kind of naked and it feels kind of wrong and I actually feel like, oh my God, I might be missing some issues here, maybe I'm not catching some bugs.

37:31 So yeah, which is quite surprising, but it says something considering I only just recently was able to really embrace all the new Python 3 stuff.

37:39 - Yeah, that's a pretty quick transition.

37:41 Find it, yeah.

37:42 I depend on this thing now that previously you had to stay away from.

37:46 Yeah, and I have static type checking enabled in Visual Studio Code.

37:51 It underlines things if you pass in the wrong types.

37:54 It gives you all these nice autocomplete prompts or previews if you want to.

37:58 That's also something I've gotten really, really used to over a really short period of time.

38:02 What about you, everyone? Do you use type pins?

38:05 Can we have an opinion, Paul?

38:07 Just raise the hand and then we summarize it for people who just--

38:10 - Everybody. - I might have to do it slowly.

38:14 I'm slowly starting to use them more and more as I see the advantages.

38:19 And in part, as I mentioned before, we started recording, like, because my students are encouraging me to use it more and more because they want it, because they come from languages where there's no such thing as optional type hints.

38:28 It's, you must do this or the program does not run.

38:30 So what do you know? They can actually be useful.

38:33 Yeah, I think that's one of the biggest benefits that I like about the type hinting system is that it's optional.

38:38 So you could choose to use it.

38:40 And then if you don't need it, and we could just take it out.

38:42 Because for me, I believe in being a lazy developer.

38:45 And stuff like type hints just make certain things a little bit more discoverable.

38:50 And kind of like what we were saying before, it just improves your tooling to be able to discover more of your code for you.

38:57 I like to use it for APIs, especially for package interfaces, so that people using a package can use the editor tools and stuff like that for the hints.

39:08 And also just wherever it makes sense.

39:10 I mean, if it looks like it actually distracts from readability, then I take it out. Readability is above all for me.

39:18 So if it's usable and readable, it helps.

39:20 I mean, I wouldn't advocate any, like, I don't know, if you have a function that takes several arguments and then returns, like, a callback that gives you this kind of typing, I wouldn't advocate for that necessarily.

39:30 But yeah.

39:31 And I feel like they're working on some of that confusion as well, and some of that, like, clutter and bloat.

39:36 I mean, that was, I talk about that 3.9 story.

39:39 It was because I was tired of typing, typing dot list or typing dot dict.

39:44 And it was like, Oh, you mean I can just do this now?

39:46 Like, and then sure enough, you know, stuff crashes and it's like, well, guess not maybe later, but it's great that for people who are looking at Python as a tool that they're using, not the language that they're writing on a day to day, but just flat out the tool that they're using.

40:04 They don't know what we're talking about half the time, but they can kind of look at something and go, "STR, hmm.

40:12 Someone mentioned a string that one time. Maybe that's what that is." You know, it's a lot easier than just looking at the code and trying to figure it out.

40:19 Yeah, and speaking of tools, I think type-ins, they're not only that syntax feature.

40:23 They've really enabled this whole new ecosystem of tools that use type-ins to accomplish something else or in a creative way.

40:31 And I think many of them have been covered on this podcast previously.

40:35 For example, there's Pydantic, which uses type-ins in actually one of my favorite creative ways for data validation, settings management.

40:41 It's actually a library we've adopted across a lot of our projects.

40:44 You just quickly, you know, you define this interface. Here's what I expect.

40:48 You get like, you know, kind of like JSON schema just with type hints.

40:51 - Identic is so wonderful. Yeah, you're right. It's really nice.

40:54 - Yeah, I would say it's one of my favorite recent discoveries in terms of Python developer tools.

40:59 And then of course, there's FastAPI and Typer developed by my colleague Sebastian, which probably has been discussed a lot on this and related podcasts.

41:08 Yeah, he was on the show long ago, like 10 shows ago or so.

41:11 And yeah, that's a fantastic framework, too.

41:13 Yeah. So that's, I think it's also partly what really popular, popular, popularized, great word, type hints in as a bot, I don't know, as a tool for, yeah, developing applications and so on, I think also, also actually Pydantic. And yeah, we've also in 2020, I think, released our So machine learning library think, which kind of functions more like an interface layer and lets you write neural networks, combine layers written in different frameworks like PyTorch, TensorFlow, and we've included custom types for arrays and all that stuff that passes through the network.

41:49 Because if you're doing machine learning, a lot of it is really abstract.

41:53 You have this multidimensional array as in 300 dimensions, and then that goes through, and then it gets transformed, and then that goes all the way back.

42:01 And at the end, you're hoping to get something out.

42:02 And there's a lot that can go wrong.

42:04 And it's not even about like, "Oh, how experienced are you as a programmer?" It's like, that shit's just abstract.

42:09 And we can make it a bit less abstract, even by just providing custom types that you can declare.

42:15 So it's easier to see what goes in, what comes out.

42:19 Do these layers fit together?

42:20 Do they not?

42:21 Because otherwise, yeah, you easily end up with these very hard to debug NumPy errors triggered by like five layers deep that tell you that you can't broadcast that shape into this other shape.

42:32 and you're like, yeah, that's gonna be fun.

42:33 - This doesn't help me, I have no idea.

42:35 Yeah, I think there's also things like mypyC, which will compile stuff that has types, and there's Cython, which will use the new Python type hints to compile to much faster code.

42:45 So there's a lot of neat things coming out of it.

42:47 - Yeah, and I think especially also for abstract stuff like machine learning, there are lots of use cases that web type hints and type-based validation is really promising, because we can also take something a developer already knows, like typing your functions, typing your layers, and it will just work do some extra stuff if you wanted to.

43:03 Watching mypy evolve is really cool.

43:06 And for Think, we've briefly dabbled in my custom mypy plugins for a bit.

43:12 Like we have a plugin that checks if two layers you're chaining together return the correct types.

43:18 Like, you know, if you have one layer that returns a two dimensional array of floats and the other layer expects a two dimensional array of integers or four dimensions, whatever, you can actually see that underlined in your editor done by by mypy as you type the code.

43:31 So you can prevent a bug before you even make it, like type it and introduce it.

43:37 So that's cool.

43:38 And I think that's all very actively developing.

43:40 So I think there's so much potential for custom plugins.

43:43 Like imagine more libraries just coming with their own mypy plugins that help the developers use those libraries better.

43:49 I think, yeah, that's all super cool.

43:51 - Yeah, it's early days and there's a bunch of great tools.

43:53 Cecil, let me round this out with, coming back to one thing you said.

43:57 You talked about how it was optional.

43:59 - The closest analogy that I think we have to this in other languages has got to be TypeScript, which is like JavaScript plus types and a little transpiling thing.

44:07 And it's all cool, but all my interactions with TypeScript are like, oh, you haven't declared this thing right.

44:11 This thing's, it's like, it's as picky as C++ or C# or Java, but it's not as like built from the start.

44:20 Like the tools aren't necessarily all there to make that fit together.

44:23 So I always feel like I'm kind of stumbling over the things.

44:26 Like 80% of the time it's helped me 20% of the time that frustrates me.

44:30 Whereas like here, you can just kind of, you can leave it off, take it or leave it if it's good, not good.

44:34 Yeah.

44:34 Yeah.

44:35 I think TypeScript definitely helped with some learnings about how developers want to interact with typing.

44:41 I think the thing with TypeScript is it is very, like there's a lot of features in there and it's very customizable, but you know, it's kind of like you have like a nail, right?

44:50 And I gave you a jackhammer.

44:51 You really don't need the jackhammer.

44:53 Exactly.

44:54 So finishing hammer would be nice.

44:56 Yeah.

44:57 There's a lot of different things that you can customize.

44:59 And some people need that.

45:00 But I think for 90% of folks, I'm making up that number, that's not a real number.

45:05 But for a lot of folks, you probably don't need a lot of those knobs to turn and options to set.

45:11 Yeah, absolutely.

45:12 There was like a two-year period with TypeScript where I'd read the roadmap.

45:16 They do a great job of running their project.

45:18 Great job.

45:19 But they put a feature in using words I didn't know from type theory.

45:23 And my answer was, Oh, that's for Angular. Oh, that's for Angular. Oh, Angular is the only one having that problem.

45:30 There's some serious machinery.

45:33 Yeah, there is. All right. Final thought on this from the chat is Eric O'Shaughnessy says, I love how typing solidifies the author's intent. Yeah. All right. Let's have the chooser choose again.

45:45 Jay.

45:46 All right. So I totally didn't peanut butter on this. But as y'all were talking, like, I just got some wonderful ideas.

45:52 And I immediately started thinking about just the doubling down of some of the technologies that we've seen kind of pop up over the last couple of years, but then truly evolve.

46:02 And like one of the biggest examples of that, and I know I'm like five years, day, late, dollar short here, but like Jupyter, anything Jupyter notebook related, like it's not just for the solo data scientist anymore, like collaborative Jupyter notebooks, like to me that I know people that just have dreams about that. And they're like the best dreams ever.

46:23 And, you know, I know, Mike, you just did a podcast episode a few weeks back about that.

46:30 And we were talking about it on Python Bytes a little bit. They're like, it's not just one option anymore. It's, you know, there's Coke out there's PyBinder, there's DeepNote. Apparently, I'm a DeepNote influencer, and I didn't even know it.

46:41 But that's awesome. You are bringing it up again.

46:46 But not just in like Jupyter land, we have like crazy ideas, like what if we could do Python on the front end with anvil? And like now anvil is saying like, hey, we're going to open source some of our server technology so that you can take this technology and do it yourself. And I think that that's just like with type hints, we're coming up with these ideas, and we're taking them a step further. I think that this is what, in my opinion, has kept me in the Python ecosystem as long as it has is because I'm constantly learning about like brand new ways to do a thing in Python. And I have to talk about this because I told people in my Slack channel that I would, but like we just thought, you know, at Elastic, we just launched a brand new tool called Elon and it's in beta. And the whole idea is it's like a panda's data frame, but instead of storing all of your data in memory, you can store it in an Elasticsearch, you know, data store.

47:38 Oh, that's an interesting idea.

47:39 So now instead of being limited to what your RAM can hold, you're limited to how much space you got.

47:47 And you can run queries on it, you can tie it into things like, scikit-learn and TensorFlow and do machine, any type of machine learning on it.

47:56 And at the end of the day, you're doing transforms on the data as it's in the data store.

48:02 So there's no more downloading a bunch of data, working on it and trying to re-upload it.

48:06 And it's off of the backs of crazy ideas from people crazier than I am that are just like, "Hey, what if we could do this?" And to me, that's all I wanted to do with Python is just make up goofy stuff and see if it would work.

48:21 - I think that's an interesting idea.

48:22 You could start using things like lots of data and things like serverless, where you get really short execution times and not much power.

48:30 But if it's already in something like Elastic, you could do pandas against it, like quote pandas, like the API against it.

48:36 without really suffering a lot of trying to download a sheer little Docker Lambda callback thing before it runs out of time to execute and whatnot.

48:44 - Yeah, I've actually, I think I've seen some people do some cool stuff with spaCy and Elastic as well, you know, processing all of, you know, they're huge volumes of text, which also I think is such an interesting part of our application of data science.

48:55 Like, you know, you wanna have, you have so much text that, you know, you can't even, nobody can read it.

49:00 You can barely store it, but you can still analyze it and process it and find something out about it Otherwise, so yeah, that's not that actually sounds quite cool. I mean the problems I was I was having initially was just this amazing problem of like I've got a million lines of data that I need to do something with and I've got this little Mac mini here That's just chugging along trying its best and everyone's like well, why don't you just use the Google API and it's like, okay Well, I did the math. I'm gonna need $20,000 and it's like, okay, so we can't use the Google API We can't solve it with that problem. What else do we got and now it's like, oh wait you mean I can just store all this data because I'm it's only a few gigs, but then I can just hit my local server as many times as I want, let, you know, search store or any type of data store that's good at retrieving the data as I want it as I need it and then let it do its thing. Like that solved a $20,000 problem. I mean, even even aside from the cash, do you want to send all your data to Google? Probably not. Yeah.

50:03 You can also send it to Facebook while you're at it.

50:05 I don't know. I was working with police call records, so I don't want to send that data anywhere.

50:10 I don't want to be on a watch list.

50:12 No, exactly. I think those are like really, a lot of these really important applications of the technology in the health domain, people are doing good and useful stuff, but they should not do that stuff at that type of cost, which means sharing all of their sensitive data with companies that will, you know, want it for some other reason.

50:29 Yeah, absolutely.

50:30 Matt, Reuven, you guys do training.

50:32 What's the Jupyter story around there?

50:34 - I did a class with, not Jupyter, but it was all Colab based this morning.

50:38 So yeah, I use it a lot.

50:40 I think it's great for beginners.

50:42 Like I said, for a lot of people who are scientists or engineers, that's what they sort of consider Excel.

50:49 And so a lot of them sort of want to sit in Jupyter all day.

50:53 They don't even want to like think about modules or code or even software engineering, right?

50:59 And so it's somewhat of a challenge for me to sort of say, okay, let's talk about some engineering skills and some good best practices for Python projects. And I'm like, I don't want to leave Jupyter, right? Can I do this all in Jupyter? I'm like, well, you could, but you kind of don't want to per se. But yeah, definitely cool. I think what I'd love to see is Inez's fancy mypy checker hooked up with pandas on Jupyter, that would make my day, because that's That's one of the things that I find confuses people just with pandas having 50 ways to do things and then objects returning five different types or methods that return five different types.

51:37 You don't get any tab completion or any of that good stuff in Jupyter when you're doing a lot of the machine learning stuff.

51:43 - Yeah, and a bunch of grown-up engineering stuff is coming to Jupyter, right?

51:47 Like NBDev, NBQA, some of these other stuff that's coming there is making it way more interesting for that regard.

51:54 Yeah, the MV dev stuff is interesting. Just coming. I mean, that's sort of taking it to that sort of literate style of programming where we're going to stay in Jupyter all day and do everything from that. And then, you know, Netflix has their stuff where they're deploying Jupyter notebooks to production.

52:09 Yeah, it was paper mill. Yeah. Reuben, how about you? And then I'm gonna run the chooser.

52:12 I use Jupyter every day for my training.

52:17 I don't use slides, I just do live coding.

52:19 I find that's a great way to show them, not just the answer, this is how you code something, but this is the process you use to get to the solution.

52:29 Because teaching people that process is really important.

52:31 It also means I can share with them everything that I typed during the day really, really easily.

52:35 People are a little resistant to using it first, especially experienced developers.

52:39 They're like, "Wait, what is this thing?

52:40 Why are you not using a normal IDE?" I've actually forced myself in the last year to remember, "Oh yeah, we should do a few exercises and tell them to use the IDE so they won't think that everything is done in Jupyter." That's a great environment, but there are other ways to develop software.

52:54 Why does Python only run in this weird web browser thing?

52:57 Right, right, exactly.

52:58 Speaking of this, I think this is another reason why Python has taken off so much, just that you want stuff to be dynamic.

53:05 If you're doing the sort of data science-y type of work, you want to run your code that even if you're running it in an IDE in a more traditional way.

53:13 But I think Jupyter has especially...

53:15 Not just like, "Oh, you want to have these little notebook cells in your browser," but more like, "This is the type of way you want to interact with code.

53:21 Run something, change a line, run again, run that part." Well, that's where they're coming from, too.

53:25 They're coming from notebook environments.

53:27 And so this makes sense for a lot of these people.

53:30 And being able to just explain it on the same...

53:34 Just in one look.

53:35 Like, instead of going, "Here's five slides explaining what I'm about to do," or here's just the code.

53:42 Now let me sit here and explain it to you.

53:44 I can literally break it down line by line.

53:46 Okay, this is what's about to happen now.

53:48 Let's run this and watch it happening as we go.

53:51 For people who don't write Python full-time, again, it's amazing because for them, they can see it piece by piece, just like, you know, my daughter, I'm just trying to teach her how to read early and it's just like, break it up piece by piece.

54:03 And like, you can do that when you're training young experience-wise Pythonistas and just say, like, hey, let's just look at what this does.

54:12 And if we want to dig into it, it's almost like having a debugger in your presentation and just being able to like stop at any point in time and go, all right, let's jump in, let's go deeper, let's really talk about this.

54:22 - Yeah, or be able to step back two lines of in your execution.

54:25 - True.

54:26 - All right, just for the time being.

54:27 - I started using.

54:28 - Go ahead, last thought, Reuven, you get the final thought on this one.

54:30 - I started using more and more markup in my notebooks as well to like, sort of document what I'm doing explicitly and sort of instead of just using hashtags and comments.

54:41 So it becomes much more obvious with bullet points.

54:43 I think people are enjoying that as well.

54:45 - Yeah, that's great.

54:46 And that's, I mean, that's part of the Zen of notebooks.

54:48 All right.

54:49 And you're up next, the random match choices selected you.

54:52 - Okay, so my big story for this year is very small story, a Microsoft story.

54:58 It's COVID, it's the coronavirus, which I think for the first time ever, it's the topic that everyone is speaking about in every country, every walk of life.

55:08 Like whoever I talk to, wherever they are, this is the only thing people talk about because it is such a profound effect on everything we've done.

55:15 Now, you know, this is a Python show, so we'll talk about the Python aspect of it.

55:18 And I'm not trying to minimize at all the extraordinary pain that people are going through physically, medically, emotionally, financially, on, on, and on.

55:27 But Python has continued sort of chugging along throughout this crisis.

55:33 In fact, we've seen that big companies, high-tech companies are actually among the winners in this as the whole world has gone digital.

55:39 And companies are hiring Python developers.

55:42 They're increasingly desperate for Python developers.

55:44 The number of people in my corporate training classes, who when we go around and everyone introduces themselves, who were hired in the last six months, is amazing to me.

55:52 Like, these are people who are working from home, were interviewed from home, hired from home, and are now doing training from home as their companies sort of grow in interest and power and everything.

56:01 So people who have Python skills have continued to be, I would even say, are increasingly in demand, which is amazing.

56:08 The bad news, or among the bad news, is, of course, so I mentioned this before, PyCon, and all the other meetings and meetups and gatherings have suffered as a result.

56:17 The good news on this front is I've been able to speak at many more conferences than I would have been able to because, hey, I can attend the conference from my home office. I don't need to fly anywhere, go anywhere.

56:26 But you know what? All of these conferences, these online solutions are still really lacking.

56:32 At the end of the day, as much as we want to say, we're virtual, we're online, we communicate this way.

56:36 And that is an important part of things, that face-to-face contact, that personal contact is really missing.

56:42 And I'm really feeling it.

56:43 I've only been going to conferences for a few years now, and boy, I really miss being in the same room as other people and talking to them and chatting and just sort of randomly bumping into people.

56:50 And then you also have the financial aspect of it that someone was mentioning earlier, the PSF, their big fundraiser each year is Python.

56:57 And I'm sure it was a hit for them, not just not to have it this year, but to have to cancel the venue.

57:02 And I am extremely skeptical that's going to happen in person.

57:05 I know EuroPython already said it's not happening in 2021.

57:08 And we'll have to see exactly how this affects the community going forward.

57:13 I'd say so far we're doing okay, but here's an opportunity for something to figure out how can we do conferences online that are effective in a closer way to what we've been doing so far.

57:24 The news is, I would say, on balance good for Python and for people who know Python, but still pretty bad overall.

57:30 And I'll also add that these vaccines, I mean, boy, oh, boy.

57:34 If they work, this is going to be like--

57:37 not as a biologist or anything, but I'm so amazingly impressed with all the science and the work that people have been doing on this front.

57:44 And I think this has a chance also to affect their lives moving forward, how they develop them.

57:49 They just need to make a PyCon bubble.

57:51 So in the United States, the NBA was continuing to play because they had a bubble where they isolated all the players for however long, right?

58:00 So we just all go to somewhere and bubble up for however long.

58:05 - Then we can have our conference.

58:06 - I'll isolate with you guys for a month, absolutely.

58:08 - I don't know if my family will let me do that.

58:11 - Jay, we'll just do it in San Diego, man.

58:12 It's nice there.

58:13 - Okay, I'm all for it.

58:14 - Coronado Island, we'll just get a section of it, okay?

58:15 - Yeah.

58:16 - But no, mandatory quarantine for like conferences.

58:20 I just did two week quarantine here to be able to enter Australia.

58:23 - Yeah, just put an Australian.

58:24 - And I can just chill around in the hotel room for two weeks and then you're free.

58:26 - Icon in Australia.

58:28 - That sounds okay to me.

58:29 - Problem solved.

58:30 - Well, I mean, to be fair, it's expensive.

58:32 So I wouldn't, you know, it's like, you know, would you want to pay like, you know, your conference plus two weeks at a hotel in order to do that?

58:40 I don't think that'd be even less inclusive than a lot of these conferences already are, so.

58:45 - I don't want to take all of his thunder, but one of the first conferences that I watched was, Cecil, please remind me now, 'cause I'm blanking.

58:53 That's how long of a year it's been.

58:55 the conference that you helped put on and the Juneteenth.

58:58 Yeah.

58:59 And, and with that, I mean, one great conference, I'm sorry, I screwed up on remembering the name.

59:06 That's my fault, not yours.

59:07 But also the sheer fact that that conference was organized in two weeks, and it was an amazing conference, like the ability.

59:16 That could never have been done if it was in person, right?

59:19 Yes.

59:20 So like the accessibility that you're providing to new speakers, people that normally would not be able to get up and give a presentation. I've had that conversation so many times of, like, the type of talks that I like to create are not the type of talks you see at PyCon, and there's a reason why. And it's not a bad thing. It's just it is what it is.

59:39 And I think when you're able to say, you know what, let's do a conference. Let's make this happen. Let's just get up. We can have fun with it. I've seen some conference talks that were literally small budget films. They had multiple casts. I've done transcriptions on conference talks that literally had... They gave me the script and said, "All right, here you go." So when you can actually do things like that and think outside the box and use this time to readjust the needle... There was a good podcast episode on More Than Just Code, I believe. Not More Than Just Code. I don't remember what it is. I'll find it. Greater Than Code. That's what it was. And they said, "If we haven't taken this time to do something better when things are different, when things have improved, then shame on all of us because yes, this sucked, but it caused us to think so much outside the box that we should be able to take all that knowledge with us when we can get back in the box and just have a ton of fun with it." Yeah, I think a lot of opportunities have been made available for people to attend conferences and for people to get jobs where companies would previously say, no, you have to move to San Francisco or to Omaha or whatever.

01:00:50 It's like, well, you know what? No one's going in the office. Let's try hiring from the best in the world instead of the best in Nebraska or whatever.

01:00:55 I think that might be the biggest fallout of this, Michael, is just the change in everything being centrally located in Silicon Valley.

01:01:03 Yeah. I've heard of a lot of people like that actually just have jobs in the valley that have moved back to where their parents are because it's like 10 times cheaper or something like that and they're not going to want to move back.

01:01:17 Have you heard that some companies have been readjusting their salaries?

01:01:21 They're saying, "Well, if you live outside of the Bay Area, then it's cheaper, so we're going to cut your salary as a result." Yeah, but people have always been doing that.

01:01:30 That's like, unfortunately, not new.

01:01:33 So if you're listening and this happens, you can say, "Well, you no longer have to pay $18 million for my office area, so let's call it an even swap." (laughing)

01:01:45 - Just keep a virtual address in the Bay Area.

01:01:49 - Exactly, my PO box is right down on Fifth Street.

01:01:51 All right, last one, this random choice was easy 'cause we're down to our final excellent guest.

01:01:56 Matt Harrison, what do you got for us?

01:01:58 - Okay, my thought was just on sort of the hardware aspect of things this year.

01:02:02 So I have a daughter who started college this year for better or for worse, and she's learning both C++ and Python right now, her linear algebra class as a Python lab, which I think is awesome.

01:02:14 I wish I had that.

01:02:15 But I was actually helping her with some homework.

01:02:19 I wasn't doing it, but she was like, "I can't figure this out." So I was her rubber duck the other night.

01:02:24 And her issue was allocating memory, right?

01:02:27 And so one of the things that's nice about Python is you don't have to worry about allocating memory.

01:02:34 She got all the output, but she was just double freeing a pointer here, which they checked on submission.

01:02:40 So pythons now now we're seeing python and i'll just tie this like I have a client who wants Next year wants me to do a training on gpus with python Which just goes to show like my clients tend to be big companies and so big companies are now leveraging like specialized technology To do things very quickly, right? And if you have a gpu you can get ten hundred thousand times performance that you get out of a CPU.

01:03:08 So I think that's super exciting.

01:03:10 There's the Rapids project, which is heavily sponsored by Nvidia.

01:03:14 And I'm super impressed with that.

01:03:17 They're basically like taking well-known APIs and basically saying, hey, your code basically change an import or with the change of an import and this hardware, your code will run X times faster or whatever.

01:03:31 And so I think that's super compelling and exciting, Not that necessarily everyone will have access to those sorts of things, but it's now getting to the point where it's becoming more common.

01:03:40 On the note of hardware, I think another thing that was really interesting this year was the Apple M1 chip coming out.

01:03:46 I actually bought a beefy PC this year, having run a MacBook for 15 years, just because due to Corona and whatnot, I needed something beefy to start doing virtual sharing and presentation like this, and my MacBook just wasn't cutting it.

01:04:02 But the thing I don't like about my PC is that it has, it sounds like it's gonna take off every time I start doing something, right?

01:04:10 Like modern Intel chips can take us to Mars if they want to.

01:04:14 We just need to like make a huge drone out of them.

01:04:17 And so I'm super excited to see, like if you can get a silent MacBook that, you know, is super powerful, runs faster than Intel chips, that's super compelling, especially if they allow you to get a little bit more RAM, which from the machine learning side is interesting to me.

01:04:34 But I'm also excited about the competition factor of that.

01:04:37 I think like Intel and the chip set is sort of stagnated for the past couple of years, we've sort of run out of the end of Moore's law per se.

01:04:46 So I think both the GPU and the M1, it'll be interesting to see what the competition is from the PC manufacturers, if they're going to do a similar everything on a chip thing just sort of go with what they've been going with.

01:05:01 I think competition is good.

01:05:02 And so I'm excited that Apple did this, even though there's going to be some growing pains and Python might have some issues at first, but I think in the end, new technologies, new hardware that allow us to write at high levels and not have to worry about freeing and allocating memory, which again, a lot of people are using Python, don't care about that.

01:05:20 They don't want to be programmers.

01:05:21 They want to leverage great code, stand on the shoulders of giants and the latest and and greatest software and hardware and have it be fast.

01:05:29 So I'm super excited about the future with them.

01:05:32 Yeah, I'm really excited about both these things.

01:05:34 Like the whole GPU revolution is incredible.

01:05:37 And the Apple M1 stuff is going to be really, really interesting.

01:05:40 Like, like you, I, I bought it maxed out Mac mini actually had already ordered a top of the line MacBook pro 16 inch, and then that came out.

01:05:49 I started seeing the reviews.

01:05:50 I'm like, you know what?

01:05:51 Forget it.

01:05:51 I'm not getting that thing.

01:05:53 I'm getting this new one to try it out.

01:05:54 And he does have some drawbacks, like Docker doesn't work right away.

01:05:58 Maybe it will, maybe it won't.

01:05:59 But I have a 2018 MacBook Pro that's pretty good.

01:06:02 I have a SimRacing 16 core machine over there.

01:06:06 Like, I've got plenty of Intel stuff to work.

01:06:08 So I'm gonna try to jump on this.

01:06:10 And I think the interesting Python angle here, besides just having stuff to run our code in, is, you know, Ines talked about like, her CI didn't support 3.9 right away.

01:06:18 Well, whose CI is gonna build wheels for Apple M1 straight away and things like that?

01:06:23 There's going to be some interesting growing pains, especially on the data science side, as this true platform.

01:06:27 Yeah.

01:06:28 It's fun.

01:06:29 You mentioned that because actually, yeah, when this came out, we were also thinking like, ah, maybe we should get one of these, you know, we want to make sure spacey compiles properly on it and we can really, you know, optimize for that.

01:06:37 And did you guys go on?

01:06:39 Yeah, because someone has to do it or so, you know, not yet.

01:06:41 No.

01:06:42 Also it's a bit difficult with the traveling, but like, yeah, that's kind of.

01:06:45 When we have other priorities too, this is not, I wouldn't put this like at the very top, but like, it's definitely something you want to do, but at the same At the same time, we also want to keep optimizing for CPU.

01:06:54 Like, that's the other thing.

01:06:55 Of course, you know, even all these new NLP models, all of that stuff obviously runs great on a GPU, and we'll see a lot of improvements, and we also see this become much more available and cheaper for people to use.

01:07:06 But at the same time, we're still optimizing our tools and libraries for CPU.

01:07:10 Like, you can run it on your random laptop, and also it's still, for a long time, it will be cheaper still.

01:07:15 If you just, you know, you want to run lots of machines, and you can get those, you know, much cheaper in the cloud.

01:07:20 And if you can parallelize, you can train your models very efficiently on CPU, if you can get it to work.

01:07:26 That will always be cheaper for a long time.

01:07:29 There's a lot to say about just the overall cost of computing as a whole.

01:07:34 I mean, this is great. This is $10.

01:07:37 And I mean, I can run a lot of stuff on that.

01:07:39 You're holding up a circuit, an Adafruit thing just for the people who are not on the video.

01:07:44 Oh, yeah, sorry. I'm holding a Raspberry Pi Zero W.

01:07:47 So it's like, you know, four gigs of RAM, like enough, you can buy a 64 gig, like flash drive for $3 now, like it's amazing.

01:07:57 Like just the overall cost of compute has dropped so much.

01:08:00 I mean, what was it?

01:08:01 The, what is the Raspberry Pi?

01:08:03 400 where it's, it's just, it looks like a keyboard, but it's actually a whole computer.

01:08:07 Yeah.

01:08:07 So it's like, as we continue to do more and more, we're doing it with less and less and it's cheaper and cheaper and cheaper. And most of us are coming from some place where the cost of living is too damn high. And we're able to say that we can afford to buy a $15 Pi Zero and not think about it too hard. But the fact that we're getting to that point where you can have people all over the world where the question isn't anymore, how do I get a computing device in my hands? It's like, okay, now I have some choices and they're all affordable. Like to me that's we're going to see a lot of things come where Brian Okken is right now where VS 2103 is in the future. There's gonna be a lot of great things coming. Yes, I'm looking at chat.

01:08:49 I have my kids to have one of those one laptop per child in their bedroom over there, right, which I think that the Raspberry Pi 400 is sort of, it's the next gen of that. It's actually something that makes sense. Everyone has a TV, right? And you can just plug it in and you're You're not in some weird, I mean, you're in Linux, so yeah, you're in a weird environment, but you can run--

01:09:13 - Is that one of the laptops with the hand crank on it?

01:09:16 - Well, it has the little wings.

01:09:17 Mine didn't have the hand crank, but--

01:09:19 - Okay.

01:09:20 - I actually, like, I kept hearing about Raspberry Pi, and so I decided to order one recently, and I haven't done anything with it.

01:09:25 It just came a few days ago.

01:09:27 But I'm looking at this thing, I'm like, oh my God, this little card here that cost me less than $100, a lot less than that, is more powerful than the computer I use to run my entire business for years, and years and years.

01:09:39 It's just astonishing to me.

01:09:41 - Yeah, so I think that's probably a good place to round out the show.

01:09:44 Like, the future is so exciting, and in some ways we're kind of living there, right?

01:09:48 Some of these things that are coming out are so amazing.

01:09:50 And I just want to say thank you all, to you all for being here.

01:09:53 You're all doing such amazing work, and it's really humbling.

01:09:56 You all took time from crazy schedules, from traveling, from weird locations, to just be here and make this whole event happen.

01:10:03 Thank you for listening to Python Bytes.

01:10:04 Follow the show on Twitter via @pythonbytes.

01:10:07 that's Python Bytes as in B-Y-T-E-S.

01:10:10 And get the full show notes at pythonbytes.fm.

01:10:13 If you have a news item you want featured, just visit pythonbytes.fm and send it our way.

01:10:18 We're always on the lookout for sharing something cool.

01:10:21 On behalf of myself and Brian Okken, this is Michael Kennedy.

01:10:24 Thank you for listening and sharing this podcast with your friends and colleagues.