Transcript #69: Digging into StackOverflow's 2018 survey results

00:00 Hello and welcome to Python Bytes where we deliver Python news and headlines directly to your earbuds.

00:06 This is episode 69, recorded March 15th, 2018. I'm Michael Kennedy.

00:11 And I'm Brian Okken.

00:13 And Brian, probably nothing to cover with notebooks or anything. Like last time we had kind of an over the top notebook thing, didn't we?

00:20 Yeah, we did. We covered a few topics on notebooks.

00:22 Good thing. We probably won't do that again this week.

00:24 - We just want to say a quick thanks to Digital Ocean.

00:27 They're sponsoring this episode and many of the episodes of Python Bytes.

00:30 So they really are helping make the show happen.

00:33 Check them out at do.co/python.

00:37 Talk more about that later.

00:39 Maybe we'll talk about notebooks, Brian.

00:41 - Yeah, let's talk about them just for a little bit.

00:42 - All right, for a little bit.

00:44 - So because we talked about notebooks last time, one of our listeners contacted us and said, we should take a look at a project called PYNB.

00:55 It says Jupyter Notebooks in plain Python code with embedded markdown text.

00:59 So the comment from our listener was, loves Jupyter Notebooks, but he also loves using his own editor.

01:06 I think he uses PyCharm.

01:08 I like doing that too, but you can't, it's like not trivial to edit.

01:14 I don't know, I haven't even tried to try to edit a notebook in PyCharm.

01:17 You probably can, but I haven't tried.

01:20 Anyway, but I like this idea of, it's basically a converter that you can convert back and forth between a plain text with, or a Python code with embedded markdown back and forth to a notebook.

01:36 So it looks pretty cool.

01:37 - Yeah, it's pretty wild.

01:38 So you define like a cells function, and then you can put like docstring type equivalent of markdown, and that becomes a cell, and then you put a little bit of Python, that becomes a Python cell, the more markdown within docstrings, and that becomes like another cell.

01:53 And yeah, it's super cool.

01:54 So really nice simple example.

01:55 It's quite cool.

01:57 - Yeah, and one of the things that it points out, which is nifty, is it allows you to use diffs better with your version control tools or things like that.

02:06 So that's, yeah, it's worth checking out if you wanna do more with notebooks, but wanna be able to store 'em in plain text better.

02:14 - It's really just right down to the essence of it, isn't it?

02:17 they're pretty verbose if you crack open the text file and look at them.

02:21 You know, you look at this and it's like literally what you would see in the notebook is like all you type.

02:26 It's pretty cool.

02:27 One of my to-do lists was to play with some notebooks a little bit, so I'll play with this also.

02:32 Yeah, so PyNB, PyNotebook, check that out. Very, very cool.

02:36 So you know how there's this really cool William Gibson quote, "The future is already here, it's just not evenly distributed"?

02:42 Yeah, I love that.

02:43 That's a cool quote. Yeah, I do too.

02:45 And I often when I think of like, oh, where's that future we were hoping for?

02:49 Well, it's not well here.

02:50 Well, maybe it is, it's just not evenly distributed.

02:53 So this next thing feels like it kind of falls into that realm.

02:56 And it's definitely like from the futuristic world.

03:00 I don't know how far futuristic is this, two years in the future, 10 years or 100 years or something like that.

03:07 Or is it just live in science fiction?

03:08 But I promise this has something to do with Python.

03:11 It just takes a moment to get to it.

03:12 So the announcement is that Microsoft has created a quantum computing language called Q#.

03:19 - Yeah, okay, this just sounds neat.

03:21 - This is awesome, right?

03:23 - Yeah.

03:23 - And it's been out for a little while, and we were talking before the show, it's like it was almost worth covering before just because, wow, there's a programming language specifically designed for quantum computers.

03:35 That's a pretty nice language, actually.

03:37 But the news is now it's available for macOS and Linux, and you can try it, right?

03:43 So there's this new language called Q#, and if you look at it, it's a little bit like C#.

03:48 It's also a little bit like either TypeScript in some ways, but also like Python.

03:53 And I was really pleased to see like, the if statements don't have parentheses around the condition and stuff like that, because we don't need them.

04:00 Why are we forced to write all those parentheses?

04:02 Like they're unnecessary.

04:04 We all know that from Python, and Q# is sort of borrowing some of these cool ideas.

04:08 The type annotations are very much similar to Python's type annotations as well.

04:13 So, quite cool.

04:14 There's this new language.

04:16 One of the first questions you might ask is like, great, there's this quantum computer programming language, but there's no quantum computers.

04:22 So, what's the point?

04:24 Like, why are we gonna run this?

04:25 I mean, I know there's like a couple of bits, but not, you can't go and get like a quantum computer to test your program on.

04:31 So how do you test it?

04:32 - Good question.

04:33 - Yeah, so they've got like this emulator type thing that emulates the behavior of a quantum computer, although I suspect it's slower than a real one.

04:40 - Well, are there, maybe I'm just not up to date, are there quantum computers now?

04:44 - There are, gosh, I can't remember who was, it feels like IBM, but there's like a couple of bits, like a two-bit quantum computer in like a super, super cold environment that can do like very simple things.

04:58 So, but it's not like a general purpose, you know, unless you've got like a hydrogen bath, like liquid hydrogen bath to put it in, and things like that.

05:08 So it's based on topological qubits and quantum computers, so hence Q#, and it's out for macOS and Linux, it's awesome.

05:16 On Windows, you can play with it in Visual Studio Standard, but on Mac and Linux, you can use Visual Studio Code.

05:23 And there's like a quantum programming code extension for Visual Studio Code, so that's pretty cool.

05:29 - Yeah, and so you're saying that this ties into Python somehow?

05:32 - Somehow.

05:33 In fact, so they said, all right, we created this new language, and we created this way to emulate it, And there's actually some stuff in their cloud computing platform, Azure, where you can plug in these things, so they all tie together in interesting ways.

05:45 But they said, "Look, we know other languages exist, and they're really important.

05:49 The one language we think is so important, it should be involved in this as well, is Python.

05:54 So you can now use Python to program quantum computers." - Oh, that is awesome. Yeah. - That's awesome, right?

06:03 It emulated quantum computers, but still super awesome.

06:07 And they said, oh, and a lot of this analysis has to do with data science-y type stuff, so we'll also support Jupyter.

06:12 - Yeah, I predict a whole bunch of people in college doing their thesis on this stuff like this.

06:18 - Yeah, that's a really good point.

06:19 Put the science-y aspect back in computer science, right?

06:23 - Yeah, definitely, that'd be neat.

06:25 - So do you speak Spanish, Brian?

06:28 - You know, this is embarrassing.

06:31 I took two years of it in high school, but I barely can order a burrito.

06:35 Yeah, my Spanish is not very good.

06:37 But one of, there is a fellow, A. Valbona, I think is his name, that does speak Spanish, and he gave a talk at PyCon ES in 2017 about pytest, and let me know about it, and showed me the slides, and this was kinda cool.

06:56 He sent it to me on Twitter and said, hey, I forgot to tell you, I gave this talk last year, and I think it was in September, And I said, "That looks really cool.

07:06 "There's a lot of neat stuff in it.

07:08 "Unfortunately, I don't speak Spanish, but neat anyway." And he said, "Oh, let me translate it for you." So he translated the slide deck for me.

07:17 And the slide deck, we have a link to the English slides.

07:21 And there's also, if you speak Spanish or want to watch anyway, there's a video on YouTube that we'll link to.

07:27 But one of the things, one of the reasons why I wanted to bring it up is because there's some neat things in there that I'd never heard of before.

07:34 There's some that I have and some that I haven't.

07:35 Like, for instance, he does talk about using pytest Django, so how to hook up pytest with Django models and stuff.

07:44 And then a thing called Model Mommy, which is a way to mock out Django models.

07:50 Haven't heard of that.

07:52 One of the things that doesn't get, that I need to talk about in full length on my podcast is pytest Lazy Fixture, which is, it's probably soon to be one of the recommended ways to have fixtures be able to go into a test function parameterized input.

08:11 Anyway, it takes a bit of explaining.

08:15 But then there's also some fun things.

08:18 Freeze gun is a way to freeze time for your testing into a specific spot in the interface that looks really cool on freeze gun.

08:25 - That's cool.

08:25 I love the names here, the model mommy and the freeze gun.

08:28 These are great.

08:29 - Yeah, and then a package called eradicate, which removes all the comments out of some code because sometimes code is easier to read if you remove the comments.

08:40 So that's fun.

08:42 - Code comments are deodorant for code spells.

08:46 Right, they're there to explain why your code is poorly written, right?

08:49 So a lot of the times, if they're not full-on documentation meant for a help statement, yeah, I totally agree.

08:54 That's awesome.

08:55 - Anyway, those are some fun things so I wanted to bring it up.

08:57 And then a bonus topic just today announced that pytest.org added a reference page which has a one page reference, full reference to the pytest API.

09:09 Oh, that's cool.

09:10 All in one place.

09:11 Yeah.

09:12 Super nice.

09:13 All right, speaking about stuff in one place, let me tell you about DigitalOcean and a cool feature they have.

09:20 So you know that you can go to DigitalOcean and create virtual machines, right?

09:22 Like Linux servers and stuff?

09:24 Yeah.

09:25 Yeah.

09:26 So, you know, that's just the start a lot of time.

09:27 What if you want to run GitLab or you want to like set up MongoDB or you're going to run like a discourse server, you got to install Ruby, you got to install the whole discourse stuff, the database, et cetera.

09:38 So one of the things you can do at DigitalOcean and when you go to create a new machine, they call them droplets, is you can create these one-click apps.

09:46 So I can click over here and say, I would like to click and say, boom, new discourse server all set up, configure it and save, go.

09:52 I want a new Go server, go.

09:54 I want a new WordPress instance, go.

09:56 Just click it and it's up and running.

09:57 It's pretty awesome.

09:58 - Really?

09:59 Okay, that's neat.

10:00 - Yeah, you can even do Docker.

10:01 Like you want a Docker server, boom, hit that button and you've got Docker 1712 running, yeah, on Ubuntu 16.04, perfect.

10:10 - Neat, I'll have to check out all the one-click things that we've got.

10:13 Sounds cool.

10:13 - Yeah, so they got a bunch of stuff that helps you get going in a nice way.

10:17 So check that out, do.co/python and let them know that they are doing the right thing, supporting our show.

10:24 - Great.

10:25 It is once again that season in which the stack of Rufalo developer survey results come out.

10:31 So the 2018 developer survey results are out.

10:35 They said they had over 100,000 developers this time.

10:39 I think last year they had 64,000.

10:41 So this is cool, it's growing.

10:43 And I thought it'd be fun to just cover some of the high points.

10:47 They spent a lot of time actually talking about history of education, how much education level did your parents have?

10:56 A lot of that kind of social stuff.

10:59 - Interesting, that sounds neat.

11:01 - Yeah, it's pretty interesting.

11:03 Yeah, things like that, there's a bunch of that.

11:05 But I'm gonna focus mostly on the tech bits.

11:08 So one of the first questions they asked was about open source, and they said, how many of you contribute to open source?

11:16 And 55% of professional developers contribute to open source.

11:20 - That's higher than I would have expected.

11:21 That's awesome. - Yeah, that's really awesome.

11:23 I think for all respondents, there were some students and stuff and people who were retired.

11:28 So it was slightly higher.

11:29 That's not surprising, but I thought the professional 55% was like the most interesting there.

11:34 And then people often feel like if they didn't get a computer science degree in school, like they don't necessarily belong in programming or it's hard for them to get into programming.

11:43 But it turns out that only 64% of the people have CS degrees.

11:47 - That's actually, yeah, I think that that's actually seems high to me.

11:51 There's only about a third or less of the people I work with have CS degrees.

11:55 - Yeah, I agree.

11:56 I think that seems even high for my experience as well.

11:58 Maybe CS degree people spend a lot of time on Stack Overflow, I don't know.

12:02 - Maybe, but why are they spending so much time?

12:04 Because supposedly they have a degree.

12:06 - Exactly, come on.

12:08 So one of the social things they talked about was sort of the sense of belonging.

12:14 - Oh, nice.

12:15 - People's perspective as developers among their peers.

12:18 So they put that under the manner of experience and belonging.

12:21 They said, they had a graph and they said, okay, how much do you feel this is true or false over time?

12:28 Like how much does this apply to you over time?

12:31 Well, not over time, but if you've had one year experience versus a people with five year experience versus people with 10 years experience, there's like a graph over years of experience and this metric.

12:39 So they said, your connection and sense of community with other developers.

12:44 And apparently the more time you spend in the programming world, the more that, you know, matches, which makes sense, right?

12:50 But they also said, how much do you feel like you're competing with your peers instead of actually working together, right?

12:57 How much do you see it as a zero-sum game, I guess?

12:59 And in the beginning, people feel quite high that it's like direct competition.

13:05 That person gets more recognition, I get less, they get a raise, I don't, I don't know.

13:09 Something like that. But over time, that diminishes quite significantly.

13:12 – That's good. – Same thing for, yeah, for sure.

13:14 And then also, the feeling of, I'm not as good of a programmer as my peers.

13:19 pretty high at the beginning, over time, people feel less and less and less like that's a true statement, which I thought was good as well.

13:25 I mean, it seems to me like all of these are going in the right direction the longer you're in programming, which is a positive statement for the whole ecosystem.

13:34 - Yeah, but it also shows that we need to try to teach students, try to get that connectedness and the less competitiveness, teach that earlier.

13:45 - Yeah, right away, right away.

13:46 You want a kind of a bleak statement that totally applies to me.

13:49 How much time do you spend on computers?

13:51 The most popular answer, nine to 12 hours a day.

13:54 - Does that include your phone?

13:56 - I think it might even be behind a screen.

13:59 I'm not entirely sure, but if you kick back and play games or watch Netflix.

14:02 It was also notable that the usage of Python has exceeded C# for the first time.

14:09 They called that out specifically, I think.

14:10 - Yeah, cool.

14:12 - Then a couple more things.

14:14 There's languages and databases.

14:16 They talk about the most loved, the most dreaded, and the most wanted.

14:20 So most loved is you're using it and you love it.

14:22 Dreaded is like you've had to use it or you never ever want to use it.

14:25 It gives you fear.

14:26 And then wanted is I'm not yet using it, but I want to use it.

14:29 So these are all interesting ways to measure things.

14:31 So languages most loved, number one is Rust.

14:34 Even though not that many people use it, among the people who use it, they love it so much apparently.

14:38 Kotlin is number two and Python is number three.

14:41 Pretty nice.

14:42 - Okay, yeah.

14:43 - Dreaded, VB6, and CoffeeScript.

14:45 Keep away.

14:46 Python is number one, the most wanted language by 25%.

14:51 Number two is JavaScript at 19, which is quite a drop.

14:54 And then Go is at 16%.

14:56 So it drops out really quick and Python is sort of ruling that.

14:59 - Yeah, that's great.

15:01 - Yeah, databases, yeah, databases, most love Postgres, most dreaded is IBM DB2, Memcached in Oracle, and Memcached was responsible for the largest distributed denial of service stack ever.

15:13 And that was put onto GitHub recently.

15:15 So, and then most wanted is MongoDB.

15:17 A most popular editor, Visual Studio Code.

15:20 Pretty interesting.

15:20 - Yeah.

15:21 - And for developers, what OS do they use?

15:23 Last thing, Windows 49%, macOS 27%, Linux 23%.

15:28 Not what do they deploy to, but what do they write their code on?

15:32 - And this is one that we need to pay attention to more, I think, because of all the, a lot of the podcasters and bloggers and all of that within the Python community, I think either Linux or macOS might be higher.

15:48 - Much higher.

15:49 - But you're teaching to a group of people that half of them are running Windows.

15:54 So you just can't ignore Windows.

15:56 - Yeah, it's a really good point that like, even though it's sort of the thought leaders and the people who see at conferences and speakers and whatnot are probably either Linux or Mac, the actual people doing the work and using the stuff is very much Windows still.

16:10 So definitely not to be forgotten.

16:12 Speaking of presenting.

16:14 Speaking of presenting, Doug Hellman, he came up with, there's got a new project called DemoShell.

16:24 And the idea is, it came out, like somebody named Gene Hack on Twitter said, "Hey speakers, if you're gonna do live demos in a shell, "clear the screen after every command "and get the prompt back to the top "so folks in the back can see what you're doing." And that's completely reasonable.

16:44 - Yeah, it's a great idea.

16:45 - But instead of fiddling with it, so this demo shell, the idea is basically just to do that, to capture a Python project, to capture your input and emulate a shell, but it really just pushes all the work off to a shell to do the work.

17:01 But between commands puts your prompt back at the top.

17:05 And it's in its very early stages, so I actually reached out to Doug and said, "Hey, do you want me to announce this yet, or you ready for it?

17:15 And he said, yeah, I put it up there to start the discussion.

17:18 Be happy if a bunch of people showed up and started participating and adding things.

17:23 But he added a warning also that too much interest is going to be met with commit privileges on the repo.

17:31 Essentially, if you want stuff done, he might say, hey, just go ahead and do it.

17:36 But actually, that's a good thing.

17:39 He's up for people to help him out.

17:40 So this is it.

17:41 - Yeah, I think it's really neat.

17:42 It's a cool idea.

17:43 It definitely makes a lot of sense.

17:45 I've done so many training classes where it's not like stadium seating, it's like flat, right?

17:50 And it's usually in some room with a crappy projector screen that's like too small, like a third of it's off, you know, people in the back can't see it at all.

17:58 And this is perfect, right?

17:59 You just, everything stays at the top.

18:01 It's really great.

18:02 And people out there looking to find an open source project to work on, right?

18:06 Go and express some interest.

18:08 You might be met with commit privileges on the repo.

18:11 But it's still in its really small stages.

18:13 There's not a lot of code there, so it's pretty easy to read.

18:16 - Yeah, that's awesome. - I think that's cool.

18:17 Nice. Alright, so let's wrap this up with a really nice definitive statement.

18:23 The last thing I want to talk about is Python 2's end of life.

18:27 So we've spoken before how there's the Python death clock, and we know that in 2020, Python 2 will go unsupported.

18:36 What exactly does that mean?

18:38 How strict are they going to be about that?

18:40 Is it going to be at PyCon 2020?

18:42 Is it going to be the end of the year, beginning of the year?

18:44 Well, my friend, Nicola Erosi, who I have more news about in just a little bit, he's put this out on Twitter and got my attention.

18:53 Basically, there was this discussion on the Python mailing list saying, "Will there be a period where Python 2.7 is in security-only status before hitting end of life?" So like, say 2020, will it go into like, "Well, we'll do security fixes for a while, but no new features and we're not changing anything or like what is that going to happen earlier?" Guido actually came out and said, "Let's not play games with semantics.

19:16 The way I see it, the situation for 2.7 is the end of life is January 1st, 2020, beginning of 2020, and there will be no updates, not even source-only security patches after that date, period.

19:28 Support from the core devs of PSF, Python.org completely stops on that date." Yeah, there's your answer.

19:35 That's pretty abundantly clear, isn't it?

19:38 - Yeah, but that's way out, that's like way in the future.

19:43 - That has to be at least a year and a half away.

19:45 So for those people with like millions of lines of code on Python 2, they probably should start checking out things like mypy and some of the other tooling on pushing that along, right?

19:58 - Yeah, or I mean if you're planning on leaving your job and going to another company within the next couple years, don't worry about it.

20:04 - Yeah, exactly.

20:06 - Or, no, I've got a better one for you, Brian.

20:08 I got a better one.

20:09 Or you want to get a really sweet consulting job migrating Python 2 to Python 3.

20:16 You know, learn all of the nuances and go and, you know, when people are freaking out in November of 2019, you're gonna be in sweet demand.

20:26 - Actually, that's not a bad idea.

20:28 - Right, my rate's $250 an hour to start, so let's go with that.

20:32 - Yeah.

20:33 - I can help you with this problem.

20:35 - It's gonna take a while though.

20:37 - I got about a year to beef up on my conversion.

20:41 - Do some conversion study and that's right.

20:42 All right. - Yeah.

20:43 - Awesome, you got any news or extra things you wanna throw out there?

20:47 You did some recent test and code episodes, right?

20:49 - Yeah, so I guess I just wanted to shout out that I'm back into recording more often.

20:54 - Yay!

20:54 - Yeah, and I thought I'd start with a little test project.

21:00 So I've got a project called Cards that is kind of a to-do app, but I'm focusing on using it just as an example bed to talk about things that I get questions about all the time.

21:13 So, okay, your book is great for how to write the tests actually, but what test do I write?

21:20 How do I pick which test to write?

21:21 How do I pick test cases?

21:23 Basically, this intro to quality assurance, but for busy people that don't have a QA team, how do I do that?

21:32 So that's where we're exploring.

21:33 I'm two episodes into it, and I'm just gonna run with it until it runs out of steam, so check those out.

21:39 - Those are really awesome.

21:40 I think that's great advice.

21:41 And when people get that wrong, testing seems super horrible, right?

21:45 You're like, what do you mean I gotta test every single thing?

21:48 If you're focused on testing the wrong thing, you can just feel super, like it's just super busy work.

21:53 - Yeah, but testing should be something to help you develop faster, not slow you down, and that's where I'm focusing my energy.

22:00 How about you?

22:01 - Nice, well, I just got back from PyCon Slovakia.

22:05 That was pretty awesome.

22:06 So I wanted to send a shout out to everyone there who I met and that was fun.

22:09 So if you're in Central Europe-ish next year, be sure to check that out.

22:14 That was a good event.

22:16 I also have a new course.

22:17 - You do and I'm excited about it.

22:19 - Yeah, it's on the most wanted database technology.

22:22 So this is the first course based on Flask that I have.

22:26 So this is on a thing called EVE, E-V-E.

22:30 the E framework, which is a way to take Flask and a MongoDB database and turn it into a really awesome RESTful API with validation and business logic and all that kind of stuff.

22:41 So it's a super cool way and it's like really declarative.

22:44 So you say, here's my models, here's my endpoints, here's the validation for them.

22:48 Make that a REST service, go.

22:50 And it's nice.

22:51 So that's out now, that's actually written by Nikola Aroshi, the guy who talked about the Python 2.7 statement.

22:58 He wrote that and we just shipped it.

23:00 He's also the creator and maintainer of Eve.

23:02 So he's pretty qualified to talk about it.

23:04 - That's great.

23:05 And I've already started listening to it because I'm very interested in this topic.

23:10 - Yeah, awesome.

23:11 Cool, well, links in the show notes.

23:13 Check that out.

23:14 And I think that might be it.

23:16 I'm really excited that we have a concrete statement on Python 2.7 and it's not like, it's just vague, right?

23:21 - Yeah, that's good to have it.

23:23 I wonder if they updated the clock.

23:24 - Exactly, I was just thinking that.

23:25 We got to recalibrate all the clocks.

23:27 - Yeah. - 'Cause they're all guessing.

23:29 They probably got shorter.

23:30 Awesome.

23:31 All right.

23:32 Well, thanks again, Brian.

23:33 Thanks everyone for listening.

23:35 Thank you for listening to Python Bytes.

23:37 Follow the show on Twitter via @PythonBytes.

23:40 That's Python Bytes as in B-Y-T-E-S.

23:43 And get the full show notes at PythonBytes.fm.

23:46 If you have a news item you want featured, just visit PythonBytes.fm and send it our way.

23:51 We're always on the lookout for sharing something cool.

23:53 On behalf of myself and Brian Okken, this is Michael Kennedy.

23:57 Thank you for listening and sharing this podcast with your friends and colleagues.