Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.
This is episode 74, recorded on April 17, 2018.
I'm Michael Kennedy.
And I'm Brian Okken.
And I'm Matt Harrison.
Yeah, that's right.
We've got a special guest here on the show.
So Brian and I decided to invite Matt Harrison to mix things up a little bit and bring a slightly different perspective.
And we decided we're going to try this little experiment from time to time, you know, once a month, once every six weeks, something like that.
So, Matt, welcome to the show.
Thanks.
Thanks for having me.
Good to be here.
Good to have you here.
So before we get into the topics, I just want to say thank you to Datadog.
They're sponsoring this episode.
Check them out at pythonbytes.fm/Datadog.
I'll tell you more about them later.
Brian, you've been running your own open source project lately, and it's been fun, right?
Yeah, I started a little project.
I called it Cards.
But the intent was to – it's on GitHub.
And the intent is to talk about, like, how I'm going to go about testing it using all the – what I think of as good test methodologies.
But doing it in the open on an open source project, I'm getting contributors already, which is really cool.
That's awesome.
I don't really know the conventions.
And even though I've been programming for decades, I'm kind of new to the – actually contributing to open source.
Well, and it's different on GitHub than it was, say, in Subversion or something like that, right?
There's the whole Git flow and PRs and that whole thing, right?
Yeah, and how to deal with, like – how to deal with branching with – branching and forking with GitHub versus doing something out in the open on GitHub is one thing.
But actually running it like a normal open source project is completely different.
So I had a – actually, Anthony Shaw did a pull request on Cards, and he did a really good job on it.
And I was – one of the things is he started it with a WIP for a work in progress.
And I didn't know that was a convention already.
So I looked around, and there's some – if people are new to this and want to learn more, I'm sure everybody out there already knows about open source stuff.
But anyway.
I would not make that assumption.
I definitely would.
I mean, it feels that way, but I think it's more of a dark matter sort of experience where, like, you see the stuff that people are doing.
But that doesn't mean everyone is participating.
And many people, I think, are just wanting to get into it.
What do you think, Matt?
I've been using open source since the mailing list days, and you had to talk over the mailing list.
And so GitHub, when it first came out, was sort of a big change for me.
And I think a lot of people use GitHub as just, I'm going to throw my source over the fence and see what happens to it, rather than maintaining it as a real open source project and trying to include community and get contributions.
So I'm curious to see how this works.
And I think, like Michael said, don't make the assumption that everyone knows how to do these things, even though some people are doing it.
There's certainly a lot of people who can learn from this.
There's a couple of things I wanted to throw out.
This is from 2015 already, but it's an article called How to Write the Perfect Pull Request.
And it kind of talks about the philosophy sort of both with approaching the pull request when you're first getting started.
And it even talks about the WIP trick to tell the person that owns the repo, you're not done with it.
You just kind of like early feedback on it.
And then there's some advice on offering feedback to pull request submitters and also responding to that feedback as the submitter as well.
It's a short read, but there's some good tips there.
And then I'm also really excited about VM's new book coming out from Pragmatic called Forge Your Future with Open Source.
And it includes things like how to deal with pull requests and everything.
That sounds like such an interesting book.
I love it.
And I just like to say, like, I think getting using open source and getting involved for someone who wants to get a break into learning Python or learning a library or even getting a job.
That's an awesome way to do it.
I agree.
And one of the problems people run into is they work in a place that doesn't use Python.
And so they don't have a place to actually practice Python outside of just like toy things.
So contributing to open source lets you make a meaningful contribution, even though maybe you're a Java or .NET shop and they're like Python?
No.
Or yeah, they might even be using Subversion or some other source control.
Absolutely.
Cool.
This is a good one.
This is a good one, Brian.
So, Matt, you and I both do in-person training periodically.
And I just did a class last week, I guess, a short class.
And it was a lot of folks coming from a traditional sort of MATLAB, Mathematica background and moving into things like Jupyter.
And I think that this might be a trend.
What do you think?
Yeah, I think we're seeing Jupyter changing from early adopter to sort of normalcy.
I found a thing going around Twitter.
Paul Romer, who's an economy professor at NYU, tweeted about his experience with Mathematica and Jupyter.
And he referenced an article in The Atlantic about both of those products, Mathematica and Jupyter.
And for me, like I said, I've been involved with open source for a long time.
You don't often see stuff in The Atlantic or professors, economy professors.
This isn't a computer science professor.
This is an economy professor posting about Python and Jupyter.
So, really cool stuff there.
This is such an interesting find.
And if you open, people should open up the Atlantic link that you're linking to here.
Because, wow, that's a pretty provocative picture.
There's like a formal paper.
It says the scientific paper is obsolete.
And there's like a paper, like an academic paper, that's literally on fire.
Like in an animated way, like full screen.
It's crazy.
Yeah, and it makes reference to the discovery of gravitational waves and how there was a paper on that.
But they also, along with that, published a Jupyter notebook where you could go out on your own.
And you could look at their code, look at their data.
And it had embedded text in it as well and basically discover gravitational waves or go through the same sequence and reproduce their science.
So, I thought that was pretty cool.
A quote from Mr. Romer.
He says that Jupyter is the new open source alternative to Mathematica that is well on its way to becoming the standard for exchanging research results.
I agree.
I think academics has been too dependent on these couple of big, really expensive lock-in type of things like Mathematica and MATLAB.
I'm also thinking of journals and stuff like that.
This sort of open source paper in the form of Jupyter kind of touches on both of those.
Brian, what do you think?
I'm actually really excited about all that.
I was just listening to a topic not too long ago about how the – actually, it was one of your podcasts about the academic journals that are – a lot of times nobody actually follows the steps along.
But having the code out on Jupyter notebooks just allows everybody to go and follow along right there.
One of the main points of these articles isn't that there's a notebook per se, but the compelling reason for using Python and Jupyter is not necessarily that the technology is better, but that there's a huge community around it.
So, you know, they make the argument that the Wolfram notebook might be prettier or whatnot, but you have so many people who are contributing to these open source projects.
You've got Matplotlib for graphics, SymPy for symbolic math, NumPy, SciPy, Pandas, NLTK.
And, you know, if you look at PyPI, there's 135,000 packages last week on that, and it's really hard to compete with that.
That's super compelling.
So, really cool.
Yeah, it is super compelling.
So, speaking of community, Brian, you know I love to, like, pull on the Stack Overflow developer survey and, like, try to dig out results from the community, right?
Yes.
Yeah, that's always fun.
So, there's another one that just came out that gives us a different perspective and also is more Python-focused than that one, right?
That's, like, broad software development.
So, JetBrains, the PyCharm team, teamed up with the PSF just at the end of last year to do a Python developer survey.
And so, the thing I'm linking to is Python developer survey 2017 results, but it's, like, December, right?
So, it's pretty relevant still.
It's pretty fresh.
They were just talking about it on their blog.
And so, this is a really nice piece of sort of almost journalism around data science, I think.
It's actually, they've really written it up nice.
They show you graphs, and they're like, here's the main takeaway from this section.
Here's the main takeaway from that section.
So, how about I share some takeaways with you?
Yes.
Yeah.
All right.
So, the first one is, says, of the people that they interviewed.
Now, this is obviously a self-selecting crowd.
But the question was, you're obviously doing Python.
Is this your main language or a secondary language?
And they said 80% of the people are, Python is their main language.
That was pretty cool.
They said data analysis is actually just as popular as web development, which is pretty cool.
So, there's basically as many Python web developers as there are data scientists.
Does that surprise either of you?
To me, Python felt like a web thing for most folks.
Yeah.
I mean, recently, it wouldn't surprise me.
But, you know, if you would have said that two years ago, it probably wasn't the case two years ago.
But now, yeah, it's not surprising to me.
Yeah.
They also talk about the growth of Python.
And Brian and I, you and I, we've touched on this a few times.
But they're also confirming, like, we think that massive hockey stick growth is largely data science people coming in.
Yeah, it could be.
There's, I think there's a lot of room to grow.
There's a lot of people who are using Excel and some of these tools that you mentioned who probably want to migrate to something like Python for the libraries and machine learning capabilities.
Yeah.
Another interesting one was Python versus legacy Python.
So, Python is at 75% usage among this group and 25% for Python 2.
And if you look at the curve, that's, like, increasing in time, like, the rate at which people are moving to Python 3.
So, that's really cool.
Yeah, that's cool.
Yeah, you wonder how much self-selecting is there, right?
The legacy laggards didn't want to participate in this.
That's right.
I don't need no stick-in surveys.
I learned everything about Python I need to know in 2008.
Okay.
So, they also talked about where code runs, where people run their Python code.
And this is, I don't think it clues, like, the hosted notebook type stuff.
So, probably not that.
But 67% AWS.
Brian, does that surprise you?
I'm going to plead.
I'm not in the field, in the web sort of space to know, really, where it's running.
My basis for sort of judging the use of AWS compared to other platforms is when AWS goes down, what parts of the internet are no longer accessible?
And they're pretty broad.
Yeah, I would think that it'd actually be a little bit higher than that.
The ones that surprised me was you've got Google App Engine at 29, Heroku at 26, Digital Ocean at 23.
And the last one they say is Microsoft Azure at 16.
And I think that 16 is probably going to change a little bit.
They've been doing a lot of hiring in the Python space and getting some prominent Python people.
So, there's a big push from Microsoft on that.
They're definitely focused on Python in a lot of important ways.
They now have Azure Notebooks.
They have Brett Cannon, Steve Dower, both Python Core devs, working there.
They brought the guy who did the Python extension for Visual Studio Code in-house.
Like, they're doing a bunch of cool stuff.
All right, so a few more takeaways.
Team size, right?
Like, you think of how big of a group do you, you know, like how large of a team do you work on?
And if you think about, like, one of the advantages of Python is you don't need a large set of people to build something interesting.
And I think that's reflected here.
So, it says, like, team size, 2 to 7 people, 75%, 74% of the respondents are in that 2 to 7 group.
And then 8 to 12 is 16.
And then, basically, above that, above 12 people, all the way up to, like, 40 or larger, is 9% of the balance, basically.
So, really small teams.
And then operating systems, Brian, you touched on this a lot.
49% of the people are still using, are currently using Windows as their OS.
Then 19% for Linux, 15% for Mac.
And, like you said before, like, Windows often gets the short end of the stick and sort of testing and examples and stuff.
But it probably shouldn't.
Yeah, one of the things I want to go back to is the cloud platforms that we talked about.
One of the things that's interesting to there is that, clearly, some people are running on multiple platforms because that's over 100%.
Yeah, that's interesting.
Like, I could tell you for sure that, like, if somebody asked me, which of these platforms do you use, I would definitely check the DigitalOcean and AWS boxes.
Okay.
Because, like, for example, the main server for our podcast and the database server runs on DigitalOcean droplets.
But when you interact with it and you get, like, say, an email, especially around the training stuff, that goes through Amazon simple email notification service and things like that.
So, like, there's this blend of them.
Yeah, I'm similar.
I've done Heroku and DigitalOcean and both had stuff in S3 as well.
So, it's not an either or.
One of the things I thought was interesting was the operating systems.
I mean, like you said, Windows tends to get, you know, people have something in their heart against it or whatnot.
But I was surprised that Mac was so low on this.
Yeah.
Interesting.
I wouldn't have thought that at all.
But you go to the, I'm telling you, Dark Matter developers, that's what it is.
Yeah.
It's interesting.
I think the story on Windows is going to get better.
I believe the new version of Python is going to use MS Build and not Visual Studio 2008 for its compilation stuff during install,
which means, like, modern versions of Windows will be able to install stuff without, like, installing a 2008 version of Visual Studio, which will be real nice.
All right.
So, before we get to the next one, I want to just tell you both a little bit about Datadog.
So, speaking of stuff hosted in the clouds and spanning multiple machines and things like that, Datadog is a monitoring solution that provides visibility and tracks down issues with distributed systems involving Python applications.
So, within just a few minutes, you can find bottlenecks in your code by exploring graphs and rich dashboards.
And you can visualize your whole performance across all of your apps, which, when you're doing distributed programming or distributed apps, microservice type things, that's a huge deal.
And you can go to pythonbytes.fm/datadog, do a quick little trial there, and you'll get a free Datadog t-shirt, which is pretty cute.
So, check them out and let them know you appreciate them supporting the show.
All right, Brian.
I'm a big fan of databases, especially shiny new ones.
You've got a really new one yet.
Like, I can't even get this one yet, but it's still pretty cool.
Yeah, you can't get it.
But one of our listeners, Arash, I think that's his name.
Anyway, he let us know about EdgeDB, and EdgeDB has a blog post up.
It says, EdgeDB, a new beginning.
And at first, I thought, yeah, okay, we'll keep an eye on this, and maybe we'll cover it later when we can actually play with it.
Because it's a new database that's not available to use yet.
However, it's going to be open source.
And the reason why I brought it up now is because it's coming from some fairly interesting people.
It has some pretty powerful Python origins, right?
Yeah.
Well, so, like, for instance, Elvis and Yuri, and I'm not going to try to pronounce their last names, or they will flame me.
Part of this, and they're the people that brought us AsyncIO and UVloop.
So, that's pretty impressive.
That's very impressive.
One of the things that's interesting is looking at the kind of code that you get with this.
So, they're trying to attack the problem they're addressing is that document databases have some issues with just scalability.
After your project gets larger, the schemaless part of it sometimes can be hard to deal with.
A lot of people deal with it fine, but they see it as part of a problem.
And relational databases are growing a lot, and Postgres, for instance, is keeping up to date.
But the interface, how you interact with the database, the schemas and the underlying API to the database, hasn't changed much in a long time.
So, they're trying to change that.
And I forget what they call it, an object relational?
They call it an object database.
Object relational database.
Yeah, yeah, yeah.
Yeah, not like the traditional ones, they say, from the 80s.
Yeah, so one of the things to look at, if you're going to look at anything, is to go to the link and look at the example.
They have a new query language called EdgeQL.
So, they have a different way to write a schema that's fairly, doesn't really look like Python, but it's type-based, and it's fairly expressive.
It's pretty interesting.
So, instead of saying, like, let's have a class and map that to the database, like, say, SQLAlchemy or MongoEngine might,
they said, we're going to define our own data definition language, our own DSL, but it's really incredibly simple.
Like, doing the relationships with cascading this and that in SQLAlchemy, I always get that wrong.
You know, I have to look it up.
And this is like, you want a foreign key relationship, you say link assignees, goes to a user definition, and the cardinality is double star,
so I'm guessing multiple, many-to-many sort of thing.
And that's on, like, incredibly short there.
So, my first impression was, like, really a new schema definition language, a new query language.
Like, seriously, like, it's just like, okay, well, I'm tired of SQL, and I'm tired of the other ways of programming.
So, we're going to invent, like, another thing that people are going to get tired of.
But it's starting to grow on me.
Matt, what do you think about this?
Whenever you say, I'm going to invent something that's going to replace SQL, I think you hear a million developers cringe because they all know SQL, right?
But I think if you can get the five-minute out-of-the-box presentation where it's like, this is a compelling reason to use it.
And, you know, everyone, or at least most Python people I know, want to use, like, an ORM and interact with the database that way.
But there is this impedance mismatch with those.
So, if you can nail that down and have, like, a really smooth five-minute out-of-the-box experience with this, I think you could get a lot of people interested in them.
Yeah, it's pretty interesting.
I'm glad you brought it, Brian.
Thanks.
All right.
Matt, so you're a fan of the Wizard of the Oz is what I'm to draw from this next one.
Yeah, yeah.
Follow the yellow brick road.
So, I've been, I do corporate training and I do consulting.
And one of the things that I do when I'm doing data stuff is visualization.
Visualization is pretty important.
I mean, I've literally found bugs by visualizing something that we couldn't have found just by looking at the data necessarily.
And so, visualization is also important in the evaluation of machine learning projects.
And one of the projects that I've been liking and using recently is a project called Yellow Brick.
So, I guess this will take you to the Wizard of Oz if you follow it.
It's not a new project necessarily, but it's a project that's alive and going and being worked on still.
And what it does is it offers visualizations for various machine learning algorithms.
So, if you use a tool like scikit-learn, you can go to their website and they'll have all these visualizations up there.
But those aren't included in the library for scikit-learn.
You either need to...
You've got to go create them yourself, right?
Yeah.
You've got to either copy and paste their code or go find some Stack Overflow.
So, what I've been doing...
I mean, I have a project on my GitHub, MLViz, that I just have my own...
Here's the visualizations that I commonly use.
And then I use my little library.
But I'm looking to replace it with this.
And I've been using this for some of my training as well recently.
So, it's got visualizations for classification, regression, clustering, and text.
One of the cool things about it is if you're familiar with SK-learn or scikit-learn is that it has a similar API to that.
So, there's a fit.
You can fit your visualization.
You can transform it.
And then you call this method called POOF.
And that will pull up a map plotlet for your POOF.
That's the magic method that they have.
How do you spell POOF?
P-U-F-F?
P-O-O-F.
POOF.
Yeah, POOF.
Gotcha.
Yeah.
Perfect.
Love it.
So, just a nice little library to, you know, one of those things that can be annoying or that you always go and copy and paste that code.
And if you can just pip install this and use it.
And it has a great interface.
It makes your life a little bit easier.
Yeah, absolutely.
So, Brian, the next one, the last one that I want to cover, comes from the whole Alexa thing.
We've had a couple of people write us about interesting things with, like, say, Flask Ask and Alexa skills, right?
Yeah.
So, this one is a little bit of a serious one or at least addressing a serious problem, right?
It's not like putting mustaches on cats, but, like, it's actually trying to solve a problem.
Although, that would be a hard thing to do audibly on Alexa.
Nonetheless, so, this one.
This one is called Depression AI.
And it's an Alexa skill.
I apologize.
Everybody's little device is probably going off.
It's a Amazon device skill for people who are suffering with depression.
And it's open source.
It's based on Flask Ask, which I covered pretty deeply on episode 146 of Talk Python.
So, that's basically a way to use Flask to write these Amazon voice assistant skills, which is pretty cool.
So, the idea is that if you are suffering from depression, one of the things that's really hard for people, apparently, who are suffering from depression is to sort of go about your normal daily routine, right?
Get up.
Make your bed.
Take a shower.
It's, like, easy to just sort of, like, stay sprawled out on the couch or the bed or whatever.
And so, it sort of helps to encourage you to keep doing those things.
And it's supposed to be able to detect your moods and kind of give you some feedback.
What do you think?
I think that's super impressive.
I mean, I have relatives who have dealt with these sorts of issues.
And I don't know that they're necessarily ones who would take to technology.
But anytime you can get some help or get, you know, some feedback or someone other, you know, you're not listening to yourself, it can be a good thing.
I think this is awesome to have.
I think there's a lot of people that would aren't somebody that wants to go talk to somebody else, but having making the decision to put this in place when they're feeling good and then have it help them through the hard times.
This would be great.
Yeah, it's pretty cool.
It won the Valley Hackathon, which I think is, I think that's in Modesto, sort of outside San Francisco.
But this was apparently built like, what is that, a weekend or something, which is also a pretty, pretty big testament.
So you can do things like it'll evaluate your mood.
It actually has suicidal intervention.
It has location-based recommendations and mostly helps you with small activities.
So you can say things like, Alexa, check on me, or I feel down, or help me feel better.
I haven't gotten out of bed today.
It'll ask you things like, have you gotten out of bed yet?
Things like that.
So it's pretty cool.
Yeah, that's also open source.
Yeah, it's open source and on GitHub and based on Python.
So if this is inspiring, even if it's a totally different subject area, take it and use it as an example.
Nice.
Well, that's it for our official news.
Brian, you got anything you want to share with the world while we're here?
I've got some good news and some bad news.
So the good news is I went to an estate sale the other day and I bought a book called How to Be Interesting in 10 Simple Steps.
So that's the good news.
The bad news is I'm a really slow reader, so it might take a while to take effect.
Step two.
No, I've just skimmed it so far, so I haven't even started yet.
Well, what's a book?
Yeah, very quick.
What's your interest?
It was printed a long time ago before we had e-books.
Is it one of those things on paper?
It's like a tablet, but it doesn't run out of batteries?
Is that what you're telling me?
I've got another book author harassing me about physical books.
How about you, Matt?
You got any books lined up?
I'm working on revamping my Pandas one.
So big demand for Pandas and I want to update mine to the latest version.
So it's .17 and that's a couple years out.
Sure, very cool.
Oh yeah, and we also have some news about maybe a course coming out from you.
We'll leave that as a teaser, but I think a video course maybe in the near future.
Yeah, yeah.
Maybe.
I don't know.
We'll have to see.
We'll have to see if we can get our act together.
Awesome.
All right.
Well, Matt, thank you for joining us and dropping in on this podcast.
And Brian, thank you as always.
Yeah, thanks.
My pleasure.
Thank you.
Yep.
Bye, guys.
Bye.
Thank you for listening to Python Bytes.
Follow the show on Twitter via at Python Bytes.
That's Python Bytes as in B-Y-T-E-S.
And get the full show notes at pythonbytes.fm.
If you have a news item you want featured, just visit pythonbytes.fm and send it our way.
We're always on the lookout for sharing something cool.
On behalf of myself and Brian Okken, this is Michael Kennedy.
Thank you for listening and sharing this podcast with your friends and colleagues.