Brought to you by Michael and Brian - take a Talk Python course or get Brian's pytest book


« Return to show page

Transcript for Episode #100:
The big 100 with special guests

Recorded on Thursday, Oct 11, 2018.

00:00 KENNEDY: Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is Episode 100, very exciting, very exciting. Episode 100, recorded October 10th, 2018, I'm Michael Kennedy.

00:14 OKKEN: And I'm Brian Okken.

00:14 KENNEDY: Hey Brian. It's not just us this time, is it?

00:16 OKKEN: No, we've got some wonderful guests.

00:19 KENNEDY: Yeah, so I want to welcome Anthony Shaw, Dan Bader, Brett Cannon, and Nina Zakharenko to the show, all of whom have been on the show before, but have come here to help us celebrate. Hello everyone.

00:31 SPECIAL GUESTS: Hey and congratulations. Hi.

00:33 KENNEDY: Hi, thank you. Thank you Dan, it's great to have you all here to celebrate this big episode and go a little bit deeper on some of the topics, it's going to be a lot of fun. So before we get to the first topic, really quickly I want to say thank you to DigitalOcean who helped us reach this milestone. Check them out at pythonbytes.fm/digitalocean. Get a $100 credit for new users. Brian, you want to kick off our first item with something poetic?

00:56 OKKEN: Yeah, definitely. Poetry is a Python project that a lot of people have, since we've talked about packaging quite a few times, a lot of people have said, hey, you should check out Poetry. And I just finally got a chance to, I've been taking a look at it this week and it's kind of cool, but it's a big, this topic is actually kind of a can of worms. There's, essentially Poetry on its tagline, it says Poetry is a tool to handle dependency installation as well as building and packaging of Python packages. And it all does it with one file, the pyproject.toml file, so that replaces the setup.py, the requirements.txt, setup.cfg, MANIFESTs and even pipfile, and that's where the can of worms comes in, so this is sort of like pipenv, but with a completely different file structure behind it. It does do virtual environment support as well, so it does a lot. Not sure, I was surprised that you can build packages and put 'em up on PyPI without a setup.py file anymore, those aren't required, I don't think. And I was surprised that, so these are also related to PEP 517 and 518, and surprisingly enough, I was surprised to find that Brett Cannon was part of those as well. So I'd like to get Brett or somebody else's opinion, or information about this.

02:28 SPECIAL GUESTS: Yeah, so the PEPs that Brian's talking about, PEP 517 and 518, 518 actually got finished first and that's what's defined the pyproject.toml file, which is originally written to provide a structured file for information required to build a package, so something you would upload to PyPI, it was not originally intended for projects, so not like your website or web app or something like that. And then subsequently PEP 517 standardized what could go in the pyproject.toml to specify how to build your package. So this is why Brian you don't need setup.py anymore, these two PEPs basically supersede the deed for that, by standardizing, because setup.py has a problem of being executable code and being fairly tied to setuptools. This completely makes it independent which lets you use projects like, such as Flit or Poetry to actually build your libraries.

03:23 KENNEDY: That's really interesting because when you do a pip install, Brett, you are basically executing the setup.py, which like you said, it could just be, you know here are my dependencies, please register 'em, or it could be you know, install this horrible thing and format my hard drive, right. So now we have a safe way to pip install a thing, do you foresee a thing marking a package as having no executable code on install?

03:48 SPECIAL GUESTS: Well so, you actually already have it any time you install a wheel. So wheels are specifically designed that it's essentially a unzipping and copying some files around, there is no actual code executed as part of the build. So only sdists source distributions are the only time you ever actually are executing code like that. And you are right, there is a potential security risk. Honestly though, the really big benefit of moving away from executable code and setup.py to something like a pyproject.toml is it allows for better introspection, because before this, if you wanted to take a package, sdist file for instance, and figure out what dependencies it had, if you had the wheel, that's in a metadata file, that's no big deal, but if you only have the setup.py, you basically had to shim out setuptools, or run setuptools and not have it run its install step to find out what's dependencies for programmatically, but with pyproject.toml, and this sort of specification that allows you to get away from that.

04:41 KENNEDY: Interesting.

04:42 OKKEN: And one of the things I wanted to mention was just that if people are considering whether to use pipenv in that route or Poetry, they're just, they're different workflows. I would recommend people just try both of them out and see which workflow fits better for you. As far as I can tell from Brett's comments offline, is that both of these PEPs are provisional but they're probably not going to go away, is it safe for me to convert my project to, away from setup.py and use pyproject.toml now, or is there a risk that that'll go away?

05:14 SPECIAL GUESTS: No, there's absolutely no risk it's going to go away. This is the future of packaging at Python, to the point that we've actually, I actually updated PEP 518 maybe two months ago to take away the specification that it was meant for libraries and just to say that it's a configuration problem, because actually when Poetry started using pyproject.toml, it kind of did it against the recommendation of the PEP, because the PEP originally said it was designed for libraries and packages, not for applications. They started to use it that way and other people such as Black, which I believe we'll maybe touch on later, also started to integrate with it and such and various other projects have, so we actually took away that wording, but the provisional basically just means we reserve the right to maybe tweak some of the wording or requirements, but this is very squarely the future of Python packaging, so they're definitely not going anywhere.

06:02 OKKEN: Okay, wonderful.

06:02 KENNEDY: Yeah, very cool.

06:04 SPECIAL GUESTS: Does that mean that the person installing the package needs to have a specific version of pip? Like to support, 'cause setup.py's cool 'cause it can work with like a really, really old version of pip, which is pretty common, so the new TOML file, does it need the users to actually have a newer version of pip to work? Yes, only though, once again, if you're installing it from a source distribution, which is once again why wheels are so important, that if you had a wheel, all of this is completely moot. I mean basically this is just how to build a wheel, and even the way pip's work flow now is structured, is pip basically takes that estest, builds a wheel, and then installs that wheel, and that's actually what PEP 517 specifies is an API to call into Python code, of how to ask a build tool such as Flit for instance or Poetry, hey, can you build me a wheel, okay now that you have a wheel, can you help me install a projector or what have you. So it is only in the newer pips, I believe it was pip either 10 or 18, 'cause they changed their numbering scheme recently, they added the score to it, so it's only within like the last two versions that they've had it, but once again, if you have wheels, it's completely unnecessary technical detail.

07:17 KENNEDY: Dan, you were trying to jump in there.

07:18 SPECIAL GUESTS: Yeah, I've never really heard before of the TOML format, and even the name is hilarious, I think isn't this like Tom's format or something like that, has like the name of the inventor in there somewhere. This is curious if this is something, I know established, for me it's sort of a like a out of the left field kind of format and I was just wondering you know, if this is something, what makes it so awesome by comparison to JSON or YAML, whatever the other options were, I guess a plain INI file or something like that? I will say that there's a very long section in the PEP explaining all of the reasons we did this. So we actually... I was going to ask, sorry, have you seen a TOML file? I just started using Black more heavily pretty much everywhere, and it required me to set up a TOML file, and I was like, yeah okay, this is pretty nice, pretty straightforward, feels like an INI file, it seems like it's a little bit different, so I was just curious. Yeah, I mean compared to JSON with all the extra curly brackets or even YAML gets pretty messy, I think it's a really nice format for configuration. Yeah, and one of the reasons we chose it is Rust standardized on TOML for all their packaging details. So we actually reached out to them before we did this and said, hey, we've noticed you've been using TOML, what some people call the only mark up language to take the creator's name out of it. They said they had no regrets, they really liked using it and it was basically readable without any ambiguous corner cases and that was basically it, really. It was as Nina said, JSONs can be really messy to read, if you don't format it well, YAML is decent but there's also some execution problems if you're not careful. INI files are under-specified and really not specified in any way other than what the module and the std library happens to say in INI files, at least in the Python community, and then TOMLs just happen to be more well established and specified, so.

09:15 KENNEDY: I for one am excited about a new text format, that doesn't let you execute Python code just by putting excpylamation marks or something in it.

09:24 SPECIAL GUESTS: Safe load, safe load all the way.

09:26 KENNEDY: Yeah, that's exactly.

09:26 SPECIAL GUESTS: Or load safe, I always forget.

09:28 KENNEDY: Nina, do you use pipenv, or Poetry, or are you just a straight pip and virtual environments person?

09:37 SPECIAL GUESTS: I actually have not heard about Poetry before today so I'll definitely be checking it out, but I use a mix of requirements.txt for simpler projects, and pipenv when things get more complex.

09:48 KENNEDY: Yeah, cool, I still haven't gotten my mind around pipenv, so I'm starting to wonder if maybe I should just learn Poetry instead, so I'm pretty excited about this actually.

09:55 SPECIAL GUESTS: There's a pipenv integration in VS Code and it's pretty nice, it takes a lot of the thinking out of it.

10:01 KENNEDY: Oh that's awesome. Yeah, very cool. Anthony, so you found a way to relate lpylamas to Python, is this correct?

10:08 SPECIAL GUESTS: Yeah, so I've been looking into tools for measuring code complexity, so I guess the idea is that less is more, so as an application grows or as a code base grows over time, you end up adding all these edge cases and unique customer requirements and stuff like that. The code gets, can get more and more unmaintainable or complicated, so there's a few ways of measuring the complexity of your code, and one tool that I came across was called Radon. It's a Python tool that leverages the AST. I guess the thing which is built in to Python which turns your code into a tree that it then executes, and one of the stats is called cyclomatic complexity, which is not unique to Python, it's used in other languages as well, but it kind of measures the number of decisions within your codebase by iterating through the tree, which is really cool.

11:11 KENNEDY: That's cool, one way I like to think of cyclomatic complexity is like, if I'm going to test this code, what is the minimum number of units I have to have to actually test all the parts. It's not exactly true, but it's sort of true, 'cause you've got to go down each path, right?

11:24 SPECIAL GUESTS: Yeah, the way I always think of it is when you go driving somewhere, how you always try and pick the route which requires the fewest number of left or right turns. So just going, even if it takes longer, it's an easier drive if you can just pick one road and stick to it, rather than taking all these back roads and having to remember all the paths and stuff, so that's how I kind of think of cyclomatic complexity. The other one that they've got is the maintainability index, which is a combination of the number of lines of code that you have and also something called the Halstead, which is the number of operations in your AST. So basically Radon is a tool that you can run over your codebase and it will tell you how complex the code is or a part of your code, by adding a special visitor to the AST, which is really cool.

12:12 KENNEDY: That's really cool, so I've got no experience with Radon, I know it as a gas that like leaks out of the ground that can be radioactive, but that doesn't have anything to do with lpylamas. What's the pylama?

12:22 SPECIAL GUESTS: The pylama, so the pylama is another tool that I found, which brings together Radon and a whole bunch of other tools used for looking at the quality of your code, in air quotes. So pycodestyle, pyflakes. It also includes a gjslint, which is a fork of jslint, which is a JavaScript linter. I'm not sure why it includes the JavaScript linter, I think the idea is that you run it over Django and other projects where you've got sort of nested JavaScripts in your code and it can basically go and give you linting in your web applications, on not just the Python, but on the JavaScript as well. It also includes another tool called McCabe, which is a project that Ned Batchelder put together, which is another way of measuring complexity by looking at the number of branches. So, that's pretty cool. Yeah, my sort of final goal was to actually write a pytest plugin that kind of benchmarks the complexity of your code in order to pass the tests, and then the idea was that you could make the code, the tests fail if someone has basically made the behavior the same but the code more complicated.

13:34 KENNEDY: That's really cool, I love it, so like if cyclomatic complexity of any function gets above 10, just boom, fail the build, that's what you're thinking?

13:44 OKKEN: More than it was before.

13:45 SPECIAL GUESTS: If I need to plug us into a pre-commit hook where you just can't even commit your code.

13:52 KENNEDY: We reject it, it's too complicated.

13:55 SPECIAL GUESTS: So have you run this on one of your existing code bases? I have done previous tours and some other code bases. It's kind of, yeah, how about you, Nina? Oh, throwing it back. I have not. Kind of just spews out a number, and you're like okay, that's interesting, is that good or bad, so it doesn't really mean anything unless you look at it historically, is what I've found. And I run a similar tool like this on .NET code bases and Java code bases, which can get extremely complicated, especially 'cause like every feature that ends up getting added is basically just adding another if statement somewhere to deal with some weird edge case.

14:38 KENNEDY: Yeah, well this is a really interesting one, I would like to propose another measure of complexity, as the number of types involved in any particular function as well, but yeah, it's cool. Brett, have you got any experience with any of this?

14:51 SPECIAL GUESTS: Not beyond the fact that pylama is supported by the Python extension for VS Code.

14:55 KENNEDY: Ooh, nice, yeah, so it builds right into the editor there. That's cool. Nice. So Nina, the topic that you want to cover is around teaching Python, which I think everybody on this call, in some way or another, is pretty actively involved in that, so tell us about your thing.

15:12 SPECIAL GUESTS: Yeah, I'm going to be teaching a two-day workshop in spring of next year, and I was having a hard time kind of deciding on what tool I wanted to use and what workflow I wanted to use, because when someone's just getting started with Python, it comes with a lot of hurdles, like the virtual environments, installing Python3, explaining why the Python that comes with your system isn't good enough, and pip, and working with the command line and all this stuff. So I put out a call on Twitter asking what software and tools people use to teach Python. I will link to that in the show notes, but there were about 50 responses, 414 votes, and I learned about a lot of new tools from that thread. So of the 414 votes, 27% said they use Python or iPython in a REPL, another 13% use the built in IDLE, which I feel like a lot of people don't even know about. There is a tool, it's janky, and it's baked into Python, 39% said they use an IDE or some other editor of Visual Studio Code, PyCharm, Atom, and then 21% used other, so a mix of local and hosted Jupyter notebooks and a handful of other responses. So I just want to cover a few tools that I learned about and a few tools that I got reminded of. The first is the Mu editor, have any of you used it?

16:35 OKKEN: Yeah, it's really cool.

16:36 SPECIAL GUESTS: Yeah, so it's just like a really simple Python editor, really great for those who are completely new to programming, it's just an editor and then really large buttons at the top with some common actions. It's got support for a bunch of educational platforms, like the adafruit Circuit Playground, the micro:bit PyGame. There's a lot of really awesome tutorials. On the other side of the complexity scale, a few people clued me unto the Neuron plugin for VS Code and the Hydrogen plugin for Atom. It kind of makes this really cool, interactive coding environment in your editor, so little bits of Jupyter notebooks, you can like interactively run commands and see the output, you can have interactive charts and graphs displayed in your editor, import to and from Jupyter notebooks, so that one is geared a little bit more towards data scientists.

17:24 KENNEDY: I really like that one, I had never heard of Neuron. I've heard of Hydrogen for Atom, and I was excited about it, but I didn't do anything, 'cause I don't really use Atom, but I do use VS Code sometimes. I'm like oh, this is pretty cool, so it's like, it's sort of the general editing, the standard editing that you have in a regular text editor, but then Jupyter-like things pop out of it, right, like a graph or something.

17:45 SPECIAL GUESTS: Yeah, and like a lot of interactivity, it looks really good.

17:48 KENNEDY: Yeah it does, nice. What else?

17:50 SPECIAL GUESTS: Someone else told me about repl.it, repl.it. They have a project goal for zero effort setup, and they actually sent me a really interesting tweet that I liked, so I'll read it out to you. They said that we believe that the initial experience of programming should be the joy of writing and running code, and delaying the setup pain is a good way to hook people and retain them to want to install locally, because after all setup is merely accidental complexity that we accepted as reality.

18:21 KENNEDY: Yeah, very cool.

18:21 SPECIAL GUESTS: Open source hosted cloud REPL, the free tier is pretty reasonable, and you don't have to log in or do anything, you can go to this site and get started right away. There's three vertical panes, there's one for files, one's your editor, and the next is your repl, and then it's got some other really cool stuff built in, visual package installation, so you don't have to use pip, you can search in a little text box and just click install, which is really nice.

18:47 KENNEDY: Be a friend to users.

18:50 SPECIAL GUESTS: Right? It automatically generates a requirements.txt, like that's pretty cool, and then it also includes a debugger, which I don't know, I don't know how they make it work, but I love it. Yeah, and the last one is a bpython. So it's a different kind of command line, interactive REPL. I've used it years ago and I was surprised to hear that it was still an active project, but what bpython is is a fancy curses interface for the Python interpreter, so you can get little popup boxes in your terminal and a lot of, kind of really fancy UI elements, which is nice. It also supports pipenvs, expected parameter lists, and then you can do things like really easily reload imported modules. You can also rewind your session, and what that does is it pops the last line and then rewinds the entire session and reevaluates it, but those are really cool features that I haven't seen anywhere else.

19:50 KENNEDY: That's a super cool one. Do you have some honorable mentions as well, right?

19:53 SPECIAL GUESTS: I do, so I saw a talk at EuroPython, from Joshua Lowe, have any of you met him?

20:00 KENNEDY: I've not met him.

20:00 OKKEN: Yeah, we met him, at the last PyCon.

20:02 KENNEDY: Nice.

20:04 SPECIAL GUESTS: Okay, yeah, he is 14 years old, which is amazing, but he's a brilliant developer and he made this open source tool called Edublocks, which is kind of a Python version of Scratch. It's a tool for kids that let's you drag and drop code blocks and see it executed instantly, open source contribute to it, check out his website, I really love that project.

20:28 KENNEDY: Yeah, that's very cool, very cool. Alright, before we get to the next one, Dan. I want to just tell you all about our sponsor, DigitalOcean, thank you to DigitalOcean for making this show possible. Like I said previously, they've sponsored this entire show for the rest of the year, so thank you to them, that's great. One of the features I want to tell you about that they're promoting these days is to bring your own image to DigitalOcean. You've heard about bring your own device, well, in the cloud world, that's bring your own image. So you can go and create a virtual machine, some kind of Linux distribution, exactly like you want it, and then just upload that image, and then press a button and create new droplets based on that. So really cool, check them out at pythonbytes.fm/digitalocean, get a $100 credit for new users. Yeah, it's great, it's been working well for us, and definitely recommend them. So Dan, the one that you'd like to bring up has already got a little bit of a cameo earlier, right?

21:22 SPECIAL GUESTS: Yeah, I just love this tool, so I've become a huge fan of the Black code formatter by Lukasz Langa. Well, what's a code formatter? It's basically a tool you run on a Python source file and it reformats it according to a built-in code style. It's sort of like PEP 8, but PEP 8 isn't super comprehensive, it's ambiguous to a certain degree. Black just makes a decision for you and reformats your code. You know with this sort of tool, it's like, it's always like a love and hate relationship I think, because if you agree with the reformatting, then it can be great, right, it's awesome, you just write your code however you like and then you reformat it, boom, it looks consistent at least. But if you don't like it or you have other idiosyncrasies that you want to keep, then that doesn't work out so great, and so with Black I've found that it's actually the first tool that I have sort of developed this blind trust for. So I pretty much started using it for everything now, and I like the way it formats my code, it pretty much always figures out some formatting solution that I like, and it's just a really, really cool tool. I think it just came out in 2018, actually heard about it on PythonBytes earlier this year, and I think Brian you found it in Episode 73, get my notes here. And yeah, it's just an amazing tool. The nice thing is, you know besides all of the auto-formatting business, you can also use it to check the formatting of an existing project or just a single file. So where this is nice, and this kind of touches on some of the code quality tools that we had talked about earlier, is that you can integrate that with your continuous integration pipeline, and then make sure that any new code that gets committed is consistent with the formatting that Black generates. So if you have multiple developers working on a code base, this just takes away all of the formatting back and forth, you know it's so easy to get sucked into a conversation of like, oh, I'm reviewing your code here, actually I think this works pretty well, you know, your 5000 line change, it's awesome, but I would really change all of these, the single quote strings to double quotes, or something like that, it just gets rid of all of these non-productive conversations, and just makes sure everybody's uses the same formatting.

23:45 KENNEDY: I love it, yeah that's awesome. Another one of the problems you have in these teams is if you have different formatting rules and you can run, like format my code options, like VS Code or PyCharm or something, and you have different ones, they can fight in version control, right. Like this one changes it, this one changes it back, this one changes it, this one changes it back, so I can just hear Nina's voice say, that should be a pre-commit hook, and then everyone's is the same, right? What do you think, Nina? Should it be a pre-commit hook?

23:45 SPECIAL GUESTS: Oh yeah, I love pre-commit hooks, absolutely.

23:45 KENNEDY: That's awesome. Brett, I wanted to ask you what do you guys do on the CoreDev team for this problem?

23:45 SPECIAL GUESTS: Lots of hand work. It's all manual, basically we don't have any tools specifically to actually maintain this. More or less all the CoreDevs have just internalized PEP 8 almost as much as you possibly can, we can just spot code that doesn't quite meet it, even though PEP 8 is not as rigorously defined as it potentially could be. The idea has come up about actually adopting Black as an official formatter, but due to the current governance situation, that hasn't really gone anywhere, but I wouldn't be shocked if Lukasz, who is at CoreDev, brings that up in the future as a way to kind of make Black a more or less official formatter for the language somehow.

23:45 KENNEDY: Yeah, that's cool, especially Lukasz is actually, you know he created Black and he's a core developer, he's very active, it would be not unreasonable to say we just run Black against everything.

23:45 SPECIAL GUESTS: Yeah, it's just the trick of, can you get enough core developers to say Black's output is pretty instead of ugly, and you could probably pull that off, but obviously formatting, much like naming, is a very opinionated subject, and it's hard to get agreement on that.

23:45 KENNEDY: It's like asking everybody to use the same editor. Somebody will be happy, but you know, somebody's going to be really unhappy about that.

23:45 SPECIAL GUESTS: One really nice thing about these tools like Black is because they're external to specific editors fail to standardize them, does help allow people to use whatever tool or editor that they want.

23:45 KENNEDY: Yeah, absolutely, absolutely, very cool. Well, one of the themes of the last PyCon Brett was, one of the places where Python belongs but is not really is on the web, on the client's side of the web, right?

23:45 SPECIAL GUESTS: Yeah, Dan Callahan had a whole keynote kind of talking about Python and where it's been and where it could potentially go, and one of his key points was Python was not really on the web as I think some of us wish it could be.

23:45 KENNEDY: Yeah, but you have your next item, is about trying to make that a bit more of a reality, right?

23:45 SPECIAL GUESTS: Yeah, so at PyCon Australia, Russell Keith-Magee who heads up the BeeWare project gave a whole talk entitled, A Web Without JavaScript, where Russell basically points out that JavaScript more or less has a monopoly on client-side programming, which is always kind of a not great thing, because you never want a specific monopoly because it's highly restrictive. It makes innovation and growth hard, various other usual reasons you don't want it, so Russell gave a whole talk about the various approaches that currently exist for trying to get Python into the browser. And he did it by using an example all the way through his talk where he implemented what's called the, I think you pronounce that, the Luhn Algorithm, it's basically a way to check some credit card numbers.

23:45 KENNEDY: Right.

23:45 SPECIAL GUESTS: And when he implemented it, it was only like .4 kilobytes, so really small. Then he just subsequently wrote it in Python and ran through all the ways you can do it, and it ranged from 32 kilobytes per transcript, which transpiles Python into JavaScript, to Brython, which is a Python compiler written in JavaScript, which had .5 kilobytes for source 'cause it's just Python, but it had a 646 kilobyte bootstrap. Batavia which is Russell's project that basically implements the eval loop for Python, which had a 1.2 kilobyte pycode size, 'cause it literally executes by the pycode, but it has a five megabyte bootstrap. Then Pyodide, which is the CPython compiled down to WASM, which is web assembly, which is kind of being viewed by some as the future way of handling this kind of a problem.

23:45 KENNEDY: I'm very excited about Pyodide.

23:45 SPECIAL GUESTS: Yeah, well it's a really cool idea, unfortunately while the code was still just .5 kilobytes for the Python code, the bootstrap's three megabytes, so it's still kind of large, but it is a nice way to short circuit getting going, because if you can just take CPython and just literally compile it straight to web assembly, there's no reimplementation of anything, right. Like Batavia and Brython and Transcript are all big projects that are having to basically reimplement Python in one way or the other, while Pyodide just said like, well let's just take CPython and just make it work.

23:45 KENNEDY: Right, compile it to WASM and then ship it.

23:45 SPECIAL GUESTS: Exactly. Which I think is why some people at least are hopeful WASM can somehow do this, whether that's writing a Python compiler straight to WASM, so we can maybe ditch the bootstrap situation or what, I don't know. The other thing for me is my interest in this is as a VS Code extension developer, 'cause that's all on TypeScript, so this and VS Code's an Electron app, so for situations where download size isn't quite as critical, something like Pyodide actually starts to become potentially feasible,

23:45 OKKEN: because shipping three megs as part of a hundreds of megabyte app, it's not a big deal.

23:45 KENNEDY: Yeah, that's where you're right, that's where it gets super exciting, because these things were, like you say, where you say oh that's three megs or that's five megs of JavaScript, that's way too much, that's true when you're getting it off the web per request, at least per session, but if it's like you said, shipped and it just runs in the Chrome that powers the whole Electron system, well then it's no big deal, right?

23:45 SPECIAL GUESTS: Exactly, and then at that point, it's a question of, which one gives the best Python to JavaScript bridge and vice versa. They all have different levels for that, based on how they're implemented. But for me, I think there's an opportunity here to not only come up with a solution that has good bridging that can work in a Electron or Node situation to get Python to there, but then also to potentially look at web assembly and see if there's a way to do like a Python compiler straight to web assembly, and then have that be the way we target Python client-side and try to keep that number down.

23:45 KENNEDY: Yeah, that's super cool, have you heard about Python Electron, Electron Python, can't remember which way it goes?

23:45 SPECIAL GUESTS: Yes, I have passively seen it, 'cause I asked on Twitter a couple times, like I really want this, like I want to, I want to write the Python extension VS Code in Python some day. Eric Snow, who's at CoreDev and also a teammate of mine, we occasionally bounce this idea back and forth, and it sometimes leaks onto Twitter. Blue skies, and yeah, and people have brought up Python Electron before.

23:45 KENNEDY: Very cool, how about the rest of you folks, have you heard about Pyodide or people doing interesting things with some of these others, what are your thoughts?

23:45 SPECIAL GUESTS: We recently added to interactive coding exercises to real Python, so there's quizzes and parts of them are, you know, solve this, like implement this little loop or whatever, and that's actually implemented in Brython, and so it compiles your Python code to JavaScript locally on the client and it runs that, and it's actually been a pretty cool experience. It's not a 100% compatible, so it's sort of like Python 3.7-ish, and that's when you run into these funny edge cases, which is interesting, because when people are trying to learn Python, what are they really learning with these exercises, but for the most part you know, it's pretty accurate. And performance-wise, it works pretty well too, so on a fast connection, when that bootstrap doesn't hit you that hard, it works quite well on mobile too. The only thing I've found where it's really starting to just churn the CPU is when you load in a bunch of stuff from their standard library, so we have the option to validate your code using regexes, and when you import re, then it can take, I don't know, five seconds on even on a notebook, and that's not ideal, but I think for a use-case like that, where you have the user type in code, for us it works really quite well. I'm not sure if I would want to rewrite like all of the front end JavaScript just if that's part of the website, but for a code runner or code exercise tool, I think it works quite well.

23:45 KENNEDY: Yeah, that sounds like a really cool use, and you don't have to worry about the security, it's not like, I'm taking their code and running it and trying to do that in Docker or something like, just hope my server doesn't get messed up from this, we'll try it.

23:45 SPECIAL GUESTS: The only thing you can break is your browser.

23:45 KENNEDY: I've hacked myself, I'm a hacker. Awesome, how about the rest of you?

23:45 SPECIAL GUESTS: No, I've not checked it out, but there's a couple other links I'd recommend people look at, one is Scott Hanselman's blog post on JavaScript as the assembly of the web, which is really interesting and paints some good context. Also from Scott there's an interview on Hanselminutes podcast between him and Patricia Aas, and that kind of covers off like the history of JavaScript engines and run times and also helps kind of explain why things are the way they are, which is really interesting to give you context as well, so.

23:45 KENNEDY: Yeah, that's cool, and you know, speaking of Scott Hanselman, they're doing really interesting stuff with web assembly in the .NET CLR with Blazer. I mean, if the Python community could just do that same thing, and it looks like Pyodide is very close, but that's more focused on data science instead of SPAs, front-end JavaScript stuff, so there's definitely some interesting stuff happening there.

23:45 SPECIAL GUESTS: I also recommend if you haven't watched Dan Callahan's keynote from PyCon that you do, it's really good.

23:45 KENNEDY: Yeah, that's a great recommendation. Alright, web assembly, it's going to make things amazing, and I'm looking forward to it. Alright, so we're down to our last item, and this one is mine, so I'll kick it off. So you all have heard of Selenium, right, you can automate browsers and do web testing and stuff with it? Yeah, so, if I wanted to use that in an async and await an async method, as an async def I'm going to call Selenium thing, there is no async option for that, and they actually talked about it on GitHub, and decided they weren't going to do it because there's this other project, which I had never heard of, called Arsenic, Arsenic I guess, something like that. It's an async version of, well basically Selenium. So if you're writing any async code, code that you're interacting with a bunch of sites, you're testing them or something like this, you could do it in an asyncio event loop very easily with this other project, so it's quite cool.

23:45 OKKEN: I'm going to definitely have to check that out, that is interesting.

23:45 KENNEDY: Yeah, so I come in here and like, I hit create one of these sessions and I say, I'd like to use the Firefox browser to do this, and then I'm going to await getting a page, and then I could actually say like, await session.waitforelement, like h1, and so you can say, put your part of the event loop to sleep until the element, you know, h1 appears and you can get the title from it and stuff, it's really quite neat.

23:45 OKKEN: Hmm.

23:45 KENNEDY: Yeah. So, we say, you know, this is good for load testing, good for automated testing of websites, you know if you want to say like, well, we have this CMS and people can just type stuff into it, but let's make sure there's no broken links, right, something like that. It uses real web browsers, so it's really using Firefox in a hidden form to do this, so it's pretty cool, very nice for testing I think.

23:45 OKKEN: Yeah, headless browsers are awesome, and it has Pytest, it works with Pytest, that's cool.

23:45 KENNEDY: Yeah, it has the special Pytest support, which is cool. So I guess, you know, this is an interesting thing for people who want to automate the web and they're doing it asynchronously and often that's like one of the times where asyncio is really helpful, because you're just waiting on other servers anyway, so you can blaze through that. But maybe just throw it out to you all, what are your thoughts on async these days, async and await and all that stuff? Brett, maybe start with you, 'cause you are on the inside. And I know you've blogged on this.

23:45 SPECIAL GUESTS: Yeah, I was going to say which makes me obviously extremely biased on the topic. So, I think async's great. I appreciate personally how we designed it, because we haven't tied ourselves to a specific event loop, which has allowed some experimentation, such as Trio and Curio. While asyncio has continued to evolve and learn from them, like I know Yury Selivanov has pulled a lot of ideas from Nathaniel Smith of Trio into asyncio and so there's been a nice feedback loop on that little experimentation. So I'm very positive on it. I wrote an entire GitHub API Library based on async, because I thought it was so great, so.

23:45 KENNEDY: Yeah, that's right, and we've covered that in the show before, but I don't remember the number, I'm afraid.

23:45 SPECIAL GUESTS: Ah, I don't know, I think Dan brought it up.

23:45 KENNEDY: That's right, Dan did bring it up.

23:45 SPECIAL GUESTS: I don't remember the number either though, but man, so many episodes, 100 episodes.

23:45 KENNEDY: I'll tell you, it's less than 100, it's either 99 or below that, we can put a pound on it, how's this. Async, the rest of you guys?

23:45 SPECIAL GUESTS: No, nothing, I still find async in a way very confusing, and the attempts that I've had at using it, I've just ended up making a mess, and causing bugs that I don't understand and can't debug, so I just kind of gave up, which is pretty sad, but I am going to watch your course Mike, at some point.

23:45 KENNEDY: Thanks.

23:45 SPECIAL GUESTS: I need to allocate the time.

23:45 KENNEDY: There you go, you know, the whole threading world, there's a phrase for the bugs that are like race condition stuff, called heisenbugs, and I just love that terminology.

23:45 SPECIAL GUESTS: I've run into a few of those.

23:45 KENNEDY: Yeah, they're...

23:45 SPECIAL GUESTS: For those who don't know, a heisenbug only appears when you're not looking at it.

23:45 KENNEDY: That's right, you observe it, too little, too late. Yeah, very cool. Alright, well people out there, you need to do some sort of automation, and you were going to use Selenium, definitely check out Arsenic, because you can do it more parallel if you're doing more than one sort of flow through your tests. It does say be careful though, because while you can async or at least start calling functions, you're interacting with like a hidden real browser, which itself can't do more than one thing at a time, and so you know, it's more for I want to test a bunch of links, or a bunch of paths, not I'm trying to make this one sequence of events go faster, that probably won't work so well.

23:45 SPECIAL GUESTS: Yeah, check out Selenium-Grid, if you want to do that, if you want to do multiple machines and multiple browsers at the same time.

23:45 KENNEDY: Is that grid computing for Selenium?

23:45 SPECIAL GUESTS: Kind of, yeah.

23:45 KENNEDY: Oh man, that's pretty awesome. Alright, well let's leave it there for our main topics, so thank you everyone for bringing these and sharing them with everyone, they were super interesting. Normally, I just ask Brian is he has anything else to share, but I know we got more than that, so we'll just go down the list here, I'll throw mine out first. So hopefully this is not out too late to make this useful, but the PSF and JetBrains last year teamed up to do the most comprehensive survey of the Python community they could, and that is now happening again this year, so just visit talkpython.fm/survey2018, and take that, but if you hear this, do that as soon as possible, 'cause there's only a week or two left of it being open. So alright, Anthony, you've got one you'd like to share.

23:45 SPECIAL GUESTS: Yeah, 3.7.1 is around the corner, and a first release candidate came out a week ago, so if you're running on 3.7 and you want to try to see if any of the bugs that you may have come across have been fixed, take a look through the release notes. Yeah, and then the next one is that on the topic of Python packaging, there's a great article that I've linked to in the show notes, and talking about the whole landscape and the comparisons with Flit and Poetry and pipenv and the reasons behind them. It's a really in depth piece, and I think it's worth a good read.

23:45 KENNEDY: That's awesome, I definitely think some comparison and exploration is needed, 'cause there's just so many ways to start solving this problem these days, but I think something will emerge, and I'm kind of hopeful for Poetry actually. Nina, how about you?

23:45 SPECIAL GUESTS: I have a few things as well, we brought up VS Code a few times during the podcast, and there's a new September release of the Python extension for VS Code, and there are lots of new features, automatic environment activation in the terminals, some debugging improvements, and a lot more. So you can check out the blog post in the show notes for some more information. Brett, I don't know why you're being so shy. You should be the one bringing this up. Ah, that's fine, and then there's one more thing I wanted to mention, PyCascades is a really nice smaller Python conference that's happening February of 2019 in Seattle, and right now the call for proposals is open. It closes on October 21st, so you should still hopefully have a little bit of time, and they're doing something great that I haven't seen from too many conferences before, which is offering mentorship.

23:45 KENNEDY: Yeah, this is for like new speakers and stuff, who maybe want to speak, but they're like, ah, I've never done this, I'm a little unsure that this is actually a good idea, could really use some help, that type of thing? Yeah, that's awesome.

23:45 SPECIAL GUESTS: Yeah, it's really great, I've been volunteering as a mentor and it's been a really rewarding experience to kind of coach people through all their fears, because at the end of the day, we can all speak at a Python conference.

23:45 KENNEDY: Yeah, absolutely, that's awesome, I'm glad you brought those up. I've already booked my travel and submitted three talks, and I even put Brian's name on one, so he has to come now too.

23:45 SPECIAL GUESTS: Awesome-- I'll say last year was my favorite conference probably, potentially ever, was the inaugural PyCascades, so you should definitely go if you can make it.

23:45 KENNEDY: I agree, that was up in Vancouver, where you and Dan are, but this time it's moving down to Seattle, and then later comes to visit us here in Portland, so I think it's a great one as well, Brett.

23:45 SPECIAL GUESTS: That's right, 2020--

23:45 KENNEDY: That's right, it'll be here, I'll drive, won't have to stay at a hotel, it'll be great. Alright, the rest of you, anything else, Dan, Brian, you want to share anything before we hit the road?

23:45 OKKEN: Just wanted to say that upcoming, I've got a recording with a guy named Anthony Shaw.

23:45 KENNEDY: Oh that guy, yeah, I've heard he's good on a podcast.

23:45 OKKEN: Yeah, we're going to talk about flaky tests on testing codes, so that should be fun.

23:45 KENNEDY: Right on. Dan, anything?

23:45 SPECIAL GUESTS: Nah, I got nothing. I hope you invite me back for Episode 200.

23:45 KENNEDY: Yeah, you all already have an invite, we'll be there in just about two years, so you all hang in there, and we'll do this again. Thank you all for coming, this was really great, and thanks for help with the show throughout the last two years, and today of course.

23:45 SPECIAL GUESTS: Thanks for having us. Yeah, thanks for doing this. Yeah. And congrats, again.

23:45 KENNEDY: Thanks. Bye everyone, and Brian, special thanks to you, you've been here for all 100 of 'em, well maybe 99, 'cause he's bowed out of one or two, but that's all good.

23:45 SPECIAL GUESTS: Alright, thanks man. Bye everyone. Thanks, bye.

23:45 OKKEN: Bye.

23:45 SPECIAL GUESTS: Bye.

23:45 KENNEDY: Thank you for listening to PythonBytes. Follow the show on Twitter via @PythonBytes, that's PythonBytes as in b-y-t-e-s, and get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way. We're always on the lookout for sharing something cool. On behalf of myself and Brian Okken, this is Michael Kennedy, thank you for listening and sharing this podcast with your friends and colleagues.

Back to show page