Transcript #219: HTMX: Dynamic and live HTML without JavaScript

00:00 - Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds.

00:05 This is episode 219, recorded February 3rd, 2021.

00:10 I'm Brian Okken.

00:12 - I'm Michael Kennedy.

00:13 - And I'm Jennifer Stark.

00:14 Hiya.

00:15 - Yay.

00:16 - Yay, welcome Jennifer.

00:17 It's so great to have you here.

00:17 - Thank you.

00:18 It's really great to be here, thank you.

00:20 - Yeah, yeah, it's been great.

00:22 It's great to have you here.

00:23 You know, we had you, or I had you as a guest over on Talk Python, and that was really fun, talking about data science stuff over there, and now we're happy to have you here on Python Bytes as well.

00:35 So really quickly, you wanna just tell people about yourself before we jump into the topics?

00:39 - Sure, yep, so I'm Jennifer, I work at Lab Bible as a lead data engineer on a really small team of three, but we're a bigger data team for research and insights as well.

00:52 We've been spending most of our time working on engineering stuff, but we've been moving gradually to include more data science tasks as well.

01:01 So looking forward to doing some more of that.

01:03 - Yeah, it sounds really fun.

01:04 Brian, you wanna kick us off?

01:06 I mean, I heard that you're supposed to use virtual environments and not mess up what you're doing, but if you don't want to, I guess you just don't do that.

01:13 - Well, I use virtual environments all the time.

01:17 But there was an article, so I wanted to cover this, and there was some discussion online.

01:22 An article from Frost Ming titled, you don't really need a virtual environment.

01:28 So what's the story there?

01:30 >> Yeah.

01:30 >> Yeah. So there's a little hint in the slug of the URL, that the slug is introducing PDM.

01:38 So I don't know if he's really saying that you don't need it, but PDM stands for, what does it stand for?

01:46 Package Python Development Master.

01:49 Well, that's cool.

01:50 I think I want to put that on my business card.

01:53 I'm a Python Development Master.

01:54 Anyway, so let's just go back up a little bit.

01:59 This is a neat tool.

02:01 It's poetry-like, but it says it doesn't use virtual environments, it uses the Dunder package directory. What is that?

02:11 We do have this problem with virtual environments.

02:14 The main problem I think is it's hard to teach people.

02:18 If you're teaching somebody new and you want them to install something, you have to say, "Okay, well, type Python-M venv or Python-M venv, space venv, and then you have to activate it.

02:34 The Mac people do this, the Windows people do something else.

02:38 Then after you've activated it, then if we've got requirements, you need to install them or install the requirements.

02:45 That's not a fun way to start teaching people how to use Python.

02:49 I think we do need to solve that, but I'm not sure this is it.

02:53 But there's--

02:54 - Sure, well, like Node.js has a similar problem, but they don't necessarily have as much of a challenge 'cause they have this like directory structure, which I think what DunderPi package is trying to do, right?

03:03 Like if you have a node modules folder, some directory up and you do something with NPM, it just goes up till it finds one.

03:11 It's like, well, there's the top of the project, we're good.

03:13 - Yeah, so there's, that's kind of what, so there's a pep582, which we'll link to, that kind of has that, has this, it's proposing to have this Dunder packages, package directory and have that sort of thing.

03:28 So if you're in a directory with one of those around and you do a pip install, I think this is how it works, either it's supposed to, I think it's just gonna install stuff there instead of in your global one.

03:39 So I actually think this would be cool, even if it's only used for teaching, it would be a cool thing to have because also you could possibly zip this whole thing up and give somebody a directory and they'd already have all the packages and everything.

03:54 That'd be kind of--

03:55 - Interesting.

03:56 I wonder if that's kind of similar to how Conda, is that similar to how Conda works?

04:01 Can I use Conda ends instead?

04:03 - Yeah, so--

04:04 - Because to me, conceptually, it feels easier.

04:06 - Yeah, yeah, you probably live way more in the Conda world than the pip world, right?

04:10 - Yeah.

04:11 - I think Conda's sort of intermediate, right?

04:14 So with Conda, you do have to say Conda activate, right?

04:18 - Yeah. - Manually.

04:19 But you don't have to be in the right place.

04:22 Like with Pip, you literally have to say--

04:23 - You could be anywhere.

04:24 - Yeah, exactly.

04:24 You say, I have an environment called this, let's roll, like activate that and then go, right?

04:29 Whereas this is, it's like, I'm in the right location, but I don't wanna have to talk about environments.

04:34 And it just happens to be that that environment has a directory structure that Python will know about.

04:40 - Yeah. - Yeah.

04:41 So there's another part of virtual environments that's a little icky, is that, I guess, it's tied, you know, maybe, it's that if you create a virtual environment, you kind of tie it to a particular Python version.

04:55 - Yes.

04:56 - And if you update your Python version, then your virtual environments aren't pointing to the new one, and I don't really know, I don't know how to, actually, I don't deal with that.

05:05 I just delete the virtual environment and recreate it.

05:08 - I've gotten really good at doing that because every time I brew upgrade my Python for a major version, it just stops working.

05:14 I'm like, oh, come on, here we go.

05:16 You know, time to erase everything and start over.

05:19 - Yeah, so the PEP 582 might fix that also 'cause you could just, I don't know if it fixes that because it's still in the directory structure.

05:27 It does have Python versions in the directory naming thing.

05:31 So I think for minor upgrades it would work, but for major ones like going from 3.9 to 3.10, I don't think it would help you there.

05:39 I don't know.

05:40 Anyway, I don't know enough about 582 to comment on this, but I do think it's cool that PDM is around so that you can play with the Dunder packages to see what it's like.

05:51 However, the workflow within PDM is way more complicated than virtual environments, in my opinion.

05:57 So I don't think that it's gonna fix the newbie problem, but it's still an easy thing.

06:04 - That's what I feel about with all of these things is it's like, it solves two problems and it adds three years like, oh, come on.

06:11 Do I really want to trade those?

06:13 A couple of comments from the live stream.

06:16 Hi Lang says, "Conda rocks," mostly.

06:18 So right there with you, Jennifer.

06:20 Yeah.

06:21 - Love Conda.

06:22 - Yeah, yeah.

06:23 And then, Chen Danelli says, "There was a way to set up Conda things "so it automatically switches to the Conda environment.

06:31 "See the environment.yaml file." I don't know anything about this.

06:35 Have you seen that, Jennifer?

06:36 - I have not used that, no.

06:39 - It sounds like we should all check it out.

06:40 - That sounds really helpful.

06:41 Yeah, it sounds really good.

06:42 - Yeah, thank you, Daniel.

06:44 All right, well, I guess we should jump over to the next one.

06:46 Something else that's really helpful is cookie cutter, right?

06:51 So often we want to go and say, well, I want to create a project and I don't want to just start from file, new project.

06:55 I want to have a certain structure.

06:57 I want to maybe have some of the files there.

06:58 So for example, if I go and create a new pyramid web app, I can use a cookie cutter to generate that.

07:04 And it'll say things like what template language do you wanna use?

07:07 Do you wanna use SQLAlchemy?

07:08 And you answer a couple of questions and it generates project already integrating those things with the right directory structure and the right extra dependencies and whatnot.

07:16 And that's cool, right?

07:17 So I think cookie cutter is really taken over as the primary way of creating projects that are structured.

07:23 It's not just Python.

07:24 You could even create like Atari 2600 assembly language projects and C++, other weird stuff like that.

07:29 Anything that has to do with projects, just here's a bunch of files, replace some conditionally include others and so on.

07:36 That's what cookie cutter does.

07:37 And so that's not what I want to talk about.

07:39 What I want to talk about is this thing called copier.

07:41 Have you guys heard of copier?

07:42 - I have not.

07:43 I have used cookie cutter, but I've not heard of this one.

07:46 - Yeah, cookie cutter is cool.

07:47 And it's way more popular than copier.

07:49 Copier is pretty relatively unknown, but I think it's worth checking out.

07:53 I don't know that I'll replace what I'm doing with cookie cutter with copier.

07:57 They're not interchangeable.

07:58 They should be, that would be a great feature, but I don't think they can share each other's templates.

08:02 That said, the thing that is interesting about Copier primarily is that it allows you to upgrade working with projects.

08:10 So if I go and make a decision to create, say, some web application or whatever else application, it even works for data science, like structuring notebooks and whatnot.

08:19 If I make a decision and then I change my mind after I've already worked on it for a while, too bad.

08:24 You don't get any choice.

08:26 Like you throw it away or you create another one and you kind of diff the files.

08:29 You're like, "Oh, well, what's the difference over here?

08:31 Oh, I should include this thing.

08:32 But with copier, you can rerun it on the project and make changes and apply those changes and different choices to an existing project you're working on.

08:40 That's why I think it's interesting.

08:42 - That's cool.

08:42 Does it have like a prompt like thing also?

08:45 I mean, because cookie cutter asks you things.

08:47 - I believe it does.

08:48 Yeah, it will ask you questions.

08:50 If you look at it, it has, yeah, it absolutely has prompts.

08:55 I can't really see a great example here.

08:57 - Okay.

08:58 - It doesn't use, I believe, cookie cutters, like native Python that you program it in.

09:03 The scripts are Python, and then they drive arbitrary text files and whatnot with replacement, and it's kind of like Jinja.

09:10 This actually uses YAML.

09:12 So if you look at an example somewhere, I'm not sure exactly where a good example is, but basically you set up YAML files, and the YAML files have different types of prompts and questions.

09:24 You can say, "Here, I want to ask for a password," and then confirm it, but don't show the output.

09:28 So there's a lot of configurability and interesting things like that.

09:31 And then if you rerun it again, it'll say, here's the project structure that you have.

09:35 Here's the project structure that we're creating.

09:37 And if it runs into a file, it'll say, this one already exists.

09:40 Do you want to override it?

09:41 Use the one we're about to generate, things like that.

09:44 So it's pretty neat.

09:45 - I think that looks pretty cool.

09:47 I definitely want to check it out.

09:48 - Yeah, yeah.

09:49 It seems to solve a slightly different problem than cookie cutter, but it's, I think cookie cutter is the right conceptualization to have when you think about it.

09:56 - Yeah, I did start using, I created my own cookie cutter for some data science-y things that I was working on.

10:02 And there is a data science cookie cutter that exists, already templates exist, but it wasn't completely sitting in my needs, so I made my own.

10:10 And then I was gonna make one for projects in our team, 'cause we do some one-off data analysis for data science projects.

10:18 And then discovered that GitHub now has, you can make a repo as a template, and you can set it as a template in GitHub, and then you just clone it and name it something else.

10:26 So that's solved part of, doesn't solve everything.

10:30 You know, if you want something different, then, - Right.

10:33 - You know, contentual with this might be really good, but.

10:35 - Yeah, yeah, yeah, that's right.

10:36 So I remember if you go to your GitHub repository under settings, there's a checkbox that's off by default.

10:41 It says, this repository is a template.

10:44 That's what you're talking about, right?

10:45 - Yeah.

10:46 - I see.

10:47 - So if you set up like empty, you can set up your file structure.

10:51 So it's got nothing about, I guess, Some of the things that you're setting up in this one are not what you set up in the GitHub template.

10:57 It's just the file structure really.

10:59 And if any files, you know, that you want to pre-populate with any files.

11:03 But yeah, so you have, that solves what I was gonna solve with cookie cutter.

11:07 Cookie cutter would have been overkill for this.

11:08 - I see.

11:09 Yeah, I had never really thought of those two things as being the same, but you're right.

11:12 They're basically the same.

11:13 'Cause normally when you fork a repo, it's like, well, now you can contribute back, but the templates are just, now you start from here and it's not really related back, right?

11:22 - Yeah. - Nice.

11:23 - Well, yeah, that brings us to your first topic, right?

11:27 Tell us about it.

11:29 - Yes, yeah.

11:30 I was thinking of data science in our team.

11:34 We had a data science project that we started a couple of weeks ago, and it had a deadline, so we weren't gonna make anything particularly pretty.

11:42 We just wanted to get something analyzed and done.

11:46 So we were using lots of tooling that we hadn't used before 'cause we were using a massive data set.

11:52 I think it was a couple of gigs worth of text.

11:55 So we had to use Google's AI platform notebook, which is just Jupyter Notebook on Google Cloud.

12:03 But you can have different sizes of your machine.

12:05 You can have as many cores as you want, different types of machines if you want, and it will just run Notebook for you.

12:11 So we thought that would solve the problem.

12:12 We just have all these cores, and we run our Notebook on that, and it would be magical.

12:17 But it wasn't, and we tried to apply, to a Panda supply to this huge data frame, it just was not working at all.

12:26 We even had the process bar on the bottom, like under the cell, and it would take, I think it was like 10 minutes to do, and it was still on zero percent.

12:35 >> Wow.

12:36 >> We don't have time for this.

12:38 Yeah, don't have time for this.

12:40 We're already on a deadline and it's like, this isn't working.

12:43 Then we went over to terminal and just checked at the top to see what processes were going on.

12:50 And this was like one Python thing.

12:52 I thought, well, that's not, we could speed that up.

12:54 Let's see what we can do there.

12:56 - Even though you have a ton of cores and a lot of high-end machine, it's still just single threaded basically, right?

13:02 - Yes, it is.

13:04 So I looked at a few alternatives and didn't want to get too much into, I think some people were suggesting there's some desk-related modules we could use.

13:12 Like I think Swifter was one, but it didn't work instantly for me.

13:17 So I looked for something else and found-

13:20 (laughing)

13:22 - You have 30 seconds library, work for me.

13:23 - Yeah.

13:24 (laughing)

13:26 I can't figure it out, bin it, start again, find something else.

13:29 - Yeah.

13:30 - I tried pandarallel, which just parallelizes any pandas apply function.

13:37 So you can tell it how many cores you wanna use.

13:41 You might not wanna use all of them.

13:43 And it's not like a linear or exponential improvement, is it?

13:46 Like doubling your cause is not necessarily half your time.

13:50 There's some overhead.

13:51 So yeah, you can tell it how many cause you want to use.

13:54 You can also opt, I think if you scroll down, it says you can also add like a progress bar to it in the options.

14:00 - Nice.

14:01 - But it's, and then yeah, it got some benchmarking there as well.

14:04 And it's just really easy to use.

14:06 So that solved our problem.

14:08 Again, like the whole project was just quick and dirty, but to get it done quickly, this is great.

14:14 And then going over to terminal and doing top again, it's like, boom, all Python.

14:18 Just Python, Python, Python, Python, Python.

14:19 I was like, yeah. - Yeah, yeah.

14:20 Beautiful, and by default, if you don't, you can specify the number of workers, but if you don't, it's just the number of CPU cores.

14:27 - Yeah, all of them.

14:28 - So just a quick question.

14:29 It looks like this is, it says that it's mostly around the apply function.

14:34 What does apply do, as a reminder?

14:36 - So if you have a, if you specify a function somewhere, And then when you hit apply, I think there's some examples actually a bit further down of the kinds of applies that you do.

14:47 So you can have, where you'd normally put apply func, so you can apply a function to that whole directory, sorry, to that whole data frame or to a specific column within that data frame.

15:00 So any function you apply will be column, will be row-wise in that column.

15:04 - Oh, okay.

15:05 - So the function only has to work on a single row, essentially.

15:09 So anywhere where you'd put apply, if you're using parallel, you just, pandernally just replace apply with parallel apply, and then it will parallelize the function.

15:18 - It's very cool.

15:19 - Yeah, that's super cool.

15:21 Daniel asks, oops, not that one, this one.

15:24 He asks, how does this compare to Dask?

15:27 - I don't know.

15:28 (laughing)

15:30 - I have not used Dask a lot either, but I think Dask can be set up to run in parallel on a given machine.

15:36 - It can also be set up to run in a distributed cluster, basically.

15:43 - Yeah.

15:43 - My feeling is--

15:44 - It's to work on large datasets, isn't it?

15:46 - Yeah, my first impression is probably, like this is about, I've got to apply this function to every element.

15:52 I want that to be fast and simple.

15:53 Let's just do that.

15:55 - Yeah.

15:56 - That's my first thought.

15:57 - I think the other option I used, or looked at for 30 seconds, was Swifter.

16:02 And I think that is a Dask-based module, I think.

16:06 that I might be misremembering.

16:07 - And then somebody else said that apply is like map in base Python.

16:13 So. - Yeah, absolutely.

16:14 Yeah, very cool.

16:16 Brian, you know what else is cool before we get on the next thing?

16:18 - What is?

16:18 - If I wanted to learn pytest, say if I was Jennifer's team, maybe, I could get a book on pytest, right?

16:24 So this episode is brought to you by us.

16:27 - Yeah, so I highly recommend a book called "Python Testing with pytest." There's a small glitch with it though.

16:33 It was written in 2017.

16:34 So if you go to pytestbook.com, there's a bunch of errata that will help you.

16:41 There's just some minor changes.

16:43 I forgot to pin a version of one of the libraries, stuff like that.

16:47 So if you go to pytestbook.com, that page has some errata that helps you through learning pytest and with the book.

16:56 There is a second edition on its way, but it is a long way out, so don't wait for it.

17:00 - That's a lot of work, yeah.

17:04 Over at TalkBytes on Trading, we're working on a bunch of courses.

17:06 Breaking news, never mentioned this before, but we may be having a Dask course coming soon.

17:12 So just so you know.

17:13 And Damon also says, probably with more experience than definitely me, or Brian, that Dask has more features.

17:20 It can do chunking on the data frame to work around the RAM size limit, for example, and whatnot, which is pretty interesting.

17:25 And also notice down over here, how is it, this option, Use Memory FS, it will actually use this memory file system thing, which sounds like it's good for lots of data as well.

17:35 - Cool.

17:36 - But, I haven't been out much.

17:38 I used to love to go out and get a milkshake or something.

17:41 But you wanted me to use Ice Cream instead for Python?

17:45 - Yes.

17:46 - What's going on here?

17:46 - Yeah, so actually, kind of love this.

17:50 There's an article from Kuyen Tran, stop using print to debug in Python, use Ice Cream instead.

17:57 And I think we've covered a couple of others, like other print alternatives.

18:01 but I went in and tried this and it's pretty cool.

18:05 So with the new fprint stuff, you can, there's, I forget when it came in, but you can do an equal sign to print and it prints a nice like variable name or whatever you want and then the value of it next to it.

18:20 So it's nice, but it's still a lot of typing.

18:23 So if you want to print something nice, you type, it's a lot of typing.

18:28 It's not tons, but when you're throwing debugging in, you're probably stressed doing it quickly is good.

18:33 So Icecream is just a way to do this faster.

18:36 So Icecream is instead of typing print for your debugging output, you type IC.

18:42 - So first of all, fewer characters.

18:44 - Yeah, fewer characters, no curly brackets.

18:47 You don't have to do quotes.

18:49 It's just IC and then you give it whatever object or expression you want to print.

18:55 So that's cool, that's it.

18:57 So even just at that, it's worth it, it's less typing.

19:00 I mean, you do have to import it, but there's that.

19:03 But there's other stuff too.

19:04 So you can, if you don't give it any arguments, it logs, it's kind of like easy flow control debugging without having a debugger.

19:14 Because if it, every time it hits an IC statement without any arguments, it'll by default print like the file and function and line number where it's at.

19:25 So you can kind of trace through your stuff.

19:27 So that's nice.

19:29 If you want to have that tracing be part of all of your statements, even the ones where you pass something in, there's a way to configure that too, which is very nice.

19:40 - Oh, nice.

19:41 - You can custom prefix, which is like super powerful.

19:45 I'm totally gonna use it for this.

19:47 So the example that they have is instead of, of course you can just put a string in or something, but it's a callback.

19:55 So you have a callback function getting called.

19:57 So you can use that.

19:58 Their example is to inject the date time, which is kind of neat.

20:02 You can inject the date time in your print statement, but I was thinking you could use that to encode system state, like which users logged in or whatever other system state you kind of want to track while you're debugging something.

20:16 This would be really cool to do since you can use a callback function.

20:21 A couple other features with it, it doesn't go to standard out, it goes to standard error by default.

20:25 So it's not cluttering up your output if you're storing an output somewhere.

20:30 And then one of the other things, I'm glad they like this article lists this as a feature.

20:36 It's not a print statement.

20:38 So when you wanna clean out all your debugging, you can just search for all of your IC lines and clean those out.

20:44 You don't, because you might have print statements that are supposed to be there and you don't need to clean those out.

20:49 So definitely. - Yeah, you could even sort of cancel it out with an import statement.

20:52 Just define import, define IC to be a function that takes whatever and does nothing.

20:57 - Yeah.

20:58 - Yeah, this is really cool.

20:58 So people who are listening, you know, you say, if I were to say, I see of a function call like plus five and give it the number four, the output would be I see colon, then actually plus five with the argument values, colon the return value.

21:11 And so it's a really nice way, instead of just printing nine and 10, it prints, I called plus five with four and got nine, I called plus five with five and got 10.

21:19 And it just, it sort of summarizes the debug information a little bit better.

21:23 Jennifer, I think this might make sense inside of even a Jupyter notebook.

21:26 - Yeah, I think it was.

21:27 I was just thinking this is even less typing than if you use f-strings.

21:33 - Yeah. - Yeah.

21:34 - Yeah. - And a little more powerful.

21:35 And like Brian said, you kind of know it's intentionally a debug thing.

21:39 You could even rename, import IC as debug and just make it really clear if you really wanted, right?

21:45 - Oh, yeah.

21:46 - You've got three extra letters to type.

21:48 - Yeah, I know.

21:49 I know it.

21:50 (both laughing)

21:52 'Cause I'm not sure if I saw IC in my code without being familiar with this that I know what that's about.

21:56 - It's your code.

21:57 - It's not a pun.

21:58 - Yeah.

21:59 - Are they making a pun?

22:00 Like, I see, I see my error.

22:02 - Oh, I see, I see.

22:04 Yes, they may be.

22:06 They may be.

22:07 And Piling is a fan of the name, brilliant name.

22:10 Yeah, it's pretty clever.

22:11 All right, good one, Brian.

22:12 So the last two, by the way, the parallel pandarello had a great visualization.

22:19 This one has some great visuals.

22:20 And this next one I wanna talk about, you know, I think might be part of the reason we love these things.

22:24 Like they communicate their values so clearly.

22:26 This thing called HTMX, high power tools for HTML.

22:31 So Brian, I know you fall into this realm.

22:34 I do some of the time as well.

22:35 Jennifer, maybe, I'm not sure.

22:38 If you're gonna write some interactive webpages, you really wanna drop in and write a ton of JavaScript and do all that interaction by hand or would you rather just have it like magic its way into interactivity?

22:48 And so that's a little bit-- - I think it's the magic.

22:50 - Yeah, who doesn't want the magic, right?

22:53 So this is kind of what this is.

22:55 Like normally if you have a webpage, you have two options.

22:58 You can have a form and that form could like post back and submit some data.

23:02 And then you could write some JavaScript.

23:04 So if I click on some element, something happens.

23:07 But what this does is it lets you to go to almost any element in your page, a picture, air graph, whatever.

23:13 And you can say, if you interact with this, here is a CSS transition to run.

23:18 Here's a WebSocket call to do.

23:19 Here is a Ajax JavaScript call.

23:22 and then it does something in reverse.

23:24 So what I could say, for example, is when somebody clicks on this picture, replace it with whatever HTML fragment I get back on the server that I told it to call.

23:33 So the picture could be like, click this for this bit of data report, and then what it does is return actually the HTML for a graph that's like a live graph with the data pre-filled.

23:43 And all you have to do is touch the picture and teach the server how to return the HTML, and now you have this interactive page that's like live with animations and stuff.

23:52 Super cool.

23:53 So let me show you probably the best way to see this is through an example.

23:56 So for example, there's a button.

23:58 If you just include the script, that's all you gotta do.

24:00 And you say button, instead of having it in a form, you just say HX-POST, that's the HTMLX thing.

24:06 You give it a URL, and when you click it, it says call slash clicked.

24:10 And when it comes back, replace what button is, the outer HTML, like button and everything in it with whatever you got back from the server, okay?

24:18 And even has a haiku in here, which is pretty cool.

24:22 JavaScript fatigue, longing for a hypertext already in hand.

24:25 But let's go look at some examples.

24:28 These are cool.

24:29 For example, let's do lazy load.

24:31 If I go over to lazy load, which is a little slow, so it probably--

24:34 - It's lazy.

24:35 - It's quite lazy, it is indeed.

24:37 Maybe we will, here we go.

24:39 So we come over here and if you just scroll down, you can see like it automatically loaded in this by refreshing, see there's a little action, Boop, boop, boop, and then off it goes.

24:50 And all I gotta do is say is this image, it has this indicator.

24:54 Here's the image to show while you're loading.

24:56 And then what you wanna do is just show whatever you get from slash graph.

25:00 Isn't that slick?

25:00 And like, that's literally what you get, have to write.

25:03 Let me show you another one.

25:04 - I might do this just for the lazy loading.

25:06 That's great.

25:07 - I know, let's go do, yeah, look at the active search.

25:09 So over here I can type J-E.

25:13 Okay, there you go.

25:14 And just as you type, all you gotta do to like type in this little text box and have all these search results come up lazy is just say, here's what you call, HTTP post and the trigger is the key chain.

25:24 So there's even a delay.

25:25 So as you type really fast, it doesn't go insane on the server.

25:28 It like waits.

25:29 Really, really cool.

25:30 Yeah, as a little indicator.

25:31 And then if you notice at the bottom, there's this thing you can show and it shows all the requests and the responses that have gone back and forth.

25:38 There's like a little debug toolbar here for the whole AJAX interaction.

25:42 - Oh, sweet.

25:43 - Isn't that sweet?

25:44 - Yeah.

25:45 - It is nice.

25:46 How did you find this?

25:47 Gosh, I don't remember.

25:48 I feel like maybe somebody, some listener told us about it or I just ran across it on Twitter or something.

25:54 I feel like I found it from the community somewhere.

25:57 - Cool.

25:58 - But I remember where, but I'm excited about this.

26:00 Include a JavaScript file, put one line, and then it becomes this cool interactive thing all over the place.

26:06 So yeah, definitely digging it.

26:07 - Yeah, totally gonna use that.

26:09 - Yeah, for sure.

26:10 - Yeah, same.

26:11 - Nice, all right.

26:12 - It might encourage me to update my website.

26:16 Exactly.

26:17 You're like, oh, it's super interactive.

26:19 Look at all this.

26:19 I rewrote it completely.

26:21 So much fun.

26:21 Yeah.

26:22 Yeah.

26:22 Cool.

26:23 All right.

26:23 What's next?

26:24 Hi, LDA Viz.

26:26 Yes.

26:27 This is also part of that quick turnaround data science project that my team did a couple of weeks ago.

26:34 We were looking at doing some topic analysis on text and our first approach was to use latent derisolate.

26:45 Nobody knows how to say it.

26:46 Just gonna say it with confidence, Leighton Derisely, I can't remember what the A stands for.

26:50 Analysis, maybe that's the A.

26:55 He applied that, but to understand what you're looking at, you can have it print out what the topics are and what words are contained in that topic, but it's really hard to get into the output of your model to evaluate if it's a good model or not.

27:13 So what some wonderful people in the R community did was they made LDAVis, which just displays the different elements of the LDA output in a really, really intuitive way.

27:27 So even if you're not too sure on the math behind LDA and what everything means, what lambda is and what all the different complex interactions are, it's quite intuitive if you spend a bit of time exploring the visualization.

27:42 So that was then ported into Python, and in Python it's called PyLDavis.

27:50 But the visualization is exactly the same.

27:52 >> Nice.

27:53 >> Yeah. So you'd have, so in this little partial screenshot of visualization, we have some bars, and the blue bars and the red bars.

28:03 So the blue bars are like how much of the overall, you have all the words in all the topics, in all the documents, so baseball.

28:12 of all the words in all the documents, how much does baseball, how much does baseball represented in all those documents?

28:20 And the red bar represents how much in that topic, topic number 19, how much topic number 19 is made up of baseball.

28:29 - Oh, I see.

28:30 So you have these different topics on the left that you can like click on them and it'll generate the bars to explain more detail?

28:36 - Yes.

28:37 - Okay.

28:38 - You can click on all the different topics.

28:40 The number of topics is determined already in the model that you've already created and you can change that, rerun the model and get that main topics out.

28:47 So yes, you can click on the different topics and explore the top words, either top words based on how it's listed there across all the documents or within that one topic.

28:57 And then there's a slider as well.

28:59 I don't know if it's an example, if you scroll down, but there'll be a slider which goes between zero and one.

29:05 And at one, it's the word order, the topic words order, order is ordered by representation across all the documents.

29:15 If you slide all the way down to zero, it's shuffled all the words to be more specific, to show the words that are more specific to that topic, that are exclusive to that topic and not in other topics.

29:30 Whereas if you have it up to one, it's prioritizing the words in the list of words that are like everywhere.

29:36 So yeah, that's just really easy and nice to play around with and explore your model.

29:41 It seems like such a powerful way to explore these models around NLP stuff.

29:45 Yeah, it just looks, it's just nice, it's just well designed and makes you feel happy playing with it.

29:51 Yeah, these pictures and live interactions are great.

29:54 Yeah, and there's really good documentation as well.

29:55 So they've got links to easy to read documents that explain way better than I did what everything and how to interpret stuff.

30:05 So definitely take a look at that.

30:07 And there's some, I think links to some YouTube videos and whatnot as well.

30:10 So yeah, the docs are really nice.

30:12 Links to academic papers explaining what everything is and topic models and yeah, good.

30:17 - Yeah, and there's some linked videos there.

30:19 I didn't pull them out 'cause I think they're probably like talks or something, but yeah, those look good as well.

30:23 - Yeah. - Cool.

30:24 - Nice.

30:25 So it says this package was ported over from R.

30:28 And I know there's a fair number of things in the Python data science world that's like that.

30:33 Do you see that still happening a lot?

30:34 Like what's this interplay between R and Python these days?

30:37 - I think, I've actually not seen that in a while.

30:41 - To me, I'm not very aware of it, but it seemed like that was really popular a couple years ago and I hear less of it now.

30:48 - Yeah, yeah, well yeah, same.

30:50 I think R is really, 'cause R's a stats language, so, and it's been around to stats longer, I think, than Python has been.

31:00 It's much more mature when it comes to that.

31:02 And I think very specific statistical applications are more advanced in R, just 'cause they've been around for longer.

31:10 Python is definitely catching up though.

31:12 But with something like this, I think it's nice that rather than reinventing the wheel in Python, they've just taken something that already works and made it work in Python.

31:20 - Exactly.

31:21 You're like, we like this, we'll just do this.

31:22 This is great.

31:23 - Yeah, like why change it?

31:24 - Yeah, Daniel Chen threw out there, going back a few topics, that there's conda-auto-env, that project, which works, I think, probably like the PDM thing that you had, Brian, what do you think?

31:39 - Say that again?

31:40 - I think we're talking about whether conda has this idea of automatically activating environments under PyPackages.

31:46 I think this project, conda-auto-env, probably does.

31:49 And apparently there's a tie-in with RStudio as well, the guy who created it.

31:53 - Nice, I was just looking at R, so R looks like it's been around since '93.

31:58 I didn't know it was that old.

31:59 - Oh wow.

32:00 - Yeah, based on F apparently.

32:03 - Based on what?

32:04 - F?

32:05 - Based on F.

32:06 - F?

32:06 - Yeah.

32:07 (laughing)

32:08 - Just one character please, one character is all we need.

32:11 - Yeah.

32:12 - C was ahead, we'll go with something to follow on.

32:15 - Yeah, R was my first programming language for data analysis, so I'm really out of touch with it.

32:20 Like now that we've got tidy R, which is supposed to be like really amazing and great for, Like it makes it easier for people new to programming to get up and running quicker.

32:31 But I look at R now and I think, oh I don't know how that works or what that is.

32:35 Since it's been out of it for so long.

32:37 - Okay, so you're completely in Python now?

32:39 - Yes, yeah.

32:40 But I'm not like no to R.

32:43 I don't know, you see it sometimes when people are like yay to Python and no to R or the other way around and I think it's just silly.

32:50 Everything's, they're both really, really good at what they do.

32:52 - Yeah, if they're doing something cool like this, like LDA viz, you know, do that here as well.

32:57 - Yeah. - Yeah.

32:58 - Speaking of visualization, I wanna remind people that are listening to the podcast that we do live stream it, this so that you can hop on on Wednesdays and watch with us or you can catch it on our YouTube channel so that you can see the things we're looking at.

33:16 We highlight, if we're looking at a webpage or a cool visualization, you can see it, so.

33:20 - Yeah, absolutely.

33:22 Is that it, Brian?

33:23 - I think it is.

33:24 - It is.

33:25 - Do you have any extra news or anything?

33:27 - Nothing super exciting, but I did want to tell people about the JetBrains survey.

33:31 And if you've ever gone to the JetBrains site, did you know that they have a little terminal command prompt for agreeing to the cookie policy, which is kind of cool.

33:40 Anyway.

33:41 - Yeah, I love it.

33:42 - I had to do that kind of, I'm like, oh, I hate these cookie things, but that's kind of cool.

33:45 I'm going to do that.

33:46 So they are launching the developer ecosystem survey for 2021.

33:51 If you participate, you get some prizes.

33:53 It does take a little while.

33:54 It took me like 15 minutes.

33:55 It's a non-trivial amount of questions, but I'm sure that we'll cover this in three months or whenever the report comes out and there'll be all sorts of cool stuff we can talk about.

34:02 So, you know, Python people, get your voice heard.

34:05 - Nice, yeah.

34:06 Gotta remember to take that.

34:07 - Yeah, how about you?

34:09 - I've got a couple of exciting bits.

34:11 I am gonna be speaking next week at a couple of places.

34:15 So I'm gonna be speaking at the Python user group for Aberdeen, which it's in the UK.

34:22 that's about all I know.

34:23 (laughs)

34:26 It's a virtual--

34:26 - It's online.

34:27 - Yeah, it's an online thing.

34:30 And so I'm gonna teach, we kind of did a survey of the people going and there's a lot of people new to testing and new to pytest.

34:40 So I'm gonna do sort of a intro to pytest sort of a thing, or at least a topic around pytest that's introductory.

34:47 And then I'm gonna do a similar talk but targeted a little bit closer to what they're doing to NOAA, which I'll probably get that wrong, National Oceanic something, something.

34:59 So I'm gonna talk with a group of those people next week.

35:02 That'll be fun.

35:04 And oh, it's in Scotland.

35:06 Aberdeen is in Scotland, sorry.

35:08 - Thank you, Alex.

35:11 - So I told my kids about both of the things and they're like, "Yeah, Aberdeen, that sounds neat." But no, I really, you're gonna be talking to them.

35:19 So my kids are excited about that.

35:20 - Yeah, that's super cool.

35:22 Jennifer, anything you wanna throw out to people listening?

35:25 You run a user group, right?

35:26 - Yeah, we've got PyData Manchester that we have going on.

35:30 And our next, it's on Meetup, and it's obviously on YouTube, 'cause where else are we gonna be?

35:38 But then our next one coming up is on agent-based models, so that's gonna be really cool.

35:44 Looking forward to that one.

35:45 And hopefully, and I'm not gonna promise too much, But we did put our own podcast on hold for a little bit.

35:51 So hopefully we will start that up again this year.

35:56 So one reason why I'm pretty interested in the tools that you guys use for your podcast, 'cause I think this makes it really interesting and engaging.

36:05 - Yeah, well, I think some of these tools, like we're using, for example, StreamYard for our live streams and stuff, I do think there's a lot of, a low bar to adopt those kind of things for a lot of meetups and stuff.

36:17 So yeah, that's cool.

36:18 If people wanna know what we're doing, they can shoot us a message and we'll let them know.

36:21 - Yeah.

36:22 - I just looked it up.

36:23 Scotland is in the UK, so I wasn't completely wrong.

36:26 - It is.

36:26 No, you're not wrong at all.

36:27 - They're squares and rectangles, come on.

36:30 (laughing)

36:31 You said it was a rectangle, it's all good.

36:32 All right, well, with that, you think we should close it out with a joke?

36:36 - Yes.

36:37 - What do you think, Brian?

36:37 - Yeah.

36:38 - All right, so this one comes to us from Edward Orochina and sent us this cool picture here.

36:45 And this has to do with an engineer helping a designer fix a problem.

36:49 I kind of feel like I want to be the developer.

36:52 Do you mind being the designer, Brian?

36:54 - Sure.

36:55 - All right, so the Brian comes to me, the designer comes and says, "There's a problem with this design." So I say, "Oh, no problem, no problem.

37:03 "We can fix this here in the terminal." And I pull open Z shell and I'm rolling along and--

37:08 - Whoa, you're a hacker.

37:09 - No, no, it's just the terminal.

37:11 - But where are all the buttons and icons and drop down menus?

37:14 Is this the matrix?

37:16 - Yes.

37:16 (laughing)

37:19 - Have you ever had one of those experiences?

37:23 - I had one of those experiences at a coffee shop.

37:26 I was doing something with the terminal and I had like three of them open and one of them was doing like tailing a log and one was running like a pip install script with a bunch of progress bars.

37:35 And people were like, "Are you trying to hack us here on the wifi?" I'm like, "No, I'm just working, leave me alone." (laughing)

37:42 - So I was on the other side of it to start with.

37:45 I was a grad student.

37:48 I shared an office with a couple of other people.

37:52 And one of the women that shared the office with us was a VIM user or VI user at the time.

37:58 And I was with tags and everything.

38:00 And I was an Emacs person at the time with menus and stuff.

38:04 And so I was watching her code once and it's just jumping all over the place.

38:09 Like she'll go to a very, and your hands on the keyboard, nothing, no mouse.

38:13 and the windows are popping back and forth and she's going all over the place.

38:17 I'm like, oh my God, she's like thinking into the computer.

38:21 So I learned to VI because of that experience.

38:26 - Oh, that's awesome.

38:27 - Yeah, when you see people using Vim or any of Emacs, whatever like that, like mind blowing.

38:33 I still need my, I like to use my mouse.

38:36 - Yeah, I like a blended experience as well.

38:39 - Cool, well, thanks everybody.

38:42 - Yeah, thanks Brian.

38:43 Thanks for coming.

38:44 >> Thank you.

38:45 Yeah, this was really fun.

38:46 Thanks for having me.

38:47 >> Thank you for listening to Python Bytes.

38:49 Follow the show on Twitter via @PythonBytes.

38:51 That's Python Bytes as in B-Y-T-E-S.

38:54 And get the full show notes at PythonBytes.fm.

38:58 If you have a news item you want featured, just visit PythonBytes.fm and send it our way.

39:02 We're always on the lookout for sharing something cool.

39:05 On behalf of myself and Brian Aukin, this is Michael Kennedy.

39:08 Thank you for listening and sharing this podcast with your friends and colleagues.