Brought to you by Michael and Brian - take a Talk Python course or get Brian's pytest book

#311: Catching Memory Leaks with ... pytest?

Published Thu, Nov 24, 2022, recorded Tue, Nov 22, 2022
Watch this episode on YouTube
Play on YouTube
Watch the live stream replay

About the show

Python Bytes 311

Sponsored by Microsoft for Startups Founders Hub.

Connect with the hosts

Michael #1: Latexify

  • We are used to turning beautiful math into programming symbols.
  • For example: amitness.com/2019/08/math-for-programmers/#sigma
  • Take this code: def do_math(a, b, c): return (-b + math.sqrt(b * 2 - 4 a * c)) / (2 * a)
  • Add @latexify.function decorator
  • display do_math in a notebook
  • Get this latex: \mathrm{do_math}(a, b, c) = \frac{-b + \sqrt{b^{{2}} - {4} a c}}{{2} a}
  • Which renders as

  • I could only get it to install with: pip install git+https://github.com/google/latexify_py

Brian #2: prefixed

Murilo #3: dbt

  • Open source tool
  • CLI tool
  • Built with Python 🐍
  • Applies “best practices” to SQL projects
  • Combines git + .sql files + jinja
  • Support many data platforms
  • Let’s you
    • Template SQL queries
      • Including loops
    • Execute DAGs
    • Data validation
    • Easily build docs (data lineage, visualize DAGs, etc.)
  • Now you can also run Python models
    • Useful if there’s a convenient python function for your data transformation or some more complex logic (i.e.:fuzzy string matching, machine learning models, etc.)
    • Available for Snowflake, Databricks, BigQuery
    • dbt’s coalesce’s announcement https://www.youtube.com/watch?v=rVprdyxcGUo

Michael #4: Memray pytest plugin

  • pytest-memray is the pytest plugin for, well, memray. :)
  • You can ensure that not too much memory is used with @pytest``**.**``mark``**.**``limit_memory``**(**``"24 MB"``**)**
  • And you get an allocation report with pytest --memray file.py
  • But coming soon, we’ll have memory leak checking too.
    @pytest.mark.check_leaks()
    def test_foobar():
      # Do some stuff and ensure
      # it does not leak memory
      pass
    

Brian #5: Stealing Open Source code from Textual

  • Will McGugan
  • Will reminds us of one of the great benefits of open source code, stealing code
    • (when allowed by the license, of course)
  • Goes as far as to point out some bits of textual that you might want to lift
    • looping with indication of when you’ve hit the first or last item
    • a LRUCache with more flexibility than lru_cache
    • a Color class with conversions for css, hex, monochrome, hsl
    • 2d geometry

Murilo #6: Shed

  • Superset of black
  • "shed is the maximally opinionated autoformatting tool. It's all about convention over configuration, and designed to be a single opinionated tool that fully canonicalises my code - formatting, imports, updates, and every other fix I can possibly automate.”
  • Also format code snippets in docstrings, markdown, restructured text
  • No configuration options
  • pre-commit hooks available
  • Bundles together:
    • black
    • isort
    • autoflake
    • pyupgrade
    • blacken-docs

Extras

Brian:

  • pytest-check (version 1.1.3) changes now live
    • New README, hopefully makes it clear how to use.
    • Use check from
      • from pytest_check import check
      • or from the check fixture: def test_foo(check): …
    • Either form returns the same object.
    • From that check object, you can
      • use helper functions like check.equal(a, b), etc.
      • use it as a context manager, with check: assert a == b
      • even grab the raises context manager: with check.raises(Exception): …
    • Intended to be backwards compatible
      • although some old use cases might be deprecated/removed in the future.

Michael:

Murilo:

Joke:


Want to go deeper? Check our projects