#322: Python Packages, Let Me Count The Ways

Play on YouTube

Watch the live stream replay

About the show

Sponsored by Microsoft for Startups Founders Hub.

Connect with the hosts

• Michael: @mkennedy@fosstodon.org
• Brian: @brianokken@fosstodon.org
• Show: @pythonbytes@fosstodon.org
• Special guest: @calvinhp@fosstodon.org

Join us on YouTube at pythonbytes.fm/stream/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too.

Brian #1: Packaging Python Projects

• Tutorial from PyPA
• This is a really good starting point to understand how to share Python code through packaging.
• Includes discussion of
  • directory layout
  • creating package files, LICENSE, pyproject.toml, README.md, tests and src dir
  • how to fill out build-system section of pyproject.toml
    • using either hatchling, setuptools, flit, or pdm as backends
  • metadata
  • using build to generate wheels and tarballs
  • uploading with twine
• However
  • For small-ish pure Python projects, I still prefer flit
    • flit init creates pyproject.toml and LICENSE
      • will probably still need to hand tweak pyproject.toml
    • flit build replaces build
    • flit publish replaces twine
  • The process can be confusing, even for seasoned professionals.
• Further discussion later in the show

Michael #2: untangle xml

• Convert XML to Python objects
• Children can be accessed with parent.child, attributes with element['attribute'].
• Call the parse() method with a filename, an URL or an XML string.
• Given this XML:

      [HTML_REMOVED]
      [HTML_REMOVED]
        [HTML_REMOVED]
      [HTML_REMOVED]
  

Access the document:

    obj.root.child['name'] # u'child1'

• A little cleaner that ElementTree perhaps.

Calvin #3: Mypy 1.0 Released

• Mypy is a static type checker for Python, basically a Python linter on steroids
• Started in 2012 and developed by a team at Dropbox lead by https://github.com/JukkaL
• What’s New?
  • New Release Numbering Scheme
    • not using symver
    • Significant backward incompatible changes will be announced in the blog post for the previous feature release
    • feature flags will allow users to upgrade and turn on the new behavior
  • Mypy 1.0 is 40% faster than 0.991 against the Dropbox internal codebase
    • 20 optimizations included in this release
  • Mypy now warns about errors used before definition or possibly undefined variables
    • for example if a variable is used outside of a block of code that may not execute
  • Mypy now supports the new Self type introduced in PEP 673 and Python 3.11
  • Support ParamSpec in Type Aliases
  • Also, ParamSpec and Generic Self types are no loner experimental
  • Lots of Miscellaneous New Features
  • Fixes to crashes
  • Support for compiling Python match statements introduced in Python 3.10

Brian #4: Thoughts on the Python packaging ecosystem

• Pradyun Gedam
• Some great background on the internal tension around packaging.
• Brian’s note: in the meantime
  • people are struggling to share Python code
  • the “best practice” answer seems to shift regularly
  • this might be healthy to arrive at better tooling in the long term, but in the short term, it’s hurting us.
• From the article:
  • The Python packaging ecosystem unintentionally became the type of competitive space that it is today.
  • The community needs to make an explicit decision if it should continue operating under the model that led to status quo.
  • Pick from N different tools that do N different things is a good model.
  • Pick from N ~equivalent choices is a really bad user experience.
  • Picking a default doesn’t make other approaches illegal.
  • Communication about the Python packaging ecosystem is fragmented, and we should improve that.
• Pradyun: “Many of the users who write Python code are not primarily full-time software engineers or “developers”.”
• from Thea: “The reason there are so many tools for managing Python dependencies is because Python is not a monoculture and different folks need different things.”
• opening up the build backend through pyproject.toml-based builds was good
• but the fracturing of multiple “workflow” tools seems bad.
• “I am certain that it is not possible to create a single “workflow” tool for Python software. What we have today, an ecosystem of tooling where each makes different design choices and technical trade-offs, is a part of why Python is as widespread as it is today. This flexibility and availability of choice is, however, both a blessing and a curse.”
• On building a default workflow tool around pip
  • interesting idea
• There’s tension between “we need a default workflow tool” and “unix philosophy: many focused tools that can work together”.

Michael #5: Top PyPI Packages

• A monthly dump of the 5,000 most-downloaded packages from PyPI.
• Also, a full copy of PyPI info too: github.com/orf/pypi-data

Calvin #6: SQLAlchemy 2.0 Released

• #57 on the Top PyPI Packages 😸
• Will be giving a SQLAlchemy tutorial at Python Web Conf
• What’s New?
  • Significant API change from 1.4
  • You’ll want to follow the migration guide and see also the what’s new in 2.0 guide
  • Fully takes advantage of Python 3 features such as dataclasses, enums and inline annotations
  • Typing support in Core and ORM, but still should be considered beta
    • all SQLAlchemy stubs packages must be uninstalled all SQLAlchemy stubs packages must be uninstalled for typing to work
    • Mypy Plugin is considered deprecated now
  • Major speed increase in the all new fully ORM-integrated bulk INSERTs
    • sorry if you are on MySQL, they don’t support INSERT RETURNING yet
    • but MariaDB does support this
  • All new bulk optimized schema reflection architecture
    • Currently enabled for PostgreSQL and Oracle
    • 250% perf increase for Postgres
    • 900% per increase for Oracle
  • Native extensions ported to Cython
    • C extensions have been replaced by Cython
    • Benchmarks as fast or sometimes faster than the previous C extensions
    • Removes some risk of memory or stability issues introduced by C
  • SQLAlchemy is now pep-517 enabled and has a pyproject.toml at the root
    • means that local source building with pip can auto install the Cython dependancy

Extras

Brian:

• Nothing to share yet, but I’m building a new alternative Python build backend.
  • which if course will be followed with a new workflow tool that follows “my workflow”.

Michael:

• “Create shortcut: New window” tip:
  • In the dock/task bar
  • Running as an app
• Speaking of Proton, started using simplelogin.io
• What’s all this banning chips about? Great documentary
• Talk Python is hiring!

Calvin:

• 5th Annual Python Web Conf 2023