Project Summaries¶
Summaries and links for the most relevant projects in the space of Python installation and packaging.
PyPA Projects¶
bandersnatch¶
bandersnatch is a PyPI mirroring client designed to efficiently
create a complete mirror of the contents of PyPI. Organizations thus
save bandwidth and latency on package downloads (especially in the
context of automated tests) and to prevent heavily loading PyPI’s
Content Delivery Network (CDN).
Files can be served from a local directory or AWS S3.
build¶
build is a PEP 517 compatible Python package builder. It provides a CLI to
build packages, as well as a Python API.
cibuildwheel¶
Docs | Issues | GitHub | PyPI | Discussions | Discord #cibuildwheel
cibuildwheel is a Python package that builds wheels for all common
platforms and Python versions on most CI systems. Also see multibuild.
distlib¶
distlib is a library which implements low-level functions that
relate to packaging and distribution of Python software. distlib
implements several relevant PEPs (Python Enhancement Proposal
standards) and is useful for developers of third-party packaging tools
to make and upload binary and source distributions, achieve interoperability, resolve
dependencies, manage package resources, and do other similar
functions.
Unlike the stricter packaging project (below), which
specifically implements modern Python packaging interoperability
standards, distlib also attempts to provide reasonable fallback
behaviours when asked to handle legacy packages and metadata that
predate the modern interoperability standards and fall into the subset
of packages that are incompatible with those standards.
distutils¶
The original Python packaging system, added to the standard library in Python 2.0 and removed in 3.12.
Due to the challenges of maintaining a packaging system where feature updates are tightly coupled to language runtime updates, direct usage of distutils has been actively discouraged, with Setuptools being the preferred replacement. Setuptools not only provides features that plain distutils doesn’t offer (such as dependency declarations and entry point declarations), it also provides a consistent build interface and feature set across all supported Python versions.
Consequently, distutils was deprecated in Python 3.10 by PEP 632 and has been removed from the standard library in Python 3.12. Setuptools bundles the standalone copy of distutils, and it is injected even on Python < 3.12 if you import setuptools first or use pip.
flit¶
Flit provides a simple way to create and upload pure Python packages and modules to PyPI. It focuses on making the easy things easy for packaging. Flit can generate a configuration file to quickly set up a simple project, build source distributions and wheels, and upload them to PyPI.
Flit uses pyproject.toml to configure a project. Flit does not rely on tools
such as Setuptools to build distributions, or twine to upload them
to PyPI. Flit requires Python 3, but you can use it to distribute modules for
Python 2, so long as they can be imported on Python 3.
The flit package is lifted by Matthias Bussonnier since October 2023 on the tidelift platform, and funds sent to the PSF and earmarked for PyPA usage.
hatch¶
Hatch is a unified command-line tool meant to conveniently manage dependencies and environment isolation for Python developers. Python package developers use Hatch and its build backend Hatchling to configure, version, specify dependencies for, and publish packages to PyPI. Its plugin system allows for easily extending functionality.
packaging¶
Core utilities for Python packaging used by pip and Setuptools.
The core utilities in the packaging library handle version handling, specifiers, markers, requirements, tags, and similar attributes and tasks for Python packages. Most Python users rely on this library without needing to explicitly call it; developers of the other Python packaging, distribution, and installation tools listed here often use its functionality to parse, discover, and otherwise handle dependency attributes.
This project specifically focuses on implementing the modern Python packaging interoperability standards defined at PyPA specifications, and will report errors for sufficiently old legacy packages that are incompatible with those standards. In contrast, the distlib project is a more permissive library that attempts to provide a plausible reading of ambiguous metadata in cases where packaging will instead report on error.
pip¶
The most popular tool for installing Python packages, and the one included with modern versions of Python.
It provides the essential core features for finding, downloading, and installing packages from PyPI and other Python package indexes, and can be incorporated into a wide range of development workflows via its command-line interface (CLI).
Pipenv¶
Pipenv is a project that aims to bring the best of all packaging worlds to the
Python world. It harnesses Pipfile, pip, and virtualenv
into one single toolchain. It can autoimport requirements.txt and also
check for CVEs in Pipfile using safety.
Pipenv aims to help users manage environments, dependencies, and imported packages on the command line. It also works well on Windows (which other tools often underserve), makes and checks file hashes, to ensure compliance with hash-locked dependency specifiers, and eases uninstallation of packages and dependencies.
Pipfile¶
Pipfile and its sister Pipfile.lock are a higher-level
application-centric alternative to pip’s lower-level
requirements.txt file.
pipx¶
pipx is a tool to install and run Python command-line applications without causing dependency conflicts with other packages installed on the system.
Python Packaging User Guide¶
This guide!
readme_renderer¶
readme_renderer is a library that package developers use to render
their user documentation (README) files into HTML from markup
languages such as Markdown or reStructuredText. Developers call it on
its own or via twine, as part of their release management
process, to check that their package descriptions will properly
display on PyPI.
Setuptools¶
Setuptools (which includes easy_install) is a collection of
enhancements to the Python distutils that allow you to more easily
build and distribute Python distributions, especially ones that have dependencies on other packages.
trove-classifiers¶
trove-classifiers is the canonical source for classifiers on PyPI, which project maintainers use to systematically describe their projects so that users can better find projects that match their needs on the PyPI.
The trove-classifiers package contains a list of valid classifiers and deprecated classifiers (which are paired with the classifiers that replace them). Use this package to validate classifiers used in packages intended for uploading to PyPI. As this list of classifiers is published as code, you can install and import it, giving you a more convenient workflow compared to referring to the list published on PyPI. The issue tracker for the project hosts discussions on proposed classifiers and requests for new classifiers.
twine¶
Twine is the primary tool developers use to upload packages to the Python Package Index or other Python package indexes. It is a command-line program that passes program files and metadata to a web API. Developers use it because it’s the official PyPI upload tool, it’s fast and secure, it’s maintained, and it reliably works.
virtualenv¶
virtualenv is a tool for creating isolated Python Virtual Environments, like venv. Unlike venv, virtualenv can create virtual environments for other versions of Python, which it locates using the PATH environment variable. It also provides convenient features for configuring, maintaining, duplicating, and troubleshooting virtual environments. For more information, see the section on Creating Virtual Environments.
Warehouse¶
The current codebase powering the Python Package Index (PyPI). It is hosted at pypi.org. The default source for pip downloads.
wheel¶
Primarily, the wheel project offers the bdist_wheel Setuptools extension for
creating wheel distributions. Additionally, it offers its own
command line utility for creating and installing wheels.
See also auditwheel, a tool that package developers use to check and fix Python packages they are making in the binary wheel format. It provides functionality to discover dependencies, check metadata for compliance, and repair the wheel and metadata to properly link and include external shared libraries in a package.
Non-PyPA Projects¶
buildout¶
Buildout is a Python-based build system for creating, assembling and deploying applications from multiple parts, some of which may be non-Python-based. It lets you create a buildout configuration and reproduce the same software later.
conda¶
Conda is a package, dependency, and environment management system for any language — Python, R, Ruby, C/C++, Fortran, and more. It is written in Python and widely used in the Python scientific computing community, due to its support for non-Python compiled libraries and extensions. It is used as the basis of the Anaconda Python distribution from Anaconda, Inc. It was originally aimed at the scientific community, but can also be used on its own, or with the miniconda, miniforge or pixi systems. It is available for Windows, Mac and Linux systems.
Conda is a completely separate tool from pip, virtualenv and wheel, but provides many of their combined features, such as package management, virtual environment management and deployment of binary extensions and other binary code.
Conda does not install packages from PyPI – it can only manage packages built specifically for conda, which can be made available on a “conda channel”, such as those hosted on anaconda.org, or a local (e.g. intranet) package server. In addition to the “default” channels managed by Anaconda, Inc., there are a wide variety of packages from the community supported conda-forge project
Note that pip can be installed into, and work side-by-side with conda for managing distributions from PyPI. It is also possible to build conda packages from Python source packages using tools such as conda skeleton: a tool to automatically make conda packages from Python packages available on PyPI.
devpi¶
devpi features a powerful PyPI-compatible server and PyPI proxy cache with a complementary command line tool to drive packaging, testing and release activities with Python. devpi also provides a browsable and searchable web interface. devpi supports mirroring PyPI, multiple package indexes with inheritance, syncing between these indexes, index replication and fail-over, and package upload.
dumb-pypi¶
dumb-pypi is a simple package index static file site generator, which then must be hosted by a static file webserver to become the package index. It supports serving the hash, core-metadata, and yank-status.
enscons¶
Enscons is a Python packaging tool based on SCons. It builds pip-compatible source distributions and wheels without using distutils or setuptools, including distributions with C extensions. Enscons has a different architecture and philosophy than distutils. Rather than adding build features to a Python packaging system, enscons adds Python packaging to a general purpose build system. Enscons helps you to build sdists that can be automatically built by pip, and wheels that are independent of enscons.
Flask-Pypi-Proxy¶
Warning
Not maintained, project archived
Flask-Pypi-Proxy is a package index as a cached proxy for PyPI.
Hashdist¶
Hashdist is a library for building non-root software distributions. Hashdist is trying to be “the Debian of choice for cases where Debian technology doesn’t work”. The best way for Pythonistas to think about Hashdist may be a more powerful hybrid of virtualenv and buildout. It is aimed at solving the problem of installing scientific software, and making package distribution stateless, cached, and branchable. It is used by some researchers but has been lacking in maintenance since 2016.
Maturin¶
Maturin is a build backend for Rust extension modules, also written in Rust. It supports building wheels for python 3.7+ on Windows, Linux, macOS and FreeBSD, can upload them to PyPI and has basic PyPy and GraalPy support.
meson-python¶
meson-python is a build backend that uses the Meson build system. It enables
Python package authors to use Meson as the build system for their package. It
supports a wide variety of languages, including C, and is able to fill the needs
of most complex build configurations.
multibuild¶
Multibuild is a set of CI scripts for building and testing Python wheels for Linux, macOS, and (less flexibly) Windows. Also see cibuildwheel.
nginx_pypi_cache¶
nginx_pypi_cache is a package index caching proxy using nginx.
pdm¶
PDM is a modern Python package manager. It uses pyproject.toml to store project metadata as defined in PEP 621.
pex¶
Pex is a tool for generating .pex (Python EXecutable)
files, standalone Python environments in the spirit of virtualenv.
PEX files are zipapps that
make deployment of Python applications as simple as cp. A single PEX
file can support multiple target platforms and can be created from standard
pip-resolvable requirements, a lockfile generated with pex3 lock ...
or even another PEX. PEX files can optionally have tools embedded that support
turning the PEX file into a standard venv, graphing dependencies and more.
pip-tools¶
pip-tools is a suite of tools meant for Python system administrators and release managers who particularly want to keep their builds deterministic yet stay up to date with new versions of their dependencies. Users can specify particular release of their dependencies via hash, conveniently make a properly formatted list of requirements from information in other parts of their program, update all dependencies (a feature pip currently does not provide), and create layers of constraints for the program to obey.
pip2pi¶
pip2pi is a package index server where specific packages are manually synchronised.
piwheels¶
piwheels is a website, and software underpinning it, that fetches source code distribution packages from PyPI and compiles them into binary wheels that are optimized for installation onto Raspberry Pi computers. Raspberry Pi OS pre-configures pip to use piwheels.org as an additional index to PyPI.
poetry¶
poetry is a command-line tool to handle dependency installation and
isolation as well as building and packaging of Python packages. It
uses pyproject.toml and, instead of depending on the resolver
functionality within pip, provides its own dependency resolver.
It attempts to speed users’ experience of installation and dependency
resolution by locally caching metadata about dependencies.
proxpi¶
proxpi is a simple package index which proxies PyPI and other indexes with caching.
Pulp-python¶
Pulp-python is the Python package index plugin for Pulp. Pulp-python supports mirrors backed by local or AWS S3, package upload, and proxying to multiple package indexes.
PyPI Cloud¶
Warning
Not maintained, project archived
PyPI Cloud is a package index server, backed by AWS S3 or another cloud storage service, or local files. PyPI Cloud supports redirect/cached proxying for PyPI, as well as authentication and authorisation.
pypiprivate¶
pypiprivate serves a local (or AWS S3-hosted) directory of packages as a package index.
pypiserver¶
pypiserver is a minimalist application that serves as a private Python package index (from a local directory) within organizations, implementing a simple API and browser interface. You can upload private packages using standard upload tools, and users can download and install them with pip, without publishing them publicly. Organizations who use pypiserver usually download packages both from pypiserver and from PyPI.
PyScaffold¶
PyScaffold is a project generator for bootstrapping Python packages, ready to be shared on PyPI and installable via pip. It relies on a set of sane default configurations for established tools (such as Setuptools, pytest and Sphinx) to provide a productive environment so developers can start coding right away. PyScaffold can also be used with existing projects to make packaging easier.
pywharf¶
Warning
Not maintained, project archived
pywharf is a package index server, serving files locally or from GitHub.
scikit-build¶
Scikit-build is a Setuptools wrapper for CPython that builds C/C++/Fortran/Cython extensions It uses cmake (available on PyPI) to provide better support for additional compilers, build systems, cross compilation, and locating dependencies and their associated build requirements. To speed up and parallelize the build of large projects, the user can install ninja (also available on PyPI).
scikit-build-core¶
Scikit-build-core is a build backend for CPython C/C++/Fortran/Cython extensions. It enables users to write extensions with cmake (available on PyPI) to provide better support for additional compilers, build systems, cross compilation, and locating dependencies and their associated build requirements. CMake/Ninja are automatically downloaded from PyPI if not available on the system.
shiv¶
shiv is a command line utility for building fully self contained Python zipapps as outlined in PEP 441, but with all their dependencies included. Its primary goal is making distributing Python applications and command line tools fast & easy.
simpleindex¶
simpleindex is a package index which routes URLs to multiple package indexes (including PyPI), serves local (or cloud-hosted, for example AWS S3, with a custom plugin) directories of packages, and supports custom plugins.
Spack¶
Docs | GitHub | Paper | Slides
A flexible package manager designed to support multiple versions, configurations, platforms, and compilers. Spack is like Homebrew, but packages are written in Python and parameterized to allow easy swapping of compilers, library versions, build options, etc. Arbitrarily many versions of packages can coexist on the same system. Spack was designed for rapidly building high performance scientific applications on clusters and supercomputers.
Spack is not in PyPI (yet), but it requires no installation and can be used immediately after cloning from GitHub.
zest.releaser¶
zest.releaser is a Python package release tool providing an
abstraction layer on top of twine. Python developers use
zest.releaser to automate incrementing package version numbers,
updating changelogs, tagging releases in source control, and uploading
new packages to PyPI.
Standard Library Projects¶
ensurepip¶
A package in the Python Standard Library that provides support for bootstrapping pip into an existing Python installation or virtual environment. In most cases, end users won’t use this module, but rather it will be used during the build of the Python distribution.
http.server¶
A package and command-line interface which can host a directory as a website, for example as a package index (see Hosting your own simple repository).
venv¶
A package in the Python Standard Library (starting with Python 3.3) for creating Virtual Environments. For more information, see the section on Creating Virtual Environments.