Publishing package distribution releases using GitHub Actions CI/CD workflows

GitHub Actions CI/CD allows you to run a series of commands whenever an event occurs on the GitHub platform. One popular choice is having a workflow that’s triggered by a push event. This guide shows you how to publish a Python distribution whenever a tagged commit is pushed. It will use the pypa/gh-action-pypi-publish GitHub Action https://github.com/marketplace/actions/pypi-publish

Attention

This guide assumes that you already have a project that you know how to build distributions for and it lives on GitHub.

Warning

At the time of writing, GitHub Actions CI/CD is in public beta. If you don’t have it enabled, you should join the waitlist to gain access.

Saving credentials on GitHub

In this guide, we’ll demonstrate uploading to both PyPI and TestPyPI, meaning that we’ll have two separate sets of credentials. And we’ll need to save them in the GitHub repository settings.

Let’s begin! 🚀

  1. Go to https://pypi.org/manage/account/#api-tokens and create a new API token. If you have the project on PyPI already, limit the token scope to just that project. You can call it something like GitHub Actions CI/CD project-org/project-repo in order for it to be easily distinguishable in the token list. Don’t close the page just yet — you won’t see that token again.

  2. In a separate browser tab or window, go to the Settings tab of your target repository and then click on Secrets in the left sidebar.

  3. Create a new secret called pypi_password and copy-paste the token from the fist step.

  4. Now, go to https://test.pypi.org/manage/account/#api-tokens and repeat the steps. Save that TestPyPI token on GitHub as test_pypi_password.

    Attention

    If you don’t have a TestPyPI account, you’ll need to create it. It’s not the same as a regular PyPI account.

Creating a workflow definition

GitHub CI/CD workflows are declared in YAML files stored in the .github/workflows/ directory of your repository.

Let’s create a .github/workflows/publish-to-test-pypi.yml file.

Start it with a meaningful name and define the event that should make GitHub run this workflow:

name: Publish Python 🐍 distributions 📦 to PyPI and TestPyPI

on: push

Defining a workflow job environment

Now, let’s add initial setup for our job. It’s a process that will execute commands that we’ll define later. In this guide, we’ll use Ubuntu 18.04:

jobs:
  build-n-publish:
    name: Build and publish Python 🐍 distributions 📦 to PyPI and TestPyPI
    runs-on: ubuntu-18.04

Checking out the project and building distributions

Then, add the following under the build-n-publish section:

    steps:
    - uses: actions/checkout@master
    - name: Set up Python 3.7
      uses: actions/setup-python@v1
      with:
        python-version: 3.7

This will download your repository into the CI runner and then install and activate Python 3.7.

And now we can build dists from source. In this example, we’ll use pep517 package, assuming that your project has a pyproject.toml properly set up (see PEP 517/PEP 518).

Tip

You can use any other method for building distributions as long as it produces ready-to-upload artifacts saved into the dist/ folder.

So add this to the steps list:

    - name: Install pep517
      run: >-
        python -m
        pip install
        pep517
        --user
    - name: Build a binary wheel and a source tarball
      run: >-
        python -m
        pep517.build
        --source
        --binary
        --out-dir dist/
        .

Publishing the distribution to PyPI and TestPyPI

Finally, add the following steps at the end:

    - name: Publish distribution 📦 to Test PyPI
      uses: pypa/gh-action-pypi-publish@master
      with:
        password: ${{ secrets.test_pypi_password }}
        repository_url: https://test.pypi.org/legacy/
    - name: Publish distribution 📦 to PyPI
      if: startsWith(github.event.ref, 'refs/tags')
      uses: pypa/gh-action-pypi-publish@master
      with:
        password: ${{ secrets.pypi_password }}

These two steps use the pypa/gh-action-pypi-publish GitHub Action: the first one uploads contents of the dist/ folder into TestPyPI unconditionally and the second does that to PyPI, but only if the current commit is tagged.

That’s all, folks!

Now, whenever you push a tagged commit to your Git repository remote on GitHub, this workflow will publish it to PyPI. And it’ll publish any push to TestPyPI which is useful for providing test builds to your alpha users as well as making sure that your release pipeline remains healthy!