How to Publish a Python Package from GitHub Actions ===================================================
A simple way to use [GitHub Actions](https://github.com/actions) to build your Python package, bump the version number, and publish it to [GitHub releases](https://github.com/seanh/gha-python-packaging-demo/releases) and [PyPI.org](https://pypi.org/project/gha-python-packaging-demo/) all with a single click of a button in the web interface.
When I want to publish a new release of one of my Python packages I just browse to [my `release.yml` workflow in the Actions tab](https://github.com/seanh/gha-python-packaging-demo/actions/workflows/release.yml) and click the Run workflow button: ![Running the release workflow]({{ "/assets/images/run_workflow_button.png" | relative_url }} "Running the release workflow") This runs a GitHub Actions workflow that increments the patch version number, creates a new [GitHub release](https://github.com/seanh/gha-python-packaging-demo/releases) with auto-generated release notes, creates a corresponding git tag, and [publishes the release to PyPI](https://pypi.org/project/gha-python-packaging-demo/#history): ![A GitHub release generated by release.py]({{ "/assets/images/release.png" | relative_url }} "A GitHub release generated by release.py") If I feel like it I can create a new release manually using GitHub's web interface instead. This allows me to choose my own version number (in case I want to bump the major or minor version instead of the patch version) and lets me write my own release notes (or I can use the Auth-generate release notes button to get GitHub's auto-generated release notes). Manually-created releases still trigger the GitHub Actions workflow that builds the package and uploads it to PyPI: ![Publishing a GitHub release]({{ "/assets/images/github_release.png" | relative_url }} "Publishing a GitHub release") In fact GitHub releases created by **any** means trigger the package build and upload workflow. For example I could run GitHub CLI's [`gh release create` command](https://cli.github.com/manual/gh_release_create) from any directory on my local machine. This doesn't require me to have a local copy of the project or to have anything setup locally other than GitHub CLI itself: ```terminal $ gh release create --repo seanh/gha-python-packaging-demo --generate-notes 0.0.1 https://github.com/seanh/gha-python-packaging-demo/releases/tag/0.0.1 ``` The `--generate-notes` option uses GitHub's auto-generated release notes again or you can use `--title '[build-system]
requires = ["setuptools>=45", "setuptools_scm[toml]>=6.2"]
build-backend = "setuptools.build_meta"
[tool.setuptools_scm]
This is the same as the
[`pyproject.toml` from the Python Packaging User Guide's packaging tutorial](https://packaging.python.org/en/latest/tutorials/packaging-projects/#creating-pyproject-toml)
but with the two parts in bold added. These two additions add
[setuptools_scm](https://github.com/pypa/setuptools_scm/) (following the
[instructions in their README](https://github.com/pypa/setuptools_scm/#pyprojecttoml-usage)).
setuptools_scm extracts the version number from the project's git tags during
the build process and injects it into the package that gets built.
This means **there's no version number anywhere in the source tree**.
There's no version number in `pyproject.toml` or `setup.cfg` or in the Python code.
The git tags are the single source for version numbers.
This is crucial to the packaging scripts we'll be using below because it means
you don't need to create and push a git commit in order to do a release.
You just need to create a new git tag.
And GitHub does that for you when you create a GitHub release.
setuptools_scm should be a solid dependency:
it's [mentioned](https://packaging.python.org/en/latest/guides/single-sourcing-package-version/?highlight=setuptools_scm#single-sourcing-the-package-version)
in the Python Packaging User Guide, it's owned by the [Python Packaging Authority](https://github.com/pypa)
on GitHub, at the time as writing it's on version 6.4.2, its been around since
2010 and it's still actively maintained.
Create a PyPI API token
-----------------------
The `publish.yml` workflow below needs a PyPI API token to upload releases of
your package to PyPI.
[Create a free PyPI.org account](https://pypi.org/account/register/) if you don't already have one,
log in,
go to your account settings,
and go to the [Add API Token](https://pypi.org/manage/account/token/) page.
You need to create an **unscoped** API token.
You can't scope the API token to the project yet because the project doesn't
exist on PyPI yet so it doesn't appear as an option in the scopes dropdown.
You can [replace the token with a scoped one](#replace-the-pypi-token-with-a-scoped-one)
after publishing the project's first release.
![Creating a PyPI API token]({{ "/assets/images/creating_a_pypi_api_token.png" | relative_url }} "Creating a PyPI API token")
Add the PyPI API token to the GitHub project's secrets
------------------------------------------------------
You need to add the PyPI API token to your GitHub project's secrets to make it
available to the `publish.yml` workflow.
Go to the GitHub project's Settings tab, go to
Secrets → Actions, and click
New repository secret.
Enter `PYPI_API_TOKEN` as the name and paste in the PyPI API token as the value
then click Add secret:
![Pasting the PyPI API token into GitHub secrets]({{ "/assets/images/github_secret.png" | relative_url }} "Pasting the PyPI API token into GitHub secrets")
The `publish` workflow
--------------------
[`publish.yml`](https://github.com/seanh/gha-python-packaging-demo/blob/main/.github/workflows/publish.yml)
is a GitHub Actions workflow that triggers whenever a GitHub release is
created. It builds the Python package and uploads it to PyPI. It's 17 lines:
```yaml
name: Publish to PyPI.org
on:
release:
types: [published]
jobs:
pypi:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
with:
fetch-depth: 0
- run: python3 -m pip install --upgrade build && python3 -m build
- name: Publish package
uses: pypa/gh-action-pypi-publish@release/v1
with:
password: {% raw %}${{ secrets.PYPI_API_TOKEN }}{% endraw %}
```
Some notes:
* This part causes the workflow to be automatically triggered whenever a new
GitHub release is published:
```yaml
on:
release:
types: [published]
```
See [Events that trigger workflows](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows)
in the GitHub Actions docs.
* The `fetch-depth: 0` causes the [checkout action](https://github.com/marketplace/actions/checkout)
to fetch all branches and tags instead of only fetching the ref/SHA that
triggered the workflow. This is necessary for setuptools_scm to work,
otherwise it wouldn't be able to find the version number from the git tags.
* The `python3 -m pip install --upgrade build && python3 -m build` is what
actually builds the Python package. These are the standard Python packaging
commands, straight from the
[Python Packaging User Guide's tutorial](https://packaging.python.org/en/latest/tutorials/packaging-projects/#generating-distribution-archives).
* Finally, we use the [pypa/gh-action-pypi-publish](https://github.com/marketplace/actions/pypi-publish)
GitHub Action from the Python Packaging authority to actually upload the
built package to PyPI.
The `{% raw %}${{ secrets.PYPI_API_TOKEN }}{% endraw %}` passes the API token
that you added to the project's GitHub secrets through to
`gh-action-pypi-publish` which will use it to authenticate to PyPI.
Create the first release
------------------------
You can now create a GitHub release and it'll trigger the publish workflow to
build the package and upload it to PyPI.
Go to the GitHub project's Releases page and click the
Create a new release button.
In the Choose a tag field enter `0.0.1` and click
Create new tag: 0.0.1 on publish.
Click Publish release:
![Publishing a GitHub release]({{ "/assets/images/github_release.png" | relative_url }} "Publishing a GitHub release")
Alternatively, instead of using the web interface you can install
[GitHub CLI](https://cli.github.com/) on your local machine and run:
```terminal
$ gh release create --repo seanh/gha-python-packaging-demo --title "First release!" --notes "This is the first release!" 0.0.1
```
Either way publishing a release will trigger the `publish.yml` workflow and the
package will be built and uploaded to PyPI. You can browse to the **Publish to
PyPI.org** workflow on the project's **Actions** page and view
[the logs](https://github.com/seanh/gha-python-packaging-demo/runs/6557828531).
You should also be able to browse to [the project's release history on PyPI](https://pypi.org/project/gha-python-packaging-demo/#history)
and see that the package release has been uploaded.
Replace the PyPI token with a scoped one
----------------------------------------
Now that you've published the first release the project will start appearing in
the list of scopes when creating an API token on PyPI. You can go back to your
PyPI account settings, delete the unscoped token, create a new token scoped to
the project, and update the value of the `PYPI_API_TOKEN` GitHub secret with
the new token.
![Creating a scoped PyPI API token]({{ "/assets/images/scoped_pypi_api_key.png" | relative_url }} "Creating a scoped PyPI API token")
That's it!
----------
From now on each time you create a new GitHub release using either the GitHub
web UI or GitHub CLI your Python package will be built and uploaded to PyPI.
You could stop here
-------------------
We've gotten as far as being able to create a GitHub release with either the
web UI or GitHub CLI and having that release be automatically built and
published to PyPI. GitHub can even generate the release notes for us. And all
we needed was setuptools_scm in `pyproject.toml`, a 17-line GitHub Actions
workflow and a PyPI API token in the project's GitHub secrets. This is really
very simple and you could just stop here and keep using it.
But having to manually enter the version number for every release could be error prone,
especially if you release a lot of packages.
It'd be nice if GitHub CLI could generate the version number for you.
Until then, we can implement that ourselves by adding just a little more code...
Automatically generating the next version number
------------------------------------------------
### The `release.py` script
[`release.py`](https://github.com/seanh/gha-python-packaging-demo/blob/main/.github/scripts/release.py)
is a short Python script that calls GitHub CLI to create new releases with
auto-incrementing version numbers and auto-generated release notes. If the
project doesn't have any releases yet it'll use `0.0.1` as the first version
number. Otherwise it'll +1 the patch number of the last release: `0.0.2`,
`0.0.3`, and so on.
Make sure you mark this script as executable:
```terminal
$ chmod u+x .github/workflows/scripts/release.py
```
As long as you have GitHub CLI installed you can run this script locally. It
should be compatible with your operating system's version of Python and it
doesn't have any dependencies, so it should just work:
```terminal
$ .github/scripts/release.py
https://github.com/seanh/gha-python-packaging-demo/releases/tag/0.0.3
```
And of course creating a release with this script will trigger the publish
workflow to upload the package to PyPI, just like creating a release in the web
interface or by using GitHub CLI would do.
If you want to increment the major or minor version number you have to create a
release manually to do that, then you can go back to using `release.py` and
it'll start generating patch numbers on top of your new major or minor numbers.
(It would be easy to add `--major` and `--minor` arguments to `release.py` if
you want to be able to use it to generate major and minor releases as well.)
### The release workflow
If you want to be able to run `release.py` on GitHub Actions you need to add a
workflow for it.
[`release.yml`](https://github.com/seanh/gha-python-packaging-demo/blob/main/.github/workflows/release.yml)
is a 12-line manually-triggered workflow that runs the `release.py` script on
GitHub Actions:
```yml
name: Create a new patch release
on: workflow_dispatch
jobs:
github:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Create new patch release
run: .github/scripts/release.py
env:
GITHUB_TOKEN: {% raw %}${{ secrets.PERSONAL_ACCESS_TOKEN }}{% endraw %}
```
Notes:
* The `on: workflow_dispatch` means that this workflow never runs
automatically, it can only be triggered by the Run workflow
button:
![Running the release workflow]({{ "/assets/images/run_workflow_button.png" | relative_url }} "Running the release workflow")
* The `{% raw %}${{ secrets.PERSONAL_ACCESS_TOKEN }}{% endraw %}` passes a
personal access token through to GitHub CLI to use to authenticate to the
GitHub API.
To get this to work you need to [create a GitHub personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
and add it to the project's GitHub secrets.
GITHUB_TOKEN
?