# Scientific Python: guide, cookie, & sp-repo-review

## Cookie

[![Actions Status][actions-badge]][actions-link]
[![GitHub Discussion][github-discussions-badge]][github-discussions-link]
[![Live ReadTheDocs][rtd-badge]][rtd-link]

[![PyPI version][pypi-version]][pypi-link]
[![Conda-Forge][conda-badge]][conda-link]
[![PyPI platforms][pypi-platforms]][pypi-link]

A [copier][]/[cookiecutter][] template for new Python projects based on the
Scientific Python Developer Guide. What makes this different from other
templates for Python packages?

- Lives with the [Scientific-Python Development Guide][]: Every decision is
  clearly documented and every tool described, and everything is kept in sync.
- Ten different backends to choose from for building packages.
- Optional VCS versioning for most backends.
- Selection for your preferred documentation engine (Sphinx, MkDocs, or
  Zensical).
- Template generation tested in GitHub Actions using nox.
- Supports generation with [copier][], [cookiecutter][], and [cruft][].
- Supports GitHub Actions if targeting a `github.com` url (the default), and
  adds experimental GitLab CI support otherwise.
- Includes several compiled backends using [pybind11][], with wheels produced
  for all platforms using [cibuildwheel][].
- Provides [`sp-repo-review`][pypi-link] to evaluate existing repos against the
  guidelines, with a WebAssembly version integrated with the guide. Checks are
  hyperlinked back to the guide.
- Follows [PyPA][] best practices and regularly updated. Recent additions:
  - [PEP 639](https://peps.python.org/pep-0639): SPDX licenses
  - [PEP 735](https://peps.python.org/pep-0735): Dependency groups
- Uses uv for high performance CI and task running.

Be sure you have read the [Scientific-Python Development Guide][] first, and
possibly used them on a project or two. This is _not_ a minimal example or
tutorial. It is a collection of useful tooling for starting a new project using
cookiecutter, or for copying in individual files for an existing project (by
hand, from `{{cookiecutter.project_name}}/`).

During generation you can select from the following backends for your package:

1. [hatch][]: This uses hatchling, a modern builder with nice file inclusion,
   extendable via plugins, and good error messages. **(Recommended for pure
   Python projects)**
2. [uv][]: The `uv_build` backend is written in Rust and is integrated into'
   `uv`, meaning it can build without downloading anything extra and can even
   avoid running Python at all when building, making it the fastest backend for
   simple packages. No dynamic metadata support.
3. [flit][]: A modern, lightweight [PEP 621][] build system for pure Python
   projects. Replaces setuptools, no MANIFEST.in, setup.py, or setup.cfg. Low
   learning curve. Easy to bootstrap into new distributions. Difficult to get
   the right files included, little dynamic metadata support.
4. [pdm][]: A modern, less opinionated all-in-one solution to pure Python
   projects supporting standards. Replaces setuptools, venv/pipenv, pip, wheel,
   and twine. Supports [PEP 621][].
5. [poetry][]: An all-in-one solution to pure Python projects. Replaces
   setuptools, venv/pipenv, pip, wheel, and twine. Higher learning curve, but
   is all-in-one. Makes some bad default assumptions for libraries.
6. [setuptools][]: The classic build system, but with the new standardized
   configuration.
7. [pybind11][]: This is setuptools but with an C++ extension written in
   [pybind11][] and wheels generated by [cibuildwheel][].
8. [scikit-build][]: A scikit-build (CMake) project also using pybind11, using
   scikit-build-core. **(Recommended for C++ projects)**
9. [meson-python][]: A Meson project also using pybind11. (No VCS versioning)
10. [maturin][]: A [PEP 621][] builder for Rust binary extensions. (No VCS
    versioning) **(Recommended for Rust projects)**

Currently, the best choice is probably hatch for pure Python projects, and
scikit-build (such as the scikit-build-core + pybind11 choice) for binary
projects.

### To use (copier version)

Install `copier` and `copier-templates-extensions`. Using [uv][], that's:

```bash
uv tool install --with copier-templates-extensions copier
```

Now, run copier to generate your project:

```bash
copier copy gh:scientific-python/cookie <pkg> --trust --vcs-ref=HEAD
```

(`<pkg>` is the path to put the new project. `--vcs-ref=HEAD` gets the current
version instead of the last tag, matching cookiecutter's behavior. Note you can
combine these two lines into one with `uvx`, just remember to pass `--with`
before the program name in that case.)

You will get a nicer CLI experience with answer validation. You will also get a
`.copier-answers.yml` file, which will allow you to perform updates in the
future.

> Note: Add `--vcs-ref=HEAD` to get the latest version instead of the last
> tagged version; HEAD always passes tests (and is what cookiecutter uses).

#### To use (cookiecutter version)

Install cookiecutter, ideally with `brew install cookiecutter` if you use brew,
otherwise with `uv tool install cookiecutter` (or prepend `uvx` to the command
below, and skip installation). Then run:

```bash
cookiecutter gh:scientific-python/cookie
```

If you are using cookiecutter 2.2.3+, you will get nice descriptions for the
options like copier!

#### To use (cruft version)

You can also use [cruft][], which adds the ability update to cookiecutter
projects. Install with `uv tool install cruft` (or prepend `uvx` to the command
below, and skip installation). Then run:

```bash
cruft create https://github.com/scientific-python/cookie
```

#### Post generation

Check the key setup files, `pyproject.toml`, and possibly `setup.cfg` and
`setup.py` (pybind11 example). Update `README.md`. Also update and add docs to
`docs/`.

There are a few example dependencies and a minimum Python version of 3.10, feel
free to change it to whatever you actually need/want. There is also a basic
backports structure with a small typing example.

#### Contained components

- GitHub Actions runs testing for the generation itself
  - Uses nox so cookie development can be checked locally
  - Uses uv for high performance CI
- GitHub actions deployment
  - C++ backends include cibuildwheel for wheel builds
  - Uses PyPI trusted publisher deployment
- A GitHub release configuration that skips common bot commits when
  auto-generating release notes
- Dependabot keeps actions up to date periodically, through useful pull requests
- A contributing guide
- Formatting handled by pre-commit or prek
  - No reason not to be strict on a new project; remove what you don't want.
  - Includes MyPy - static typing
  - Includes Ruff - standard formatting, linting and autofixes
    - Replaces Flake8, isort, pyupgrade, yesqa, pycln, and dozens of plugins
  - Includes spell checking
- One of several popular licenses
- A pylint nox target can be used to run pylint, which integrated GHA
  annotations
- A ReadTheDocs-ready `docs/` folder and `docs` dependency-group
- A `tests/` folder and pytest `test` dependency-group
- A dev group for `uv run` integration
- A noxfile is included with a few common targets
- A README
- Code coverage reporting with automatic uploads to Codecov after tests run

#### For developers

You can test locally with [nox][]:

```console
# See all commands
nox -l

# Run a specific check
nox -s "lint(scikit-build)"

# Run a noxfile command on the project noxfile
nox -s "nox(hatch)" -- docs
```

If you don't have `nox` locally, you can use [uv][], such as `uvx nox` instead.

#### Other similar projects

[Hypermodern-Python][hypermodern] is another project worth checking out with
many similarities, like great documentation for each feature and many of the
same tools used. It has a slightly different set of features, and has a stronger
focus on GitHub Actions - most our guide could be adapted to a different CI
system fairly easily if you don't want to use GHA. It also forces the use of
Poetry (instead of having a backend selection), and doesn't support compiled
projects. It currently dumps all development dependencies into a shared
environment, causing long solve times and high chance of conflicts. It also does
not use pre-commit the way it was intended to be used. It also has quite a bit
of custom code.

#### History

A lot of the guide, cookiecutter, and repo-review started out as part of
[Scikit-HEP][]. These projects were merged, generalized, and combined with the
[NSLS-II][] guide during the 2023 Scientific-Python Developers Summit.

[actions-badge]: https://github.com/scientific-python/cookie/actions/workflows/ci.yml/badge.svg
[actions-link]: https://github.com/scientific-python/cookie/actions
[cibuildwheel]: https://cibuildwheel.readthedocs.io
[cookiecutter]: https://cookiecutter.readthedocs.io
[copier]: https://copier.readthedocs.io
[cruft]: https://cruft.github.io/cruft
[flit]: https://flit.readthedocs.io/en/latest/
[github-discussions-badge]: https://img.shields.io/static/v1?label=Discussions&message=Ask&color=blue&logo=github
[github-discussions-link]: https://github.com/scientific-python/cookie/discussions
[hatch]: https://github.com/ofek/hatch
[hypermodern]: https://github.com/cjolowicz/cookiecutter-hypermodern-python
[maturin]: https://maturin.rs
[meson-python]: https://meson-python.readthedocs.io
[nox]: https://nox.thea.codes/en/stable/
[nsls-ii]: https://nsls-ii.github.io/scientific-python-cookiecutter/
[pdm]: https://pdm.fming.dev
[pep 621]: https://www.python.org/dev/peps/pep-0621
[poetry]: https://python-poetry.org
[pybind11]: https://pybind11.readthedocs.io
[pypa]: https://www.pypa.io
[pypi-link]: https://pypi.org/project/sp-repo-review/
[pypi-platforms]: https://img.shields.io/pypi/pyversions/sp-repo-review
[pypi-version]: https://badge.fury.io/py/sp-repo-review.svg
[rtd-badge]: https://readthedocs.org/projects/scientific-python-cookie/badge/?version=latest
[rtd-link]: https://scientific-python-cookie.readthedocs.io/en/latest/?badge=latest
[scikit-build]: https://scikit-build.readthedocs.io
[setuptools]: https://setuptools.readthedocs.io
[uv]: https://docs.astral.sh/uv

---

## sp-repo-review

<!-- sp-repo-review -->

`sp-repo-review` provides checks based on the [Scientific-Python Development
Guide][] at [scientific-python/cookie][] for [repo-review][].

This tool can check the style of a repository. Use like this:

```bash
uvx sp-repo-review[cli] <path to repository>
```

This will produce a list of results - green checkmarks mean this rule is
followed, red x’s mean the rule is not. A yellow warning sign means that the
check was skipped because a previous required check failed. Some checks will
fail, that’s okay - the goal is bring all possible issues to your attention, not
to force compliance with arbitrary checks. Eventually there might be a way to
mark checks as ignored.

For example, `GH101` expects all your action files to have a nice `name:` field.
If you are happy with the file-based names you see in CI, you should feel free
to simply ignore this check (you can specify ignored checks in pyproject.toml or
by passing args to repo-review, see the [repo-review
docs][repo-review configuring]).

All checks are mentioned at least in some way in the [Scientific-Python
Development Guide][]. You should read that first - if you are not attempting to
follow them, some of the checks might not work. For example, the guidelines
specify pytest configuration be placed in `pyproject.toml`. If you place it
somewhere else, then all the pytest checks will be skipped.

This was originally developed for [Scikit-HEP][] before moving to Scientific
Python.

## Extras

- `cli`: Dependencies to run the CLI (not needed for programmatic access, like
  on Web Assembly)
- `pyproject`: Include validate pyproject with schema store.
- `all`: All extras

## Helper utility

There's also a script, accessible as `sp-ruff-checks`, that will compare your
ruff checks to the known values. It's a little more elegant on the command line
than the Ruff family description, which will only print out a basic list.

## Other ways to use

You can also use GitHub Actions:

```yaml
- uses: scientific-python/cookie@<version>
```

Or pre-commit:

```yaml
- repo: https://github.com/scientific-python/cookie
  rev: <version>
  hooks:
    - id: sp-repo-review
```

If you use `additional_dependencies` to add more plugins, like
`validate-pyproject`, you should also include `"repo-review[cli]"` to ensure the
CLI requirements are included.

## List of checks

<!-- rumdl-disable MD013 -->

<!-- [[[cog
import itertools

from repo_review.processor import collect_all
from repo_review.checks import get_check_url, get_check_description
from repo_review.families import get_family_name

collected = collect_all()
print()
for family, grp in itertools.groupby(collected.checks.items(), key=lambda x: x[1].family):
    print(f'### {get_family_name(collected.families, family)}\n')
    if note := collected.families.get(family, {}).get("readme_note"):
        print(note, end="\n\n")
    for code, check in grp:
        url = get_check_url(code, check)
        link = f"[`{code}`]({url})" if url else f"`{code}`"
        print(f"- {link}: {get_check_description(code, check)}")
    print()
]]] -->

### General

- [`PY001`](https://learn.scientific-python.org/development/guides/packaging-simple#PY001): Has a pyproject.toml
- [`PY002`](https://learn.scientific-python.org/development/guides/packaging-simple#PY002): Has a README.(md|rst) file
- [`PY003`](https://learn.scientific-python.org/development/guides/packaging-simple#PY003): Has a LICENSE* file
- [`PY004`](https://learn.scientific-python.org/development/guides/packaging-simple#PY004): Has docs folder
- [`PY005`](https://learn.scientific-python.org/development/guides/packaging-simple#PY005): Has tests folder
- [`PY006`](https://learn.scientific-python.org/development/guides/style#PY006): Has pre-commit config
- [`PY007`](https://learn.scientific-python.org/development/guides/tasks#PY007): Supports an easy task runner (nox, tox, pixi, etc.)
- [`PY008`](https://learn.scientific-python.org/development/guides/packaging-simple#PY008): Has a .gitignore file

### PyProject

- [`PP002`](https://learn.scientific-python.org/development/guides/packaging-simple#PP002): Has a proper build-system table
- [`PP003`](https://learn.scientific-python.org/development/guides/packaging-classic#PP003): Does not list wheel as a build-dep
- [`PP004`](https://learn.scientific-python.org/development/guides/packaging-simple#PP004): Does not upper cap Python requires
- [`PP005`](https://learn.scientific-python.org/development/guides/packaging-simple#PP005): Using SPDX project.license should not use deprecated trove classifiers
- [`PP006`](https://learn.scientific-python.org/development/guides/packaging-simple#PP006): The dev dependency group should be defined
- [`PP301`](https://learn.scientific-python.org/development/guides/pytest#PP301): Has pytest in pyproject
- [`PP302`](https://learn.scientific-python.org/development/guides/pytest#PP302): Sets a minimum pytest to at least 6 or 9
- [`PP303`](https://learn.scientific-python.org/development/guides/pytest#PP303): Sets the test paths
- [`PP304`](https://learn.scientific-python.org/development/guides/pytest#PP304): Sets the log level in pytest
- [`PP305`](https://learn.scientific-python.org/development/guides/pytest#PP305): Specifies strict xfail
- [`PP306`](https://learn.scientific-python.org/development/guides/pytest#PP306): Specifies strict config
- [`PP307`](https://learn.scientific-python.org/development/guides/pytest#PP307): Specifies strict markers
- [`PP308`](https://learn.scientific-python.org/development/guides/pytest#PP308): Specifies useful pytest summary
- [`PP309`](https://learn.scientific-python.org/development/guides/pytest#PP309): Filter warnings specified

### GitHub Actions

- [`GH100`](https://learn.scientific-python.org/development/guides/gha-basic#GH100): Has GitHub Actions config
- [`GH101`](https://learn.scientific-python.org/development/guides/gha-basic#GH101): Has nice names
- [`GH102`](https://learn.scientific-python.org/development/guides/gha-basic#GH102): Auto-cancel on repeated PRs
- [`GH103`](https://learn.scientific-python.org/development/guides/gha-basic#GH103): At least one workflow with manual dispatch trigger
- [`GH104`](https://learn.scientific-python.org/development/guides/gha-wheels#GH104): Use unique names for upload-artifact
- [`GH105`](https://learn.scientific-python.org/development/guides/gha-basic#GH105): Use Trusted Publishing instead of token-based publishing on PyPI
- [`GH200`](https://learn.scientific-python.org/development/guides/gha-basic#GH200): Maintained by Dependabot
- [`GH210`](https://learn.scientific-python.org/development/guides/gha-basic#GH210): Maintains the GitHub action versions with Dependabot
- [`GH211`](https://learn.scientific-python.org/development/guides/gha-basic#GH211): Do not pin core actions as major versions
- [`GH212`](https://learn.scientific-python.org/development/guides/gha-basic#GH212): Require GHA update grouping

### MyPy

- [`MY100`](https://learn.scientific-python.org/development/guides/style#MY100): Uses a type checker (pyproject config)
- [`MY101`](https://learn.scientific-python.org/development/guides/style#MY101): Type checker in strict mode
- `MY102`: MyPy show_error_codes deprecated
- [`MY103`](https://learn.scientific-python.org/development/guides/style#MY103): MyPy warn unreachable
- [`MY104`](https://learn.scientific-python.org/development/guides/style#MY104): MyPy enables ignore-without-code
- [`MY105`](https://learn.scientific-python.org/development/guides/style#MY105): MyPy enables redundant-expr
- [`MY106`](https://learn.scientific-python.org/development/guides/style#MY106): MyPy enables truthy-bool

### Nox

Will not show up if no `noxfile.py` file is present.

- [`NOX101`](https://learn.scientific-python.org/development/guides/tasks#NOX101): Sets minimum nox version
- [`NOX102`](https://learn.scientific-python.org/development/guides/tasks#NOX102): Sets venv backend
- [`NOX103`](https://learn.scientific-python.org/development/guides/tasks#NOX103): Set default per session instead of session list
- [`NOX201`](https://learn.scientific-python.org/development/guides/tasks#NOX201): Set a script block with dependencies in your noxfile
- [`NOX202`](https://learn.scientific-python.org/development/guides/tasks#NOX202): Has a shebang line
- [`NOX203`](https://learn.scientific-python.org/development/guides/tasks#NOX203): Provide a main block to run nox

### Pre-commit

Will not show up if using lefthook instead of pre-commit/prek.

- [`PC100`](https://learn.scientific-python.org/development/guides/style#PC100): Has pre-commit-hooks
- [`PC110`](https://learn.scientific-python.org/development/guides/style#PC110): Uses black or ruff-format
- [`PC111`](https://learn.scientific-python.org/development/guides/style#PC111): Uses blacken-docs
- [`PC140`](https://learn.scientific-python.org/development/guides/style#PC140): Uses a type checker
- [`PC160`](https://learn.scientific-python.org/development/guides/style#PC160): Uses a spell checker
- [`PC170`](https://learn.scientific-python.org/development/guides/style#PC170): Uses PyGrep hooks (only needed if rST present)
- [`PC180`](https://learn.scientific-python.org/development/guides/style#PC180): Uses a markdown formatter
- [`PC190`](https://learn.scientific-python.org/development/guides/style#PC190): Uses a linter (Ruff/Flake8)
- [`PC191`](https://learn.scientific-python.org/development/guides/style#PC191): Ruff show fixes if fixes enabled
- [`PC192`](https://learn.scientific-python.org/development/guides/style#PC192): Ruff uses `ruff-check` instead of `ruff` (legacy)
- [`PC901`](https://learn.scientific-python.org/development/guides/style#PC901): Custom pre-commit CI update message
- [`PC902`](https://learn.scientific-python.org/development/guides/style#PC902): Custom pre-commit CI autofix message
- [`PC903`](https://learn.scientific-python.org/development/guides/style#PC903): Specified pre-commit CI schedule

### ReadTheDocs

Will not show up if no `.readthedocs.yml`/`.readthedocs.yaml` file is present.

- [`RTD100`](https://learn.scientific-python.org/development/guides/docs#RTD100): Uses ReadTheDocs (pyproject config)
- [`RTD101`](https://learn.scientific-python.org/development/guides/docs#RTD101): You have to set the RTD version number to 2
- [`RTD102`](https://learn.scientific-python.org/development/guides/docs#RTD102): You have to set the RTD build image
- [`RTD103`](https://learn.scientific-python.org/development/guides/docs#RTD103): You have to set the RTD python version
- [`RTD104`](https://learn.scientific-python.org/development/guides/docs#RTD104): You have to specify a build configuration now for readthedocs.

### Ruff

- [`RF001`](https://learn.scientific-python.org/development/guides/style#RF001): Has Ruff config
- [`RF002`](https://learn.scientific-python.org/development/guides/style#RF002): Target version must be set
- [`RF003`](https://learn.scientific-python.org/development/guides/style#RF003): src directory doesn't need to be specified anymore (0.6+)
- [`RF101`](https://learn.scientific-python.org/development/guides/style#RF101): Bugbear must be selected
- [`RF102`](https://learn.scientific-python.org/development/guides/style#RF102): isort must be selected
- [`RF103`](https://learn.scientific-python.org/development/guides/style#RF103): pyupgrade must be selected
- `RF201`: Avoid using deprecated config settings
- `RF202`: Use (new) lint config section

### Security

- [`SEC001`](https://learn.scientific-python.org/development/guides/security#SEC001): Use zizmor to check the GitHub Actions

### Setuptools Config

Will not show up if no `setup.cfg` file is present.

- [`SCFG001`](https://learn.scientific-python.org/development/guides/packaging-classic#SCFG001): Avoid deprecated setup.cfg names

<!-- [[[end]]] -->

[repo-review]: https://repo-review.readthedocs.io
[repo-review configuring]: https://repo-review.readthedocs.io/en/latest/intro.html#configuring
[scientific-python development guide]: https://learn.scientific-python.org/development
[scientific-python/cookie]: https://github.com/scientific-python/cookie
[scikit-hep]: https://scikit-hep.org
[conda-badge]: https://img.shields.io/conda/vn/conda-forge/sp-repo-review
[conda-link]: https://github.com/conda-forge/sp-repo-review-feedstock

<!-- rumdl-enable MD013 -->