# Contributing to Engram

Have a question or want to talk through an idea before you start? Join us on [Discord](https://discord.gg/G8fjjGswdc) — it's the quickest way to reach the maintainers and other contributors.

Found a bug or have a feature idea but don't plan to work on it yourself? Run `/bug` or `/feature` in the Discord server and **Trace**, the support bot, will help turn it into a GitHub issue. See [Getting Help & Reporting Issues](https://jsakkos.github.io/engram/getting-help/).

## Local development setup

### Prerequisites

- Python 3.11
- Node.js 24
- [uv](https://github.com/astral-sh/uv) for Python dependency management
- MakeMKV with a valid license (for end-to-end work with real discs)
- Pre-commit framework: `pipx install pre-commit` or `uv tool install pre-commit`

### One-time setup after cloning

```bash
git clone https://github.com/Jsakkos/engram.git
cd engram

# Install pre-commit hooks into .git/hooks (runs ruff + ESLint on commit)
pre-commit install

# Backend
cd backend && uv sync && cd ..

# Frontend
cd frontend && npm install && cd ..
```

### Start the dev servers

Run the backend and frontend in separate terminals:

```bash
# Backend (API on port 8000)
cd backend
uv run uvicorn app.main:app
```

```bash
# Frontend (dashboard on port 5173)
cd frontend
npm run dev
```

Open <http://localhost:5173>. The Vite dev server proxies `/api` and `/ws` to the backend at `localhost:8000`.

> **Do not start the backend with `--reload`.** It spawns a child process with its own drive sentinel, which produces duplicate disc events.

#### With a Procfile runner

The root `Procfile` starts both servers with one command under a Procfile runner such as [foreman](https://github.com/ddollar/foreman), [overmind](https://github.com/DarthSim/overmind), or [honcho](https://github.com/nickstenning/honcho):

```bash
foreman start
```

Dependencies are installed from the lockfiles before the servers start: the backend runs `uv sync --locked`, and the frontend runs `npm ci` only when `package-lock.json` has changed since the last install (`scripts/ensure_frontend_deps.py`). Neither ever rewrites a lockfile. Both processes start at the same time, so on a first run the `uv sync` and `npm ci` output is interleaved.

> **Windows:** the `Procfile` commands run through `sh -c`, which native Windows shells don't provide. Run the Procfile runner from WSL or Git Bash, or use the two-terminal steps above.

- The backend starts with `DEBUG=true` by default, which enables the `/api/simulate/*` endpoints. Set `DEBUG=false` to turn them off.
- If the default ports are taken, set `VITE_BACKEND_PORT` (backend port and the frontend's proxy target) and `VITE_PORT` (dashboard port). A gitignored `.envrc` is a convenient place for these, as long as direnv exports it into the shell you start the runner from; the runners themselves read `.env`, not `.envrc`.
- Stopping the runner stops both servers. Check that no `uvicorn` or `makemkvcon` processes are left behind.

## Code quality

```bash
# Backend
cd backend
uv run ruff check .       # Lint
uv run ruff format .      # Format

# Frontend
cd frontend
npm run lint              # ESLint
npm run build             # TypeScript check + production build
```

Ruff config: line length 100, target Python 3.11, rules `E/F/I/UP/B`, double quotes.

## Running tests

```bash
# Backend — all tests
cd backend
uv run pytest

# Backend — by category
uv run pytest tests/unit/
uv run pytest tests/integration/
uv run pytest tests/pipeline/

# Frontend — E2E tests (requires backend with DEBUG=true)
cd frontend
npx playwright install    # first time only
npm run test:e2e
npm run test:e2e:ui       # interactive browser UI
```

See the [testing guide](https://jsakkos.github.io/engram/development/testing/) for detailed test categories, fixtures, and patterns.

## Pre-commit hooks

`.pre-commit-config.yaml` runs on every commit:

- **Backend**: `ruff check --fix` and `ruff format` on changed Python files in `backend/`
- **Frontend**: `eslint --max-warnings 0` on changed `*.ts`/`*.tsx`/`*.js`/`*.jsx` files in `frontend/`
- **Repo hygiene**: trailing whitespace, EOF newlines, YAML/TOML validity, merge-conflict markers, large-file guard (1 MB)

To run hooks against all files manually (useful after a rebase):

```bash
pre-commit run --all-files
```

To skip a commit for a genuine emergency:

```bash
git commit --no-verify -m "..."
```

Don't make a habit of it.

## CI pipeline

`.github/workflows/ci.yml` runs the following jobs on every PR and push to `main`. All jobs run in parallel unless `needs:` is specified.

| Job | What it checks | Runs on |
|-----|---------------|---------|
| `Backend Lint` | `ruff check` + `ruff format --check` | Linux |
| `Backend Tests (unit)` | `pytest tests/unit/` with coverage | Linux + Windows |
| `Backend Tests (integration)` | `pytest tests/integration/ tests/pipeline/` (after unit passes) | Linux |
| `Backend Smoke Test` | Imports `app.main:app` and probes `GET /api/jobs` | Linux |
| `Alembic Migration Roundtrip` | `upgrade head → downgrade base → upgrade head` | Linux |
| `Frontend Lint & Build` | ESLint + `tsc` + Vite production build + bundle-size budget | Linux |
| `Frontend Unit Tests` | Vitest with v8 coverage | Linux |
| `E2E Tests` | Playwright against a real backend+frontend (after Lint & Build) | Linux |
| `CodeQL Analyze` | Security/quality scan for Python and TypeScript | Linux |

Concurrency control cancels stale runs when you push fixups to a PR.

### Bundle size budget

Configured in `frontend/scripts/check-bundle-size.mjs`. Current thresholds (gzipped):

- Total JS: 600 KB
- Total CSS: 60 KB
- Largest single JS chunk: 350 KB

Edit the `BUDGETS_KB` constant in that file when you intentionally cross a threshold. Don't bump it casually — investigate first.

### Coverage

Backend coverage XML and frontend coverage HTML are uploaded as workflow artifacts on every run. To wire up Codecov for PR comments, add a `CODECOV_TOKEN` secret and a `codecov/codecov-action` step.

## Branch protection (configured in GitHub settings)

Under **Settings → Branches → Branch protection rules** for `main`:

- **Require a pull request before merging**: enabled
- **Require status checks to pass before merging**: enabled
  - Required checks:
    - `Backend Lint`
    - `Backend Tests (unit) (ubuntu-latest)`
    - `Backend Tests (unit) (windows-latest)`
    - `Backend Tests (integration)`
    - `Backend Smoke Test`
    - `Alembic Migration Roundtrip`
    - `Frontend Lint & Build`
    - `Frontend Unit Tests`
    - `E2E Tests`
- **Require branches to be up to date before merging**: enabled
- **Require linear history**: enabled (matches the single-commit-per-issue convention)
- **Require conversation resolution before merging**: enabled

## Git workflow

- Work on **feature branches**, never directly on `main`
- Branch naming: `fix/32-movie-track-state`, `feat/34-metadata-logging`
- One commit per issue/feature, referencing the issue number (e.g., `fix: correct DiscDB scan log URL (#124)`)
- Conventional-commit prefixes: `feat:`, `fix:`, `chore:`, `docs:`, `refactor:`, `test:`
- Open a PR to merge into `main` (linear history is enforced)

## Key conventions

- **Async everywhere** — use `async`/`await` for database, subprocess, and I/O operations
- **Singleton services** — `job_manager`, `ws_manager`, `curator` are module-level singletons
- **Error hierarchy** — use specific exceptions from `app/core/errors.py`, never bare `except`
- **State machine** — all job lifecycle is tracked through `JobState` transitions
- **Tailwind v4** — uses `@theme inline` blocks in CSS, not `tailwind.config.js`

## Dependency updates

Renovate (`.github/renovate.json`) opens PRs Monday mornings:

- **Patch + pin updates**: auto-merge after CI passes
- **Minor npm updates (non-0.x)**: auto-merge after CI passes
- **Major updates**: land in the dashboard for manual review

Install the [Renovate GitHub App](https://github.com/apps/renovate) on the repo to activate.

## Releases

Tag a commit with `v*` (e.g., `v0.7.0`) to trigger `.github/workflows/release.yml`:

1. Builds PyInstaller bundles for Windows, Linux, and macOS
2. Smoke-tests each built executable by starting it and hitting `/api/jobs`
3. Uploads `.zip` / `.tar.gz` artifacts to a GitHub Release with auto-generated notes

A failing smoke test blocks the release. Do not bypass.

## Acknowledging contributors

External contributors are credited automatically and durably:

- **Release notes** — every release body gets a **Contributors** section listing
  external contributors (first-timers flagged 🎉), generated by
  `backend/scripts/contributors.py --release-section` inside `release.yml`. No
  manual step.
- **PR thank-you** — `.github/workflows/contributor-welcome.yml` comments a
  thank-you when an external PR merges (extra-warm for first-timers).
- **Changelog inline credit (convention)** — when a changelog entry describes an
  external contribution, append `(#NNN, thanks @user!)`. This is curation, not a
  CI gate; the release-notes section is the can't-forget safety net.
- **Roster** — `CONTRIBUTORS.md` lists external contributors. Regenerate it as a
  step in each `chore: release vX.Y.Z` PR (a human commit — protected `main`
  rejects bot pushes), so it never needs a workflow push:

  ```bash
  python backend/scripts/contributors.py --roster --repo Jsakkos/engram > CONTRIBUTORS.md
  # (requires an authenticated `gh`; include the diff in the release PR)
  ```

The "who counts as external" rules (owner + bots excluded) live in one place —
`backend/scripts/contributors.py` — and are shared by the release section and the
roster.

## Visual regression baselines

`frontend/e2e/visual-regression.spec.ts` snapshots key pages. To generate or update baselines:

```bash
cd frontend
npx playwright test visual-regression --update-snapshots
git add e2e/visual-regression.spec.ts-snapshots/
```

Only update baselines when an intentional UI change requires it.

## Building the docs

Documentation is built with [MkDocs Material](https://squidfunk.github.io/mkdocs-material/) and auto-deploys to GitHub Pages on push to `main`.

```bash
pip install mkdocs-material "mkdocstrings[python]"
mkdocs serve
```

Edit files under `docs/`; they are picked up automatically. The root `CHANGELOG.md` and `CONTRIBUTING.md` are transcluded into the site, so edit those at the repo root rather than under `docs/`.