# syncstorage-rs

Firefox Sync storage and token server, written in Rust. The workspace contains a
Rust multi-crate server plus Python utilities and integration/end-to-end tests.

## Agent rules

Security takes absolute precedence. This repository handles synchronization of
user data and token management to access storage nodes.

### 1) Scope & writes

- Operate strictly within this repo root; normalize paths; do not follow
  symlinks outside the repo.
- Writes are allowed to the working tree. Always present a diff for review
  before any staging or commit.
- Do not modify files adjacent to a requested change. Mention issues found
  but do not fix them unless asked.
- Ask before running any command (build, test, install, DB ops, git,
  service management). Do not run `git add`, `git commit`, `git push`, or
  `git rebase` unless explicitly instructed.

### 2) Non-negotiables

- **Secrets:** never read, print, summarize, or transmit secret files or
  values. Use placeholders (`YOUR_API_KEY_HERE`) in examples.
- **External network:** only with explicit approval to a trusted, documented
  endpoint.
- **Pipelines & contracts:** flag breaking API or contract changes. Do not
  alter GitHub Actions defined in `.github/`, any configuration for CI/CD workflows or git hooks without explicit, reviewed
  justification.
- **Published DB migrations:** never edit existing published migration files or schema definitions (ex. `syncstorage-spanner/src/schema.ddl`), whether for MySQL, Postgres, or Spanner.
  Always add a new forward migration and a separate rollback migration.
- **Workspace recommendations:** the repo deliberately ships no
  `.vscode/extensions.json`. Do not re-add one. Useful extensions are listed
  in `.vscode/README.md` for contributors to install at their own discretion.

### 3) Do-not-touch paths

Never read or write these paths. This list mirrors `.gitignore` — if there is
a discrepancy, `.gitignore` is the source of truth.

```
service-account.json      # GCP credentials
.sentryclirc              # Sentry auth token
.envrc                    # direnv secrets
config/local.toml         # local server config (may contain secrets)
tools/tokenserver/loadtests/*.pem
tools/tokenserver/loadtests/*.pub
target/                   # Rust build artifacts
*.xml                     # test result files
openapi.json              # generated artifact
book/                     # generated docs
docs/output/              # generated docs
venv/                     # Python virtual environments
.install.stamp            # Poetry install marker
workspace/                # CI artifact directory
```

### 4) Git commit messages

Follow the commit message format defined in [CONTRIBUTING.md](CONTRIBUTING.md) Git Commit Guidelines.

- Format: `type(scope): subject` — `type` is one of `feat`, `fix`, `docs`,
  `style`, `refactor`, `perf`, `test`, `chore`
- Subject: imperative present tense, no capital first letter, no trailing period
- Body: explain motivation and contrast with previous behavior
- Footer: `Closes STOR-1234` (Mozilla engineers — matches the Jira ticket) or
  `Closes #N` / `Issue #N` (community contributors — matches the GitHub issue number);
  `BREAKING CHANGE:` for breaking changes
- Branch naming (Mozilla engineers): `type/description-STOR-1234` where `1234` must match the numeric suffix of the associated Jira ticket
- Branch naming (community): `type/description-1112`
- All commits must be GPG/SSH signed (enforced by CI)

Example:

```
feat: add node reassignment on capacity overflow

Nodes at 100% capacity previously returned 503 rather than
assigning the user to an available node.

Closes #42.
```

## Project layout

```
syncserver/            Main Actix-web server (entry point)
syncstorage-db/        Syncstorage database abstraction layer
syncstorage-db-common/ Syncstorage shared database types and traits
syncstorage-mysql/     MySQL backend for syncstorage
syncstorage-postgres/  PostgreSQL backend for syncstorage
syncstorage-spanner/   Google Cloud Spanner backend for syncstorage
syncstorage-settings/  Syncstorage configuration types
tokenserver-db/        Tokenserver database abstraction layer
tokenserver-db-common/ Tokenserver shared database types
tokenserver-mysql/     MySQL backend for tokenserver
tokenserver-postgres/  PostgreSQL backend for tokenserver
tokenserver-auth/      HAWK token generation and verification
tokenserver-common/    Shared tokenserver utilities
tokenserver-settings/  Tokenserver configuration types
syncserver-common/     Shared server utilities
syncserver-db-common/  Shared database utilities
syncserver-settings/   Syncserver configuration types
glean/                 Glean telemetry crate
tools/
  integration_tests/   Pytest-based integration + e2e tests
  tokenserver/         Tokenserver utilities and load tests
  spanner/             Spanner utilities
  hawk/                HAWK token generation utility
  syncstorage-loadtest/ Molotov load tests
docker/                Docker Compose files for each backend
config/                Local TOML configuration files
docs/                  mdBook documentation source
.github/               GitHub CI Actions and configuration
```

## Rust builds

The default feature set targets MySQL, however production uses the Spanner database with MySQL Tokenserver. Backend features are mutually exclusive.

```bash
# MySQL (default)
cargo build

# PostgreSQL
cargo build --no-default-features --features=syncstorage-db/postgres --features=tokenserver-db/postgres --features=py_verifier

# Spanner
cargo build --no-default-features --features=syncstorage-db/spanner --features=py_verifier
```

## Rust tests

Tests require `RUST_TEST_THREADS=1` (many tests share a real database).

```bash
# MySQL unit tests (default DATABASE_URL envs set in Makefile)
make test

# Specific package
make test ARGS="--package syncstorage-db"

# With coverage
make test_with_coverage          # MySQL
make postgres_test_with_coverage
make spanner_test_with_coverage
```

## Clippy / formatting / audit

Formatting checks are necessary before applying any code changes, as they
can capture problematic patterns or compiler issues in advance.

```bash
cargo fmt -- --check             # format check
cargo fmt                        # auto-format

make clippy_mysql
make clippy_postgres
make clippy_spanner

make clippy_release_mysql        # release-mode (catches dead code etc.)
make clippy_release_postgres
make clippy_release_spanner

cargo audit                      # check dependencies for known CVEs (run by CI on every push)
```

After editing Rust code, run clippy and the unit tests for the affected backend before finishing the task; the `rust-code-smell` skill covers changed files across backends.

## Python tooling

All Python tooling uses [Poetry](https://python-poetry.org/). The root
`pyproject.toml` covers the `tools/` tree. Each subdirectory under `tools/` has
its own `pyproject.toml` for its specific dependencies.

All of these Python checks must be run with any related change to a Python
utility or test.

```bash
make install                     # install root dependencies
make integration-test            # install integration test dependencies
make tokenserver                 # install tokenserver utility dependencies
```

Lint / format / type-check:

```bash
make ruff-lint                   # ruff check tools/
make ruff-format                 # ruff format tools/
make ruff-fmt-chk                # format diff check
make mypy                        # mypy type checking
make pydocstyle                  # docstring validation
make bandit                      # security linting
```

## Integration / E2E tests

### Local (against a running server)

```bash
# Run all integration tests except real FxA e2e
PYTHONPATH=$(pwd)/tools \
SYNC_MASTER_SECRET=secret0 \
SYNC_TOKENSERVER__FXA_OAUTH_SERVER_URL=http://localhost:6000 \
TOKENSERVER_HOST=http://localhost:8000 \
poetry -C tools/integration_tests run pytest . --ignore=tokenserver/test_e2e.py
```

Or via the Makefile shortcut (sets the env vars above):

```bash
make run_local_e2e_tests
```

### Docker-based E2E (full stack)

```bash
make docker_run_mysql_e2e_tests
make docker_run_postgres_e2e_tests
make docker_run_spanner_e2e_tests
```

Each command spins up two compose stacks (with and without JWK cache), copies
JUnit XML results out of the container, then tears everything down.

## Running the server locally

```bash
# MySQL
make run_mysql

# Spanner (requires GCP service account key at ./service-account.json)
make run_spanner
```

Server listens on `http://localhost:8000`. Health check: `GET /__heartbeat__`.

## Key environment variables

| Variable | Purpose |
|---|---|
| `SYNC_MASTER_SECRET` | HAWK token signing secret |
| `SYNC_SYNCSTORAGE__DATABASE_URL` | Syncstorage DB connection string |
| `SYNC_TOKENSERVER__DATABASE_URL` | Tokenserver DB connection string |
| `SYNC_TOKENSERVER__NODE_TYPE` | `mysql` / `postgres` / `spanner` |
| `SYNC_TOKENSERVER__RUN_MIGRATIONS` | Auto-migrate on startup (`true`/`false`) |
| `SYNC_TOKENSERVER__FXA_OAUTH_SERVER_URL` | FxA OAuth endpoint |
| `RUST_TEST_THREADS` | Set to `1` for all test runs |
| `RUST_LOG` | Log level (`debug`, `info`, etc.) |
| `SYNC_TEST_LOG_HTTP` | Set any value to log all HTTP req/resp in integration tests |
| `SYNC_SERVER_URL` | Integration test base URL (default: `http://localhost:8000`) |
| `TOKENSERVER_HOST` | Tokenserver host for integration tests |
| `PYTHONPATH` | Must point to `$(pwd)/tools` when running integration tests |

## Docker Compose files

| File | Purpose |
|---|---|
| `docker/docker-compose.mysql.yaml` | MySQL syncstorage + MySQL tokenserver |
| `docker/docker-compose.postgres.yaml` | PostgreSQL syncstorage + PostgreSQL tokenserver |
| `docker/docker-compose.spanner.yaml` | Spanner emulator + MySQL tokenserver |
| `docker/docker-compose.e2e.{mysql,postgres,spanner}.yaml` | E2E test runner overlay |
| `docker/docker-compose.e2e.jwk-cache.yaml` | Enable JWK caching overlay |
| `docker/docker-compose.e2e.no-jwk-cache.yaml` | Disable JWK caching overlay |

## Documentation

```bash
make doc-install-deps            # install mdBook + mdBook-mermaid
make doc-watch                   # live preview at localhost
make doc-prev                    # build and serve
make doc-test                    # validate markdown

make api-prev                    # generate OpenAPI spec + serve Swagger UI on :8080
```


## Test structure

- `tools/integration_tests/conftest.py` — pytest fixtures only (`st_ctx`)
- `tools/integration_tests/helpers.py` — retry helpers, auth state, `switch_user`
- `tools/integration_tests/test_storage.py` — storage protocol tests
- `tools/integration_tests/tokenserver/conftest.py` — tokenserver fixtures only
- `tools/integration_tests/tokenserver/helpers.py` — tokenserver DB/auth helpers
- `tools/integration_tests/tokenserver/test_*.py` — tokenserver tests

## MCP integrations

Two MCP servers are configured for this repository: **Atlassian Jira** (`mcp__atlassian__`) and **Sentry** (`mcp__sentry__`).

Use the skills below to query them. Do not call MCP tools ad-hoc without reading the shared context first.

| Skill | Invocation | Purpose |
|---|---|---|
| Jira status | `/jira-status [me\|epic KEY\|epics\|sprint\|summary ...\|create ...\|update STOR-#### ...]` | Task landscape, epics, personal view, summaries, create/edit tickets |
| Sentry health | `/sentry-health [new\|regressions\|volume\|full]` | Production error trends, regressions, new issues |

**Shared context:** `.claude/mcp-context.md` contains project IDs, JQL patterns, Sentry project slug, what to flag, and known noise. Skills read this at runtime. Read it before any ad-hoc MCP query.

**Identity when using these tools:** You are acting as both engineer and engineering manager. Surface patterns, not just raw lists. Distinguish "needs action now" from "worth monitoring". Do not fabricate data — if a query returns nothing, say so.

**Jira project:** `STOR` — `https://mozilla-hub.atlassian.net`
**Sentry project:** `syncstorage-prod` — org `mozilla`