# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

Laminar is an open-source observability platform for AI agents: OpenTelemetry-native tracing, evaluations, AI monitoring, and SQL access to all data.

## Repository Structure

Multi-service monorepo:

- **app-server/** — Rust backend (Actix-web HTTP, Tonic gRPC). SQL query validation + JSON↔SQL conversion run in-process (`src/query_engine/`, built on `sqlparser`).
- **frontend/** — Next.js/TypeScript web UI.
- **pii-redactor/** — optional standalone Rust gRPC service running a HuggingFace PII model on CPU via ONNX Runtime. See `pii-redactor/README.md`.

Some features (Signals evaluation, Laminar Agent, clustering) are enterprise-only and live in the private `lmnr-private` repo behind the `signals` cargo feature; OSS ships stubs and public scaffolding for them. Don't document them as OSS features.

## Development Commands

### Frontend (Next.js)

```bash
cd frontend
pnpm install        # Install dependencies
pnpm run dev        # Start dev server (Turbopack)
pnpm lint           # oxlint (lint:fix to auto-fix)
pnpm format:write   # oxfmt
pnpm type-check     # TypeScript type checking
pnpm test           # Run tests (tsx --test tests/**/*.test.ts)
pnpm build          # Production build
```

- The sandbox's global pnpm is v10 but the lockfile is written by pnpm 12 (`.github/workflows/fe-build-check.yml` pins 12.14.0, `frontend/Dockerfile` 12.8.1). `pnpm add` on v10 rewrites hundreds of `(supports-color@…)` peer suffixes. Add or bump deps with `npx -y pnpm@12.14.0 add <pkg> --lockfile-only`, then `npx -y pnpm@12.14.0 install --frozen-lockfile` to sync `node_modules`.
- **Dependency build scripts are denied.** Every `allowBuilds` entry in `frontend/pnpm-workspace.yaml` stays `false` unless the package does not work without its script (supply-chain risk). A new dep with a build script fails `pnpm install` until it is listed. Policy and review steps: `docs/internal/frontend-dependencies.md`.
- Format only the files you changed: `npx oxfmt --write <files>`. `prettier` is present in `node_modules` (transitive dep) but is NOT the project formatter — running it reformats files oxfmt then flags in the pre-commit hook.
- In a fresh checkout, `pnpm type-check` (and the husky pre-commit hook) fails with `TS2307: Cannot find module '@/assets/...svg'` errors — `next-env.d.ts` is gitignored. Fix: `npx next typegen` (or any `next dev`/`next build` run).
- `tsconfig.json` sets `"incremental": true`, so a bare `npx tsc --noEmit` can report **zero errors on files it skipped** and give a false green. When verifying a type fix, run `npx tsc --noEmit --incremental false` (the pre-commit hook does a full check and will catch what you missed otherwise). On an 8 GB box the full check OOMs at Node's default heap; prefix `NODE_OPTIONS=--max-old-space-size=8192`.
- `pnpm test` is gated in CI by `.github/workflows/fe-test.yml` on any PR touching `frontend/**`, so it must stay fully green. A red test on a clean checkout is a real bug, not noise: check the source against the test's intent before editing the assertion.
- **A stale `node_modules` fakes ~47 "pre-existing" type errors.** An installed tree older than `pnpm-lock.yaml` (e.g. recharts 2.15.4 where the lock pins 3.10.1) makes `tsc --noEmit` and `pnpm build` fail on `BarStack` / `MouseHandlerDataParam` / `useYAxisScale` / `<YAxis width="auto">` across `chart-builder/`, `charts/`, `evaluation/`, `landing/`. They look exactly like the recharts v2→v3 debt they are not: `pnpm install --frozen-lockfile` clears all of them. Verify the install matches the lock before writing any of it off as pre-existing.
- **Turbopack is the Next 16 default for both commands; `build` opts out with `--webpack`, `dev` does not.** Turbopack's production output miscompiled chunks (`module factory is not available` on client navigation). So both bundler blocks in `next.config.ts` are live, and a production repro needs `pnpm build`, not a bare `next build`. `next dev` generates `frontend/AGENTS.md` on every run; it is gitignored (do not commit it).
- `frontend/.dockerignore` excludes only `node_modules` and `.next`, so a local `docker build frontend/` ships `.env.local` into the builder and `next build` inlines its `NEXT_PUBLIC_*` values. For a CI-faithful image, build from a copy without it (`tar --exclude=./node_modules --exclude=./.next --exclude=./.env.local -cf - . | tar -xf - -C /tmp/ctx`). The sandbox has no Docker by default: `apt-get install -y docker.io` then start `dockerd` in the background works.

### Backend (Rust)

```bash
cd app-server
cargo r                    # Run in development mode
cargo build --release      # Production build
cargo test -- --nocapture  # Run tests
```

- The `aws-*` crates in `Cargo.lock` require **rustc ≥ 1.94.1**; on 1.94.0 `cargo check` fails during resolution ("requires rustc 1.94.1") before compiling anything — `rustup update stable`.
- `cargo check --features signals` and `cargo fmt` on `main.rs` both fail in OSS — the `signals` feature gates modules that live only in `lmnr-private`. Default-feature `cargo check` is the real gate; format leaf files individually with `rustfmt --edition 2024 <file>`. Full stub workaround list: `docs/internal/app-server.md`.
- NEVER run `cargo fmt`, even as `cargo fmt -- <file>` (the file arg does NOT scope it). The tree is not rustfmt-clean at HEAD, so it rewrites ~40 unrelated files and buries the real diff. Use `rustfmt --edition 2024 <file>` on the files you changed.
- `cargo test --lib` fails with "no library targets found" — `app-server` is a binary crate. Use `cargo test --bin app-server <filter>`; the filter takes a single path prefix, not a list.

## Local Development Setup

```bash
cp app-server/.env.example .env   # app-server reads .env at the repo root (dotenv)
cp frontend/.env.local.example frontend/.env.local
```

**PostgreSQL** and **ClickHouse** are required. Everything else degrades gracefully: RabbitMQ → in-memory queue, Redis → in-memory cache, Quickwit → search disabled, S3 → MockStorage.

Docker-based:

```bash
# Frontend-only (pre-built app-server image):
docker compose -f docker-compose-local-dev.yml up
cd frontend && pnpm run dev

# Full-stack (all dependencies, run app-server + frontend yourself):
docker compose -f docker-compose-local-dev-full.yml up
cd app-server && cargo r      # Terminal 1
cd frontend && pnpm run dev   # Terminal 2
```

**Gotcha:** `dotenv` does NOT override already-exported env vars. A shell that exports `PORT` silently breaks the app-server HTTP listener (bind error is swallowed) and `next dev` (`EADDRINUSE`). Launch with explicit `PORT=8000 cargo r` / `PORT=3000 pnpm run dev`, or `env -u PORT`. Details: `docs/internal/app-server.md`.

## Architecture

```
Frontend (5667 prod / 3000 local-dev) ────┐
                                          │
App Server                                │
├─ REST API (8000) ◄──────────────────────┘
├─ gRPC ingestion (8001) ◄─── SDK traces
└─ Realtime SSE (8002)
   │
   ├──► PostgreSQL (5433)  - main database        [required]
   ├──► ClickHouse (8123)  - analytics/spans      [required]
   ├──► RabbitMQ (5672)    - async processing     [optional]
   └──► Quickwit (7280/81) - full-text search     [optional]
```

## Database Migrations

Schema is managed with Drizzle ORM; the database itself is the source of truth — do NOT edit schema files directly.

```bash
cd frontend
pnpm db:generate   # generate migrations AND strip "public". qualifiers (required)
# Migrations are applied automatically on frontend startup
```

- Migration SQL must stay schema-neutral (no `"public".` qualifiers) — `pnpm db:generate` handles the strip; if you generate by hand, run `pnpm db:strip-schema` after.
- Hand-written migrations also need a `meta/NNNN_snapshot.json` or the next generate produces a duplicate migration.
- ClickHouse migrations (`frontend/lib/clickhouse/migrations/`) run once and are checksummed — NEVER modify an applied migration file; always add a new numbered one.
- Full details (drizzle-kit quirks, snapshots, `POSTGRES_SCHEMA`): `docs/internal/database.md`.

## Comments

Comments are welcome when they add a WHY that names cannot: a constraint, invariant, or workaround. Keep them to a line or two. Skip comments that restate the next lines, and skip changelog notes ("previously X, now Y") — describe the current code, not the diff. Longer design notes belong in `docs/internal/`.

```rust
// Exclusive parks happen after admission; a wait_count>0 wake already owns the claim.
```

## App-server conventions

- Every env var is registered in `app-server/src/env/` (typed `NumEnv`/`StringEnv`/`BoolEnv` descriptors) — never inline a string-literal env name at a call site.
- `mod env` shadows `std::env`: inside files with `use crate::env;`, write `std::env::var(...)` fully qualified.
- Backend `Feature` flags are fine-grained — one flag per feature; never gate a new feature on another feature's flag.
- More: `docs/internal/app-server.md`, and `docs/internal/rust-best-practices.md` for reuse/layering/scoping rules.

## Frontend conventions

- One component per file; keep components <150 lines; related components in a folder with `index.tsx`.
- Complex state belongs in a Zustand store (with `shallow` selectors); use nuqs for URL param state — never sync URL params into a store via `useEffect`.
- Client fetches to our own `/api/...`: use `fetchApi` (`@/lib/api/fetch-api`) and toast on `error`; raw `fetch` only for streaming, non-JSON, or external calls. API routes: wrap with `apiHandler` (`@/lib/api/api-handler`), no `try/catch`; throw `ApiError` subclasses (`@/lib/api/errors`) for expected 4xx; `ZodError` → 400, anything else → 500 + Sentry, always `{ error }` JSON. App-server-only `/api/internal` routes use `internalApiHandler` (`@/lib/api/internal`). Use `AbortController` for superseded in-flight fetches.
- Recharts is on v3 (`^3.10.1`). `CategoricalChartFunc` is defined from `MouseHandlerDataParam` in `components/charts/time-series-chart/types.ts` (the v2 `recharts/types/chart/generateCategoricalChart` path is gone). Use `<YAxis width="auto">` and `<BarStack>` for stacked rounded bars — do not reintroduce a custom bar `shape`.
- New data tables MUST follow the `InfiniteDataTable` split pattern (index/contents/controls/constants). Full patterns: `docs/internal/frontend-best-practices.md`.

## Key Technical Details

- **Rust edition**: 2024
- **Node version**: 26 (`frontend/Dockerfile` is `node:26-alpine`)
- **Package managers**: Cargo (Rust), pnpm (frontend)
- **Git workflow**: Submit PRs to `dev` branch, which merges to `main` periodically

## Pre-commit Hooks

Frontend uses Husky with lint-staged: oxfmt, oxlint, a circular-import check, and `tsc --noEmit` run on staged files. Config lives in `frontend/.oxfmtrc.json` and `frontend/.oxlintrc.json`. `✖ Task failed to spawn: oxfmt … ENOENT` means `frontend/node_modules` is partial, and the commit was NOT made: run `pnpm install --frozen-lockfile` in `frontend/` and commit again. If type-check fails on pre-existing SVG/PNG asset-import errors, verify your own files are clean (`npx tsc --noEmit 2>&1 | grep "your-file"`) before using `--no-verify`.

## Detailed topic notes (read on demand)

`docs/internal/` holds detailed, hard-won working notes. **Before working in one of these areas, read the matching file** — it documents invariants, security boundaries, and gotchas that are not inferable from the code:

| File | Read when touching |
|---|---|
| `docs/internal/rust-best-practices.md` | Any app-server change — reuse, layering, db/cache scoping, named types, error retryability |
| `docs/internal/database.md` | Postgres migrations, `POSTGRES_SCHEMA`, name-sort collation |
| `docs/internal/sql-query-engine.md` | `query_engine/` validator (a security boundary), SQL editor schema/autocomplete, `/v1/sql/query` guards, rate limiting |
| `docs/internal/clickhouse-traces.md` | `traces_agg`/`traces_static`/`traces_v0`, spans query scoping, trace aggregation, async-insert tuning, traces-table filters, project data deletion |
| `docs/internal/dedup-search.md` | `unique_content` group-scoped dedup (`traces/dedup/`), `spans_v0` reconstruction, Quickwit indexing/search |
| `docs/internal/ingestion.md` | OTLP `/v1/traces`, GenAI semconv parsing, trace metadata patches, input/output extraction, system-prompt extraction, system-prompt / user-template versioning, agent versioning (`agent_versioning/`), 413s |
| `docs/internal/observability.md` | App-server self-tracing, Sentry layers/sampling |
| `docs/internal/mq-redis.md` | RabbitMQ queues + streams transport, Redis resilient connections, readiness probes |
| `docs/internal/audit.md` | Workspace audit log (`audit_events`), `handlerWithAudit`/`handlerNoAudit` route wrappers, `/v1` audit middleware, `sql.query` events, `SqlQuerySource` |
| `docs/internal/auth.md` | Better Auth, OAuth providers, CLI device-flow auth, project API keys |
| `docs/internal/rbac.md` | `piiMode` (off/redact/dual), PII masks (`*_masks`, `pii_checked`, redactor contract), `spans_v1`/`traces_v1` policy param, `AccessPolicy` (`maskPii`, `traceFilters`), `trace_access_policy_dict`, SQL route `actor`, role-based PII masking |
| `docs/internal/billing.md` | Tiers, Starter free trial, usage warnings/hard limits, signal cost metering, custom model costs |
| `docs/internal/signals.md` | Signals, alerts, signal events, CLI CRUD (`trigger`/`filters`/`mode`) |
| `docs/internal/slack.md` | Slack OAuth broker + brokered self-hosted integration |
| `docs/internal/ai-features.md` | `getLanguageModel`, LLM provider config, `llm_feature_routes` (per-feature model routing), Vercel AI SDK v7 |
| `docs/internal/analytics.md` | PostHog, Loops sync, self-hosted telemetry heartbeat |
| `docs/internal/labeling-queues.md` | Labeling queues (ClickHouse RMT items, dirty-state, push-to-dataset) |
| `docs/internal/debugger.md` | Debugger replay cache, debugger session blocks/timeline |
| `docs/internal/frontend-tables.md` | Data-table filters, advanced search, saved views, evaluations page, dashboards |
| `docs/internal/frontend-trace-view.md` | Trace view, span rendering/message parsing, ContentRenderer, rrweb replay |
| `docs/internal/frontend-app.md` | Settings pages, onboarding, base-path serving, render templates, SSE proxy routes, landing page, agent stubs |
| `docs/internal/frontend-best-practices.md` | Any new frontend component/table/store work |
| `docs/internal/frontend-dependencies.md` | Adding/bumping frontend deps, `pnpm-workspace.yaml` `allowBuilds` (build-script policy) |
| `docs/internal/app-server.md` | App-server env vars, local dev env quirks, signals-feature build stubs, TLS/crypto-backend deps |
| `docs/internal/hosted-evals.md` | Hosted Harbor evals (`/v1/hosted-evals/*`, `src/hosted_evals/`): sandbox provider abstraction (Daytona), image push to ECR, remote-run orchestrator |
| `docs/internal/oss-release.md` | `.github/workflows/build-push.yml` (GHCR images, attestations, release-source gate) |
| `docs/internal/backend-ci.md` | `.github/workflows/backend-build-and-test.yml`, `app-server/Dockerfile` (branch-scoped Actions cache, cargo-chef cook layers, CI timing baseline) |