# Helix Foundry: agent guide The Helix Foundry site, in two builds: the hosted site, foundry.helix-db.com (branch `main`) and dev.foundry.helix-db.com (branch `dev`) on Vercel, and the local build that `local/` runs on one machine. A Next.js 16 app that serves three things: - the Foundry UI: a single-page app (react-router 7, TanStack Query), mounted client-only; - sign-in: helix-website's WFE-brokered WorkOS flow, run as the Foundry AuthKit application; - `/papi`: a same-origin BFF that forwards an explicit allowlist of WFE routes with the session's bearer token. Foundry's API is `backend/` (Backend, below): `helix.foundry.v1`, every route under `/v1/foundry` and `/v1/workspaces/{workspaceId}/foundry`. Hosted, the Helix WFE serves those routes beside its own, which the site also calls (whoami, workspaces, members, projects, tenants, billing). Their REST contract is vendored in two parts, Foundry's in `contract/foundry.swagger.json` and the WFE's own in `contract/wfe.swagger.json`, with the ontology agent's Read and Ask JSON Schemas, whose JSON Foundry carries as strings, in `contract/ontology_agent/` with their `VERSION`. `pnpm contract:sync` copies Foundry's contract and the schemas from `backend/`, and the WFE's from `WFE_REPO`, a checkout of hosted Helix's backend, which only Helix's maintainers have. Next.js 16 differs from older versions. Read the matching guide in `node_modules/next/dist/docs/` before using an API you have not used here. ## Commands | Command | Does | | --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `pnpm install` | Node 24, pnpm (version in `packageManager`) | | `pnpm dev:mock` | The app on http://localhost:3000 against the mock WFE on :4010. No accounts or secrets. Sign in with any provider; `ada@example.com` already has the Acme workspace, any other email starts with none. | | `pnpm dev:mock:local` | A local build (below) on http://localhost:3000 against the mock on :4010 playing its API: no sign-in, Ada owns Acme, every connector is open. | | `pnpm dev` | https://local.foundry.helix-db.com:3000 against the WFE in `.env.local` (copy `.env.example`). Needs mkcert and `127.0.0.1 local.foundry.helix-db.com` in `/etc/hosts`. | | `pnpm typecheck` | `tsc --noEmit` | | `pnpm lint` | ESLint, then `prettier --check` (`pnpm format` fixes formatting) | | `pnpm test` | Vitest unit tests in `tests/unit` | | `pnpm test:coverage` | The unit tests with coverage of `src/server`, `src/shared`, the browser API client and the areas' logic (`model.ts`, `plan.ts`) | | `pnpm e2e` | Playwright in `tests/e2e`: a production build in `.next-e2e` against the mock, on ports 3100 and 4110, and a local build in `.next-e2e-local` against its own mock (`tests/e2e/local`), on 3101 and 4111. Needs Chromium (`pnpm exec playwright install chromium`). | | `pnpm e2e:stack` | `tests/stack/run.sh`: the real local stack (`local/`), fresh, with a sample PostgreSQL database, driven in Playwright's image. Runs the site and foundry built from source (`helix-foundry/{site,foundry}:local`; it builds foundry's) and the release's ontology agent, or the images `FOUNDRY_*IMAGE` name. It starts as a fresh clone does, with nothing set up (the workspace the install makes, its secrets written into the stack's volume), and with no AI provider (`FOUNDRY_AI_PROVIDER=none`, so nothing on the machine is found instead), so uploads and the draft wait for one; `FOUNDRY_STACK_MODELS=1` makes Claude the install's provider with the shell's `ANTHROPIC_API_KEY`, uploads, waits for the draft, then loads it and asks. | | `pnpm docs:media` | The docs' screenshots and videos (`tests/docs`) into `docs-media/`. `pnpm docs:media:upload` publishes them to assets.helix-db.com, which docs.helix-db.com links. | Run all four checks (`typecheck`, `lint`, `test`, `e2e`) before you finish. CI (`.github/workflows/ci.yml`) runs the same. ## Layout | Path | Contents | | ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | | `src/app` | Next routes only: `auth/login`, `auth/logout`, `login/callback`, `papi/[...path]`, and `[[...slug]]`, which mounts the SPA client-only | | `src/server` | Server-only code: auth config, cookies, sessions and single-flight refresh, CSRF, the BFF handler and **the allowlist** (`papi-allowlist.ts`) | | `src/shared` | Used by both sides: `wfe.ts` (every WFE payload type), route templates, return paths, the CSRF contract | | `src/web` | The SPA: `main.tsx`, `api.ts`, `paths.ts`, `ui.tsx`, `kit/`, `shell/`, `areas//`, the sign-in pages and the global CSS | | `mock` | The mock WFE (`server.ts`), its state, tokens, the WorkOS stand-in and `routes/` | | `contract` | The vendored WFE swagger and the commit it came from; `ontology_agent/`, the agent's Read and Ask schemas | | `tests/unit`, `tests/e2e` | Vitest and Playwright | | `tests/docs` | The docs' captures: one spec per docs section, `support.ts` for screenshots, the recorded pointer and captions | ## Rules - **The allowlist.** The browser reaches the WFE only through routes listed in `src/server/papi-allowlist.ts`: one entry per method and path, written exactly as in the swagger, with `{param}` matching one path segment. No wildcards, no prefixes. Add only routes the UI calls. Never add sign-in, SSO management, role changes or member removal. `tests/unit/papi-allowlist.test.ts` fails unless every entry is in the contract and served by the mock. - **WFE types** live in `src/shared/wfe.ts` only, written from the swagger, under a heading per area. grpc-gateway JSON is camelCase, enums are names, int64 are strings, and default values may be missing. - **Agent JSON** (Read results, Ask events and results) is the engine's, typed from `contract/ontology_agent/` in `src/shared/foundry-read.ts` and `wfe.ts` "Ontology: questions", and parsed without trust. The mock emits exactly those shapes, and `tests/unit/agent-contract.test.ts` validates every mock answer, every fixture and the reads the SPA sends against the schemas (`mock/agent-contract.ts`, ajv), so a contract change fails CI. - **Calls** go through `api()` in `src/web/api.ts` with WFE paths (`api("/v1/whoami")`). It adds the CSRF header on unsafe methods and sends a signed-out user to sign-in. - **Look.** Everything uses the kit, shell, tokens and copy tone (`src/web/kit`, `src/web/shell`, `src/web/workspace.css`). Build a new screen from the kit and an existing area's CSS rather than inventing styles, with colors as tokens. Do not edit the kit to suit one screen. - **Colors** are `--hf-*` tokens in `src/web/workspace.css` only: the light set on `:root`, the dark set in two identical blocks (`@media (prefers-color-scheme: dark)` under `:root:not([data-theme="light"])`, and `:root[data-theme="dark"]`). A new color gets a value in all three, or is listed in `SAME_IN_BOTH`. No other CSS has a color literal; `tests/unit/theme-css.test.ts` checks that, and the dark theme's contrast (4.5:1 text, 3:1 icons and marks). - **Cookies** are Foundry's own and host-only: `__Host-foundry-session` (`WORKOS_COOKIE_NAME`), `__Host-foundry-csrf`, `__Host-foundry-login-`. On loopback http (mock, e2e) they drop the `__Host-` prefix. Never set a cookie domain. - **Secrets** never go in the repo. `.env.example` names the variables. The values in `mock/env.ts` are mock-only fixtures. ## Adding an area An area (Workspace, Data, Ontology) owns its folder (step 1) and adds to four shared files, each in a section of its own (steps 2 to 5): 1. `src/web/areas//`: `index.tsx` (route components; Data and Ontology export a `*Routes` table for their URL prefix, and Ontology also `ExploreRoutes` and `AskRoutes` for `/explore` and `/ask`), `paths.ts` (URL helpers, merged into `src/web/paths.ts`), `shell.ts` (sidebar children, a section's panel and command-palette sources), and its CSS. 2. `src/web/main.tsx`: the area's routes, if it adds a new prefix. 3. `src/server/papi-allowlist.ts`: one line per WFE route the area calls, in the area's section. 4. `src/shared/wfe.ts`: the payload types, under the area's heading. 5. `mock/routes/.ts`: the same routes with in-memory state, spread into `mock/routes/index.ts`. Use `membership()` from `mock/state.ts` for workspace-scoped routes and `onReset()` for the area's state. Then add the area's main flow to `tests/e2e/` and its logic to `tests/unit/`. ## Things to know - Names and Records: wherever an entity shows (the Explorer's canvas, tooltips, inspector and search, Records, an entity's page, a document's view) it reads by the version of the load the page reads (`explore/naming.ts`, `useNaming(versionId)`; Other data reads by the current load's version): the first non-empty `names` prop of its type, else its key props joined, else its stored `name`, else a short id; a label the version doesn't know, or no current version, keeps the raw label and the explorer's own caption. `__nn` (normalized names), an entity's `entity_key` and `norm_name`, a fact's `fact_key`, anything's `foundry_load` and Foundry's other stamps (`foundry_*`, such as `foundry_tx`) on all but Foundry's own labels never show in property lists; those stamps (`isFoundryStamp`) also never match the Explorer's search or take its `contains` scan's keys. Records (`/explore/{id}/records`, `areas/records`) is the Explorer's table view: its types are listed in the Explorer's panel (above the table while the panel isn't on screen); every query in `records/model.ts`, each bounded (a page is `range` over one type, one row more than a page, at most 10,000 rows deep; searches and filters count up to 1,000). - **Running Foundry locally**: `local/` (compose, README, `.env.example` of optional overrides) runs the local build, with nothing to set up: `docker compose up -d --wait` from the root (its `compose.yaml` includes `local/compose.yaml`: one project, `helix-foundry`, the same volumes from either place, and Compose reads `local/.env` for that file from either) or from `local/`. Its first service, `init-secrets` (`foundry init-secrets` in the foundry image, offline), writes the secrets the `secrets` volume lacks, once: an old install's `FOUNDRY_*` from its `local/.env` (setup.sh's, before 0.1.1), else random ones; every other service waits for it and reads its own from the volume (the site `FOUNDRY_LOCAL_TOKEN_FILE`, MySQL `MYSQL_PASSWORD_FILE`, the store its file, foundry `local.secrets_dir`). The connector key there seals every stored credential and is never made again. foundry makes the owner's first workspace, My workspace, until the install has had one, and with no `FOUNDRY_AI_PROVIDER` finds its AI default on the machine (`backend/ai/detect`: the shell's `ANTHROPIC_API_KEY`, then `OPENAI_API_KEY`, then an Ollama with a model that calls tools, then LM Studio; `none` is none). It runs the standalone `foundry` image (Foundry's backend in one process: its API, a directory of the one owner's workspaces, projects and databases, and its workers; built from `backend/`), the ontology agent, SeaweedFS, MySQL and the public helix-db image. That HelixDB keeps no tenants apart: every ontology's loads share one database, so one ontology's Other data shows labels another's loads wrote. The site's `Dockerfile` builds its image. - **Local build** (`NEXT_PUBLIC_FOUNDRY_MODE=local`, fixed at `next build`; `src/shared/deployment.ts`): a copy someone runs on their own machine. It has one owner and no sign-in, members, billing or sign-out. `/papi` (`src/server/local.ts`) answers only the Host of `FOUNDRY_ORIGIN`, which must be a loopback origin, and forwards as the owner with `FOUNDRY_LOCAL_TOKEN`, else the token in the file `FOUNDRY_LOCAL_TOKEN_FILE` names (read once it reads; shared with the API, 32+ bytes; the CSRF secret is derived from it) under the same allowlist, origin and CSRF checks. The API refusing the token is shown as a setup error, never a sign-in. Anything the hosted site shows about members, the plan or the Helix dashboard is behind `LOCAL`, as are People enrichment's settings (locally the API reads the setting as unavailable, so Edit offers no Enrich people). A local build offers no API source (`connectorKindsQuery` drops the `events` kind, and the palette has no New API source): their senders reach Foundry through the Helix MCP, which a local install doesn't run, and the local API refuses their send tokens. - Refresh: Foundry access tokens live 300 s and WorkOS refresh tokens are single use. AuthKit refreshes 30 s before expiry, and a failed refresh while the token is still valid keeps the session. Once the token has run out, a second refresh of the same cookie fails and ends the session, so: every `/papi` answer says how long the token has left (`x-foundry-session-expires-in`, `src/shared/session-contract.ts`), and once that is under 10 s, or not known yet, `api.ts` sends its calls one at a time under a Web Lock shared by the origin's tabs until one reports a fresh token. That holds across server instances. On one instance, parallel calls also share one refresh per session cookie (`src/server/single-flight.ts`), and a rotated session is reused for 30 s by requests still carrying the old cookie. - Onboarding: a workspace with no ontology shows nothing but the new-ontology wizard (`main.tsx`'s `WorkspaceRoutes`, `onboarding` on `useWorkspace()`). The sidebar keeps only the workspace switcher and the account menu (no nav, no search, no palette). Every other path redirects to `/ontology/new`, except an ontology's own pages (the one just created opens before the list has it) and `/settings/*` (the plan a database connection needs). The Helix MCP's connector links (`/data?add=`) open their form over the wizard. To test the full app in the mock, sign in as `ada@example.com`; a new email lands in onboarding. - Workspace switching is client-side (`src/web/workspaces.ts`): a Foundry session works on every workspace the user belongs to. When sessions become workspace-scoped, the token swap goes in `chooseWorkspace`. - Links into the app may name their workspace with `?workspace=`: `SignedIn` opens and remembers it when the user belongs to it, says so when not, and drops the parameter. `/data?add=` opens that kind's form in the Add data modal (`?add=1` the picker). The Helix MCP's `helix_foundry_connector_link` sends users to `/data?add=&workspace=`, so connector credentials are entered here, never through an agent. - Sign-out returns to `/login`. WorkOS must list `https:///login/callback` as a redirect URI and `https:///login` as a logout redirect for the Foundry application. - Password, SSO and email-verification sign-in are not offered: the callback sends those cases to `/login?error=sso|verify`. - Foundry Pro and Max (`src/web/areas/workspace/plan.ts`, which holds the prices: Pro $100 and Max $500 a month; the WFE returns none): one Autumn group, so a workspace holds Free, Pro or Max, and Max grants everything Pro grants. Autumn is the system of record and the WFE reads it live, so the plan query is cached for a minute. Every stamp's plans offer every connector kind (paid plans sync more often); the WFE refuses a kind a plan leaves out (create, test, update and sync) with FAILED_PRECONDITION and reason `FOUNDRY_PRO_REQUIRED`, which the mock keeps for PostgreSQL, MySQL and S3 on Free. The Data area renders `` (`areas/workspace/pro-gate.tsx`) for a kind with `available: false` (the picker, and a Pro connection's page after the workspace left Pro) and in place of a form error for which `isProRequired(error)` is true. It links owners and admins to `/settings/billing`, and the billing page refreshes the connector kinds once the plan bought shows. A Max workspace passes every Pro gate, so it never sees one. The plan's tier shows on Plan & billing and the account menu. - Billing: checkout (`CreateFoundryProCheckout`) sends a new `clientToken` per click and always the `tier` (`FOUNDRY_TIER_PRO` or `FOUNDRY_TIER_MAX`; unset is Pro). From Free, the upgrade dialog picks Pro or Max and takes a promotion code; a paid workspace switches to the other tier from its card, behind a confirm dialog, by checking that tier out (asking for the tier held attaches nothing). Cancel and resume act on whichever tier is held. After an answer that may have charged (`checkoutFailure` is `unsure`) the page reads the plan before offering checkout again, and Autumn refuses a second attach of a held plan. Stripe Checkout returns to `/settings/billing` (the WFE's `billing.autumn.foundry.return_url`), and that page keeps reading the plan for a while after a checkout this tab started, until the tier it bought shows. The mock's checkout and billing portal are `/mock/checkout` and `/mock/billing-portal`; `POST /__mock/config {"checkoutCardOnFile": true}` charges without a page. - `/` is Ask (`main.tsx` redirects it to `/ask`, keeping the link's query), on the ontology last opened, else the newest; a workspace without an ontology goes to the wizard. Ask is the sidebar's first section and is current on `/`, with its threads under the sections. There is no Home page. - Mock users: `alan@example.com` is a plain member of Acme, for what non-managers see; `POST /__mock/config {"alanRole": "WORKSPACE_ROLE_ADMIN"}` makes him an admin until the next reset, for what admins (not owners) see. Any other new email starts with no workspace, named from the email unless the mock sign-in page's Name is filled in. - Members are read-only here and invitations are create-only. Role changes, removal and pending-invitation management stay in the Helix dashboard (`NEXT_PUBLIC_HELIX_DASHBOARD_URL`, set per stamp in `next.config.ts`), where invitations also land. - The PostgreSQL and MySQL guides list the stamp's outbound addresses from `NEXT_PUBLIC_HELIX_EGRESS_IPS` (set per Vercel environment: the stamp's NAT egress IPs); without it they only say to allow Helix's outbound address. - API sources over HTTP: `NEXT_PUBLIC_HELIX_EVENTS_URL` (per stamp in `next.config.ts`: `https://ingest.helix-db.com` on prod, `https://ingest.dev.helix-db.com` on dev and in the mock, empty in local mode) is where Foundry's `foundry-events` role takes `POST /v1/sources/{id}/pieces` with a send token as the bearer. Empty, every mention of HTTP is hidden: the send panel's "Send over HTTP" (`eventSourceCurl()`, which reads the token from `HELIX_EVENT_TOKEN`, and `eventSourcePiecesExample()`), the prompts' HTTP section and `send_url` line, and the "app or agent" copy of the picker, palette, form, page and token. The mock doesn't serve the endpoint, and the site never calls it. - The mock has one Foundry plan per workspace, in `mock/routes/billing.ts`: Acme starts on Pro, a new workspace on Free, and checkout moves it to the tier asked for. A switch between Pro and Max charges the card on file at once (no checkout page) and reads the old tier once, as a card-on-file purchase reads Free once. The Data routes' Pro gate reads it (`isFoundryPro`, true on Pro and Max); unit tests set it with `setFoundryPlan`. To test an upgrade end to end, start from a new workspace (`signInToNewWorkspace` in `tests/e2e/support/mock.ts`); a switch starts from Acme. - Amazon S3 (`areas/data/connection-s3.tsx`, a Pro kind): files in one folder of a bucket become tables (the WFE lays keys out into tables; the user picks tables by name, `tables` empty meaning every one). Access is an IAM role or an access key. A role is offered only when the kind's `s3` setup (`principalArn`, the workspace's `externalId`) is in the connector kinds (an edit keeps a stored role, which without the setup can be renamed but not tested); the role panel shows the external ID and the trust and permissions policies to paste, written by `s3TrustPolicy` and `s3PermissionsPolicy` in `model.ts` exactly as the backend's S3 plan has them. A key's secret is write-only, and an edit that leaves it blank keeps the stored one; a role stores no secret. A test copies the region it detected into the form, and its tables carry `format`, `fileCount`, `bytes` (int64 strings, shown in binary units by `formatBytes`), up to 5 `preview` rows and a `problem` (the sync fails until it is fixed); Connect is disabled while the chosen tables' bytes are over the discovery's `maxBytes`. Changing the bucket clears an AWS region, which the next test detects again. - Data (`areas/data`): every Data page sits beside the Data explorer (`explorer.tsx`), the workspace's data as a tree, as in an IDE: at its root a folder per connector kind that has a connection, the workspace's own folders of connections, and `documents/` (`documents-tree.tsx`) with a folder per document set holding its documents (`tree.tsx`). It is the Data section's panel beside the sidebar (`panel.data` in `shell.ts`, `