# nexus-ai-pro: the full documentation

> The typed AI framework for TypeScript. One API for provider routing and failover, agent graphs and workflows that run branches in parallel and survive a restart, retrieval over any vector store, MCP, durable background operations, guardrails, cost control, images, voice, and evals — with a studio a team can share.

The README and every guide, in the order the README lists them, each after a line naming its source.
The guides' generated API references are left out: the published type declarations carry the same
doc comments. The index is at https://raw.githubusercontent.com/mkhitar-abrahamyan/nexus-ai/main/llms.txt.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/README.md

# nexus-ai-pro

The typed AI framework for TypeScript. One API for provider routing and failover, agent graphs and
workflows that run branches in parallel and survive a restart, retrieval over any vector store, MCP,
durable background operations, guardrails, cost control, images, voice, and evals — with a studio a
team can share.

Import only what you use. The root import is the client and nothing else, and every other capability
has its own entry point with a size budget CI enforces; [the packaging guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/packaging.md)
publishes what each one costs. The package has no required dependencies: a graph-only application
loads 81 KB, and installs nothing but this package.
The client, the provider adapters, graphs, agents, and the agent protocols run on Node.js, Deno,
Bun, edge runtimes, and browsers; [the runtimes guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/runtimes.md)
lists the entry points that need Node.js, and why.

- NPM: https://www.npmjs.com/package/nexus-ai-pro
- GitHub: https://github.com/mkhitar-abrahamyan/nexus-ai
- Contributing, testing, and release procedure: [CONTRIBUTING.md](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/CONTRIBUTING.md)
- What is planned next: [ROADMAP.md](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/ROADMAP.md)
- For AI assistants: [llms.txt](https://raw.githubusercontent.com/mkhitar-abrahamyan/nexus-ai/main/llms.txt) indexes every guide, and [llms-full.txt](https://raw.githubusercontent.com/mkhitar-abrahamyan/nexus-ai/main/llms-full.txt) holds them all in one file

## Install

```bash
npm install nexus-ai-pro
```

Requires Node.js 22 or newer. The package is authored as ESM and ships both an ESM and a CommonJS build,
so `import` and `require()` both work — including from NestJS and other apps compiled with
`"module": "commonjs"`. Types are shared between both builds.
The main Nexus pipeline is Node-oriented; the isolated realtime subpaths use portable structural
interfaces and can be bundled for modern browsers.

Install only the provider SDKs your app uses. The OpenAI SDK also powers OpenAI-compatible adapters such as OpenRouter, Groq, Mistral, DeepSeek, Azure OpenAI, LM Studio, and llama.cpp.

```bash
npm install openai @anthropic-ai/sdk ollama
```

Everything else is optional too. Add what your code needs:

| Package | When |
| --- | --- |
| `@types/node` | A TypeScript project on Node. The client, graphs, and the lifecycle compile without it. |
| `zod` | You write schemas with zod. The package uses the zod your schema came from. |
| `ajv`, `ajv-formats` | A response format checks output against a JSON Schema object. |

The Google Gemini provider uses Node.js `fetch`, so it does not need an extra SDK.
Realtime transports use injected or platform WebRTC, WebSocket, and `fetch` interfaces and do not
require the OpenAI SDK.

## Quick Start

```ts
import { createNexus } from 'nexus-ai-pro';

const ai = createNexus({
  provider: 'openai',
  apiKey: process.env.OPENAI_API_KEY!,
  model: 'gpt-5.4-mini',
  security: 'standard',
});

const response = await ai.complete({
  model: 'auto',
  messages: [{ role: 'user', content: 'Explain RAG in 3 bullet points.' }],
});

console.log(response.content);
console.log(response.meta.providerUsed);
console.log(response.meta.modelUsed);
```

For a fuller typed config, use the builder:

```ts
import { createNexusConfig } from 'nexus-ai-pro';

const ai = createNexusConfig()
  .openai(process.env.OPENAI_API_KEY!)
  .anthropic(process.env.ANTHROPIC_API_KEY!)
  .deepseek(process.env.DEEPSEEK_API_KEY!)
  .lmstudio({ baseUrl: 'http://localhost:1234/v1' })
  .auto('quality')
  .security('standard')
  .retry({ enabled: true, maxRetries: 2 })
  .create();
```

You can still pass a plain `NexusAIConfig` to `new NexusAI(...)` when you want full object-literal control.

## Why Use It

`nexus-ai-pro` is useful when an app needs more than a direct SDK call:

- route across multiple providers and model aliases
- fail over on provider errors, timeouts, or unhealthy providers
- stream through one normalized interface
- protect inputs and outputs with guardrails
- compact long conversations before they hit model limits
- estimate tokens and cost before provider calls
- cache exact or semantically similar prompts
- add tools, agents, RAG context, evals, batch jobs, and queues
- embed text through the same routing, caching, batching, budget, retry, and metrics as a completion
- run long operations that survive a restart, with leases, retries, dead-lettering, and signed webhooks
- build stateful graphs with cycles, fan-out, subgraphs, human approval, and resumable checkpoints
- make ordinary async functions durable step by step, and recover a crashed run at the step it died on
- load files, web pages, sitemaps, and Git repositories into retrieval, over memory, Postgres, SQLite,
  Redis, Qdrant, Pinecone, Weaviate, or Chroma
- retrieve with hybrid keyword and vector search, reranking, and diversity, and measure it with evals
- reach tools from many MCP servers under one configuration, with allowlists and credentials from the
  environment
- version prompts, instructions, tools, and skills together, and promote them through evaluation gates
- find failing and slow runs, detect regressions, and review evaluated fix proposals in a shared studio
- serve assistants as revisions, with canaries that roll back on a regression, workers that scale on
  the queue under Kubernetes, and per-tenant quotas, rate limits, and budgets
- trip routing away from a failing provider, and share one rate-limit budget across workers
- reach the providers' half-price asynchronous batch tier behind one operation handle
- persist generated media to disk or S3 with tenant isolation, retention, and checksums
- build persistent realtime voice agents with interruption, live tools, and normalized conversation state
- authorize, budget, audit, and observe every call of every family in one place, with hooks that see
  each one finish
- pass images a tool made back to the model by reference, never as base64 text
- keep TypeScript types around every request and response

## Guides

Each feature has its own guide, covering every export of its entry points with a reference generated
from the doc comments. The guides live in the repository, so these links go to GitHub.

| Guide | Covers |
| --- | --- |
| [The client](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/core.md) | Requests, context windows, token optimization, cost checks, reasoning, prompt caching, streaming |
| [The lifecycle](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/lifecycle.md) | Authorization, a shared budget, hooks, audit, and metrics for every call of every family |
| [Providers and routing](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/providers.md) | The twelve completion providers, routing and failover, the model registry |
| [Guardrails](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/security.md) | Injection, PII, secrets, schema validation, output redaction |
| [Graphs](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/graphs.md) | Parallel branches, interrupts, checkpoints, time travel, subgraphs, caching, diagrams |
| [Agents and tools](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/agents.md) | Graph-based agents with approvals, capabilities and permission policies, a sandbox contract, the middleware catalog, and the simple tool loop |
| [The deep-agent preset](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/deep-agents.md) | Long, multi-step agents: a plan, a sandboxed workspace, helpers, skills, and a context that stays small |
| [Long-term memory](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/memory.md) | Namespaced memory with semantic search, in memory or Redis |
| [MCP](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/mcp.md) | Borrowing tools from MCP servers, a registry of many, and serving your own |
| [Agent protocols](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/protocols.md) | Serving an agent to user interfaces over AG-UI, to other agents over A2A, and to code editors over ACP, and calling remote agents as tools |
| [Traces](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/tracing.md) | Run trees, queries, feedback, comparison, and alerts |
| [Evaluation](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/evaluation.md) | Datasets, evaluators, experiments, comparisons with a verdict, review queues, LLM judges, and tools emulated by a model so an evaluation sends nothing |
| [Context hub](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/context-hub.md) | Prompts, instructions, tools, and skills versioned together, promoted through gates, and moved between projects |
| [Insights](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/insights.md) | Failing and slow runs clustered, regressions between time windows, and evaluated fix proposals |
| [Prompts](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/prompts.md) | Typed templates, content versions, gated promotion, rollback, A/B splits, serving through outages |
| [Postgres](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/postgres.md) | One adapter family for operations, memory, traces, evaluation, circuits, and prompts |
| [SQLite](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/sqlite.md) | Durable operations, checkpoints, and memory on one machine, over any SQLite driver |
| [Durable operations and jobs](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/operations.md) | Operations that survive a restart, webhooks, and in-process job helpers |
| [Resilience and observability](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/resilience.md) | Circuit breaking, distributed rate limits, metrics, logs, and audit |
| [Batch tiers](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/batch.md) | The providers' discounted batch APIs, resumable from any process |
| [Caching](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/caching.md) | Exact and semantic response caches, with memory, Redis, and SQLite adapters |
| [Grounding](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/grounding.md) | Retrieval, citations, verification, self-consistency, and knowledge graphs |
| [Loaders](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/loaders.md) | Files, Markdown, HTML, CSV, JSON, PDF, web pages, sitemaps, and Git repositories into retrieval |
| [Retrieval](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/retrieval.md) | Hybrid search, reranking, MMR, parent documents, and Redis, Pinecone, Weaviate, and Chroma stores |
| [Embeddings and retrieval](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/embeddings.md) | Embeddings as a routed operation family, and RAG helpers |
| [Media understanding](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/media.md) | Reading documents and images page by page, describing images, and answering questions about them, through any vision model or OCR |
| [Typed decisions](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/decisions.md) | A yes or no, a choice, or a score, answered with probabilities and a calibrated confidence, from TypeSafe AI's Jev or any model, so a graph, an approval, or a guardrail acts when it is sure and asks a person when not |
| [Video generation](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/video.md) | Videos from a prompt or a first frame, on Google Veo and Runway, as durable jobs streamed into an asset store and priced per second |
| [Images (experimental)](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/images.md) | Generation and masked edits over three backends, input safety, moderation, asset stores |
| [Voice](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/voice.md) | Transcription, speech, voice turns, and voice sessions |
| [Realtime voice](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/realtime.md) | Browser and server realtime sessions with barge-in, tools, and exports |
| [Telephony](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/telephony.md) | Calls, webhooks, phone numbers, and phone agents on realtime sessions |
| [Testing](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/testing.md) | Recording and replaying provider traffic, and conformance suites |
| [Adapter kit](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/adapter-kit.md) | Building a provider, store, or retriever outside this repository, verifying it against the same contracts, and its compatibility record on the [compatibility page](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/COMPATIBILITY.md) |
| [Studio](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/studio.md) | A shared UI for traces, threads, approvals, experiments, prompts, deployments, costs, and health, as a separate package |
| [Agent server](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/server.md) | Self-hosted HTTP server for assistants: threads, durable runs, resumable streams, cron, a worker queue, and metrics |
| [Deployments](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/deployments.md) | Revisions and canaries with automatic rollback, autoscaling on Kubernetes and Helm, and per-tenant limits |
| [Tenancy](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/tenancy.md) | One tenant model: a tenant's view of every store, tenant graph runs, and the isolation suite |
| [Policy](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/policy.md) | Guardrails, routing, budgets, residency, and credentials as one versioned file, validated as the client starts, checked by `nexus policy check` in CI, and recorded on every operation |
| [Regions and credentials](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/regions.md) | Agent servers across regions behind a gateway, residency rules that keep each tenant's requests, traces, and assets in its regions, and each tenant's provider keys from a secret manager |
| [Workflows](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/workflows.md) | Ready-made chains and domain workflows |
| [Command line](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/cli.md) | The `nexus` command |
| [Packaging and install weight](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/packaging.md) | Every entry point and what it costs |
| [Runtimes](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/runtimes.md) | The portable kernel, and running the client, providers, graphs, and agents on Deno, Bun, edge runtimes, and browsers |

## Examples

```bash
npm run example:minimal
npm run example:create-nexus
npm run example:config-builder
npm run example:custom-provider
npm run example:feature-flags
npm run example:basic
npm run example:security
npm run example:optimizer
npm run example:agent
npm run build && npx tsx examples/realtime-scheduling.ts
npx tsc -p tsconfig.examples.browser.json
```

Repository example files cover:

- OpenTelemetry with Node HTTP and Express
- BullMQ workers
- OCR/PDF ingestion
- persisted vector stores
- classifier calibration
- domain workflows
- custom providers
- deterministic realtime scheduling with tools, confirmation, metrics, and conversation exports
- import examples under `examples/exports`

## Test Commands

```bash
npm install
npm run check
npm run check:release
```

`check` runs formatting and lint gates, source/test/example type checks, the build, unit and mock
conformance tests, coverage thresholds, package import checks, and an external type-consumer test.
`check:release` additionally holds the scheduling and runtime budgets — speed against a calibration
loop, and memory by retention — and verifies the dry-run tarball and a clean packed-package install.

`npm run docs:check` fails when any public export, or any public member of an exported class,
interface, or enum, has no doc comment, and `npm run docs:guides` fails when a guide does not name
every export of the entry points it covers, or its generated reference is stale. Both hold at 100%;
`--list` names what is missing, and `npm run docs:update` regenerates the references.

Real provider conformance, the vector store contract against real servers, and the experimental
families against their live services are opt-in:

```bash
npm run test:conformance:real
npm run test:vectors:live
npm run test:families:live
```

## Roadmap

See [ROADMAP.md](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/ROADMAP.md) for what is
planned, and [CHANGELOG.md](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/CHANGELOG.md) for what has shipped. It is a design proposal, not a compatibility promise; the guarantees
live in [API_STABILITY.md](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/API_STABILITY.md).

Shipped so far:

- completion controls and capability negotiation;
- durable operations, provider batch tiers, and distributed resilience, now including shared circuit
  state and a Postgres adapter family;
- embeddings, graphs, long-term memory, agents on graphs, and MCP;
- queryable traces, an evaluation platform, and a CI gate on top of it;
- record and replay of provider traffic;
- prompt versioning: typed templates, content versions, gated promotion, and serving through outages;
- a self-hosted agent server, and a local studio as a separate package;
- per-entry-point size budgets;
- vector stores behind one contract, and deprecation warnings at run time;
- durable functional workflows, step-level recovery in the server, SQLite persistence, and diagrams
  as images;
- document loaders, five more vector stores, hybrid retrieval with reranking, and an MCP registry;
- a shared studio with accounts, roles, an audit log, and comments; a context hub; insights with
  evaluated fix proposals; and evaluation caching;
- self-managed deployment at scale: revisions and canaries with automatic rollback, a worker queue
  that autoscales on Kubernetes, and per-tenant limits;
- the bridge to 2.0: every capability on a subpath of its own, everything 2.0 removes deprecated, and
  a codemod that moves your imports;
- 2.0 consolidation: one lifecycle for every call of every family, a shared spend budget, a root
  import that holds only the client, no required dependencies, and a model registry checked against
  every provider;
- 2.1 fault tolerance and crash-safe observability: checkpoints written on your terms, recovery after
  retries, idle timeouts and a clean drain, traces that survive a crash and reach any OpenTelemetry
  backend, one event stream, and dashboard rollups;
- 2.2 production scale and safe upgrades: versioned database migrations, queue claims and routing
  that cost the same at any size, a circuit breaker that counts across workers, one tenant model
  across every store, and `nexus doctor` to check a deployment before it serves traffic;
- 2.3 agents that act safely, on production retrieval: capabilities and permission policies, a
  sandbox contract, a middleware catalog, server authentication with one principal on every run,
  keyword search in Postgres and Elasticsearch with rerankers, durable ingestion, and a deep-agent
  preset;
- 2.4 graduation and interoperability: SQLite, loaders, retrievers, the MCP registry, the context
  hub, insights, and tenant limits out of experimental with published evidence; canaries judged with
  confidence and on quality; an adapter kit; AG-UI, A2A, and ACP; and a portable kernel, with the
  client, providers, graphs, and agents running on Deno, Bun, edge runtimes, and browsers;
- 2.5 graduation finished, and conversations evaluated: deployments that change atomically across
  replicas, revision worker pools, fair canaries judged on every run, and the studio out of
  experimental after a threat review; Postgres tenant usage; signed context named on every trace;
  simulated users and thread evaluators; and a doctor that knows the opt-in settings;
- 2.6 protocols complete, and progress on every stream: a tool's progress reaching the event stream,
  the protocols, traces, and the studio; AG-UI state deltas and frontend tools; A2A tasks and ACP
  sessions that survive a restart, with push notifications, `session/load`, and the editor's files,
  terminal, and MCP servers; and every protocol proven in a browser, against the previous release,
  and against its reference client;
- 2.7 agents that do more, and evaluation that runs on its own: workspace search and a multimodal
  workspace, a container sandbox, an interpreter, helpers that run in the background and survive a
  crash, and tool search on every provider; datasets generated from documents and from the
  guardrails' attacks; and online evaluation on workers, within a budget;
- 2.8 retrieval at production scale, and media that is understood: the Qdrant, Redis, Weaviate, and
  Chroma stores out of experimental after their checklist ran against the real servers; a shared
  embedding cache; images, audio, and documents as embedding input; a half-price embedding batch tier
  that ingestion survives a crash through; streaming asset stores; registry data you supply and
  refresh; and reading, describing, and searching media, with answers given the media themselves;
- 2.9 video, and everything shipped, proven: videos from a prompt or a first frame on Google Veo and
  Runway, as durable jobs another worker collects after a crash, streamed into an asset store and
  priced per second; online evaluation that reads every run however many share a millisecond, and
  budget reservations a dead worker leaves settled by what its call did; a deep agent's helper limit
  across processes; a media index that remembers what it holds; and the experimental families run
  against their live services every week;
- 2.10 durable conversations, and a second realtime provider: A2A tasks, ACP turns, and realtime
  sessions taken over by another worker when theirs dies, each proven killed at every write; realtime
  on the client's lifecycle; Gemini Live behind the same session, with video and images; phone calls
  answered over SIP; helpers on another server whose runs outlive their worker; and realtime over
  real WebRTC in a browser, with published budgets for turn latency and session load;
- 2.11 workspaces, and telephony in full: an agent's files apart from where its code runs, in memory,
  on disk, in any long-term store, read-only, or a composite of several; and a second phone provider,
  Telnyx, beside Twilio, both answering, transferring, conferencing, recording under a retention
  policy, and sending and receiving messages;
- 2.12 observability that loses nothing, and media you can evaluate: spans spooled through a collector
  outage, OTLP over gRPC, deployment evaluation that reads each run's whole trace, evaluators that
  read and judge images, documents, audio, and video, a catalog of every stored asset, and automation
  rules that roll a canary back when a score falls;
- 2.13 regions and residency, and credentials per tenant: agent servers across regions behind a
  gateway that fails over only within a tenant's regions, residency rules that keep each tenant's
  provider calls, traces, and assets in its regions, and each tenant's provider keys from a secret
  manager, rotated without a restart and kept out of every error;
- 2.14 policy you can review, and evaluation without side effects: guardrails, routing, budgets,
  residency, and credentials as one versioned file that `nexus policy check` compares with the version
  before it in CI and every operation records, tools emulated by a model so an evaluation sends
  nothing, and each tenant's key in every family;
- 2.15 a compatibility record for community adapters: `nexus adapters verify` seals the contract an
  adapter passed, against which release, with its benchmark; `nexus adapters recheck` turns a listed
  adapter red before a release that would break it; and one
  [page](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/COMPATIBILITY.md) lists them;
- 2.16 A2A over every binding: one agent served over JSON-RPC, HTTP+JSON, and gRPC from one task
  store, so a caller on any binding continues a task another started, and webhooks told the answer
  as it is written;
- 2.17 the editor's MCP servers over every transport, and video you can extend or edit: an ACP agent
  connects stdio, HTTP, and SSE servers and decides each tool by what its server says it does, and a
  video is extended from its own job or edited on Gemini Omni Flash, Veo, and Runway;
- 2.18 span events and links, as OpenTelemetry has them, media that knows its length from its own
  headers, a tenant's threads and runs listed and paged across all of its regions, MCP servers that
  reconnect and change their tools within a session, and videos made after reference videos;
- 2.19 typed decisions: a yes or no, a choice, or a score, answered with probabilities and a
  confidence, from TypeSafe AI's Jev or from any model's log probabilities, gated by thresholds a
  calibration chooses, so a graph's branch, an approval, or a guardrail acts when it is sure and asks
  a person when it is not;
- 2.20 tools read from a service's own OpenAPI document, which an agent calls or an emulator plays,
  an emulator that holds what a whole evaluation did so examples that build on each other are
  evaluated with nothing sent, and one A2A handler serving many tenants, each seeing only its own;
- 2.21 typed decisions about pictures, a yes or no, a choice, a score, or a box around something in
  one, from TypeSafe AI's Jev or any vision model; community adapters' compatibility records gathered
  from their own CI, a failing one turning red; and 3.0's defaults behind one switch today;
- 2.22 MCP over streamable HTTP both ways, with prompts, resources, and a server's questions back to
  the client; OpenAPI tools that send each operation's credentials; tokens, API keys, and sessions the
  server issues; and a studio that signs people in with your identity provider and serves many
  tenants, each seeing only its own;
- 3.0 the three defaults 2.21 previewed, on: tracing written as it runs, the Redis dispatch index, and
  `hash-v2` for a new deployment, with `useDefaults('2.x')` for an application that upgrades first;
  and the studio's sign-in, stable, proven against a real OpenID Connect provider on every build.

Upgrading from 2.x: 3.0 turns on three defaults 2.x kept opt-in, and `useDefaults('2.x')` keeps them
off until you are ready; from 1.x, `npx nexus migrate src --write` moves your imports.
[MIGRATING.md](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/MIGRATING.md) covers both.
Next, in 3.1, Pinecone, images, the Gemini embedding batch tier, media understanding, video, typed
decisions, Gemini Live, SIP, both phone providers, and Gemini's voice graduate once the live workflow
passes against each. The
[roadmap](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/ROADMAP.md) lists what comes after, in the order
applications ask for it.

## Before Production

Each guide ends with its limitations. The ones worth reading first: the
[client guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/core.md#in-production) on latency, traces,
and shared adapters; the [security guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/security.md#limitations)
on what guardrails do not replace; and the
[grounding guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/grounding.md#limitations) on embeddings
and verification. Voice, telephony, images, realtime, local models, and custom providers are optional
layers; enable only what you use.

## Donations

- USDT Tron: `TNbS2ub2Wys6j8yrv57bWg3Ke21ZNwt115`

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/core.md

# The client: requests, context, cost, and streaming


`NexusAI` is the client every other part plugs into. It gives you one request shape across providers,
and handles what sits around each call: long conversations, token budgets, cost checks, reasoning and
prompt-caching controls, options a model cannot honour, and streaming.

Import it from the root. In 2.0 the root holds the client and nothing else: its config builders, its
types, the errors it throws, and the lifecycle every call runs through. Every family is on a subpath
of its own.

```ts
import { createNexus } from 'nexus-ai-pro';

const ai = createNexus({ provider: 'openai', apiKey: process.env.OPENAI_API_KEY!, model: 'gpt-5.4-mini' });
const response = await ai.complete({ model: 'auto', messages: [{ role: 'user', content: 'Hello' }] });
```

## Creating a client

There are three ways in, and they produce the same `NexusAI`.

```ts
// 1. The shorthand: one provider, one model, credentials from the environment.
import { createNexus } from 'nexus-ai-pro';
const ai = createNexus({ provider: 'openai', apiKey: process.env.OPENAI_API_KEY!, model: 'gpt-5.4-mini' });

// 2. The typed builder, for several providers and a routing strategy.
import { createNexusConfig } from 'nexus-ai-pro';
const ai = createNexusConfig()
  .openai(process.env.OPENAI_API_KEY!)
  .anthropic(process.env.ANTHROPIC_API_KEY!)
  .auto('quality')
  .create();

// 3. The full configuration object, when it comes from a file or a secrets store.
import { NexusAI, defineNexusConfig } from 'nexus-ai-pro';
const ai = new NexusAI(defineNexusConfig({ providers: { openai: { apiKey } }, routing: { mode: 'auto' } }));
```

The shorthand takes `CreateNexusOptions`: a `CreateNexusProvider` name, its key, a base URL, the
Azure endpoint and deployment, and a default model. `normalizeCreateNexusConfig()` turns those into a
full `NexusAIConfig`, so you can see what the shorthand produced. `NexusConfigBuilder` is the
builder's type. `defineNexusConfig()` does nothing at run time; it types a plain object, so a
configuration kept in its own file is checked where it is written.

### The configuration

`NexusAIConfig` is the whole surface. Each part is explained in the guide for its feature, and the
reference below links every name to its summary.

| Part | Type | Guide |
| --- | --- | --- |
| Providers | `ProvidersConfig` | [Providers](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/providers.md) |
| Routing | `RoutingConfig` | [Providers](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/providers.md) |
| Retries | `RetryConfig` | [Resilience](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/resilience.md) |
| Response cache | `CacheConfig` | [Caching](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/caching.md) |
| Guardrails | `SecurityConfig` | [Security](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/security.md) |
| Rate limits | `RateLimitConfig` | [Resilience](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/resilience.md) |
| Token and cost budgets | `BudgetConfig`, `CostBudgetConfig` | this guide |
| Capability checks | `CapabilityConfig` | this guide |
| Token optimization | `TokenOptimizerConfig` | this guide |
| Structured output | `ResponseFormatConfig` | this guide |
| Logging and audit | `LoggerConfig`, `AuditLogConfig` | below, and [Security](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/security.md) |
| Pipeline hooks | `PipelineConfig` | this guide |
| Authorization, a shared budget, and hooks for every call | `LifecycleConfig` | [Lifecycle](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/lifecycle.md) |

`ProvidersConfig` has one typed entry per provider: `OpenAIProviderConfig`, `AnthropicProviderConfig`,
`GoogleProviderConfig`, `AzureOpenAIProviderConfig`, `OllamaProviderConfig`, `GroqProviderConfig`,
`MistralProviderConfig`, `CohereProviderConfig`, `DeepSeekProviderConfig`, `LMStudioProviderConfig`,
`LlamaCppProviderConfig`, and `CustomProviderConfig` for your own.

`RoutingConfig` has four parts:

- a `RoutingStrategy` for the auto-router: `cost`, `speed`, `quality`, or `privacy`;
- an ordered list of `RoutingRule` values, each sending matching requests to a model;
- the models each strategy prefers, as `RoutingModelPreference` entries (a name, or a name with a
  weight);
- a `FallbackConfig`, which adds failover models, a first-attempt timeout, and rate-limit handling to
  every route — even a request that names its model.

`LoggerConfig` receives each `LogEvent`: a `LogLevel` (`info`, `warn`, or `error`), a message, a
timestamp, structured data, and the error when there is one. Send them to your own sink, the console,
or both.

`AuditLogConfig` sends each `AuditLogEvent` to your sink. Every operation, in every family, writes a
`request` event when it is admitted. It then writes a `response`, a `blocked` when a check refused
it, or an `error`. Each event carries the operation's family, its name, and its request id.

## A request and its response

`CompletionRequest` is the request every provider takes. It needs a `model` and `messages`; the rest
is what a provider may honour:

| Settings | Fields |
| --- | --- |
| Sampling | `temperature`, `topP`, `topK`, penalties, `seed`, `stop` |
| Length and time | `maxTokens`, `timeoutMs`, `signal` |
| Tools | `tools`, and a `ToolChoice` |
| Output | `responseFormat`, `reasoning`, `cache` |
| Identity | `requestId`, `userId`, `tenantId`, `idempotencyKey` |
| Application data | `metadata` |

The identity fields follow the call everywhere. `requestId` is on the audit log, the provider call,
and `response.meta.requestId`; one is generated when you set none. `tenantId` decides which budget
the call is charged to. `idempotencyKey` goes to providers that deduplicate retried requests.

Each tool is a `ToolDefinition`: a name, a description written for the model, and a JSON Schema for
its arguments. It can also carry an `execute` function, for loops that run tools themselves, a
cache breakpoint, and `deferred`, which loads the tool only once a tool search finds it.

A `Message` has a `MessageRole`, and either text or a list of `ContentPart` values:

| Part | What it carries |
| --- | --- |
| `TextContent` | Text. |
| `ImageContent` | An image from a path, a URL, bytes, base64 text, or an asset. |
| `AudioContent`, `VideoContent` | Audio or video, or a transcript standing in for the audio. |
| `DocumentContent` | A document, such as a PDF, from a path, a URL, bytes, or base64 text. |
| `ToolReferenceContent` | A tool a search found, which the model may now call. |
| `AssetContent` | An asset by reference: an image or file named by where it lives, never its bytes. |

`AssetContent` is how a tool's image reaches the model without base64 text. Every adapter writes it
as one short reference line, which the model can pass back to another tool. Use an image part when
the model should see the image itself. A tool returns either kind as a `ToolOutput`, built with
`toolOutput()`; the [agents guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/agents.md) shows it. `BinaryBuffer` stands in for Node's `Buffer`,
so the types also compile for a browser.

A document part reaches Anthropic as a document block, OpenAI as a file, and Google as inline data.
Audio reaches Google and OpenAI's audio models. Any other provider, or a source it cannot read, such
as a path, receives one line naming the part instead.

`NexusResponse` comes back with:

- the text, the `ToolCall` values the model made, and a `finishReason`;
- `assets`, when a model whose output includes images returned some, as bytes;
- `ResponseMeta`: which provider and model answered, the latency, `TokenUsage`, `ResponseCost`,
  whether it was a cache hit, the guardrails applied, and the `PipelineTrace` when tracing is on;
- `failedAttempts`, when the first attempt did not answer: each `ResponseAttempt` that failed before
  it, retried on the same provider or failed over to the next, with its provider, model, error,
  category, and the time it failed. A stream's last chunk carries them in its `meta`;
- `logprobs`, when the request asked for them and the provider reported them: each `TokenLogprob` of
  the answer, its log probability, and its most likely alternatives, as the
  [providers guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/providers.md) describes.

After running a tool, you send back a `ToolCallResult`. `NexusStream` and `StreamChunk` are the
streaming equivalents.

```ts
const response = await ai.complete({ model: 'auto', messages: [{ role: 'user', content: 'Hello' }] });
response.meta.providerUsed;     // who answered
response.meta.usage.totalTokens;
response.meta.cost.amount;      // a number, in USD
```

## Structured output

`responseFormat` asks for JSON, optionally against a schema, and the client validates what comes
back rather than trusting it: output that is not valid JSON, or does not match the schema, raises a
`ResponseFormatError` rather than reaching your code as if it were valid. `ResponseFormatConfig` sets the same thing for every request.

```ts
const response = await ai.complete({
  model: 'auto',
  messages: [{ role: 'user', content: 'Extract the invoice' }],
  responseFormat: { type: 'json_schema', schema: { type: 'object', required: ['total'], properties: { total: { type: 'number' } } } },
});
```

The schema can be written two ways, and neither validator is installed with the package:

| Schema | What checks it | What to install |
| --- | --- | --- |
| A zod shape, such as `{ total: z.number() }` | Each field's own `safeParse` | Nothing beyond the zod you wrote it with |
| A JSON Schema object | ajv, loaded on the first check | `npm install ajv ajv-formats` |

A JSON Schema without ajv installed fails with a `ResponseFormatError` that says what to install.

## Context windows

Long chats can be compacted before token optimization and provider calls.

```ts
const ai = new NexusAI({
  providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } },
  routing: { mode: 'direct' },
  defaultModel: 'openai/best',
  contextWindow: {
    strategy: 'last-messages-with-summary',
    lastMessages: 8,
    summary: {
      model: 'openai/fast',
      maxTokens: 500,
      fallbackToLocal: true,
    },
  },
});
```

Strategies:

- `last-messages` - keep system messages and the latest N conversation messages
- `last-tokens` - keep recent messages up to `maxInputTokens`
- `last-messages-with-summary` - summarize older messages, then keep the latest N
- `last-tokens-with-summary` - summarize older messages, then keep recent messages near the token limit
- `auto` - compact only when message or token limits are exceeded

Summaries can be local, provider-generated, or custom:

```ts
contextWindow: {
  strategy: 'auto',
  lastMessages: 12,
  maxInputTokens: 12000,
  summary: {
    mode: 'custom',
    summarizer: async ({ serializedMessages }) => {
      return myMemoryService.summarize(serializedMessages);
    },
  },
}
```

The result is visible in `response.meta.contextWindow` and `ai.plan(...).contextWindow`.

`ContextWindowConfig` is the whole setting:

| Option | Default | What it sets |
| --- | --- | --- |
| `strategy` | — | A `ContextWindowStrategy`, from the list above. |
| `lastMessages` | 20 | Conversation messages kept. |
| `maxInputTokens` | — | The token ceiling. |
| summary reserve | — | Tokens held back for the summary. |
| keep system messages | on | System messages always survive. |
| `summary` | — | A `ContextSummaryConfig`, below. |

`ContextSummaryConfig` decides how cut messages are summarized. Its `ContextSummaryMode` is one of:

- `local`: an extractive summary, with no model call;
- `provider`: a model writes it, through the client. If that fails, it falls back to `local` unless
  `fallbackToLocal` is `false`;
- `custom`: your own `ContextSummarizer`, which receives a `ContextSummaryInput` — the request, the
  messages to fold, those messages as one text, the token limit, the model, and the instructions.

It also sets the summary's length (512 tokens), model, temperature, instructions, heading, and whether
it goes in as a system or a user message.

`ContextWindowManager` does the work, and you can use it on its own. `optimize()` trims a request,
calling the summarizer from a `ContextWindowRuntime` when given one. `preview()` reports what trimming
would do, without calling anything.

Both return a `ContextWindowResult`: the trimmed request, the techniques applied, and warnings (such as
a summary that fell back to local). Its `ContextWindowUsage` shows the strategy `auto` chose, tokens
and messages before and after, and how many messages were kept, summarized, or dropped.

## Token optimization

Use `tokenOptimizer` when you want lightweight token estimation, prompt densification, and budget enforcement.

```ts
tokenOptimizer: {
  densification: { enabled: true, preserveCodeBlocks: true },
  budget: {
    enabled: true,
    maxInputTokens: 4000,
    warnAt: 0.8,
    onExceeded: 'truncate',
  },
}
```

Budget actions, the `BudgetExceededAction` values:

- `error` - throw `TokenBudgetError` (the default)
- `truncate` - remove or slice older prompt content
- `densify` - compact prompt text before checking budget
- `allow` - warn but send as-is

`DensificationConfig` picks the techniques — whitespace cleanup, phrase compression, and list
compaction, all three by default — and leaves fenced code blocks alone unless told otherwise. A
warning is recorded once a request reaches `warnAt` of the budget, 80% by default.

The three pieces are exported for use outside the client. `TokenOptimizer` runs both steps on a
request and returns an `OptimizationResult`: the request, the techniques applied, warnings, and a
`TokenUsageSnapshot` of estimated tokens before and after, saved, and the share saved.
`PromptDensifier` rewrites text on its own, and `BudgetEnforcer` checks a request against a budget
(`check()`) or applies the over-budget action (`enforce()`).

## Planning a request before sending it

```ts
const plan = ai.plan({
  model: 'auto',
  messages: [{ role: 'user', content: 'Compare the last 3 releases.' }],
  maxTokens: 800,
  maxEstimatedCost: 0.05,
});

console.log(plan.model);
console.log(plan.estimatedCost.formatted);
console.log(plan.fitsContext);
console.log(plan.warnings);
```

`plan()` does not call a provider. It estimates route, token use, context fit, cost, guardrail findings, and warnings, including any option the routed model cannot honor.

The `NexusPlan` it returns answers the questions you would ask before paying for a call:

| Question | Field |
| --- | --- |
| Which provider and model would answer? | The model asked for, the one chosen, and the full routing decision with fallbacks. |
| Will it fit? | The model's context window, and whether the request fits it. |
| What would trimming do? | The context-window and token-optimization results. |
| What will it cost? | The estimated cost. |
| Could the cache answer it? | Whether the response cache has it. |
| Would guardrails block it? | What the input checks found, and which guardrails would apply. |

## Reasoning

```ts
const response = await ai.complete({
  model: 'auto',
  messages: [{ role: 'user', content: 'Find the bug in this migration plan.' }],
  reasoning: { effort: 'high', summary: 'auto' },
});

console.log(response.meta.usage?.reasoningTokens);
```

`ReasoningConfig` has three fields, and leaving it unset keeps the model's own default. `effort` is
the portable control, a `ReasoningEffort` from `none` through `minimal`, `low`, `medium`, `high`, and
`xhigh` to `max`. It maps to OpenAI `reasoning_effort` and the Responses `reasoning` field, Claude
thinking, and Gemini `thinkingConfig`. Use `reasoning.maxTokens` when you want to set a thinking
budget directly instead of by level, and `summary` (`none`, `auto`, or `detailed`) to stream
reasoning summaries.

Claude takes reasoning two ways, and the Anthropic adapter picks by model:

| Claude models | What is sent | `effort: 'none'` |
| --- | --- | --- |
| 4.6 onward | Adaptive thinking and `output_config.effort`. A `maxTokens` becomes the smallest effort whose budget covers it. | Turns thinking off. Sonnet 5.5 can only skip up-front thinking. Opus 5.5 and Fable always think, so they get effort `low`. |
| Before 4.6 | A thinking budget, from `maxTokens` or the effort. | Sends no thinking. |

Thinking counts toward the output limit. When you set no `maxTokens`, the adapter leaves room for
the thinking the effort asks for. Your own `maxTokens` is always kept as the hard cap. The newest
Claude models hide thinking text unless asked, so a `summary` also asks them to show it.

Requesting a summary emits `reasoning` stream chunks, which stay separate from visible output:

```ts
for await (const chunk of ai.stream({ model: 'auto', messages, reasoning: { summary: 'auto' } })) {
  if (chunk.type === 'reasoning') process.stderr.write(chunk.content ?? '');
  if (chunk.type === 'text') process.stdout.write(chunk.content ?? '');
}
```

## Prompt caching

Providers charge far less for a prompt prefix they have already processed. `mode: 'auto'` is the
default and reports whatever the provider reused on its own. `mode: 'explicit'` sends caller-placed
breakpoints to providers that accept them, such as Anthropic:

```ts
const response = await ai.complete({
  model: 'anthropic/best',
  cache: { mode: 'explicit', ttl: '1h' },
  messages: [
    { role: 'system', content: longPolicyDocument, cache: true },
    { role: 'user', content: 'Does clause 14 apply here?' },
  ],
});

console.log(response.meta.usage?.cachedReadTokens);
console.log(response.meta.cost?.amount);
```

Mark the end of the stable prefix, not every message. A provider caps how many breakpoints it
accepts (Anthropic allows four); when there are more marks than slots, the deepest ones are kept,
because a deeper breakpoint caches strictly more of the prompt.

`PromptCacheConfig` is the request's `cache` field: the `mode`, a `ttl` for breakpoints that set none
of their own, and `maxBreakpoints` to send fewer than the provider allows. Messages and tools carry
the breakpoints themselves. `off` suppresses caller-placed breakpoints; it cannot turn off caching a
provider does on its own.

## Usage and cost

Every response carries a numeric cost and a full token breakdown:

```ts
const { usage, cost } = response.meta;

console.log(usage?.inputTokens);        // billed at the standard input rate
console.log(usage?.cachedReadTokens);   // served from the provider cache
console.log(usage?.cachedWriteTokens);  // written into the provider cache
console.log(usage?.reasoningTokens);    // share of output spent on reasoning
console.log(cost?.amount, cost?.currency, cost?.basis);
```

`meta.tokensInput` keeps its original meaning of every prompt token, cached or not. Bundled prices
are defaults, not financial truth; override them with `models.registry`, or adjust cache rates with
`models.cachePricing`.

The client prices each response on the model it routed the request to, as the registry names it.
Providers often answer with another name, such as a dated snapshot of the model, which the registry
does not file. Pricing on that echo would cost the call nothing. A charge the provider reports
itself, with `basis: 'reported'`, is kept as it is.

## Counting and pricing tokens

`Tokenizer` estimates tokens without a provider — `estimateTextTokens()`, `estimateMessageTokens()`,
and `estimateRequestTokens()` — which is what budgets, planning, and context-window trimming run on.

Pricing is separate and explicit:

| Function | What it does |
| --- | --- |
| `estimateCost()` | Prices a `CostEstimateInput` against the model registry, returning a `CostEstimate`. |
| `priceUsage()` | Prices a `TokenUsage` a provider actually reported, with `PriceUsageOptions`. |
| `buildUsage()`, `buildMeta()` | Turn raw provider counts into `TokenUsage` and `ResponseMeta`, from `UsageInput` and `BuildMetaOptions`. For adapter authors. |
| `ensureUsageAndCost()` | Fills both in on a response from an older custom provider. |
| `costAmount()`, `formatCost()` | Read the number out, and render it. |
| `assertWithinCostBudget()` | Throws `CostBudgetError` when an estimate exceeds what you allow, so a request is stopped before it is sent, not after it is billed. |

`DEFAULT_CURRENCY` and `DEFAULT_CACHE_PRICING` are the defaults behind them.

## Capability negotiation

Requests are reconciled against the routed model before the provider is called:

```ts
const ai = new NexusAI({
  providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } },
  capabilities: { policy: 'warn' }, // 'strict' | 'warn' (default) | 'off'
});

const response = await ai.complete({ model: 'auto', messages, seed: 7 });
console.log(response.meta.capabilityWarnings);
```

- `strict` throws a `NexusCapabilityError` before spending anything.
- `warn` drops or clamps the option and records it on `meta.capabilityWarnings`.
- `off` sends the request exactly as written.

Some models take only part of an option. Claude 4.7 and later accept only the default temperature
and top-p (`sampling: false`), so a `temperature` sent to them is dropped. Opus 5.5, Sonnet 5.5, and
Fable 5.1 cannot be forced to call a tool (`toolChoice: false`). For them, `required` or a named tool
is dropped, while `auto` and `none` still pass.

Those are the three `CapabilityPolicy` values. Each `CapabilityWarning` names the option by its
dotted path, such as `reasoning.effort`, with the model and provider, the value requested, the
`CapabilityWarningAction` taken — `dropped`, or `adjusted` with the value actually sent — and why.

`negotiateCompletionRequest()` is the negotiation on its own, for a custom provider or a gateway that
checks requests before forwarding them. It takes a request, the model's capabilities, and
`NegotiateOptions` (the policy and the provider name for messages), and returns a
`NegotiationResult` with the adjusted request and its warnings. It checks only the options a request
sets and copies the request only when something changes, so an ordinary call allocates nothing.

An option a model does not declare is always passed through — absence means the registry does not
know, not that the provider refuses — so a model you register yourself is never restricted by fields
it omits. Set `capabilityPolicy: 'off'` on a single request to reach a provider feature that is
newer than the bundled registry.

Registry provenance is inspectable, and a release check can fail on stale data:

```ts
import { describeModel, assertRegistryFreshness } from 'nexus-ai-pro/models';

console.log(describeModel('anthropic/best'));
// { model: 'claude-opus-4-8', verifiedAt: '…', alias: { stage: 'stable', floating: true }, … }

assertRegistryFreshness({ maxAgeDays: 120 });
```

A `floating` alias resolves by intent rather than to a pinned model, so its target can change between
releases. Pin the resolved model when a run has to be reproducible.

## Streaming

```ts
const stream = ai.stream({
  model: 'auto',
  messages: [{ role: 'user', content: 'Write a haiku about TypeScript.' }],
});

for await (const chunk of stream) {
  if (chunk.type === 'text') process.stdout.write(chunk.content);
}
```

All providers normalize streaming chunks to `text`, `reasoning`, `tool_call`, `done`, or `error`.
Reasoning summaries only arrive when the request asks for them, so a consumer that switches on
`chunk.type` keeps receiving visible output only.

A stream is an operation like a completion. It opens when you first read it: authorization, the rate
limit, guardrails, routing, and the budget run then. So an error from any of them comes out of the
`for await` loop, not out of the `stream()` call. The `done` chunk carries the request id and the
cost of the usage the provider reported. The operation finishes with the stream; one you stop reading
early finishes as cancelled.

When security is enabled, output streams are correctness-first: Nexus buffers and validates the
complete output before yielding any chunk, up to 1 MiB or 100,000 chunks. This prevents secrets,
PII, and blocked phrases split across provider chunks from leaking partially, at the cost of
token-by-token latency. `security: 'off'` preserves immediate streaming when that trade-off is
explicitly acceptable for the application.

Three helpers work on any `NexusStream`. `collectStream()` reads one to the end and returns its text,
`mapStream()` transforms each chunk as it passes, and `createTextStream()` makes a stream that yields
one text chunk and finishes, for tests and for answering from a cache in the streaming shape.

## The request pipeline

Every request runs through the same stages: security, context window, optimization, routing, the
provider call, output checks, and caching. `PipelineConfig` is where you step into them.

| Piece | Use it to |
| --- | --- |
| `PipelineHooksConfig` | Run your function at a stage, named by a `PipelineHookName`. |
| `PipelineMiddleware`, `PipelineStep` | Add a stage of your own. |
| `PipelineRunner`, `createPipelineContext()` | Run the whole pipeline yourself, as the client does. From `nexus-ai-pro/pipeline`. |

Every hook receives the `PipelineContext`:

- the request, as earlier stages rewrote it, and the response once there is one;
- the routing decision, and what trimming and optimization did;
- the security findings and guardrails so far;
- the timings, and a `metadata` object hooks can share.

A hook may return a new context, a replacement request, a replacement response, or nothing.

`PipelineContext` is generic over the request and the response, and both default to a completion's.
That lets you run the same hooks over another family's types. Give `PipelineRunner` a
`PipelineShapes` that says how to tell your request and response apart from a returned context.

Each stage is timed into a `PipelineTrace` of `PipelineTraceStep` values. `PipelineStepName` names
the stage, and the list is closed, so a `switch` over it can be exhaustive. It holds the hook points,
the built-in stages, and the lifecycle's `authorize`, `reserveBudget`, `reconcileCost`, and `audit`.
A step you add with `use()` is recorded as `customStep`, with its own name in the step's `custom`
field. The trace is returned on `response.meta.pipeline` unless `includeTraceInResponse` is off.

The pipeline is a completion's own stages. The [lifecycle](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/lifecycle.md) around it is shared by
every family: authorization, the budget, audit, metrics, and hooks that see every call.

## Serving a request from a framework

`createNexusRouteHandler()`, from `nexus-ai-pro/next`, returns a Next.js route handler. It completes the
posted request, or streams it as server-sent events when `NexusRouteHandlerOptions.stream` is on:

```ts
// app/api/chat/route.ts
import { createNexusRouteHandler } from 'nexus-ai-pro/next';
export const POST = createNexusRouteHandler({ ai, stream: true });
```

For a full HTTP surface — threads, background runs, resumable streams — see the
[agent server guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/server.md).

## In production

- Keep what adds latency off the routes that are sensitive to it: the semantic cache and the
  semantic injection check each embed the request, and every pipeline hook runs on every call.
- Turn `pipeline.includeTraceInResponse` off when responses leave your service, so stage timings stay
  internal.
- Replace process-memory caches, rate limits, circuit state, and stores with their shared adapters
  before running more than one process; the [resilience guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/resilience.md) and the
  [Postgres guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/postgres.md) list them.
- Bundled prices and context limits are defaults, not a pricing contract; see the
  [providers guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/providers.md).
- Guardrails reduce risk; keep server-side authorization and provider moderation around high-impact
  actions, as the [security guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/security.md) says.

## 3.0's defaults, and keeping 2.x's

3.0 turns on three settings that 2.x kept opt-in. One switch keeps all three as 2.x had them, so an
application can upgrade first and turn them on when it is ready:

```ts
import { useDefaults } from 'nexus-ai-pro/config';

useDefaults('2.x'); // before creating a tracer, an operation store, or a deployment
```

or, without a change to the code, `NEXUS_DEFAULTS=2.x` in the environment. `DEFAULT_CHANGES` lists
each `DefaultChange`: the setting, where it is set, what 2.x did, what 3.0 does, and what to do first.

| Setting | 2.x, and `useDefaults('2.x')` | 3.0 |
| --- | --- | --- |
| `Tracer` `incremental` | A trace is written when its root run finishes | Each run is written as it starts and ends, on a durable trace store: one whose `durable` is not `false`, as `MemoryTraceStore`'s is |
| `RedisOperationStore` `index` | Finding queued work reads every record | Queued and leased work is kept in sorted sets, where the client has `zadd`, `zrem`, and `zrangebyscore`; first run 2.2 or later on every worker sharing the prefix, and call `reindex()` once |
| `Deployments` `bucketStrategy` | `fnv1a-v1` for a deployment first recorded | `hash-v2` for a deployment first recorded; one already recorded keeps its own |

Each is read when the object is created, so switch before creating anything, and on every replica
together. An option set explicitly wins under either: `incremental: false`, `index: false`, or
`bucketStrategy: 'fnv1a-v1'` keeps 2.x's behaviour for that one object. `nexusDefaults()` returns the
`NexusDefaults` in effect, `3.0` unless chosen otherwise, and a value of `NEXUS_DEFAULTS` that is
neither is refused rather than ignored. Under 2.x's, `nexus doctor --upgrade 3.0` says what turning
3.0's on would change for a deployment, as the [CLI guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/cli.md#checking-a-deployment) shows.
[MIGRATING.md](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/MIGRATING.md) covers each change.

## The registry's types, in passing

The root keeps the types a model entry is made of, which the [providers guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/providers.md)
explains: `ModelCapabilities`, `InputModality`, `OutputModality`, `ModelEndpoint`, `ModelStatus`,
`AliasMetadata`, `AliasStage`, `ProviderCapabilities`, `PromptCachingCapability`, `CacheTtl`, and
`CacheHint`. The registry data and its lookups are on `nexus-ai-pro/models`.

## The root import

The root import is the core client and nothing else:

| On the root | Examples |
| --- | --- |
| The client and its config builders | `NexusAI`, `createNexus`, `createNexusConfig` |
| Request helpers | `tool()`, `toolOutput()`, the streaming helpers, the usage and pricing helpers |
| The lifecycle | `OperationLifecycle`, its errors, and its types |
| Types of configuration, requests, and responses | `NexusAIConfig`, `CompletionRequest`, `NexusResponse` |
| Errors the client throws | `NexusProviderError`, `NexusSecurityError`, `CostBudgetError` |

Importing it costs about 325 KB, against 572 KB when the root re-exported most of the package.
Provider adapters load on their first call, and so do the features that answer asynchronously: agents,
evals, verification, the semantic cache, and the image and embedding engines.

Code written for 1.x imported families from the root. The command line moves those imports to their
subpaths:

```bash
npx nexus migrate src --write
```

The [migration guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/MIGRATING.md) lists every change 3.0 and 2.0 make, with before-and-after code.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/lifecycle.md

# One lifecycle for every operation


Every call the client makes passes through the same stages, whichever family makes it. A completion,
a stream, an embedding, a transcription, a phone call, an image, a realtime session, a graph run, an
agent run, a queued job, and a provider batch share:

- one authorization callback, which can refuse any of them before a provider sees it;
- one spend budget, held before each call and charged what the call actually cost;
- one set of hooks, which see every call start and finish, including cache hits;
- one audit log and one metrics collector, labelled by family.

You configure all of it once, as `lifecycle` on the client. `nexus-ai-pro/lifecycle` holds the budget
and the types; the runner and its errors are on the root as well.

```ts
import { NexusAI, OperationDeniedError } from 'nexus-ai-pro';
import { budgetLedger } from 'nexus-ai-pro/lifecycle';

const ai = new NexusAI({
  providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } },
  lifecycle: {
    authorize: (operation) => operation.userId !== undefined,
    budget: budgetLedger({ limit: 50, period: 'day' }),
    hooks: {
      onFinish: (operation, outcome) => console.log(operation.family, operation.operation, outcome.status),
    },
  },
});
```

## The stages

`LIFECYCLE_STAGES` lists them in order, each a `LifecycleStage`. A family skips a stage it has nothing
for.

| Stage | What happens |
| --- | --- |
| `validate` | The family checks the request's shape. |
| `authorize` | The rate limit, then your `authorize` callback. A refusal stops the call here. |
| `inputPolicy` | Guardrails on the input: security checks, safety policies, trimming. |
| `resolveAssets` | Images and files the request names are loaded. |
| `route` | The provider and model are chosen. |
| `reserveBudget` | The estimated cost is held against the budget. |
| `execute` | The provider is called, or a cache answers. |
| `outputPolicy` | Guardrails on the output. |
| `persist` | What outlives the call is stored, such as a cache entry. |
| `reconcileCost` | The hold is replaced by what the call cost. |
| `audit` | The response is audited and counted, and `onFinish` runs. |

Every operation is described by an `OperationDescriptor`: its `OperationFamily`, its name, its request
id, the provider and model once routing chose them, and the user and tenant it runs for. That is what
your callback, the budget, and your hooks receive.

| Family | Operations |
| --- | --- |
| `completion` | `complete`, `stream`, and `summarize` for a context-window summary |
| `embedding` | `embed` |
| `voice` | `transcribe`, `speak` |
| `telephony` | `createCall`, `getCall`, `endCall`, `listPhoneNumbers`, `updatePhoneNumber` |
| `image` | `images.generate`, `images.edit` |
| `realtime` | `realtime.session`, from connect to disconnect |
| `graph` | `graph.run`, `graph.<name>` for a named graph, and `workflow.<name>` |
| `agent` | `agent` for `ai.agent()`, and `agent.<name>` for `createAgent()` |
| `job` | `job`, once per attempt |
| `batch` | `batch.submit`, from submission to collection |
| `decision` | `decision.decide`, a typed decision's questions about one state |

## Authorization

`authorize` receives each operation and decides whether it may run. Return `false`, or throw, to
refuse it. Either way the call fails with `OperationDeniedError` before any provider is called, and
the error carries the operation it refused. A throw's message becomes the reason.

```ts
lifecycle: {
  authorize: async (operation) => {
    if (operation.family === 'telephony' && !(await canPlaceCalls(operation.userId))) {
      throw new Error('this account cannot place calls');
    }
    return true;
  },
},
```

The callback runs for every family. An admin-only operation, such as `updatePhoneNumber`, is refused
the same way as a completion.

## A shared budget

`budgetLedger()` builds a `BudgetLedger` that every family draws on. Before a call runs, its estimated
cost is held; afterwards the hold becomes what the call actually cost, and a failed call gives it back.
Adding first and then checking the new total is what keeps concurrent calls from overshooting the
limit together.

| `BudgetLedgerOptions` | Default | What it sets |
| --- | --- | --- |
| `limit` | — | US dollars per period: one number for every budget, or a function of the budget's key. |
| `period` | `month` | A `BudgetPeriod`: `hour`, `day`, `week`, `month` in UTC, or `{ windowMs }`. |
| `key` | the tenant, then `default` | Which budget an operation draws on. Return `undefined` to leave it untracked. |
| `store` | process memory | Where totals live, a `BudgetUsageStore`. |
| `leaseMs` | 10 minutes | How long a recorded reservation is held without `renew()` before `recover()` may settle it. |
| `uncertain` | `charge` | What a reservation whose operation's outcome is unknown costs: kept charged, or given back. |

```ts
import { budgetLedger } from 'nexus-ai-pro/lifecycle';
import { RedisTenantUsage } from 'nexus-ai-pro/server/tenancy';

const budget = budgetLedger({
  limit: (tenant) => plans[tenant]?.dailyUsd,
  period: 'day',
  store: new RedisTenantUsage(redis),
});
console.log(await budget.spent('acme'));
```

A call that does not fit fails with `BudgetExceededError`. It names the budget, the estimate, and
what was left. A budget that is already spent refuses even a call estimated at nothing, since an
estimate can be missing.

The pieces underneath:

- `BudgetUsageStore` needs two operations, `add()` and `total()`. `MemoryBudgetUsage` is the default.
  `MemoryTenantUsage`, `RedisTenantUsage`, and `PostgresTenantUsage` fit as they are, so one Redis or
  one database counts both tenant runs and model spend.
- `BudgetLedgerWithReport` is what `budgetLedger()` returns: the ledger, plus `spent(key)` for this
  period's total, and the reservation methods below.
- A `BudgetReservation` is one hold: the budget's key, the amount, and its `id` when the store
  records it. Write your own `BudgetLedger` with `reserve()`, `reconcile()`, and `release()` when your
  spend lives elsewhere.

### Reservations that outlive their process

A process that dies between reserving and reconciling used to leave its estimate charged for the rest
of the period. Over a store that records reservations, every bundled store, each reservation is a
`BudgetReservationRecord`, held under a lease of `leaseMs`, and its `BudgetReservationStatus` moves
with its total in one step, so a crash never applies one without the other:

| Status | Charged | Reached by |
| --- | --- | --- |
| `reserved` | the estimate | `reserve()` |
| `settled` | what the operation cost | `reconcile()`, or recovery of a finished operation |
| `released` | nothing | `release()`, or a reservation the limit refused |
| `expired` | nothing | recovery of an operation that never started |
| `uncertain` | the estimate, or nothing under `uncertain: 'release'` | recovery of an operation whose outcome is unknown |

`recover(resolve)` reads the reservations whose lease lapsed and asks `resolve` what became of each
operation, as a `BudgetReservationOutcome`: `finished` at a cost, `never-started`, `unknown`, or
`running`, which renews its lease. `resolve` returns `undefined` for an operation it does not know,
and `BudgetRecoverOptions.operation` reads only one kind; the `BudgetRecoveryReport` counts what moved
and what was skipped. `settle(id, outcome)` does the same for one reservation from any process, such
as the worker that took over its operation, and `renew()` extends the lease of an operation that runs
longer than it. A late `reconcile()` from the operation itself corrects what recovery decided; a
second reconcile changes nothing. `reservation(id)` reads one back.

```ts
const budget = budgetLedger({ limit: 500, store: new RedisTenantUsage(redis), uncertain: 'charge' });

// On a schedule, in any process: settle what dead workers held, by what your records say.
await budget.recover(async (reservation) => {
  const job = await jobs.get(reservation.operationId);
  if (!job) return undefined;
  if (job.finished) return { kind: 'finished', cost: job.cost };
  return job.startedProviderCall ? { kind: 'unknown' } : { kind: 'never-started' };
});
```

A store of your own records reservations by implementing the five optional methods together:
`hold()`, `settle()` with a `BudgetReservationChange`, `renew()`, `lapsed()`, and `reservation()`. A
ledger refuses a store with only some of them; a store with none works as before, its reservations
held in memory.

The budget is separate from `costBudget`, which caps one request's estimate. Both apply when both are
set.

## Provider health

`reportHealth(provider, outcome)` reports how a provider answered a call a family made itself, such
as a realtime connection, to the client's provider health and circuit breaker, as a completion's
attempts are. A `ProviderHealthOutcome` is `{ ok: true, latencyMs }` or `{ ok: false, error }`.
`ai.lifecycle` forwards it; a lifecycle built on its own does so through
`LifecycleRuntime.providerHealth`, and without it reports go nowhere. It is optional on
`OperationLifecycleLike`, so a family checks before it reports.

## Hooks

`LifecycleHooks` has two:

- `onStart` runs once an operation is admitted;
- `onFinish` runs once it ends, however it ends, with an `OperationOutcome`.

The outcome's `status` is `succeeded`, `failed`, `cancelled`, or `denied` when authorization, a rate
limit, a budget, or a guardrail refused the call. It also says whether a cache answered, how long the
call took, what it cost, and the error when there was one. A throwing hook is ignored, so
observability can never fail a call.

`LifecycleConfig` is the whole setting: `authorize`, `budget`, `hooks`, `credentials`, a
`CredentialResolver` that gives each provider call its tenant's own key, and `residency`, a
`ResidencyPolicy` that keeps each tenant's provider calls in its regions, as the
[regions and credentials guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/regions.md) describes; `providers`, a `ProviderGate` that says which
providers and models each operation may reach, and `policy`, the `PolicyReference` every operation and
audit event records, as the [policy guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/policy.md) describes. `applyPolicy()` sets them all from
one file.

`callContext()` gives a provider call that continues an operation admitted earlier, such as a video
job polled by its ref, what that operation's calls get: a `ContinuedCallContext` with the tenant's
credentials, its residency, and a `scrub()` for the call's errors, without authorizing, budgeting, or
auditing another operation. A family calls it through `OperationLifecycleLike.callContext`, which is
optional.

## Graphs, agents, sessions, queues, and batches

Code outside the client takes the client's lifecycle as an option, `ai.lifecycle`. Its runs then
become operations of the client too.

| Where | Option |
| --- | --- |
| `graph.compile()` | `lifecycle` |
| `workflow()` | `lifecycle` |
| `createAgent()` | `lifecycle`, labelled `agent` |
| `createRealtimeSession()` | `lifecycle`, from connect to disconnect |
| `new JobQueue()` | `lifecycle`, once per attempt |
| `new BatchManager()` | `lifecycle` in its runtime |

```ts
const graph = builder.compile({ lifecycle: ai.lifecycle, name: 'triage' });
await graph.invoke(input); // one `graph.triage` operation, and one per model call inside it
```

These options take an `OperationLifecycleLike`: anything with a `start()` that admits an operation.
So `nexus-ai-pro/graph` and the other families depend on the lifecycle's types only.

## Writing a provider

Every provider call receives a `ProviderCallContext` beside its request. It holds the operation's
request id, its abort signal, its deadline, its trace headers, its idempotency key, its tenant, and,
with a credential resolver, `credential(provider)`: the `ProviderCredential` to send in place of the
configured key. A provider of your own sends `(await context?.credential?.(name))?.apiKey`, falling
back to its own. Under a residency policy, `residency` says which providers the call may reach. A chat
provider gets it as the second argument of `complete()` and `stream()`. Each family's own context
extends it: an image provider's adds the operation id, and an embedding provider's adds the attempt
and the batch index.

```ts
class MyProvider extends BaseProvider {
  async complete(request: CompletionRequest, context?: ProviderCallContext) {
    const response = await fetch(url, {
      signal: context?.signal,
      headers: { 'Idempotency-Key': context?.idempotencyKey ?? '' },
    });
    // ...
  }
}
```

The bundled OpenAI and Anthropic adapters send the idempotency key and the trace headers on.

## Running an operation of your own

`OperationLifecycle` is the runner, and the client builds one from `LifecycleRuntime`: its metrics,
audit log, rate limiter, and `LifecycleConfig`. `ai.lifecycle` is the client's own. There are two ways
to use it.

`run()` takes an `OperationPlan` and runs every stage in order. A plan's `execute` is required; its
`validate`, `inputPolicy`, `resolveAssets`, `route`, `estimate`, `outputPolicy`, `persist`, and
`settle` are optional.

```ts
const summary = await ai.lifecycle.run(
  { family: 'job', operation: 'nightly-summary', tenantId: 'acme' },
  {
    estimate: () => 0.02,
    execute: async (context) => summarize(documents, { signal: context.signal }),
    settle: (result) => ({ cost: result.cost }),
  },
);
```

`start()` is for an operation that outlives one call, such as a stream or a session. It admits the
operation and returns an `OperationTicket`:

- `context`, the `ProviderCallContext` to hand every provider call;
- `reserve()`, which holds an estimate against the budget;
- `succeed()`, with an `OperationResultInfo` of cost, cache hit, provider, and model;
- `fail()`, with the error that ended it, cleaned first of every credential the operation resolved;
- `refreshCredential()`, which resolves a provider's credential again after the provider refused it;
- `scrub()`, which removes those credentials from any other error, in place.

`OperationStartOptions` add a caller's signal, a timeout, an idempotency key, and trace headers.

## In production

- Give every request a `tenantId` when a budget is kept per tenant. Without it, calls share the
  `default` budget.
- Use a shared store, such as Redis, once more than one process charges the same budget.
- Keep `authorize` fast: it runs before every call, cache hits included.
- Budgets hold estimates. A model with no registry price is estimated at nothing; register its price
  with `models.registry` so the budget sees it.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/providers.md

# Providers, routing, and the model registry


Twelve completion providers behind one contract, the routing that chooses between them, and the model
registry routing reads. Each provider has its own entry point, so an application loads only the ones
it uses. For a provider that is not bundled, implement `BaseProvider`.

## Providers and Routing

```ts
const ai = new NexusAI({
  providers: {
    openai: { apiKey: process.env.OPENAI_API_KEY! },
    anthropic: { apiKey: process.env.ANTHROPIC_API_KEY! },
    groq: { apiKey: process.env.GROQ_API_KEY! },
    deepseek: { apiKey: process.env.DEEPSEEK_API_KEY! },
    openrouter: { apiKey: process.env.OPENROUTER_API_KEY! },
    ollama: { baseUrl: 'http://localhost:11434' },
    lmstudio: { baseUrl: 'http://localhost:1234/v1' },
  },
  routing: {
    mode: 'auto',
    strategy: 'quality',
    requiredCapabilities: {
      streaming: true,
      minContextTokens: 128000,
    },
  },
});
```

First-class provider adapters:

- OpenAI
- Anthropic
- Google Gemini
- Ollama
- OpenRouter
- Groq
- Mistral
- Cohere
- DeepSeek
- Azure OpenAI
- LM Studio
- llama.cpp
- custom OpenAI- or Anthropic-compatible endpoints through `providers.custom` or `registerProvider(...)`

Routing modes:

- `direct` - always use the requested model or `defaultModel`
- `auto` - choose from available providers by cost, speed, quality, or privacy
- `rules` - route from request metadata
- `hybrid` - try rules first, then fall back to auto routing

## The adapters

Each adapter has its own entry point, so an application that uses one provider loads one:

| Adapter | Entry point | Notes |
| --- | --- | --- |
| `OpenAIProvider` | `/providers/openai` | Chat Completions and the Responses API; the base of the OpenAI-compatible adapters below |
| `AnthropicProvider` | `/providers/anthropic` | Messages API, with explicit cache breakpoints. `reasoning.effort` becomes adaptive thinking and an effort from Claude 4.6 on, and a thinking budget before it |
| `GoogleProvider` | `/providers/google` | Gemini, with thinking budgets mapped from effort and cached tokens separated out of the prompt count |
| `AzureOpenAIProvider` | `/providers/azure-openai` | OpenAI on an Azure endpoint and deployment |
| `OpenRouterProvider` | `/providers/openrouter` | Many vendors' models through one OpenAI-compatible API |
| `GroqProvider` | `/providers/groq` | Groq's OpenAI-compatible chat API |
| `MistralProvider` | `/providers/mistral` | Mistral's chat API |
| `DeepSeekProvider` | `/providers/deepseek` | DeepSeek's OpenAI-compatible API |
| `CohereProvider` | `/providers/cohere` | Cohere's Chat API |
| `OllamaProvider` | `/providers/ollama` | Models on this machine or a local server; local, so privacy routing prefers it |
| `LMStudioProvider` | `/providers/lmstudio` | LM Studio's local OpenAI-compatible server |
| `LlamaCppProvider` | `/providers/llamacpp` | llama.cpp's local OpenAI-compatible server |

The client constructs them from `providers` configuration, and each adapter's module loads on that
provider's first call. So configuring a provider you never call costs no import time. Construct one
yourself to call it directly or to register it under another name.

Every adapter sends a conversation's tool calls and results back the way its API expects:

| Adapter | The model's tool calls | Their results |
| --- | --- | --- |
| OpenAI chat | `tool_calls` on the assistant message | One `tool` message per call, under the call's id; images follow as one user message |
| OpenAI Responses | `function_call` items | `function_call_output` items, under the call's id |
| Anthropic | `tool_use` blocks | `tool_result` blocks, every result of a turn in one user message, images included |
| Google | `functionCall` parts | `functionResponse` parts, named for the function that asked, images beside them |

An asset part in a message is sent as one reference line, never as its bytes.

### Log probabilities

A request's `logprobs` asks for each token of the answer with its log probability, and with `top`,
its most likely alternatives, up to 20; `response.logprobs` carries them, each a `TokenLogprob`.
OpenAI's Chat Completions and Responses API, the OpenAI-compatible adapters that report them (Azure
OpenAI, OpenRouter, DeepSeek, LM Studio, and llama.cpp), Gemini, Ollama, and Cohere send it. Cohere
reports a piece of text at a time, with no alternatives: each piece is one entry, its log probability
the sum of its tokens'. Groq and Mistral refuse the option, so it is not sent to them, and a model
whose `ModelCapabilities` declare `logprobs: false` has it removed, with a capability warning. A model
that reports none leaves `response.logprobs` absent. Typed decisions read a model's probabilities
this way; see the [decisions guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/decisions.md).

A stream reports them too: each `text` chunk's `StreamChunk.logprobs` holds its tokens', in order, so
the chunks' together are what `complete()` reports. Text the output guard changes, such as a redacted
address, keeps none, streamed or not, since they would spell out what it removed.

## Writing a provider

`BaseProvider`, on `/providers/base`, is the contract for a provider that is not bundled. Implement
three things:

- `info`, a `ProviderInfo`: the name, and whether it runs locally, which privacy routing prefers;
- `complete()`, for one response;
- `stream()`, for a streamed one.

Both calls receive the operation's `ProviderCallContext` as a second argument: its request id, abort
signal, deadline, trace headers, and idempotency key. Ignore it, or pass the parts your API accepts;
the [lifecycle guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/lifecycle.md) shows how.

`healthCheck()` is optional and defaults to healthy. Protected helpers do the shared work: build the
base response, extract message text, format tools, normalize errors, honour an aborted signal, and
turn an async generator into a `NexusStream`. So a new adapter is mostly request and response mapping.

```ts
import type { CompletionRequest } from 'nexus-ai-pro';
import { BaseProvider } from 'nexus-ai-pro/providers/base';

class GatewayProvider extends BaseProvider {
  readonly info = { name: 'gateway', isLocal: false };

  async complete(request: CompletionRequest) {
    this.throwIfAborted(request);
    const response = this.createBaseResponse('gateway', request.model);
    response.content = await gateway.chat(this.extractTextContent(request.messages), request.signal);
    return response;
  }

  stream(request: CompletionRequest) {
    return this.createStream(async function* () {
      for await (const text of gateway.chatStream(request.messages)) yield { type: 'text', content: text };
    }, request.signal);
  }
}

ai.registerProvider('gateway', new GatewayProvider());
```

`/providers/type-guards` has small readers for untyped JSON:

| Function | What it does |
| --- | --- |
| `isRecord()` | Is the value a plain object? |
| `asArray()`, `asString()`, `asNumber()` | Coerce a value, with a fallback. |
| `getRecord()`, `getArray()`, `getString()`, `getNumber()` | Read one key safely. |

## Provider errors

Every adapter reports failure as a `NexusProviderError`. It carries the provider, the model, the HTTP
status, a category, whether a retry could succeed, and the cause. Retries, failover, and the circuit
breaker all decide from these fields.

The `NexusProviderErrorCategory` is one of `auth`, `abort`, `timeout`, `rate-limit`, `server-error`,
`network`, `bad-response`, or `unknown`. `NexusProviderErrorOptions` constructs an error, inferring
any category and retryability you leave out.

`/providers/errors` has the builders adapters share:

| Function | Use it for |
| --- | --- |
| `createProviderHttpError()` | A failed response, keeping the first 1,000 characters of its body. |
| `toNexusProviderError()` | Any error. A provider error is returned unchanged. |
| `createAbortProviderError()` | A request the caller aborted. Never retryable. |
| `createTimeoutProviderError()` | A request that ran past its timeout. Retryable. |
| `categorizeProviderError()` | Inferring a category from the status, then the error's name, code, and message. |
| `isRetryableProviderError()` | Deciding whether to retry: timeouts, rate limits, server errors, and network failures are worth it. |
| `isAbortError()` | Recognizing an abort. |

## Routing internals

`Router` makes the routing decision, in this order:

1. a model the request names directly;
2. a matching rule;
3. the auto-router's ranking;
4. then the fallbacks that `routing.fallback` adds.

A routing strategy sees a `RouterContext`: the request, the configuration, the providers, their
health, and the providers whose circuit is open. Providers with an open circuit are excluded, not
just ranked lower.

The auto-router plans once per configuration. Which models are candidates, which pass the allow and
deny lists and `requiredCapabilities` (its boolean requirements checked as bitmasks), and how each
scores for the strategy are computed into a plan the first time a configuration routes, and kept
until the routing settings, the model settings, or the providers change. A request then pays only
for open circuits and provider health: health moves a whole provider up or down, so the plan's
per-provider lists are merged rather than re-sorted. Over 1,000 models a warm plan routes in about
15 µs at p50; 2.1 took most of a second, rebuilding the registry for every candidate. Decisions are
exactly those 2.1 made, which a test checks against the 2.1.0 router on 300 random configurations.
A registry changed in place keeps the old plan; replace `models.registry` to change it.

It returns a `RouteDecision`: the provider and model to try first, why, limits for that first attempt,
and the fallbacks in order. Each fallback is a `RouteAttempt`. `FailoverExecutor` runs the decision.
It tries each attempt with its own timeout, rate-limit retries, and circuit check. If none succeeds, it
throws with every attempt's error. Both are on `nexus-ai-pro/router`, for a gateway that routes
requests without the rest of the client.

## Model Registry Generation

The registry is generated from versioned provider data in `data/models/`, not edited by hand. A
hand-written list of 100+ models invites mistakes; the Claude 5 reasoning bug fixed in 1.4.0 was one.

```bash
npm run registry:generate   # data/models/*.json -> src/models/generated.ts
npm run registry:check      # fails if the committed output is stale
```

`registry:check` runs as part of `npm run check`, so committed data and output cannot drift apart. The
generator checks required fields, price signs, context bounds, and status values. It also checks that
every alias points at a real model; a dangling alias would otherwise fail only when a request used it.

The runtime still reads the hand-written `KNOWN_MODELS`, and a test asserts the generated registry
matches it exactly. Switching the runtime over is a separate change, so adding the generator could not
quietly change prices in the same release.

`src/models/generated.ts` and `data/` are build-time files and are **not** published. They duplicate
`KNOWN_MODELS`, and shipping them in both builds would add about 310KB to every install for data
nothing reads. Both live in the repository, where a diff is what you want.

## Reading the registry

`KNOWN_MODELS` maps each model to its `ModelCapabilities`:

| Group | Fields |
| --- | --- |
| Identity | Provider and family, `ModelStatus`, release date, knowledge cutoff. |
| Modalities | What it accepts and what it produces. See below. |
| Features | Streaming, tool calling, structured output and JSON mode, reasoning and the efforts it takes. |
| Caching | A `PromptCachingCapability`: whether it takes caller-placed breakpoints, which `CacheTtl` lifetimes, the minimum prefix, and how many breakpoints. |
| Options honoured | Forced tool choice, parallel tool calls, non-default sampling (temperature and top-p), seed, top-k, penalties, log probabilities. |
| Limits and cost | Context and output limits, prices, quality and speed scores. |
| Serving | The `ModelEndpoint` values it is served on. |
| Provenance | When and where the entry was verified. |

### What a model accepts and produces

Every entry says which way each modality flows, in two lists:

- `inputModalities` lists each `InputModality` the model accepts: text, image, audio, video, or pdf.
- `outputModalities` lists each `OutputModality` it produces: text, image, or audio.

Both are required, on a bundled entry and on one you register. `modalitiesOf()` returns copies of
the two, in one call:

```ts
import { getModelCapabilities, modalitiesOf } from 'nexus-ai-pro/models';

const { input, output } = modalitiesOf(getModelCapabilities('gemini-3-pro-image')!);
// input: ['text', 'image'], output: ['text', 'image']
```

Routing requires them the same way. `routing.requiredCapabilities` takes `inputModalities` and
`outputModalities`, so a request with a PDF attached can route only to models that read one.

`ProviderCapabilities` groups a provider's models with its name and locality. A `CacheHint` is a
breakpoint on a message or tool.

Aliases such as `openai/best` resolve by intent. `MODEL_ALIAS_METADATA` gives each alias's
`AliasMetadata`: its `AliasStage`, and whether it floats. `REGISTRY_PROVENANCE` says where the bundled
data came from and when it was verified. `resolveProvider()` names the provider a model belongs to.

`/models` reads the registry through your configuration. A model you register with `models.registry`,
or an alias with `models.aliases`, is seen like a bundled one. A name with a provider prefix, such as
`google/gemini-3.5-flash`, finds the entry filed without it, but only when that entry belongs to the
provider the prefix names.

`DEFAULT_CACHE_PRICING` holds each provider's cache rates, as multiples of its standard input price.
It applies when a model declares no cached-read or cache-write price of its own.

| Function | Returns |
| --- | --- |
| `resolveModel()` | A `ResolvedModel`: the model, provider, capabilities, and alias metadata. It runs on every completion, so it reads the maps directly instead of merging them. |
| `resolveModelAlias()` | The resolved name alone. |
| `getModelCapabilities()` | One model's entry. |
| `getModelRegistry()`, `getModelAliases()`, `getAliasMetadata()` | The merged maps. Your entries win. |
| `listKnownModels()`, `listModelsForProvider()` | Model names, sorted. |
| `describeModel()` | `ModelProvenance`: the resolution, plus when the entry was verified and where its data came from. |
| `checkRegistryFreshness()` | `RegistryFreshness`: when the data was verified, its age, the window (180 days), whether it is stale, and entries older than the window. |
| `assertRegistryFreshness()` | Throws when the data is stale, for a release check. |

## Registry data you supply

Prices and models change faster than a runtime is released. Keep the registry as a file of your own,
check it, and hand it to the client: the next call is priced from it. Nothing is fetched from a
server; the file lives in your repository, a volume, or your configuration.

```ts
import { readFile } from 'node:fs/promises';
import { NexusAI } from 'nexus-ai-pro';
import { parseRegistryData, withRegistryData } from 'nexus-ai-pro/models/data';

const data = parseRegistryData(await readFile('./registry.json', 'utf8'));
const ai = new NexusAI(withRegistryData({ providers }, data));
```

A `RegistryData` file declares its format, `REGISTRY_DATA_FORMAT` (`nexus-registry/1`), its own
`version`, when it was verified, and where. It holds completion `models`, `aliases`, and
`aliasMetadata`, and `embeddingModels` and `embeddingAliases`. Every entry has the shape of a bundled
one.

| Function | Does |
| --- | --- |
| `bundledRegistryData()` | The data this release bundles, as a file to start from or compare with. |
| `checkRegistryData()` | Every problem with a file, each with its path, such as `models.gpt-5.4-mini.costPer1kInput`; empty when it is valid. An alias must name a model the file or the bundled data has. |
| `parseRegistryData()` | Reads JSON text or a parsed value, and throws a `RegistryDataError` listing every problem. |
| `withRegistryData()` | A client configuration with the file layered in: your own `models.registry` entries first, then the file's, then the bundled ones. Embedding entries go to `embeddings.models` the same way. |
| `diffRegistryData()` | A `RegistryDiff`: models added and removed, each changed model's `RegistryFieldChange` list, and aliases moved. |
| `registryDataAgeDays()` | How many days old a file is. |

`withRegistryData()` also sets `models.verifiedAt`, so `checkRegistryFreshness()` and `nexus doctor`
measure the registry's age from the file. `nexus models export`, `check`, and `refresh` work with the
file from the command line; the [CLI guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/cli.md#registry-data) shows them. In the test suite, a
price raised tenfold in a supplied file makes the next call cost ten times as much.

## Limitations

- Bundled prices and capabilities are defaults, verified on the date the registry records. A
  provider can change either between releases; supply newer registry data, or override entries with
  `models.registry`.
- No provider publishes prices in its model list, so `nexus models refresh` finds new and retired
  models, and a changed price is still checked against the provider's pricing page.
- An option a model does not declare passes through untouched, so a registry that does not know a
  feature never blocks it; it also never warns about it.
- Log probabilities come from Chat Completions, OpenAI's Responses API, Gemini, Ollama, and Cohere,
  without alternatives from Cohere: not from Anthropic, Groq, or Mistral, whose APIs report none.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/security.md

# Guardrails and security


Guardrails check what goes into a model and what comes out. They come from `nexus-ai-pro/security`:

- on input: schema validation, prompt-injection and personal-data detection, secret and URL checks,
  and tool allowlists;
- on output: redaction, moderation, and a grounding check;
- for files: upload scanning.

They reduce risk. They do not replace authorization, or a person reviewing high-impact actions.

## Turning it on

Security is on by default, at the `standard` level. The client's `security` option takes a level, or a
full `SecurityConfig`: a level, a preset, and detailed input and output settings.

```ts
const ai = new NexusAI({
  providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } },
  security: {
    level: 'strict',
    input: {
      injectionDetection: { enabled: true, onDetection: 'block' },
      pii: { enabled: true, action: 'mask', detect: ['email', 'phone', 'credit-card', 'aws-key', 'private-key'] },
      secrets: { enabled: true, action: 'block' },
    },
    output: { piiRedaction: true, maxContentLength: 8000 },
  },
});
```

A blocked request or response raises `NexusSecurityError`. Its `findings` say what was found. Every
matched value in them is redacted, so the error never repeats the leak.

## Levels

A `SecurityLevel` sets what runs when nothing more specific is configured:

| Level | What it does |
| --- | --- |
| `off` | Nothing. Streams are not buffered. |
| `basic` | Schema, secret, URL, and tool checks on input; redaction on output. |
| `standard` | Adds injection and PII detection. Both record findings rather than block. |
| `strict` | Injection blocks, and PII is masked. |
| `paranoid` | PII blocks too, after masking. |

Detailed settings override the level. For example, `onDetection: 'block'` blocks injection at
`standard`, and `pii.enabled: false` turns PII detection off at `paranoid`.

A `SecurityPreset` is a starting configuration for a kind of deployment: `developer`, `startup`,
`enterprise`, `healthcare`, or `finance`. Your own settings are merged over it.

## What the guardrails report

Every guardrail reports a `SecurityFinding`:

| Field | What it holds |
| --- | --- |
| guardrail | Which guardrail found it. |
| severity | From `low` to `critical`. |
| message, location | What was found, and where. |
| match, metadata | The matched value, and details specific to the guardrail. |

What a guardrail does with a match is its `SecurityAction`: let it through, block the request, mask
the match, or only record it.

A check returns a `SecurityResult`. It says whether the value may go ahead, and gives the value after
masking and redaction, the findings, and the guardrails that ran.

## Input guardrails

| Guardrail | What it checks | What it does by default |
| --- | --- | --- |
| Schema | The request's shape: roles, content parts, tool definitions. | Always blocks a malformed request. |
| Prompt injection | Known attack phrasings, plus your own patterns. | Depends on the level. |
| Semantic injection | Similarity to example attacks. Catches rephrasings no pattern lists. | Off until enabled. |
| Personal data | Email addresses, phone and card numbers, IP addresses, AWS keys, private keys. | Depends on the level. |
| Secrets | Private keys; cloud, payment, GitHub, Slack, and Google keys; JWTs; `password = "…"`. | Blocks. |
| URLs | Loopback and cloud-metadata addresses, onion services, secret-like query parameters. | Records. |
| Tools | Tools offered to the model that are not on `input.tools.allowedNames`. | Blocks. |
| Length | Text longer than `input.maxContentLength`. | Blocks. |

Each one is a class you can use on its own:

- `SchemaValidator` checks the request's shape: every field, by path. The checks are written out rather than
  declared through a schema library, so the guardrails add no dependency to an install.
- `InjectionDetector` matches instruction overrides, role jailbreaks, prompt exfiltration, and
  synthetic role tags, plus your `customPatterns`. `InjectionDetectionConfig.onDetection` blocks the
  request, records the finding, or, with `transform`, rewrites the attack phrases into inert markers
  and sends the request on.
- `SemanticInjectionClassifier` compares each message with example attacks by vector similarity.
  Enabled through `injectionDetection.semantic`, it uses hashed term vectors and needs no provider.
  For a real embedding model, construct it with `embed` and call its asynchronous `detect()`.
- `PIIDetector` finds each `PIIType`. `PIIConfig` chooses the types and the action: `mask` replaces
  the match, keeping its shape unless `preserveFormat` is off; `remove` deletes it; `block` refuses
  the request; and `flag` records it.
- `InputGuard` is the secret, URL, tool, and length checks together.

## Output guardrails

`OutputGuard` checks a response and its tool-call arguments. It:

- redacts secrets and personal data;
- removes database connection strings;
- records proprietary terms you name;
- cuts a response that is too long;
- applies moderation terms and forbidden topics;
- checks that the answer's words overlap the context you give it.

Moderation and topics block only when you tell them to. A blocked response's text is never returned,
not even inside the failed result.

## Running them yourself

`SecurityPipeline` runs everything, and it is what the client uses. Use it on its own to check text
that never goes through the client:

```ts
import { SecurityPipeline } from 'nexus-ai-pro/security';

const pipeline = new SecurityPipeline({ level: 'strict' });
const input = pipeline.protectInput(request);
pipeline.assertSafe(input); // throws NexusSecurityError when it failed
```

| Method | What it does |
| --- | --- |
| `protectInput()` | Checks a request. |
| `protectOutput()` | Checks a response. |
| `assertSafe()`, `assertOutputSafe()` | Throw `NexusSecurityError` for a result that failed. |

When security is on, streamed output is buffered and checked before any chunk is released. So a
secret split across chunks cannot leak in part. The [client guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/core.md#streaming) has the
limits.

## Policies

`GUARDRAIL_POLICIES` holds bundled configurations, each named by a `GuardrailPolicyName`:

| Policy | What it does |
| --- | --- |
| `owasp-llm` | Blocks injection and secrets, flags risky URLs, and redacts output and connection strings. |
| `pii-safe` | Masks personal data on input, and redacts it on output. |
| `rag-grounded` | Checks that answers overlap the context you pass as `output.grounding.context`. |
| `tool-safe` | Blocks injection, and enforces a tool allowlist once you give it the names. |
| `enterprise-strict` | The enterprise preset, with URL blocking and data-loss checks. |

`guardrailPolicy()` returns one with your overrides merged over it:

```ts
import { guardrailPolicy } from 'nexus-ai-pro/security';

const ai = new NexusAI({
  providers,
  security: guardrailPolicy('tool-safe', { input: { tools: { allowedNames: ['search', 'lookup_order'] } } }),
});
```

## Hardening a prompt

`hardenPrompt()` wraps each plain-text user message in delimiters. It then puts a system instruction
first, telling the model to treat delimited text as data, not instructions. It detects nothing; it
makes an injection less likely to work. `PromptHardeningOptions` sets the delimiter and the
instruction.

## Calibrating the semantic classifier

The right similarity threshold depends on your traffic. `calibrateSemanticInjectionClassifier()`
scores the classifier at several thresholds against labelled prompts:

```ts
import { calibrateSemanticInjectionClassifier } from 'nexus-ai-pro/security';

for (const result of calibrateSemanticInjectionClassifier(myLabelledPrompts)) {
  console.log(result.threshold, result.precision, result.recall);
}
```

Each prompt is an `InjectionCalibrationExample`: the prompt, whether it is an attack, and a category.
Without your own, it uses `SEMANTIC_INJECTION_CALIBRATION_SET`: five attacks, and five safe prompts
that resemble them. Each threshold gets an `InjectionCalibrationResult`: accuracy, precision, recall,
and the false-positive and false-negative rates. It scores the hashed-vector check the pipeline runs.

## Scanning uploads

`UploadScanner` checks files before they reach a model or a store:

```ts
import { scanUploads } from 'nexus-ai-pro/security';

const result = scanUploads([{ name: 'notes.txt', content: text }]);
if (!result.ok) console.log(result.findings);
```

It checks four things, each set by `UploadScannerOptions`:

- the file's size;
- a blocked extension, executables and scripts by default;
- an allowed MIME type;
- forbidden patterns in text contents: private keys, cloud and GitHub tokens, and instruction
  overrides by default.

A `FileUpload` is a name, an optional MIME type and size, and the contents. An `UploadScanResult`
fails on any high or critical `UploadScanFinding`. Each finding names the file, the severity, the
problem, and the matched text. `scanUploads()` is a one-off scan. `ingestFilesAfterScan()`, in the
[grounding guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/grounding.md), scans before it ingests.

## Elsewhere

- `createFetchUrlTool()`, in the [agents guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/agents.md), fetches URLs safely. It pins DNS,
  validates redirects, blocks private networks, and bounds how much it reads.
- CLI scans and audit records redact detected values by default. `nexus scan --reveal-values` works
  only in an interactive terminal. Raw audit data needs both `includeSensitiveData: true` and an
  explicit custom sink.

## Limitations

- Detection works by pattern and similarity. It catches known phrasings and common formats, not
  every attack or every piece of personal data. Treat it as a layer, not a boundary.
- An answer whose text the output guard changes keeps no log probabilities, streamed or not: they name
  the tokens the model wrote, so a redacted address would still be spelled out in them.
- Blocking applies to critical and high findings. An instruction block (`BEGIN SYSTEM`) or a role tag
  (`<system>`) is medium, so even `strict` records it and lets the request through. Pair the input
  guard with output moderation, `onViolation: 'redact'` for what must never leave. `proprietaryTerms`
  only records a match.
- Each pattern names an example of the attack it catches. `attackSeeds()` returns them, as
  `AttackSeed`s with an `AttackCategory`, for an adversarial dataset; the
  [evaluation guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/evaluation.md) builds one.
- The grounding check compares words, not meaning. The [grounding guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/grounding.md) has
  claim-level checks.
- Buffered streaming trades token-by-token latency for safety. Turn security off for a stream only
  when that trade is acceptable.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/graphs.md

# Graphs


Stateful graphs from `nexus-ai-pro/graph`: typed channels, parallel branches, conditional and deferred edges, interrupts for human input, checkpoints that survive a restart, time travel, subgraphs, per-node caching, and a Mermaid visualizer on `nexus-ai-pro/graph/visualize`. A graph-only application imports no third-party package.

## Overview

A graph is nodes, edges, and typed state. Cycles, fan-out, subgraphs, and human approval are
supported shapes rather than workarounds, and every run is checkpointed, so it is resumable by
construction rather than after configuring a checkpointer.

```ts
import { createGraph, appendList, counter, END, MemoryGraphCheckpointer } from 'nexus-ai-pro/graph';

const graph = createGraph({
  channels: { messages: appendList<string>(), turns: counter() },
})
  .addNode('research', async (ctx) => ({ messages: [await search(ctx.state.messages)], turns: 1 }))
  .addNode('answer', async (ctx) => ({ messages: [await ai.complete(...).then((r) => r.content)] }))
  .setEntry('research')
  .addConditionalEdges('research', (state) => (state.turns >= 3 ? 'answer' : 'research'))
  .addEdge('answer', END)
  .compile({ checkpointer: new MemoryGraphCheckpointer() });

const result = await graph.invoke({ messages: ['who won?'] }, { threadId: 'q-42' });
```

**Channels, not assignment.** Each state slot declares how writes combine — `lastValue`,
`appendList`, `appendSet`, `mergeObject`, `counter`, or your own `reducerChannel`. That is what makes
fan-out safe: two branches running in the same superstep can both write, and the channel decides
whether that means overwrite, append, or sum. Assignment would silently drop one branch's work.

**Cycles are first-class**, so `maxSteps` is what stands between a mistaken router and an infinite
loop. Exceeding it names the nodes still pending rather than hanging.

**Branches run in parallel.** Every task in a superstep starts together, so four branches of three
seconds each finish in about three seconds rather than twelve:

```ts
.addConditionalEdges('plan', () => ['research-a', 'research-b', 'research-c', 'research-d'])
```

`maxConcurrency` caps how many run at once — 16 by default, settable per compile and per run, and
`1` restores strictly sequential execution. Whatever the timing, writes are reduced in task order, so
a replay produces the same state. A superstep with one task skips the scheduler entirely.

**Fan out over data decided at run time.** Edges can only name nodes that already exist. `Send`
creates one task per value instead, each reading its own `context.input`:

```ts
import { Send } from 'nexus-ai-pro/graph';

.addNode('research', async (ctx) => ({ findings: [await fetch(ctx.input as string)] }), { ends: [END] })
.addConditionalEdges('plan', (state) => state.urls.map((url) => new Send('research', url)))
```

Fifty-seven URLs become fifty-seven tasks of one node in a single superstep, bounded by
`maxConcurrency` and checkpointed individually, so a resume re-runs only the copies that did not
finish. Declaring `ends` keeps compile-time reachability checks exact.

**Retries and timeouts per node.** A node that calls a flaky service can retry on its own, without
wrapping every node body in the same try/catch:

```ts
.addNode('fetch', fetchNode, {
  retry: { maxAttempts: 3, initialIntervalMs: 500, backoffFactor: 2, jitter: true },
  timeoutMs: 10_000,
})
```

Retrying defaults to off, because only you know whether a node is idempotent; `compile({ retry })`
sets a default for every node. Interrupts, aborts, and validation errors are never retried.
`context.attempt` tells a node which try it is on, and a timeout aborts the node's signal and fails
that attempt. When one task fails, its siblings are aborted by default; `onNodeError: 'settle'` lets
them finish first, and either way what they already wrote is kept.

**Several questions at once.** Parallel tasks can each interrupt. The paused result carries every
pending question, and they can be answered together or a few at a time:

```ts
const run = await graph.invoke(input, { threadId });
await graph.resumeInterruptsWith(threadId, {
  [run.interrupts[0].id]: true,
  [run.interrupts[1].id]: 'use the second draft',
});
```

**Human in the loop.** A node calls `interrupt()`; the graph checkpoints and stops:

```ts
.addNode('approve', (ctx) => ({
  approved: ctx.interrupt<boolean>({ reason: 'Publish this draft?', payload: { chars: 1200 } }),
}))
```

```ts
const run = await graph.invoke(input, { threadId });
if (run.status === 'awaiting_input') {
  // Hours later, in another process:
  await graph.resumeWith(threadId, true);
}
```

The interrupted node runs again from the top and `interrupt()` returns the supplied value instead of
throwing, so the node body reads as straight-line code either way. Keep the work before an interrupt
cheap, because it happens twice. A sibling branch that already finished is **not** re-run — its
writes were checkpointed before the graph suspended.

**Time travel and inspection.** Every superstep is a checkpoint:

```ts
await graph.state(threadId);        // latest checkpoint
await graph.history(threadId);      // newest first
graph.resumeFrom(threadId, 3);      // rewind and run forward
```

**Route from inside a node.** Sometimes a node already knows where to go: an agent that just picked a
tool, or a triage step that classified a ticket. Return a `Command` instead of adding a separate
router:

```ts
import { Command } from 'nexus-ai-pro/graph';

.addNode('triage', (ctx) =>
  new Command({ update: { label: classify(ctx.state) }, goto: isUrgent(ctx.state) ? 'page-oncall' : 'queue' }),
  { ends: ['page-oncall', 'queue'] },
)
```

`goto` takes node names or `Send`s, and `ends` keeps compile-time checks and diagrams exact. A node
inside a subgraph can return `new Command({ graph: Command.PARENT, goto: 'human' })` to hand control
back to the graph that contains it — how a nested agent escalates.

**Aggregate after branches of different lengths.** `{ defer: true }` holds a node until every other
pending task has finished. An aggregator after a two-step branch and a one-step branch then runs once,
after both, instead of once per arrival.

**Pause for inspection.** Breakpoints stop a run before or after chosen nodes, checkpointed, without
changing the nodes themselves:

```ts
const graph = builder.compile({ interruptBefore: ['publish'] });
const paused = await graph.invoke(input, { threadId });   // status: 'interrupted', breakpoint: { when: 'before', ... }
await graph.updateState(threadId, { draft: fixedDraft }); // correct something first, if needed
for await (const _ of graph.continue(threadId)) {}         // then carry on
```

Set them per compile or per run; a run's list replaces the compiled one.

**Edit state, or fork a thread.** `updateState(threadId, update)` merges a correction into the latest
checkpoint through the channel reducers, leaving any pending question in place.
`updateState(threadId, update, { asNode: 'research' })` applies it as if that node had produced it,
so the next step follows that node's edges. `fork(threadId, { step })` copies the history up to a step
into a new thread and leaves the original untouched, so two answers to the same question can be run
side by side.

**Watch it run.** `onEvent` receives every task starting, retrying, and finishing, and every
checkpoint written. It also receives whatever a node passes to `context.emit()`, such as model tokens
as they stream, a status line, or an intermediate result:

```ts
.addNode('write', async (ctx) => {
  for await (const token of streamAnswer(ctx.state)) ctx.emit({ token });
  return { answer };
})

await graph.invoke(input, {
  onEvent: (event) => event.type === 'custom' && process.stdout.write((event.data as { token: string }).token),
});
```

A listener that throws never fails the run it is watching.

**See the shape.** `graph.describe()` returns nodes, edges, and which routes are decided at run
time, as plain JSON. `toMermaid()` from `nexus-ai-pro/graph/visualize` draws it — solid arrows are
always taken, dotted ones are chosen at run time, and subgraphs are drawn inside the node that runs
them:

```ts
import { toMermaid } from 'nexus-ai-pro/graph/visualize';

console.log(toMermaid(graph, { highlight: checkpoint.next }));
```

This diagram is that function's actual output for a research-and-review graph:

```mermaid
flowchart TD
  __start__([start])
  __end__([end])
  n_plan["plan"]
  n_research["research<br/><small>retries</small>"]
  n_summarize["summarize<br/><small>deferred</small>"]
  n_review["review"]
  n_publish["publish"]
  __start__ --> n_plan
  n_summarize --> n_review
  n_publish --> __end__
  n_plan -.-> n_research
  n_research -.-> n_summarize
  n_review -.-> n_publish
  n_review -.-> n_plan
```

The visualizer is its own subpath and imports no runtime code, so drawing a graph costs nothing to an
application that only runs one. No image renderer is bundled; any Mermaid renderer turns the output
into PNG or SVG.

**Durable by construction.** The default checkpointer is in-process, holding up to 1,000 threads;
pass `checkpointer: false` to turn checkpointing off. Point it at the operation store
that already backs durable operations and a thread survives a restart — a different worker resumes
what another suspended:

```ts
import { OperationStoreCheckpointer } from 'nexus-ai-pro/graph';
import { RedisOperationStore } from 'nexus-ai-pro/operations/adapters';

const checkpointer = new OperationStoreCheckpointer(new RedisOperationStore(redis));
```

Writes are compare-and-set on the record's sequence, so two workers advancing the same thread cannot
both win. `PostgresOperationStore` and `SqliteOperationStore` work the same way; see
[Postgres](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/postgres.md) and [SQLite](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/sqlite.md).

**Subgraphs.** A compiled graph is a node:

```ts
parent.addNode('research', researchGraph.asNode());
```

Channels shared by name are passed in and merged back; anything the parent does not declare stays
private to the subgraph.

**Input and output channels.** Working channels can stay internal:

```ts
const research = createGraph({
  channels: { question: lastValue(''), notes: appendList<string>(), answer: lastValue('') },
  input: ['question'],
  output: ['answer'],
});
```

`invoke()` accepts only `question` and returns only `answer`, in the types as well as at run time. As
a subgraph it receives only its inputs from the parent and merges back only its outputs, so a parent
with its own `notes` channel never sees these. Checkpoints, `state()`, and stream events still carry
the whole state, because they describe the thread rather than answer the caller.

**Caching node results.** An expensive node whose result depends only on what it reads can skip work
it has already done:

```ts
graph.addNode('classify', classifyTicket, {
  cache: { ttlMs: 10 * 60_000, key: ({ state }) => `classify:${(state as Ticket).body}` },
});
```

The default key hashes the graph name, the node, the state it sees, and its `Send` input — always
correct, sometimes too specific, which is what `key` is for. Results live in a bounded in-process map
by default, or in any cache adapter: `compile({ cache: new RedisCacheAdapter(redis) })` shares them
across workers. A failure, an interrupt, a node that consumed a human's answer, and a command for a
parent graph are never cached, and a cache that is down is a miss rather than a failed node. The
caching code loads the first time a caching node runs.

The graph is not in the root import. It costs nothing to a user who does not build graphs.

## Building a graph

`createGraph()` returns a `StateGraph` — the builder — over a `ChannelSchema`, the channels by name.
Each `Channel` is a `reduce()` that combines the value already there with a write, and an optional
`initial()`. `StateOf` is the state a schema describes, and `StateUpdate` is what a node may write
back: any subset of the channels. `GraphInput` is what `invoke()` accepts, narrowed to the declared
input channels.

`addNode()` takes a name, a `NodeFn`, and `NodeOptions`:

| Option | What it sets |
| --- | --- |
| `retry` | A `RetryPolicy`: attempts, first delay, backoff factor, a ceiling, jitter, and a `retryOn` test. |
| `timeoutMs` | How long one attempt may take. |
| `ends` | The nodes it can route to, for the diagram. |
| `defer` | Runs it only after every other branch has finished. |
| `cache` | A `NodeCachePolicy`: the `key`, a `ttlMs` (5 minutes), and a `store`. A cached result is a `NodeCacheEntry`: the updates the node returned, and where it routed. |

Edges connect nodes:

- `setEntry()` names the first node, and `addEdge()` joins two.
- `addConditionalEdges()` routes by state. It takes an `EdgeRouter`: a function of state that returns a
  `GraphRouteTarget` — a node name, a `Send`, or a list of either. An optional map turns its answers
  into node names.
- `START` and `END` are the entry and exit markers.

A node receives a `NodeContext`:

| Field | What it is |
| --- | --- |
| state | The state so far, frozen. |
| name, step, thread, task id | Where the node is running. |
| `Send` input, attempt | What this task was given, and which try this is. |
| signal | Aborted on cancellation or timeout. |
| `interrupt()` | Asks a person and waits for the answer. |
| `store` | Long-term memory. |
| `report()`, `emit()` | Progress, and custom events. |

It returns a `NodeResult`: an update, a `Command`, or nothing. A `Command` names what runs next with a
`CommandTarget`: a node, a `Send`, or a list.

`compile()` checks the graph and returns a `CompiledGraph`. `CompileOptions` holds the `store`, the
default node `cache`, breakpoints, `maxConcurrency`, a default `retry`, `onNodeError`, the
`checkpointer`, `maxSteps`, a `name` recorded on every checkpoint, and a clock for tests.

## Running a graph

`invoke()` runs to the end and returns a `GraphResult`: the thread, the status, the output state, the
steps completed, and any interrupts, breakpoint, or error. The `GraphStatus` is one of `running`,
`awaiting_input`, `completed`, `failed`, or `interrupted`.

`stream()` yields a `GraphStepEvent` per superstep: the nodes and tasks that ran, retry attempts, the
state, and the status. It ends with an `interrupt`, `breakpoint`, or `done` event.

A `GraphTask` is one unit of work: its id, its node, and its `Send` input.

Compile with `lifecycle: ai.lifecycle`, and each run becomes one operation of the client: authorized
before its first step, audited and counted when it ends. `lifecycleFamily` labels it `graph`, or
`agent` as `createAgent()` sets. The [lifecycle guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/lifecycle.md) has the details.

`GraphRunOptions` sets the `threadId`, `maxSteps`, a signal, metadata for every checkpoint,
`maxConcurrency`, breakpoints for this run, `onEvent`, `onProgress`, its `durability`, and a `control`
to drain it. `onEvent` receives each `GraphEvent`: `task_start`, `task_retry`, `task_failed`,
`task_end` with the update and whether it was served from the cache, `checkpoint`, `custom`, and
`step`, with the state after each superstep. `onProgress` receives each `GraphProgress` a node reported.

A node asks a question with an `InterruptRequest`: a `reason` and a JSON `payload`. Each question
waits as a `PendingInterrupt`, with its id, node, task, step, position within the node, and when it
was asked.

| Method | What it does |
| --- | --- |
| `resume()`, `resumeWith()` | Answer the one pending question. |
| `resumeInterrupts()`, `resumeInterruptsWith()` | Answer several, by id. |
| `continue()` | Carries on from a breakpoint, a drain, or a failure. On a finished thread it changes nothing and returns its state. |
| `continueWith()` | The same, returning the final result: what a replica calls to finish a run whose process stopped. |
| `resumeFrom()` | Rewinds to a step and runs forward. |

`interruptKey()` is the stable key an answer is stored under: task, step, and index. That is how
several `Send` tasks of one node keep their own answers.

### A run for one tenant

A `tenantId` run option ties a thread to a tenant. It is recorded on every checkpoint as
`metadata.tenantId`, nodes read it as `context.tenantId`, and `context.store` becomes the tenant's own
view of the graph's store. Another tenant cannot use the thread: `resume()`, `continue()`,
`resumeFrom()`, `fork()`, and `updateState()` with a different `tenantId` throw
`GraphThreadNotFoundError` before anything runs, `state()` and `history()` given one read it as
absent, and a new run on its id is refused. A run that names no tenant carries on the thread's, so the
owner resumes as before.

A `principal` run option says who the run is for, as the server's authentication reports them.
Nodes read it as `context.principal`, and an agent passes it on to its tools and middleware. Its
`tenantId` becomes the run's tenant when `tenantId` is not given, and a run whose two tenants differ
is refused. Its `userId` is recorded on every checkpoint as `metadata.userId`. Nothing else in it is
stored: a resumed run carries the principal of whoever resumed it. A subgraph runs for the same
tenant and principal as its parent.

```ts
await graph.invoke(input, { threadId, principal });   // its tenant, and who asked
await graph.invoke(input, { threadId, tenantId: principal.tenantId });
await graph.resumeWith(threadId, answer, { tenantId: principal.tenantId });
const checkpoint = await graph.state(threadId, undefined, { tenantId: principal.tenantId });
```

The [tenancy guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/tenancy.md) covers the tenant model across every store.

## Streaming events

`stream()` yields one event per superstep. A chat window, a progress view, or a live trace needs more
than that: model output as it is written, tool calls as they start and finish. `events()` runs the
graph as one stream of typed events instead, and every part of the package that follows a run reads
the same one.

```ts
const run = agent.events(agentInput('What changed in the contract?'), { threadId });

for await (const { chunk } of run.messages()) process.stdout.write(chunk.content);
const result = await run.result;
```

Each event is a `GraphStreamEvent`, tagged by its `GraphStreamProjection`:

| Projection | What it carries |
| --- | --- |
| `values` | The state and status after each superstep |
| `updates` | Each task's write, with its node |
| `messages` | A `GraphMessageChunk` of model output: its text, `text` or `reasoning`, and an optional message id |
| `tools` | A `GraphToolEvent`: a call starting with its arguments, its progress, then its result or error |
| `tasks` | Each task starting, retrying, failing, and ending |
| `checkpoints` | Each checkpoint written |
| `custom` | What nodes pass to `emit()` |

`GraphEventsOptions` are the run's own options plus `include`, which defaults to `values`, `updates`,
`messages`, `tools`, and `custom`. `resumeEvents()` and `continueEvents()` do the same for an
answer to an interrupt and for a thread carried on. A node streams model output with
`context.message()` and reports tool calls with `context.tool()`; `createAgent()` does both for you.

The returned `GraphEventStream` can be iterated for everything included, or read one projection at a
time through `messages()`, `tools()`, `values()`, and `updates()`. Create every reader before
reading any: the run starts at the first read, and stops once every reader has stopped. `result`
resolves with the run's result.

**Subgraphs.** A graph used as a node keeps its events to itself by default. With `subgraphs: true`
they come through too, each with a `namespace` naming the nodes it came through, such as
`['review']`. `GraphRunOptions.subgraphEvents` does the same for `onEvent`.

**Slow readers.** Every reader has its own buffer of `maxBuffered` events (1,000). At the end of each
superstep the run waits for its slowest reader, so a reader that falls behind slows the graph rather
than filling memory. Within a superstep, a node can still write faster than a reader reads, and
`overflow`, a `GraphStreamOverflow`, decides what happens when a buffer is full:

| Policy | When the buffer is full |
| --- | --- |
| `coalesce` (the default) | A model chunk is merged into the one before it from the same task, and a state snapshot replaces the older one, so no text is lost. When nothing can merge, the oldest event goes. |
| `drop-oldest`, `drop-newest` | Events are lost, and counted. |
| `error` | The stream ends with `GraphStreamOverflowError` and the run stops. |

`streamStats()` returns `GraphStreamStats`: the most any reader held at once, and how many events were
merged or dropped. A reader a hundred times slower than a node that streams 100,000 chunks still holds
no more than its buffer.

## State and checkpoints

A `GraphCheckpoint` is everything needed to resume a thread elsewhere:

- its schema `version`, 2, and its `id`, as `<threadId>:<step>`;
- the step, and every channel's value;
- the nodes to run next, and the `tasks` that run them, always spelled out;
- for a paused step, the tasks that already finished and the routes they chose;
- the status, every pending question in `interrupts`, and the answers already given;
- an error, and a `GraphBreakpoint` (`before` or `after`, and its nodes);
- when it was written, and its metadata.

`state()` and `history()` read checkpoints, `updateState()` edits one, and `fork()` copies a thread.

A `GraphCheckpointer` stores them: `put()`, `get()` the latest or one step, `history()` newest first,
and optionally `delete()` and `threadIds()`. `MemoryGraphCheckpointer` is the default, bounded by
`MemoryGraphCheckpointerOptions` — `maxPerThread` checkpoints (50) and `maxThreads` (1,000, after
which the least recently written thread is dropped). `OperationStoreCheckpointer` writes each
checkpoint as an operation record, with `OperationStoreCheckpointerOptions.maxPerThread` (50), so any
operation store — memory, Redis, or Postgres — makes a thread durable.

### Checkpoints 1.x wrote

2.x and 3.0 write version 2 and read version 1, so a thread a 1.x deployment left
waiting resumes after the upgrade. A `GraphCheckpointV1` is that older shape: no version or id, tasks
only when `next` could not say them, and the first question repeated in `interrupt`.

A checkpointer's `get()` and `history()` return a `StoredGraphCheckpoint`, which is either shape. The
bundled checkpointers read version 1 into version 2, and so does the graph for a checkpointer of your
own. `migrateCheckpoint()` does it for code that reads a store directly:

```ts
import { migrateCheckpoint } from 'nexus-ai-pro/graph';

const stored = await checkpointer.get('order-991');
const checkpoint = stored && migrateCheckpoint(stored);
checkpoint?.id;         // 'order-991:3'
checkpoint?.tasks;      // always present, one per node or Send
checkpoint?.interrupts; // every pending question, empty when none
```

A version 1 checkpoint gains its id, its tasks spelled out from `next` when it carried none, and its
question moved into `interrupts`. A version 2 checkpoint passes through unchanged.

To build a checkpoint by hand, for a seed or a test, write a `CheckpointDraft` — a checkpoint without
its version, id, tasks, and questions — and pass it to `toCheckpoint()`. It fills those in, and always
derives the id again, so a copy moved to another thread never keeps a stale one.

## Surviving failure

A graph in production meets slow databases, failing services, hung calls, and workers that are
stopped mid-run. Each of those has a setting, and none changes how a graph behaves until you use it.

### When checkpoints are written

`compile({ durability })` takes a `DurabilityMode`. `GraphRunOptions.durability` overrides it for
one run.

| Mode | Writes | A crash loses | For |
| --- | --- | --- | --- |
| `sync` (the default) | Each checkpoint, before the next superstep starts | Nothing that finished | Side effects that must line up with state, such as payments |
| `async` | Each checkpoint, in the background while the next superstep runs | The last few supersteps, which run again | Most agents |
| `exit` | Only where the run stops: an interrupt, a breakpoint, completion, a failure, an abort, or a drain | The whole run | Short, high-throughput graphs |

`async` keeps every write and its order. The writes still go one at a time, because a checkpointer
replaces anything at a later step, so the gain is that the graph stops waiting: it does the next
step's work while the store catches up. Against a store with an 8 ms round trip, a 50-step graph
whose steps each take about as long finishes in about half the time. The run waits for every write
before it returns, pauses, fails, or drains. `maxPendingWrites` (8) caps the writes in flight, so a
slow store slows the graph down rather than filling memory. A write that fails in the background fails
the run. `flush()` on a compiled graph waits for the writes of every run in progress.

### Recovering after retries

A node's retry policy decides how often it tries. `onError` decides what happens when the tries run
out. It receives a `NodeFailure` and returns what a node returns: a state update, a `Command`, or
nothing to carry on along the node's own edges. Throwing fails the graph as before.

```ts
import { Command } from 'nexus-ai-pro/graph';

.addNode('charge', chargeCard, {
  retry: { maxAttempts: 3 },
  ends: ['refund'],
  onError: (failure) =>
    new Command({ update: { status: 'payment_failed' }, goto: 'refund' }),
})
```

A `NodeFailure` holds the node and task, the step, the attempts made, when the first attempt started
and the last one failed, the error, whether the retry policy ran out or the error was not retryable,
the timeout that ended the attempt if one did, the `Send` input, and the thread. A `NodeErrorHandler`
is the function's type. It decides and should not act: compensation with side effects belongs in the
node it routes to.

The decision is part of durable state. The checkpoint after the step lists each recovery as a
`RecoveredFailure` in `recovered`, with the error, the attempts, and where the run went, and that
checkpoint is written before anything the decision routes to can run, in every durability mode. A
crash after it resumes into the same compensation path, and the failed node is not run again. The step
event carries the same list, and `onEvent` gets a `task_failed` event, with `recovered` set when
`onError` handled it. A mistake in the graph itself, a `GraphValidationError`, is never recovered.

`compile({ nodeDefaults })` sets a retry policy, timeouts, and an `onError` for every node at once.
A node's own options win, field by field.

### Run and idle timeouts

A `NodeTimeout` sets two limits. `runMs` caps an attempt: it may never take longer. `idleMs` caps
the time between signs of progress: the attempt may run as long as it likes while it keeps showing it
is alive. `timeoutMs` is the same as `timeout.runMs`.

```ts
.addNode('crawl', async ({ heartbeat, emit }) => {
  for (const page of pages) {
    emit({ page: await fetchPage(page) }); // progress, which also refreshes the idle timer
    heartbeat();                           // or just that, for work with nothing to report
  }
}, { timeout: { runMs: 10 * 60_000, idleMs: 30_000 } })
```

`context.heartbeat()`, `context.emit()`, and `context.report()` each refresh the idle timer. Either
limit aborts the node's signal and fails the attempt with `GraphNodeTimeoutError`, whose `kind` says
which. The retry policy may retry it, and `onError` sees it in `failure.timeout`.

A `Send` can give the task it creates its own limits through `SendOptions`, such as more time for one
large document. The override is saved with the task, so it holds after a resume:

```ts
new Send('summarize', doc, { timeout: { runMs: 5 * 60_000 } })
```

### Draining a run

An abort signal stops a run in the middle of a superstep and cancels the nodes in flight. A
`RunControl` stops it cleanly instead: the superstep in flight finishes, its checkpoint is written,
and the run ends with `GraphDrainedError`, which names the thread and the step. Any process can then
`continue()` the thread.

```ts
import { RunControl } from 'nexus-ai-pro/graph';

const control = new RunControl();
process.on('SIGTERM', () => control.drain('sigterm'));

await graph.invoke(input, { threadId, control });
```

One control drains any number of runs. `onDrain()` registers a listener. The checkpoint records the
drain in `drained`, with its reason. `GraphRunOptions.control` takes any `RunControlLike`, an object
with `draining` and `reason`. The agent server passes one to every graph it serves, and drains
them when the replica drains (see the [server guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/server.md)).

### What repeats after a crash

A crash can run some work twice. This is exactly what, so that a side effect can be written to be
safe:

| Primitive | Runs again after a crash? | What is durable | A side effect must |
| --- | --- | --- | --- |
| Graph node, `sync` | The superstep it was in, if its checkpoint was not written | Every finished superstep | Be idempotent |
| Graph node, `async` | The supersteps not yet written | Every superstep the store has acknowledged | Be idempotent |
| Graph node, `exit` | The whole run | Pauses, failures, drains, and the end | Be idempotent, or the graph has none |
| A node before an interrupt | Yes, once more when the answer arrives | The question and its answer | Come after the interrupt |
| A cached node | No, on a hit | The cached result | Not exist |
| `onError` | Only if the crash came before its checkpoint | The decision, before its target runs | Not exist: route to a node instead |
| Functional workflow step | Only the step in flight | Each finished step's result | Be idempotent |
| Operation attempt | The whole attempt | The record, and its last heartbeat details | Be idempotent, and resume from `previousHeartbeat` |

Nothing here runs a side effect exactly once. An idempotency key on the call that makes it is what
turns "at least once" into "effectively once".

### Linting a graph

`lintGraph()` from `nexus-ai-pro/graph/lint` reads a compiled graph's shape — or a description saved
as JSON — and returns a `GraphLintFinding` for each design that works in a demo and fails in
production. Nothing runs. `nexus graph lint` runs it from the command line (see the
[command-line guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/cli.md)).

| `GraphLintCode` | `GraphLintSeverity` | Finds |
| --- | --- | --- |
| `UNBOUNDED_CYCLE` | error | A cycle of static edges, which only the step limit can stop |
| `RETRY_WITHOUT_IDEMPOTENCY` | warning | A retried node not declared `idempotent` |
| `RETRY_WITHOUT_TIMEOUT` | warning | A retried node with no timeout, whose hung attempt is never retried |
| `DEFERRED_DURABILITY_WITH_SIDE_EFFECTS` | warning | `async` or `exit` durability with nodes not declared `idempotent` |
| `MEMORY_CHECKPOINTER` | warning, or error when `deployed` | The in-process default, which a restart loses |
| `NO_CHECKPOINTER` | info | No checkpoints, so no interrupt, resume, drain, or recovery |
| `UNBOUNDED_CONCURRENCY` | warning | A superstep with no limit on tasks at once |
| `DYNAMIC_ROUTE` | info | A router with no mapping and no `ends`, whose routes cannot be checked |
| `SENSITIVE_TOOL_WITHOUT_APPROVAL` | warning, or error when `deployed` | A tool that writes, runs commands or code, or reaches the network, with no approval and no permission policy |
| `UNDECLARED_TOOL_CAPABILITIES` | info | A tool with no capabilities and no approval, whose effects cannot be judged |
| `SIDE_EFFECT_BEFORE_INTERRUPT` | warning | A node that declares `effects` and `interrupts`, so its effects repeat when it runs again on the answer |

Each finding has its code, severity, node (a path such as `review/approve` inside a subgraph), what
is wrong, and the fix. `GraphLintOptions` sets `deployed` for a graph that runs where restarts and
hand-offs are normal, and `ignore` for codes to skip.

A node declares `idempotent: true` when running it twice is safe. Nothing at run time depends on it;
it is what lets the linter tell a safe retry from a dangerous one. A node can also declare its
`effects`, as capabilities, and that it `interrupts`. A graph compiled with `tools` — `createAgent()`
passes its own — describes each as a `GraphToolDescription`: its name, its fixed capabilities or
that it computes them per call, and how its calls are approved (`interrupt`, `policy`, or `none`).
The [agents guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/agents.md) covers capabilities and permission policies.

```ts
import { lintGraph } from 'nexus-ai-pro/graph/lint';

for (const finding of lintGraph(graph, { deployed: true })) {
  console.log(`${finding.severity} ${finding.code}: ${finding.message}`);
}
```

## Errors

Every graph error extends `GraphError` and carries a stable `code`:

- `GraphValidationError` — the definition is invalid: an unknown node, no entry point, an
  unreachable node, a bad channel.
- `GraphInterrupt` — control flow, not a failure. `interrupt()` throws it, and the runtime catches it
  and pauses. It reaches you only when a node interrupts on a graph compiled with `checkpointer:
  false`, which has nowhere to keep the question.
- `GraphStepLimitError` — the run exceeded `maxSteps`, naming the nodes still pending.
- `GraphNodeError` — a node threw; the original error is its `cause`.
- `GraphNodeTimeoutError` — a node outlived its run limit, or went past its idle limit without
  showing progress; `kind` is `run` or `idle`. It is separate because it is often worth retrying.
- `GraphDrainedError` — not a failure: the run stopped because its `RunControl` drained, after its
  superstep finished and was checkpointed. `continue()` resumes it.
- `GraphThreadNotFoundError` — no checkpoint exists for the thread.
- `GraphNotInterruptedError` — a resume was asked of a thread that is not waiting for input.

## Describing and drawing

`describe()` returns a `GraphDescription`:

- the name, and each node with its options and any subgraph;
- each edge, conditional or not, with its label;
- routers whose targets cannot be known in advance;
- the input and output channels;
- the durability mode, which checkpointer it writes to (`memory`, `custom`, or `none`), and its
  concurrency limit.

Each node lists its retry attempts, timeouts, whether it has an `onError`, and whether it is declared
`idempotent`. That is what `lintGraph()` reads.

`toMermaid()` draws anything `Describable` — a description, or anything with `describe()`. Its
`MermaidOptions` set the direction (`TD` or `LR`), whether subgraphs are expanded or collapsed, and
which nodes to highlight. `toGraphJSON()` returns a copy of the description, for a UI or a test.

`toSvg()` draws the graph as a standalone SVG document with no dependency — write it to a file,
inline it in a page, or hand it to any SVG rasterizer for PNG. `SvgOptions` sets the nodes to
highlight, a title for screen readers, and the colours: background, nodes, lines, text, highlights,
and edges that point back up. Names are escaped, conditional edges are dashed, and cycles curve round
the side.

```ts
import { writeFileSync } from 'node:fs';
import { toSvg } from 'nexus-ai-pro/graph/visualize';

writeFileSync('support-agent.svg', toSvg(graph, { highlight: checkpoint.next }));
```

`layoutGraph()` is the layered layout underneath, the same one the studio draws with. Each node sits
one layer below the nearest node that leads to it, and unreachable nodes go at the bottom.

It returns a `GraphLayout`. Each `LaidOutNode` has its position, size, kind, and marks for deferred or
cached nodes. Each `LaidOutEdge` has a `back` flag for an edge that points up — a cycle.

## Functional workflows

Some programs are better written as plain control flow — loops, early returns, `Promise.all` — but
still need what a graph gives. `workflow()`, on `nexus-ai-pro/graph/functional`, makes an ordinary
function durable.

Each `step()` records its result. When the workflow runs again — resumed, continued, or recovered on
another worker — finished steps return their recorded result without running. The function carries on
from the first step that did not finish.

```ts
import { workflow } from 'nexus-ai-pro/graph/functional';

const refund = workflow(
  async (input: { orderId: string }, { step, interrupt }) => {
    const order = await step('load', () => orders.get(input.orderId));
    const [risk, history] = await Promise.all([
      step('score-risk', () => risk.score(order), { timeoutMs: 5_000 }),
      step('load-history', () => orders.history(order.customerId)),
    ]);
    if (!interrupt<boolean>({ reason: `Refund ${order.total}?`, payload: { order, risk, history } })) {
      return { refunded: false };
    }
    await step('refund', () => payments.refund(order.id, { idempotencyKey: order.id }), {
      retry: { maxAttempts: 3 },
    });
    return { refunded: true };
  },
  { name: 'refund', checkpointer },
);

const paused = await refund.invoke({ orderId: 'o-42' }, { threadId: 'refund-o-42' });
// Hours later, on any worker with the same checkpointer:
const done = await refund.resumeWith('refund-o-42', true);
```

The function receives its input and a `WorkflowContext`: the thread id, a signal, the long-term
`store`, `step()`, `interrupt()`, and `emit()` for custom events. A `WorkflowFn` is that function.

- **`step(name, run, options)`** runs `run` once per thread. It receives a `StepContext` — the step's
  name, the key its result is recorded under, the attempt, a signal, and `heartbeat()` — and
  `StepOptions` set a `retry` policy, the limits below, and an `onError`. Name steps so the same
  call has the same name every time; a name used more than once is numbered in call order, `name#2`.
  Steps started together run in parallel, up to the workflow's `maxConcurrency`, and when one fails
  its siblings still finish and are recorded, so a retry repeats only the failure.
- **`interrupt(request)`** stops the run with status `awaiting_input` and the question pending. After
  an answer, the function runs again from the top — finished steps return their recorded results —
  and `interrupt()` returns the answer instead of stopping.

`WorkflowOptions` names the workflow and sets:

| Option | Default | Meaning |
| --- | --- | --- |
| `checkpointer` | memory | Where step results are kept. `false` keeps none. |
| `store` | — | Long-term memory for steps. |
| `retry` | — | A default retry policy for every step. |
| `stepDefaults` | — | Defaults for every step's `retry`, `timeout`, and `onError`; a step's own options win, field by field. |
| `maxConcurrency` | 16 | Steps running at once. |
| `lifecycle` | — | Runs each invocation as one operation of a client: `ai.lifecycle`. |
| clock | — | For tests. |

The `Workflow` it returns has these methods:

| Method | What it does |
| --- | --- |
| `invoke()`, `stream()` | Runs a thread, returning the result or streaming its events. |
| `resume()`, `resumeWith()` | Answers the pending question and runs on. |
| `resumeInterrupts()` | Answers several questions by id. |
| `continue()` | Runs a thread on after a crash, a failure, or a cancellation. |
| `state()`, `history()` | Reads its checkpoints. |
| `describe()` | Its shape, for a diagram. |

`WorkflowRunOptions` sets the thread id, a signal, and metadata for every checkpoint. Its `onEvent`
receives the same step events a graph's tasks produce, so `traceGraph()` records a workflow too.

### Steps that survive failure

Workflow steps have what graph nodes have:

- **Run and idle timeouts.** `timeout: { runMs, idleMs }`, a `StepTimeout`, caps an attempt and the
  time it may go without calling `context.heartbeat()`. `timeoutMs` is `runMs` by its older name.
  Either raises `WorkflowStepTimeoutError`, whose `kind` is `run` or `idle`.
- **Recovery after retries.** A step's `onError` receives a `StepFailure` — the name, key, attempts,
  and last error — once its retries run out. What it returns becomes the step's result, recorded
  with a `StepRecovery` under `recovered`, so a resumed run neither repeats the step nor its
  recovery; throwing fails the step. A `task_failed` event reports it.
- **Drain.** `control`, a `RunControl`, stops the run between steps: steps in flight finish and are
  recorded, no new one starts, and the run ends with `GraphDrainedError`. `continue()` runs only what
  was left, on this worker or another, and the agent server drains a workflow it serves this way.
- **Tenants.** `tenantId` is recorded on every checkpoint and read as `context.tenantId`; the
  workflow's store becomes the tenant's view of it, and another tenant's thread is not found. A
  `principal` is read as `context.principal`, and supplies the tenant when `tenantId` is not given.

```ts
const charge = await step('charge', () => payments.charge(order), {
  retry: { maxAttempts: 3 },
  timeout: { runMs: 30_000, idleMs: 5_000 },
  onError: async ({ error }) => ({ charged: false, refunded: await payments.refund(order), reason: String(error) }),
});
```

`invoke()` returns a `WorkflowResult`: the thread, the status, the `output`, the `WorkflowState` —
the input, every recorded step result, and the output — the number of recorded steps, and any
pending questions. `stream()` yields a `WorkflowEvent` for each finished step, then one for the
outcome. A failed workflow records a `failed` checkpoint and throws, exactly as a graph does.

A workflow's checkpoints are ordinary graph checkpoints, so every checkpointer stores them —
memory, Redis, Postgres, and SQLite — and `graphAssistant()` serves a workflow on the agent server
like a graph.

## Limitations

- A node before an interrupt runs twice: once to ask, once to receive the answer. Keep that work
  cheap or idempotent.
- `async` writes go one at a time to keep their order, so a graph whose steps are much faster than
  its store gains little: the run still waits for the writes before it returns.
- A drain is honoured between supersteps. A superstep that runs for an hour finishes first; an abort
  signal is the way to stop one sooner.
- `events()` streams what nodes report: a node that calls a model without `context.message()`, or a
  tool without `context.tool()`, streams no tokens or tool events. `createAgent()` reports both.
- The linter reads the graph's shape. It cannot see a side effect before an interrupt, a `Send`
  fan-out too wide for its services, or a tool that should need approval.
- The in-process checkpointer loses threads on restart; use an operation store for durability.
- Cached node results must survive the cache store's serialization.
- A workflow's code between steps runs again on every resume. Keep side effects inside steps.
- Steps are matched to their recorded results by name and call order, so call them in the same order
  on every run. Step results must survive JSON serialization.
- Catching the error `interrupt()` throws inside a workflow hides the question; let it propagate.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/agents.md

# Agents and tools


Tool-calling agents, from `nexus-ai-pro/agent`. There are two:

- `createAgent()` builds an agent on the graph runtime. It has checkpoints, approvals for sensitive
  tools, and middleware around model and tool calls.
- `AgentLoop` is the minimal loop, for when none of that is needed.

## A first tool

A tool is a name, a description, a JSON Schema for its arguments, and a function:

```ts
import { NexusAI, tool } from 'nexus-ai-pro';

const getCurrentTime = tool({
  name: 'get_current_time',
  description: 'Get the current ISO timestamp.',
  parameters: { type: 'object', properties: {} },
  execute: async () => ({ now: new Date().toISOString() }),
});

const result = await ai.agent({
  model: 'auto',
  goal: 'What time is it now?',
  tools: [getCurrentTime],
  maxIterations: 5,
});
```

## Tools that report progress

A tool that takes a while can say how far it has got. Give it `stream` instead of `execute`: an
async generator that yields progress and returns the result.

```ts
const importFiles = tool({
  name: 'import_files',
  description: 'Imports the files in a folder.',
  parameters: { type: 'object', properties: { folder: { type: 'string' } } },
  async *stream({ folder }, { signal }) {
    const files = await list(String(folder));
    for (const [index, file] of files.entries()) {
      await importOne(file, { signal });
      yield { done: index + 1, of: files.length, file };
    }
    return `imported ${files.length} files`;
  },
});
```

An agent turns each yield into a `progress` tool event while the tool runs, between its `start` and
its `result`. From there it reaches everything that reads the agent's events:
- `events()`, and the server's event stream for a graph assistant recording `tools`;
- AG-UI, A2A, and ACP, each in its own way;
- an MCP client that asked for progress;
- traces, which keep it on the tool's run;
- the studio, which shows it with the run.

An `execute` tool can report too, through `context.progress`. `tool()` gives a `stream` tool an
`execute` as well, which runs the stream and reports each yield to `context.progress`, so it works
wherever a tool is executed. `ToolBody` is the type of the choice between the two. A tool an MCP server
runs also finds `context.mcp`, which asks the client that called it for a completion from its model,
an answer from its person, or its roots, as the [MCP guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/mcp.md#the-server) describes.

## Tools that return images and files

A tool's result usually goes back to the model as JSON. A tool that made an image, or found a file,
should not send it that way: base64 text costs tokens and the model cannot see it. Return a
`toolOutput()` instead. Its parts are sent as the tool message itself.

```ts
import { tool, toolOutput } from 'nexus-ai-pro';

const renderChart = tool({
  name: 'render_chart',
  description: 'Renders a sales chart and stores it.',
  parameters: { type: 'object', properties: { year: { type: 'number' } } },
  execute: async ({ year }) => {
    const chart = await render(year);
    const stored = await assets.put(chart.bytes, { mimeType: 'image/png' });
    return toolOutput(
      `Rendered the ${year} chart.`,
      { type: 'asset', asset: stored },                     // named by reference
      { type: 'image', source: { asset: { location: { kind: 'bytes', data: chart.bytes }, mimeType: 'image/png' } } }, // shown
    );
  },
});
```

| Part | What the model receives |
| --- | --- |
| A string, or a text part | The text. |
| `{ type: 'asset', asset }` | One reference line: the asset's id, where it lives, and its type. Never its bytes. |
| `{ type: 'image', source }` | The image itself, for a model that reads images. |

Each adapter sends these the way its API expects. Anthropic and Google take images inside the tool
result. The OpenAI chat API takes only text in a tool message, so the images follow the results in one
user message. `isToolOutput()` tells a `toolOutput()` from any other result, and
`toolMessageContent()` turns a `ToolExecutionResult` into the content the model reads. Both are for a
loop of your own.

## Agents

An agent is a graph. So it gets what a graph has: checkpoints, human approval, parallel work, forks,
events, and a diagram.

```ts
import { createAgent, agentInput, tool } from 'nexus-ai-pro/agent';

const agent = createAgent({
  client: ai,                        // anything with complete(); NexusAI qualifies
  systemPrompt: 'You are a support engineer.',
  tools: [refundTool, emailTool],
  interruptOn: { send_email: true }, // this one waits for a human
  store,                             // long-term memory
  checkpointer,                      // survives a restart
});

const run = await agent.invoke(agentInput('Refund order 1182 and tell the customer'), { threadId });
run.state.answer;      // the final text
run.state.stopReason;  // 'completed' | 'max_iterations' | 'stopped' | 'client_tools'
```

**Tools that depend on the run.** `toolSource` supplies tools per run: it is called with a
`ToolSourceContext` (the thread, the tenant, the principal, and the signal) before each model call
and each tool call, and its tools are offered and run like `tools`. They pass the same permission
policy and approvals. A tool in `tools` wins over a source's tool of the same name. It is how an
editor's files serve one session, or how the MCP servers one user connected serve that user.

**Tools the client runs.** An input may carry `clientTools`: each a `ClientToolDefinition` with a
name, a description, and a schema, and no implementation. The model is offered them with the
agent's own. When it calls one, the agent reports the call as started, runs its own calls of that
step, and ends the turn with `stopReason: 'client_tools'`. The client runs the tool and sends the
transcript back with the result, and the next run carries on from there. A client tool named like
one of the agent's own is ignored. AG-UI's frontend tools work this way.

**Tool calls run in parallel.** Each call the model asks for becomes its own task, up to
`toolConcurrency` at once. So three lookups take as long as the slowest one.

**Approval is an interrupt, not a callback.** A tool listed in `interruptOn` pauses the run and saves
a checkpoint. The answer approves, refuses with a reason the model sees, or approves with corrected
arguments:

```ts
if (run.status === 'awaiting_input') {
  await agent.resumeWith(threadId, { approved: true, args: { to: 'billing@example.com' } });
}
```

Because it is a checkpoint, the approval can arrive days later, from another process.

**Its output streams as it is written.** An agent reports what its model says and every tool it
calls on the graph's event stream, with nothing to wire in its nodes. With `streamTokens: true` and a
client that has `stream()`, as `AgentModelClient` allows, the answer arrives token by token; without
it, each answer arrives as one message once the model returns. Middleware sees the assembled response
either way.

```ts
const agent = createAgent({ client: ai, tools, streamTokens: true });
const run = agent.events(agentInput('Refund order 1182'), { threadId });

const tokens = run.messages();
const calls = run.tools();
// Read both; each has its own buffer. See "Streaming events" in the graph guide.
```

**An agent is an operation of its client.** Pass `lifecycle: ai.lifecycle`, and every run is
authorized, counted, and audited as an `agent` operation, alongside the model calls it makes.
`ai.agent()` runs as one already. The [lifecycle guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/lifecycle.md) explains what that shares.

## Who a tool acts for

An agent passes every tool a `ToolContext` as the second argument of `execute`, with these fields:
- `toolCallId`, the call's id;
- `threadId` and `tenantId`;
- `signal`, the run's signal;
- `store`, which in a tenant's run is the tenant's view of it;
- `principal`, who the run is for: the caller the server authenticated, with their roles and
  scopes;
- `progress`, which reports how far the tool has got.

A tool can therefore act as that user and no one else, stop when the run is cancelled, and remember
things per tenant. Middleware reads the principal as `context.principal`, and a permission policy
as `request.principal`. A tool run outside an agent receives an empty context.

```ts
const myTickets = tool({
  name: 'my_tickets',
  description: "Lists the caller's open tickets",
  parameters: { type: 'object' },
  capabilities: ['network:helpdesk.internal'],
  execute: async (_args, { principal, signal }) =>
    helpdesk.tickets({ assignee: principal?.userId, tenant: principal?.tenantId, signal }),
});
```

## What a tool may do

Approval by tool name does not scale: a tool that runs commands is harmless running `git status`
and dangerous running `rm -rf`. So a tool declares what each call does, as capabilities, and a
permission policy decides every call from them, before it runs.

**Capabilities.** `ToolDefinition.capabilities` is a list, or a function of the call's arguments so
the policy sees the path, host, or command the call actually uses:

| Capability | Means |
| --- | --- |
| `filesystem:read`, `filesystem:read:<path>` | Reads files, or one path |
| `filesystem:write`, `filesystem:write:<path>` | Writes, creates, or deletes files |
| `network`, `network:<host or URL>` | Opens a network connection |
| `shell`, `shell:<command line>` | Runs a shell command |
| `code` | Runs code it was given |
| `mcp.read:<server>/<tool>` | An MCP tool its server says only reads |
| `mcp.write:<server>/<tool>` | An MCP tool its server says changes things but destroys nothing |
| `mcp.destructive:<server>/<tool>` | An MCP tool its server says may destroy or overwrite |
| `api.read:<api>/<operation>` | An operation of an OpenAPI document whose method only reads |
| `api.write:<api>/<operation>` | One whose method changes things |
| `api.destructive:<api>/<operation>` | One whose method deletes |
| `<name>`, `<name>:<detail>` | An effect of your own, such as `payments:refund` |

`capabilities: []` says a tool changes nothing; a tool with no list is undeclared. From
`nexus-ai-pro/agent/permissions`, `parseCapability()` splits one into a `ParsedCapability` (kind,
access, target), `capabilitiesOf()` resolves a tool's list for one call (`undefined` when
undeclared, or when its function throws), and `isSensitiveCapability()` says whether one does
anything but read.

```ts
const fetchUrl = tool({
  name: 'fetch_url',
  description: 'Fetches a web page',
  parameters: { type: 'object', properties: { url: { type: 'string' } } },
  capabilities: (args) => [`network:${args.url}`],
  execute: ({ url }) => fetch(String(url)).then((response) => response.text()),
});
```

**A permission policy.** `permissionPolicy()` builds a `PermissionPolicy` from `PermissionRules`:

| Rule | Grants |
| --- | --- |
| `filesystem` | `read` and `write` globs: `/workspace/**` is the workspace and everything in it. A write grant also allows reading. |
| `network` | `allow` hosts: `api.github.com`, `*.openai.com` for its subdomains, `*` for any. |
| `shell` | `allow` command prefixes: `git` allows every git command, `npm test` only that. A command line that chains, pipes, redirects, or substitutes is never granted by a prefix. |
| `code` | A decision for running code: `deny` by default. |
| `custom` | Decisions for your own capabilities, by kind or in full: `{ payments: 'ask', 'payments:refund': 'deny' }`. |
| `ask` | Patterns that need approval even when granted: `shell`, `filesystem:write`, `network:*.stripe.com`. |
| `undeclared`, `ungranted` | What happens to a tool with no capabilities, and to a capability no grant covers: `deny` by default. |
| `tools` | A decision per tool name, applied first. |
| `root` | What relative paths resolve against. |

Paths are normalized before any grant sees them — `normalizePath()` resolves `.` and `..` and
backslashes — so `/workspace/output/../../etc/passwd` is judged as `/etc/passwd`.

```ts
import { permissionPolicy } from 'nexus-ai-pro/agent/permissions';

const agent = createAgent({
  client: ai,
  tools,
  checkpointer,
  permissions: permissionPolicy({
    filesystem: { read: ['/workspace/**'], write: ['/workspace/output/**'] },
    network: { allow: ['api.github.com'] },
    shell: { allow: ['git', 'npm test'] },
    ask: ['shell'],
  }),
});
```

`decide()` takes a `PermissionRequest` — the tool, its arguments, its capabilities — and returns a
`PermissionVerdict`: a `PermissionDecision` of `allow`, `ask`, or `deny`, and a reason. A call is
denied when any capability is, asked about when any needs approval, and allowed only when every
one is granted. In `createAgent({ permissions })`, which takes any `PermissionPolicyLike`, a denied
call never runs and the model reads `Permission denied: <reason>`; an `ask` interrupts exactly as
`interruptOn` does, with the call's `capabilities` and the policy's reason on the `AgentToolCall`
it asks about. Arguments an approver corrects are decided again, so an edit cannot reach what the
policy denies.

**A workspace.** An agent's files live in a `Workspace`, on `nexus-ai-pro/agent/workspace`, apart
from where its code runs. A workspace reads, writes, and lists files, and may search them with
`glob()` (`WorkspaceGlobOptions`) and `grep()` (`WorkspaceGrepOptions`, returning
`WorkspaceMatch`es), remove them, and hold bytes. Its paths mean the same everywhere: relative to its
root, a leading `/` naming the root, and one that would leave it refused with a `WorkspacePathError`;
`resolveWorkspacePath()` gives a workspace of your own those same paths. A missing file rejects with
`ENOENT`, and every method rejects rather than throws. The package ships five:

| Workspace | Where the files are |
| --- | --- |
| `memoryWorkspace()` | In the process, from `MemoryWorkspaceOptions`, with files to start with. For tests and scratch work. |
| `fileWorkspace()` | In a directory on disk, from `FileWorkspaceOptions`, on Node. It runs nothing. |
| `storeWorkspace()` | In a long-term `Store` (memory, Redis, Postgres, or SQLite) under the `StoreWorkspaceOptions` namespace, so files outlive the process and every replica sees them. |
| `readOnlyWorkspace()` | Another workspace, read and searched, with every write refused as a `WorkspaceReadOnlyError`. |
| `compositeWorkspace()` | Several workspaces as one, each mounted at a path, as a `WorkspaceMount` that may be read-only, with `CompositeWorkspaceOptions`. |

A store workspace keeps each file as its contents and a small entry in a tree, so listing and
searching read entries, not contents, and nothing is indexed for semantic search. A composite sends
a path to the deepest mount it is under, lists the mounts below a directory as directories, and
searches every mount under the directory searched, through that mount's own search when one mount
holds it all:

```ts
import { compositeWorkspace, fileWorkspace, storeWorkspace } from 'nexus-ai-pro/agent/workspace';

const workspace = compositeWorkspace({
  '/': sandbox,                                                      // the code, where commands run
  memories: storeWorkspace(store, { namespace: ['agents', tenantId] }), // what outlives the run
  docs: { workspace: fileWorkspace({ root: './docs' }), readOnly: true },
});
```

`workspaceTools()` turns a workspace into `read_file`, `write_file`, and, where it can, `list_files`,
`glob_files`, `grep_files`, and `read_media`, each declaring the path a policy sees, under the
`WorkspaceToolsOptions` mount. `runWorkspaceConformance()` checks any workspace against the
contract, with no test framework, and returns a `WorkspaceConformanceReport` of
`WorkspaceConformanceCheck`s: files that round-trip and are replaced, missing files and escaping
paths refused, listings, glob and grep with their limits, bytes, and removal. In the test suite,
every workspace above passes it, the store workspace on each of the four stores, and so do the
process sandbox, the container sandbox on a real container runtime, and a composite.

**A sandbox.** Commands and code belong in a sandbox the application provides — a container, a VM,
a remote interpreter — through the `Sandbox` interface on `nexus-ai-pro/agent/sandbox`. A sandbox is
a workspace that can also run commands, `exec()` with `SandboxExecOptions` (a working directory,
variables, standard input, a timeout, a signal), returning a `SandboxExecResult`, and says what it
isolates in its `SandboxIsolation`. So a sandbox goes wherever a workspace goes, and its commands see
its files. `sandboxTools()` turns a sandbox into `run_command`, `read_file`, `write_file`, and,
where the sandbox implements them, `list_files`, `glob_files`, `grep_files`, and `read_media`,
each declaring exactly what one call does, with files under the `mount` a policy sees
(`/workspace` by default); `SandboxToolsOptions` chooses them.

**Workspace search and media.** A sandbox searches and reads bytes as a workspace does, with
`SandboxGlobOptions`, `SandboxGrepOptions`, and `SandboxMatch`es, the workspace's own types. The tools
built on them are bounded: `glob_files` lists at most 500 paths and `grep_files` at most 200 lines,
each cut to 300 characters. `read_media` reads an image, audio, or a PDF as a `ToolOutput` the model
sees rather than as text: an image part, an audio part, or a `DocumentContent` part, each as base64,
so it survives a checkpoint. A file over `maxMediaBytes`, 10 MB by default, is refused. Anthropic
receives a PDF inside the tool result; OpenAI and Google receive a tool's media right after the
results. Audio reaches Google and OpenAI's audio models; any other model receives its transcript, or
a line saying it is there.

`runSandboxConformance()` checks any sandbox against the contract, with no test framework, and
returns a `SandboxConformanceReport` of `SandboxConformanceCheck`s: output, errors, and exit codes;
standard input and variables, with none of the host's; a timeout that stops the command; files that
round-trip where commands run; and paths outside the sandbox refused, as `SandboxPathError`, a
`WorkspacePathError` whose code is `SANDBOX_PATH`.
`SandboxConformanceOptions` names the JavaScript command and the timeout it tests.

`processSandbox()` is a reference for development: commands run as ordinary processes in a
directory, with only `PATH` and the variables given, and its file methods refuse any path outside
`root` (`ProcessSandboxOptions`). A command it runs can still touch anything you can, and its
`isolation` says so. It is never a security boundary. Its search follows no links, skips `.git` and
`node_modules`, and walks at most 20,000 entries; `grep()` skips files over 1 MB and binary files,
reads at most 64 MB, and searches the first 4,000 characters of a line.

```ts
import { processSandbox, runSandboxConformance, sandboxTools } from 'nexus-ai-pro/agent/sandbox';

const sandbox = processSandbox({ root: './.agent-workspace' });   // development only
const report = await runSandboxConformance(() => mySandbox());    // prove your own
const agent = createAgent({ client: ai, tools: sandboxTools(sandbox), permissions, checkpointer });
```

**A container sandbox.** `containerSandbox()`, on `nexus-ai-pro/agent/sandbox/container`, is a sandbox
for production. It drives the `docker` or `podman` binary, as the git loader drives `git`, with no
dependency, and keeps each sandbox in one container, started on first use and removed by
`dispose()`. `ContainerSandboxOptions` sets the image, the binary, the working directory, the
network, and the limits. By default the container:
- has no network at all;
- drops every capability and cannot gain privileges;
- is held to 512 MB of memory and swap, one CPU, and 256 processes;
- mounts nothing of the host, so its files, search, and media live only inside it.

A command past its timeout is killed inside the container, with everything it started. The
container sandbox passes `runSandboxConformance()`. CI also proves that a command cannot see the
host's files or reach the host's network, and that the memory limit holds.

```ts
import { containerSandbox } from 'nexus-ai-pro/agent/sandbox/container';

const sandbox = containerSandbox({ image: 'node:22-alpine' });   // no network, 512 MB, one CPU
const agent = createDeepAgent({ client: ai, sandbox, commands: ['node', 'npm test'] });
// …
await sandbox.dispose?.();
```

The image needs a POSIX shell, `cat`, `find`, and `tar`, which every common base image has, and the
commands the agent runs. `network: true` joins the runtime's default network, and a string names one;
`isolation` then no longer claims the network. `runArgs` passes anything else to `run`, such as
`--user` with a volume that user can write.

**An interpreter.** A model that needs fifty tool calls, three at once, or a filter on a large
result can write code that makes them, run in one step, instead of a model round trip per call.
The `Interpreter` contract, on `nexus-ai-pro/agent/interpreter`, runs a snippet: the body of an
async function that calls `await tools.<name>(args)`, logs with `console`, and `return`s a value.
`run()` takes `InterpreterRunOptions`: the `InterpreterFunction`s the code may call, a timeout, and a
signal. It returns an `InterpreterResult` with the value or the error, the logs, and each
`InterpreterCall` the code made. A run never throws.

`codeTool()` puts an interpreter in front of a model as `run_code`, with `CodeToolOptions` naming the
tools the code may call. It declares `code`, so a policy must grant code before it runs at all. The
policy then decides each call the code makes, as it decides the model's own. A call it would ask a
person about is refused inside code, since code cannot pause; the model can make that call
directly.

`workerInterpreter()` is the reference, for JavaScript. Each run gets a fresh worker thread, under
`WorkerInterpreterOptions`:
- a heap of 64 MB;
- a timeout of 10 seconds;
- 100 calls;
- 64,000 characters of logs.

An endless loop or a runaway allocation ends the run, not the process. Like the reference sandbox,
it is not a security boundary: the code runs with the process's own access. Run untrusted code
behind a container or a VM. `runInterpreterConformance()` checks any JavaScript interpreter against
the contract, as an `InterpreterConformanceReport`: values, logs, and calls in order and at once;
failures caught in the code; errors reported; and a loop and an allocation stopped, with the
interpreter usable afterwards.

```ts
import { codeTool, workerInterpreter } from 'nexus-ai-pro/agent/interpreter';

const runCode = codeTool(workerInterpreter(), { tools: [lookupOrder, refundOrder], permissions });
const agent = createAgent({ client: ai, tools: [runCode, lookupOrder, refundOrder], permissions });
```

## Middleware

`AgentMiddleware` wraps the parts of a run worth controlling. It has four hooks, each optional,
applied in order:

| Hook | What it can do |
| --- | --- |
| `beforeModel` | Adjust the request: trim history, add context, change the model or the tools. |
| `wrapModelCall` | Surround the model call itself: retry it, fall back to another model, or end the run without it. |
| `afterModel` | Inspect or replace the response. |
| `wrapToolCall` | Surround each tool call: log, time, retry, refuse, or change its arguments. |

Every hook receives an `AgentMiddlewareContext`: the state at that step, the store, the thread
and tenant, the run's signal, and `interrupt()`, which suspends the run for a person exactly as
`interruptOn` does. `wrapModelCall` receives an `AgentModelCallContext`, which adds `stop()`: it
ends the run with an answer and `stopReason: 'stopped'`, with no model call. `next()` may be called
more than once in either wrapper. When a `wrapToolCall` passes `next()` a call with changed
arguments, the permission policy decides them again before the tool runs. A middleware can
therefore never reach what the policy denies.

```ts
const timing: AgentMiddleware = {
  name: 'timing',
  async wrapModelCall(request, next) {
    const started = performance.now();
    try {
      return await next(request);
    } finally {
      metrics.observe('model_ms', performance.now() - started);
    }
  },
};
```

Three small middleware come with `nexus-ai-pro/agent`:

| Middleware | What it does |
| --- | --- |
| `limitToolCalls()` | Caps how often a tool may be called in one run, so a loop cannot bill forever. Calls are counted from the transcript, so the count survives a restart and two threads never share one. |
| `redactMessages()` | Removes matching text before it reaches a provider or a log. `RedactOptions` sets the patterns, the replacement, and whether responses are redacted too. |
| `summarizeHistory()` | Replaces the older half of a long transcript with a summary, so a long thread fits the context window. `SummarizeOptions` sets how many messages stay verbatim, when summarizing starts, and the `summarize` function. |

`nexus-ai-pro/agent/middleware` has those three and ten more. Each is a separate import, and a
bundler keeps only the ones you use:

| Middleware | What it does |
| --- | --- |
| `modelRetry()` | Tries a failed model call again, with backoff. |
| `modelFallback()` | Tries other models, or other clients, when the model call fails. |
| `dynamicModel()` | Chooses the model for each call from the run so far. |
| `toolRetry()` | Runs a failed tool call again, only when repeating it is safe. |
| `toolSelector()` | Sends the model only the tools relevant to the turn. |
| `contextEditor()` | Clears old tool results from what the model is sent. |
| `piiMiddleware()` | Redacts, masks, hashes, or blocks personal data and secrets. |
| `humanApproval()` | Asks a person before a call that a rule picks. |
| `modelCallLimit()` | Caps the model calls of a run, of a thread, or both. |
| `filesystemContext()` | Puts instruction files in context, and moves large results out of it. |

```ts
import { createAgent } from 'nexus-ai-pro/agent';
import { contextEditor, humanApproval, modelRetry, toolRetry, toolSelector } from 'nexus-ai-pro/agent/middleware';

const agent = createAgent({
  client: ai,
  tools: mcpTools,                       // 150 of them
  checkpointer,
  middleware: [
    modelRetry(),
    toolSelector({ embed, maxTools: 12, always: ['search_docs'] }),
    contextEditor({ triggerTokens: 40_000 }),
    humanApproval({ when: (call) => call.name === 'refund' && Number(call.args.amount) > 100 }),
    toolRetry(),
  ],
});
```

**Retries.** `modelRetry()` retries the agent's step after the client's own retries gave up.
`ModelRetryOptions` sets the attempts, the `RetryBackoff` (an initial wait, a maximum, a factor,
and jitter), `retryOn` for which errors to retry, and `retryResponse` for retrying a response that
arrived but is unusable, such as an empty one. An error marked `retryable: false`, such as a
provider's authentication error, is never retried, and neither is a cancellation. Waits stop when
the run is cancelled. `onRetry` receives a `RetryEvent` before each new attempt.

`toolRetry()` repeats a failed tool call. By default it repeats only a call whose tool declares
capabilities and nothing beyond reading, because retrying a payment or a write that timed out
after it landed would do it twice. `ToolRetryOptions.tools` names the tools that are idempotent, or
says `'all'`, or takes a function of the call. Put `humanApproval()` before `toolRetry()`, so a
retried call is not asked about twice.

**Models.** `modelFallback()` tries each `ModelFallbackTarget` in order when the call fails: a
model name for the agent's own client, or `{ client, model }` for another one. That covers what the
router cannot: a client of a different kind, or a fallback after the client's own routes are used
up. `ModelFallbackOptions` sets which errors fall back, and `onFallback` receives a
`ModelFallbackEvent`. `dynamicModel()` takes a function of a `DynamicModelContext`, the step's
context plus the request about to go out, and returns the model to use. For example, a cheap model
for the first steps and a stronger one once the task proves hard:

```ts
dynamicModel(({ state }) => (state.iterations >= 4 ? 'gpt-5' : 'gpt-5-mini'))
```

**Tool selection.** An agent with 150 tools otherwise sends 150 schemas on every call. Those tokens
are paid every time, the call is slower, and a model choosing among that many picks worse.
`toolSelector()` chooses a dozen, by a rule, by embedding similarity, or by both
(`ToolSelectorOptions`):

- A rule, `select`, receives a `ToolSelectionContext`: the step's context, the request, the
  tools offered, and the query.
- `embed` is any embedding function. Each tool is embedded once, from its name and description;
  the query once per turn.

The tools named in `always`, a tool that `toolChoice` forces, and the tools the model already
called this turn are always kept. The agent can still run every tool; only what the model is
offered changes. `onSelect` receives a `ToolSelection`: the query, the tools sent, how many were
offered, and each tool's similarity score.

In the benchmark in the test suite, `toolSelector()` sends 12 tools instead of 150 over 40 tasks:
- every task still succeeds;
- input tokens per task fall from about 64,000 to about 5,200;
- latency per task falls by about 70%. Latency there is the model's time, computed from
  each request's size as a provider's time to first token grows with it, plus the agent's own time,
  measured, with the selector's work included.

`compareExperiments()` reports the comparison.

**Tool search.** A tool marked `deferred` stays out of the model's way until it finds it. An agent
with a deferred tool offers `search_tools` (`TOOL_SEARCH_NAME`), built by `toolSearchTool()` with
`ToolSearchOptions`. It ranks the deferred tools by BM25 over their names, descriptions, and
arguments, and answers with the five best by default, as tool references. `searchTools()` runs the
same ranking alone and returns `ToolSearchHit`s.

How a found tool reaches the model depends on the provider:
- On Claude 4.5 and later, every tool is sent on every call, a deferred one with `defer_loading`.
  It stays out of the model's context until a reference in the conversation names it. The tool list
  at the front of the prompt never changes, so the cached prefix holds.
- On any other model, and through any other provider, a deferred tool is sent only once a search has
  found it.

Either way the model calls a found tool on its next turn, and the agent runs it like any other. In
the test suite, an agent with 500 deferred tools finishes every one of 40 tasks on Anthropic, OpenAI,
and Google. Its model calls a tool only once that provider's request has loaded it. `toolSelector()`
and tool search combine: put `search_tools` in `always`.

**Context.** `contextEditor()` replaces old tool results with a short placeholder once the request
passes `triggerTokens`, keeping the most recent few whole. Results are cleared in batches, so the
edited prefix stays the same between edits and a provider's prompt cache keeps working. Only the
request changes: the transcript in state keeps every result. `ContextEditorOptions` also sets the
batch, the tools never cleared, whether arguments are cleared too, and the placeholder, which a
function can build from a `ClearedToolResult`. `onEdit` receives a `ContextEdit` with the tokens
before and after. `estimateRequestTokens()` is the default estimate, about four characters a
token over messages, tool calls, and tool schemas.

`filesystemContext()` treats the filesystem as context, from a `FilesystemContextSource`: a
`Sandbox`, or anything with its file methods. Through `FilesystemContextOptions`, it does two
things:

- Instruction files such as `AGENTS.md` reach the model on every call, right after the system
  prompt, with an optional listing of a directory. They are read once per run.
- With `offload` (`ContextOffloadOptions`), a tool result larger than the limit is written to a file
  instead. The model reads a preview and a path it can open with `read_file`.

The transcript stays small however much a tool returns, and nothing is lost.

```ts
filesystemContext({ source: sandbox, include: ['AGENTS.md'], listing: {}, offload: { overChars: 20_000 } })
```

**Personal data.** `piiMiddleware()` finds each `PiiKind`:
- emails and phone numbers;
- card numbers, checked with Luhn;
- US social security numbers;
- IBANs, checked with their checksum;
- IP addresses;
- common API keys and private keys;
- any patterns of your own.

The checksums keep false positives low: an order number is not taken for a card. Each match gets a
`PiiStrategy`, for every kind or per kind:
- `redact` replaces it with its kind;
- `mask` keeps its last four characters;
- `hash` gives a stable token, so the model can still tell two values apart;
- `block` refuses the message with a `PiiBlockedError`.

`PiiMiddlewareOptions.apply` chooses where it looks:
- the model's input, which is everything but the system prompt;
- its output;
- tool results, which are cleaned before they enter the transcript, so they are never checkpointed.

`onDetect` receives a `PiiFinding`, which gives the kind and its `PiiWhere` but never the value.

```ts
piiMiddleware({ strategy: { email: 'hash', 'credit-card': 'block' }, apply: { input: true, toolResults: true } })
```

**Approvals and limits.** `humanApproval()` asks before the calls a rule picks: a refund over a
limit, or a write outside a draft folder. `interruptOn` instead asks before every call to a named
tool. By default it asks about every call that declares nothing, or declares more than reading.
`HumanApprovalOptions.approve` asks out of band, by posting to a chat and waiting for the answer,
instead of suspending the run. An approver who corrects the arguments has them decided again by the
permission policy.

`modelCallLimit()` caps model calls per run, per thread across runs, or both
(`ModelCallLimitOptions`):
- The run count is read from the transcript.
- The thread count is kept in the agent's store when it has one, so every process shares it.

At a limit, the run ends with an answer and `stopReason: 'stopped'`. With `onLimit: 'error'`, it
fails with a `ModelCallLimitError` instead. `maxIterations` still stops a single run that keeps
calling tools.

## The deep-agent preset

For long, multi-step work, `createDeepAgent()` on `nexus-ai-pro/deep-agent` is `createAgent()`
with a kit already assembled:
- a plan it keeps current;
- a sandboxed workspace;
- helpers to delegate to;
- skills loaded on demand;
- context offloading and editing;
- a permission policy.

The [deep-agent preset guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/deep-agents.md) covers it.

## An agent as a tool

`agentAsTool()` turns an agent into a `ToolDefinition`, so one agent can call another. A supervisor
hands a question to a specialist and gets the answer back as a tool result. Neither knows the other's
tools or state.

```ts
const researcher = createAgent({ client: ai, tools: [searchTool], name: 'researcher' });
const supervisor = createAgent({
  client: ai,
  tools: [agentAsTool({ agent: researcher, name: 'research', description: 'Researches a question' })],
});
```

## Running tools yourself

`ToolExecutor` runs tool calls by name, outside an agent: in a custom loop, or in a server route that
runs what a model asked for. A failure comes back as a `ToolExecutionResult` with `ok: false`, not a
thrown error, so one bad call does not end a run. An unknown tool name is reported the same way.

## Tools that reach the web

`nexus-ai-pro/connectors` ships two tools, built on `tool()`:

- `createFetchUrlTool()` reads text from a URL. It refuses private and link-local addresses, follows
  a bounded number of redirects, truncates a long response, and times out. `WebConnectorOptions` sets
  those limits and the SSRF policy, including a resolver that answers each host name with its
  `WebResolvedAddress` values.
- `createSearchTool()` wraps a search function you supply, so the search provider stays yours.

## Tools from an OpenAPI document

`openApiTools()`, from `nexus-ai-pro/agent/openapi`, reads a service's own API description into tools: an
`OpenApiDocument`, OpenAPI 3.0 or 3.1, or Swagger 2.0, as parsed JSON (parse a YAML one first).
`loadOpenApiTools()` fetches one as JSON and reads it, resolving a relative server against where it came
from and reading in the other documents its references name.

```ts
import { loadOpenApiTools } from 'nexus-ai-pro/agent/openapi';

const shop = await loadOpenApiTools('https://shop.example.com/openapi.json', {
  // The document's security schemes, by name: a key per tenant, and an OAuth 2.0 client.
  credentials: {
    shopKey: ({ context }) => keys.for(context?.tenantId),
    shopOAuth: { clientId: 'agent', clientSecret: secrets.shop },
  },
});
const agent = createAgent({
  client: ai,
  tools: shop.tools,
  permissions: permissionPolicy({ custom: { 'api.read': 'allow', 'api.write': 'ask' }, network: { allow: ['shop.example.com'] } }),
});
```

Each operation becomes one tool, described as an `OpenApiOperation`:
- its name is its `operationId`, or its method and path, such as `get_orders_orderId`; two the same
  get a number, and `prefix` keeps two APIs' apart;
- its description is the operation's summary and description;
- its parameters are its path, query, and header parameters, and the fields of a JSON, form, or
  multipart body, flattened, unless one is named as a parameter, when the body is one `body`
  parameter. A property the API marks `readOnly` is left out. Cookies, and the `Accept`,
  `Content-Type`, and `Authorization` headers, are the client's, as OpenAPI says;
- a body of text or XML is a string; a file field of a multipart form, and a body of bytes such as an
  image, is its base64, alone or as `{ base64, filename, mimeType }`, and is sent as the bytes. A
  multipart field the API takes as base64 text is text;
- it declares `api.read`, `api.write`, or `api.destructive` with the API's name and its own, as its
  method says, and `network` for the API's host when it runs, so a permission policy decides it as it
  decides any tool.

References within the document are followed, a schema that refers back to itself is described rather
than repeated, and OpenAPI 3.0's `nullable` and `example` become JSON Schema's. An `OpenApiTools` holds
the tools, their operations, `results`, the schema of what each returns, from its first success
response, which `emulatedTools()` shows the emulator, and `webhooks`.

`OpenApiToolsOptions` sets `baseUrl`, where the API lives, which defaults to the document's first
server; without one, the tools describe the API and run nowhere, as an emulated tool wants them.
`headers` are sent with every call, or chosen per call from an `OpenApiHeadersRequest`, the operation
and the call's context with its tenant, and win over a header the model sets. `include` chooses
operations, and `timeoutMs` (30 seconds), `maxResponseBytes` (1 MB), and `fetch` bound each call.

### Credentials

Each operation's security, or the document's, is its `security`: alternatives, each the schemes it
needs at once. `credentials` gives one for each scheme, by its name in the document, as an
`OpenApiCredential`, or a function choosing one per call from the same `OpenApiHeadersRequest`. A call
sends the first alternative it has every credential for:
- an `apiKey` in the header, query parameter, or cookie the scheme names, from a token, as text or
  `{ token }`;
- `http` `basic` from `{ username, password }`, and `bearer`, or another HTTP scheme by its own name,
  from a token;
- `oauth2` and `openIdConnect` from a token the application holds, or, for the client-credentials
  flow, `{ clientId, clientSecret }`: a token is fetched from the scheme's `tokenUrl` for the scopes
  the operation asks, the client authenticating with HTTP Basic, and kept until a minute before it
  expires.

An operation none of whose alternatives can be met throws `OPENAPI_INVALID`, naming them, before
anything is sent; one that asks for `mutualTLS` throws `OPENAPI_UNSUPPORTED`. An operation that asks for
no security is sent with none. Without `credentials`, nothing is added, and `headers` carry what the
application sends. A credential goes where the document says, its token URL included, so give
credentials only to tools from a document you trust.

### Swagger 2.0 and other documents

A Swagger 2.0 document is read as the OpenAPI 3.0 document it describes, which `swaggerToOpenApi()`
returns: its host, base path, and first scheme as the server; its definitions, shared parameters and
responses, and security definitions as components; a body parameter, or form data, as a request body of
the type it consumes, a `file` field making it multipart; and each response's schema as content of the
type it produces. Swagger 1.2 is refused.

A reference to another document is read in by `bundleOpenApiDocument()`, which `loadOpenApiTools()`
calls when the document has one: each document is fetched once, as JSON, relative to the one naming it;
its own references are followed; a reference back into the first document stays one; and a cycle
across documents is described rather than repeated. Its `OpenApiBundleOptions` take the
`documentUrl` references are read against; `references`, which documents may be fetched, `same-origin`
by default, `any` web URL, or a function; and `maxDocuments` (50) and `maxBytes` (5 MB), which
`loadOpenApiTools()` takes too, the first document's size included. `openApiTools()` fetches nothing
itself: a document with a
reference outside it throws `OPENAPI_UNSUPPORTED`, naming the two ways to read it.

### Webhooks and callbacks

`webhooks` lists what the API sends the application, each an `OpenApiWebhook`: the document's webhooks,
and each operation's callbacks, named for the operation and the callback with the runtime expression of
where it is sent, each with its method and the schema of its JSON body. They are not tools: they
describe what the application's own endpoint receives.

### Calls

A call encodes each path parameter, so none can reach another path, and refuses one that is only dots.
It is never sent when its call was stopped first, never follows a redirect, reads no more of an answer
than the limit, and returns the JSON, the text, or `null` for nothing. A refusal, or a call that
cannot be made, throws an `OpenApiError` in the API's own words, such as `createRefund answered 409:
already refunded`, with its `status`, which the agent sees as the tool's failure. A document that
cannot be read throws one too: `OPENAPI_UNSUPPORTED` for Swagger 1.2, and `OPENAPI_INVALID` for a
malformed one.

## The pieces, by name

| Name | What it is |
| --- | --- |
| `CreateAgentOptions` | What `createAgent()` takes. |
| `AgentGraph` | What it returns: a compiled graph over `AgentChannels`. |
| `AgentState` | The graph's state: the transcript, the iterations used, the final answer, and an `AgentStopReason`: `completed`, `max_iterations`, `stopped` when a middleware ended the run, or `client_tools` when it waits for a client. |
| `agentInput()` | Wraps a question into that state. |
| `AgentApprovalPolicy` | Decides which tools pause for a person. |
| `AgentApproval` | A person's answer: allow, refuse with a reason the model sees, or allow with corrected arguments. |
| `AgentToolCall`, `AgentToolResult` | What the model asked for, and what came back. |
| `AgentModelClient` | The one-method client contract the agent needs. |

The simpler loop is `AgentLoop`. It is configured with `AgentConfig` and driven by an
`AgentLoopModelClient`. It returns an `AgentResult`: the answer, the model calls made, and every
`AgentStep` along the way. It has no checkpoints and no approvals, which is the point of it.

## Limitations

- Progress is whatever a tool reports. Nothing estimates it for a tool that reports nothing, and
  the runtime does not throttle reports, so a tool that loops quickly should report every so often,
  not on every pass.
- Tool search uses the agent's own ranking, not a provider's server-side search, so it works the same
  on every provider; Anthropic loads what it finds through tool references.
- OpenAPI credentials are sent for flows that need no person: an OAuth 2.0 authorization-code or
  device flow is the application's, which passes the token it gives. `mutualTLS` is for a `fetch` that
  holds the certificate.
- Other documents a reference names are read as JSON; a `fetch` that answers a YAML one as JSON reads it.
- Webhooks and callbacks are listed, not received: the application's own endpoint receives them.
- A query parameter is sent as OpenAPI's default form, an array as the parameter repeated; an object in
  a query or a header is sent as JSON.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/deep-agents.md

# The deep-agent preset


Agents for long, multi-step work. A deep agent has:
- a plan it keeps current;
- a workspace it reads, writes, and runs commands in;
- helpers it hands self-contained work to;
- skills it loads only when a task needs them;
- a context that stays small however long the work runs.

`createDeepAgent()` is a preset, not a third engine. It composes `createAgent()` with tools and
middleware from the agent kit, and returns the same compiled graph. Checkpoints, approvals, events,
the server, and the linter all work as they do for any agent.

```ts
import { agentInput } from 'nexus-ai-pro/agent';
import { processSandbox } from 'nexus-ai-pro/agent/sandbox';
import { createDeepAgent } from 'nexus-ai-pro/deep-agent';

const agent = createDeepAgent({
  client: ai,
  model: 'claude-sonnet-5-5',
  instructions: 'You maintain this repository.',
  sandbox: processSandbox({ root: './checkout' }),   // development only; containerSandbox() in production
  commands: ['node', 'npm test', 'git diff'],
  subagents: [
    { name: 'reviewer', description: 'Reviews a change for bugs', instructions: 'You review changes for bugs.', tools: ['read_file'] },
  ],
  skills: [{ name: 'changelog', description: 'How to record a change', file: 'skills/changelog.md' }],
  checkpointer,
});

const run = await agent.invoke(agentInput('The test fails. Fix it.'), { threadId });
```

## What it is made of

| Part | What the agent gets | Built from |
| --- | --- | --- |
| Planning | `write_todos`: the plan, sent whole each time, each step `pending`, `in_progress`, or `completed`. It lives in the transcript, so it is checkpointed with the run and never cleared by context editing. | A tool |
| A workspace | `read_file`, `write_file`, `list_files`, and `edit_file` in the `workspace`, or the sandbox, mounted at `/workspace`; `glob_files`, `grep_files`, and `read_media` where it can search and read bytes. | `workspaceTools()` |
| Commands | `run_command` in the `sandbox`, when `commands` names any. | `sandboxTools()` |
| Helpers | `delegate`: hands a task to a `DeepSubagent`, which runs to an answer in a fresh context, with the tools it was given, for the same principal. | `createAgent()` per helper |
| Skills | `load_skill`: only each `DeepSkill`'s name and description sit in the prompt until the agent loads one, from its `instructions` or a `file` in the sandbox. | A tool |
| Instruction files | `AGENTS.md`, or the files `include` names, on every call, after the system prompt. | `filesystemContext()` |
| Offloading | A tool result over 20,000 characters is written to `.context/` and left as a preview and a path. | `filesystemContext()` |
| Context editing | Old tool results are cleared from what the model is sent past 60,000 estimated tokens. | `contextEditor()` |
| Tool selection | With `toolSelection`, only the relevant tools are sent; the deep agent's own are always kept. | `toolSelector()` |
| Code | With `interpreter`, `run_code`: the model's code calls the agent's own tools in one step, each call decided by the policy. A tool that needs approval is not callable from code. | `codeTool()` |
| Permissions | A policy that grants the workspace, the `commands`, the agent's own tools, and code when it has an interpreter, and denies everything else. | `permissionPolicy()` |

`DeepAgentOptions` takes everything `createAgent()` takes, except the system prompt, which the
preset composes from `instructions`, plus the parts above. Each part can be turned off or replaced:
- `planning: false`;
- `offload: false`;
- `contextEditing: false`;
- `permissions: false`, or a policy of your own;
- more `middleware`, which runs after the preset's own.

A tool of your own in `tools` needs a grant in your policy, because the default policy denies what
it does not know.

## Where its files live

A deep agent's files and its commands live apart. `workspace` is where its files are: what it reads,
writes, searches, and edits, its skills' files, its `AGENTS.md`, and the results it offloads. Any
`Workspace` will do: in memory for a run that keeps nothing, in a `Store` for files every replica
sees and that outlive the process, on disk, or a composite of several. `sandbox` is where it runs
commands. Without a `workspace`, the sandbox is both, as before.

With both, its commands see the sandbox's own files. To let a command see the agent's code, mount
the sandbox in the workspace:

```ts
const agent = createDeepAgent({
  client: ai,
  workspace: compositeWorkspace({
    '/': sandbox,
    memories: storeWorkspace(store, { namespace: ['agents', userId] }),
    policies: { workspace: fileWorkspace({ root: './policies' }), readOnly: true },
  }),
  sandbox,
  commands: ['node', 'npm test'],
});
```

The agent edits the code in the sandbox and runs its tests there; what it learns goes to the store,
where the next run in any process reads it; and the policies are there to read, never to change. A
write to them reaches the model as a refusal.

## Helpers and skills

A helper is for work that would otherwise fill the main context: reviewing a change, researching a
question, summarizing a long file. It sees only the task it is given, works with the tools named in
its `tools` (all of the agent's but `delegate`, by default), and returns its answer as the tool
result. It cannot delegate in turn. Because it cannot pause, it never gets a tool named in
`interruptOn`: work that needs a person's approval stays with the main agent, which can pause.

**In the background.** With `background`, `delegate` returns a task id at once and the agent goes
on working; `collect_helper` returns the helper's answer when the agent needs it, waiting up to
`wait_seconds`, and lists the tasks when called without one. At most `maxBackgroundHelpers`, four
by default, work at once in one thread. A helper the agent waits for keeps no checkpoints. One in
the background keeps them in the deep agent's checkpointer, on a thread named after the call that
started it, and `delegate` returns only once its first checkpoint is written. So a crash leaves the
helper's finished steps behind it:
- collecting the task in a new process carries the helper on from its last checkpoint;
- a `delegate` call the agent repeats after the crash finds the same thread, and carries that on
  instead of starting the helper again.

Without a checkpointer, a background helper lives only as long as its process.

**Helpers it creates.** With `createHelpers`, the agent can create a helper for work no declared
helper fits: it gives `delegate` a new name and the helper's instructions. `DeepHelperLimits` bound
what it creates: the tools a created helper may use, its model, its model calls (10 by default), and
how many the agent may create in one thread (5 by default). With a checkpointer, each helper the
agent creates is recorded, with its instructions, on a thread beside the agent's own,
`<thread>/helpers-created`, before the call that created it returns. An agent resumed in another
process counts those helpers against the limit and delegates to them again by name.

**On another server.** A helper with `remote` runs on another agent server instead: give it a
`DeepRemoteHelper`, such as `createRemoteGraph()` returns for one of that server's assistants. It is
sent its `instructions` and the task, and its run's output is its answer: text, an agent's `answer`,
or its last message, from a `DeepRemoteRunResult`. Its `tools`, `model`, and model calls are the
remote assistant's own.

The run is an operation on that server, leased to the worker that runs it, so when that worker stops,
another takes the run over. The agent starts the run under a key made from the call that started
it, and the server answers a second start under the same key with the run it already has. In the
background, the task is marked on its thread, with what it was asked, before `delegate` returns, so:
- collecting it in another process, after a crash, waits for that same run;
- a run that never reached the server, because it was down when the agent delegated, is started by
  the process that collects it, once.

```ts
import { createRemoteGraph } from 'nexus-ai-pro/server/remote';

const agent = createDeepAgent({
  client,
  checkpointer,
  subagents: [
    {
      name: 'researcher',
      description: 'Researches a question in depth',
      instructions: 'Research the question; answer with sources.',
      remote: createRemoteGraph({ url: 'https://agents.internal/api', assistant: 'researcher' }),
    },
  ],
});
```

**Its events.** Streamed with `subgraphs: true`, a helper's model output, tool calls, and steps reach
the agent's own event stream under the helper's namespace, such as `tools/researcher@call-7`, in the
background as in the foreground. A tool of your own that starts a run of its own can do the same
through `ToolContext.forward`.

A skill is for instructions that are long and rarely needed, such as a release procedure or a house
style. The model sees one line per skill until it calls `load_skill`, so ten skills cost ten lines of
context, not ten documents.

## Proof

In the test suite, a deep agent works on a fixture repository inside the reference sandbox. In
order, it:
1. plans;
2. runs the failing test;
3. reads the code and edits the bug;
4. runs the test again;
5. delegates a review to a helper;
6. loads a skill;
7. records the change.

Afterwards the test passes when run outside the agent. Along the way:
- the 30,000-character test log is offloaded to a file;
- the plan finishes at 5/5;
- the default policy refuses an unlisted command, a chained command, a write outside the workspace,
  and an ungranted tool of the application's own.

Workspaces have proofs of their own:
- The same deep agent does the same work over every workspace: in memory, on disk, in each of the
  four stores, a process sandbox, and a composite of a sandbox, a store, and read-only docs. It reads
  the `AGENTS.md` it is given, finds files by glob and a TODO by grep, edits the code, loads a skill
  from a file, looks at an image, has a large result offloaded, and writes its notes; every workspace
  ends holding the same files.
- An agent with a composite edits code in the sandbox mounted at its root, and its test command sees
  the edit; its notes reach the store; and the read-only docs refuse its write, which the model reads.

Helpers have proofs of their own:
- A background researcher answers only after the agent has written a note, and the agent collects
  the answer; the researcher's tool calls reach the agent's stream under its namespace.
- The agent creates a helper on instructions it wrote, which gets only the tools the limits allow; a
  second one past `max` is refused, also after the agent is killed and resumed in another process,
  where it still delegates to the helper it created by name.
- A crash while the agent waits for its helper, and a crash before the agent has saved the call that
  started it, both end with one lookup, one note, and one start of the helper.
- A remote helper runs on two replicas of a real agent server. The agent's process dies while the
  helper works, and so does the server worker running it. The second replica takes the run over when
  its lease lapses, and the agent, resumed in a new process, collects that same run's answer: one run,
  and the agent's note not written again. A helper whose start never reached the server is started by
  the process that collects it, once.

## Limitations

- The reference sandbox, `processSandbox()`, is for development. It is never a security boundary.
  Use `containerSandbox()`, from `nexus-ai-pro/agent/sandbox/container`, or a VM of your own behind
  the `Sandbox` interface, in production.
- A helper without `remote` runs in the agent's process. A remote helper's events stay on its
  server; they do not reach the agent's stream.
- Workspace search is bounded: `glob_files` lists at most 500 paths and `grep_files` 200 lines.
  Every workspace skips files over 1 MB and binary files when it searches, and those on disk skip
  `.git` and `node_modules` too. A wider search is a command the sandbox runs.
- A store workspace writes a file's contents before its entry, so a process that stops between the
  two leaves a file that reads but is not listed until it is written again. Its search reads every
  entry under the directory searched.
- A composite searches a mount only if the mount has `glob`.
- `read_media` sends a file of at most 10 MB, as the model's provider accepts it: a PDF reaches
  Anthropic, OpenAI, and Google; audio reaches Google and OpenAI's audio models, and any other model
  is told the audio is there.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/memory.md

# Long-term memory


A checkpoint remembers one conversation. A store remembers across them. `nexus-ai-pro/store` keeps
namespaced items that outlive a run or a thread, with optional semantic search through an embedding
function you inject.

```ts
import { MemoryStore } from 'nexus-ai-pro/store';

const store = new MemoryStore({ index: { embed, fields: ['text'] } }); // embed is yours
await store.put(['tenant-7', 'users', 'alice'], 'tone', { text: 'prefers brief answers' });

// In any node or tool, in any thread, later:
const memories = await context.store.search(['tenant-7', 'users', 'alice'], { query: 'how do they like answers?' });
```

A namespace is a tuple, so `['tenant-7', 'users', 'alice']` is both a place to put something and a
prefix to search. Items can expire with `ttlMs`, and can be filtered by field.

| Store | Where | Guide |
| --- | --- | --- |
| `MemoryStore` | This process. | Here. |
| `RedisStore` | Redis, shared between processes. From `nexus-ai-pro/store/redis`. | Here. |
| `PostgresStore` | Postgres, ranking in the database with pgvector. | [Postgres](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/postgres.md) |
| `SqliteStore` | One SQLite file. | [SQLite](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/sqlite.md) |

The store never imports the embeddings runtime. Without an index, a query matches text instead.

## The contract

`Store` is five methods, and every adapter has all of them. Each may answer synchronously or with a
promise, so a read from the in-memory store costs no microtask.

| Method | What it does |
| --- | --- |
| `put()` | Stores a value under a namespace and key. It replaces any item there, keeping its creation time. |
| `get()` | Reads one item, or returns `undefined` when it does not exist or has expired. |
| `delete()` | Removes one item. |
| `search()` | Returns the items under a namespace prefix, newest first, or ranked by similarity to a query. |
| `listNamespaces()` | Lists the namespaces under a prefix, for browsing what an agent remembered. |

`putIfVersion()` is optional. It stores an item only when the stored one's `version` field is still
the version given, or, given `null`, only when none is stored, and it resolves whether it wrote. The
check and the write are one atomic step in every included store, so two processes deciding on one
version cannot both write. The memory, Postgres, and SQLite stores have it, and so does the Redis
store when its client has `eval`. Deployments use it to change atomically across replicas.

A `StoreNamespace` is a tuple of strings. A `StoreItem` carries:

- its namespace and key, and the value;
- when it was created and last written, and when it expires;
- on semantic search results, its `score`.

`StorePutOptions` has two fields:

- `ttlMs` sets a lifetime.
- `index` names the fields of the value to embed, as dot paths. It defaults to the store's configured
  fields; `false` stores the item without indexing it.

`StoreSearchOptions` has four:

- `query`, a natural-language question;
- `filter`, exact matches on dot-path fields;
- `limit`, 20 by default, and `offset`, for paging.

## Semantic search

`StoreIndexOptions` turns search on:

- `embed` turns a batch of texts into vectors. Pass the embeddings family's function to inherit its
  routing, batching, and caching.
- `fields` names what to embed. By default, every string field in the value.

Items are embedded when they are written. A query is embedded when it is searched, then ranked with
`cosine()`: cosine similarity, from -1 to 1. It is exported for adapters of your own.

Without an index, a query is a case-insensitive text match over the stored values. That is useful,
but not semantic.

## The adapters

`MemoryStoreOptions` has three fields:

| Field | Default | What it does |
| --- | --- | --- |
| `index` | none | Semantic search, as above. |
| `maxItems` | 10,000 | Past this, the least recently written item is dropped, so a store that only grows cannot leak. |
| `now` | the clock | Replaces the clock, for tests. |

`size()` counts the items it holds.

`RedisStore` takes a `RedisStoreLikeClient` and `RedisStoreOptions`:

```ts
import Redis from 'ioredis';
import { RedisStore } from 'nexus-ai-pro/store/redis';

const store = new RedisStore(new Redis(process.env.REDIS_URL!), { prefix: 'support:memory', index: { embed } });
```

The client needs six commands: `get`, `set` with `PX` expiry, `del`, `sadd`, `srem`, and `smembers`.
An `ioredis` client has them as it is; wrap another client in those six. `RedisStoreOptions` sets a key
`prefix` (`nexus:store` by default, so one Redis can hold several stores), an `index`, and a clock.

Expiry is Redis's own TTL. Each namespace keeps a set of its keys, so a prefix search never scans the
keyspace.

## Using a store

- In a graph: `compile({ store })`, and nodes reach it as `context.store`.
- In an agent: `createAgent({ store })`, and its tools and middleware receive it as `store`.

## Limitations

- The memory, Redis, and SQLite stores rank semantic search in the process. Every item under the
  prefix is read and scored, so keep namespaces focused. `PostgresStore` with pgvector ranks in the
  database.
- The Redis, Postgres, and SQLite stores keep values as JSON, so a value must survive
  `JSON.stringify()`. The memory store holds the object itself, so changing it after `put()` changes
  what is stored.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/mcp.md

# MCP


The Model Context Protocol, in both directions, from `nexus-ai-pro/mcp`. `McpClient` borrows another
server's tools as ordinary tool definitions. `McpServer` lends your tools to other assistants. Both run
over stdio, HTTP, or any transport you supply, and `McpRegistry` manages many servers from one file.

## Overview

The Model Context Protocol, both directions, implemented directly rather than through an SDK:

```ts
import { McpClient, createStdioTransport } from 'nexus-ai-pro/mcp';

const client = new McpClient(createStdioTransport({ command: 'npx', args: ['-y', 'some-mcp-server'] }));
const agent = createAgent({ client: ai, tools: await client.toNexusTools({ prefix: 'files' }) });
```

That is how this package reaches a broad tool ecosystem without maintaining a catalogue of
integrations: anything exposed over MCP becomes tools an agent can call. The server direction lends
your tools out the same way, so a tool written once with `tool()` serves an agent here and an editor
elsewhere:

```ts
import { McpServer, createStdioServerTransport } from 'nexus-ai-pro/mcp';

await new McpServer({ name: 'my-app', tools: [refundTool] }).connect(createStdioServerTransport());
```

Stdio, streamable HTTP, and SSE transports are included, both ways, and the transport is an
interface, so a test can wire a client to a server in memory.

## The client

`McpClient` takes a transport and `McpClientOptions`: the `clientInfo` name and version it reports,
`timeoutMs`, how long a request waits (30 seconds), and `reconnect`. `connect()` performs the
handshake; the first request that needs it calls it for you.

| Method | Returns |
| --- | --- |
| `listTools()` | Each tool as an `McpToolDescriptor`: name, description, and JSON Schema. |
| `callTool()` | An `McpToolResult`: a list of `McpContent` parts (text, or base64 data with a MIME type), and `isError` when the tool reported a failure. |
| `listResources()` | `McpResourceDescriptor` values: URI, name, description, and MIME type. |
| `readResource()` | One resource, as content parts. |
| `listPrompts()`, `getPrompt()` | The server's prompts, and one rendered into messages. |
| `toNexusTools()` | The tools as `ToolDefinition` values an agent or `ToolExecutor` can run. |
| `onToolsChanged()`, `onPromptsChanged()`, `onResourcesChanged()` | Hears each time the server says that list changed; each returns a function that stops hearing. |
| `subscribeResource()`, `unsubscribeResource()`, `onResourceUpdated()` | Asks to hear when a resource changes, stops, and hears its URI each time it does. |
| `rootsChanged()` | Tells the server this client's roots changed. |
| `close()` | Closes the transport, rejecting requests still waiting. |

### What a server asks of the client

A server may ask its client for things while it works: a completion from the client's model, an
answer from the person using it, or the roots it may work within. `McpClientOptions` answers each it
is given, and declares it to the server at the handshake:
- `sampling` answers `sampling/createMessage`, an `McpSamplingRequest` of `McpSamplingMessage` values,
  each `McpSamplingContent` text, an image, or audio, with `McpModelPreferences`, with an
  `McpSamplingResult`. `samplingFrom()` is one, from any `McpSamplingClient`, such as `NexusAI`, so
  each request is routed, budgeted, traced, and priced as every completion is. `McpSamplingOptions`
  set the `model`, the `models` a server's hints pick from, a `maxTokens` cap, whether to keep its
  `systemPrompt`, and `approve`, which sees each request first, as MCP expects a person to;
- `elicitation` answers `elicitation/create`, an `McpElicitationRequest` with the form of its answer,
  with an `McpElicitationResult`: accepted with its fields, declined, or cancelled;
- `roots` answers `roots/list` with each `McpRoot`.

```ts
const client = new McpClient(createHttpTransport({ url }), {
  sampling: samplingFrom(ai, { model: 'gpt-5-mini', maxTokens: 1_000, approve: (request) => confirm(request) }),
  roots: () => [{ uri: 'file:///work/acme', name: 'acme' }],
});
```

A request it was not given is refused with method-not-found, as is anything else a server asks but
`ping`, which it answers. A request the server cancels stops its handler, through the `signal` it is
given, and is not answered.

Two kinds of failure are kept apart. A tool that fails answers with `isError`; it does not throw. A
protocol failure throws `McpError`.

With `toNexusTools()`, a `prefix` keeps two servers' `search` tools apart. Each tool returns its text,
and a failure throws, so the agent sees a failed tool call.

### What a server says its tools do

A server describes each tool with `McpToolAnnotations`: whether it only reads (`readOnlyHint`), may
destroy what it changes (`destructiveHint`), repeats safely (`idempotentHint`), and reaches beyond the
server (`openWorldHint`). `McpToolsOptions.capabilities: true` turns them into the capabilities each
tool declares, so a permission policy decides a server's tools as it decides the agent's own:

```ts
const tools = await client.toNexusTools({ prefix: 'github', capabilities: true });
const agent = createAgent({
  client: ai,
  tools,
  permissions: permissionPolicy({ custom: { 'mcp.read': 'allow', 'mcp.write': 'ask' }, network: { allow: ['*'] } }),
});
```

`mcpToolCapabilities()` gives each tool, under the server's name (the `server` option, then the
prefix, then the name the server reports):
- `mcp.read:<server>/<tool>` when it only reads;
- `mcp.write:<server>/<tool>` when it changes things but destroys nothing;
- `mcp.destructive:<server>/<tool>` when it may destroy or overwrite;
- and `network` beside it, unless the server says it stays within itself.

A hint left out takes the specification's default: not read-only, destructive, and reaching out. A
tool with no annotations declares nothing, and a policy decides it as undeclared. A function as
`capabilities` decides per tool instead. Annotations are the server's word, so give `true` for
servers you trust, and decide the others yourself.

`callTool()` takes a `progress` listener. The call then names a progress token, and each
`notifications/progress` the server sends for it arrives there, with its progress, total, and
message. A tool from `toNexusTools()` passes them to the agent's `context.progress`, so a remote
tool's progress reaches the agent's events like a local one's.

### When the connection drops, or the tools change

With `reconnect`, a client whose transport closed opens it again at the next request and greets the
server again, since the server's session went with the connection:
- an SSE stream that ends, as a proxy's timeout or a server's restart ends it, is opened anew by
  `createSseTransport({ reopen: true })`;
- a stdio server that exits is started again, as a new process.

A request in flight when the connection drops still fails, since it may have run. A client closed with
`close()` stays closed. Without `reconnect`, a closed transport stays closed, as before 2.18.

```ts
const client = new McpClient(createSseTransport({ url, reopen: true }), { reconnect: true });
const stop = client.onToolsChanged(() => {
  tools = undefined; // list them again before the next turn
});
```

A server that says its tools change sends `notifications/tools/list_changed` when they do, and
`onToolsChanged()` hears each one. What `listTools()` returned is then out of date; the client keeps
no list of its own, so the next `listTools()` or `toNexusTools()` reads the new one. A listener that
throws keeps no other from hearing.

## Transports

`createStdioTransport()` starts a server as a child process and speaks to it over stdin and stdout.
`StdioClientOptions` gives the `command`, `args`, `env`, `cwd`, and `inheritEnv`. `node:child_process`
is imported only when the transport starts, so a browser or edge build that uses HTTP never reaches
for it. When the process exits, the transport closes; started again, it runs a new process, which
reads its own output from the start.

A server is often third-party code, so it does not inherit this process's environment. It starts
with `env` and only what a process needs to run:
- the path, and the home and temporary directories;
- the user, the locale, and the terminal;
- the Windows system variables;
- proxy and certificate settings.

An API key or a database URL reaches a server only when `env` names it. `inheritEnv` widens this: a
list adds variables by name, `true` passes the whole environment, as releases before 2.4 did, and
`false` passes nothing but `env`.

`createHttpTransport()` speaks streamable HTTP, MCP's transport since 2025-03-26: one POST per
message, carrying the `mcp-session-id` the server assigns and the protocol version, answered as JSON
or as a stream of events. A stream is read as it comes, so the server's progress, and a question it
asks the client in the middle of a call, arrive while the call runs. With `listen`, a `GET` stream
carries what the server sends of its own accord, such as its tools changing; a server that answers it
with 405 offers none. `close()` ends the session with a `DELETE`. `HttpClientOptions` gives the
`url`, `headers` such as authorization, `listen`, and a `fetch` to use instead of the global one.

`createSseTransport()` speaks to a server over HTTP with Server-Sent Events, the transport before
streamable HTTP that many servers still serve. It opens one `GET` on the server's event stream, posts
each message to the endpoint the stream names, and reads every answer from the stream.
`SseClientOptions` gives the `url`, `headers` for the stream and every message, a `fetch`, and
`timeoutMs`, how long to wait for the endpoint (10 seconds). The endpoint must be on the stream's
origin, so a server cannot send the messages, and their headers, anywhere else. When the stream ends,
the transport closes and the requests waiting on it fail. So does a stream that cannot open, or names
no endpoint in time: the transport lets it go and closes at once. With `reopen`, the next message
after that opens a new stream; only the current stream's end closes the transport, never one it
replaced. `close()` closes it at once and for good.

`McpTransport` is the whole contract, for a transport of your own: `send()` a message, `onMessage()`
to receive them, and optional `start()` and `onClose()` with `close()`. The messages are JSON-RPC:
`JsonRpcRequest` expects a `JsonRpcResponse` with the same id, carrying a result or an error, and a
`JsonRpcNotification` expects nothing; `JsonRpcMessage` is any of the three. `LineDecoder` splits a
byte stream into newline-delimited messages, holding a partial message until the rest arrives and
skipping a malformed line rather than losing the stream.

## The server

`McpServer` takes `McpServerOptions`: a `name` and `version` to report, the `tools` to expose, its
`resources`, each an `McpServerResource`, a URI with a name, description, MIME type, and a `read()`
function, its `prompts`, and `timeoutMs`, how long a question it asks a client waits (60 seconds).
`register()`, `registerPrompt()`, and `registerResource()` add one later and tell every connected
client that list changed. `connect()` starts serving a client over a transport, and serves each
client connected to it at once; `close()` stops, and `handle()` takes one message directly, for a
host that carries messages itself.

An `McpServerPrompt` has a name, a title and description, the `McpPromptArgument` values it takes,
and `get()`, which renders it: text, which is one user message, or each `McpPromptMessage`. A
required argument a client leaves out is refused with invalid-params. A client may subscribe to a
resource; `resourceUpdated()` tells each subscribed client it changed, and only those.

A tool the server runs may ask the client that called it for what it needs, through `context.mcp`, the
`McpCallerRequests` of that call: `createMessage()` asks its model, `elicit()` asks its person, and
`listRoots()` its roots. Each is refused before it is sent when the client did not declare it, waits
`timeoutMs`, and is cancelled, and the client told, when the call is. `createMessage()`, `elicit()`,
and `listRoots()` on the server itself ask the one client connected, and refuse when several are,
since only a tool's context knows which called. `onRootsChanged()` hears a client's roots change.

```ts
const summarize = tool({
  name: 'summarize',
  description: 'Summarizes a document with the caller’s own model',
  parameters: { type: 'object', properties: { id: { type: 'string' } } },
  execute: async ({ id }, context) => {
    const answer = await context?.mcp?.createMessage({
      messages: [{ role: 'user', content: { type: 'text', text: `Summarize: ${await read(String(id))}` } }],
      maxTokens: 300,
    });
    return answer?.content.type === 'text' ? answer.content.text : '';
  },
});
```

Each tool is listed with the annotations its capabilities give it, from `mcpToolAnnotations()`, read
so a client is never told less than the tool may do. It is read-only only when every capability reads
files or is `mcp.read`, or there are none. It is not destructive only when everything else it does is
`mcp.write`. It stays within itself only when every capability is the filesystem's or an MCP tool's.
A tool whose capabilities depend on its arguments, or that declares none, is listed without
annotations, which a client reads as able to do anything.

It answers `initialize`, `ping`, `tools/list`, `tools/call`, `resources/list`, `resources/read`,
`resources/subscribe`, `resources/unsubscribe`, `prompts/list`, and `prompts/get`, and stops a tool's
call when its client cancels it, through the call's `signal`.
A tool with neither `execute` nor `stream` is a schema for a model to call elsewhere, so the server
reports it as unknown when called. A `stream` tool runs too, and when the call names a
`progressToken`, each report goes back as `notifications/progress`: a number is the progress, an
object's `progress`, `total`, and `message` are used, and anything else is the message. A tool that throws answers with `isError` and the message rather
than failing the request. `createStdioServerTransport()` serves over this process's stdin and stdout,
which is how an editor or desktop assistant launches it; nothing else may be written to stdout.

### Serving over HTTP

`mcpHttpHandler()` serves an `McpServer` over streamable HTTP as a Web handler, an `McpHttpHandler`,
for any server that speaks `Request` and `Response`. Each client that sends `initialize` gets a
session of its own, named in the `Mcp-Session-Id` header it sends from then on. A request is answered
as a stream of events that carries the server's progress, and its questions for the client, before
the answer; two requests in flight are each answered on their own stream. A notification, or an
answer to the server's question, is taken with 202. A `GET` holds the session's own stream, for what
the server sends of its own accord, and a `DELETE` ends it.

```ts
const handle = mcpHttpHandler(new McpServer({ tools: [summarize], prompts: [triage] }));
Bun.serve({ port: 3001, fetch: handle });
```

`McpHttpOptions` sets `allowedOrigins`: a request from a web page on any other origin than the
endpoint's own is refused with 403, so a page cannot reach a server on the person's machine through
their browser. `json` answers each request as JSON, once the server finishes. `sessionTtlMs` ends a
session idle that long (30 minutes), unless a stream of it is open; `maxSessions` refuses more than
that many at once with 503 (1,000); and `sessionId` makes the ids. A message without a session, or
naming one that has ended, is refused with 400 or 404, a protocol version it does not speak with 400,
and a batch, which 2025-06-18 dropped, with 400.

## A registry of servers

`McpRegistry`, on `nexus-ai-pro/mcp/registry`, runs many MCP servers from one configuration. It is the
package's integration catalogue: whatever an MCP server exposes, under your policy.

An `McpRegistryConfig` lists the `servers` and named `bundles` of their tools. It also accepts
`mcpServers`, so the file desktop MCP clients use works as it is. `McpRegistry.fromFile()` reads one
from JSON. `validateMcpConfig()` checks one, and an `McpRegistryError` names the server and field at
fault.

```json
{
  "mcpServers": {
    "github": {
      "command": "github-mcp-server",
      "env": { "GITHUB_TOKEN": "${GITHUB_TOKEN}" },
      "allowTools": ["list_*", "get_*", "create_issue"],
      "denyTools": ["delete_*"]
    },
    "docs": { "url": "https://docs.example.com/mcp", "headers": { "authorization": "Bearer ${DOCS_KEY}" } }
  },
  "bundles": { "support": ["github/create_issue", "docs"] }
}
```

```ts
import { McpRegistry } from 'nexus-ai-pro/mcp/registry';

const registry = await McpRegistry.fromFile('mcp.json');
const agent = createAgent({ client: ai, model: 'gpt-5.4', tools: await registry.tools('support') });
```

An `McpServerConfig` describes one server:

| Field | What it sets |
| --- | --- |
| `command`, `args`, `env`, `cwd`, `inheritEnv` | A local server, run over stdio, with what it inherits from this process's environment. |
| `url`, `headers` | A remote server, reached over HTTP. |
| `allowTools`, `denyTools` | Which tools agents see. `*` matches any run of characters, and deny wins. |
| `prefix` | The prefix of tool names agents see. Defaults to the server's name, so two servers' `search` stay apart; `false` keeps names as they are. |
| `timeoutMs` | How long each request may take. |
| `enabled` | `false` keeps the server in the file but out of the registry. |

Any string may hold `${NAME}` placeholders, filled from the environment when the server connects. So
the file names credentials without containing them. A missing variable is named in the error, never
its value.

Servers connect on first use and stay connected.

| Method | What it does |
| --- | --- |
| `tools()` | Tools for an agent, from bundles or server names; every server's tools when given none. Each tool appears once. A bundle entry is a server, or `server/tool` with a pattern such as `github/list_*`. |
| `client()` | One server's `McpClient`. |
| `serverNames()`, `bundleNames()` | What is configured. |
| `refresh()` | Forgets the tool lists read so far. |
| `health()` | Connects to every server in parallel and reports each as an `McpServerHealth`: whether it answered, how long it took, and how many tools it offers. One failure does not fail the check. |
| `close()` | Closes every connection. |

`McpRegistryOptions` sets where placeholders are read from (`process.env`), the handshake's
`clientInfo`, a `fetch` for HTTP servers, and a `transport` factory for custom transports and tests.

## Errors and versions

`McpError` carries a JSON-RPC error code, with `JSON_RPC_ERRORS` naming the standard ones: parse,
invalid request, method not found, invalid params, and internal. The server answers an unknown method
with method-not-found, so a client can tell an unsupported feature from a failure.
`MCP_PROTOCOL_VERSION` is the protocol revision both directions speak, `2025-06-18`.

## Limitations

- A stream that drops is not resumed from where it was: the HTTP handler numbers no events, and a
  client reads a dropped answer's request as failed.
- A request in flight when a connection drops fails, rather than being sent again, since it may have
  run. A registry's tool lists are read again only after `refresh()`.
- A tool's capabilities from its annotations are only as true as its server is honest.
- The server offers no resource templates and no completions, and a client's sampling handler is
  given no other server's context: `includeContext` is the server's request, not a promise.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/protocols.md

# Agent protocols


Three open protocols put an agent where other software already expects one:
- **AG-UI** connects an agent to a user interface. Any AG-UI frontend renders the run as it happens:
  text as it streams, each step, each tool call with its result, and questions for the person.
- **A2A**, the Agent2Agent protocol, connects agents to each other. Other agents discover yours from
  its card and send it tasks. Yours calls theirs as a tool.
- **ACP**, the Agent Client Protocol, puts an agent inside a code editor. The editor starts it as a
  subprocess and shows its edits, its plan, and its requests for approval.

Each protocol has its own entry point, and none needs a dependency. An entry point never imports
the graph runtime: it drives a `ProtocolGraph`, the part of a compiled graph a protocol needs.
- `invoke()` starts a run.
- `resumeInterruptsWith()` answers a paused run's questions by id.

An agent from `createAgent()` or `createDeepAgent()` already satisfies it, and so does any compiled
graph. The protocol passes `ProtocolRunOptions`:
- the thread;
- a signal that cancels the run;
- an `onEvent` callback for the run's fine-grained events, which the protocol translates as they
  arrive;
- the `principal`, when the transport authenticated the caller.

A run ends with a `ProtocolResult`: its status, its state, and any `ProtocolInterrupt` it is
waiting on. Each interrupt carries an id, the question, and a payload. An agent's tool approval
carries the call as its payload, and every protocol below shows that call as an approval rather than
as a general question.

## AG-UI: an agent in a user interface

`agUiHandler()` serves a graph to AG-UI frontends. It takes a `Request` and returns a `Response`,
so it mounts in any framework that uses them:

```ts
import { createAgent } from 'nexus-ai-pro/agent';
import { agUiHandler } from 'nexus-ai-pro/protocols/ag-ui';

const agent = createAgent({ client: ai, tools, streamTokens: true, interruptOn: { send_email: true } });

// A Next.js route handler: app/api/agent/route.ts
export const POST = agUiHandler(agent);
```

The frontend POSTs an `AgUiRunInput`, AG-UI's `RunAgentInput`, with:
- the thread and run ids;
- the whole conversation as `AgUiMessage` values;
- any tools and context the frontend offers;
- its shared state;
- whatever it forwards.

`agUiMessages()` converts the conversation into the messages an agent takes. A developer message
becomes a system message, and an assistant's tool calls and the tool results that answer them are
kept. The default graph input is `{ messages }`. `AgUiOptions.input` builds another input from the
run and the converted messages, for a graph with other state or an application that uses the
frontend's tools.

The response is a stream of Server-Sent Events, each an `AgUiEvent` that `encodeAgUiEvent()`
writes as one `data:` frame:
- `RUN_STARTED` first;
- `STEP_STARTED` and `STEP_FINISHED` around each node the graph runs;
- `TEXT_MESSAGE_START`, `TEXT_MESSAGE_CONTENT` with each streamed piece, and `TEXT_MESSAGE_END`;
- `TOOL_CALL_START`, `TOOL_CALL_ARGS`, and `TOOL_CALL_END` for each call, then `TOOL_CALL_RESULT`.
  While the tool runs, each report of its progress is a `CUSTOM` event named `tool_progress`, whose
  value has the call's id and name and the progress;
- `STATE_SNAPSHOT` with the final state, unless `stateSnapshot` is false;
- last, `RUN_FINISHED` with the answer as its `result`, or `RUN_ERROR` with the failure's message
  and code.

A subgraph's events belong to the parent's step, so only the graph's own nodes are reported as
steps. Reasoning is not sent as message text.

### State, transcripts, and the frontend's tools

Three options turn on the rest of the protocol. Each is off by default, so a frontend sees the
events above and nothing more until it asks:

| Option | What the frontend gets |
| --- | --- |
| `stateDeltas` | The state as it changes: a `STATE_SNAPSHOT` after the first step, then a `STATE_DELTA` after every step that changed it. A delta is an RFC 6902 JSON Patch, a list of `AgUiPatchOperation` values. A message added to the transcript is one `add` at `/messages/-`, so a long conversation costs only what is new. |
| `messagesSnapshot` | A `MESSAGES_SNAPSHOT` with the whole transcript before the run finishes. The frontend's messages keep their ids, a tool's result has the id of its `TOOL_CALL_RESULT`, and the rest are named after the run. |
| `frontendTools` | The tools the frontend sends, offered to the model. |

A tool the frontend offers runs in the frontend. When the model calls one, the frontend sees
`TOOL_CALL_START`, `TOOL_CALL_ARGS`, and `TOOL_CALL_END` for it and no result, and the run finishes
once the agent's own calls of that step are done. The frontend runs the tool and sends the
conversation back with a `tool` message carrying the result, and the turn carries on from there.
The agent's state says why it stopped: `stopReason: 'client_tools'`. A frontend tool named like one
of the agent's own is ignored, so a page can never replace a tool that runs on the server.

```ts
export const POST = agUiHandler(agent, { stateDeltas: true, messagesSnapshot: true, frontendTools: true });
```

### Questions and approvals

A run that pauses ends with `RUN_FINISHED` and an interrupt outcome. Each question is an
`AgUiInterrupt`:
- its id;
- a reason: `tool_call` for an approval, or `input_required` for anything else;
- the question as `message`;
- for an approval, the `toolCallId`;
- the whole payload in its `metadata`.

The frontend answers with a new run on the same thread. It carries one `AgUiResumeEntry` per
question:
- `resolved`, with a payload;
- or `cancelled`.

By default:
- a resolved entry with a payload answers with that payload;
- a resolved entry without one answers `true`, which approves a call;
- a cancelled entry answers `false`, which refuses it.

`AgUiOptions.answer` maps entries another way. Pass `{ approved: true, args }` as the payload to
approve with corrected arguments.

### Callers and other transports

`AgUiOptions.principal` resolves the caller from each request, such as from a session cookie or a
bearer token. The run then carries that `principal`, so tools act for that user and a permission
policy decides by their roles. The request's signal cancels the run when the frontend disconnects.

`agUiEvents()` is the same translation without HTTP: an async generator of events for a WebSocket,
a queue, or a test. It takes `AgUiEventsOptions`, which is `AgUiOptions` with the caller already
known and a signal.

## A2A: agents calling agents

`a2aHandler()` serves a graph as an A2A agent, speaking version `A2A_PROTOCOL_VERSION` (1.0) over
JSON-RPC and HTTP+JSON, and over gRPC with `a2aGrpcService()`:

```ts
import { a2aHandler } from 'nexus-ai-pro/protocols/a2a';

const handler = a2aHandler(researcher, {
  card: {
    name: 'Researcher',
    description: 'Answers research questions with sources',
    version: '1.0.0',
    url: 'https://agents.example.com/a2a',
    skills: [{ id: 'research', name: 'Research', description: 'Finds and cites sources', tags: ['research'] }],
  },
  principal: (request) => verifyAgentToken(request.headers.get('authorization')),
});
```

A `GET` returns the `A2aAgentCard` at `/.well-known/agent-card.json`. `A2aServerOptions.card` gives
the name, description, version, provider, and each `A2aSkill`. The handler fills in the rest:
- an interface for each binding: JSON-RPC and HTTP+JSON at the card's `url`, as `bindings` chooses
  (both by default), then each of `interfaces`, such as the gRPC service's address;
- the capabilities: streaming, and push notifications and an extended card when they are set up;
- the input and output media types, unless given.

A `POST` to the card's `url` is a JSON-RPC call. HTTP+JSON and gRPC call the same methods:
- `SendMessage` starts a task from an `A2aMessage` and returns the `A2aTask` when it is done.
  With `returnImmediately`, it returns the task while it runs.
- `SendStreamingMessage` streams `A2aStreamEvent` values: the task, then the answer as artifact
  updates while it streams, then the final status. A tool's progress is a `TASK_STATE_WORKING`
  status update while it runs, whose message says what is happening in text and carries the
  progress as data.
- `GetTask` reads a task, with as much of its history as `historyLength` asks for.
- `ListTasks` lists tasks, most recent first, by `contextId` and by `status`, a page at a time:
  `pageSize` (50, at most 100) and the `nextPageToken` of the page before. A page is an
  `A2aTaskPage`, with `totalSize` when the store counts.
- `SubscribeToTask` follows a task still running, from now until it finishes or waits: the task,
  then its events. It works on any replica. Where the task runs, the events come as they happen;
  elsewhere, the replica reads the task every `pollMs` (1 second) and reports each change.
- `CancelTask` stops a task that is still running. On another replica, it stands too: the replica
  running the task finishes its work but does not record it over the cancel.
- `CreateTaskPushNotificationConfig`, `GetTaskPushNotificationConfig`,
  `ListTaskPushNotificationConfig`, and `DeleteTaskPushNotificationConfig` manage a task's
  webhooks, when the agent pushes. `SendMessage` takes one too, in its configuration.
- `GetExtendedAgentCard` returns the card for an authenticated caller, when the agent has one.

A message is made of `A2aPart` values: text, data, or a file by URL or bytes. Who sent it is an
`A2aRole`. A task moves through the `A2aTaskState` values. A finished task carries its answer as an
`A2aArtifact` and as the status message.

### HTTP+JSON and gRPC

HTTP+JSON is served at routes under the card's `url`, so mount the handler at that path and every
path under it. Each route is one method:

| Route | Method |
| --- | --- |
| `POST /message:send` | `SendMessage` |
| `POST /message:stream` | `SendStreamingMessage`, as Server-Sent Events of the events themselves |
| `GET /tasks/{id}` | `GetTask`, with `historyLength` in the query |
| `GET /tasks` | `ListTasks`, with its filters and paging in the query |
| `POST /tasks/{id}:cancel` | `CancelTask` |
| `POST /tasks/{id}:subscribe` | `SubscribeToTask`; a `GET` works too |
| `POST /tasks/{id}/pushNotificationConfigs` | `CreateTaskPushNotificationConfig` |
| `GET /tasks/{id}/pushNotificationConfigs` | `ListTaskPushNotificationConfigs` |
| `GET /tasks/{id}/pushNotificationConfigs/{configId}` | `GetTaskPushNotificationConfig` |
| `DELETE /tasks/{id}/pushNotificationConfigs/{configId}` | `DeleteTaskPushNotificationConfig` |
| `GET /extendedAgentCard` | `GetExtendedAgentCard` |

An error is the JSON of Google's AIP-193: an HTTP status, and a `google.rpc.ErrorInfo` whose reason
names the A2A error, such as `TASK_NOT_FOUND` with 404. `a2aErrorStatus()` gives each code's HTTP
status, gRPC status, and reason, as an `A2aErrorStatus`; `a2aErrorCode()` reads a reason back as its
code. `bindings: ['JSONRPC']` serves JSON-RPC alone.

`a2aGrpcService()`, on `nexus-ai-pro/protocols/a2a/grpc`, serves the gRPC binding over the same
handler, and so over the same tasks:

```ts
import { a2aHandler } from 'nexus-ai-pro/protocols/a2a';
import { a2aGrpcService } from 'nexus-ai-pro/protocols/a2a/grpc';

const handler = a2aHandler(researcher, {
  card,
  interfaces: [{ url: 'agents.example.com:50051', protocolBinding: 'GRPC' }],
});
const grpc = await a2aGrpcService(handler).listen(50051, '0.0.0.0');
```

An `A2aGrpcService` answers each method of `lf.a2a.v1.A2AService` as the specification's proto
defines it, over `node:http2` and with no dependency:
- a stream is a server-streaming call;
- an error is a gRPC status, such as `NOT_FOUND`, with its `ErrorInfo` in `grpc-status-details-bin`;
- the caller is read by the handler's `principal`, from a `Request` with the call's metadata as its
  headers;
- `grpc-timeout` cancels a call at its deadline, and a gzip or deflate message is read.

`listen()` serves plain HTTP/2, for a server behind a proxy that ends TLS. To end TLS here, create
your own server with `http2.createSecureServer()` and give each of its streams to `handle()`. List
the address on the card as `host:port`, the form gRPC names a target in. `close()` stops taking
calls, tells each connected client to go, and resolves once the calls in flight finish.
`A2aGrpcServiceOptions.maxMessageBytes` caps a request's message, 4 MiB by default.

A task is the same task on every binding: a caller can start it over gRPC, answer its question over
HTTP+JSON, and read it over JSON-RPC, with the same history. Each binding runs a method through
`A2aHandler.call()`, with an `A2aCallContext` naming its `A2aBinding`, and gets back an
`A2aCallOutcome`: a result, an error, or a stream of events. A binding of your own serves the same
tasks the same way, with the caller read by `authenticate()`.

### Conversations and questions

A message's `contextId` is the conversation. Each task in a context sees the context so far: the
agent's transcript, tool calls included. `A2aServerOptions.input` builds another graph input from
the message and that history.

A task that pauses is `TASK_STATE_INPUT_REQUIRED`. Its status message asks the question in text and
lists the interrupts as data. The calling agent answers with a message that names the task's
`taskId`, and the task continues. By default:
- the message's first data part is the answer, such as `{ approved: true }` for a tool approval;
- otherwise, its text is the answer;
- text that answers an approval is read as words: yes, ok, allow, approve, or confirm allows, and
  anything else refuses, so "no" never reads as consent.

`A2aServerOptions.answer` maps messages another way.

Errors use A2A's codes, listed in `A2A_ERRORS`, beside JSON-RPC's own:
- a task that does not exist;
- a task that cannot be cancelled because it finished;
- a message to a task that is not waiting for input;
- an `A2A-Version` header this agent does not speak.

`A2aServerOptions.principal` authenticates the calling agent, and its tasks run as that principal.

### Tasks that outlive the process

Tasks and conversations live in an `A2aTaskStore`. Each task is an `A2aTaskRecord`: the task, its
messages, the questions it waits on, and its webhooks. `A2aTaskQuery` is what a listing asks for.
- `MemoryA2aTaskStore` is the default, for one replica. `maxTasks` bounds the tasks and contexts
  it keeps, 10,000 each.
- `storeA2aTasks()` keeps them in any long-term `Store`: the memory, Redis, Postgres, or SQLite
  store, so every replica shares them. Listing filters in the store and does not count.

A task paused for an answer then survives a restart: another replica reads it, and the message
that answers it resumes the run there. The graph resumes from its own checkpoints, so give it a
shared checkpointer too.

```ts
import { a2aHandler, storeA2aTasks } from 'nexus-ai-pro/protocols/a2a';

const agent = createAgent({ client: ai, tools, interruptOn: { send_email: true }, checkpointer: postgresCheckpointer });
export const handler = a2aHandler(agent, {
  card,
  store: storeA2aTasks(postgresStore),
  recoverEveryMs: 15_000, // take over what a stopped replica was working on
});
```

A task that is working survives its replica too. `a2aHandler()` returns an `A2aHandler`: the
request handler, plus `recover()` and `close()`. While a replica runs a task, the record holds its
`A2aTaskLease`: the replica's `owner` name and when the lease lapses. The replica renews it every
third of `lease.ms` (30 seconds). The record also holds the `A2aTaskRun`: whether a message started
the run or answers resumed it, that message, the answers, and the principal. A new task is written
with its run from its first write, so a task submitted to a replica that stops before starting it is
started by another, for the same principal.

`recover()` reads the working tasks whose lease has lapsed, and the submitted tasks no replica
started, and takes each over, up to `limit` (10).
It claims the lease, then carries the run on: from the graph's last checkpoint through the graph's
`continueWith()`, by answering the questions again when a resume never reached the graph, or from
the start when the run left no checkpoint. It resolves to an `A2aRecoveryReport` once the tasks are
claimed. `recoverEveryMs` runs it on a timer, which `close()` stops.

`A2aTaskStore.claim()` takes a lease in one step, so of two replicas recovering at once, one takes
the task. `MemoryA2aTaskStore` has it, and `storeA2aTasks()` builds it on the store's
`putIfVersion()`, bumping the record's `version`. A replica that finds another holding its task's
lease stops the run, and never records its result over the other's. A graph of your own without
`continueWith()` cannot carry a run on: its task fails with that reason rather than staying working.

In the test suite, a task whose replica dies halfway through a tool is taken over on every store.
The model call the dead replica finished is not made again, the email it never sent is sent once,
and two replicas recovering at the same moment take the task over once. A crash-injection suite then
kills the replica at every write a task makes, and another replica takes it over each time and ends
it as a task never interrupted ends. A replica that dies before the task's first write has told its
caller so. A store that cannot record a task's failure leaves the task working, with its lease, for
the replica that takes it over.

### Push notifications and the extended card

With `pushNotifications`, a caller can give a task a webhook, an `A2aPushNotificationConfig` with a
URL, a token, and optionally its authentication. Each change of the task's state is then POSTed to
it as a status update, in order:
- with the caller's token as `X-A2A-Notification-Token`, and its `authentication` as the
  `Authorization` header, such as `Bearer` and a credential the receiver checks;
- signed with `A2aPushOptions.secret` as the server's operation webhooks are, in
  `x-nexus-signature`, so the receiver checks it with `verifyOperationWebhook()`;
- within `timeoutMs` (5 seconds), without following a redirect. A failed delivery goes to `onError`
  and never fails the task.

With `A2aPushOptions.text`, a webhook also receives the answer as it is written: each piece is an
artifact update, the same pieces a stream sends, delivered before the state that follows them.
`text: { everyMs }` gathers the pieces written within that many milliseconds into one delivery,
for a receiver that wants fewer calls.

```ts
const handler = a2aHandler(researcher, { card, pushNotifications: { text: { everyMs: 250 } } });
```

A webhook is a URL a caller chose, which the agent then fetches. So it must be `https`, and not a
loopback, private, or link-local address or name such as `localhost` or `metadata.internal`.
`allowPrivateNetworks` turns that off, for tests or trusted callers. A name is checked as written,
not as it resolves; to close a name that resolves to a private address, pass a `fetch` that checks
the address it connects to, or send deliveries through an egress proxy.

`extendedCard` gives an authenticated caller more of the card, such as skills only partners see. It
is called with the caller's principal, and `GetExtendedAgentCard` refuses a caller without one.

A card says how a caller authenticates. `securitySchemes` names each `A2aSecurityScheme`: an API key in
a header, a query, or a cookie; HTTP authentication, such as a bearer token; OAuth 2.0, with its
`A2aOAuthFlows`, authorization code, client credentials, or device code; OpenID Connect; or mutual
TLS. `securityRequirements`, on the card or on one skill, list each `A2aSecurityRequirement`: the
schemes a call needs, by name, with their scopes. `signatures` carry each `A2aCardSignature`, a JSON
Web Signature over the card. Every binding carries them, the gRPC card included.

### Signed cards

A card is signed as protocol 1.0 signs one: a flattened JWS over its canonical form, which
`canonicalA2aCard()` writes, the card as its schema has it, without its `signatures` and empty values,
its keys in order as RFC 8785 serializes JSON. That is the form the A2A project's SDK computes, so a
card either signs, the other verifies. `sign`, an `A2aCardSigner` or several, signs every card the
handler serves, each tenant's and the extended card included:

```ts
const handler = a2aHandler(agent, {
  card,
  sign: { key: privateJwk, alg: 'ES256', kid: 'agents-2026', jku: 'https://agents.example.com/jwks.json' },
});

const client = a2aClient({
  url: 'https://agents.example.com/.well-known/agent-card.json',
  verifyCard: (kid) => trustedKeys[kid], // the public key, or undefined for one not trusted
});
```

`A2aClientOptions.verifyCard`, an `A2aCardKeyResolver` given each signature's key id, its key set's URL,
and its algorithm, makes the client check the card before its first call: until a signature holds,
every call fails with `A2aCardSignatureError`, which says why each signature did not, and nothing is
sent. Given an endpoint rather than the card, the client reads the card from its origin to check it;
`a2aGrpcClient()` checks what `card()` and `extendedCard()` read. `signA2aCard()` and `verifyA2aCard()`
do the same for any card. A key is a Web Crypto key, an `A2aCryptoKey`, or a JWK, an `A2aJsonWebKey`, and an
`A2aSignatureAlgorithm` is ES256,
ES384, ES512, RS256, RS384, RS512, PS256, PS384, PS512, or Ed25519, by that name or `EdDSA`; a JWK that
names another algorithm is refused, and a header naming `none`, or an HMAC, never verifies.

### Several tenants behind one handler

`tenants` serves several tenants from one handler, each request routed by the `tenant` it names, as
protocol 1.0 carries it: in its params over JSON-RPC, as the first segment of its path over HTTP+JSON,
such as `/acme/message:send`, and in its message over gRPC.

```ts
const handler = a2aHandler(agent, {
  card,
  principal: (request) => verify(request.headers.get('authorization')), // { tenantId, ... }
  tenants: true,
});
```

Each tenant's tasks and conversations are its own. Another tenant's task is not found, however a
request names it, and a conversation's context id is the tenant's alone. A run carries the tenant on
its principal, so its credentials and its budget are the tenant's.

`A2aTenancyOptions` decides who may act for which tenant. `authorize`, given the tenant and the
caller's principal, defaults to the principal's `tenantId` being that tenant, so a caller nobody
authenticated acts for none; an operator's role can be let in by an `authorize` of your own.
`required` (true) refuses a request that names no tenant. A caller refused is answered with
`A2A_ERRORS.permissionDenied`, this package's own code from the range JSON-RPC leaves to servers:
HTTP 403, and `PERMISSION_DENIED` over gRPC. Nothing runs.

A store keeps each record's `tenant`, and lists one tenant's when `A2aTaskQuery.tenant` names it, as
the memory store and `storeA2aTasks()` do, so a tenant pages through full pages of its own. The handler
filters what any store lists by tenant too, so a store of your own that ignores it leaks nothing.

A client names a tenant on every request, as the specification requires: `A2aClientOptions.tenant`, or
else the tenant of the interface the card lists for its binding. `a2aGrpcClient()` takes the same
option.

Each tenant has a card of its own, at `/{tenant}/.well-known/agent-card.json`, from the root or under the
card's URL: the same card, every interface naming the tenant, so a client that reads it calls as that
tenant over whichever binding it picks, this package's and the A2A project's reference client alike.
An interface that names its own tenant keeps it. Give a client the tenant's card, or, for one that
resolves the card from a base URL, the tenant's base ending in a slash, such as
`https://agents.example.com/acme/`. Without `tenants`, every such path is the card itself, as before.

### Calling other agents

`a2aClient()` talks to any A2A agent, over JSON-RPC or, with `binding: 'HTTP+JSON'`, over HTTP+JSON.
`A2aClientOptions` takes the agent's card URL, whose interface for the binding it then calls, or the
endpoint itself, whose origin serves the card. Pass `fetch` and `headers` for authentication or a
proxy. `a2aGrpcClient()`, on `nexus-ai-pro/protocols/a2a/grpc`, is the same client over gRPC. Give
`A2aGrpcClientOptions` the service as `host:port`, or an `https://` URL for TLS, and the `cardUrl`.
An `A2aGrpcClient` holds one HTTP/2 connection until its `close()`.
An `A2aClient` can:
- read the `card()`, and the `extendedCard()`;
- `send()` a message and wait for its task;
- `stream()` a task's events;
- read a task with `getTask()`, list them with `listTasks()`, and follow one with `subscribe()`;
- `cancel()` one;
- manage a task's webhooks with `setPushNotification()`, `listPushNotifications()`, and
  `deletePushNotification()`.

`A2aSendOptions` continues a conversation by `contextId`, answers a task by `taskId`, sends data
beside the text, gives a webhook as `push`, returns at once with `returnImmediately`, or cancels by
signal. `A2aListTasksOptions` filters and pages a listing. An error on any binding raises an
`A2aError` with A2A's code.

`a2aTool()` makes a remote agent one more tool. `A2aToolOptions` gives the tool's name and a
description written for the model. Each call is a task, and the result reports:
- its state;
- its task and context ids;
- the remote agent's answer, or its question.

The model can answer a question by calling again with the `taskId`, and continue a conversation by
`contextId`. The tool declares the host it reaches as `network:<host>`. A permission policy grants
or denies it like any other network access.

```ts
import { a2aTool } from 'nexus-ai-pro/protocols/a2a';

const research = a2aTool({
  name: 'ask_researcher',
  description: 'Asks the research agent a question; it answers with sources.',
  url: 'https://agents.example.com/.well-known/agent-card.json',
  headers: { authorization: `Bearer ${process.env.RESEARCH_TOKEN}` },
});
const agent = createAgent({ client: ai, tools: [research] });
```

## ACP: an agent in an editor

`serveAcp()` serves a graph to an editor that speaks the Agent Client Protocol, version
`ACP_PROTOCOL_VERSION` (1). The editor starts the agent as a subprocess and writes JSON-RPC to its
standard input, one message per line. `AcpStreams` names the two ends: `process.stdin` and
`process.stdout`, or any pair like them.

```js
#!/usr/bin/env node
// reviewer.mjs: started by the editor
import { createDeepAgent } from 'nexus-ai-pro/deep-agent';
import { serveAcp } from 'nexus-ai-pro/protocols/acp';

const agent = createDeepAgent({ client: ai, interruptOn: { edit_file: true, write_file: true } });
serveAcp(agent, { input: process.stdin, output: process.stdout }, { agentInfo: { name: 'reviewer', version: '1.0.0' } });
```

Nothing else may write to standard output. Logs go to standard error.

Each editor session is a conversation on its own thread, and `AcpSessionInfo` describes it:
- its id;
- the directory it was opened in;
- the MCP servers the editor offered.

A prompt is a list of `AcpContentBlock` values. Text is read as written, an embedded file is
fenced under its URI, and a link is named. An image or audio block reaches the model as an image or
audio part beside the text: the agent tells the editor in `initialize` that it reads both. The
default graph input is `{ messages }`: the session's conversation and the prompt. `AcpOptions.input`
builds another from the prompt text, the session, the history, and the blocks.

While the prompt runs, the editor receives each `AcpSessionUpdate`:
- the agent's text as it streams, and its reasoning as thought;
- each tool call, then its result:
  - the call has a title and an `AcpToolKind`, and the file it touches when it has a `path`
    argument;
  - while it runs, each report of its progress updates the call, still `in_progress`;
  - the result is text, and an edit's result is also a diff;
- the plan, whenever `write_todos` records one.

`AcpOptions.kind` decides a tool's kind. By default, the kind is guessed from the name:
`read_file` reads, `edit_file` edits, and `execute` runs a command.

### Sessions an editor reopens

Sessions live in an `AcpSessionStore`, each as an `AcpSessionRecord`: the session, its
conversation, the question it waits on, and the person's standing choices to always allow or
reject a tool.
- `MemoryAcpSessionStore` is the default.
- `storeAcpSessions()` keeps them in any long-term `Store`: the memory, Redis, Postgres, or SQLite
  store.

The agent offers `session/load`. After the agent restarts, the editor reopens a session by its id.
The agent then replays the conversation as updates (the person's messages, its own, and each tool
call with its result) and asks again any question it was waiting on, so the next prompt answers it.
A store outside the process makes a session survive a restart; give the graph a durable
checkpointer too, so the paused run resumes. Audio is kept in a session as a note that it was
heard, not as its bytes.

A turn the agent was in the middle of survives too. While a prompt runs, its record holds an
`AcpTurnRecord`: whether the prompt started a run or answered the last turn's questions, the prompt,
and the answers. When `session/load` finds one, the agent carries the turn on once the load has
answered, streaming its updates as a prompt would:
- from the graph's last checkpoint, so steps that finished are not run again;
- by answering again a resume that never reached the graph;
- or from the start, when the run left no checkpoint.

A turn stopped while the editor was deciding an approval asks for it again. A prompt sent meanwhile
waits for the carried-on turn. A graph without `continueWith()` cannot carry a turn on, so the agent
says so in the session, and the turn ends. In the test suite, an agent that stops halfway through a
tool is started again and its session reopened: the model call it finished is not made again, the
email it never sent is sent once, and the answer reaches the editor.

### The editor's files, terminal, and MCP servers

`acpEditor()`, on `nexus-ai-pro/protocols/acp/editor`, gives an agent what the editor offers, as an
`AcpEditor`. Pass it to `serveAcp()` as `editor`, an `AcpEditorBridge`, and its `toolSource` to the
agent:

```ts
import { createAgent } from 'nexus-ai-pro/agent';
import { serveAcp } from 'nexus-ai-pro/protocols/acp';
import { acpEditor } from 'nexus-ai-pro/protocols/acp/editor';

const editor = acpEditor();
const agent = createAgent({
  client: ai,
  toolSource: editor.toolSource,
  interruptOn: { editor_write_file: true, editor_run_command: true },
});
serveAcp(agent, { input: process.stdin, output: process.stdout }, { editor });
```

Each session gets the tools the editor said it can serve, in `initialize`, as `AcpClientCapabilities`:
- `editor_read_file` reads through the editor, including changes the person has not saved;
- `editor_write_file` writes through the editor, which shows the change;
- `editor_run_command` runs a command in the editor's terminal, which the person sees, and returns
  its output and exit code. A cancelled run kills the command, and the terminal is always released;
- the tools of each MCP server the editor offered for the session, prefixed by the server's name.
  Servers started over stdio run in the session's directory, and HTTP and SSE servers are reached as
  given, with the editor's headers. The agent tells the editor, in `initialize`, that it connects
  servers over HTTP and SSE, as the bridge's `mcpCapabilities`, so the editor offers those too.

A relative path is taken from the session's directory. Each tool declares what it touches, as
`filesystem:read:<path>`, `filesystem:write:<path>`, or `shell:<command>`, so a permission policy
decides it, and an approval asks the person in the editor. An MCP server's tools declare what their
server says they do: `mcp.read`, `mcp.write`, or `mcp.destructive` under the server's name, and
`network` for one that reaches beyond it. A policy that lets MCP tools read and nothing else,
`{ custom: { 'mcp.read': 'allow' } }`, then runs a server's lookup and refuses its delete, with no
rule naming either. The bridge asks the editor through an
`AcpConnection`, which `serveAcp()` attaches. When the editor closes its end, the bridge closes the
MCP servers it started.

A session's servers can change while it lasts. An SSE server whose stream ends, as a proxy's timeout
or a restart ends it, is reached again at its next call, on a new stream, and greeted again. A server
that says its tools changed has them listed again before the agent's next turn, so a tool it adds
between turns is offered at the next one. When listing them again fails, the agent keeps the tools as
last listed, `onError` hears why, and they are listed again at the turn after.

`AcpEditorOptions` turns off `files`, `terminal`, or `mcp`, and sets `mcpTimeoutMs` (30 seconds),
`mcpFetch`, `maxOutputBytes` (64 KB), and `onError` for a server that fails to start.
`mcpToolCapabilities` decides what MCP tools declare: `true`, the default, reads their annotations, a
function decides per tool and server, and `false` declares nothing, as before 2.17.

The prompt answers with an `AcpStopReason`:
- `end_turn` when the agent answered;
- `max_turn_requests` when it ran out of iterations;
- `cancelled` when the editor cancelled the session.

### Approvals

A tool call that needs approval asks the editor, which shows the person four choices: allow once,
always allow, reject once, and always reject. An always choice holds for that tool for the rest of
the session. A rejected call runs nothing, and the agent is told it was refused. Cancelling the
session while the editor asks ends the turn as cancelled.

ACP asks only for permissions, so the agent asks any other question in its message, and the turn
ends there. The next prompt is the answer. `AcpOptions.answer` maps it, and defaults to the
prompt's text.

`AcpOptions.principal` is who the agent acts for: the person at the editor. The returned
`AcpServer` resolves `closed` when the editor closes its end. `close()` stops every prompt still
running.

## Proof

Each protocol is tested from the other side of the wire, with no Nexus client code on that side:
- **AG-UI.** A plain `fetch` and Server-Sent Events parser renders a streaming agent's run. The
  render includes the transcript, the tool call with its arguments and result, and the final answer.
  Every event carries the fields the specification requires. A run that needs approval finishes
  with an interrupt, and a resume entry on the same thread continues it.
- **A2A, served.** A client that writes raw JSON-RPC reads the card, sends messages, and continues
  a conversation by context. It answers an approval, streams a task, and cancels one. It also gets
  each of the specification's error codes.
- **A2A, calling.** The client and tool are tested against a stub agent written from the
  specification alone. The stub answers with a task, with a bare message, and with a stream split
  mid-frame. Then one Nexus agent calls another through the tool.
- **A2A, on every binding.** One agent is served over JSON-RPC, HTTP+JSON, and gRPC from one
  handler. A task started over gRPC is answered over HTTP+JSON, and all three bindings read it back
  the same, with the same history. A webhook asked for text receives, piece by piece, the text a
  stream receives. Each binding's errors carry the protocol's codes, and a gRPC message is written
  byte for byte as the specification's proto numbers it.
- **A2A, for several tenants.** One handler serves two tenants over all three bindings. Each lists,
  reads, and continues only its own tasks and conversations, in full pages, and each run is its
  tenant's. A caller naming another tenant is refused on every binding, with nothing run. Over gRPC, a
  card's every security scheme, OAuth flow, requirement, and signature reads as its JSON has them.
- **ACP.** An editor's end of the connection, written line by line, does the following:
  - it initializes;
  - it opens a session;
  - it sends a prompt with an embedded file;
  - it approves an edit with "always allow";
  - it sees the plan, the diff, and the answer.

  A second prompt in the session asks nothing and carries the transcript over. Rejecting runs
  nothing, cancelling ends the turn as cancelled, and a question is asked in words.
- **ACP, with the editor's MCP servers.** An editor offers one server over stdio, one over HTTP, and
  one over SSE. The agent says it connects HTTP and SSE, calls a tool on each, and a policy that only
  lets MCP tools read refuses each server's destructive tool before the server sees it, with no rule
  naming a tool.
- **ACP, with an MCP server that changes.** Between two turns, an editor's SSE server adds a tool,
  says so, and its stream drops, and its sessions are its streams, so a new stream not greeted again
  is refused. At the next turn the agent is offered the new tool and calls it over a new stream.

Each protocol is also tested across releases and against the reference client of its own project:
- **The previous minor.** A 2.5 client drives this release's server, and this release's client a
  2.5 server, on the server's HTTP API and on A2A. One harness written against AG-UI and ACP drives
  both releases' servers. A 2.6 client asking a 2.5 agent for something 2.6 added gets a protocol
  error, never a broken answer.
- **Reference clients, nightly.** `scripts/interop/run.mjs` installs the latest `@ag-ui/client`,
  `@a2a-js/sdk` with its gRPC transport's peers, `@agentclientprotocol/sdk`, and
  `@modelcontextprotocol/sdk` in a directory of their own, without their scripts. Each drives its protocol:
  - the AG-UI client pauses a run, resumes it, and rebuilds the state from its deltas;
  - the A2A client sends, approves, reads, lists, and streams tasks, and sets their webhooks,
    over each of JSON-RPC, HTTP+JSON, and gRPC; a task it starts over gRPC it answers over
    HTTP+JSON, and every binding reads the same history;
  - the ACP client starts an agent as an editor would. The agent reads a file through it, writes
    once the client allows it, and has its session reloaded;
  - the ACP client offers that agent MCP servers built with `@modelcontextprotocol/sdk` over stdio,
    HTTP, and SSE. Each server's tool that reads runs, and its destructive one is refused.

  The `Interoperability` workflow runs it every night.

## Limitations

- **AG-UI.** A state delta compares JSON: a value JSON cannot carry, such as a `Date` or a `Map`,
  reaches the frontend as JSON makes it. A change inside an array other than appending replaces the
  whole array.
- **A2A tasks running when their process stops.** A step that was halfway through when the
  process stopped runs again on the replica that takes it over, as a graph's steps do after any
  crash, so a tool with side effects needs an idempotency key. A slow replica that loses its lease
  stops its run, but a tool already running finishes.
- **A2A cards.** A signature covers what the card's schema names: a field outside it is not covered,
  as the A2A project's SDK says of its own. The client trusts the keys `verifyCard` gives; fetching a
  key set from a card's `jku` is the resolver's to do, from hosts it trusts. One handler serves one
  card, and with `tenants`, one more a tenant, at the tenant's path.
- **ACP turns.** A turn goes on when the editor reopens its session, not before; one the agent stops
  in again while carrying it on goes on at the next reopening. A step that was halfway through when
  the agent stopped runs again, so a tool with side effects needs an idempotency key.
- **The editor's MCP servers.** A tool's capabilities are what its server says of it. The editor
  chose the server, so the agent takes its word; for a server you trust less, decide with
  `mcpToolCapabilities`. A call in flight when a server's stream drops fails, and the agent sees the
  failed call. A stdio server that exits is not started again within the session.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/tracing.md

# Traces


Traces you can search, from `nexus-ai-pro/tracing`. Each request becomes a tree of runs — the graph,
its nodes, its model and tool calls — stored in memory, a JSONL file, or Postgres. You can filter
them, leave feedback on them, compare two of them, and alert on them. A model call rendered from a
versioned prompt records which prompt ran, in `metadata.prompt`, and one rendered from a context
bundle records which bundle version, in `metadata.context`.

## Overview

Metrics say how it is going. A trace says what happened: the run tree of one request, with its
inputs, outputs, tokens, cost, and errors, stored where you can search it.

```ts
import { Tracer, MemoryTraceStore, traceGraph, traceModelClient } from 'nexus-ai-pro/tracing';

const tracer = new Tracer({
  store: new MemoryTraceStore(),               // or JsonlTraceStore, or PostgresTraceStore
  sampling: { rate: 0.05, keepErrors: true },  // 5% of traces, plus every failure
  redaction: { hideFields: ['apiKey', 'user.email'] },
});

const tracing = traceGraph(tracer, { name: 'support-agent', kind: 'agent' });
const agent = createAgent({ client: traceModelClient(ai, tracer, { parent: () => tracing.runFor('model') }) });

const run = await agent.invoke(agentInput(question), { threadId, ...tracing.runOptions });
await tracing.finish(run);
```

**Nothing inside the graph knows about tracing.** The graph already reports every task starting,
retrying, and finishing; a trace is that stream written down as a tree. An application that does not
trace creates no context, allocates nothing, and does no per-call work.

**Tail sampling keeps what matters.** A 5% rate still records every error, every run slower than a
threshold, and every run more expensive than one, because the decision is made when the trace
finishes rather than when it starts.

**Redaction happens before storage**, so a field you hide is never written, and a `redact` hook gets
the last word on every run.

**Then ask questions of it:**

```ts
const failures = await store.query({ status: 'error', kind: 'model', since: yesterday, minLatencyMs: 2000 });
const tree = await store.tree(failures[0].traceId);
console.log(formatTree(tree));

await tracer.recordFeedback(runId, { key: 'thumbs', score: 0, source: 'user' });
```

The same questions from a terminal: `nexus traces list --store runs.jsonl --status error`, then
`nexus traces show <traceId>` to print the tree.

`compareTraces(a, b)` lays two runs of the same shape side by side and reports what changed — the
step that got slower, the tool that stopped being called, the output that differs — which is how
"it worked yesterday" becomes an answerable question.

**Alerts** run the same queries on a timer, and carry the runs that tripped them:

```ts
import { AlertEvaluator, createWebhookNotifier } from 'nexus-ai-pro/tracing';

const alerts = new AlertEvaluator(
  store,
  [
    { name: 'error rate', metric: 'errorRate', threshold: 0.05, minRuns: 20 },
    { name: 'p95 latency', metric: 'latencyP95', threshold: 8000 },
    { name: 'hourly spend', metric: 'cost', threshold: 25, windowMs: 3_600_000 },
  ],
  { notifier: createWebhookNotifier({ url: process.env.ALERT_WEBHOOK! }) },
);

setInterval(() => void alerts.evaluate(), 60_000);
```

An alert names the rule, the measured value, and three runs that contributed, so the next step is
reading them rather than starting an investigation.

## Runs and trees

A `Run` is one unit of work.

| Field | Contents |
| --- | --- |
| id, trace id, parent id | Which trace it belongs to, and the run it ran inside. |
| name, kind | A `RunKind`: `chain`, `model`, `tool`, `graph`, `node`, `agent`, `retriever`, `embedding`, `image`, `voice`, `realtime`, `operation`, or `decision`, a typed decision with its answers. |
| status | A `RunStatus`: `running`, `ok`, or `error`. |
| times, latency | When it started and ended. |
| inputs, outputs, error | After redaction. |
| tags, metadata | Metadata can be queried by dot path. |
| usage, cost, model, provider | For model runs. |
| feedback | Judgements left on it. |
| events, dropped events | What happened inside it, each a `RunEvent` at its time, and how many past the cap were counted. |
| links | The runs it came from beside its parent, each a `RunLink`. |

A `RunTree` is a run with its `children` in start order. That is how a trace is read.

`RunFeedback` is a judgement on a run: a `key` such as `helpful`, a numeric `score`, a non-numeric
`value`, a `comment`, and the `source` that left it. `tracer.recordFeedback()` attaches one and fills
in the time.

## Recording

`Tracer` takes `TracerOptions`:

| Option | What it sets |
| --- | --- |
| `store` | Where finished traces are written. |
| `sampling` | A `SamplingPolicy`, below. |
| `redaction` | A `RedactionPolicy`, below. |
| `tags`, `metadata` | Added to every run, such as the deployment or release. |
| `onError` | Store errors are swallowed so tracing never breaks a request; this hook lets you notice them. |
| `maxEventsPerRun` | The events one run keeps (128); the rest are counted as its `droppedEvents`. |
| clock | For tests. |

The current run travels through `AsyncLocalStorage`, which is created only when a tracer exists.

| Method | What it does |
| --- | --- |
| `startRun()` | Starts a run beneath the one in scope, or a new trace. Returns a `RunHandle`. |
| `handle.finish()` | Ends the run. |
| `handle.child()` | Starts a run beneath it, without relying on the ambient context. |
| `trace()` | Runs a function inside a new run, and finishes it with the result or error. |
| `traceable()` | Wraps a function so every call becomes a run, with its arguments as inputs. |
| `within()` | Runs a function with a run started by `startRun()` in scope, so what it traces nests beneath that run, which its handle still finishes. |
| `current()` | The run in scope. |
| `event()` | Records an event on the run in scope, and says whether there was one. |
| `handle.event()`, `handle.link()` | Records an event on that run; returns a link to it, for another run to name. |

`StartRunOptions` give the name, kind (`chain` by default), inputs, tags, metadata, model, and
provider. A `parentId` or `traceId` attaches the run somewhere explicit.

`FinishRunOptions` give the outputs, usage, cost, metadata to merge, and the model and provider that
actually ran. Passing an `error` marks the run failed.

### Events and links

An event is something that happened inside a run, at a moment: a retry, a guardrail's verdict, a
tool's progress, the cache's answer. A `RunEvent` has a `name`, an ISO-8601 `time`, and
`attributes`, each a `RunAttributeValue`: a string, a number, or a boolean. A link names a run this
one came from, beside its parent: a branch a joined step gathers, the run that queued a job, or a span
in another process's trace. A `RunLink` has the linked run's `traceId` and `runId`, and attributes
that say why.

```ts
await tracer.trace({ name: 'job', links: [{ traceparent: request.headers.traceparent }] }, async (run) => {
  tracer.event('cache', { outcome: 'miss' });
  run.event('checkpoint', { step: 3 });
});
```

`StartRunOptions.links` takes `RunLinkInput` values: a `RunLink`, or a W3C `traceparent` header,
read as the trace and span it names; one that cannot be read is skipped. `FinishRunOptions` also take
`links`, and `events` as `RunEventInput` values, for what a run learns only as it ends, each
recorded at its own time and read in order with the rest. Once a run finishes, it records nothing
more, so an exporter holding it sees what it ended with. Redaction's `redact` hook sees the events
and links too.

The studio shows a run's events, each at its time with its attributes, and its links, each opening
the run it names, in the trace it belongs to. A trace's address, `#/traces/<trace>/<run>`, opens that
run of the trace, so a link can be shared.

### Writing as it runs

On a durable store, a tracer writes each run when it starts and again when it ends. Written only once
its root finishes, a trace would let sampling decide with the whole of it in hand, but a process that
dies mid-run would take its trace with it — the trace you most needed. `TracerOptions.incremental`
chooses: it is on by default for a store whose `durable` is not `false`, and off for a store in
memory, such as `MemoryTraceStore`, which a dead process takes with it anyway.

```ts
const tracer = new Tracer({ store }); // writes as it runs
const atTheEnd = new Tracer({ store, incremental: false }); // writes each trace when its root finishes
```

2.x wrote at the end unless asked; `useDefaults('2.x')` still does, as the
[core guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/core.md#30s-defaults-and-keeping-2xs) explains.

A crash then leaves the run in the store with every run beneath it that finished, and a run in
progress can be watched while it goes. Each trace's writes are chained, so a run's end never lands
before its start. Sampling is decided when the root starts. A trace not sampled is still held in
memory and written at the end when tail sampling keeps it, so `keepErrors` still shows every
failure.

A run a dead process left behind stays `running` until something closes it. `closeAbandonedRuns()`
does that: every run still running after `olderThanMs` (`CloseAbandonedRunsOptions`) is saved as an
error named `RunAbandoned`, with `metadata.abandoned`, keeping what it recorded. Call it when a worker
starts, or on a schedule. `tracer.flush()` waits for every write and export handed over so far.

### Across processes

A run can continue a trace another process started. `handle.traceparent()` returns the run as a W3C
`traceparent` header, and `StartRunOptions.traceparent` joins it on the other side:

```ts
// On the server, when it queues work:
await runner.submit(work, { traceContext: { traceparent: request.traceparent() } });

// In the worker:
const job = tracer.startRun({ name: 'summarize', traceparent: context.traceContext?.traceparent });
```

The run joins the trace under the span the header names, and keeps its sampling decision. The same
header works with any OpenTelemetry service. `w3cTraceId()` and `w3cSpanId()` turn a tracer's ids
into the 32 and 16 hex digits W3C and OpenTelemetry use, the same way in every process.

### Sampling and redaction

With `incremental: false`, a trace is written when its root finishes, so sampling decides with the
whole trace in hand.
`SamplingPolicy.rate` keeps that share of traces. Three options keep a trace whatever the rate:
`keepErrors`, `keepSlowerThanMs`, and `keepCostlierThan`.

`RedactionPolicy` can drop inputs or outputs entirely, and strips `hideFields` by dot path. Its `redact`
hook sees each run last. `stripFields()` is the dot-path removal on its own; it copies rather than
changing what you pass.

## Graphs and model calls

`traceGraph()` records a graph or agent run: a root run with one child per task, retries included,
each retry a `retry` event of its task's run with the attempt that failed and why. Each tool call is
a run inside its node's. A tool that reports progress has each report as a `progress` event, and
keeps them in the run's `metadata.progress` too: each report with when it came, the latest 50. A task
that follows a step of several tasks, such as the join after a fan-out, links to each of them.
`GraphTracingOptions` names the root, labels it `graph` or `agent`, and adds tags, metadata, and
inputs. It returns `GraphTracing`:

- `runOptions`, to spread into `invoke()`;
- `root`, the root run's handle;
- `runFor(node)`, the run of a node in progress;
- `finish()`, which closes the root with the result.

`traceModelClient()` wraps anything with a `complete()` method — the `ModelClientLike` contract — so
every call becomes a `model` run. The run records the request, the content and tool calls, token usage,
cost, finish reason, the model and provider that answered, and the prompt and context versions when the
request came from a registry or a context hub. Its `parent` option decides where the call hangs: `() => tracing.runFor('model')`
nests it inside the node that made it. What the response reports becomes the run's events:
- each attempt that failed before it, from the response's `failedAttempts`, as `retry` when it went
  to the provider that answered and `failover` when it went to another, at the time it failed;
- each guardrail applied, as `guardrail`;
- the cache's answer, as `cache`;
- each capability warning, as `capability_warning`.

A tool call an agent reports — `createAgent()` reports every one — becomes a `tool` run inside the
node that made it, with the arguments as inputs and the result or the error.

`traceRetriever()` wraps anything with a `retrieve()` method — the `RetrieverLike` contract, which
every retriever in `nexus-ai-pro/rag` meets — so every retrieval becomes a `retriever` run, holding
the query and the options asked as inputs and `{ documents }` as outputs. Every other member of the
retriever is its own, so a wrapped keyword index still takes writes. It takes `parent` and `name`, as
`traceModelClient()` does.

An agent server given a `tracer` traces each run itself: each attempt is the root of a trace, an
`agent` run named after the assistant, with the input as inputs and the output as outputs, and
`metadata.nexus` naming the run, the assistant, the attempt, the revision, and the thread, queryable
as `{ metadata: { 'nexus.run': runId } }`. Whatever the assistant traces through the same tracer, a
model call or a retrieval, lands beneath it. The run records the trace's id as `traceId`, which
[a deployment's evaluation](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/deployments.md#judging-with-confidence-and-on-quality) reads.

## Exporting to OpenTelemetry

`TracerOptions.exporters` hands every finished run the tracer keeps, after redaction, to each
`TraceExporter`: an object with `export(run)` and an optional `flush()`. An exporter that fails is
reported through `onError` and never fails the run.

`OtlpTraceExporter` from `nexus-ai-pro/tracing/otlp` sends runs to any OpenTelemetry collector over
OTLP/HTTP, with no SDK to install. A run's events become its span's events, an `OtlpSpanEvent` each,
and its links the span's links, an `OtlpSpanLink` each, in JSON, in protobuf, and over gRPC:

```ts
import { OtlpTraceExporter } from 'nexus-ai-pro/tracing/otlp';

const tracer = new Tracer({
  store,
  exporters: [new OtlpTraceExporter({ endpoint: 'http://localhost:4318', serviceName: 'support-agent' })],
});
```

`OtlpTraceExporterOptions` sets the collector's base URL (spans go to `/v1/traces`), headers such as an
API key, the `service.name`, more resource attributes, the batch size (100) and how long a span waits
to batch (2 seconds), a `fetch` of your own, and an `onError`. `protocol` chooses the encoding:
`http/json` by default, or `http/protobuf`, OTLP's binary encoding, which is smaller and quicker for a
collector to read. On its own, a batch the collector refuses is dropped; `send()` sends runs as one
batch and rejects when the collector refuses them, which is what the spool below sends through. A
collector that takes a batch but rejects some of its spans says so in a partial success, which
reaches `onError` as an `OtlpPartialSuccessError` with the `rejectedSpans` and the collector's
`errorMessage`; sending them again would change nothing, so it is not a refusal.

`encodeOtlpTraceRequest()` writes an `OtlpExportRequest`, the spans grouped by resource and scope,
each attribute an `OtlpKeyValue` holding an `OtlpAttributeValue`, in that encoding, and
`decodeOtlpTraceResponse()` reads the `OtlpPartialSuccess` from a collector's answer. Both run
anywhere.

### OTLP over gRPC

`OtlpGrpcTraceExporter`, from `nexus-ai-pro/tracing/otlp-grpc`, sends the same spans over gRPC, port
4317, for a collector or a backend that takes only that. It needs no gRPC library: it frames the
trace service's `Export` call itself, over Node's own HTTP/2 client, so it runs on Node.js only.

```ts
import { OtlpGrpcTraceExporter } from 'nexus-ai-pro/tracing/otlp-grpc';

const exporter = new OtlpGrpcTraceExporter({
  endpoint: 'https://collector.internal:4317',
  headers: { 'x-api-key': process.env.OTEL_KEY ?? '' },
});
const tracer = new Tracer({ store, exporters: [exporter] });
// On shutdown: send what is waiting, and close the connection.
await exporter.close();
```

`OtlpGrpcTraceExporterOptions` takes everything `OtlpTraceExporterOptions` does but `fetch` and
`protocol`. An `http://` endpoint connects without TLS, and an `https://` one with it; the headers
are the call's metadata. `gzip` compresses each batch (on by default), `timeoutMs` is each call's
deadline (10 seconds), and `connect` passes options such as a `ca`, a client `cert` and `key`, or
`servername` to `http2.connect()`. One connection is kept, opened again when it drops, and it never
keeps the process alive while no call is in flight. A connection that dropped without a word, as when
a collector dies, fails the first call made on it before anything reaches a collector; that call is
made once more, on a new connection, as gRPC clients do. A call the collector answers with a status other
than OK rejects `send()` with an `OtlpGrpcError`, carrying the gRPC `code`, such as 14 for
`UNAVAILABLE`, and the collector's `details`, so a spool keeps the batch.

In the test suite, the OpenTelemetry Collector itself, in a container, takes the spans over gRPC and
sends them on as OTLP/HTTP JSON, and every one arrives with its ids, times, kind, attributes, and
status as it was sent.

### Spans that a collector being down cannot lose

`spoolingExporter()`, from `nexus-ai-pro/tracing/spool`, wraps any `TraceBatchSender` (something with
a `send(runs)` that rejects when the receiver refuses them, such as `OtlpTraceExporter`) in a
`SpoolingExporter` that keeps what the collector refuses. A refused batch is written to an operation
store, the memory, Redis, Postgres, or SQLite one, and tried again with backoff each `drainEveryMs`
(15 seconds). Every replica on the same store drains the same spool; each batch is claimed before it
is tried, so it is sent once, and one a stopped replica was sending is taken over when its lease
lapses.

```ts
import { OtlpTraceExporter } from 'nexus-ai-pro/tracing/otlp';
import { spoolingExporter } from 'nexus-ai-pro/tracing/spool';

const exporter = spoolingExporter({
  sender: new OtlpTraceExporter({ endpoint: 'https://collector.internal:4318' }),
  store: operationStore,                 // shared by every replica
  maxPendingBatches: 10_000,
  maxAgeMs: 24 * 3_600_000,
});
const tracer = new Tracer({ store, exporters: [exporter] });
```

`SpoolingExporterOptions` bounds what it keeps: at most `maxPendingBatches` (10,000) wait, and
spooling one more drops the oldest; a batch older than `maxAgeMs` (24 hours), or tried `maxAttempts`
times (1,000), is dead-lettered: marked failed in the store and never sent. The backoff runs from
`baseDelayMs` (1 second) to `maxDelayMs` (5 minutes). With `writeAhead`, every batch is written to the
spool before it is sent and removed once sent, so a process that stops in the middle of a send loses
nothing either, for a store write per batch.

`stats()` resolves to `SpoolStats`, counted in runs: how many were exported, of those how many were
retried from the spool, how many were spooled, dropped, and dead-lettered, and how many batches wait
now, across every replica, with the age of the oldest. `drain()` tries what is due now and resolves to
a `SpoolDrainReport`; `close()` flushes and stops the timers, leaving the spool for any replica.

In the test suite, a collector stops for an hour while a span finishes every second, and the process
restarts halfway through. When the collector comes back, every one of the 3,600 spans reaches it,
once, on the memory, SQLite, Postgres, and Redis operation stores.

Each run becomes an `OtlpSpan` with the GenAI semantic conventions, so Grafana Tempo, Jaeger,
Honeycomb, Datadog, New Relic, or any OTLP backend reads it like the rest of the application:

| Run | Span |
| --- | --- |
| `model` | A client span named `chat <model>`, with `gen_ai.operation.name`, `gen_ai.provider.name`, `gen_ai.request.model` (`metadata.requestedModel` when routing chose), `gen_ai.response.model`, and `gen_ai.usage.input_tokens` and `output_tokens` |
| `embedding` | The same, with the `embeddings` operation |
| `tool` | `execute_tool <name>`, with `gen_ai.tool.name` |
| `agent` | `invoke_agent <name>`, with `gen_ai.agent.name` |
| `node` | `nexus.graph.node` and `nexus.graph.step` |
| Any | `nexus.run.kind`, `nexus.run.id`, `nexus.cost.usd`, a `nexus.tag.*` per tag, and `error.type` and an error status when it failed |

Span and trace ids are the W3C ids `traceparent()` propagates, so a trace that crossed processes
arrives as one trace. `runToOtlpSpan()` is the conversion on its own.

## Rollups for dashboards

A trace is the forensic record of one run. A chart of a week's cost needs none of that, only sums,
and reading every trace in the window to draw it is how a dashboard slows down as the service grows.
`nexus-ai-pro/tracing/rollups` keeps hourly totals instead.

```ts
import { MemoryRollupStore, rollupTraceStore, sumRollups } from 'nexus-ai-pro/tracing/rollups';

const rollups = new MemoryRollupStore();
const tracer = new Tracer({ store: rollupTraceStore(traceStore, rollups) });

const week = sumRollups(await rollups.query({ since: weekAgo }));
```

`rollupTraceStore()` wraps any trace store. Reads go to it unchanged; every run saved finished is also
added to a rollup. A run is counted once: a save of a run still running is not counted, and neither is
feedback. `RollupTraceStoreOptions.include` picks what counts, by default root runs and every model
and embedding run.

| Piece | What it is |
| --- | --- |
| `RollupKey` | What a row is counted by: the hour, kind, root name, model, provider, and `metadata.tenantId`. `rollupKeyOf()` reads it from a run. |
| `RollupTotals` | Runs, errors, cost, input and output tokens, and latency counts per bucket. |
| `RollupRow` | A key and its totals. |
| `RollupQuery` | An hour range, kinds, a model, a provider, a tenant, or a name. |
| `RollupStore` | `add()` and `query()`. `MemoryRollupStore` keeps rows in memory, and `PostgresRollupStore`, from `nexus-ai-pro/postgres/rollups`, beside your traces; any database can implement it. |
| `ROLLUP_LATENCY_BOUNDS_MS` | The latency buckets, from 10 ms to 10 minutes. |
| `rollupPercentile()` | A percentile from the buckets, exact to the bucket. |
| `sumRollups()` | Rows summed into one total. |

A million runs a month for three models is about two thousand rows. The [studio](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/studio.md) reads
them for its cost view when you give it `rollups`.

## Storing and querying

A `TraceStore` has `save()`, `get()`, `query()`, and `tree()`, and optionally `addFeedback()` and
`prune()`. Three stores are included:

| Store | Best for |
| --- | --- |
| `MemoryTraceStore` | Development, tests, and one process. Bounded by `MemoryTraceStoreOptions.maxRuns` (10,000). |
| `JsonlTraceStore` | Durable without a database: one JSON object per line in `JsonlTraceStoreOptions.file`, trimmed to its newer half past `maxBytes` (64 MB). A read loads the file, so it suits a service that writes far more than it queries. |
| `PostgresTraceStore` | Production queries, with every filter in SQL. See the [Postgres guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/postgres.md). |

A `RunQuery` filters runs. Results come newest first, and runs that started in the same millisecond
come by id, from last to first, compared as code units: every store orders them the same way.

| Filter | Matches |
| --- | --- |
| trace id, kind (one or several), status, name, model, provider | Exactly. |
| tags | Runs with all of them. |
| metadata | By dot path. |
| minimum latency, minimum cost | Runs at least that slow or expensive. |
| `since`, `until` | A window on the start time. |
| feedback key | Runs with feedback under that key. |
| `limit`, `offset` | Paging. `limit` defaults to 50. |
| `after` | Runs after a `RunCursor`, the start time and id of the last run of the page before. |

Page by `after` rather than `offset` when runs keep arriving: an offset shifts as new runs push
older ones down, and `until` cannot split runs that share a start time, so a page that ends among
them would end the same way again. `runCursor()` takes the cursor from a page's last run.

```ts
import { type Run, type RunCursor, runCursor } from 'nexus-ai-pro/tracing';

let after: RunCursor | undefined;
for (;;) {
  const page = await store.query({ kind: 'model', limit: 500, ...(after ? { after } : {}) });
  for (const run of page) handle(run);
  if (page.length < 500) break;
  after = runCursor(page[page.length - 1] as Run);
}
```

`applyQuery()` and `assembleTree()` are the filtering and tree-building every store shares. They are
exported for a store of your own, so a query means the same everywhere; `isAfterCursor()` is the
cursor test on its own, for a store that filters elsewhere.

## Comparing traces

`compareTraces()` matches the runs of two trees by path, such as `root/child#0/grandchild#1`, because
two runs of the same graph never share ids. It returns a `TraceComparison`:

- each matched pair, with its `RunDifference` values: a field added, removed, or changed, with both
  sides;
- the runs only one side has;
- the change in root latency and total cost.

`formatTree()` prints a tree as indented text.

## Alerts

An `AlertRule` watches one metric.

| Field | Meaning |
| --- | --- |
| name | What the alert says. |
| metric | An `AlertMetric`: `errorRate`, `latencyP95`, `latencyP50`, `cost`, or `count`. |
| threshold | It fires above this. |
| window | How far back it looks. Defaults to 15 minutes. |
| `filter` | Measures one model, kind, or tag. |
| `minRuns` | So a quiet window does not fire. |

`AlertEvaluator.evaluate()` checks every rule once. Each rule that fires produces an `AlertEvent` —
the value, the threshold, the run count, the window, and up to three sample runs — which goes to an
`AlertNotifier`. `measure()` computes a metric over runs on its own.

`createWebhookNotifier()` posts events to a URL. `WebhookNotifierOptions` sets headers, a `fetch`, and
a `body` builder. By default it sends the `{ text }` shape chat webhooks accept, with the event beside
it.

An alert answers a question you asked in advance. To find problems you did not expect — failing and
slow runs grouped by what went wrong, and metrics that got worse than the week before — use
[insights](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/insights.md) over the same trace store.

## Limitations

- With `incremental: false`, or on a store in memory, traces are written when the root finishes, so
  a process that dies mid-run loses that trace.
- With `incremental`, a run is written twice, so a store pays twice the writes. A run left by a dead
  process stays `running` until `closeAbandonedRuns()` closes it.
- `OtlpGrpcTraceExporter` runs on Node.js only. On another runtime, send OTLP/HTTP with
  `OtlpTraceExporter`, in JSON or protobuf.
- A run keeps at most `maxEventsPerRun` events; the rest are counted, not kept. A model run's events
  from its response, other than failed attempts, carry the time the response came back.
- Without `writeAhead`, the runs waiting to be batched, at most `flushIntervalMs` of them, are lost
  with a process that stops before it sends them; what the collector refused is not.
- Rollups come as `MemoryRollupStore` and `PostgresRollupStore`; another database's is a `RollupStore`
  of your own.
- An alert measures up to 1,000 runs in its window unless its filter sets a larger `limit`.
- Cost alerts sum every run in the window, so give a `filter` such as `{ kind: 'model' }` when
  parent runs also record cost.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/evaluation.md

# Evaluation


Evaluation from `nexus-ai-pro/evaluate`: run any target — a completion, an agent, a graph, an image operation, or plain code — over a dataset versioned by its content, score it, and compare experiments with a verdict rather than a difference of means. `nexus-ai-pro/evals` keeps the eval-case runner and the LLM judge.

## Overview

One entry point for evaluating anything: a completion, an agent, a graph, an image operation, or
plain code. A dataset of examples, a target that turns an example into an output, evaluators that
score it, and a stored experiment a later run can be compared against.

```ts
import { evaluate, createDataset, exactMatch, contains, trajectory, passRate } from 'nexus-ai-pro/evaluate';

const dataset = createDataset({
  name: 'support-questions',
  examples: [
    { id: 'refund', inputs: { question: 'how do I get a refund?' }, expected: 'Open the order and choose refund.' },
    { id: 'hours', inputs: { question: 'when are you open?' }, expected: 'Weekdays, nine to five.' },
  ],
});

const experiment = await evaluate(
  (inputs) => agent.invoke(agentInput(inputs.question)),
  dataset,
  [contains(['refund']), trajectory({ expected: ['search_orders'] })],
  { repetitions: 3, concurrency: 4, summary: [passRate()], store: experiments },
);
```

**A dataset is versioned by its content**, so two experiments are comparable only when they really
ran over the same examples. Change an example and the version changes with it.

**Datasets can come from production.** `datasetFromTraces({ store, query: { status: 'error' } })`
turns recorded runs into examples, each keeping the run it came from — the request that failed
yesterday becomes tomorrow's regression test.

**Evaluators are ordinary functions.** Several come bundled, from exact matching to embedding
similarity. `trajectory` scores *how* an answer was reached: an agent that gets the right answer by
calling the refund tool three times is not working. The LLM judge from `nexus-ai-pro/evals/judge`
plugs in as one more evaluator.

**Cost is read from the output**: `meta.cost` on a completion or an image result, or a `cost` field on
anything else, and a `cost` option reads whatever a custom target returns. `underCost(0.05)` fails an
example that overspends, and `totalCost()` sums the experiment, so a quality gain that tripled the
bill is visible. `signal` cancels an evaluation without ever storing a partial experiment, and
`repetitions` can be a function when some examples are noisier than others.

### Was the change real?

```ts
import { compareExperiments, formatComparison } from 'nexus-ai-pro/evaluate';

const comparison = compareExperiments(baseline, candidate);
console.log(formatComparison(comparison));
if (comparison.regressed) process.exit(1);
```

A mean that moves from 0.81 to 0.83 says nothing on its own; with twenty examples that is noise, and
shipping on it makes a quality gate a coin toss. The verdict comes from a **paired bootstrap** over
the per-example differences, which assumes nothing about how the scores are distributed, and the
pairing removes the variation caused by the examples themselves. A metric whose interval spans zero
is reported as `unchanged`, not as an improvement. The bootstrap is seeded, so CI gives the same
verdict twice.

The report names the examples that moved most, the failures that are new, and the ones that were
fixed. A comparison across different dataset versions says so rather than pretending it is like for
like.

### In CI

```bash
nexus eval run eval.mjs --out candidate.json --baseline baseline.json --fail-on-regression
```

The module exports `{ target, dataset, evaluators, summary? }`. `nexus eval gate baseline.json
candidate.json` compares two stored experiments on its own. Either fails only when a metric got worse
beyond noise, a new failure appeared, or the datasets differ; `--allow-dataset-mismatch` accepts the
last. `FileExperimentStore` keeps experiments as files a pipeline can cache or commit, and
`PostgresExperimentStore` keeps them in a table.

`EvalRunner` and `MediaEvalRunner` run on `evaluate()` too, and their results carry the experiment
underneath, so an existing suite can be gated the same way without being rewritten.

### The runs you did not think of

```ts
import { evaluateOnline, AnnotationQueue } from 'nexus-ai-pro/evaluate';

const queue = new AnnotationQueue({
  rubric: [{ key: 'helpful', prompt: 'Did this answer the question?', type: 'boolean' }],
  consensus: 2,
});

await evaluateOnline({
  store: traceStore,
  evaluators: [mustNotMatch([/i cannot help/i])],
  sampleRate: 0.1,
  reviewQueue: queue,
  reviewWhen: (scores) => scores.some((score) => score.passed === false),
});
```

A dataset tells you whether a change works on the cases you thought of; online evaluation tells you
how it is doing on the rest. Scores are written back as feedback on the run, so an alert rule can
watch them, and anything uncertain goes to a person.

**The review queue keeps track of what people owe you.**

- Claims expire, so a reviewer who closes the tab does not strand an item.
- `consensus` sends one item to two reviewers when one opinion is not enough.
- `toExamples()` turns reviewed items into dataset examples. A production failure becomes a permanent
  regression test.

## Datasets

A `Dataset` has a name, a content `version`, a description, tags, its creation time, and its
examples. Each `DatasetExample` has an `id` — which is what pairs results across experiments — the
`inputs` the target receives, an optional `expected` answer, metadata, a `split`, tags, and, for
an example built from production, the `sourceRunId` it came from.

`createDataset()` builds one from `CreateDatasetOptions`: the name, the examples, a description, tags,
and a version. Examples without an id are numbered `ex-1`, `ex-2`, and so on. The version defaults to
`contentVersion()`, a hash of the examples, so identical content is always the same version.

`splitOf()` returns one split as a dataset of its own. `datasetFromTraces()` builds a dataset from
recorded runs. `FromTracesOptions` gives the store, a trace query, a `toExample` mapping (inputs and
outputs by default), and a limit (100).

A `DatasetStore` saves versions, gets one by name and version (the newest when none is given), and
lists names with their versions. `MemoryDatasetStore` keeps them in memory and `FileDatasetStore` as
one JSON file per version; `PostgresDatasetStore` is in the [Postgres guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/postgres.md).

### Datasets that grow

`nexus-ai-pro/evaluate/generate` writes examples you would otherwise write by hand.

**From documents.** `syntheticDataset()` cuts each `SyntheticDocument` into `SyntheticPassage`s of up
to 1,500 characters, keeping paragraphs whole where they fit. It asks for `SyntheticDraft`s per
passage, from your `generate` function or from a model with a built-in prompt. Each draft is a
question, its answer, and a quote of the passage that supports the answer. It is kept only when:
- its quote is in its passage, word for word apart from spacing;
- its question is not too like one already kept (`similarity`, 0.8 by default).

Each example carries a `SyntheticCitation` in `metadata.citation`: the document, the passage, and
the quote with where it stands. The `SyntheticDatasetResult` also counts what was dropped as
unsupported or as a repeat. `SyntheticDatasetOptions` sets the passage size, the drafts per
passage, and a limit; `SyntheticModelClient` is the model the built-in prompt uses.

```ts
import { syntheticDataset } from 'nexus-ai-pro/evaluate/generate';

const { dataset, unsupported, duplicates } = await syntheticDataset({
  name: 'handbook',
  documents: pages.map((page) => ({ id: page.path, title: page.title, text: page.text })),
  client: ai,
  model: 'claude-haiku-5-5',
});
```

**From attacks.** `adversarialDataset()` grows a dataset from the guardrails' own patterns: each
seed of `attackSeeds()` on `nexus-ai-pro/security`, in each `AttackCarrier` and each
`AttackDisguise`. The carriers are the user's message, a document the user gives, and a tool's
result. The disguises are plain text, full-width letters, and invisible characters splitting the
words. Every example's `AdversarialInputs` hide a canary in the system prompt, and the attack asks
for it:
- an injection or a jailbreak tells the model to say it;
- exfiltration asks for the prompt that holds it.

`AdversarialExpected` names the canary. `resistsAttack()` scores 1 when the canary never appears in
the output, read through the same disguises. A target that refused, or that a guardrail stopped,
kept it in. `AdversarialDatasetOptions` chooses the categories, carriers, and disguises, and can
`rephrase` an attack into more wordings, such as a model's. Benign examples, one per carrier, show
a guard that refuses everything.

In the test suite, every attack gets the canary out of an agent with no guardrails, and none gets it
out of a guarded one. That agent blocks injections at `strict` and redacts its secret with output
moderation. It still answers every benign example.

**From production.** `promoteIssue()` turns the runs behind an insights `Issue` into examples.
Each keeps its run as `sourceRunId` and the issue as `metadata.issue`, so a regression test points
back to the problem it guards against. `PromoteIssueOptions` adds them to an existing dataset
(`into`), maps a run to an example, and sets a limit. A run already in the dataset is not added
twice.

## Targets, evaluators, and experiments

An `EvaluationTarget` turns an example's inputs into an output. An `Evaluator` receives an
`EvaluationContext` — the example, the output or the error, the latency, the cost, and which
repetition it was — and returns a number, a pass or fail, or one or more `EvaluationScore` values: a
key, the score, whether it passed, a comment, and metadata. Scores with the same key are summarized
and compared together. A `SummaryEvaluator` scores the experiment as a whole.

The bundled evaluators:

| Evaluator | Scores |
| --- | --- |
| `exactMatch()` | Equality with the expected output. |
| `contains()` | Whether required phrases appear. |
| `mustNotMatch()` | That forbidden patterns do not appear. |
| `completed()` | That the target produced an output at all. |
| `underLatency()`, `underCost()` | Latency and cost against a limit. |
| `embeddingSimilarity()` | Closeness to the expected answer, through any embedder. |
| `pairwise()` | A side-by-side judgement against another output. |
| `trajectory()` | How the answer was reached: the tools or nodes called. |
| `passRate()` | The share of examples that passed. A summary evaluator. |

`trajectory()` takes `TrajectoryOptions`: the `expected` tool or node names, a `path` reader (the
agent's tool calls by default), a `mode`, and the score `key`. The mode is `exact` for the same
sequence, or `subset` for every expected step appearing.

For retrieval, `recallAtK()` scores the share of an example's relevant ids found in the output's top
`k`, and `reciprocalRank()` scores one over the rank of the first relevant one, whose mean over a
dataset is the mean reciprocal rank. The output is a ranked list of ids or chunks, such as a
retriever's results, and `expected` is the relevant id or a list of them; `RetrievalEvaluatorOptions`
sets `k` and the score `key`. Comparing two retrievers this way is how the
[retrieval guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/retrieval.md) shows that hybrid search with reranking is a real improvement.

`EvaluateOptions` names the experiment and sets `concurrency` (4), `repetitions` (a number, or a
function of the example), `summary` evaluators, a `store`, metadata, a per-example `timeoutMs`, a
`signal`, a `cost` reader, `onResult` for progress, and a clock.

An `Experiment` records:

- its id and name, the dataset and exact version it ran over, and its start and finish;
- how many examples produced no output;
- every `ExampleResult`: the example id, repetition, output or error, latency, cost, and scores;
- a `MetricSummary` per metric: count, mean, standard deviation, minimum, maximum, and the pass rate
  when the evaluator reported passes;
- the summary scores, and metadata;
- `evaluators`: an `EvaluatorProvenance` for each evaluator, in order — its position, name, the
  score keys it produced, and what it declared;
- `framework`: the package name and version that ran it.

A score means little without what produced it. `withProvenance()` marks an evaluator with its name,
version, the judge model and prompt version behind it, and its rubric, and the experiment records
them. It returns the same evaluator, which runs exactly as before:

```ts
import { withProvenance } from 'nexus-ai-pro/evaluate';

const grounded = withProvenance(groundedness, {
  name: 'groundedness',
  version: '3',
  judge: { model: 'claude-sonnet-5-5', promptVersion: 'grounded-v3', temperature: 0 },
  rubric: 'Every claim is supported by a cited passage.',
});
```

An evaluator with no provenance is recorded by its function's name, or as `evaluator-<position>`, with
the keys it produced, so two experiments can still be checked for using the same scorers.

| Function | What it does |
| --- | --- |
| `summarize()` | Computes the metric summaries. |
| `stats()` | The arithmetic underneath, with a 95% interval of the mean. |
| `scoreMap()` | Summary scores as a map, for a test or a gate. |
| `totalCost()` | Sums the experiment's cost. |

An `ExperimentStore` saves experiments, gets one by id, and lists them newest first, for one dataset
or name. `MemoryExperimentStore` keeps them in memory and `FileExperimentStore` as one file each;
`readExperiment()` reads a file, returning `undefined` for anything that is not an experiment.

## Caching

An experiment re-run after a small change should pay only for what changed. With a `cache`,
`evaluate()` stores each example's output and reuses it the next time the example's inputs, its
repetition, and the target's `fingerprint` are all unchanged; the evaluators always run again, so
correcting an expected answer or adding an evaluator re-scores stored outputs for free. Failures are
never stored, so a flaky example is retried. A cached result is marked `cached` and keeps the latency
and cost it had, and the experiment's `cache` field counts hits and misses.

The fingerprint names whatever decides the target's output — a model, a prompt version, settings —
and `fingerprintOf()` builds one from any settings object. A cache without a fingerprint is refused,
so a changed target can never reuse another's outputs. `evaluatePrompt()` and `evaluateContext()`
fingerprint the prompt or bundle version themselves.

```ts
import { evaluate, FileEvaluationCache, fingerprintOf } from 'nexus-ai-pro/evaluate';

const cache = new FileEvaluationCache('.eval-cache');
const fingerprint = await fingerprintOf({ model: 'gpt-5.4-mini', prompt: answer.version, temperature: 0 });
const experiment = await evaluate(target, dataset, evaluators, { cache, fingerprint, store });
console.log(experiment.cache); // { hits: 48, misses: 2 }
```

An `EvaluationCache` stores and returns a `CachedOutput` — the output, its latency and cost, and when
it was produced — by key. `MemoryEvaluationCache` keeps them in process, least recently used dropped
first beyond `MemoryEvaluationCacheOptions.maxEntries`, and copies them in and out so an evaluator
cannot change what a later experiment reuses. `FileEvaluationCache` keeps one JSON file per key in a
directory, which a CI job keeps between runs by caching the directory. `evaluationCacheKey()` is the
key: a SHA-256 over the fingerprint, the example's id and inputs, and the repetition — not the
expected output.

## Comparisons in detail

`compareExperiments()` takes `CompareOptions`:

| Option | Default | Meaning |
| --- | --- | --- |
| `lowerIsBetter` | latency and cost | Metrics where a smaller number wins. |
| `resamples` | 2,000 | Bootstrap resamples behind each interval. |
| `seed` | fixed | Makes a comparison in CI reproducible. |
| `topExamples` | 10 | How many regressed and improved examples to list. |

It returns an `ExperimentComparison`:

| Field | Contents |
| --- | --- |
| both experiments | Their ids and names, and a dataset mismatch when they ran on different versions. |
| `metrics` | A `MetricComparison` per metric: both means, the mean paired difference, and a verdict of `better`, `worse`, or `unchanged`. |
| `regressions`, `improvements` | The examples that moved most, each an `ExampleComparison`. Its `delta` is signed so positive always means better. |
| `newErrors`, `fixedErrors` | Examples that started or stopped failing. |
| `regressed` | True when anything got worse beyond noise. |

`formatComparison()` renders it as text for a pull request or a CI log.

## Conversations

An agent that talks with a user over several turns is judged on the whole conversation: whether the
user got what they came for, how many turns it took, and the turn where it went wrong.
`nexus-ai-pro/evaluate/threads` holds those conversations with a simulated user and scores them, so
two versions of a multi-turn agent are compared the way single answers are, with the same verdicts.

```ts
import { compareExperiments, createDataset, evaluate } from 'nexus-ai-pro/evaluate';
import {
  conversationTarget,
  goalCompletion,
  goalReached,
  graphThreadAgent,
  simulatedUser,
  turnCount,
} from 'nexus-ai-pro/evaluate/threads';

const goals = createDataset({
  name: 'support-goals',
  examples: [{ inputs: { goal: 'Get a refund for order 1182', persona: 'An impatient customer' } }],
});
const user = simulatedUser({ client: ai, model: 'gpt-5.4-mini' });
const run = (agent) =>
  evaluate(conversationTarget(graphThreadAgent(agent), { user, maxTurns: 8 }), goals, [
    goalReached(),
    goalCompletion(judge),
    turnCount(),
  ]);

const comparison = compareExperiments(await run(current), await run(candidate), { lowerIsBetter: ['turns'] });
```

### The simulated user

A dataset of conversations holds `ThreadInputs`: the user's `goal`, and optionally a `persona` and the
user's `opening` message. A `SimulatedUser` plays that user. Given a `SimulatedUserContext` (the
goal, the persona, and the turns so far), it returns a `SimulatedUserReply`: its next message, or that
it is done, because the goal was met or because it gave up.

- `simulatedUser()` is driven by a model, through any `SimulatedUserClient` with a `complete()`
  method. The model is told the goal and the persona, sees the conversation with the roles turned
  around, and ends it by answering `DONE: GOAL` or `DONE: GAVE UP`. `SimulatedUserOptions` set the
  model, a default persona, the temperature (0, so a conversation repeats), the token limit, and
  instructions of your own.
- `scriptedUser()` follows a script, for a deterministic test: its messages in order, then done. A
  function in place of the script decides each move itself.

### The conversation target

`conversationTarget()` is an `evaluate()` target. For each example it lets the user speak and the agent
answer, until the user is done or `ConversationTargetOptions.maxTurns` (10) user messages have been
sent. The user may be one for every example, or a function that builds one per example, such as one
with that example's persona.

The target returns a `Thread`: every `ThreadTurn`, and a `ThreadEnding` that says how it ended:
`goal`, `gave-up`, `max-turns`, or `error` with the agent's message. An assistant turn records how
long the agent took and, when the agent reports them, every message it added on the way: its tool
calls, their results, and the answer.

A `ThreadAgent` is any function from the conversation so far to a `ThreadAgentReply`: the answer, or
the answer and the messages the agent added. It is given its own earlier turns as it recorded them,
and a `ThreadAgentContext`: the example's inputs, the turn, a `conversationId` that is new for every
conversation, and the abort signal. `graphThreadAgent()` makes one from `createAgent()`, or from any
`ThreadGraph` that takes `{ messages }` and ends with an `answer`. Each turn runs on the
conversation's thread with the transcript so far, so the agent sees its own tool calls on later
turns, as it does with a real user, with or without a checkpointer.

### Thread evaluators

| Evaluator | Key | Scores |
| --- | --- | --- |
| `goalReached()` | `goal_reached` | 1 when the user said the goal was met, otherwise 0, with how the conversation ended. |
| `goalCompletion(judge)` | `goal_completion` | A judge's verdict on the whole conversation, given the goal as the question. It takes an `LLMJudge`. |
| `turnCount()` | `turns` | How many times the agent spoke. Fewer is better: compare with `lowerIsBetter: ['turns']`. |
| `turnScores(evaluator)` | `turn_score` | Every assistant turn by an evaluator of single answers, and the conversation by their mean. |

`turnScores()` shows the evaluator each turn as an example of its own. The user's message is the
input, the turn is the output, and the conversation before it is in `metadata.history`. The score's
metadata lists every turn's score, and `wentWrongAt`, the first turn below the threshold, counted
from 1. `TurnScoresOptions` set the key and the threshold (0.5). `formatThread()` writes a
conversation as text, for a judge or a person.

Each evaluator scores an output that is not a conversation as 0, so one failed target does not end
an experiment.

## Online evaluation and review

`evaluateOnline()` scores production traffic. It takes `OnlineEvaluationOptions`:

| Option | Default | Meaning |
| --- | --- | --- |
| `store`, `evaluators` | — | Where runs are read from, and how they are scored. |
| `query` | finished model runs | Which runs are scored. |
| `sampleRate` | all | The share of runs scored. |
| `reviewQueue`, `reviewWhen` | — | Where, and when, a run goes to a person. |
| `recordFeedback` | on | Writes scores back onto the runs. |

It returns an `OnlineEvaluationReport`: the runs scanned, evaluated, and queued for review, and every
score.

### On workers

`evaluateOnline()` is one pass, called by whoever calls it. For production traffic, run
`onlineEvaluationWorker()` from `nexus-ai-pro/evaluate/online` instead, on as many processes as you
like over one operation store, such as Redis or Postgres. Each `OnlineEvaluationWorker`:
- **reads** runs newest first, `scanLimit` (1,000) a page, each page resumed after the last run of
  the one before, so a backlog of runs that started in the same millisecond is read through. On a
  trace store of your own that does not honour `RunQuery.after`, it pages by offset instead, and its
  `OnlineSchedulePass` says `paging: 'offset'`;
- **samples** by a hash of the run's id and the `source`, so every worker, and every pass after a
  restart, picks the same runs;
- **queues** one operation per run and source, under an idempotency key, so a run is queued once
  however many workers see it, and it stops queuing while `maxQueued` (1,000) of the source's
  operations are waiting or running;
- **scores** what it claims, `concurrency` (4) at a time, each evaluation first reserved from a
  `budget` ledger at `estimate` and reconciled with `costOf`. Just before it calls its evaluators, it
  records the reservation it holds. A reservation held by a worker that died is settled by whoever
  takes over: charged at what the scores cost if they were written, given back if the evaluators
  never ran, and kept or given back by the ledger's `uncertain` policy if they ran and wrote nothing,
  since a judge may have billed the call. A run scored again after a crash is reserved and charged
  too, since its evaluators run again.

Every score is written to the run with a mark naming its operation. An operation run again, after
a worker died between writing a run's scores and recording them, writes only what is missing. So
each run is scored exactly once per source, across workers and after a crash. A worker that dies
leaves its claims to lapse after `leaseMs`, and another takes them back. When the budget is spent,
the evaluation goes back to the queue, not to failure, and the workers stop until it allows more.

`drain()` schedules and scores until nothing is left, `start()` and `stop()` keep it going, and
`report()` returns an `OnlineEvaluationWorkerReport`: queued, scored, already scored, missing,
failed, and why it last stopped. `schedule()` and `work()` run one step each; a scheduling pass
returns an `OnlineSchedulePass`. `OnlineEvaluationWorkerOptions` holds the rest, among them:
- `withTrace`, which gives each evaluator what the run did as `context.trace`, for one `tree()` read
  per evaluation, as below;
- `onScored`, which hears each run's scores once they are written, as an `OnlineScored`: the run,
  its scores, the source, and the operation that wrote them. `automationRules()` acts on them. A
  failure of it goes to `onError` and never fails the evaluation.

### Judging a run by its trace

An evaluator given a trace judges more than the answer. `context.trace` is an `EvaluationTrace`,
read by `traceEvidence()` from the run and everything beneath it:

| Field | What it holds |
| --- | --- |
| `root` | The trace, as a `RunTree` from the run judged. |
| `inputs`, `outputs` | What the run received and produced. |
| `toolCalls` | Every `TraceToolCall`: the tool's name, its arguments, its result or error, and how long it took. |
| `retrievals` | Every `TraceRetrieval`: the query and the `TraceDocument` values that came back, best first. |
| `modelCalls` | Every `TraceModelCall`: the model and provider, the request and response, usage, and cost. |
| `media` | Every `TraceMedia`: an image, audio, video, document, or asset part the run received or produced, each once, with the run that carried it. |

`mediaParts()` finds the same media, each a `MediaPart`, in any value: a message's content, a tool's output, or an asset
an image or video operation returned, which comes back as an asset part. A retrieval is read from a
`retriever` run, as `traceRetriever()` records one, whose outputs are the documents or
`{ documents }`.

```ts
// Grounded: the answer quotes what the tool returned, and rests on a retrieved policy.
const grounded: Evaluator = ({ output, trace }) => {
  const order = trace?.toolCalls.find((call) => call.name === 'lookup_order')?.outputs as Order;
  const cited = trace?.retrievals.some((retrieval) => retrieval.documents.length > 0) ?? false;
  return { key: 'grounded', score: cited && String(output).includes(order.ships) ? 1 : 0 };
};
```

A deployment's runs are judged the same way: `Deployments.evaluate()` and
`Deployments.evaluationWorker()` run this worker over the server's runs, as the
[deployments guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/deployments.md#judging-with-confidence-and-on-quality) shows.

## Media you can evaluate

An image, a document, an audio clip, or a video can be what is judged. `nexus-ai-pro/evaluate/media`
gives two evaluators that read the media themselves, so they score an experiment over a target that
draws, an online evaluation of runs that make images, or a deployment's runs, like any other.

```ts
import { mediaChecks, mediaJudge } from 'nexus-ai-pro/evaluate/media';

const evaluators = [
  // Read from the media: a PNG or WebP, at least 1,024 pixels wide, at 16:9.
  mediaChecks({ mimeTypes: ['image/png', 'image/webp'], width: { min: 1024 }, aspectRatio: { ratio: 16 / 9 } }),
  // Shown to a vision model, beside the rubric and the brief.
  mediaJudge({ client: ai, model: 'gpt-5', rubric: 'On brand, sharp, and legible at a glance.', key: 'banner' }),
];
```

**Which media.** `mediaOf()` finds them, as both evaluators do: the media in the output, or, with
none there, those the run's trace says it produced, such as an image a tool drew; `from: 'input'`
takes the example's inputs instead, and `from: 'all'` both. `select` picks them from the whole
`EvaluationContext` instead. They are `MediaSelection` options.

**Reading them.** `resolveMedia()` reads one medium into a `ResolvedMedia`: its `MediaSubjectKind`
(`image`, `audio`, `video`, `document`, or `other`), its MIME type as its bytes say, its bytes, and
what its headers say: an image's width and height, and the length of audio or video. It reads base64,
bytes, a data URL, and an asset's bytes as they are. `MediaResolveOptions` reach the rest: `assets`, an asset store
and the tenant, for a stored asset; `fetch`, for a URL, which is otherwise never fetched; `readFile`,
for a path; and `maxBytes`, 20 MiB of any one medium. A medium that cannot be read says why in
`unreadable`.

**Lengths.** `readMediaDuration()` reads the length of audio or video from its own headers, with
nothing decoded and no dependency, as a `MediaDuration`: the milliseconds, the container's MIME type,
and whether it holds a picture. It reads WAV, MP4 and MOV (a fragmented MP4 by its fragments), WebM
and Matroska, MP3 (by its Xing, Info, or VBRI frame count, or at its constant bitrate), Ogg Vorbis and
Opus (without Opus's pre-skip, as RFC 7845 counts it), and FLAC. Bytes it cannot read have no length,
and never throw. Against ffmpeg's own encodings, each container's length is read to the millisecond.

**Checks.** `mediaChecks()` takes `MediaCheckOptions`, each a `MediaRange` or a list, and makes only
the checks asked: `count` (at least one), `kinds`, `mimeTypes` (with `image/*` for any image),
`bytes`, `width`, `height`, `aspectRatio` with a `tolerance` (2%), and `durationMs`. Each is a score,
`media.format` for instance, the share of media that pass it, with a comment naming each that fails;
`media` is the share that pass every check, and 0 when the count is wrong. A measurement that cannot
be read fails its check, or passes it with `unknown: 'pass'`. `key` changes the prefix.

**A judge.** `mediaJudge()` takes `MediaJudgeOptions`: a `client` with `complete()`, a `model` that
can see the media, a `rubric`, and a `key` (`media-judge`). Each medium goes to the model as itself,
read and sent as base64 where it can be, since a provider cannot read an asset store; the brief beside
it is the example's inputs as text, with any media in them named rather than spelled out, or what
`brief` returns. It asks for JSON with a score, a verdict, and a rationale, within `range` (0 to 1)
and against `passThreshold`, at `temperature` 0, showing at most `maxMedia` (8). With nothing to
judge it scores 0, or nothing with `whenMissing: 'skip'`, and calls no model. An experiment records
the judge's model and rubric.

A [canary is rolled back on a media evaluator's score](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/deployments.md#automation-rules) when
those scores fall.

```ts
import { onlineEvaluationWorker } from 'nexus-ai-pro/evaluate/online';

const worker = onlineEvaluationWorker({
  source: 'helpfulness-v2',
  traces,
  operations: new RedisOperationStore(redis, { prefix: 'evaluations:', index: true }),
  evaluators: [judge],
  sampleRate: 0.1,
  budget: budgetLedger({ limit: 50 }),
  estimate: 0.002,
});
worker.start();
```

In the test suite, ten workers score 10,000 runs while one of them dies right after writing a run's
scores. Every run ends with each score exactly once; the run the dead worker never recorded is taken
back and found already scored; and the queue stays near its limit. Workers on a budget stop at it,
leave the rest queued, and carry on the next period without scoring a run twice.

**A spend cap for experiments.** `evaluate()` takes `budget`: a ledger and an estimate per example
run. Each run is reserved before it starts and reconciled with its cost, and a cached output costs
nothing. Once the budget runs out, no further example starts. `evaluate()` then rejects with an
`EvaluationBudgetError` that carries the results it paid for, and the experiment is not stored.

`AnnotationQueue` takes `AnnotationQueueOptions`:

- the `rubric`, a list of `ReviewQuestion` values: a key, the prompt, a type, and choices for a choice
  question;
- a claim `leaseMs` (15 minutes), and `consensus`, the answers an item needs (1);
- a clock.

| Method | What it does |
| --- | --- |
| `enqueue()` | Adds a `ReviewItem`: its subject, rubric, status (`pending`, `claimed`, or `reviewed`), live claims, answers, creation time, and metadata. |
| `claim()` | Hands the next item to a reviewer. |
| `submit()` | Records a `ReviewAnswer`: the reviewer, their answers as scores, a comment, and when. |

## Tools that touch nothing

An agent whose tools charge a card, send an email, or write to a production database cannot be
evaluated against them. `emulatedTools()`, from `nexus-ai-pro/evaluate/emulate`, returns the same
tools, by name, description, parameters, and capabilities, each answered by a model playing it, or by
an answer a store kept, and never run: nothing they would send is sent.

```ts
import { createAgent, agentInput } from 'nexus-ai-pro/agent';
import { emulatedTools, MemoryEmulationStore } from 'nexus-ai-pro/evaluate/emulate';

const store = new MemoryEmulationStore(JSON.parse(await readFile('emulated.json', 'utf8')));
const experiment = await evaluate(
  async (inputs, { example }) => {
    const tools = emulatedTools([lookupOrder, refund], {
      client: ai,
      model: 'gpt-5-mini',
      scenario: example.metadata?.scenario as string, // "Order 1042 was delivered and cost $59."
      store,
    });
    return createAgent({ client: ai, model: 'gpt-5', tools }).invoke(agentInput(inputs.request));
  },
  dataset,
  [answered, trajectory({ mode: 'exact' })],
);
await writeFile('emulated.json', JSON.stringify(store));
```

Each call asks the `EmulatorClient`, the client or anything with its `complete()`, to answer as the
real tool would, as `{"result": ...}`, or as `{"error": "..."}` for a failure, which the tool then
throws, so the agent sees it as it would the real tool's. The emulator is told, by
`EmulatedToolsOptions`:
- the tool's name, description, and parameters, and `results`, a JSON Schema of what it returns;
- `instructions`, each tool's own notes, and the `scenario`, the world the tools act on, such as an
  example's;
- `examples`, real calls of the tool with what it returned, such as from production traces;
- the calls this set has answered before, up to `history` (20), so an order looked up after a refund
  shows it.

It answers at `temperature` 0 by default, with `model`, `maxTokens`, and the call's signal and tenant.
An answer that is not JSON, or that `validate` refuses with a reason, is asked for again once with
that reason, and then fails the call with an `EmulationError`, `EMULATION_INVALID`. Tools named in
`passthrough`, those that read and change nothing, run for real.

The answers go to an `EmulationStore`, each an `EmulatedResult` of the tool, its arguments, and its result or error, by `emulationKey()`: the tool, the arguments, the scenario, and
which call with those arguments it is. `mode` is `auto` with a store, replaying an answer it has and
asking for the rest; `replay`, which asks for nothing and fails a call it has no answer for with
`EMULATION_MISS`; `record`, which asks every time; or `emulate`, without a store. `MemoryEmulationStore`
keeps them in memory, and `toJSON()` writes them where the next run reads them back, so a suite runs
the same answers in CI without a model. `onCall` sees each `EmulatedToolCall`: the tool, the arguments,
the result or the error, the key, and who answered, `model`, `store`, or `real`.

In the test suite, a support agent looks orders up and refunds them through a shop of the test's own.
Evaluated over the real tools and then over emulated ones told each example's scenario, it reaches the
same verdict on all four examples, a refund, an order not yet delivered, one escalated, and a refund
the shop refuses, and the emulated run sends the shop nothing. Replayed from the kept answers, it
reaches that verdict again with the emulator never asked; told a different scenario, the evaluation
reports the different answer it leads to.

### Tools from an API description

A service that publishes an OpenAPI document can be emulated before an agent is wired to it: read the
document into tools with `openApiTools()`, from the [agents guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/agents.md), and emulate them,
showing the emulator what each returns:

```ts
const shop = openApiTools(JSON.parse(await readFile('shop.openapi.json', 'utf8')));
const tools = emulatedTools(shop.tools, { client: ai, model: 'gpt-5-mini', results: shop.results, world });
```

The tools have the service's names, parameters, and descriptions, so the agent is evaluated against
the service it will call; read without a base URL, they could not reach it if they ran.

### A world across an evaluation

An emulator holds what its tools did in one run; the next run starts again from its scenario. An
`EmulatedWorld`, given as `EmulatedToolsOptions.world`, holds it across a whole evaluation instead: the
scenario it started from, and each call made in it as an `EmulatedWorldCall`, so an order one example
refunds is refunded for the next, as on a staging copy of the service. The emulator is told every
change made in the world, however long ago, and the latest `history` calls.

```ts
const world = new EmulatedWorld({ scenario: 'Order 1042 is delivered and cost $59.' });
await evaluate(
  (inputs) => agentOver(emulatedTools(shop.tools, { client: ai, results: shop.results, world })).invoke(agentInput(inputs.request)),
  dataset,
  evaluators,
  { concurrency: 1 },
);
```

A call is a change when it succeeds and its tool does more than read: a tool named in `reads`, or one
whose capabilities are all reads, `api.read`, `mcp.read`, or `filesystem:read`, with `network` beside
them, changes nothing, nor does one that declares none. `changes` lists them, and `calls` everything.

Runs that share a world run one at a time, in order. To run them side by side, start each from the
same point: `snapshot()` writes the world as an `EmulatedWorldSnapshot`, plain JSON for a file, and
`new EmulatedWorld(snapshot)` reads it back; `fork()` does both at once. A world's `digest()` names
its scenario and its changes, and the key of each answer kept carries it, with `ordinal()` counting
the same call since the last change, so a recorded evaluation replays only after the same changes.
Without a world, keys are those of every earlier release.

In the test suite, a shop serves its own OpenAPI document. A support agent is evaluated on six
examples, two of which depend on a refund the first makes: over tools written by hand for the shop,
over tools read from its document, which send the same requests byte for byte, and over those tools
emulated in one world, all three reaching the same verdict, the emulated run sending the shop nothing.
Emulated without a world, the two dependent examples come out otherwise. Started each from a snapshot
taken after the refund, the other five reach their verdicts side by side, and replayed from the
answers kept, the six reach them again with the emulator never asked.

## LLM judges and eval cases

Use a plain assertion for small evals, or add an LLM judge when a rubric is more useful than exact matching:

```ts
import { LLMJudge } from 'nexus-ai-pro/evals';

const judge = new LLMJudge({
  client: ai,
  model: 'openai/gpt-4.1-mini',
  rubric: 'Score whether the answer is correct, concise, and grounded in the supplied context.',
  passThreshold: 0.7,
});

const run = await ai.runEvals([
  {
    name: 'support answer quality',
    request: {
      model: 'auto',
      messages: [{ role: 'user', content: 'Explain our refund policy.' }],
    },
    expected: 'Refunds are available within 30 days.',
    judge: judge.asEvalJudge(),
  },
]);
```

`judge.asEvaluator()` is the same judge as an evaluator for `evaluate()`. It scores each output under
`key` (`judge` by default); `input` maps an evaluation context to what the judge sees, and defaults to
the output as the answer, the expected output as the reference, and the inputs as the question. The
experiment records the judge's model, temperature, rubric, and a hash of its instructions:

```ts
const experiment = await evaluate(target, dataset, [judge.asEvaluator({ key: 'quality' })]);
experiment.evaluators?.[0]?.judge; // { model, promptVersion, temperature }
```

## Eval cases in detail

`EvalRunner` runs `EvalCase` values through anything with `complete()`: the `EvalClient` contract.

A case has a name and a `request`, and is checked by an `assert` function, a `judge`, or both. It can
also carry an `expected` answer, a `metrics` function for `calculateEvalMetrics()`, and tags.

An `EvalJudge` returns true or false, or an `EvalJudgment`: a score from 0 to 1, whether it passed, a
rationale, labels, the raw output, and the judge's provider and model.

`EvalRunOptions` names the run and sets `concurrency` (1, so cases run in order), a `store`, and
metadata. Each `EvalResult` says whether its case passed and how long it took, with the response,
metrics, judgment, or error. The `EvalRunResult` totals them. It also carries the run as an
`experiment`, so it can be compared and gated like any other.

`LLMJudge` takes `LLMJudgeOptions`:

| Option | Default | Meaning |
| --- | --- | --- |
| `client` | — | Anything with `complete()`: the `JudgeClient` contract. |
| `model`, `rubric` | — | Who judges, and against what. |
| `systemPrompt` | built in | Replaces the judge's instructions. |
| `range` | 0 to 1 | The score range. |
| `passThreshold` | middle of the range | The score that counts as a pass. |
| `temperature` | 0 | Kept at 0 for repeatable verdicts. |
| `maxTokens`, metadata | 512 | — |

It judges an `LLMJudgeInput` — the answer, a reference, the question, a rubric for this answer, and
source passages. It returns an `LLMJudgeResult`, with the score clamped to the range.
`parseJudgeResponse()` reads the judge's reply, whether JSON or a number in prose.

`createLLMJudgeEval()` and `asEvalJudge()` adapt a judge to an eval case. An `LLMJudgeInputMapper` turns
a response and its case into what the judge sees.

## Metrics

`calculateEvalMetrics()` computes every metric its `MetricInputs` make possible. It needs the answer.
Everything else is optional, and each input you add unlocks more metrics:

- the expected answer, the question, and the contexts;
- ranked retrieved chunks and the relevant ids;
- candidates, and which of them passed;
- token log-probabilities, latency, time to first token, token counts, and cost;
- a policy of forbidden and required terms, and an embedder.

It returns `EvalMetrics`, grouped by kind:

- `QualityMetrics`: `exactMatch()` after normalizing case, punctuation, and whitespace;
  `semanticSimilarity()` under the embedder; `passAtK()`; and `perplexity()` from log-probabilities.
  `f1Score()` gives token-overlap F1 on its own.
- `RagMetrics`: `faithfulness()`, the share of factual statements the contexts support lexically;
  `contextualPrecision()` and `contextualRecall()` of the ranked chunks; and answer relevancy.
- `SafetyMetrics`: hallucination rate (one minus faithfulness), `toxicityScore()`, `biasScore()`,
  `policyAdherence()`, and `refusalRate()`.
- `OperationalMetrics`: `tokensPerSecond()`, time to first token, latency, cost, and token counts.

The safety metrics are heuristics — short term lists and fixed patterns — suited to catching a
regression in a test, not to judging content in production; use a model judge for that.

## Limitations

- A simulated user is as good as the model that plays it. Hold its model and temperature fixed
  between the experiments you compare, and read some conversations before trusting a verdict.
- A conversation runs its turns in order, so 50 conversations of 5 turns are 250 agent calls and 250
  user-model calls. `concurrency` runs conversations side by side, not turns.
- A synthetic example is as good as the model that wrote it. Its quote proves the passage says
  something, not that the question is a good one: read a sample before trusting a verdict.
- An adversarial dataset covers the attacks the guardrails know. A new phrasing needs a new pattern,
  or `rephrase`.
- An emulated tool is as faithful as its scenario and its emulator. Check a suite against the real
  tools, or a staging copy, once, and keep the answers; a verdict that differs is a scenario to fix.
- `emulatedTools()` replaces `execute` and drops `stream`: an emulated tool reports no progress.
- A world tells the emulator every change made in it, so an emulator's prompt grows with an
  evaluation's changes; start long evaluations' runs from snapshots to keep it short.
- Runs that share a world must run one at a time: side by side, their calls interleave differently
  each time, so the world, and the keys of its answers, differ between runs. Fork the world per run.
- `mediaChecks()` reads the dimensions of PNG, JPEG, GIF, WebP, and BMP images, and the length of
  WAV, MP4 and MOV, WebM and Matroska, MP3, Ogg (Vorbis and Opus), and FLAC. Other audio and video are
  checked for kind, type, and size; a judge can watch them, on a provider that takes them. A file cut
  short before the part that says its length, such as an MP4 whose movie header comes last, has none.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/context-hub.md

# Context hub


Everything an agent runs with, versioned together as one context bundle: the prompts it uses, its
instructions, the tools it is offered, its skills, and its settings.

You commit a bundle, label it, promote it through evaluation gates, diff it, roll it back, and export
it to another project. It is what the [prompt registry](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/prompts.md) does for one prompt, applied to
the whole context around it.

```ts
import { PromptRegistry } from 'nexus-ai-pro/prompts/registry';
import { ContextHub, contextExperimentGate, evaluateContext } from 'nexus-ai-pro/context-hub';

const hub = new ContextHub({
  prompts: registry,
  gates: { production: [contextExperimentGate({ store: experiments, noRegression: true })] },
});

const bundle = await hub.commit(
  {
    name: 'support-agent',
    prompts: { answer: { name: 'support-answer', version: answer.version } },
    instructions: { policy: 'Never promise a refund.', tone: 'Be brief.' },
    tools: [{ name: 'lookup_order', description: 'Finds an order', parameters: orderSchema }],
    skills: { refunds: { description: 'Refund questions', instructions: 'Check the order first.' } },
    config: { model: 'gpt-5.4-mini' },
  },
  { message: 'Cite the order number', author: 'ana', label: 'staging' },
);

await evaluateContext(bundle, dataset, [correctness], runAgent, { store: experiments });
await hub.promote('support-agent', { from: 'staging', to: 'production', by: 'ana' });
```

## Bundles

A `ContextBundleDefinition` names the bundle and holds its parts, each optional:

- `prompts` pins prompts from the registry by role, each a `ContextPromptPin` of a prompt name and an
  exact content version, so a bundle never changes because a prompt label moved.
- `instructions` are named texts, such as a policy and a tone guide.
- `tools` are `ContextTool` definitions — a name, a description, and an argument schema. The bundle
  decides what the model may call; the implementations stay in code.
- `skills` are `ContextSkill` values: when a skill applies, how to do the task, and the reference
  material it needs.
- `config` holds settings the application reads, and `metadata` application data that is not part of
  the version.

`commit()` stores a definition as a `ContextBundle` under its content version: `c` and 12 hex digits
of a SHA-256 over every part but the metadata, which `contextVersion()` computes on its own. The same
content is always the same version, in any project, and committing unchanged content records
nothing. With a prompt registry, every pin must exist in it.

Using a bundle:

| Function | What it does |
| --- | --- |
| `get()` | Finds a version by content version, label, or `latest`. |
| `resolve()` | The same, with the `ContextReference` experiments and traces record. |
| `renderPrompt()` | Renders the prompt a bundle pins for a role, recording the bundle in the request's `metadata.context`. |
| `contextInstructions()` | Joins the instructions into one system text. |
| `bindTools()` | Turns the bundle's tools into callable tools, matching each to your implementation by name. An implementation the bundle does not offer is never exposed. |

```ts
const { bundle } = await hub.resolve('support-agent', 'production');
const agent = createAgent({
  client: ai,
  model: String(bundle.config?.model),
  systemPrompt: contextInstructions(bundle),
  tools: bindTools(bundle, { lookup_order: lookupOrder }),
});
```

## Labels, promotion, and history

Labels point at versions, and every change is recorded, as in the prompt registry.

| Method | What it does |
| --- | --- |
| `label()`, `unlabel()` | Point a label at a version without gates, or remove it. |
| `promote()` | Moves a label to a version — named directly, or whatever another label serves — once every gate agrees. Returns a `ContextPromotionResult` with each verdict. |
| `rollback()` | Moves a label back to where it pointed before. |
| `history()`, `versions()`, `labels()`, `names()` | Read the record. |

A refused promotion raises a `ContextPromotionError`, unless `force` is set; forcing is recorded in
the note. A `ContextLabel` and a `ContextHistoryEntry` have the same shapes as their prompt
counterparts.

A `ContextPromotionGate` is a function of a `ContextPromotionContext` — the bundle, the version being
promoted, the label, and what it serves now — that returns a verdict. Two come bundled:

- `contextExperimentGate()` refuses until an experiment has passed for the exact version, taking the
  same `ExperimentGateOptions` as the prompt gate: thresholds, a maximum number of failures, and
  `noRegression` against the version it would replace.
- `servedByContextGate()` requires another label, such as `staging`, to serve the version first.

`evaluateContext()` runs a bundle over a dataset. You give it a `ContextTarget`: your function that
builds a request from the bundle and runs it. The experiment records the bundle in
`metadata.context`, which is what the experiment gate looks for.

With an evaluation cache and no fingerprint, the bundle's version is the fingerprint. An unchanged
bundle is then scored from its stored outputs; see [evaluation](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/evaluation.md#caching).

`ContextHubOptions` takes the `store`, the `prompts` registry, the `gates` by label, an `onChange`
callback for every recorded change, and a clock.

Errors are `ContextHubError` values with a stable code:

| Error | When |
| --- | --- |
| `ContextNotFoundError` | A bundle, version, or label does not exist. |
| `ContextConflictError` | A label moved while it was being changed. |
| `ContextDefinitionError` | A definition, pin, or import is invalid. |

## Diffs

`diff()` compares two versions or labels, and `diffContexts()` two bundles you already have. The
`ContextDiff` lists each `ContextChange` — an added, removed, or changed prompt pin, instruction,
tool, skill, or config key — with line diffs for text. `formatContextDiff()` writes it as text for a
terminal or a pull request.

## Moving bundles between projects

The hub keeps its state in a prompt store of its own, so every prompt adapter serves bundles:
`MemoryPromptStore`, Redis, Postgres, and `FilePromptStore`, which writes one reviewable JSON file per
version for version control.

`export()` writes a bundle version as one `ContextBundleExport` document, carrying every prompt
version it pins; `import()` commits those prompts into the other project's registry and then the
bundle. Each version is checked on the way in, so an export edited by hand is refused rather than
imported under a version that no longer describes it.

```ts
await writeFile('support-agent.json', JSON.stringify(await hub.export('support-agent', 'production')));
await otherHub.import(JSON.parse(await readFile('support-agent.json', 'utf8')), { label: 'staging' });
```

## Signed bundles

A version check proves an export was not edited by accident. A signature proves who published it.
A hub with a `signer` signs every new version, and a hub with a `keyring` checks those signatures.

- **What a signature covers.** It covers the bundle's content and the content of every prompt it
  pins, under the full SHA-256 that `contextDigest()` computes over a `ContextSignedContent`. A
  version is only the first 12 hex digits, enough to name content but not to stand behind a signature.
  Signing a bundle that pins prompts needs the hub's prompt registry, which is where that content
  comes from.
- **The keys.** Signatures are Ed25519, through Web Crypto, so on any runtime.
  `generateContextKey()` creates a `ContextSigningKey`: a `ContextSigner`, and both halves as a
  `ContextJsonWebKey` to keep in a secret store. `ed25519Signer()` loads a private key as a signer,
  and `ed25519Keyring()` loads trusted public keys, by key id, as a `ContextKeyring`. Either takes a
  JSON Web Key or a `ContextCryptoKey`, which any runtime's `CryptoKey` is.
- **What is stored.** Each `ContextSignature` names its key id, the digest, the signature, and when it
  was signed. It is stored with the version and travels in its export, but it is not part of the
  version.
- **Checking.** `verify()` returns a `ContextVerification`: whether a trusted key signed exactly this
  content, the digest it has now, and the signature that verified. A version's answer is checked once
  and remembered.
- **Requiring it.** With `requireSignature`, the hub refuses, with a `ContextSignatureError`, a bundle
  with no signature from a key in its keyring. That holds on `import()`, and whenever `resolve()` or
  `renderPrompt()` serves a version. A version edited where it is stored is refused when it is served.
- **Imports keep signatures.** An import keeps the signatures that claim its exact content, and a hub
  with its own signer adds one. That is how a key rotates: re-import the bundles through a hub that
  trusts the old key and signs with the new one.

```ts
import { ContextHub, ed25519Keyring, ed25519Signer } from 'nexus-ai-pro/context-hub';

const publisher = new ContextHub({ prompts, signer: await ed25519Signer('release-2026', privateKey) });
const production = new ContextHub({
  prompts,
  keyring: await ed25519Keyring({ 'release-2026': publicKey }),
  requireSignature: true,
});
```

## On every trace

`renderPrompt()` records the bundle in the request's `metadata.context` and the prompt in
`metadata.prompt`, and a traced model call keeps both. A trace query by
`{ metadata: { 'context.version': version } }` finds every call one version made, and the `query`
of an insight narrows the same way. An agent that is not rendered from the hub can still name its
bundle: pass the `reference` that `resolve()` returns as `metadata: { context: reference }` to
`createAgent()`.

## Limitations

- A bundle pins prompt versions; it does not follow prompt labels. Commit a new bundle version to
  take a new prompt version, which is what makes a bundle reproducible.
- Versions are immutable, so one committed before a signer was configured stays unsigned. Re-import
  it through a hub that signs.
- A signature covers what a bundle names. The tool implementations `bindTools()` binds are the
  application's own code, which the bundle cannot change.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/insights.md

# Insights


Insights find problems in your traces without being asked. They do three things:

- **Group failures.** Failing and slow runs are grouped by what went wrong.
- **Spot regressions.** A metric that got worse between two sets of runs is reported.
- **Propose fixes.** Opt-in: a fix is written, evaluated on your dataset, and waits for a person.

```ts
import { detectRegressions, findIssues, proposeFix, modelFixProposer, ProposalInbox } from 'nexus-ai-pro/insights';

const issues = await findIssues({ store: traces, slowMs: 8000 });
const regressions = await detectRegressions({
  store: traces,
  current: { since: yesterday },
  baseline: { since: lastWeek, until: yesterday },
});
```

## Clusters and issues

`clusterRuns()` groups runs that failed, or behaved, the same way. `ClusterOptions` picks what to
group `by`. That is a `ClusterBy`:

| Value | Groups runs by |
| --- | --- |
| `error` | The run name and the error message. `errorSignature()` normalizes the message first: it takes out ids, numbers, quoted values, and addresses. |
| `trajectory` | The path a run took: the steps `trajectoryOf()` reads from its trace tree. |
| `meaning` | The run's text, embedded with the function you pass. Use it when messages differ but mean the same. |

Each group is a `RunCluster`: its runs, a stable id, what they share, their run names and models, and
when the group was first and last seen.

`findIssues()` reads a window of traces and returns its problems, most common first. Each `Issue` is a
cluster of failing runs, or of slow runs when `slowMs` is set. It also gives the share of the window it
covers and a readable summary.

```ts
const issues = await findIssues({ store: traces, since: lastHour, by: 'error', minCount: 3 });
for (const issue of issues) console.log(issue.summary);
```

`FindIssuesOptions` sets:

- the window, and a query to narrow the runs;
- whether to count only root runs, one per request, which is the default;
- how many runs to read, and the smallest cluster that counts;
- the clustering options, which pass through. Clustering by trajectory reads each run's trace tree.

## Regressions

`detectRegressions()` compares two time windows, each a `TimeWindow`. It judges each run name, model,
or provider on its own, and reports each `Regression`: the metric, both values, the change, and the
sample sizes.

`DetectRegressionsOptions` chooses the `RegressionMetric` values to compare:

| Metric | Counts as worse when |
| --- | --- |
| error rate | The rise passes a two-proportion z-test at 95%. |
| p95 latency | It rises by a relative margin, a quarter by default. |
| mean cost | It rises by a relative margin, a quarter by default. |
| a feedback score | Its mean falls by more than twice its standard error. You name the keys. |

A group needs `minRuns` in both windows before it is judged, 20 by default. So a quiet endpoint does
not raise alarms on noise.

### Comparing any two sets of runs

`compareRuns()` applies the same tests to two lists of runs you already have. It is what
`detectRegressions()` runs for each group, and what the [canary guard](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/deployments.md#canary-guard)
runs to compare a canary with the live revision.

```ts
import { compareRuns } from 'nexus-ai-pro/insights';

const regressions = compareRuns(liveRuns, canaryRuns, { group: 'support@v2', metrics: ['error-rate', 'latency'] });
```

Each run is a `RunSample`: a `status` (`error` counts as a failure), and optionally `latencyMs`,
`cost`, and `feedback`. A traced `Run` already fits. `CompareRunsOptions` has the group name for the
summaries, the metrics, the feedback keys, `minRuns`, and the latency and cost margins.

With `confidence`, such as 0.95, each metric is judged instead by a seeded bootstrap. A metric then
regresses only when its interval does not reach zero and the change is at least its minimum effect:
- `minErrorRateIncrease` for the error rate;
- the latency and cost margins;
- `minFeedbackDrop` for a feedback score.

So neither noise nor a trivially small change counts. `resamples` and `seed` control the bootstrap,
and each `Regression` then carries its `interval`. The
[canary guard](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/deployments.md#judging-with-confidence-and-on-quality) judges canaries this way.
`minLatencyChangeMs` sets the smallest latency rise that counts, in milliseconds, which keeps noise on
fast runs from reading as a regression. `detectRegressions()` takes it too.

## Proposed fixes

`proposeFix()` answers an issue with a candidate fix, and measures it before anyone is asked. It takes
four steps:

1. A `FixProposer` gets a `FixRequest`: the issue, the version its label serves, and some runs that
   show it. It returns a `FixCandidate`: a new definition, and why it should help.
2. The candidate is committed without a label, to the prompt registry or context hub the
   `FixSubject` names.
3. Your `evaluate` function scores the current version and the candidate on your dataset. The
   comparison decides the verdict.
4. An improvement becomes a `pending` `FixProposal` for a person. Anything else is `discarded`, with
   its evidence kept.

```ts
const proposal = await proposeFix({
  issue: issues[0],
  subject: { kind: 'prompt', registry, name: 'support-answer', label: 'production' },
  propose: modelFixProposer(ai, { model: 'gpt-5.4' }),
  evaluate: (version) => evaluatePrompt(version, dataset, [correctness], { client: ai, store: experiments, cache }),
  store: proposals,
});
```

With an evaluation cache, the current version's outputs are reused across proposals. Each proposal
then pays only for its candidate.

`ProposeFixOptions` also takes:

| Option | What it does |
| --- | --- |
| `compare` | Options for the comparison that decides the verdict. |
| `store` | Where proposals are kept. |
| `pullRequest` | Opens a pull request for an improving fix, through a `PullRequestClient` you inject. It is a small interface over your forge, so no forge is a dependency. |

`modelFixProposer()` is a proposer over a chat model, a `FixModelClient`. It rewrites a prompt's
messages, or a bundle's instructions, from the issue and example runs. `ModelFixProposerOptions`
names the model.

### Keeping proposals, and acting on them

A `ProposalStore` keeps proposals. `MemoryProposalStore` keeps them in process, and
`FileProposalStore` as one JSON file each.

`ProposalInbox` is what a person acts on. It lists the pending proposals. `promote()` moves one
through the label's gates, like any promotion, and `reject()` turns it down. `ProposalInboxOptions`
gives the inbox its store, registry, and hub. The [studio](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/studio.md) shows the inbox, behind roles.

## Limitations

- Clustering by meaning embeds every run's text. On a busy store, bound the window and the `limit`.
- A proposed fix is only as good as the dataset it is evaluated on. Nothing is promoted without a
  person, and promotion still runs the label's gates.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/prompts.md

# Prompts: templates, versions, promotion, and serving


Prompts that ship like code:

- **Templates** whose variables are typed from their text.
- **Versions** derived from their content.
- **Labels**, such as `staging` and `production`, that move only when their gates agree.
- **A serving client** that keeps answering through a registry outage.

The family is split by what a process does, so each process pays only for its part:

| Entry point | For | Size |
| --- | --- | --- |
| `nexus-ai-pro/prompts` | Defining and rendering prompts. | Smallest. |
| `nexus-ai-pro/prompts/client` | Serving prompts by label in an application. | Small. |
| `nexus-ai-pro/prompts/registry` | Committing, promoting, comparing, and evaluating, in CI and admin scripts. | Larger. |
| `nexus-ai-pro/prompts/file`, `/prompts/redis`, `nexus-ai-pro/postgres/prompts` | Where versions and labels live. | Each on its own. |

None of them imports a third-party package. `/prompts` and `/prompts/client` run without Node's APIs.

## Templates

`definePrompt()` compiles a prompt once. Its variables are read from the template text, so `render()`
is type-checked. A variable without a default is required, and a misspelled one is a compile-time
error, not a blank in production.

```ts
import { definePrompt } from 'nexus-ai-pro/prompts';

const summarize = definePrompt({
  name: 'summarize',
  messages: [
    { role: 'system', content: 'You summarize {{kind}} for {{audience}}. {{> tone}}' },
    { placeholder: 'history', optional: true },
    { role: 'user', content: '{{ticket.body}}' },
  ],
  partials: { tone: 'Keep it {{tone}}.' },
  defaults: { audience: 'engineers', tone: 'brief' },
  config: { model: 'gpt-5.4-mini', temperature: 0 },
});

const request = summarize.render({ kind: 'incident reports', ticket: { body: text } });
const response = await ai.complete(request);
```

| Syntax | What it does |
| --- | --- |
| `{{name}}`, `{{name.path}}` | Inserts a value. Objects render as JSON. |
| `{{> partial}}` | Includes a named fragment. Partials may include partials. |
| `{ placeholder: 'history' }` | Inserts whole messages, such as the conversation so far. Required unless `optional` is set. |

An unknown partial, a cycle, or a malformed tag fails when the prompt is defined, not when it is first
rendered. There is no escape syntax: to render a literal `{{`, pass it in through a variable.

- `config` holds any completion field: model, temperature, output format, tools. It is versioned with
  the prompt. `overrides` in the render options replace it for one call, and `model` supplies a model
  when neither names one.
- A missing variable throws `PromptRenderError`, naming every missing variable. `missing: 'empty'`
  renders it as an empty string, and `missing: 'keep'` leaves the tag in place.
- The rendered request records `metadata.prompt`: the name, and, from a registry, the version, label,
  and A/B arm.
- `TemplateVariables` gives the variable names of any template string. `PromptInput` gives a prompt's
  render input.

`compilePrompt()` and `renderCompiled()` are the two halves of `render()`, for code that manages its
own compiled cache.

## Versions

A version is derived from content. `promptVersion()` hashes the name, messages, partials,
configuration, and defaults. It hashes a canonical encoding, `canonicalJson()`, so key order never
changes it. Metadata is not part of it.

So the same prompt has the same version in a test, in CI, and in production. `prompt.version()` gives
the version a registry will assign, before anything is committed. Hashing uses Web Crypto, so the value
is the same in Node, a browser, and an edge runtime.

## The registry

`PromptRegistry`, from `nexus-ai-pro/prompts/registry`, stores versions and moves labels. It keeps no
state beyond a compiled cache, so any number of processes can share one store.

```ts
import { PromptRegistry, experimentGate, servedByGate } from 'nexus-ai-pro/prompts/registry';
import { PostgresPromptStore } from 'nexus-ai-pro/postgres/prompts';

const registry = new PromptRegistry({
  store: new PostgresPromptStore(pool),
  gates: {
    production: [
      servedByGate('staging'),
      experimentGate({ store: experiments, thresholds: { correctness: 0.9 }, noRegression: true }),
    ],
  },
});

const version = await registry.commit(summarize, { message: 'Tighter tone', author: 'ada', label: 'staging' });
await registry.promote('summarize', { from: 'staging', to: 'production', by: 'ada' });
```

| Method | What it does |
| --- | --- |
| `commit()` | Stores a definition under its content version. Unchanged content returns the existing version and records nothing, so a deploy can commit every prompt. |
| `get()` | Reads by version, by label, or `latest`. |
| `resolve()`, `render()` | The same, for serving. They choose an A/B arm by key. |
| `label()`, `unlabel()` | Point a label at a version without gates, or remove it. |
| `promote()` | Moves a label once every gate for the destination allows it. |
| `rollback()` | Moves a label back to where it pointed before its last change. |
| `split()` | Shares a label's traffic between versions, for an A/B test. The first arm is the control. |
| `history()` | Every change, newest first, optionally for one label. |
| `versions()`, `labels()`, `names()` | List the rest. |

A refused promotion throws `PromptPromotionError`, with each gate's verdict. `force: true` promotes
anyway and records which gates were overridden. `promote()` returns a `PromotionResult`, with
`changed: false` when the label already served the version.

`chooseVariant()` gives each key the same A/B arm every time.

Labels are written with compare-and-set. When another process moved a label in the meantime, the
change is refused with `PromptConflictError`, not overwritten.

### Hearing about changes

- `onChange` is called after each change.
- `webhooks` post changes — `promote`, `rollback`, and `split` by default — signed like operation
  webhooks. `deliverPromptWebhook()` sends one, and a receiver checks it with `verifyPromptWebhook()`.
- A failing receiver is reported to `onWebhookError`. It never fails the change.

### Errors

Every error extends `PromptError` and has a stable `code`:

| Error | When |
| --- | --- |
| `PromptDefinitionError` | A definition is malformed. |
| `PromptRenderError` | A variable is missing. |
| `PromptNotFoundError` | No prompt, version, or label by that name. |
| `PromptConflictError` | Another process moved the label first. |
| `PromptPromotionError` | A gate refused. Its `results` list each `GateResult`. |

## Promotion gates

A gate is a function of a `PromotionContext` that returns a `GateResult`. The context holds the
version, the label it goes to, the version that label serves now, and the registry. Two gates are
bundled:

- `experimentGate()` requires a passing experiment for the exact version being promoted, one whose
  `metadata.prompt` names it. `ExperimentGateOptions` can also require an experiment name or dataset.
  `thresholds` sets a minimum mean per metric, and `maxErrors` the examples that may fail outright.
  `noRegression` compares with the newest experiment for the version being replaced, and refuses a
  change that is worse beyond noise.
- `servedByGate()` requires the version to be what another label serves, such as staging before
  production.

Any function with the `PromotionGate` signature works as a gate: a human approval recorded elsewhere,
a check on the model's price, or a freeze window.

## Evaluating a version

`evaluatePrompt()` is the headless playground. It renders a prompt for each example, runs it through a
client, and scores it with ordinary evaluators. It returns an experiment tagged with the prompt's name
and version: the tag `experimentGate()` looks for.

```ts
import { evaluatePrompt } from 'nexus-ai-pro/prompts/registry';
import { createDataset, contains, FileExperimentStore } from 'nexus-ai-pro/evaluate';

const experiment = await evaluatePrompt(version, dataset, [contains(['refund'])], {
  client: ai,
  store: new FileExperimentStore('./experiments'),
  variables: (inputs) => ({ kind: 'support tickets', ticket: inputs }),
});
```

`EvaluatePromptOptions` accepts everything `evaluate()` does, plus two fields:

- `variables` maps an example's inputs to template variables.
- `render` passes render options, such as a model override. That is how one prompt is compared across
  models.

With an evaluation `cache` and no `fingerprint`, the prompt version and render options are the
fingerprint. So re-evaluating an unchanged version re-scores its stored outputs. Pass your own
fingerprint when the client or model changes. The evaluation runtime loads only when
`evaluatePrompt()` runs.

To version a prompt together with the instructions, tools, and skills around it, use the
[context hub](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/context-hub.md). A bundle pins prompt versions from this registry, and is promoted the
same way.

## Comparing versions

`diff()` on the registry, or `diffPrompts()` on two versions, compares them. Messages are compared line
by line; partials, configuration, and defaults field by field. `formatPromptDiff()` renders the result
for a terminal or a pull-request comment, and `diffLines()` is the line diff underneath.

A `PromptDiff` holds a `PromptMessageDiff` per message, made of `PromptLineChange` lines, and a
`PromptFieldChange` per changed field.

## Serving

`PromptClient`, from `nexus-ai-pro/prompts/client`, serves prompts by label to application code.

```ts
import { PromptClient } from 'nexus-ai-pro/prompts/client';

const prompts = new PromptClient({
  source: new PostgresPromptStore(pool),
  label: 'production',
  ttlMs: 60_000,
  fallbacks: [summarize],
  onError: (error, name) => logger.warn({ error, name }, 'prompt refresh failed'),
});

const request = await prompts.render('summarize', { kind: 'tickets', ticket }, { key: user.id });
```

How it stays fast and up:

- A label is read from the source at most once per `ttlMs`.
- For `staleWhileRevalidateMs` after that, calls are answered from cache while the label refreshes in
  the background. Concurrent refreshes share one request.
- When the source is unreachable, the last version seen keeps serving, however old, unless
  `serveStaleOnError` is off.
- A process that starts during an outage serves its `fallbacks`: the definitions bundled in code.
- A split label picks its arm by `key`, so the same user sees the same version every time.

`get()` returns a `ServedPrompt`. It says which version was served, and whether it came from cache, a
stale cache, the source, or a fallback. `refresh()` rereads a label now, and `clear()` forgets
everything.

The source needs only `getLabel()` and `getVersion()`. Any `PromptStore` fits, or a `PromptSource` of
your own, such as an HTTP endpoint in front of the registry.

## Stores

A `PromptStore` holds versions, labels, and history. Versions are immutable and keyed by content, so
saving one twice is harmless. Labels are the only mutable state, and `setLabel()` compares before it
writes.

| Store | Entry point | Use it for |
| --- | --- | --- |
| `MemoryPromptStore` | `/prompts/registry` | Tests, and prompts defined in code in one process. |
| `FilePromptStore` | `/prompts/file` | Prompts in version control: one reviewable JSON file per version and label. |
| `RedisPromptStore` | `/prompts/redis` | Sharing between processes. Label compare-and-set is one Lua call when the client has `eval`. |
| `PostgresPromptStore` | `/postgres/prompts` | Sharing between processes, with the schema from `promptStoreMigration()`. |

`RedisPromptLikeClient` lists the Redis commands the adapter needs, in `ioredis` argument order, so no
Redis client becomes a dependency. `MemoryPromptStoreOptions`, `RedisPromptStoreOptions`, and
`PostgresPromptStoreOptions` set the history length, key prefixes, and table names.

## Traces

A model call wrapped with `traceModelClient()` from `nexus-ai-pro/tracing` records the rendered
request's `metadata.prompt` on its run. So a trace answers which prompt version produced an output, and
a trace query can filter on it.

## Types

| Type | What it is |
| --- | --- |
| `PromptDefinition` | Everything that defines a prompt. |
| `PromptVersion` | A committed prompt, with its version, variables, and provenance. |
| `PromptMessage` | A `PromptMessageTemplate`, or a `PromptMessagePlaceholder` whose variable holds `PlaceholderMessages`. |
| `PromptModelConfig` | The versioned request configuration. |
| `RenderOptions` | Configures one render. |
| `RenderedPrompt`, `PromptReference` | The result of a render, and the reference it records. |
| `PromptLabel`, `PromptVariant` | A label, and one arm of a split. |
| `PromptHistoryEntry`, `PromptHistoryAction` | One recorded change, and what kind it was. |
| `Prompt`, `PromptSpec` | What `definePrompt()` returns, and what it takes. |
| `CompiledPrompt`, `CompiledMessage` | What `compilePrompt()` produces. |
| `ResolvedPrompt` | What `resolve()` returns. |
| `PromptRegistryOptions`, `PromptWebhookConfig`, `PromptClientOptions` | Configure the registry, its webhooks, and the client. |

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/postgres.md

# Postgres


Postgres storage for everything that has to outlive a process or be shared between workers: durable
operations and graph checkpoints, long-term memory, traces and their hourly rollups, datasets and
experiments, shared circuit state, tenant usage, prompt versions, and retrieval vectors with pgvector. Every schema
is versioned, so an upgrade applies only what changed.

Each adapter has its own entry point and works with the Postgres client you already use. None creates
a schema until you ask. `PostgresPromptStore` is covered in the [prompts guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/prompts.md).

## Overview

One adapter family for everything that has to outlive a process or be shared between workers, over
the Postgres client you already have:

```ts
import pg from 'pg';
import { PostgresOperationStore, PostgresStore, PostgresTraceStore } from 'nexus-ai-pro/postgres';

const pool = new pg.Pool({ connectionString: process.env.DATABASE_URL });
const operations = new PostgresOperationStore(pool);
await operations.migrate(); // or: nexus db migrate --client db.mjs
```

| Adapter | Subpath | Serves |
| --- | --- | --- |
| `PostgresOperationStore` | `/postgres/operations` | durable operations, and graph checkpoints through `OperationStoreCheckpointer` |
| `PostgresStore` | `/postgres/store` | long-term memory, ranked by pgvector when `vectorDimensions` is set |
| `PostgresTraceStore` | `/postgres/traces` | traces, with every query filter in SQL |
| `PostgresDatasetStore`, `PostgresExperimentStore` | `/postgres/evaluate` | datasets and experiments |
| `PostgresCircuitStateStore` | `/postgres/circuits` | circuit state shared between workers, and the failure window they count together |
| `PostgresRollupStore` | `/postgres/rollups` | hourly totals of runs, for dashboards |
| `PostgresVectorStore` | `/postgres/vectors` | retrieval chunks, ranked by pgvector in the database |

**No driver is a dependency.** Each adapter takes anything with a `query(text, values)` method that
resolves to `{ rows }`: `pg`'s `Pool` and `Client`, `@neondatabase/serverless`, PGlite.
`fromPostgresJs(sql)` adapts `postgres.js`. Parameters are only ever strings, numbers, and `null`,
cast in SQL, so a driver's own conversion of arrays or dates never changes what is stored.

**Nothing creates a schema at import.** `migrate()` applies an adapter's pending migrations when you
call it, and `nexus db migrate` does it for every adapter. `postgresMigration()` and `nexus db sql`
produce the same SQL for the migration tooling you already use. Table names are options,
schema-qualified if you like.

**What Postgres adds over the other stores:**

- An operation update is one `UPDATE … WHERE sequence = expected`, atomic without a transaction.
- Idempotency keys are unique in the table. When two workers race to submit the same key, the loser
  attaches to the winner's operation instead of running the work twice.
- Trace feedback is appended in one statement, so two evaluators scoring one run at once both land.

A graph thread started on one worker and finished on another is the same code as before, pointed at a
different store:

```ts
const graph = builder.compile({ checkpointer: new OperationStoreCheckpointer(new PostgresOperationStore(pool)) });
```

The adapters are tested on every run against a real Postgres engine — PGlite, PostgreSQL compiled to
WebAssembly — with the same contract tests the in-memory stores pass.

## The client contract

`PostgresLikeClient` is the one method every adapter needs: `query(text, values)`, resolving to
`{ rows }` and, optionally, a `rowCount`. `PostgresJsLike` is the part of a `postgres.js` instance
that `fromPostgresJs()` adapts — its `unsafe(text, values)` — so the tagged-template driver works too:

```ts
import postgres from 'postgres';
import { fromPostgresJs, PostgresTraceStore } from 'nexus-ai-pro/postgres';

const traces = new PostgresTraceStore(fromPostgresJs(postgres(process.env.DATABASE_URL!)));
```

Every adapter selects jsonb as text and parses it itself, because drivers disagree about whether
they parse jsonb; reading text everywhere is what makes every driver return the same value.

## Options and schemas

Each adapter takes an options object whose `table` names its table, optionally as `schema.table`. A
name that is not letters, digits, and underscores is refused, so a table option can never inject SQL.

| Options | Fields |
| --- | --- |
| `PostgresOperationStoreOptions` | `table` (`nexus_operations`). |
| `PostgresStoreOptions` | `table` (`nexus_store`); the `index` that embeds items for semantic search; `vectorDimensions`, to store embeddings in a pgvector column and rank in the database; and a `now` clock for tests. |
| `PostgresTraceStoreOptions` | `table` (`nexus_runs`). |
| `PostgresEvaluationStoreOptions` | `datasetsTable` (`nexus_datasets`) and `experimentsTable` (`nexus_experiments`), shared by the dataset and experiment stores. |
| `PostgresCircuitStateStoreOptions` | `table` (`nexus_circuits`), and the clock for probe leases. It must be the breaker's own clock. |
| `PostgresPromptStoreOptions` | `table`, used as a prefix: `nexus_prompt` gives `nexus_prompt_versions` and the tables beside it. |
| `PostgresVectorStoreOptions` | `dimensions`, the width of every vector (required); `embed` (hashed term vectors by default); `table` (`nexus_vectors`); and `index`: `hnsw` (the default) searches without scanning every row, `none` scans exactly. |

Without `vectorDimensions`, `PostgresStore` keeps embeddings as JSON and ranks in the process. That
works on any Postgres and suits namespaces of a few thousand items.

### Versioned migrations

Each adapter's schema is a list of numbered migrations, and the database records which ran. A
`SchemaMigration` names its `component` — the adapter and its table, such as
`operations:nexus_operations` — its `version`, counted from 1 within the component, a `name`, and
its `statements`. Version 1 of every component is the schema 2.0 created, statement for statement,
so a database made by 2.0 or 2.1 is recognized as it is: its first migration records version 1 and
applies only what came later.

`migrate()` applies an adapter's pending migrations and resolves to a `MigrationResult`: the
migrations it `applied`, the `statements` it ran, and whether it was a `dryRun`. Each is recorded in
`nexus_schema_migrations` with `migrationChecksum()` of its statements, a hash that ignores line
endings. A migration whose statements changed after it ran — an edited migration, or adapter options
such as a vector width that differ from those it ran with — is refused with a `SchemaMigrationError`
before anything is written. Its `code` is `MIGRATION_CHANGED`, `MIGRATION_INVALID` for a list with a
gap or a duplicate, or `MIGRATION_LOCKED` when another migrator held the lock too long.

`PostgresMigrateOptions` tunes a run:

- `dryRun` plans without writing anything, not even the migrations table.
- `table` renames `nexus_schema_migrations`.
- `transaction` runs each migration and the row recording it in one transaction on one connection.
  Pass it for a pool, where `BEGIN` and `COMMIT` would otherwise reach different connections;
  PGlite needs only `transaction: (run) => db.transaction(run)`. Without it, statements run one at a
  time and the record is written last. Every bundled statement is idempotent, so a migration that was
  interrupted runs again next time.
- `lockTtlMs` and `lockTimeoutMs`: the lock is a row with a lease rather than a session advisory
  lock, so it holds through a connection pool and serverless drivers alike. A second migrator waits up
  to `lockTimeoutMs` (2 minutes), and the lease (10 minutes, renewed before each migration) lets one
  take over from a migrator that died.

`applyPostgresMigrations(client, migrations, options)` and `postgresMigrationStatus(client,
migrations)` do the same for any list, such as every adapter's together. A `MigrationStatus` lists
what is `applied` (each an `AppliedMigration` with its checksum and `appliedAt`), what is `pending`,
what `changed`, and what is `unknown`: migrations a newer release recorded. Unknown ones are expected
during a rollout and are left alone, and `current` is true when nothing is pending or changed. Both
functions live in `nexus-ai-pro/postgres/migrations` too, for an application that imports one adapter.

`postgresMigrations()` lists the migrations of the chosen adapters on their default tables, in the
order `POSTGRES_ADAPTERS` gives. `PostgresMigrationOptions` chooses the `adapters`, each a
`PostgresAdapter` name: `operations`, `store`, `traces`, `evaluation`, `circuits`, `prompts`,
`rollups`, or `vectors`. Every one but `vectors` is included by default, and `vectorDimensions` sets
the store's and the `vectors` table's width. A store on a custom table migrates through its own
`migrate()`, under a component of its own.

```ts
import { applyPostgresMigrations, postgresMigrations, postgresMigrationStatus } from 'nexus-ai-pro/postgres';

const migrations = postgresMigrations({ adapters: ['operations', 'traces', 'rollups'] });
const status = await postgresMigrationStatus(pool, migrations);
if (!status.current) await applyPostgresMigrations(pool, migrations, { transaction });
```

`nexus db status --client db.mjs` and `nexus db migrate --client db.mjs` do the same from a shell,
through a module that exports your client; the [CLI guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/cli.md) has the details.

What each version adds:

| Component | Version | What it does |
| --- | --- | --- |
| `operations:nexus_operations` | 1 | the 2.0 table |
| | 2 | indexes queued work by age, so a claim reads the oldest few rows instead of sorting the queue |
| | 3 | indexes queued work by tenant |
| `traces:nexus_runs` | 1 | the 2.0 table |
| | 2 | indexes runs by `metadata.tenantId`, which a tenant's trace queries read |
| | 3 | indexes runs in query order, start time then id, which a page resumed by cursor seeks to |
| `circuits:nexus_circuits` | 1 | the 2.0 table |
| | 2 | the failure window workers count together, and its streak |
| `rollups:nexus_rollups` | 1 | the rollups table |
| `tenancy:nexus_tenant_usage` | 1 | the usage table |
| | 2 | the budget reservations table beside it, `nexus_tenant_usage_reservations` |
| `store`, `evaluation`, `prompts`, `vectors` | 1 | the 2.0 tables |

**Schema changes expand, then contract.** A column or index is added in one minor release, written
by the next, and removed only in a major, so a worker of the previous minor keeps running against a
migrated database during a rollout. That is tested against the published packages: a database 2.0.0
created is migrated by 2.2, and a 2.1.0 worker and a 2.2 worker drain one queue together with nothing
run twice.

Each adapter still has a function returning its statements, for your own tooling, and one returning
its versions; both take the same table options:

| Adapter | Statements | Versions |
| --- | --- | --- |
| `PostgresOperationStore` | `operationStoreMigration()` | `operationStoreMigrations()` |
| `PostgresStore` | `storeMigration()`, which also creates the pgvector extension when `vectorDimensions` is set | `storeMigrations()` |
| `PostgresTraceStore` | `traceStoreMigration()` | `traceStoreMigrations()` |
| `PostgresDatasetStore`, `PostgresExperimentStore` | `evaluationStoreMigration()` | `evaluationStoreMigrations()` |
| `PostgresCircuitStateStore` | `circuitStoreMigration()` | `circuitStoreMigrations()` |
| `PostgresPromptStore` | `promptStoreMigration()` | `promptStoreMigrations()` |
| `PostgresRollupStore` | `rollupStoreMigration()` | `rollupStoreMigrations()` |
| `PostgresVectorStore` | `vectorStoreMigration()` | `vectorStoreMigrations()` |

`postgresMigration()` joins every chosen adapter's statements into one script, which is what
`nexus db sql` prints. Every statement is idempotent, so the script runs again safely after an
upgrade. `PostgresMigrationScriptOptions` adds `record`, which also creates the migrations table
— the statements `migrationsTableStatements()` returns — and records every migration in the script as
applied, so `nexus db status` agrees with a database your own tooling built
(`nexus db sql --record`).

```ts
import { writeFileSync } from 'node:fs';
import { postgresMigration } from 'nexus-ai-pro/postgres';

writeFileSync('migrations/0007_nexus.sql', postgresMigration({ adapters: ['operations', 'traces'], record: true }));
```

## Dashboard rollups

`PostgresRollupStore` keeps the hourly rollups `rollupTraceStore()` produces (see the
[tracing guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/tracing.md)) beside the traces themselves, so a dashboard's totals survive a restart
and every replica adds to the same rows. Adding to a row is one `INSERT … ON CONFLICT DO UPDATE`
that sums every total, latency buckets included, so two workers finishing runs in the same hour both
count. A query reads only the rows of its range and filters exactly as `MemoryRollupStore` does.
`PostgresRollupStoreOptions` has one field, `table` (`nexus_rollups`).

```ts
import { PostgresRollupStore } from 'nexus-ai-pro/postgres/rollups';
import { rollupTraceStore, sumRollups } from 'nexus-ai-pro/tracing/rollups';

const rollups = new PostgresRollupStore(pool);
await rollups.migrate();
const traces = rollupTraceStore(new PostgresTraceStore(pool), rollups);
const month = sumRollups(await rollups.query({ since: '2026-10-01T00:00:00Z', tenant: 'acme' }));
```

## Tenant usage

`PostgresTenantUsage` is a `TenantUsageStore` for `tenantLimits()` (see the
[deployments guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/deployments.md)). Every replica enforces one limit through it, which until now
needed Redis. Each key is one row, holding a tenant's active slots as a JSON map from run to lease
expiry, or its spending in the current period. Every operation is one statement on that row.
Postgres applies concurrent updates of one row one at a time, checking each against the newest
version, so two replicas admitting runs for one tenant at once cannot both take its last slot.
`PostgresTenantUsageOptions` has two fields: `table` (`nexus_tenant_usage`), and `now`, a clock for
tests. `tenantUsageMigrations()` and `tenantUsageMigration()` give the schema, as every adapter's
migrations do. Version 2 adds `<table>_reservations`, where a `budgetLedger()` over this store
records each budget reservation: a reservation and its total move in one statement, and of two
processes settling one reservation, the second finds it moved (see the
[lifecycle guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/lifecycle.md#reservations-that-outlive-their-process)).

```ts
import { PostgresTenantUsage } from 'nexus-ai-pro/postgres/tenancy';
import { tenantLimits } from 'nexus-ai-pro/server/tenancy';

const usage = new PostgresTenantUsage(pool);
await usage.migrate();
const tenants = tenantLimits({ default: { maxActiveRuns: 10, budget: { usd: 50, period: 'day' } }, usage });
```

The long-term store's `putIfVersion()` is one statement too, so a server whose state lives in
`PostgresStore` changes deployments atomically across replicas.

## Retrieval with pgvector

`PostgresVectorStore` implements the retrieval contract every vector store shares: `add()`, `search()`,
`searchVector()`, and `delete()`. The [grounding guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/grounding.md) covers the contract.

Only the nearest chunks leave the database, so a search costs the same for a thousand chunks or ten
million. Adding a chunk whose id exists replaces it. A metadata filter runs in SQL as a jsonb
containment check. A vector of the wrong width is refused before it reaches the database, and
`size()` counts the chunks.

```ts
import { PostgresVectorStore } from 'nexus-ai-pro/postgres/vectors';

const chunks = new PostgresVectorStore(pool, {
  dimensions: 1536,
  embed: toEmbeddingFunction(ai, { model: 'text-embedding-3-small' }),
});
await chunks.migrate();
await chunks.add(ingestDocuments(docs).chunks);
const context = await chunks.search(question, { topK: 5, filter: { tenant: 'acme' } });
```

## A shared embedding cache

`PostgresEmbeddingCache`, on `nexus-ai-pro/postgres/embedding-cache`, keeps embedding vectors for
every process that reaches the database. Give it to `embeddings.cache.store`, as the
[embeddings guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/embeddings.md#sharing-the-cache) shows. A batch is one `SELECT` and one
upsert, and a vector is packed float32 in a `bytea`: 6 KB for 1,536 dimensions.

```ts
import { PostgresEmbeddingCache } from 'nexus-ai-pro/postgres/embedding-cache';

const cache = new PostgresEmbeddingCache(pool, { table: 'app.embedding_cache' });
await cache.migrate();
const ai = new NexusAI({ providers, embeddings: { cache: { store: cache, ttlSeconds: 30 * 86_400 } } });
```

`PostgresEmbeddingCacheOptions` sets the `table` (`nexus_embedding_cache`).
`embeddingCacheMigrations()` returns its versioned schema, which `migrate()` applies. `purgeExpired()`
deletes what has lapsed; an expired vector is never returned before that.

## Keyword search

`PostgresKeywordIndex`, on `nexus-ai-pro/postgres/fulltext`, is keyword search with Postgres's own
full-text search: stemmed terms, a GIN index, and `ts_rank_cd` ranking from 0 to 1. Only the best
matches leave the database. It is a `SparseRetriever`, so `hybridRetriever()` fuses it with
`PostgresVectorStore`.

`PostgresKeywordIndexOptions`:
- `table`;
- `language`, the text search configuration, `english` by default;
- `match`: `any` finds chunks with any of the query's terms and ranks partial matches; `all` needs
  every term and reads the query as a web search, with quoted phrases and `-` exclusions;
- `shared`.

With `shared`, the index searches a `PostgresVectorStore`'s own table:
- `migrate()` adds a generated `tsvector` column and its index to that table;
- the store's writes keep the column current;
- `add()` and `delete()` leave writing to the store.

So the hybrid search runs over one copy of every chunk, and keyword and vector search never disagree
about what exists. `keywordIndexMigrations()` is the versioned schema, and `migrate()` applies it.

```ts
import { PostgresKeywordIndex } from 'nexus-ai-pro/postgres/fulltext';
import { hybridRetriever, vectorRetriever } from 'nexus-ai-pro/rag/retrievers';

const keywords = new PostgresKeywordIndex(pool, { table: 'nexus_vectors', shared: true });
await keywords.migrate();                          // after chunks.migrate()
const retriever = hybridRetriever([vectorRetriever(chunks), keywords]);
```

## Limitations

- Migrations only add. A column or index is never dropped in a minor release, so the schema of an
  upgraded database is a superset of what each release reads.
- `nexus db migrate` migrates the default tables. A store on a custom table migrates through its own
  `migrate()`, or through a `migrations` list the database module exports.
- Without `vectorDimensions`, semantic search in `PostgresStore` reads and ranks every item under the
  prefix in the process.
- The HNSW index is approximate: it can miss a match that an exact scan would return, and a metadata
  filter is applied to the candidates it returns. For small tables, or when every match must be
  found, use `index: 'none'`.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/sqlite.md

# SQLite


Durable persistence on one machine with no server to run: operation records — and through them graph
and workflow checkpoints — long-term memory, and retrieval vectors, in a SQLite file. Each adapter takes the database
you already open, so no SQLite driver is a dependency of this package.

```ts
import { DatabaseSync } from 'node:sqlite';
import { OperationStoreCheckpointer } from 'nexus-ai-pro/graph';
import { SqliteOperationStore, SqliteStore } from 'nexus-ai-pro/sqlite';

const db = new DatabaseSync('agent.db');
const operations = new SqliteOperationStore(db);
const memory = new SqliteStore(db, { index: { embed } });
await operations.migrate();
await memory.migrate();

const graph = builder.compile({ checkpointer: new OperationStoreCheckpointer(operations), store: memory });
```

A thread checkpointed this way survives a restart of the process, and a second process that opens
the same file continues it. Every update is one `UPDATE … WHERE sequence = expected`, so two processes
sharing the file cannot both advance the same operation or thread.

## Drivers

`SqliteLikeClient` is the contract the adapters use: `run()` for a statement that changes rows,
reporting how many, `all()` for a query, and `exec()` for a migration. Each may answer synchronously
or with a promise, and parameters are only ever a `SqliteValue` — a string, a number, `null`, or bytes,
which every driver binds as a blob.

- **`node:sqlite`** and **`better-sqlite3`** are passed directly: a handle with `prepare()` and `exec()`,
  the `SqliteDatabaseLike` shape, is adapted automatically. `fromSqliteDatabase()` is that adapter on
  its own; it caches each prepared statement, so a statement the adapters run on every call is
  compiled once.
- **libSQL** and Turso are asynchronous: `fromLibsql()` adapts a client with `execute()` and
  `executeMultiple()`, the `LibsqlLikeClient` shape.

```ts
import { createClient } from '@libsql/client';
import { fromLibsql, SqliteOperationStore } from 'nexus-ai-pro/sqlite';

const operations = new SqliteOperationStore(fromLibsql(createClient({ url: process.env.LIBSQL_URL! })));
```

## Operation records

`SqliteOperationStore` implements the operation store contract, the same one the memory, Redis, and
Postgres stores keep, and passes the same tests. It backs an `OperationRunner`, the agent server, and —
through `OperationStoreCheckpointer` — graphs and workflows.

It does what every operation store does:

- creates, reads, and updates records, with a compare-and-set on their sequence;
- finds lapsed leases, for recovery;
- enforces unique idempotency keys; a duplicate raises `OperationDuplicateError`;
- lists records, and `prune()` deletes finished ones older than a cutoff.

`SqliteOperationStoreOptions` names the `table` (`nexus_operations`). `sqliteOperationStoreMigration()`
returns its schema, for your own tooling, and `sqliteOperationStoreMigrations()` its versions: version
2 indexes queued work by age, so a claim reads the oldest few rows instead of sorting the queue, and
version 3 indexes it by tenant.

## Long-term memory

`SqliteStore` implements the long-term store contract: namespaced items with expiry, filters, text
queries, and semantic search through the embedding function in its `index`.

The namespace prefix and expiry are matched in SQL. Filters, text queries, and semantic ranking run in
the process, with the same helpers the in-memory store uses, so every store answers a search the same
way.

`SqliteStoreOptions` takes the `table` (`nexus_store`), the `index`, and a clock. `sweep()` deletes
expired items. `sqliteStoreMigration()` returns the schema, and `sqliteStoreMigrations()` its versions.

## Retrieval vectors

`SqliteVectorStore`, on `nexus-ai-pro/sqlite/vectors`, implements the `VectorStore` contract and passes
the same contract test as every other store. Vectors are stored as float32 blobs. Metadata filters run
in SQL against the metadata JSON, and match each value's type as well as its value.

`SqliteVectorStoreOptions` sets the `dimensions`, the `table` (`nexus_vectors`), the `embed` function,
and how similarity is computed:

| `search` | How it ranks | Needs |
| --- | --- | --- |
| `scan` (default) | In JavaScript, over the rows that pass the filter. | Nothing. |
| `sqlite-vec` | Inside SQLite, with `vec_distance_cosine()`, so only the best rows leave the database. | The sqlite-vec extension, loaded into the handle first. |

`sqliteVectorStoreMigration()` returns the table's schema, and `sqliteVectorStoreMigrations()` its
versions.

```ts
import Database from 'better-sqlite3';
import * as sqliteVec from 'sqlite-vec';
import { SqliteVectorStore } from 'nexus-ai-pro/sqlite/vectors';

const db = new Database('knowledge.db');
sqliteVec.load(db);
const vectors = new SqliteVectorStore(db, { dimensions: 1536, embed, search: 'sqlite-vec' });
await vectors.migrate();
```

## A shared embedding cache

`SqliteEmbeddingCache`, on `nexus-ai-pro/sqlite/embedding-cache`, keeps embedding vectors in a table,
for every process on one machine, or a team's through libSQL. Give it to `embeddings.cache.store`, as
the [embeddings guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/embeddings.md#sharing-the-cache) shows. Vectors are packed float32 blobs, and
a batch of keys is read in statements of 500.

```ts
import Database from 'better-sqlite3';
import { SqliteEmbeddingCache } from 'nexus-ai-pro/sqlite/embedding-cache';

const cache = new SqliteEmbeddingCache(new Database('embeddings.db'));
await cache.migrate();
```

`SqliteEmbeddingCacheOptions` sets the `table` (`nexus_embedding_cache`) and a clock for tests.
`sqliteEmbeddingCacheMigrations()` returns its versioned schema, and `purgeExpired()` deletes what has
lapsed.

## Versioned migrations

Each adapter's `migrate()` applies its pending migrations and records each in
`nexus_schema_migrations` with a checksum, as the Postgres adapters do; the
[Postgres guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/postgres.md) explains the shared types: `SchemaMigration`, `AppliedMigration`,
`MigrationStatus`, `MigrationResult`, `migrationChecksum()`, and `SchemaMigrationError`. Version 1 is
the schema 2.0 created, so a file made by 2.0 or 2.1 is recognized as it is, and an edited migration
is refused before anything runs.

Each migration runs inside `BEGIN IMMEDIATE`, which takes SQLite's write lock: its statements and the
row recording it land together or not at all, and a second process migrating the same file waits for
the first. `SqliteMigrateOptions` takes `dryRun` and the migrations `table`.

`sqliteMigrations()` lists the chosen adapters' migrations on their default tables; each is a
`SqliteAdapter` name, `operations`, `store`, or `vectors`, and all three are included by default.
`applySqliteMigrations()` and `sqliteMigrationStatus()` apply and inspect any list, from
`nexus-ai-pro/sqlite` or `nexus-ai-pro/sqlite/migrations`. `sqliteMigrationsTableStatement()` is the
statement that creates the migrations table, for your own tooling.

```ts
import { applySqliteMigrations, sqliteMigrations, sqliteMigrationStatus } from 'nexus-ai-pro/sqlite';

const status = await sqliteMigrationStatus(db, sqliteMigrations());
if (!status.current) await applySqliteMigrations(db, sqliteMigrations());
```

`nexus db migrate --client db.mjs` does the same from a shell; the [CLI guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/cli.md) has the
details.

## Limitations

- SQLite is one machine. Several processes can share a file, but workers on several machines need
  Redis or Postgres.
- Semantic search reads every item under the prefix and ranks it in the process, which suits
  namespaces of a few thousand items; `PostgresStore` with pgvector ranks in the database.
- Vector search is exact in both modes, which suits one node's corpus. Beyond a few hundred thousand
  chunks, use pgvector or a dedicated vector database.
- `node:sqlite` is still marked experimental by Node.js and prints a warning when first loaded.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/operations.md

# Durable operations and jobs


Background work that survives a restart, from `nexus-ai-pro/operations`. A submitted operation has an
id, a status, progress, cancellation, and a stream of events. Give it a store and a dispatcher, and
it survives a crash and can run on another worker.

When durability is not needed, the in-process helpers on `nexus-ai-pro/jobs` are lighter.

## Durable Operations

Long-running work — an image render, a batch, anything asynchronous — runs through one lifecycle:
`queued → running → succeeded | failed`, with `retrying`, `cancelling`, `cancelled`, and `expired`
covering the rest. `OperationRunner` owns it end to end, so a crashed worker does not lose work.

The default store is in-process, so the small case needs no infrastructure:

```ts
import { OperationRunner } from 'nexus-ai-pro/operations';

const runner = new OperationRunner<string>({ retry: { maxAttempts: 3, baseDelayMs: 500 } });

const handle = await runner.submit(async (context) => {
  context.report({ completed: 1, total: 3 });
  return doTheWork(context.signal);
});

for await (const event of handle.events()) {
  console.log(event.type, event.sequence);
}

const value = await handle.result();
```

Swapping the store makes the same code survive a restart. Nothing else changes:

```ts
import { OperationRunner } from 'nexus-ai-pro/operations';
import { BullMQOperationDispatcher, RedisOperationStore } from 'nexus-ai-pro/operations/adapters';

const runner = new OperationRunner({
  store: new RedisOperationStore(redis),
  dispatcher: new BullMQOperationDispatcher(queue),
  owner: process.env.HOSTNAME,
  leaseMs: 30_000,
  webhook: { url: 'https://app.example/hooks/operations', secret: process.env.HOOK_SECRET! },
});
```

**How restart survival actually works.** A worker claims a lease before running and heartbeats
while it works. If the process dies, the lease simply lapses; another worker's `recover()` sweep
finds the record and resumes it. There is no distributed lock — every store write is a
compare-and-set on the record's `sequence`, so two workers racing on the same operation cannot both
win.

```ts
// On startup, in each worker.
const resumed = await runner.recover(executor);
```

Recovery is deliberately conservative: a record past its `expiresAt` is **expired** rather than
re-run, and one that has used every attempt is **dead-lettered** and parked for inspection, so a
permanently failing operation cannot be recovered forever.

**Idempotency.** An `idempotencyKey` that matches an existing record replays that operation instead
of starting a second one, which is what stops an ambiguous timeout from double-charging:

```ts
const handle = await runner.submit(chargeAndRender, { idempotencyKey: `render:${orderId}` });
```

**Webhooks** are signed with HMAC-SHA256 over `${timestamp}.${body}`, so a captured delivery cannot
be replayed indefinitely. The verifying half ships too — a receiver should never have to hand-roll a
constant-time comparison:

```ts
import { OPERATION_WEBHOOK_SIGNATURE_HEADER, verifyOperationWebhook } from 'nexus-ai-pro/operations/webhooks';

app.post('/hooks/operations', (request, response) => {
  const ok = verifyOperationWebhook(
    request.rawBody,
    request.header(OPERATION_WEBHOOK_SIGNATURE_HEADER),
    process.env.HOOK_SECRET!,
  );
  response.sendStatus(ok ? 204 : 400);
});
```

A failed delivery is reported through `onWebhookError` and never turns a completed operation into a
failed one.

**Binary payloads are refused, not truncated.** Persisting a result that carries a `Uint8Array`,
`Buffer`, or `Blob` throws `OperationSerializationError` naming the exact path. Base64 in a job
payload inflates it by a third and most queue backends cap job size well below one image, so the
bytes belong in an `AssetStore` with only a reference on the record. The BullMQ dispatcher likewise
queues the operation id and nothing else.

The image family already runs on this lifecycle, so `ai.images.submit()` reports the same events —
through an in-process handle, not a store, so an image operation does not survive a restart.

## The lifecycle

This is the life of a durable record. It is not the [operation lifecycle](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/lifecycle.md) every call
runs through, which authorizes, budgets, and audits each call whether it is durable or not.

An `OperationStatus` is one of `queued`, `running`, `retrying`, `succeeded`, `failed`, `cancelling`,
`cancelled`, or `expired`.

Four are final — succeeded, failed, cancelled, and expired — and `TERMINAL_OPERATION_STATUSES` lists
them. Nothing leaves a final status, so a late provider callback cannot bring a cancelled operation
back. One subtlety: `cancelling` can still reach `succeeded` or `failed`, because a cancel asks the
executor to stop but cannot promise it stopped in time.

| Function | Answers |
| --- | --- |
| `isTerminalOperationStatus()`, `isSettled()` | Is the status final? |
| `isClaimable()` | May a worker start it? True for `queued` and `retrying`. |
| `canTransition()` | Is a move allowed? |
| `assertTransition()` | The same, throwing `OperationTransitionError` when not. |
| `allowedTransitions()` | Every move from a status, for a custom store or runner. |

## Running operations

`OperationRunner` takes an `OperationRunnerConfig`:

| Option | Default | What it sets |
| --- | --- | --- |
| `store` | memory | Where records live. |
| `dispatcher` | none | Hands work to other workers. |
| `owner` | — | This worker's name, recorded on its leases. |
| `leaseMs`, `heartbeatMs` | 30 s, a third of it | How long a claim lasts, and how often it is renewed. |
| `timeoutMs` | none | A limit for a whole operation. |
| `retry` | one attempt | An `OperationRetryConfig`: attempts, first and longest delay, fixed or exponential backoff, jitter, and an `isRetryable` test. By default everything but a cancellation is retried. |
| `webhook`, `onWebhookError` | — | Notifications, and where delivery failures are reported. |
| id generator, clock | — | For tests. |

The runner's methods:

| Method | What it does |
| --- | --- |
| `submit()` | Starts an operation here. Takes an `OperationExecutor` and `OperationSubmitOptions`. |
| `enqueue()` | Records an operation as queued without running it, for any worker to claim. |
| `claimQueued()` | Claims queued operations no worker holds, oldest first, and runs them here. |
| `release()` | Hands an operation this runner is executing back to the queue. |
| `read()` | Returns the stored record. |
| `cancel()` | Cancels by id, from any worker. |
| `recover()` | Resumes operations whose lease lapsed. Call it on each worker at startup. |

`executing` lists the ids this runner is running now.

`OperationSubmitOptions` can set an id, a `kind` such as `image.generate`, an `idempotencyKey`,
`maxAttempts` and `timeoutMs` for this operation, an `expiresAt`, trace headers, metadata, and a signal.

The executor is a function of an `OperationContext`, which gives it:

- the id and the attempt number;
- a signal, aborted on cancellation, timeout, or a lost lease;
- `report()` for progress, and `heartbeat()` to extend the lease before a long step;
- `previousHeartbeat`, what an earlier attempt last passed to `heartbeat()`;
- the trace headers and metadata from submission.

**Resuming from progress.** An attempt that fails is run again from the top. `heartbeat(details)`
stores `details` on the record as `heartbeatDetails`, and the next attempt reads them as
`previousHeartbeat`, so a long job resumes where it stopped instead of starting over:

```ts
await runner.submit(async (context) => {
  const from = (context.previousHeartbeat as { row: number } | undefined)?.row ?? 0;
  for (let row = from; row < rows.length; row += 1) {
    await importRow(rows[row]); // idempotent: keyed by the row
    await context.heartbeat({ row: row + 1 });
  }
});
```

The details must survive the store's serialization. They say where to resume, not that the work
before them happened exactly once, so the work itself still needs an idempotency key.

`submit()` returns a `DurableOperationHandle`. It implements the `OperationHandle` contract — `id`,
`status()`, `result()`, `cancel()`, and `events()` — and adds `record()` for the stored snapshot and
`progress()`.

An `OperationProgress` has any of a ratio, a count done and a total, a message, and metadata. Show
what is present rather than assuming a percentage.

`LocalOperationHandle` is the in-process handle the runner builds on, for a family that needs a handle
without a store. `LocalOperationHandleOptions` sets its clock, the error a cancellation rejects with,
and an event listener.

## Queued work and hand-offs

`submit()` runs an operation on the worker that accepted it. A pool of workers works the other way:
one process accepts work, and whichever worker is free runs it.

```ts
// The process that accepts work.
const record = await runner.enqueue({ kind: 'report.build', metadata: { reportId } });

// Each worker, whenever it has room.
const handles = await runner.claimQueued(buildReport, freeSlots, { kindPrefix: 'report.' });
```

`enqueue()` takes `OperationEnqueueOptions`, which are the submit options without a signal. It
resolves to the stored record, or to the existing one when the idempotency key matches.
`claimQueued()` claims up to `limit` operations, oldest first. Each claim is a compare-and-set, so
two workers polling the same store never run the same operation. It returns the handles it claimed.

`release()` is for a worker that is shutting down. It aborts the executor's signal with an
`OperationReleasedError`, and puts the record back in the queue as its next attempt. The hand-off
adds one to the attempt budget, so it does not use up a retry. An executor that sees that reason on
its signal should stop without recording a failure. No `failed` webhook is sent. The agent server's
`drain()` is built on it.

A worker never takes over a live lease another worker holds. Only lapsed leases are recovery's to
take.

## Events

Every `OperationEvent` carries the `OperationEventBase` fields: the operation id, the record's
sequence (for ordering and de-duplication), and a timestamp. `OperationEventType` names the kinds:

| Event | Extra fields |
| --- | --- |
| `queued` | — |
| `running` | The attempt. |
| `progress` | The progress. |
| `retrying` | The delay, and the error. |
| `cancelling`, `cancelled`, `expired` | — |
| `succeeded` | The result. |
| `failed` | The error, and whether it was dead-lettered. |

A failure is recorded as an `OperationErrorDescriptor`: the error's name, message, stable code, and
whether it was retryable. That shape survives a store and a webhook. `describeOperationError()` builds
one from any error.

## Records and stores

An `OperationRecord` is what a store keeps:

- the id, status, attempt, and attempt budget;
- the `sequence`, bumped on every write;
- when it was created, started, completed, and when it expires;
- the `OperationLease`: the owner, when it lapses, and the last heartbeat;
- progress, the result or error, and whether it was dead-lettered;
- the idempotency key, kind, trace headers, and metadata.

`assertSerializableRecord()` refuses a record that carries raw bytes, naming the path.

An `OperationStore` needs three methods: `create()`, `read()`, and `update()`. `update()` is a
compare-and-set: it returns `false` when another worker changed the record first. Optional methods add
`delete()`, `claimExpired()` for recovery, `findByIdempotencyKey()`, and `list()`.

Two more optional methods serve queues and autoscalers:

- `listQueued(limit, filter)` returns queued records no worker holds, oldest first.
- `stats(now, filter)` returns an `OperationStoreStats`: unfinished records per status, the oldest
  queued record's time, running records whose lease has lapsed, and queued records per kind.

An `OperationStoreFilter` narrows both by `kindPrefix`, such as `assistant:` for the agent server's
runs, and by `kinds`, a list of exact kinds. `claimQueued()` checks a candidate against the filter
itself too, so a store of your own that ignores a filter field never hands a worker the wrong work. `operationStats()` calls `stats()`, or counts `list()` for a store without it. Every store
here has both methods. Postgres and SQLite answer `listQueued()` from an index of queued work by age,
so a claim reads the oldest few rows however long the queue is. Redis does too, with its index;
without it, Redis reads every record.

Every store refuses a second record under an idempotency key another record holds, with
`OperationDuplicateError`, so ten workers submitting one key at the same moment run the work once and
all ten handles resolve to its result. The in-memory and Redis stores enforce it from 2.2 on; with
`eval`, Redis checks and writes in one Lua call.

| Store | Notes |
| --- | --- |
| `MemoryOperationStore` | The default. Evicts the oldest settled records beyond `MemoryOperationStoreOptions.maxRecords` (1,000). |
| `RedisOperationStore` | Takes a `RedisOperationLikeClient` (`hget`, `hset`, `hdel`, `hvals`, and optionally `eval`, `hmget`, and the sorted-set commands) and `RedisOperationStoreOptions` (a key `prefix`, `useEval`, and `index`). With `eval`, the compare-and-set is one atomic Lua call. Without it, a read-then-write narrows the race but cannot close it. |
| `PostgresOperationStore` | See the [Postgres guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/postgres.md). |
| `SqliteOperationStore` | See the [SQLite guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/sqlite.md). |

### The Redis dispatch index

With its index, `RedisOperationStore` keeps queued work in a sorted set by age and leased work in
one by lease expiry. `listQueued()` reads the oldest few records from the first, and `claimExpired()`
reads only lapsed leases from the second, so a claim reads the same handful of records with 100,000
operations queued as with 1,000. The records stay the source of truth: an index entry only says where
to look, and one that no longer matches its record is skipped, then removed by a Lua script that
checks the record in the same step. It needs `zadd`, `zrem`, and `zrangebyscore`, which `ioredis`
has; with `eval`, a record and its index entries change together.

It is on by default where the client has those commands, and `index: false` turns it off; 2.x kept it
off unless asked, as `useDefaults('2.x')` still does. A worker without the index writes records
without index entries, so what it queued is found once `reindex()` indexes what is already stored: call
it once no worker without the index shares the prefix. `reindex()` is safe while workers are busy, and
resolves to how many records it read.

```ts
const operations = new RedisOperationStore(redis);
await operations.reindex(); // once no worker without the index shares the prefix
```

An `OperationDispatcher` hands an accepted record to a worker. `BullMQOperationDispatcher` queues only
the id and routing metadata on a `BullMQLikeOperationQueue`. It uses the operation id as the job id, so
a duplicate dispatch is ignored. `BullMQOperationDispatcherOptions` sets the job name and options.

## Errors

Every operation error extends `OperationError` and has a stable code.

| Error | When | What to do |
| --- | --- | --- |
| `OperationCancelledError` | `result()` of a cancelled operation. | — |
| `OperationExpiredError` | `result()` of an expired operation. | — |
| `OperationConflictError` | A compare-and-set lost to another worker. | Reload the record; do not retry the write. |
| `OperationLeaseLostError` | Another worker took over the operation. | Stop working on it. |
| `OperationReleasedError` | The signal's reason when a worker released the operation. | Stop without recording a failure; another worker continues it. |
| `OperationDuplicateError` | A store refused a second record with the same idempotency key. | The runner attaches to the existing operation. |
| `OperationNotFoundError` | No record has that id. | — |
| `OperationTransitionError` | The lifecycle forbids the move. | A bug in a runner or store. |
| `OperationSerializationError` | A result carries raw bytes. | Put the bytes in an asset store. |

## Webhooks in detail

`OperationWebhookConfig` has the `url`, the `secret`, the `events` to send (terminal ones by default),
extra headers, a `fetch`, and a timeout (10 seconds).

| Function | What it does |
| --- | --- |
| `deliverOperationWebhook()` | Sends one event. Throws on a failed delivery, which the runner reports and moves past. |
| `signOperationWebhook()` | Builds the `t=…,v1=…` signature over the timestamp and the body. |
| `verifyOperationWebhook()` | Checks a signature in constant time. Rejects a delivery older than `VerifyOperationWebhookOptions.toleranceSeconds` (300). |

## In-process jobs

`nexus-ai-pro/jobs` holds lighter helpers. None of them survives a restart the way an operation does.

```ts
import { runBatch } from 'nexus-ai-pro/jobs/batch';

const results = await runBatch(documents, (doc) => ai.complete(summarizeRequest(doc)), { concurrency: 4 });
```

| Helper | What it does |
| --- | --- |
| `runBatch()` | Runs a worker over items with bounded concurrency. Returns a `BatchItemResult` per item, in input order: its index, whether it succeeded, and the value or error. `BatchOptions` sets `concurrency` (3) and `stopOnError`, which stops starting items after the first failure. |
| `JobQueue` | Runs a worker over enqueued payloads, with `QueueOptions` for `concurrency` (1), `maxAttempts` (1), and a client's `lifecycle`, which runs each attempt as a `job` operation. Each `QueueJob` has an id, the payload, its attempts, a status (`queued`, `running`, `completed`, or `failed`), and the result or error. `list()` and `get()` read them. |
| `DurableQueueAdapter` | Storage for job records: `enqueue()`, `get()`, `update()`, and `list()`. It stores jobs; running them is up to your worker. |
| `RedisQueueAdapter` | That storage in Redis, through a `RedisQueueLikeClient` (`lpush`, `hset`, `hget`, `hvals`). |
| `BullMQQueueAdapter` | Hands jobs to an existing BullMQ queue through the `BullMQLikeQueue` slice. BullMQ's own workers run them, and it reads a job's result back by id. |

When work must survive a restart or run on another worker, use an operation instead.

## Limitations

- A queue filter is applied while the Redis index is read in order, so a `kindPrefix` that matches
  little of a long shared queue reads far into it. Give very different queues their own `prefix`.
- The in-memory operation store and `JobQueue` live in one process. `SqliteOperationStore` shares
  records between processes on one machine; a distributed deployment needs `RedisOperationStore` or
  `PostgresOperationStore` and a dispatcher such as BullMQ.
- Recovery re-runs a whole attempt, not the step it died on, so an executor with side effects should
  be idempotent, using the operation id or its idempotency key. To lose only a step, run the work as
  a graph or a [functional workflow](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/graphs.md#functional-workflows), which checkpoints each step.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/resilience.md

# Resilience and observability


Staying up when providers do not: health-aware routing, a circuit breaker whose state can be shared across workers through `nexus-ai-pro/ops/circuit-store`, rate limits that hold across processes through `nexus-ai-pro/ops/rate-limit-adapters`, and the metrics, traces, audit log, and logger that show what happened.

## Resilience: Circuit Breaking and Distributed Limits

Health monitoring ranks a struggling provider lower. A circuit breaker is the stronger step: while a
circuit is open the provider is removed from routing entirely, so a hard-down provider stops
absorbing one failed attempt per request.

```ts
const ai = new NexusAI({
  providers: { openai: { apiKey: process.env.OPENAI_API_KEY! }, anthropic: { apiKey: process.env.ANTHROPIC_API_KEY! } },
  health: { enabled: true },
  circuitBreaker: {
    enabled: true,
    failureThreshold: 5,        // consecutive failures that open the circuit
    failureRateThreshold: 0.5,  // or half the calls failing in the window
    minimumThroughput: 10,
    resetTimeoutMs: 30_000,     // cooldown before a probe
    halfOpenMaxCalls: 1,
    onStateChange: (event) => logger.warn('circuit', event),
  },
});

ai.getCircuitBreakerStatus();   // state, failure rate, retryAt per provider
ai.resetCircuitBreaker('openai');
```

Two independent triggers, because they catch different failures. A consecutive count catches a
provider that is hard down. A failure *rate* catches one that fails half its calls without ever
failing several in a row — invisible to a consecutive counter. The rate is only considered once
`minimumThroughput` calls have been seen, since one failure out of two is not evidence of anything.

After the cooldown the circuit goes half-open and admits a limited number of probes. A successful
probe closes it; a failed probe reopens it and restarts the cooldown. If *every* circuit is open the
router routes anyway — that usually means a shared dependency is down, and one attempt beats a
certain failure with no attempt at all.

A request the caller cancelled never counts against the provider. `isFailure` keeps anything else
that is not the provider's fault out of the calculation:

```ts
circuitBreaker: {
  enabled: true,
  isFailure: (error) => !(error instanceof NexusProviderError && error.category === 'bad-response'),
}
```

Probe limits hold on every attempt, retries included: while a probe is in flight, other requests go
to the next provider instead of piling onto one that has only just come back.

**Sharing circuit state across workers.** By default each worker learns on its own. Give the breaker
a store and a provider that fails in one worker is taken out of routing in all of them, and when the
cooldown ends only one worker probes it:

```ts
import { RedisCircuitStateStore } from 'nexus-ai-pro/ops/circuit-store';

const ai = new NexusAI({
  providers,
  circuitBreaker: { enabled: true, store: new RedisCircuitStateStore(redis), workerId: process.env.HOSTNAME },
});
await ai.syncCircuitBreaker(); // optional: learn what is open elsewhere before the first request
```

Only decisions are shared — open, closed, and who may probe. Each worker still counts its own
failures, and every check stays synchronous: shared state is refreshed in the background at most once
a second, so the breaker never puts a network call in front of a request. If the store is
unreachable, each worker decides for itself rather than holding circuits open. `PostgresCircuitStateStore`
does the same in Postgres, and the coordination code loads only when a store is configured.

**Distributed rate limiting.** The built-in limiter is process-local, which multiplies the real
limit by the number of workers. Pointing it at a shared store fixes that, and the same budget then
covers completions and embeddings alike:

```ts
import { RedisRateLimitStore } from 'nexus-ai-pro/ops/rate-limit-adapters';

const ai = new NexusAI({
  providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } },
  rateLimit: {
    enabled: true,
    maxRequests: 100,
    windowMs: 60_000,
    key: 'userId',
    store: new RedisRateLimitStore(redis),
  },
});
```

Prefer a client exposing `eval`: the increment and the expiry then happen in one atomic round trip.
Without it the store falls back to `INCR` plus `PEXPIRE` and re-arms any missing TTL it sees, so a
crash between the two calls cannot block a key forever.

`NexusRateLimitError` carries `resetAt` and `retryAfterSeconds`, so a gateway can answer with a
real `Retry-After` header:

```ts
catch (error) {
  if (error instanceof NexusRateLimitError) {
    response.setHeader('Retry-After', String(error.retryAfterSeconds ?? 60));
  }
}
```

Omitting `store` keeps the original synchronous in-memory path, which costs no extra microtask per
request.

## Observability and Reliability

```ts
const response = await ai.complete({
  model: 'auto',
  messages: [{ role: 'user', content: 'Trace this request.' }],
});

console.log(response.meta.pipeline?.steps);
console.log(ai.getMetricsSnapshot());
console.log(ai.getPrometheusMetrics());
console.log(await ai.checkProviders());
```

Structured logging can be injected without replacing audit logs or metrics:

```ts
const ai = createNexusConfig()
  .openai(process.env.OPENAI_API_KEY!)
  .direct('gpt-5.4-mini')
  .logger({
    console: false,
    sink: (event) => myLogger.info(event),
  })
  .create();
```

Production controls include:

- request timeouts
- provider retries
- health-aware routing
- estimated cost budgets
- pipeline traces
- Prometheus metrics
- OpenTelemetry metrics and trace export helpers
- audit log sink
- structured logger sink

## The circuit breaker in detail

`CircuitBreakerConfig` is the client's `circuitBreaker` option. A circuit moves between three
`CircuitState` values — `closed`, `open`, and `half-open` — and these settings decide when:

| Option | Default | Meaning |
| --- | --- | --- |
| `failureThreshold` | 5 | Consecutive failures that open a circuit. |
| `failureRateThreshold` | — | Or a share of failing calls, over `windowMs`. |
| `windowMs` | 60 s | The rolling window for the failure rate. |
| `minimumThroughput` | 10 | Calls needed in the window before the rate counts. |
| `resetTimeoutMs` | 30 s | How long a circuit stays open. |
| `halfOpenMaxCalls` | 1 | Probe calls allowed while half-open. |
| `successThreshold` | 1 | Probe successes that close it again. |
| `isFailure` | — | Which errors count as failures. |
| `onStateChange` | — | Receives every `CircuitStateChange`: the provider, the states it moved from and to, when, and why. |

For sharing state between workers:

| Option | Default | Meaning |
| --- | --- | --- |
| `store` | — | The shared `CircuitStateStore`. |
| `shareObservations` | `false` | Count every worker's calls together, so nine failures across three workers count as nine. |
| `workerId` | — | This worker's name in the store. |
| `syncIntervalMs` | 1 s | How often shared state is pulled. |
| `probeLeaseMs` | 30 s | How long one worker holds the right to probe. |
| `onStoreError` | — | Receives store failures. |

`now` replaces the clock in tests.

`CircuitBreaker` also works on its own, around any call:

| Method | What it does |
| --- | --- |
| `allowRequest()` | May this call go? |
| `recordSuccess()`, `recordFailure()` | Report how it went. |
| `state()`, `isOpen()`, `openProviders()` | Read the circuits. |
| `snapshot()` | A `CircuitSnapshot` per provider: state, consecutive failures, calls and failures in the window, failure rate, when it opened, when it will probe, the last error, and with `shareObservations` the `shared` window every worker counts. |
| `reset()` | Closes one circuit, or all of them. |
| `sync()`, `flush()` | With a store: pull shared state, and wait for pending writes before shutting down. |

A `CircuitStateStore` holds the decisions every worker must agree on, and optionally the calls they
count together:

| Method | What it does |
| --- | --- |
| `read()` | Every `SharedCircuitState`: a provider, `open` or `closed`, when it opened, when and by which worker it was last written, and why. |
| `write()` | Records one provider's state. |
| `claimProbe()` | Lets exactly one worker probe an open circuit, for a lease. |
| `observe()` | Optional. Adds one worker's calls to the provider's shared window and returns the window across every worker. |

### Counting failures across workers

Without `shareObservations`, each worker counts only its own calls: three workers each seeing three
failures never reach a threshold of nine. With it, each worker adds what it saw to a rolling window in
the store, and the circuit opens everywhere once the shared count crosses `failureThreshold` — counted
as failures since any worker last succeeded — or `failureRateThreshold`. Checks stay synchronous:
calls are batched and reported when the worker syncs, so a worker learns the shared count at most
`syncIntervalMs` late, and a request never waits on the store.

A report is a `CircuitObservation`: calls and failures per window bucket, whether any call
succeeded, the failures after the last success, the window and bucket widths, and the time of the
circuit's last close, before which nothing counts, so the failures that opened a circuit do not
reopen it the moment a probe closes it. The store answers with a `CircuitWindow`: the calls and
failures in the window, and the consecutive failures. Every bundled store implements `observe()`:
the memory store in process, Redis in one Lua call (with `eval`), and Postgres in one statement, after
version 2 of its schema.

```ts
const ai = createNexus({
  circuitBreaker: { enabled: true, store: new RedisCircuitStateStore(redis), shareObservations: true },
});
```

Ten workers sharing one circuit are tested on all three stores: ten failures, one per worker, open the
circuit on all of them, exactly one probes after the cooldown, and its success closes it everywhere.

Two stores are included:

- `MemoryCircuitStateStore` shares state within one process, for tests and for several clients in one
  process.
- `RedisCircuitStateStore` shares it between processes. It takes a `RedisCircuitLikeClient` — `hgetall`,
  `hget`, `hset`, `set` with `PX` and `NX`, `get`, `del`, and optionally `eval`, in `ioredis` argument
  order — and `RedisCircuitStateStoreOptions`: a key `prefix`, and `useEval: false` to skip Lua.

With `eval`, a change is written only when it is newer than the stored one, in one step. Without it, a
stale change can briefly win; it heals after one cooldown.

## Rate limits in detail

`RateLimiter` counts calls keyed per user, per model, per tenant, or globally as the client's
`rateLimit.key` says, from a `RateLimitedRequest` (the model, user, and tenant). `check()` counts in memory, synchronously;
`checkAsync()` counts through the configured store. Both throw `NexusRateLimitError` once a bucket
is full, with `resetAt` set to when a call would pass again.

`rateLimit.algorithm` decides how calls are counted:

| Algorithm | Counts | At a window's edge |
| --- | --- | --- |
| `fixed-window` (the default) | Calls per window of `windowMs` | A burst at the end of one window and another at the start of the next pass nearly twice the limit in a moment |
| `gcra` | The token-bucket algorithm, kept as one timestamp per key: `maxRequests` per `windowMs` on average, and at most `burst` at once | There is no edge: two milliseconds earn back a fifth of a call at 100 a second |

```ts
rateLimit: { enabled: true, maxRequests: 100, windowMs: 1_000, algorithm: 'gcra', burst: 20, store }
```

`burst` defaults to `maxRequests`. With a store, GCRA needs the store's `gcra()`, which returns a
`RateLimitDecision`: whether the call may go, and from when a refused one would. Both bundled stores
have it. `RedisRateLimitStore` runs it as one Lua script on Redis's own clock, so every worker agrees
on the time and two cannot both take the last slot; it needs a client with `eval`. `gcraDecide()` is
the arithmetic on its own, for a store of your own.

A `RateLimitStore` has `hit(key, windowMs)`, which returns a `RateLimitHit`: the count in the current
window, and when the window resets. It may also have `reset()`, and `gcra()` for the GCRA algorithm.

- `MemoryRateLimitStore` keeps counters in the process, with `reset()` and `clear()`.
- `RedisRateLimitStore` shares them between processes. It takes a `RedisRateLimitLikeClient` (`incr`,
  `pexpire`, `pttl`, and optionally `eval`) and `RedisRateLimitStoreOptions` (a key `prefix` and
  `useEval`).

## Health

`HealthConfig` turns on provider health tracking. A provider is unhealthy after `failureThreshold`
consecutive failures (3), or when its score drops below `minScore` (20). The router ranks an unhealthy
provider lower.

The score starts at 100. It loses 20 per consecutive failure, up to 60, and 1 per second of average
latency, up to 30.

`ProviderHealthMonitor` does the tracking. `recordSuccess()` takes a latency and `recordFailure()` the
error; `score()`, `isHealthy()`, `isUnknown()`, and `snapshot()` read the result. Each
`ProviderHealthSnapshot` has the counts, consecutive failures, average latency, last error, score, a
`status`, and whether it is `stale`.

A `ProviderHealthStatus` is `unknown` with nothing recent to judge by, `unhealthy` past the failure
threshold or under the minimum score, `degraded` when calls fail or are slow but not enough to stop
routing to it, and `healthy` otherwise.

**Health that expires.** Health moves only with traffic. Without a limit, a provider that failed on
Monday and got no calls since is still unhealthy on Wednesday, and one that worked last week is still
healthy after it broke. `observationTtlMs` bounds how long an outcome counts. A provider whose last
call is older is `unknown` and `stale`, and is routed to as a provider never seen, with its last error
still on record. `ai.checkProviders({ staleOnly: true })` runs the health check of every unknown
provider; run it on a schedule to keep an idle provider current.

## Metrics and export

`MetricsConfig` turns metrics on, with a `prefix` for names (`nexus_ai` by default) and a `sink`. A
`MetricsSink` takes counters (`increment()`), histograms (`observe()`), and optionally gauges.
`InMemoryMetrics`, the default, keeps them for `getMetricsSnapshot()` and renders Prometheus text for
`getPrometheusMetrics()`. `MetricsCollector` is what the client records through: requests,
responses with latency and cost, errors, cache hits, and each pipeline step.

Every family records through the same collector, under the same labels: `family`, `operation`, and
the provider and model once routing chose them. So one query covers completions, embeddings, images,
voice, calls, and graph runs alike. `cache_hits` counts operations a cache answered without a provider.

OpenTelemetry is reached through structural interfaces, so no OpenTelemetry package is a dependency.

| Piece | What it does |
| --- | --- |
| `OpenTelemetryMetricsSink` | Sends metrics through an `OpenTelemetryLikeMeter` — anything with `createCounter()` and `createHistogram()` — creating each instrument on first use. |
| `OpenTelemetryTraceExporter` | Turns a pipeline trace into spans: one for the pipeline, and one per step with its timing, outcome, and metadata as attributes. It uses an `OpenTelemetryLikeTracer` (`startSpan()`) and `OpenTelemetryLikeSpan` (`setAttribute()` and `end()`). |

```ts
import { metrics, trace } from '@opentelemetry/api';
import { OpenTelemetryMetricsSink, OpenTelemetryTraceExporter } from 'nexus-ai-pro/ops';

const ai = new NexusAI({ providers, metrics: { enabled: true, sink: new OpenTelemetryMetricsSink(metrics.getMeter('app')) } });
const exporter = new OpenTelemetryTraceExporter(trace.getTracer('app'));

const response = await ai.complete(request);
if (response.meta.pipeline) exporter.exportTrace(response.meta.pipeline, { 'app.route': '/chat' });
```

## Audit log

`AuditLogger` writes the client's audit events to the `auditLog.sink`, or to the console without
one. Values are redacted — secrets, tokens, credentials, and personal data — unless
`includeSensitiveData` is set, and that is refused without an explicit sink, so raw data never lands
on a console by accident.

Every operation writes a `request` event when it is admitted. Then it writes one more: a `response`,
a `blocked` when a check refused it, or an `error` when it failed after it started. A `blocked` event
carries the guardrail findings that caused it.

## Other families

`FamilyTelemetry` runs an image, voice, or telephony call through the client's
[lifecycle](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/lifecycle.md): its authorization, budget, hooks, rate limit, audit log, and metrics.
`run()` takes a `FamilyCallDescriptor` — the operation, provider, model, user, tenant, request id,
metadata, signal, idempotency key, and an estimate for the budget — and a function that receives the
call's `ProviderCallContext`. `FamilyRuntime` is the wiring the client hands down: its lifecycle, or
the collector, audit logger, rate limiter, and rate-limit settings to build one from. A family reports
cost only when it can price the call in dollars.

## Limitations

- Failure counts are shared only with `shareObservations`; otherwise a shared store shares open,
  closed, and who may probe, and each worker counts its own failures. Shared, a worker learns the
  others' failures at its next sync, so it may let a call or two through, up to `syncIntervalMs`
  after the circuit opened elsewhere.
- The default fixed windows let a burst straddling a window boundary reach twice the limit for a
  moment; `algorithm: 'gcra'` does not. GCRA through Redis needs `eval`.
- Health comes from calls. Without `observationTtlMs`, a provider that receives no traffic keeps its last
  score; with it, an idle provider is `unknown` until a call or `checkProviders()` says more.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/batch.md

# Provider batch tiers


The providers' discounted batch tiers from `nexus-ai-pro/batch`: submit many requests at roughly half price, poll with backoff, collect and price the results, and resume a batch from its reference in another process. The OpenAI and Anthropic adapters have their own entry points, with a mock for tests.

## Provider Batch Tiers

Both OpenAI and Anthropic sell an asynchronous tier at roughly half price, in exchange for a
completion window measured in hours. Local `runBatch()` concurrency cannot reach it — it is a
different API. `BatchManager` puts both behind one operation handle.

```ts
import { createGraph } from 'nexus-ai-pro/graph';
import { BatchManager } from 'nexus-ai-pro/batch';
import { OpenAIBatchProvider } from 'nexus-ai-pro/batch/openai';

const batch = new BatchManager({
  providers: { openai: new OpenAIBatchProvider({ apiKey: process.env.OPENAI_API_KEY! }) },
  defaultProvider: 'openai',
});

const handle = await batch.submit({
  model: 'gpt-5.4-mini',
  idempotencyKey: `nightly-${date}`,
  items: documents.map((document) => ({
    customId: document.id,
    request: { model: 'gpt-5.4-mini', messages: [{ role: 'user', content: document.text }] },
  })),
});

console.log(handle.id);        // persist this, do not block a request on the result
```

`customId` is required, not optional. A batch provider does not guarantee output order, and matching
results by position is exactly the bug that silently mislabels every row. Duplicates are refused
before submission for the same reason.

The handle settles when the provider finishes, which can be hours later, so treat it as a background
operation:

```ts
const result = await handle.result();

result.status;              // completed | failed | expired | cancelled
result.counts;              // { total, completed, failed }
result.items;               // one entry per customId, with response or error
result.cost.amount;         // priced per item, then discounted at the provider's batch rate
```

A mixed batch is normal: individual items carry their own `error` while the batch still reports
`completed`. Nothing is invented for a failed batch — `items` comes back empty rather than padded.

**Surviving a restart.** Everything after `submit` takes only a `BatchJobRef`, which is
JSON-serializable. A worker that never submitted the batch can collect it:

```ts
const ref = { id: savedBatchId, provider: 'openai' };

await batch.status(ref);     // provider-side state, without waiting
const result = await batch.resume(ref);
await batch.cancel(ref);
```

Polling backs off from the configured interval up to `maxPollIntervalMs`, so a 24-hour batch does
not generate 2,880 polls while a fast one is still caught by the first few short intervals.

A submission records its batch the moment the provider accepts it. A worker that dies while it waits
is replaced by one that collects that batch, never a second, which the provider would run and bill.

## Configuring the manager

`BatchConfig` registers providers, names the default, and sets the poll interval, its upper bound
once backoff has grown it, and the timeout — 26 hours by default, just past the 24-hour tier.
`BatchManagerRuntime` supplies what the manager runs on: the operation runner configuration that
makes a submitted batch durable, the model registry it prices results against, and optionally a
client's `lifecycle`. With it, each submission is authorized before the provider sees it, and its
priced cost is charged to the client's budget when the batch is collected. Providers can also be added later with
`registerBatchProvider()`, and `hasBatchProvider()` and `listBatchProviders()` report what is
registered.

```ts
const batch = new BatchManager(
  { providers: { openai: new OpenAIBatchProvider({ apiKey }) }, defaultProvider: 'openai' },
  { operations: { store: new RedisOperationStore(redis) } },
);
```

## Statuses

A batch moves through `BatchJobStatus`: `validating`, `in_progress`, `finalizing`, then one of
`completed`, `failed`, `expired`, `cancelling`, or `cancelled`. `TERMINAL_BATCH_STATUSES` lists
the ones that never change again and `isTerminalBatchStatus()` is the check, which is what a poll
loop or a dashboard should use rather than comparing strings.

`BatchJobState` is a poll's answer: the `BatchJobRef`, the status, `BatchCounts` of total,
completed, and failed items, timestamps, any batch-level error, and the provider's raw payload.
`BatchJobResult` is the settled result, with one `BatchOutputItem` per `customId` — a response or
an error — and the priced cost.

## Errors

Every failure is a `BatchError` with a stable `code`:

| Error | When |
| --- | --- |
| `BatchValidationError` | A request is refused before it is sent: a missing or duplicated `customId`, or an empty batch. |
| `BatchProviderError` | The provider failed. |
| `BatchProviderNotFoundError` | No provider is registered under that name. |
| `BatchCapabilityError` | The provider cannot do what was asked, such as cancelling. |
| `BatchProviderResponseError` | The provider's answer does not have the shape the adapter expects. |

## Writing a provider

`BatchProvider` has four methods: `submit`, `poll`, `results`, and an optional `cancel`. Each takes a
`BatchProviderCallContext` with the abort signal and request id. A `BatchSubmitRequest` carries
`BatchInputItem` values in, and `BatchOutputItem` values come back.

`BatchProviderInfo` declares the provider's name and its `BatchProviderCapabilities`:

- how many items and bytes a batch may hold;
- which completion windows it offers;
- whether it can cancel;
- the discount it applies, which is what the manager prices with.

Two adapters are bundled. Both take an API key, a base URL, headers, and a replacement `fetch`:

| Adapter | Options | Also sets |
| --- | --- | --- |
| `OpenAIBatchProvider` | `OpenAIBatchProviderOptions` | The endpoint each item targets. Results include the failed requests, from the error file. Embeddings are batched on `nexus-ai-pro/embeddings/batch/openai`. |
| `AnthropicBatchProvider` | `AnthropicBatchProviderOptions` | The output token limit for an item that names none. |

## Testing without a provider

`MockBatchProvider` runs the whole submit, poll, and collect cycle in memory, in milliseconds, with no
account and no 24-hour wait. `MockBatchProviderOptions` scripts what each poll returns, which
`customId` values come back as errors, and the token usage to report. A test can then drive a mixed
batch, where some items fail, and check exactly what the manager does.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/caching.md

# Caching


Response caching, from `nexus-ai-pro/cache`. The exact cache answers a request it has seen before. The
semantic cache also answers a request that means nearly the same. Cached values live in memory, Redis,
or SQLite.

## Two caches, one switch

There are two response caches, and `strategy` picks which run:

| Strategy | What it reuses |
| --- | --- |
| `exact` | A request whose key matches exactly: same model, messages, and sampling settings |
| `semantic` | A request that *means* nearly the same as an earlier one |
| `hybrid` | Exact first, then semantic. The default worth using: an exact hit costs no embedding |

```ts
const ai = new NexusAI({
  providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } },
  cache: {
    enabled: true,
    strategy: 'hybrid',
    ttlSeconds: 600,
    semantic: { enabled: true, similarityThreshold: 0.86 },
  },
});

const response = await ai.complete(request);
response.meta.cacheHit; // true when the answer came from a cache
```

Caching is off until `enabled` is set, so nothing is reused by accident. `ai.clearExpiredCache()`
removes lapsed entries and returns how many went; `ai.getCacheStats()` reports size, capacity, and
expired entries not yet swept.

## Namespaces and outcomes

A cached response is reused only within its namespace. `CacheConfig.namespace` sets one, fixed or
computed per request, as a `CacheNamespace`: the `tenant` (the request's `tenantId` by default), the
`environment`, a `modelRevision`, and a `contextVersion`. Changing the revision or the context
version starts from an empty cache, so a prompt or model change never serves answers written for the
old one. Without a namespace, exact-cache keys are what they were before 2.2.

```ts
const ai = new NexusAI({
  providers,
  cache: {
    enabled: true,
    strategy: 'hybrid',
    namespace: (request) => ({ environment: 'production', contextVersion: String(request.metadata?.context ?? '') }),
  },
});
```

The semantic cache only ever matches entries of the same tenant, namespace or not: before 2.2, a
tenant asking another tenant's question could be answered with that tenant's response.

Every response carries what the cache did as `meta.cache`, a `CacheOutcome`, which the request's
trace records too:

| `outcome` | Meaning |
| --- | --- |
| `hit` | Answered from the exact or the semantic `layer`. |
| `miss` | Looked, found nothing. |
| `stale` | Found an entry that had expired. |
| `bypass` | The request skipped the lookup: `responseCache: 'bypass'` neither reads nor writes, and `'refresh'` writes the fresh answer. |
| `error` | The cache failed; the request went to the provider as on a miss. |

Each has a `reason` and the `namespace` it ran in, and `writeError` says when storing the answer
failed — the answer is returned either way. A cache that throws never fails a request.

This is the library's own cache, which stores whole responses. It is unrelated to provider-side
prompt caching, which discounts the *input* tokens of a long shared prefix and is covered in the
[client guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/core.md).

## The exact cache

`MemoryCache` is the store behind it: a bounded cache with per-entry expiry that evicts the least
recently used entry when full, holding 500 entries by default. `CacheEntry` is what it holds — the
value and when it expires — and its `get`, `set`, `delete`, `clear`, `clearExpired`, and `stats`
methods are the whole surface, with `lookup()`, which also says whether a missing value had expired.

The key is what decides whether two requests are "the same". `createCacheKey()` builds it from any
value with object keys sorted, so two requests that differ only in property order share a key, and
circular structures and dates do not throw:

```ts
import { createCacheKey, MemoryCache } from 'nexus-ai-pro/cache/memory-cache';

const cache = new MemoryCache<string>(1_000);
cache.set(createCacheKey({ model, messages }), answer, 300);
```

## The semantic cache

`SemanticCache` answers a request that means nearly the same as an earlier one. It embeds the
request, compares it with what it holds, and returns the closest entry above `similarityThreshold`.
The default, 0.88, is high enough that "reset my password" never answers "cancel my account".
`SemanticCacheOptions` also sets `maxEntries`, `ttlSeconds`, and `embed`. `get()` and `set()` take a
`scope`, and a lookup compares only entries stored under the same one; the client passes the
request's tenant and namespace.

`embed` is the part worth configuring. The default is hashed term vectors, which need no provider and
suit tests; for production, pass an embedding function so similarity means what you expect:

```ts
import { SemanticCache } from 'nexus-ai-pro/cache/semantic-cache';
import { toEmbeddingFunction } from 'nexus-ai-pro/embeddings';

const cache = new SemanticCache({
  enabled: true,
  similarityThreshold: 0.9,
  embed: toEmbeddingFunction(ai, { model: 'text-embedding-3-small' }),
});
```

A semantic lookup costs an embedding call, which is why `hybrid` tries the exact key first.

## Where cached values live

`CacheAdapter` is the storage contract: `get`, `set` with a TTL, `delete`, and `clear`, each of which
may be synchronous or asynchronous. Three adapters ship:

| Adapter | Entry point | Use it for |
| --- | --- | --- |
| `MemoryCacheAdapter` | `/cache/adapters` | One process. Expired entries are dropped when read |
| `RedisCacheAdapter` | `/cache/adapters` | Sharing a cache between processes; values are JSON under a key prefix |
| `SQLiteCacheAdapter` | `/cache/adapters` | One machine, surviving restarts |

The Redis and SQLite adapters take the client you already have, so neither library is a dependency.
`RedisLikeClient` is the `get`, `set`, and `del` of an `ioredis`-style client. `SQLiteLikeDatabase` is the
`prepare` and `exec` of a `better-sqlite3` handle:

```ts
import { RedisCacheAdapter, SQLiteCacheAdapter } from 'nexus-ai-pro/cache/adapters';

const shared = new RedisCacheAdapter(redis, 'nexus:cache:');
const local = new SQLiteCacheAdapter(new Database('cache.db'));
```

Writing your own adapter is implementing those four methods against whatever store you already run.

## Limitations

- A process-memory cache is per replica: two replicas each keep their own, and neither sees the
  other's hits. Use the Redis adapter when that matters.
- Default hash embeddings make the semantic cache cheap but crude. Give it a real embedding function
  before trusting it in production.
- Keep the semantic cache off latency-sensitive routes unless the embedding call is cheaper than the
  completion it saves.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/grounding.md

# Grounding: retrieval, citations, and verification


Grounding makes an answer follow from evidence, and catches it when it does not. Four steps:

1. **Ingest.** Split documents into chunks.
2. **Retrieve.** A vector store finds the chunks that match a question.
3. **Answer.** `withRagContext()` puts them in front of the model, with rules for citing them.
4. **Check.** The verification helpers test what came back.

Every export here comes from the root import, except ingestion, which is on `nexus-ai-pro/rag`.

```ts
import { withRagContext } from 'nexus-ai-pro/grounding';
import { MemoryVectorStore } from 'nexus-ai-pro/rag';

const store = new MemoryVectorStore();
await store.add([{ id: 'doc-1', content: 'NexusAI supports RAG context with citations.', source: 'docs' }]);

const chunks = await store.search('How does NexusAI ground answers?', { topK: 3 });

const response = await ai.completeVerified(
  withRagContext(
    { model: 'auto', messages: [{ role: 'user', content: 'How does NexusAI ground answers?' }] },
    { chunks, requireCitations: true },
  ),
  { context: chunks.map((chunk) => chunk.content), minSupportRatio: 0.85 },
);
```

## Ingesting documents

`ingestDocuments()` and `ingestText()` split text into overlapping chunks. They return `RagChunk` values
ready for a vector store, and an `IngestionResult` that says how much was read.

Each `DocumentSource` is a text, with an optional id, source, and metadata. All of them are carried
onto its chunks. `IngestionOptions` sets the split:

| Option | Default | What it does |
| --- | --- | --- |
| `chunkSize` | 1,200 characters | How long a chunk may be. |
| `overlap` | 150 | How much consecutive chunks share. |
| `splitOnMarkdownHeadings` | off | Keeps every chunk inside one section. |

To read files, directories, web pages, sitemaps, CSV, JSON, PDF, and Git repositories, use the
[loaders](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/loaders.md). `loadIntoStore()` streams what they read through this split into one or more
stores.

### Scanning uploads first

`ingestFilesAfterScan()` goes one step earlier, for files people upload:

1. It scans the files for unsafe content. Anything high or critical throws, before any file is read.
2. It extracts each file's text, and skips what it cannot read.
3. It chunks the text.

```ts
import { ingestFilesAfterScan, createPdfExtractor } from 'nexus-ai-pro/rag/files';

const { chunks, skippedFiles } = await ingestFilesAfterScan(uploads, {
  extractors: [createPdfExtractor(myPdfToText)],
  chunkSize: 1200,
  splitOnMarkdownHeadings: true,
});
```

`FileIngestionOptions` adds the scan settings, and the extractors to try in order, to the split
options. Text files need no extractor. `FileIngestionResult` adds how many files were scanned and, for
each skipped file, its name and why.

`createPdfExtractor()` and `createOcrExtractor()` wrap your own PDF or OCR function as a
`FileTextExtractor`. So the library carries no parsing dependency of its own.

## Retrieval

Every retrieval store implements one contract, `VectorStore`:

| Method | What it does |
| --- | --- |
| `add()` | Stores chunks. A chunk whose id exists is replaced, so re-ingesting a document never duplicates it. |
| `search()` | Finds the chunks closest to a text. |
| `searchVector()` | Finds the chunks closest to a vector computed elsewhere. |
| `delete()` | Removes chunks by id. |

`VectorSearchOptions` sets `topK` (5 by default) and `minScore` (0). Its `filter` matches top-level
metadata fields exactly, such as `{ tenant: 'acme' }`; `matchesMetadata()` is that test on its own.

A `VectorDocument` is a chunk with an optional precomputed embedding. A `VectorSearchResult` is a chunk
with its cosine similarity.

These stores implement it, and pass the same contract tests:

| Store | Entry point | Where the vectors live |
| --- | --- | --- |
| `MemoryVectorStore` | `nexus-ai-pro/rag` | In process. For development, tests, and a few thousand chunks. |
| `PostgresVectorStore` | `nexus-ai-pro/postgres/vectors` | Postgres with pgvector, ranked in the database. See the [Postgres guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/postgres.md). |
| `QdrantVectorStore` | `nexus-ai-pro/rag/qdrant` | A Qdrant collection, through its REST API. |
| `SqliteVectorStore` | `nexus-ai-pro/sqlite/vectors` | A SQLite file, optionally ranked by sqlite-vec. See the [SQLite guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/sqlite.md). |

Redis, Pinecone, Weaviate, and Chroma are in the [retrieval guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/retrieval.md), with hybrid search,
reranking, and the other retrievers.

### Embeddings

Each store takes any `EmbeddingProvider`:

- `toEmbeddingFunction()` adapts the embeddings family, so retrieval inherits its routing, batching,
  and caching.
- Without one, a store falls back to `createHashEmbeddings()`: hashed term vectors that are
  deterministic and need no provider. They suit tests, not production.

`cosineSimilarity()` and `normalizeVector()` are the arithmetic underneath.

### Qdrant

```ts
import { ingestDocuments } from 'nexus-ai-pro/rag';
import { QdrantVectorStore } from 'nexus-ai-pro/rag/qdrant';

const store = new QdrantVectorStore({
  url: process.env.QDRANT_URL!,
  apiKey: process.env.QDRANT_API_KEY,
  collection: 'support-docs',
  dimensions: 1536,
  embed: toEmbeddingFunction(ai, { model: 'text-embedding-3-small' }),
});
await store.migrate({ filterFields: ['tenant'] });

await store.add(ingestDocuments(docs).chunks);
const chunks = await store.search(question, { topK: 5, filter: { tenant: 'acme' } });
const answer = await ai.complete(withRagContext({ model: 'auto', messages: [{ role: 'user', content: question }] }, { chunks }));
```

`QdrantVectorStore` needs no Qdrant client. It speaks HTTP through `fetch` and hashes ids with Web
Crypto, so it runs on edge runtimes too. `QdrantVectorStoreOptions` takes the `url`, the `collection`,
the `dimensions`, an `apiKey`, the `embed` function, a `fetch`, extra `headers`, and a `timeoutMs`
(30 seconds).

`migrate()` creates the collection with cosine distance unless it exists. It also indexes the metadata
fields you filter on: a list as keywords, or a map naming each field's type.

Qdrant point ids must be UUIDs, so each chunk id maps to a stable UUID, and the original id travels in
the payload. A failed request raises `QdrantError`, with the status and Qdrant's response body.

## Answering from context

`withRagContext()` turns a request into a grounded one:

- the chunks become a system message;
- the model is told to answer only from them, and to cite them by id;
- sampling defaults low.

`RagOptions` sets the citation style, how many chunks are included, and what the model should say when
the context does not answer the question. `extractCitations()` reads the citations back out of an
answer. `validateCitations()` checks that each one names a chunk that was actually supplied.

### From a knowledge graph

`withKnowledgeGraphContext()` does the same for relationships instead of passages. It ranks a
`KnowledgeGraph` of `KnowledgeGraphNode` and `KnowledgeGraphEdge` values against the question, and
states the best ones as facts. It tells the model not to infer relationships the graph does not
contain. `selectGraphFacts()` is the ranking on its own, and `KnowledgeGraphOptions` configures it.

### With no retrieval

`withFactualDefaults()` is lighter, for when there is no retrieval at all. It asks for conservative
answers, sets an explicit "I do not know" fallback, and keeps sampling low. `FactualOptions` sets the
wording and worked examples. `asJsonOnly()` does the same for JSON-only output.

## Checking the answer

| Function | What it does |
| --- | --- |
| `verifyAgainstContext()` | Splits an answer into claims and checks each against the context. |
| `completeVerified()` | Completes, verifies, and asks once for a revision when claims are unsupported. |
| `completeWithSelfConsistency()` | Samples several answers and returns the one the others agree with most. |

`verifyAgainstContext()` splits the answer with `extractFacts()`. It returns a `VerificationReport` of
`VerificationFact` values, and the share that is supported. The default check is
`lexicalEntailment()`: term overlap, with no model. For a real entailment model, supply an
`NliVerifier`.

`completeVerified()` runs that whole loop, and attaches the report to the response's
`meta.verification`. `VerificationOptions` sets the minimum share supported and the fallback answer.
`VerificationClient` is the small client contract it needs.

`completeWithSelfConsistency()` attacks the problem from the other side. It picks the most consistent
answer with `selectMostConsistent()` and `textSimilarity()`, or with a judge you supply in
`SelfConsistencyOptions`. It survives failed samples as long as one succeeds. `ConsistencyClient` is
its minimal client contract.

## Limitations

- The default hash embeddings suit tests and demos, not real semantic search. Use a real embedding
  provider for production retrieval.
- NLI verification is an interface, not a model. The bundled check is lexical, so bring a specialized
  verifier when you need high-confidence entailment.
- `MemoryVectorStore` lives in one process. Use a real vector database, or the long-term store with an
  index, for anything shared or persistent.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/loaders.md

# Loaders


Getting documents into retrieval: files, pages, rows, records, PDFs, web pages, and Git
repositories, each read into the `DocumentSource` values that ingestion splits into chunks. Every
format has its own entry point, and none adds a dependency — PDF parsing is a function you inject,
and web pages go through the same SSRF-safe fetch as the `fetch_url` tool.

```ts
import { MemoryVectorStore } from 'nexus-ai-pro/rag';
import { loadIntoStore } from 'nexus-ai-pro/loaders';
import { loadMarkdown } from 'nexus-ai-pro/loaders/markdown';
import { loadDirectory } from 'nexus-ai-pro/loaders/text';
import { loadSitemap } from 'nexus-ai-pro/loaders/web';

const store = new MemoryVectorStore(embed);
await loadIntoStore(
  store,
  [
    loadDirectory('./docs', { parsers: { '.md': (file) => loadMarkdown(file) } }),
    loadSitemap('https://example.com/sitemap.xml', { include: (url) => url.includes('/help/') }),
  ],
  { splitOnMarkdownHeadings: true },
);
```

## Loading into a store

Every loader returns a `DocumentLoader`: an async iterable of `DocumentSource` values, each with an id,
the text, a source, and metadata. Documents arrive one at a time, so a corpus larger than memory
streams through instead of being read whole. A plain array of documents is a loader too.

`loadIntoStore()` streams loaders into one or more stores. Each batch of documents is split with the
ingestion options and added to every store at once. So a vector store and the `KeywordIndex` that
hybrid search reads fill up in one pass. A store is any `ChunkSink`: anything with an `add()` for
chunks.

`LoadIntoStoreOptions` adds a `batchSize` (32 documents), a `signal` that stops between batches, and an
`onBatch` progress callback. The `LoadIntoStoreResult` counts documents, chunks, and characters.

Reloading is safe. Chunk ids come from document ids, and loaders use the file path, URL, or row key as
the id. Loading the same corpus again replaces its chunks instead of duplicating them.

`collectDocuments()` reads loaders into an array instead, for inspection or for code that wants
the documents themselves.

## Keeping an index current

`loadIntoStore()` embeds everything it reads, every time. For a corpus that changes, use the durable
pipeline on `nexus-ai-pro/rag/pipeline`.

`createIngestionPipeline()` keeps a manifest in any `Store`. For each document, an
`IngestedDocument` records:
- the version of its content;
- its chunking;
- the embedding model;
- each chunk's id with the version of its text.

A run of the `IngestionPipeline` does six things:
- It reads the corpus, from any `IngestionSource`: a loader, an array, or an iterable.
- It skips every document whose content, chunking, and model are as recorded.
- It embeds only the chunks that are new or changed. A chunk is named by its text, so one whose text
  did not change keeps its embedding wherever it moved.
- It deletes the chunks a document no longer has.
- It deletes the documents the corpus no longer has, unless `deleteMissing: false` says the run
  loads only part of it.
- It writes every change to the stores before the manifest records it, so a crash repeats a write
  but never loses one.

A different embedding model, or different chunking, re-embeds everything. Chunks are cut by length,
within each Markdown section when `splitOnMarkdownHeadings` is on. An edit therefore re-embeds the
chunks of its own section, and adding a section leaves the others alone.

`IngestionPipelineOptions`:
- the pipeline's `name`, which keeps two manifests apart;
- the `manifest` store;
- the vector store;
- any keyword indexes kept in step;
- the `embed` function and the `embeddingModel`, or a `batchEmbed` tier instead of `embed`;
- the chunking and the batch size.

A `MemoryStore` keeps 10,000 items unless given a larger `maxItems`. A manifest has one item per
document, so size it to the corpus or keep it in a database.

`IngestionRunOptions`: a `signal`, `deleteMissing`, `onProgress`, and the durable `operation` the
run belongs to. `run()` returns an `IngestionReport`, which counts:
- documents read, unchanged, added, updated, and deleted;
- chunks embedded, kept, and deleted;
- documents a resumed run skipped;
- with a batch tier, the provider `batches` submitted, and the `batchesRecovered` from an earlier
  attempt.

`document()` reads one document's manifest record, and `remove()` removes documents from every
store.

`ingestionExecutor()` makes the pipeline the executor of a durable operation. Submit it to an
`OperationRunner`. After each batch, the run records how far it got, and a retry, on this worker or
another, starts at the document the failed attempt stopped on.

```ts
import { OperationRunner } from 'nexus-ai-pro/operations';
import { loadDirectory } from 'nexus-ai-pro/loaders/text';
import { createIngestionPipeline, ingestionExecutor } from 'nexus-ai-pro/rag/pipeline';

const pipeline = createIngestionPipeline({
  name: 'help-center',
  manifest: store,
  vectors,
  keywords: [elasticsearch],
  embed,
  embeddingModel: 'text-embedding-3-small',
  chunking: { splitOnMarkdownHeadings: true },
});
const runner = new OperationRunner({ store: operations, retry: { maxAttempts: 5 } });
const handle = await runner.submit(ingestionExecutor(pipeline, () => loadDirectory('./docs')), {
  idempotencyKey: 'help-center',
});
```

In the test suite, an ingestion of 100,000 documents dies halfway, and its retry embeds no chunk that
had finished. A second run after ten documents are edited embeds exactly ten chunks.

### Through a provider's batch tier

A large ingestion can embed at a provider's batch price, usually half, if it can wait hours instead
of seconds. Give `batchEmbed` a `batchEmbedder()` from the
[embeddings guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/embeddings.md#a-batch-tier-for-large-jobs) in place of `embed`:

```ts
import { batchEmbedder, EmbeddingBatchManager } from 'nexus-ai-pro/embeddings/batch';
import { OpenAIEmbeddingBatchProvider } from 'nexus-ai-pro/embeddings/batch/openai';

const batches = new EmbeddingBatchManager({ providers: { openai: new OpenAIEmbeddingBatchProvider({ apiKey }) } });
const pipeline = createIngestionPipeline({
  name: 'archive',
  manifest: store,
  vectors,
  batchEmbed: batchEmbedder(batches, { model: 'text-embedding-3-small', size: 20_000 }),
  batchesInFlight: 4,
  embeddingModel: 'text-embedding-3-small',
});
```

The run gathers the chunks to embed into windows of the embedder's `size`, and submits each window
as one provider batch. It writes each window once its batch completes, in order, with
`batchesInFlight` (4) batches outstanding at most. A batch that does not complete, or fails any
input, fails the run before its window is written.

Inside a durable operation, each batch is recorded the moment it is submitted. A retry rebuilds the
same windows from the same corpus, and collects a recorded batch instead of submitting it again. Only
a batch submitted in the instant before a crash, and not yet recorded, can be submitted twice. In the
test suite, an ingestion of 100,000 chunks dies while it collects its third window, and across both
attempts each of its ten windows is submitted once.

## Files

The file loaders take one input or many. A `FileInput` is one of:

- a path, or a `file:` URL;
- content already in memory, with a name for it: `{ source: 'upload.md', content }`. That is how a
  loader reads an upload, a database blob, or a file on a runtime without a filesystem;
- a stream of it, with a name: `{ source: 'export.csv', stream }`, any iterable or async iterable of
  bytes or text, such as a response's body or an object store's download.

`node:fs` is imported only when a path is read. `FileLoaderOptions` adds `metadata` to every document.
The CSV and JSON loaders read a file as it arrives, a path in chunks of 64 KiB and a stream piece by
piece, and yield each row or record as soon as it is read, so a file larger than memory streams through
to the store. The other loaders read a file whole, a stream included.

- **Text.** `loadText()` reads plain text files, one document each, dropping a byte-order mark.
- **Directories.** `loadDirectory()` walks a directory in sorted order, so a load is reproducible. It
  uses paths relative to the directory as ids. `DirectoryLoaderOptions` sets:
  - `extensions`, the files to read;
  - `parsers`, a map from extension to a `FileParser` — any loader given one file, such as
    `loadMarkdown`;
  - `recursive`, and an `ignore` test (by default, `node_modules` and dot directories are skipped);
  - `maxFiles` (10,000), a guard against pointing at the wrong directory.
- **Markdown.** `loadMarkdown()` reads Markdown with its front matter and title in the metadata, so a
  filter can select on them; ingest with `splitOnMarkdownHeadings` so no chunk spans two sections.
  `parseMarkdown()` is the parser on its own, returning a `MarkdownDocument` with the body, the
  `frontMatter` fields, and the `title`. Front matter is read as flat `key: value` lines — strings,
  numbers, booleans, and inline lists — so no YAML parser is needed. `MarkdownLoaderOptions` can set
  `plainText`, which runs `markdownToPlainText()` to strip link targets, emphasis, and fences.
- **HTML.** `loadHtml()` reads HTML files, with the page title and description in the metadata.
  `htmlToText()` does the conversion without a DOM. It drops scripts, styles, and the head, turns
  block elements into line breaks, and decodes entities. Headings become Markdown headings, so HTML
  splits by section too. It returns an `HtmlText`: the text, `title`, `description`, and every link.
  `HtmlToTextOptions` keeps only `<main>` or `<article>` by default (`mainContent`), and resolves links
  against a `baseUrl`. `HtmlLoaderOptions` combines them with the file options.
- **CSV.** `loadCsv()` makes one document per row, writing each column as `name: value` so a model
  reading a chunk knows what each value is. `CsvLoaderOptions` chooses the `textColumns`, an
  `idColumn` for stable ids, and `metadataColumns` to filter on. `parseCsv()` is an RFC 4180 parser
  — quoted delimiters, line breaks, and doubled quotes — and `CsvParseOptions` sets the `delimiter`.
  `CsvRowParser` is the same parser given text a piece at a time: `push()` returns the rows a piece
  completed and `end()` the last, a quote or a CRLF split between two pieces read as one.
- **JSON.** `loadJson()` makes one document per record, from a JSON file or JSON Lines. The items of
  a top-level array, or of the array at `recordsAt`, and the lines of JSON Lines, are read one at a
  time as the file arrives; a file that is not an array is one record. `JsonLoaderOptions` sets:
  - `recordsAt`, object keys from the top, dotted, such as `data.items`, to the array of records. A
    file with no array there is refused;
  - `records`, which picks the records out of the parsed value instead, reading the file whole;
  - `text` and `id`, each a field by dotted path, or a function;
  - `metadataFields`, copied into the metadata;
  - `lines`, which reads one value per line. On by default for `.jsonl` and `.ndjson`; a line that
    does not parse is named by its number.

```ts
import { loadJson } from 'nexus-ai-pro/loaders/json';

const response = await fetch('https://exports.example.com/tickets.json');
// { "meta": { ... }, "data": { "items": [ ...millions of tickets... ] } }, a ticket at a time.
const tickets = loadJson({ source: 'tickets.json', stream: response.body ?? [] }, { recordsAt: 'data.items', id: 'id', text: 'body' });
await loadIntoStore(tickets, store);
```
- **PDF.** `loadPdf()` hands each file's bytes to your parser, a `PdfTextExtractor` wrapping
  `pdfjs-dist`, `unpdf`, or an OCR service, which returns the text or one string per page.
  `PdfLoaderOptions` takes that `extract` function and `splitPages`: on by default, one document per
  page with `page` in the metadata, so a citation can name it. A file without the PDF signature is
  refused with a `PdfLoaderError` before it reaches the parser.

```ts
import { loadPdf } from 'nexus-ai-pro/loaders/pdf';
import { extractText, getDocumentProxy } from 'unpdf';

const manuals = loadPdf(['manual.pdf', 'warranty.pdf'], {
  extract: async (bytes) => (await extractText(await getDocumentProxy(bytes))).text,
});
```

## Web pages and sitemaps

`loadWebPages()` fetches pages through the SSRF-safe fetch. Private, loopback, and cloud-metadata
addresses are refused unless explicitly allowed, and each redirect hop is checked again. HTML becomes
text through `htmlToText()`; plain text, Markdown, and JSON are kept as they are.

`WebLoaderOptions` extends that `SafeFetchPolicy` (`allowedDomains`, `allowPrivateNetworks`, a resolver):

| Option | Default |
| --- | --- |
| `maxBytes` | 5 MB |
| `maxRedirects` | 5 |
| `timeoutMs`, per page | 15 seconds |
| `concurrency` | 4 pages at once, still yielded in the order given |
| `signal`, `metadata`, `mainContent` | — |

A page that fails — a status other than 2xx, a type that is not text, a timeout — raises a
`WebLoaderError` with the URL and status. Set `onError: 'skip'` to pass over it and hear about it in
`onSkip`.

A page whose content a script builds holds little in the HTML a server sends. `render` is a
`WebPageRenderer` that renders one with a browser of the application's own: given the
`WebPageToRender`, the page's URL and the HTML fetched, it returns the HTML the browser ends with, and
the document is read from that, marked `rendered`. It is called for an HTML page that runs a script
and whose text is shorter than `renderBelow`, 200 characters by default; `Infinity` renders every
page that runs one. `browserRenderer()` makes one from a `RenderingBrowser`, such as Playwright's or
Puppeteer's: each page is opened, waited on until `waitUntil` (`networkidle`), read, and closed, within
`timeoutMs` (30 seconds), as `BrowserRendererOptions` say. The package depends on no browser. The
safe fetch checks a page's URL first; the browser then fetches what the page loads under its own
network policy, not the loader's.

```ts
import { chromium } from 'playwright';
import { browserRenderer, loadWebPages } from 'nexus-ai-pro/loaders/web';

const browser = await chromium.launch();
const pages = loadWebPages(urls, { render: browserRenderer(browser) });
```

`sitemapUrls()` reads a sitemap's page URLs, following a sitemap index into its sitemaps and
decompressing gzipped ones; `loadSitemap()` loads those pages. `SitemapOptions` adds an `include`
test and a `limit` (1,000 URLs).

## Git repositories

`loadGitRepository()` reads the files a repository tracks, one document each.

- A remote URL is shallow-cloned into a temporary directory, which is removed afterwards.
- A local path is read in place, unless a `ref` asks for another branch or tag.
- Binary files, and files over the size limit, are skipped.
- Each document carries the repository, the commit, and the path in its metadata, so a citation can
  link to the exact revision.

`GitLoaderOptions` takes the `ref`, `extensions`, an `ignore` test, `parsers` as for directories,
`maxFileBytes` (1 MB), the `git` executable, and a `timeoutMs` for the clone (2 minutes).

It runs `git` without a shell, with credential prompts and the `ext::` transport disabled. It keeps
committed line endings, so a commit loads the same on every platform. A failure raises a
`GitLoaderError` with what `git` printed.

## Limitations

- The HTML conversion is a heuristic, not a browser: a page built by script needs a `render` of the
  application's own, and an HTML file read from disk is never rendered.
- A JSON file's records stream from an array reached by object keys alone; one inside another array,
  or picked by a `records` function, is read whole.
- The Git loader needs the `git` executable on the path.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/retrieval.md

# Retrieval


Retrieval quality and where the vectors live. Composable retrievers — hybrid search, reranking,
diversity, parent documents, and several phrasings of a query — work over any store, and five more
stores implement the `VectorStore` contract beside the memory, pgvector, Qdrant, and SQLite ones.
The contract itself, ingestion, and answering from retrieved context are in the
[grounding guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/grounding.md).

```ts
import { loadIntoStore } from 'nexus-ai-pro/loaders';
import { hybridRetriever, KeywordIndex, modelReranker, rerankRetriever, vectorRetriever } from 'nexus-ai-pro/rag/retrievers';

const keywords = new KeywordIndex();
await loadIntoStore([store, keywords], loaders);

const retriever = rerankRetriever(
  hybridRetriever([vectorRetriever(store), keywords]),
  modelReranker(ai, { model: 'gpt-5.4-mini' }),
);
const chunks = await retriever.retrieve('what does error E4012 mean', { filter: { tenant: 'acme' } });
```

## Retrievers

A `Retriever` answers a query with ranked chunks through `retrieve()`. `RetrieveOptions` asks for a
`topK`, a metadata `filter` passed to every store searched, and a `signal`.

Every retriever here takes other retrievers and returns one, so they stack in any order:

| Retriever | Use it when |
| --- | --- |
| `vectorRetriever()` | You want a store's search as a retriever, with defaults for every call. |
| `KeywordIndex` | Queries contain exact terms — codes, error numbers, names — that vectors miss. |
| `hybridRetriever()` | You want both, fused. The usual production setup. |
| `rerankRetriever()` | The right answer is in the top 20 but not the top 5. |
| `mmrRetriever()` | Results are near-duplicates of each other. |
| `parentDocumentRetriever()` | Small chunks match well but are too short to answer from. |
| `multiQueryRetriever()` | Users phrase questions differently from your documents. |

### Keyword and hybrid search

`KeywordIndex` ranks chunks in memory by BM25. `KeywordIndexOptions` tunes the BM25 constants `k1` and
`b`, and can replace the tokenizer. The default, `tokenize()`, splits text into lower-cased runs of
letters and digits in any script. Adding a chunk whose id is already held replaces it, as in a store.

```ts
const keywords = new KeywordIndex();
await loadIntoStore([store, keywords], loaders); // fill both in one pass

const hybrid = hybridRetriever([vectorRetriever(store), keywords], { topK: 5 });
const results = await hybrid.retrieve('error E4012');
```

`hybridRetriever()` runs its retrievers in parallel and fuses their rankings.
`reciprocalRankFusion()` is the fusion on its own: each chunk scores the sum of `weight / (k + rank)`
over the rankings it appears in. It uses ranks, not scores, so BM25 and cosine combine without
normalizing.

| Option | Type | Default |
| --- | --- | --- |
| `k` | `FusionOptions` | 60 |
| `weights` | `FusionOptions` | 1 for each ranking |
| `topK` | `FusionOptions` | every chunk, for the function; 5, for the retriever |
| `candidates` | `FusedRetrieverOptions` | four times `topK`, asked of each retriever |

### Keyword search at scale

`KeywordIndex` is one `SparseRetriever`. That is the contract for keyword search that holds its own
chunks: it is a `Retriever`, so it fuses with vector search, and it takes `add()` and `delete()`, so
ingestion keeps it in step with a vector store. Two more ship, each checked by the same contract
test:

- `PostgresKeywordIndex`, on `nexus-ai-pro/postgres/fulltext`, uses Postgres full-text search. The
  [Postgres guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/postgres.md#keyword-search) covers it.
- `ElasticsearchKeywordIndex`, on `nexus-ai-pro/rag/elasticsearch`, searches Elasticsearch or
  OpenSearch through the REST API, with no client library, as the vector stores do.
  `ElasticsearchKeywordIndexOptions` sets the cluster, the index, an API key or a user and password,
  the analyzer, and when writes become searchable (`refresh`, `wait_for` by default).
  - `migrate()` creates the index: `content` as analyzed text, and every metadata string as a keyword,
    so a filter matches it exactly.
  - Writes go in one NDJSON bulk request.
  - Scores are the engine's own BM25.
  - A failure raises an `ElasticsearchError` with the status and the body.

```ts
import { ElasticsearchKeywordIndex } from 'nexus-ai-pro/rag/elasticsearch';

const keywords = new ElasticsearchKeywordIndex({ url: 'https://search.internal:9200', index: 'kb', apiKey });
await keywords.migrate();
const hybrid = hybridRetriever([vectorRetriever(store), keywords]);
```

### Reranking

`rerankRetriever()` asks its base for more candidates than it returns, scores them all, and keeps the
best. The scorer is a `Reranker`: any function that scores chunks against a query, such as a
cross-encoder or a hosted rerank API. `RerankRetrieverOptions` sets `topK` (5), `candidates` (20), and
a `minScore`.

`modelReranker()` is a reranker over a chat model. One call scores every candidate from 0 to 10; a
reply it cannot read scores them all 0 instead of failing the search. `ModelRerankerOptions` names the
`model` and limits the characters shown per chunk. The client is a `RetrievalModelClient`: a
`NexusAI` instance, or anything with the same `complete()`.

```ts
const reranked = rerankRetriever(hybrid, modelReranker(ai, { model: 'gpt-5.4-mini' }), { topK: 5 });
```

`nexus-ai-pro/rag/rerankers` has rerankers that are faster and cheaper than a chat model. None
needs an SDK.

Hosted rerank APIs, each taking `HostedRerankerOptions` (the API key, the model, and an optional
endpoint):
- `cohereReranker()`;
- `voyageReranker()`;
- `jinaReranker()`.

Self-hosted and in-process:
- `teiReranker()` calls a cross-encoder that Hugging Face's text-embeddings-inference serves on your
  own machine or cluster (`TeiRerankerOptions`).
- `httpReranker()` covers any other rerank API. `HttpRerankerOptions` describes its request `body`
  and how to read its `scores`.
- `crossEncoderReranker()` runs a `CrossEncoder` in process. That is anything that scores
  (query, passage) pairs, so an ONNX or transformers model fits in a few lines, the passages never
  leave the machine, and there is no per-query price. `CrossEncoderRerankerOptions` sets the batch
  size, the characters scored per chunk, and a sigmoid for a model that returns logits.

Every HTTP reranker takes `RerankerHttpOptions`: `fetch`, headers, a timeout, and the characters sent
per chunk. They all do the same three things:
- put the scores back in the order of the chunks;
- give 0 to a passage the API left out;
- raise a `RerankerError` with the status when the API refuses.

```ts
import { cohereReranker, crossEncoderReranker } from 'nexus-ai-pro/rag/rerankers';

const hosted = rerankRetriever(hybrid, cohereReranker({ apiKey, model: 'rerank-v3.5' }), { candidates: 20 });
const local = rerankRetriever(hybrid, crossEncoderReranker({ score: (pairs) => model.score(pairs) }));
```

In the test suite, an experiment on the stored support dataset runs on real Postgres with
full-text search and pgvector. Its findings:
- Hybrid retrieval beats vector-only: recall@5 0.71 → 0.97, and mean reciprocal rank 0.51 → 0.71,
  both with a 95% interval above zero.
- A hosted reranker and a local cross-encoder, compared in the same experiment, each lift mean
  reciprocal rank to 0.97.

### Diversity

`mmrRetriever()` reorders its base's results by maximal marginal relevance, so near-duplicates stop
crowding out other evidence. `MmrRetrieverOptions` takes the `embed` function and the number of
`candidates`. `MmrOptions` sets `lambda`: 1 is pure relevance, 0 pure diversity, and the default is
0.5. `maximalMarginalRelevance()` is the selection on its own, over vectors you already have.

```ts
const diverse = mmrRetriever(vectorRetriever(store), { embed, lambda: 0.5, topK: 5 });
```

### Parent documents

Small chunks match a query precisely, but are often too short to answer from.
`parentDocumentRetriever()` searches small chunks and returns the larger parents they came from. Each
parent comes back once, in the order of its best child.

`splitParentChild()` produces both sides from your documents, with `ParentChildOptions` for each
split. Parents are found through a `ParentLookup`: a map, or a function that fetches them by id.
`ParentDocumentRetrieverOptions` names the child's `parentKey`, which defaults to the `documentId`
that ingestion sets.

```ts
const { parents, children } = splitParentChild(documents);
await store.add(children);
const retriever = parentDocumentRetriever(vectorRetriever(store), {
  parents: new Map(parents.map((parent) => [parent.id, parent])),
});
```

### Several phrasings

`multiQueryRetriever()` searches several rewrites of a query in parallel and fuses the results. A
`QueryVariants` function produces the rewrites. `modelQueryVariants()` asks a chat model for them;
`ModelQueryVariantsOptions` names the `model` and the `count`. `MultiQueryRetrieverOptions` can leave
the original query out.

```ts
const retriever = multiQueryRetriever(hybrid, modelQueryVariants(ai, { model: 'gpt-5.4-mini', count: 3 }));
```

### Measuring a retriever

Run a retriever over a dataset with `recallAtK()` and `reciprocalRank()` from the
[evaluation guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/evaluation.md), then compare two retrievers with `compareExperiments()`. That is
how you know a change helped rather than guessing.

## More vector stores

Each store implements `VectorStore`, takes any embedding function, and passes the same contract test:
ranking, typed metadata filters, replacement by id, and deletes. None adds a dependency. The REST
stores speak HTTP through `fetch`, and the Redis store sends commands through the client you pass.
Call `migrate()` once where a store has one; it never runs on its own.

The Qdrant, Redis, Weaviate, and Chroma stores are stable. Each runs its contract, ten racing writers,
an upgrade, tenant isolation, a load budget, and a soak against the real server in a scheduled
workflow. Pinecone runs the same checks against a stand-in, and is experimental until it runs them
live.

What every store promises:
- **A replacement is whole.** Adding a chunk whose id exists replaces it. Metadata, filter fields, and
  the source that the new version lacks are gone, even when writers race on one id.
- **A filter is exact.** A result always matches its filter, by type as well as value.
- **`topK` is a positive integer.** Anything else is refused before a request is sent. An absent
  `topK` means 5.

| Store | Entry point | Where the vectors live |
| --- | --- | --- |
| `RedisVectorStore` | `nexus-ai-pro/rag/redis` | Redis with RediSearch: Redis Stack, Redis 8, or Redis Cloud. |
| `PineconeVectorStore` | `nexus-ai-pro/rag/pinecone` | A Pinecone index, through its data-plane API. |
| `WeaviateVectorStore` | `nexus-ai-pro/rag/weaviate` | A Weaviate collection, through REST and GraphQL. |
| `ChromaVectorStore` | `nexus-ai-pro/rag/chroma` | A Chroma collection, through its v2 API. |
| `SqliteVectorStore` | `nexus-ai-pro/sqlite/vectors` | A SQLite file, optionally ranked by sqlite-vec. See the [SQLite guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/sqlite.md). |

```ts
import Redis from 'ioredis';
import { RedisVectorStore } from 'nexus-ai-pro/rag/redis';

const store = new RedisVectorStore(new Redis(process.env.REDIS_URL!), {
  dimensions: 1536,
  embed,
  filterFields: ['tenant'],
});
await store.migrate();
```

### Redis

`RedisVectorStore` keeps each chunk as a hash and searches a RediSearch vector index. The client is a
`RedisVectorLikeClient`: ioredis, with `call()`, or node-redis, with `sendCommand()`. Either protocol
works: RESP2, or RESP3, which ioredis 6 speaks by default.

`RedisVectorStoreOptions` sets the `dimensions`, the `index` and key `prefix`, the `algorithm` (`HNSW`,
or exact `FLAT`), and the `filterFields`. RediSearch can only filter on fields declared in the index.
So declared fields are indexed as tags with their type, and a filter on any other field is refused
instead of being silently ignored. A chunk is replaced by one atomic script, so a field the new
version lacks never survives. Under `tenantVectorStore()`, declare `tenantId` too.

### Pinecone

`PineconeVectorStore` upserts, queries, and deletes in batches. `PineconeVectorStoreOptions` takes the
index `host`, the `apiKey`, a `namespace`, the `apiVersion` header, and the upsert `batchSize` (100). A
failed request raises a `PineconeError` with the status and body.

`PineconeVectorStore.create()` makes the index first, through Pinecone's control plane, and returns a
store over its host. It takes `PineconeCreateOptions`: the store's, without the host, and the index's
from `PineconeIndexOptions`. `ensurePineconeIndex()` does that part alone and returns a `PineconeIndex`,
its `host` and whether it was `created`:
- a new index is serverless, with the cosine metric and the store's width, on the `cloud` and `region`
  given, `aws` and `us-east-1` by default, with `deletionProtection` and `tags` if asked;
- an index of that name that exists is used when its width and metric are the store's, and refused
  otherwise, as is one another process creates at the same moment;
- it waits until the index is ready, asking every `pollMs` (2 seconds) for up to `readyTimeoutMs` (5
  minutes), and refuses one whose initialization failed or that is being deleted.

```ts
import { PineconeVectorStore } from 'nexus-ai-pro/rag/pinecone';

const store = await PineconeVectorStore.create({ apiKey: process.env.PINECONE_API_KEY ?? '', name: 'docs', dimensions: 1536, region: 'eu-west-1', embed });
```

### Weaviate

`WeaviateVectorStore` brings its own vectors to a collection with no vectorizer and cosine distance.
`migrate()` creates the collection and adds a property for each field you filter on. Each field's type
is a `WeaviateFieldType`: `text`, `number`, or `boolean`.

A filter follows the collection's schema, which the store reads the first time a filter needs it. A
value of another type than its property matches nothing, as in every other store, and costs no
request. A property added later, by `migrate()` or by Weaviate's auto-schema, is picked up when a
filter first names it. A collection made elsewhere with `int` properties works too.

Every filtered hit is checked against the object's own fields before it is returned. Weaviate's
filter index can keep an entry for a value an object no longer has, after two writers replace it at
once. When such an entry took a place, the store asks for more, so the page stays full.

`WeaviateVectorStoreOptions` takes the `url`, the `collection` (`NexusChunk`), an `apiKey` sent as a
bearer token, and the usual `fetch`, `headers`, and `timeoutMs`. Errors raise a `WeaviateError`,
including errors Weaviate reports inside a successful batch.

### Chroma

`ChromaVectorStore` uses a collection with cosine distance, which `migrate()` creates.
`ChromaVectorStoreOptions` takes the `url`, the `collection`, the `tenant` and `database`, and an
`apiKey` sent as `x-chroma-token`. A failed request raises a `ChromaError`.

Chroma merges metadata when it replaces a record, so the store removes each field the new version
lacks. How it knows which fields to remove depends on `filterFields`:

| `filterFields` | Filterable | A replacement |
| --- | --- | --- |
| Not set | Every top-level field | Reads which fields the stored version has, then removes the rest: one more request per batch. |
| A list | Only the listed fields; a filter on another is refused | Sets every listed field, or removes it, in the same request. Whole under any race. |

```ts
const store = new ChromaVectorStore({
  collection: 'docs',
  dimensions: 1536,
  embed,
  filterFields: ['tenantId', 'lang'],
});
```

### How the REST stores keep metadata

In the REST stores, each chunk's top-level string, number, and boolean metadata fields are copied under
`meta_` names; that is what filters match on. The whole metadata also travels as JSON, so nested values
still come back. Scores are cosine similarity in every store, so a `minScore` means the same wherever
the vectors live.

## Limitations

- `KeywordIndex` holds its chunks in memory. For a corpus that does not fit, use
  `PostgresKeywordIndex` or `ElasticsearchKeywordIndex`, or another engine behind a `SparseRetriever`
  of your own.
- `modelReranker()` and `modelQueryVariants()` cost a model call per retrieval; a cross-encoder or a
  hosted rerank API is faster and cheaper where you have one.
- `ensurePineconeIndex()` creates serverless indexes alone; a pod-based index, or one with another
  metric, is made in the console or with Pinecone's client, and given to the store by its host.
- The Pinecone store is experimental until it passes the live checks the other remote stores pass.
- A Chroma store without `filterFields` reads before it replaces, so two writers replacing one id at
  once can leave a stale filter field. Declare `filterFields` when writers can race.
- The Redis store has no timeout of its own: the client's command timeout applies.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/embeddings.md

# Embeddings and retrieval


Turning text, images, audio, and documents into vectors, from `nexus-ai-pro/embeddings`. An
embedding call gets what a completion gets: routing across providers, batching, caching, budgets, and
retries. It also refuses an option a model cannot honour, such as a dimension it does not produce, so
a store never mixes incompatible vectors. Adapters for OpenAI, Google, Cohere, Mistral, Ollama, and compatible servers are on
`nexus-ai-pro/embeddings/adapters`.

Searching the vectors, grounding answers in them, and checking those answers are in the
[grounding guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/grounding.md).

## Embeddings

`ai.embed()` is a first-class operation, not a helper: it gets the same routing, caching, batching,
budget, retry, audit, and metrics as a completion, so embedding spend shows up next to completion
spend instead of being invisible.

The smallest call is one line, and needs no embedding-specific configuration — adapters are derived
from the provider credentials already in `providers`:

```ts
import { NexusAI } from 'nexus-ai-pro';

const ai = new NexusAI({ providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } } });

const vector = await ai.embedOne('Provider-neutral embeddings.');
```

A batch returns vectors in input order regardless of how many provider calls the model's batch
limit required, and reports what the call cost:

```ts
const response = await ai.embed({
  input: documents.map((document) => document.text),
  model: 'embed-quality',
  inputType: 'document',
  normalize: true,
});

response.vectors;                  // number[][], in input order
response.meta.batches;             // provider calls the split required
response.meta.usage.inputTokens;   // reported by the provider, or estimated when it reports none
response.meta.cost.amount;         // numeric, priced from the embedding registry
response.meta.cachedInputs;        // inputs answered from cache
response.meta.deduplicatedInputs;  // repeated inputs answered from one call
```

Caching is per input rather than per request, so a partially repeated batch only sends the texts it
has not seen. Repeated texts inside one batch are collapsed into a single provider call:

```ts
const ai = new NexusAI({
  providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } },
  embeddings: {
    cache: { enabled: true, ttlSeconds: 86_400 },
    costBudget: { enabled: true, maxEstimatedCost: 5 },
    retry: { enabled: true, maxRetries: 2 },
    fallback: [{ provider: 'cohere', model: 'embed-v4.0' }],
  },
});
```

Unlike a completion, an unsupported embedding option is **refused rather than dropped**. A vector
built with different dimensions or a different input type is silently incompatible with the vectors
already in a store, and the mismatch only surfaces later as unexplained retrieval quality loss:

```ts
await ai.embed({ input: 'x', model: 'embed-english-v3.0', dimensions: 512 });
// EmbeddingCapabilityError: model has a fixed size of 1024 dimensions
```

An option the registry says nothing about is still passed through, because an undeclared capability
means unknown rather than unsupported, and `providerOptions` reaches the provider body untouched for
anything the neutral contract does not model yet.

Existing vector stores keep their contract. `toEmbeddingFunction()` adapts the family to the plain
function `MemoryVectorStore`, the semantic cache, and RAG ingestion already accept:

```ts
import { toEmbeddingFunction } from 'nexus-ai-pro/embeddings';
import { MemoryVectorStore } from 'nexus-ai-pro/rag';

const store = new MemoryVectorStore(toEmbeddingFunction(ai, { inputType: 'document' }));
```

Bundled adapters cover OpenAI (and any OpenAI-compatible `/embeddings` server), Google, Cohere,
Mistral, and Ollama. A custom adapter implements `EmbeddingsProvider` and is checked against the
neutral contract by `runEmbeddingProviderConformance()`:

```ts
import { OpenAIEmbeddingProvider } from 'nexus-ai-pro/embeddings/adapters';
import { MockEmbeddingProvider } from 'nexus-ai-pro/embeddings/mock';

ai.registerEmbeddingProvider('gateway', new OpenAIEmbeddingProvider({
  apiKey: process.env.GATEWAY_KEY!,
  baseUrl: 'https://gateway.internal/v1',
  providerName: 'gateway',
}));
ai.registerEmbeddingProvider('mock', new MockEmbeddingProvider());
```

Bundled embedding dimensions and prices are defaults, not financial truth. Override them through
`embeddings.models.registry` when exact numbers matter.

## Images, audio, video, and documents

An input can be media as well as text: an image, audio, a video, a PDF, or text with them. A multimodal
model embeds each in the same space as text, so a text query finds the image it describes. Give the
input as `EmbeddingParts`: the same text, image, audio, and document parts a message takes, and a video
by its content.

```ts
const photos = await ai.embed({
  model: 'gemini-embedding-2',
  inputType: 'document',
  input: images.map((bytes) => ({ parts: [{ type: 'image', source: { buffer: bytes, mimeType: 'image/jpeg' } }] })),
});
await store.add(images.map((_, index) => ({ id: `photo-${index}`, content: '', embedding: photos.vectors[index] })));

const [query] = (await ai.embed({ model: 'gemini-embedding-2', inputType: 'query', input: 'a red bicycle' })).vectors;
const found = await store.searchVector(query, { topK: 5 });
```

An `EmbeddingPart` is a `TextContent`, `ImageContent`, `AudioContent`, `DocumentContent`, or
`EmbeddingVideoPart`. Every part of one input becomes one vector, such as a product photo with its
caption. A part must hold its content, as bytes, base64, or a URL. A path or a transcript is refused,
since an embedding never reads a file. That is why a video has a part of its own: a message's video
part names a file, and an `EmbeddingVideoPart` holds the video's bytes or base64, with its container,
`mp4` or `mov`, or a URL such as a Files API URI.

```ts
const clip = await ai.embed({
  model: 'gemini-embedding-2',
  input: { parts: [{ type: 'video', source: { buffer: await readFile('demo.mp4'), format: 'mp4' } }] },
});
```

Media is sent only to a model that takes it. An `EmbeddingModality` is `text`, `image`, `audio`,
`video`, or `document`, and `inputModalities` on a model's registry entry says which it takes. A model without an
entry is judged by its adapter's capabilities. Anything else is refused with an
`EmbeddingCapabilityError` before any call, and a fallback that cannot take the media is skipped.

| Model | Takes | Notes |
| --- | --- | --- |
| `gemini-embedding-2` | text, image, audio, video, document | Up to 6 images, 180 seconds of audio, 120 seconds of video, its sound not heard, or a 6-page PDF an input. The task is a prefix on the text, which the adapter writes from `inputType`. |
| `embed-v5.0-pro`, `embed-v5.0-fast` | text, image | One vector space: index with Pro, query with Fast. 2,048 dimensions, or 256 to 1,536. |
| `embed-v4.0` | text, image | Images as JPEG, PNG, WebP, or GIF, 20 MB a request. |
| Every other bundled model | text | |

Google takes media inline, or by a Files API URI; Cohere takes images inline. Any other URL is refused
with the reason, since neither provider fetches it.

Media is priced beside the text's tokens, at its model's registry prices:
- an image at `costPerImage`;
- audio at `costPerAudioSecond`, each part's length counted up to the next second;
- video at `costPerVideoFrame`, for each frame the model takes, as its `videoFrames` says:
  gemini-embedding-2 takes one a second, at most 32, so a minute's video is 32 frames.

A length is read from the media's own headers, without decoding it: WAV, MP3, MP4, MOV, and the other
containers the media family reads. Audio or video longer than a model's `maxAudioSeconds` or
`maxVideoSeconds` is refused before any call, naming the input. A part whose length cannot be read,
one given by URL or in a container no reader knows, is sent, and counted in the response's
`unpricedMedia` rather than priced at nothing. A budget counts the media before the call, and a
repeated medium is embedded once, and cached under a hash of its content.

## Sharing the cache

A shared cache lets every process reuse what any of them embedded. Re-indexing a corpus, or several
workers ingesting overlapping documents, then pays the provider only for what is new. Give
`cache.store` an `EmbeddingCacheStore`, from `nexus-ai-pro/embeddings/cache` or a database entry
point:

```ts
import Redis from 'ioredis';
import { NexusAI } from 'nexus-ai-pro';
import { MemoryEmbeddingCache, RedisEmbeddingCache, tieredEmbeddingCache } from 'nexus-ai-pro/embeddings/cache';

const ai = new NexusAI({
  providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } },
  embeddings: {
    cache: {
      store: tieredEmbeddingCache(new MemoryEmbeddingCache(), new RedisEmbeddingCache(new Redis(process.env.REDIS_URL!))),
      ttlSeconds: 30 * 86_400,
    },
  },
});
```

A store differs from a `CacheAdapter` in three ways:

- **One round trip a request.** It reads every input in one `getMany()` and writes what it embedded
  in one `setMany()`, so 2,000 cached texts cost one call, not 2,000.
- **Keys are hashes.** `embeddingCacheKey()` is a SHA-256 of the text and its
  `EmbeddingCacheContext`: the provider, model, input type, size, normalization, and truncation. No
  text is ever a key in a shared store, and the same text under another model is another entry.
- **Vectors are packed.** `packVector()` and `unpackVector()` store four bytes a dimension, against
  about twenty as JSON. Providers send float32 precision, so nothing they sent is lost.

`EmbeddingCacheConfig` is the `CacheConfig` every cache takes, plus `store`. A store turns the cache
on unless `enabled` is `false`, and takes the place of `adapter`. A store that fails fails the
request, as an adapter does. Wrap it to treat a failure as a miss.

| Store | Entry point | Where the vectors live |
| --- | --- | --- |
| `MemoryEmbeddingCache` | `nexus-ai-pro/embeddings/cache` | This process, least recently used first out. `MemoryEmbeddingCacheOptions` sets `maxEntries` (10,000). |
| `RedisEmbeddingCache` | `nexus-ai-pro/embeddings/cache` | Redis: one `MGET` and one script a batch. |
| `PostgresEmbeddingCache` | `nexus-ai-pro/postgres/embedding-cache` | A Postgres table. See the [Postgres guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/postgres.md). |
| `SqliteEmbeddingCache` | `nexus-ai-pro/sqlite/embedding-cache` | A SQLite table. See the [SQLite guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/sqlite.md). |

`tieredEmbeddingCache()` puts two stores in front of each other. The near one answers first, and
the far one answers the rest in one call. A far hit is copied near, and a write goes to both.

`RedisEmbeddingCache` takes a `RedisEmbeddingCacheClient`: ioredis, with `call()`, or node-redis,
with `sendCommand()`. `RedisEmbeddingCacheOptions` sets the key `prefix` (`nexus:embedding:`) and
`multiKey`. On Redis Cluster, set `multiKey: false`: each key then travels as its own command, all
sent at once, since a multi-key command needs every key in one slot.

An `EmbeddingCacheEntry` is a key and its vector. A store of your own implements the two methods.

A `cache.adapter`, without a store, keeps working with the keys it always had. It is now asked for
every input at once rather than one after another.

## A batch tier for large jobs

OpenAI and Google embed at half price through their batch tiers, in exchange for a completion window
measured in hours. `EmbeddingBatchManager`, on `nexus-ai-pro/embeddings/batch`, puts both behind one
durable handle: it submits, polls with backoff, collects, and prices at the batch rate.

```ts
import { EmbeddingBatchManager } from 'nexus-ai-pro/embeddings/batch';
import { OpenAIEmbeddingBatchProvider } from 'nexus-ai-pro/embeddings/batch/openai';

const batches = new EmbeddingBatchManager(
  { providers: { openai: new OpenAIEmbeddingBatchProvider({ apiKey: process.env.OPENAI_API_KEY! }) } },
  { operations: { store: operationStore } },
);
const handle = await batches.submit({
  model: 'text-embedding-3-small',
  items: chunks.map((chunk) => ({ customId: chunk.id, input: chunk.content })),
  idempotencyKey: 'nightly-reindex',
});
const result = await handle.result(); // hours later, or in another process
```

An `EmbeddingBatchSubmitRequest` holds `EmbeddingBatchItem` values: a `customId` each, which must be
unique, and an input, which is a text or `EmbeddingParts`. It also names the concrete `model`, and can
set the input type, `dimensions`, `normalize`, a completion window, an idempotency key, and labels.

A submission is checked before anything is sent: ids, the adapter's item limit, and whether the model
takes the inputs' media. It is then a durable operation that records its batch the moment it exists.
A worker that dies while it waits is replaced by one that collects that batch, never a second. The
same holds for completion batches on `nexus-ai-pro/batch`.

| Method | Does |
| --- | --- |
| `submit()` | Submits, and returns a durable handle that settles with an `EmbeddingBatchResult`. |
| `start()` | Submits, and returns the batch's ref at once, for a caller that keeps it. |
| `resume()` | Waits for a batch by its ref, submitted here or elsewhere, and collects it. |
| `status()`, `cancel()` | Reads or cancels a batch by its ref. |
| `registerEmbeddingBatchProvider()`, `hasEmbeddingBatchProvider()`, `listEmbeddingBatchProviders()` | Manage adapters. |

`EmbeddingBatchConfig` registers adapters and sets the poll interval, its maximum, the timeout
(26 hours), and the embedding `models` used for checks and prices. `EmbeddingBatchManagerRuntime`
gives it an operation store, and a client's `lifecycle` to authorize and budget each submission.

An `EmbeddingBatchResult` has every input's `EmbeddingBatchOutputItem`, matched by `customId`: a
vector, or an error. It also has counts, usage, and the cost with the provider's discount. Tokens are
reported where the provider reports them, and estimated where it does not, never priced at zero.

`batchEmbedder()` turns a manager into the `BatchEmbedder` an ingestion pipeline takes as
`batchEmbed`; the [loaders guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/loaders.md#through-a-providers-batch-tier) shows it.
`BatchEmbedderOptions` names the model and sets the window `size` (10,000 texts), the input type
(`document`), `dimensions`, and normalization.

### Batch adapters

| Adapter | Entry point | Notes |
| --- | --- | --- |
| `OpenAIEmbeddingBatchProvider` | `nexus-ai-pro/embeddings/batch/openai` | `/v1/embeddings` lines, 50,000 a batch; failed lines are read from the error file. `OpenAIEmbeddingBatchProviderOptions`: the key, base URL, organization, headers, and `fetch`. |
| `GoogleEmbeddingBatchProvider` | `nexus-ai-pro/embeddings/batch/google` | `asyncBatchEmbedContent`: inline under 20 MB, and a larger batch, up to 2 GB, uploaded as a JSONL file through the Files API, its results read from the file the batch writes, matched by key; takes media on a model that does. `GoogleEmbeddingBatchProviderOptions`: the key, base URL, headers, and `fetch`. Experimental until a live run is recorded. |
| `MockEmbeddingBatchProvider` | `nexus-ai-pro/embeddings/batch/mock` | In memory, for tests: scripted statuses and failures, and the vectors `MockEmbeddingProvider` gives. `MockEmbeddingBatchProviderOptions` sets them. |

An adapter of your own implements `EmbeddingBatchProvider`: `submit`, `poll`, `results`, and an optional
`cancel`. Its `EmbeddingBatchProviderInfo` declares its `EmbeddingBatchProviderCapabilities`: the
batch limits, discount, and `inputModalities`. `submit` receives an `EmbeddingBatchProviderRequest`,
whose `EmbeddingBatchProviderItem` values hold each input's text and, for media, its parts.

## Requests and responses

`EmbeddingRequest` needs only the `input`: an `EmbeddingInput`, which is one text or a list, where
media is `EmbeddingParts`. The rest is optional.

| Field | What it does |
| --- | --- |
| `model`, `provider` | A model or alias, and a provider, when routing should not choose. |
| `inputType` | An `EmbeddingInputType`: `document`, `query`, `classification`, or `clustering`. Store with `document` and search with `query` on providers that embed the two differently. |
| `dimensions` | Shortens the vector on models that can. Refused on a model with a fixed size. |
| `encodingFormat` | An `EmbeddingEncodingFormat`: `float`, or `base64`, which is smaller on the wire. Both come back as numbers. |
| `normalize` | Scales vectors to unit length when the provider does not. |
| `truncate` | An `EmbeddingTruncateMode`: `none` refuses a long input; `start` or `end` cuts from that side. |
| `user`, `userId` | For provider abuse monitoring, and for your rate limits and audit. |
| `requestId`, `idempotencyKey`, `signal`, `timeoutMs` | Per-call identity and control. |
| `retry`, `cache`, `concurrency` | Overrides for this call. |
| `metadata`, `providerOptions` | Application data, and fields merged into the provider body as they are. |

`EmbeddingResponse` returns:

- `vectors`, in input order;
- `embeddings`, each an `Embedding` with its index, values, size, and whether the provider cut its
  text;
- the last batch's raw payload;
- `EmbeddingMeta`: the provider and model, dimensions, count, latency, batches, usage, cost, cache and
  de-duplication counts, retries, and the routing decision.

## Configuration

`EmbeddingConfig` is the client's `embeddings` option.

| Option | What it sets |
| --- | --- |
| `providers`, `defaultProvider`, `defaultModel` | Adapters by name, and what a request uses by default. |
| `fallback` | Routes tried, in order, on a retryable failure. |
| `models` | An `EmbeddingModelRegistryConfig`: aliases, registry entries, and whether to keep the bundled ones. |
| `cache` | Caching per input, so a repeated text is embedded once. |
| `rateLimit`, `retry`, `timeoutMs` | Limits and retries, as for completions. |
| `costBudget` | An `EmbeddingCostBudgetConfig`: a limit in US dollars, and `error` to refuse or `warn` to report. |
| `concurrency` | Batches sent at once when a request is split. Defaults to 4. |
| `deduplicate` | Sends repeated texts once. On by default. |
| `autoRegisterProviders` | Builds adapters from the chat credentials already in `providers`. `createConfiguredEmbeddingProviders()` does this step on its own. |

`EmbeddingManager` is the family behind `ai.embed()`, and it works without a client too. Its methods
are `registerEmbeddingProvider()`, `hasEmbeddingProvider()`, `listEmbeddingProviders()`, `embed()`,
and `embedOne()`. A client exposes its own as `ai.embeddings`.

Anything with an `embed()` method is an `EmbeddingSource`. That is what `toEmbeddingFunction()` adapts
into the plain function vector stores take.

## Writing an adapter

An adapter implements `EmbeddingsProvider`: an `info` description and an `embed()` method.

`info` is an `EmbeddingProviderInfo`: a name, locality, version, default model, and
`EmbeddingProviderCapabilities`. Capabilities declare models, batch size, input length, dimensions,
input types, encodings, truncation, whether vectors come back normalized, and the `inputModalities` it
takes. A field you leave out means unknown, not unsupported, except `inputModalities`: an adapter that
does not declare media is sent text only.

`embed()` is called once per batch:

| Type | Contents |
| --- | --- |
| `EmbeddingProviderRequest` | One batch, already split to fit and resolved to a concrete model. Its `contents` holds each input's parts when any holds media. |
| `EmbeddingProviderPart` | A part as an adapter receives it: an `EmbeddingProviderTextPart`, or an `EmbeddingProviderMediaPart` with its MIME type and base64 `data` or `url`. |
| `EmbeddingProviderCallContext` | The `ProviderCallContext` every provider receives — request id, signal, deadline, idempotency key, and trace headers — with the attempt and the batch index. |
| `EmbeddingProviderResult` | What you return: one vector per input in order, the model actually used, whether it truncated, and the raw payload. |
| `EmbeddingProviderUsage` | Token counts, when the provider reports them. |

### Bundled adapters

The adapters on `nexus-ai-pro/embeddings/adapters` take `EmbeddingAdapterOptions`: an API key, a base
URL, a model, headers, and a `fetch`.

| Adapter | Notes |
| --- | --- |
| `OpenAIEmbeddingProvider` | Also serves any OpenAI-compatible `/embeddings` server, through `OpenAICompatibleEmbeddingOptions`: a `providerName` and the `capabilities` it declares. |
| `GoogleEmbeddingProvider` | Maps input types to Google's task names. |
| `CohereEmbeddingProvider` | Maps input types to Cohere's. |
| `MistralEmbeddingProvider` | OpenAI-compatible. |
| `OllamaEmbeddingProvider` | Local models. |

`MockEmbeddingProvider`, on `nexus-ai-pro/embeddings/mock`, is deterministic, for tests.
`MockEmbeddingProviderOptions` sets its name, model, dimensions (16), capabilities, and reported
usage. It can also fail on a chosen call, for retry and failover tests, and add latency. It takes
every modality, and `describeMedia` turns a media part into the text its vector is hashed from, so a
test can put an image and its caption in one space.

A vector store that needs no routing or retries can use a bare function instead:
`createOpenAIEmbeddingProvider()`, `createGeminiEmbeddingProvider()`, or
`createCohereEmbeddingProvider()`. They take `OpenAIEmbeddingOptions`, `GeminiEmbeddingOptions`, and
`CohereEmbeddingOptions`.

## The embedding registry

`KNOWN_EMBEDDING_MODELS` maps each model to its `EmbeddingModelCapabilities`: provider and family,
native dimensions and the sizes it can shorten to, input and batch limits, the longest audio and video
it takes, the price per 1,000 input tokens, per image, per second of audio, and per video frame, with
how many frames it takes a second, whether it normalizes, the input types and `inputModalities` it
takes, and its status. Each entry also records when and where it was verified.

`EMBEDDING_MODEL_ALIASES` holds stable names such as `embed-quality`. `EMBEDDING_REGISTRY_PROVENANCE`
says when the bundled data was checked.

`nexus-ai-pro/embeddings/models` reads the registry through your configuration:

| Function | Returns |
| --- | --- |
| `resolveEmbeddingModel()` | A `ResolvedEmbeddingModel` for a name or alias. |
| `getEmbeddingModelCapabilities()` | One model's entry. |
| `getEmbeddingModelRegistry()`, `getEmbeddingModelAliases()` | The merged maps. |
| `listEmbeddingModels()`, `listEmbeddingModelsForProvider()` | Model names. |
| `estimateEmbeddingCost()` | The price of an `EmbeddingCostEstimateInput`, before the call. |
| `priceEmbeddingUsage()` | The price of reported usage. Embeddings bill on input only. |
| `embeddingDimensions()` | The size a model produces, honouring a requested truncation. |
| `resolveMaxBatchSize()` | The smaller of the model's and the adapter's batch limits. |

## Errors

Every embedding error extends `EmbeddingError` and has a stable code.

| Error | When |
| --- | --- |
| `EmbeddingValidationError` | The request is malformed. |
| `EmbeddingProviderError` | The provider failed. The next three extend it. |
| `EmbeddingProviderNotFoundError` | No adapter serves the request. |
| `EmbeddingCapabilityError` | The model cannot honour an option. |
| `EmbeddingProviderResponseError` | The response has the wrong shape, such as the wrong number of vectors. |
| `EmbeddingModelNotFoundError` | The registry does not know the model. |

## Limitations

- Bundled dimensions and prices are defaults, verified on the date the registry records.
- Cost is estimated from tokens when a provider reports none.
- Two processes that miss the same text at once both embed it; the shared cache keeps one vector.
- A Google batch file leaves its input file in the Files API, which deletes it after 48 hours.
- Audio or video given by URL is not priced, since its length is not read; `unpricedMedia` counts it.
- Cohere publishes no per-token price for `embed-v5.0-pro` or `embed-v5.0-fast`, so their cost reads
  as zero and a budget cannot hold them. Give a price through `embeddings.models.registry`.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/media.md

# Media understanding


Reading what a document or an image says, describing an image, and answering a question about
either, from `nexus-ai-pro/media`. One contract covers any vision model, reached through the client,
and Mistral's dedicated OCR service. A conformance suite checks any provider against it. Media search
and RAG, on `nexus-ai-pro/media/rag`, find the media a question is about and hand the model the media
themselves.

```ts
import { NexusAI } from 'nexus-ai-pro';
import { MediaManager, modelMediaProvider } from 'nexus-ai-pro/media';
import { MistralOcrProvider } from 'nexus-ai-pro/media/mistral';

const ai = new NexusAI({ providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } } });
const media = new MediaManager(
  {
    providers: {
      ocr: new MistralOcrProvider({ apiKey: process.env.MISTRAL_API_KEY! }),
      vision: modelMediaProvider(ai, { model: 'gpt-5.4' }),
    },
    defaults: { read: 'ocr', describe: 'vision', answer: 'vision' },
  },
  { lifecycle: ai.lifecycle },
);

const invoice = { type: 'document' as const, source: { buffer: await readFile('invoice.pdf') } };
const { pages } = await media.read({ media: invoice });
const { caption, altText } = await media.describe({ media: { type: 'image', source: { url: photoUrl } } });
const { answer } = await media.answer({ media: invoice, question: 'What is the total due?' });
```

## Tasks

A `MediaInput` is an image or a document part, the same `ImageContent` and `DocumentContent` a message
takes. It holds bytes, base64, a URL, or an asset. A path is refused, since nothing here reads a file.
A `MediaKind` is `image` or `document`, and a `MediaTask` is one of three:

| Method | Request | Result |
| --- | --- | --- |
| `read()` | `MediaReadRequest`: the medium, and optionally which `pages`, from 0. | `MediaText`: each `MediaPage` with its index, its text as Markdown, its size, and the boxes of the pictures on it; and all the text joined. |
| `describe()` | `MediaDescribeRequest`: the medium, and a `brief` or `detailed` caption. | `MediaDescription`: the caption, short alternative text for a screen reader, and tags. |
| `answer()` | `MediaAnswerRequest`: one medium or several, and the question. | `MediaAnswer`: the answer, and the pages it rests on when the provider says. |

Every result is `MediaResultMeta` too: the provider, the model, the `MediaUsage` (tokens for a model,
pages for an OCR service), and the cost where it is priced.

## The manager

`MediaManager` sends each task to the provider a request names, then to the task's default in
`MediaConfig.defaults`, then to the first provider that performs it on that kind of medium. It
refuses a request no provider can serve with a `MediaError`, whose code is `MEDIA_NO_PROVIDER`,
`MEDIA_UNSUPPORTED`, or `MEDIA_INVALID`. `registerMediaProvider()`, `hasMediaProvider()`, and
`listMediaProviders()` manage its providers.

`MediaRuntime.lifecycle` runs every call as one operation of the client's lifecycle, family `media`,
so it is authorized, budgeted, and audited as every other call is. A vision model is reached through
the client, so its completion is too; the lifecycle covers a provider that calls its service
directly, such as OCR.

## Providers

| Provider | Entry point | Does |
| --- | --- | --- |
| `modelMediaProvider()` | `nexus-ai-pro/media` | All three tasks, through any vision model. |
| `MistralOcrProvider` | `nexus-ai-pro/media/mistral` | Reading: PDFs, slides, documents, and images into Markdown pages. |
| `MockMediaProvider` | `nexus-ai-pro/media/mock` | All three, from what its options say each medium holds, for tests. |

`modelMediaProvider()` takes a `CompletionClientLike`, anything with `complete()` such as a client, and
`ModelMediaProviderOptions`: the `model`, a name, the media kinds it takes, and the most tokens a
reading may produce (8,192). It asks for JSON matching each task's shape. A model that answers in
plain text still gives one page, one caption, or one answer, and the call's cost is the client's own.

`MistralOcrProvider` takes `MistralOcrProviderOptions`: the key, the model (`mistral-ocr-latest`), the
table format, headers, and `fetch`. A medium is sent as a data URI or by its URL. Mistral prices OCR by
the page: set `pricePerPage` to the rate your account pays, and each reading reports its cost.

`MockMediaProviderOptions` gives each medium's pages, caption, and answer as functions, and names the
tasks it performs. Its `calls` keeps every request.

A provider of your own implements `MediaProvider`. Its `MediaProviderInfo` declares its
`MediaProviderCapabilities`, the tasks and kinds it takes, and each task method receives a
`MediaCallContext` with the signal and the request id.

## Conformance

`runMediaConformance()` checks a provider against the contract with `MediaConformanceFixtures`: an
image, a document, a pattern the read text must match, and a question with a pattern for its answer.
Every task it declares must be implemented and return the right shape. It never throws; the
`MediaConformanceReport` lists what passed and what failed. All three bundled providers pass it in the
test suite.

## Media search and RAG

`createMediaIndex()` keeps media in any vector store, and `answerFromMedia()` answers from them:

```ts
import { MemoryAssetStore } from 'nexus-ai-pro/images/assets';
import { answerFromMedia, createMediaIndex } from 'nexus-ai-pro/media/rag';

const index = createMediaIndex({
  vectors,
  media,
  embeddings: ai.embeddings,
  assets: { store: new MemoryAssetStore(), tenantId: 'acme' },
});
await index.add([
  { id: 'invoice-17', media: invoice, source: 'mail/invoice-17.pdf', metadata: { customer: 'globex' } },
  { id: 'q3-chart', media: { type: 'image', source: { buffer: chartPng, mimeType: 'image/png' } } },
]);

const { answer, hits } = await answerFromMedia(media, index, 'Which quarter had the most revenue?');
```

`createMediaIndex()` returns a `MediaIndex`: `add()`, `search()`, and `resolve()`, and, with a
manifest, `delete()`, `reindex()`, `describe()`, and `list()`.
Each `MediaDocument` is an id, a medium, and optionally a source and metadata. `add()` turns it into
chunks:

| Chunk | From | When |
| --- | --- | --- |
| Each page's text, cut at paragraphs past `maxChunkChars` (4,000) | `read()` | By default. |
| A caption | `describe()` | By default, for an image. |
| The medium's own vector | The embedding model, given the medium as parts | With `index: { vectors: true }`, on a model that takes the medium, such as gemini-embedding-2. |

`MediaIndexOptions` takes the vector store, the media manager, a `MediaEmbeddingSource` such as a
client's `embeddings`, the embedding model, what to index, and an asset store. A medium held as bytes
or base64 is kept in the asset store, and one at a URL is kept by its URL, so every hit can hand its
medium back. A medium held as bytes without an asset store is refused. Adding an id again replaces
its chunks. The `MediaIndexReport` counts the media indexed and left `unchanged` and the chunks, and
lists each medium's chunk ids.

### The manifest

Give the index a `manifest`, any `Store`, and a `name`, and it remembers what it holds: one
`MediaIndexEntry` per medium, with a hash of its content, source, metadata, and how it was indexed,
where it is kept, how many pages were read, every chunk's id, the embedding model, and the
`MediaIndexReader` that read and described it. With it:

- `add()` upserts: a medium at the same version writes nothing, and one that changed replaces exactly
  its chunks, removing those of pages it no longer has, and the asset of its old content;
- `delete(ids)` removes media with their chunks and assets;
- `reindex(ids)` indexes media again from where each is kept, after a new embedding model or reader;
- `describe(id)` returns a medium's entry, and `list()` the entries, newest first.

```ts
const index = createMediaIndex({ vectors, media, embeddings, assets, manifest: store, name: 'invoices' });
await index.add(documents); // run again: only what changed is read and embedded
await index.delete(['invoice-17']);
```

A change is recorded as pending before anything is written, and as done only once its chunks are
written, its stale chunks removed, and its old asset deleted. Chunk ids are fixed by the medium and
its pages, so a write repeated after a crash replaces rather than adds, and the next `add()`,
`reindex()`, or `delete()` of a medium a crash left pending finishes it. In the test suite, a medium
indexed, changed, or deleted with a crash at each write ends with exactly its chunks and its asset.

`search()` returns `MediaHit` values, each medium once at its best chunk, the best first, as
`MediaSearchOptions` asks: `topK` (5) and a metadata `filter`. A hit says what matched (a page, the
caption, or the medium's vector), the page and text, the source, the medium's own metadata, and its
`MediaReference`: where the medium is kept. `resolve()` hands back the medium behind a hit, as the
model takes it.

`answerFromMedia()` searches (`topK` 3), resolves each hit, and asks the media manager with the media
themselves, so the model sees what a page or a chart shows, not only what was written about it. Its
`MediaRagAnswer` carries the hits it was given. In the test suite, with stand-in models, answers given
the retrieved charts are right on every question, and answers from the same hits' text on none: the
text says what each chart is, and only the chart shows which bar is tallest.

## A catalog of every stored asset

`mediaCatalog()`, from `nexus-ai-pro/media/catalog`, keeps one record of every stored asset in any
long-term store: the Postgres, SQLite, Redis, or memory one. `catalogedAssetStore()` wraps an asset
store, the S3, filesystem, or memory one, so every asset stored through it is recorded and every one
deleted through it is forgotten; an entry expires with its asset, and the store's `purgeExpired()`
removes both. Wrap each store an application keeps media in, with a `name` for each, and the catalog
covers them all. Everything else about each store is its own.

```ts
import { catalogedAssetStore, mediaCatalog } from 'nexus-ai-pro/media/catalog';

const catalog = mediaCatalog({ store: new PostgresStore(pool) });
const assets = catalogedAssetStore(new S3AssetStore({ client, bucket: 'media' }), catalog, { name: 's3' });

// Every image a model made this week that a judge scored below 0.5.
const page = await catalog.list({
  tenantId: 'acme',
  kind: 'image',
  since: '2026-10-12T00:00:00Z',
  scoreBelow: { key: 'sharpness', below: 0.5 },
});
```

A `MediaCatalogEntry` holds what the store knows of the asset (its id, tenant, `CatalogMediaKind`,
MIME type, URI, size, dimensions, file name, and SHA-256), where it came from (the provider, model,
operation, request, parent assets, and the run and trace its provenance names as `runId` and
`traceId`), its tags, its metadata, and a `CatalogScore` per key, the latest each evaluation gave
it. Each change raises its `version`, and is written by compare-and-set on a store that has
`putIfVersion()`, as all four included stores do, so ten evaluators scoring one asset at once keep
all ten scores.

`MediaCatalog` methods:

| Method | What it does |
| --- | --- |
| `record()` | Records an asset from its `AssetStat`, with the `MediaCatalogRecordOptions`: tags, metadata, the store's name, and a run and trace. The wrapper calls it. |
| `forget()`, `get()` | Removes or reads one entry. |
| `list()` | Entries that match a `MediaCatalogQuery`, newest first, as a `MediaCatalogPage` with a `next` cursor: by tenant, kind, MIME type, provider, model, operation, run, store, tag, time, a score below or at least a value, or no score yet. |
| `score()`, `tag()` | Records a score, keeping the latest per key, or adds and removes tags. |
| `attach()` | Records scores on every stored asset a value refers to, such as the images in a run's output. An online evaluation's `onScored` calls it with the run's output. |
| `stats()` | `MediaCatalogStats`: assets and bytes, per kind. |
| `purgeExpired()` | Removes the entries of assets that expired, which are never read again. |
| `reconcile()` | Removes the entries of assets gone from their store, as when they were deleted around the catalog. |

`MediaCatalogOptions` take the `store`, a `namespace` (`['nexus', 'media']`), and a clock.
`CatalogedAssetStoreOptions` take the store's `name`, the `tags` each asset is recorded with, and an
`onError`. Without `onError`, an asset the catalog cannot record is deleted again and the `put()`
rejects, so the catalog and the store always agree; with it, the failure is reported and the asset
kept.

In the test suite, forty assets of five kinds, two providers, and two tenants are stored, some
deleted and some expiring, on each of the four long-term stores: every page lists each remaining
asset once, every filter finds exactly its assets, and the expired ones are never read and are
purged with their assets.

## Limitations

- A vision model's reading is only as faithful as the model; an OCR service keeps layout better.
- Mistral's price per page is not bundled, since it differs by account and model; set `pricePerPage`.
- Video and audio are not media here; transcribe audio with the voice family.
- Without a manifest, a media index does not remember which media it holds: keep the report's chunk
  ids to remove one.
- The catalog records what is stored through a wrapped asset store, or given to `record()`. Assets
  stored before it existed are recorded by passing each one's `stat()` to `record()`.
- A catalog query filters by score, time, and absent scores after reading, so `list()` reads at most
  `scanLimit` entries (10,000) to fill a page and returns a `next` cursor to carry on from.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/decisions.md

# Typed decisions


Software asks a model many small questions it then acts on: which team takes a ticket, whether a
message is an attack, how well an answer meets a rubric. A decision, from `nexus-ai-pro/decisions`, is
such a question asked as a type: a yes or no, a choice among options, a score on a rubric, or where
something is in a picture, about one state, text, JSON, or the pictures it holds. Each answer comes back
with its probabilities, or its box, and a confidence, so the application acts on it without parsing
text, and knows when not to.

```ts
import { NexusAI } from 'nexus-ai-pro';
import { completionDecisions, DecisionManager, gate } from 'nexus-ai-pro/decisions';
import { TypeSafeDecisionProvider } from 'nexus-ai-pro/decisions/typesafe';

const ai = new NexusAI({ providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } } });
const decisions = new DecisionManager(
  {
    providers: {
      jev: new TypeSafeDecisionProvider({ apiKey: process.env.TYPESAFE_API_KEY! }),
      gpt: completionDecisions(ai, { model: 'gpt-4.1-mini' }),
    },
    fallbacks: ['gpt'],
  },
  { lifecycle: ai.lifecycle },
);

const { answers } = await decisions.decide({
  state: ticket,
  questions: {
    team: { type: 'choice', options: { billing: 'Charges and invoices', technical: 'Errors and outages' } },
    urgent: { type: 'yes-no', question: 'Does the customer need an answer today?' },
    severity: { type: 'score', levels: ['Cosmetic', 'Degraded', 'Blocking'] },
  },
});
if (gate(answers.team, { act: 0.8 }) === 'act') route(answers.team.choice); // 'billing' | 'technical'
```

## Questions and answers

A `DecisionRequest` holds a `DecisionState`, text or any JSON value, and `DecisionQuestions`: each a
`DecisionQuestion` under the name its answer comes back by, asked in isolation against the same state.
A `DecisionQuestionType` is one of four:

| Question | What it asks | Its answer |
| --- | --- | --- |
| `YesNoQuestion` | A yes or no, with `criteria` for what counts as each | A `YesNoAnswer`: `yes`, its `probability`, and `confidence` |
| `ChoiceQuestion` | One of its `options`, by name, with what each means, up to 255 | A `ChoiceAnswer`: the `choice`, every option's `probabilities`, and `confidence` |
| `ScoreQuestion` | A level of its `levels`, lowest first, two to ten | A `ScoreAnswer`: the weighted `score`, the most probable `level`, each level's `probabilities`, and `confidence` |
| `BoxQuestion` | Where something is in a picture the state holds, its `question` naming the picture | A `BoxAnswer`: the `box`, a `DecisionBox`, and `confidence` |

The answers are typed by their questions: `AnswerOf` gives a question's `DecisionAnswer`, and
`DecisionAnswers` the whole set, so a choice's answer names only its own options, and a misspelt one
fails to compile. A request also names its `provider` and `model`, its `tenantId`, a `signal`,
`metadata` for the trace, and `cache: false` to ask afresh. A `DecisionResult` holds the answers, the
`provider` and `model` that gave them, its `DecisionUsage`, its `cost`, how long it took, and whether
it was `cached`.

Confidence is read from the probabilities the same way whichever provider gave them, as TypeSafe AI
documents it for its own:
- `yesNoConfidence()`: how far the probability is from an even chance, `|2p - 1|`;
- `choiceConfidence()`: how far the top probability is above an even split, `(pmax - 1/n) / (1 - 1/n)`;
- `scoreConfidence()`: one less the probability-weighted distance from the most probable level, over
  that distance for an even spread, and 0 at an even spread or wider.

Confidence is a margin, not a probability: a yes at 0.75 has confidence 0.5. `answerProbability()` is
the probability an answer gives its own answer, which is what calibration compares with accuracy.

## Decisions about pictures

A state that is an object or a list may hold pictures anywhere in it: image parts,
`{ type: 'image', source }`, held as bytes, base64, a file, a URL, or an asset, and the assets an image
operation returns. A question names each by its path in backticks, `photo`, `faces.happy`, or
`frames[2].frame`, and every question type may ask about one.

```ts
import { readFile } from 'node:fs/promises';

const photo = { type: 'image', source: { buffer: await readFile('claims/7.jpg'), mimeType: 'image/jpeg' } } as const;
const { answers } = await decisions.decide({
  state: { claim: 'Water damage to the kitchen door', photo },
  questions: {
    damaged: { type: 'yes-no', question: 'Does `photo` show damage to a door?' },
    where: { type: 'box', question: 'the damage in `photo`' },
  },
});
if (answers.damaged.yes && gate(answers.where, { act: 0.7 }) === 'act') crop(answers.where.box);
```

A box question always gets a box, even for something the picture does not show, so ask a yes or no
beside it, as above. A `DecisionBox` has four edges, each a fraction from 0 to 1 of the picture's
height or width from its top-left corner; an answer whose edges fall outside that, or cross, is refused
as malformed. `boxOverlap()` measures how far two boxes overlap, the area they share over the area they
cover together, and `answeredRight()` counts a box right when it overlaps the expected one by half or
more, or by its `overlap` option. A box has no probabilities, so its confidence stands in for one, and
`calibrate()` chooses a threshold for boxes as it does for any answer.

The manager refuses a picture that is the whole state, which no path names, and a box asked of a state
with no picture. A provider declares `DecisionProviderCapabilities.images` when it reads pictures; one
that does not is refused a decision about a picture, and passed over as a fallback. A picture is never
written into a trace or a cache key: both hold a digest of its bytes, or its path or URL, in its place.
The budget holds each picture at 1,600 tokens, what one within 1,568 pixels a side costs at most, and
never reads its base64 as text.

`statePictures()` lists a state's pictures, each a `DecisionPicture` with its path, and `readPicture()`
reads one into a `ReadDecisionPicture`, with its bytes, its MIME type as its bytes say, and its size, by
the media family's readers: a file only with `readFile`, a URL only with `fetch`, and a stored asset
from an asset store. A picture it cannot read, or whose bytes are not an image, fails the decision,
naming its path.

## Gates and calibration

`gate()` turns a confidence into a `GateOutcome`: `act` at or above `GateThresholds.act`, `review` (a
person decides) at or above `review`, and `abstain` below. Thresholds belong to the risk of what the
answer does: a low bar to route a ticket, a high one to refund an order.

Choose them from data. `calibrate()` takes `CalibrationSample` values, each an answer and whether it
was right, as `answeredRight()` reads it, and returns a `CalibrationReport`:
- `expectedCalibrationError`, the gap between the probability answers gave themselves and how often
  they were right, from the `CalibrationBin` ranges it counts in;
- with `CalibrateOptions.targetAccuracy`, the `threshold`: the lowest confidence at which the answers
  at or above it hold the target, keeping at least `minSamples`, and the `coverage` it acts on.

```ts
const samples = labelled.map(({ answer, expected }) => ({ answer, correct: answeredRight(answer, expected) }));
const { threshold, coverage } = calibrate(samples, { targetAccuracy: 0.95, minSamples: 100 });
```

A model whose confidence runs high is chosen a higher threshold, and the threshold holds on answers
like the ones it was chosen from, not on a different kind of question or state.

## The manager

`DecisionManager` sends each request to the provider it names, the `DecisionConfig.defaultProvider`,
or the first registered, and checks it first: a malformed question, or one the provider cannot take,
is refused with a `DecisionError` before anything is sent. Its `DecisionErrorCode` is
`DECISION_INVALID`, `DECISION_UNSUPPORTED`, `DECISION_NO_PROVIDER`, or `DECISION_FAILED`, and
`retryable` says whether another provider may help. `fallbacks` are asked in order when one is: a
timeout, a refused connection, or an overloaded service, never a malformed request or a refused key.
`registerDecisionProvider()`, `hasDecisionProvider()`, and `listDecisionProviders()` manage providers
by name, and `decide()` asks one provider with none of this.

`DecisionConfig.cache` keeps each decision's answers in any cache adapter, the memory, Redis, or SQLite
one, as `CachedDecision` values, so the same questions about the same state, for the same tenant, are
answered once. `prices` add `DecisionPrice` rates, per million tokens, to `DECISION_PRICES`, which holds
TypeSafe AI's published price for Jev, $42 per billion input tokens with output free, and
`priceDecision()` prices a decision's usage, leaving one without a price unpriced, never free.

`DecisionRuntime.lifecycle` runs every decision as an operation of the client, family `decision`:
authorized, rate limited, audited, on the tenant's own key, and held against the budget at its model's
price before it is sent. `DecisionRuntime.tracer` records each one as a run of kind `decision`, with its
answers, usage, and cost, and the provider's own calls nested beneath; `DecisionTracerLike` and
`DecisionRunLike` are the parts of a tracer it uses.

## Providers

`TypeSafeDecisionProvider`, from `nexus-ai-pro/decisions/typesafe`, speaks TypeSafe AI's System One API,
`POST /v1/systemone`. Its model, Jev (`jev-latest`, `jev-preview`, or a version), answers only typed
questions, all of a decision in one call, each in isolation, with no text to parse.
`TypeSafeDecisionProviderOptions` sets the `apiKey`, a `baseUrl` (a proxy that passes the API through
works), the `model`, `longContext`, which sends the header TypeSafe AI requires for a long state,
`media`, the readers for pictures held by reference, a `timeoutMs` (30 seconds), `headers`, and a
`fetch`. A refusal of a field names it, from the API's validation error, and a 400 or 413 is a refused
decision, not a failure to try again; `listModels()` reads `GET /v1/models`.

The pictures a state holds go to Jev as the API's assets, each under a new reference in its place in
the state, with the `TypeSafe-Danger-Zone-Images` header that pictures and boxes require. Each goes
as a part of its own in a multipart request, as bytes, after the request itself; `pictures: 'base64'`
sends them within the JSON instead, a third larger. TypeSafe AI takes a still PNG or JPEG of at most
16 MiB, from 280 to 1,568 pixels a side, in a request of at most 32 MiB, counted in bytes as sent, and
refuses one outside them rather than resize it, so the adapter refuses it first, naming its path,
before anything is sent. Three pictures of 10 MiB fit as parts; as base64 they do not. A box needs Jev 1.14 or later, which `jev-latest` serves; a decision
asking one of an earlier version is refused as unsupported.

`completionDecisions()` makes any completion model a provider, through a client's own `complete()`, so
every call is routed, budgeted, traced, and priced as every completion is. Each question is asked on its
own, at temperature 0, to answer with one code: a letter per option, Y or N, or a level's digit. Its
probabilities are read from the log probabilities of that token's alternatives, on a provider that
reports them, OpenAI's models on its Responses API among them; elsewhere it is asked `samples` times
and each answer counted. A model that reports the chosen token alone, with no alternatives, as Cohere's
does, is sampled too: the chosen token's probability says nothing of the others'.
`CompletionDecisionsOptions.method`, a `CompletionDecisionMethod`, is `logprobs`, `sampling`, or `auto`,
the default, which samples a model that reports none or refuses them, and remembers it. A
`DecisionCompletionClient` is anything with `complete()`, such as `NexusAI`. Its calls are charged
where they are made, so its decisions charge nothing again.

A vision model is shown the pictures a state holds: the state's text with each picture's place marked,
then each picture labelled by its path, then the question. A box is asked `samples` times at the
sampling temperature, and is the median of each edge; its confidence is how far the samples overlap
that box, a sample with no box agreeing with nothing, and `method: 'logprobs'` refuses one. Set
`images: false` for a text-only model, which takes no box and no picture.

`MockDecisionProvider`, from `nexus-ai-pro/decisions/mock`, answers as `MockDecisionProviderOptions.answer`
says, or with an even spread, and keeps every request it was asked.

A provider of your own implements `DecisionProvider`: `decide()` takes a `DecisionProviderRequest`, its
model, state, and each `DecisionProviderQuestion`, and a `DecisionCallContext`, and returns a
`DecisionProviderResult` of each question's `DecisionDistribution`. The manager reads every answer from
those with `decisionAnswer()`, so confidence is the same everywhere. Its `DecisionProviderInfo` names it
and declares its `DecisionProviderCapabilities`: the question types, options, levels, and questions it
takes, whether it reads pictures, and its default model. Its state holds the pictures themselves, for
`statePictures()` and `readPicture()`, and a box comes back as `{ type: 'box', box, confidence }`.

## Decisions where the package decides

Each takes a `DecisionManagerLike`, such as a `DecisionManager`, and `DecisionRequestOptions` for whom
and what to ask:

| Function | Where it decides | What an uncertain answer does |
| --- | --- | --- |
| `decisionRoute()` | A graph's branch, for `addConditionalEdges()`, by a choice whose options name nodes, or by `routes` | Goes to `review`, such as a node that asks a person with `interrupt()` |
| `decisionPermissions()` | An agent's tool calls, after a `base` policy: what it allows or refuses stands | Still asks a person |
| `decisionGuard()` | Agent middleware asking a yes or no of each message and each answer: a confident yes stops it | Waits for a person, who answers a `DecisionGuardReview` |
| `decisionModel()` | Agent middleware sending each turn to a model by a choice about its message, every call of the turn | Keeps the agent's own model |
| `decisionEvaluator()` | A judge's score, with the answer's confidence in its metadata | Goes to people with `reviewUnsure()`, as online evaluation's `reviewWhen` |

`DecisionRouteOptions`, `DecisionPermissionsOptions`, `DecisionGuardOptions`, `DecisionModelOptions`, and
`DecisionEvaluatorOptions` set each one's question, threshold, and state. An evaluator may ask a box,
and given `where` the thing is, such as from the example's expected output, scores how far the box
overlaps it, passing at `overlap`, one half by default. `classifyRoute()`, from the
workflows, takes `decisions` too, and then answers with every label's probability.

## Conformance

`runDecisionConformance()` checks any provider against the contract and lists what fails, as a
`DecisionConformanceReport`: its capabilities are coherent, each question type it declares comes back
as probabilities, or a box, that fit, a provider that reads pictures answers about one, several
questions in one decision each get their own answer, a stopped call is not answered, and each
`DecisionConformanceCase` of `DecisionConformanceOptions.cases`, a question whose answer is known, is
answered right; a case's `expected` may be a box. The checks of pictures and boxes need a `picture`;
without one they are listed as `skipped`, not passed.

In the test suite, one decision suite passes on TypeSafe AI's API, against a stand-in written from its
documentation, on a model's log probabilities, and on sampling. On 2,000 labelled invoices, the
threshold a calibration chooses holds its accuracy on 2,000 it never saw, for a calibrated model and for
an overconfident one, where a threshold taken on trust does not. A support graph routes tickets by
decision, and every ticket it is unsure of reaches a person. One decision suite about pictures, of yes
or no, choice, score, and box cases, passes on TypeSafe AI's API, against a stand-in that checks its
documented asset rules and looks at the pixels, and on a vision model, with each box landing where the
picture has it; a model that reads only the text fails it.

## Limitations

- The family is experimental until the live workflow passes against TypeSafe AI's Jev and a model that
  reports log probabilities.
- Confidence is only as calibrated as the model is; choose thresholds from the application's own
  labelled answers, and choose them again when the model or the questions change.
- A completion model's decision costs a call per question, or `samples` calls where it reports no log
  probabilities, and sampled probabilities come in steps of one sample. A choice of more than 20 options
  is sampled.
- TypeSafe AI marks pictures, boxes, and its long-context mode as experimental, and their request
  format and limits may change between its releases.
- A completion model's box costs `samples` calls, and its confidence is how far they agree, not a
  probability the model gave it.
- The adapter does not resize a picture TypeSafe AI would refuse; a decision's pictures, parts or
  base64, travel within the 32 MiB a request may hold.
- Jev is in early access; its adapter has passed against a stand-in written from TypeSafe AI's
  documentation, not yet against the live service.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/video.md

# Video generation


Videos from a prompt, a first or last frame, reference images or videos, or a video to extend or
edit, from `nexus-ai-pro/video`. One contract covers Google's models, Gemini Omni Flash on the Gemini
API and on Vertex AI, and Veo on Vertex AI, and Runway, whose task API also serves Veo, Gemini Omni
Flash, Seedance, and the other models it hosts. A generation is a durable job: it survives the process that submitted it, streams
its video into an asset store, and is priced per second of video.

```ts
import { NexusAI } from 'nexus-ai-pro';
import { FilesystemAssetStore } from 'nexus-ai-pro/images/stores';
import { RedisOperationStore } from 'nexus-ai-pro/operations';
import { VideoManager } from 'nexus-ai-pro/video';
import { GoogleVideoProvider } from 'nexus-ai-pro/video/google';
import { RunwayVideoProvider } from 'nexus-ai-pro/video/runway';

const ai = new NexusAI({ providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } } });
const video = new VideoManager(
  {
    providers: {
      google: new GoogleVideoProvider({ apiKey: process.env.GEMINI_API_KEY! }),
      runway: new RunwayVideoProvider({ apiKey: process.env.RUNWAYML_API_SECRET! }),
    },
    store: new FilesystemAssetStore({ directory: './videos' }),
    tenantId: 'acme',
  },
  { operations: { store: new RedisOperationStore(redis) }, lifecycle: ai.lifecycle },
);

const handle = await video.generate({ prompt: 'A lighthouse at dusk, waves below', durationSeconds: 8 });
const { videos, cost } = await handle.result(); // minutes later: persist handle.id instead of waiting
```

## Requests

A `VideoGenerateRequest` holds the prompt and, as the model allows:

| Field | What it asks for |
| --- | --- |
| `model`, `provider` | The model and provider. Default to the provider's model and the manager's default provider. |
| `image`, `lastFrame`, `references` | A first frame, a last frame, and images whose subject or style the video keeps, as an `AssetInput`: bytes, a URL, or an asset in the manager's store. |
| `referenceVideos` | Videos whose motion, subject, or look the video takes after, as `AssetInput` values of a video type. Their sound is ignored. |
| `video` | A video to extend or edit, as a `VideoSource`: below. |
| `durationSeconds`, `aspectRatio`, `resolution` | Its length, shape (`16:9`, `9:16`), and size (`720p`, `1080p`, `4k`). Extending, the length is what is added. |
| `audio`, `negativePrompt`, `seed`, `count` | Sound, what it must not show, repeatability, and how many videos. |
| `providerOptions` | Fields only one provider knows, sent as they are. |
| `delivery`, `tenantId`, `ttlSeconds`, `metadata` | Where the video goes, who owns it, how long it is kept, and labels for it. |
| `idempotencyKey`, `pollIntervalMs`, `timeoutMs`, `signal` | Replays, polling, and when to stop waiting. |

Before any job is submitted, the request is checked against the model's capabilities. A
`VideoProviderCapabilities` declares the provider's models and what they take, a
`VideoModelCapabilities` for each model where it differs (`byModel`), and `capabilitiesOf()` merges
the two for one model. A `VideoInputKind` is `image`, `last-frame`, `references`,
`reference-videos`, `extend`, or `edit`; a `VideoAudioSupport` is `always`, `optional`, or `none`. A length, shape, size, input, seed, or count
the model does not take is refused with a `VideoError` of code `VIDEO_UNSUPPORTED`, naming what it
does take. A provider receives a `VideoProviderRequest`: the model chosen, and a stored input read
into bytes.

### Extending and editing a video

A request's `video` is a `VideoSource`. Its `mode` is `extend`, which continues the video from its
end, or `edit`, which changes it as the prompt says. The video comes one of two ways:
- `input`, an `AssetInput` of a video type: bytes, a URL the provider reads, or a stored asset;
- `from`, a finished job of the same provider and which of its videos, which the provider continues
  from what it already holds, with nothing uploaded.

```ts
const first = await (await video.generate({ provider: 'google', prompt: 'A marble rolling down a track' })).result();
const longer = await (
  await video.generate({
    provider: 'google',
    prompt: 'The marble lands in a cup',
    video: { mode: 'extend', from: { ref: first.ref } },
    durationSeconds: 4,
  })
).result();
```

A model says what it takes in its capabilities:
- `inputs` with `extend` or `edit`;
- `extendDurations`, the seconds an extension adds;
- `continuesJobs`, whether `from` may name a job of its own;
- `requires`, what every request must give, such as `edit` for a model that only changes videos;
- `maxInputVideoSeconds`, the longest video it extends or edits: 10 seconds on Omni, 30 on Veo;
- and the provider's `inputVideoMimeTypes`.

A mode the model does not take, a job of another provider, a length it does not add, a video type it
does not read, a video longer than it takes, read from the video's own headers, or a request without
what the model requires is refused before anything is sent. The
input travels with a durable generation, so a worker that takes it over sends the same video.

| Provider | Extends | Edits | Continues its own job |
| --- | --- | --- | --- |
| Gemini Omni Flash, on the Gemini API | 3 to 10 seconds, a video of up to 10 seconds | Yes | By the interaction's id, nothing uploaded |
| Veo 3.1 on Vertex AI | 7 seconds, a video in Cloud Storage, up to 37 seconds in all | No | When `storageUri` keeps its videos in Cloud Storage |
| Runway's `gemini_omni_flash_1.1` | 3 to 10 seconds | Yes | By its task's signed URL |
| Runway's `gemini_omni_flash` and `aleph2` | No | Yes, and `aleph2` only edits | By its task's signed URL |

### Reference videos

`referenceVideos` gives a model videos to take after: a motion to follow, a subject to keep, a look to
match. A model that takes them lists `reference-videos` in its `inputs`, the most it takes as
`maxReferenceVideos`, and how long they may be as `referenceVideoSeconds`, each and together. A
video given as bytes says its length in its headers, so one too long is refused before anything is
sent.

| Model | Reference videos |
| --- | --- |
| Gemini Omni Flash, on the Gemini API and Vertex AI | Up to three, of three seconds each; not with a `video` to extend or edit |
| Runway's `gemini_omni_flash_1.1` | Up to three, of three seconds each |
| Runway's Seedance 2 models and `hailuo3` | Up to 15 seconds together, beside up to nine reference images |
| Runway's `wan3` and `wan3_prime` | Up to 15 seconds together, beside up to ten reference images, with or without a first frame |

```ts
const dance = await video.generate({
  provider: 'runway',
  model: 'seedance2',
  prompt: 'The robot dances as the woman in the clip does',
  references: [{ location: { kind: 'url', url: robotPng }, mimeType: 'image/png' }],
  referenceVideos: [{ location: { kind: 'url', url: danceClip }, mimeType: 'video/mp4' }],
});
```

## Durable jobs

`VideoManager.generate()` runs a generation as a durable operation, in the operation store the
`VideoRuntime` names. It records the provider's job, a `VideoJobRef` of plain JSON, the moment the job
exists, and each video as it is stored. A worker that dies is replaced: `recover()`, run on a
schedule in every worker that shares the operation store, takes over a generation whose worker
stopped heartbeating, collects its job instead of submitting another, and skips a video already
stored. A failed job is submitted again only when its `VideoJobError` says that may help, so a prompt
a safety filter refused is never billed twice. A generation has three attempts unless
`VideoRuntime.operations.retry` says otherwise.

In the test suite, a generation whose process dies after submitting, and one whose process dies
after storing its video, are each collected by another worker with one submission to the provider,
one video in the store, and a price for eight seconds.

Every generation runs as one operation of the client's lifecycle, in the `video` family, when the
runtime's `lifecycle` is given: authorized and budgeted before the provider sees it, and its cost
reconciled when the video is collected.

The rest of the manager works from a ref, which any process can hold:

| Method | What it does |
| --- | --- |
| `start(request)` | Submits a job and returns its ref at once, without an operation. |
| `resume(ref, options)` | Waits for a job and collects its videos, as `VideoCollectOptions` asks. |
| `status(ref)` | Reads the job's `VideoJobState` now: its `VideoJobStatus`, progress, and outputs. |
| `download(ref, index)` | One finished video as a stream, a `VideoContent`, to send on without storing it. |
| `cancel(ref)` | Cancels a running job, on a provider that can. |

`registerVideoProvider()`, `hasVideoProvider()`, and `listVideoProviders()` manage providers by name.

## Where videos go

A `VideoResult` holds each video as a `VideoAsset`: an asset descriptor with its length, provenance
naming the provider, model, job, and request, and the request's labels. `generate()` streams each
video into the manager's `store` through `putStream()`, so a large video never sits in memory; a store
that cannot stream buffers it up to `maxBytes`. An operation keeps no bytes, so `generate()` needs a
store. `resume()` can also return the bytes themselves, with `delivery: 'bytes'`, up to `maxBytes`
(512 MiB).

`VideoConfig` sets the providers and default provider, the store, tenant, and `ttlSeconds`, the poll
interval and its backoff, the timeout (1 hour), prices, and `removeFromProvider`, which deletes the
provider's copy once a video is collected. A deletion that fails is a `VideoWarning` on the result,
never a failure. A job that ends without a video raises `VideoJobFailedError`, carrying the ref and
its final state; every failure is a `VideoError` with a `VideoErrorCode`, and `retryable` says
whether trying again may help.

## Prices

`priceVideo(model, seconds, options)` prices seconds of video from `VIDEO_PRICES`, the Gemini API's
list prices published on 2026-10-20, or from the account's own prices. Gemini Omni Flash is billed by
output token; its 720p rate is 5,792 tokens a second at $17.50 a million, about $0.10 a second, and
its other sizes, for which Google publishes no rate, stay unpriced. A
`VideoPrice` is a rate per second, one for every size or one per resolution, and optionally a lower
rate without sound. A model without a price is left unpriced, never recorded as free. A result's
`cost` is what the provider reported, or this price times the length of each video.

## Providers

`GoogleVideoProvider` serves Google's video models. With an `apiKey` it uses the Gemini API, whose
model is Gemini Omni Flash, `gemini-omni-1.1-flash`, since the Veo 3.1 previews shut down on
2026-10-22. Omni runs on the Interactions API as a background interaction: its id is the ref, any
worker reads it, and its video comes back with it, inline or by a Files API URI read with the key.
Omni makes 3 to 10 seconds at 360p to 4k, in `16:9` or `9:16`, with sound, from a prompt, a first
frame, or reference images, and extends and edits videos. `cancel()` and `remove()` cancel and
delete the interaction, which the free tier keeps a day.

With an `accessToken`, a string or a function that returns a fresh one, and a Vertex AI `baseUrl`, it
serves Veo 3.1 on Vertex: `veo-3.1-generate-001`, Fast, and Lite, in preview. It serves Gemini Omni
Flash there too, as `gemini-omni-1.1-flash-preview`, on the same Interactions API at Vertex AI's
global location, the only one that serves it, for the project `baseUrl` names;
`interactionsBaseUrl` names another. A job is a long-running
operation, polled with `fetchPredictOperation`, and each video is read from Cloud Storage, under
`storageUri`, or from the bytes Vertex returns inline. Veo makes 4, 6, or 8 seconds with sound, up to
four videos a request, and extends a video in Cloud Storage by seven seconds. Fast and Lite make no 4k
video, and Lite takes no reference images. `GoogleVideoProviderConfig` also sets the models, where a
name beginning `gemini-omni` is reached through the Interactions API and any other through Veo's
protocol, `byModel`, and `personGeneration`.

`RunwayVideoProvider` speaks Runway's task API: a prompt alone is text to video, a first frame
makes it image to video, and a video makes it video to video. Reference images and videos go with a
prompt alone, and reference videos with a first frame on Wan only. A request's shape and size become Runway's `ratio`, such as `1280:720`, and a
ratio of Runway's own passes through as `aspectRatio`. Each video is read from the signed URL the
finished task lists, which is fetched without the API secret. `cancel()` and `remove()` delete the
task. A video goes by URL, or inline up to 5 MB. `RunwayVideoProviderConfig` sets the key, the API
version (`2024-11-06`), the models (every text-to-video model Runway names, and `aleph2`; `gen4.5`
by default), and `byModel`. Runway's prices are in credits
that depend on the account, so none are bundled: set `prices`.

`MockVideoProvider`, from `nexus-ai-pro/video/mock`, finishes each job after `pollsToComplete` polls
and makes each video on the fly, as large as `bytes` asks, streamed a chunk at a time. Its videos
begin as an MP4 file does, without being playable. It extends and edits too, and continues its own
jobs. `MockVideoProviderOptions` can fail each job, and
two mocks given one `jobs` map, of `MockVideoJob` entries, stand for two processes in front of one
provider.

A provider of your own implements `VideoProvider`: `submit()`, `poll()`, and `download()`, and
optionally `cancel()` and `remove()`, each taking a `VideoCallContext`. Its `VideoProviderInfo` names
it and declares its capabilities.

## Conformance

`runVideoConformance(provider, options)` checks any provider against the contract and lists what
fails, as a `VideoConformanceReport`. Its capabilities must be coherent; its ref must survive JSON as
it would a restart; a job must settle when polled through a ref parsed afresh; and the video must
stream as non-empty bytes of a video type, an MP4 beginning with its `ftyp` box.
`VideoConformanceOptions` adds an image-to-video check; a `video` check, which extends or edits the
video given and, on a model that continues its own jobs, extends the first job too; a
`referenceVideo` check, which makes a video after the one given, where the model takes reference
videos; and a cancellation check, which costs a job and is off by default. In the test suite the mock, Omni, Veo on
the Gemini API and on Vertex AI, and Runway pass it against stand-ins for their APIs.

## Limitations

- The family is experimental until Gemini Omni Flash, Veo, and Runway pass the conformance suite
  against the live services.
- OpenAI removed its Videos API, and the Sora 2 models, on 2026-09-24, so there is no OpenAI adapter.
- Gemini Omni Flash on Vertex AI is in preview, at the global location only.
- Veo extends videos on Vertex AI only, and edits none. Editing or extending an uploaded video on Omni
  is unavailable in the EEA, Switzerland, and the UK.
- A video's length, and a reference video's, is read from its headers when it is given as bytes, and
  one longer than the model takes is refused before it is sent. A video given by URL, or in Cloud
  Storage, is refused by its provider when it is too long.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/images.md

# Image generation and editing (experimental)


Image generation and editing from `nexus-ai-pro/images`, experimental: one contract over OpenAI, Google Imagen, and ComfyUI, masked edits with the mask converted for each provider, input validation that stops a decompression bomb from its header, visual moderation, asset stores for results, and media evaluation. The family leaves experimental once recorded live conformance passes on all three backends.

## Image Generation and Editing (Experimental)

Image operations use a separate provider-neutral manager because generated assets have different
capabilities, delivery modes, safety checks, and lifecycles from text completions.

```ts
import { NexusAI } from 'nexus-ai-pro';
import { OpenAIImageProvider } from 'nexus-ai-pro/images/openai';

const openaiImages = new OpenAIImageProvider({
  apiKey: process.env.OPENAI_API_KEY!,
});

const mediaAi = new NexusAI({
  providers: {},
  images: {
    defaultProvider: 'openai',
    providers: { openai: openaiImages },
  },
});

const image = await mediaAi.images.generate({
  model: 'auto',
  prompt: 'A clean product photograph of a red mechanical keyboard',
  dimensions: { width: 1536, height: 1024 },
  quality: 'high',
  delivery: { kind: 'bytes', format: 'png' },
});

console.log(image.assets[0]?.location);
```

`ImageProvider` is independent from completion providers. Explicit options are negotiated against the
selected provider's declared capabilities, so unsupported formats, delivery kinds, masks, seeds, or
dimensions fail before a provider call rather than being silently dropped.

### Three backends, one contract

The same request runs against OpenAI, Google Imagen, or a self-hosted ComfyUI server. Each adapter is
its own subpath, so an application loads only the backends it registers.

```ts
import { ComfyUIImageProvider } from 'nexus-ai-pro/images/comfyui';
import { GoogleImageProvider } from 'nexus-ai-pro/images/google';

const portable = new NexusAI({
  providers: {},
  images: {
    defaultProvider: 'google',
    providers: {
      openai: openaiImages,
      google: new GoogleImageProvider({ apiKey: process.env.GEMINI_API_KEY! }),
      local: new ComfyUIImageProvider({ baseUrl: 'http://127.0.0.1:8188' }),
    },
  },
});
```

The backends really do differ, and negotiation says so instead of hiding it. Imagen sizes output by
aspect ratio and refuses `dimensions`. It honours `seed` and `negativePrompt`, which OpenAI refuses, and
setting a seed turns its watermark off, which the result reports as a warning. When Imagen filters some
images in a batch, you get the ones that passed plus a withheld finding for each missing one. ComfyUI
queues work and polls for it, records the seed it used so any run can be reproduced, and removes an
abandoned prompt from the server queue. Its graph is yours: pass a `workflow` builder, or use the bundled
`comfyTextToImageWorkflow` and `comfyInpaintWorkflow`.

### Masked edits

Draw the mask once, with either polarity. Each adapter converts it to what its backend expects:
OpenAI's alpha channel, or Imagen's and ComfyUI's white-is-editable greyscale.

```ts
const edited = await portable.images.edit({
  provider: 'openai',
  prompt: 'Replace the sky with a sunset',
  input: { location: { kind: 'bytes', data: photo }, mimeType: 'image/png' },
  mask: {
    location: { kind: 'bytes', data: maskPng },
    mimeType: 'image/png',
    polarity: 'white-is-editable',
    resizeMode: 'reject', // or 'stretch' | 'contain' | 'cover'
  },
});
```

A mask whose size differs from the image is refused unless `resizeMode` allows resampling.
Partly transparent mask pixels count as non-editable, so a soft brush edge never widens the edit.
The PNG codec behind the bundled `PngMaskTransformer` loads on the first masked request, so an
application that never masks never loads it. To accept JPEG or WebP masks, pass your own
`maskTransformer`.

### Validating inputs

Byte uploads, remote URLs, and stored assets all go through the same checks before any provider sees
them:

```ts
import { createImageInputResolver } from 'nexus-ai-pro/images/inputs';

const guarded = new NexusAI({
  providers: {},
  images: {
    providers: { google: googleImages },
    inputResolver: createImageInputResolver({
      maxBytes: 10 * 1024 * 1024,
      maxPixels: 16_000_000,
      allowedDomains: ['cdn.example.com'],
    }),
  },
});
```

The file's own bytes decide its type. A declared or served MIME type that disagrees is refused. Pixel
limits are checked from the header before decoding, so a small file that claims a huge canvas costs
nothing. Remote URLs use the same SSRF protection as the web connector: pinned DNS, redirect
revalidation, and blocking of private networks and cloud metadata endpoints. A refusal is an
`ImageInputError` whose `reason` is machine-readable (`too-large`, `mime-mismatch`, `blocked-url`, and
so on). Without a resolver, nothing runs and nothing is loaded.

### Visual moderation

```ts
import { combineSafetyPolicies, createOpenAIVisualModeration } from 'nexus-ai-pro/images/moderation';

const safety = combineSafetyPolicies(
  createOpenAIVisualModeration({ apiKey: process.env.OPENAI_API_KEY!, reviewThreshold: 0.4 }),
  myBrandPolicy,
);
```

Visual moderation screens the prompt, the input and reference images, and every generated image. An
innocuous prompt can still produce an unsafe image, so text-only checks are not enough. Scores map to
`block` or `review` findings, with per-category thresholds. If the moderation call itself fails, the
request is blocked unless you set `failOpen`.

### Evaluating media

`MediaEvalRunner` from `nexus-ai-pro/images/evals` scores what a string golden file cannot:

- prompt alignment, scored by your judge;
- rendered text, through your OCR function and edit distance;
- whether an edit preserved the rest of the image, through a perceptual hash that survives re-encoding;
- whether content was blocked when it should have been, reported as false-positive and false-negative
  rates.

Each case runs several times and reports mean, spread, and a 95% interval. Scores inside a configured
uncertainty band go to a `ReviewQueue` for a person to judge instead of being decided automatically.

Image support is still marked experimental. The adapters are verified against recorded wire shapes
and a shared conformance suite, which now includes a masked-edit case. The label comes off after the
opt-in live conformance suite passes against each hosted backend.

Use `submit()` for a cancellable local operation handle with replayable lifecycle events:

```ts
const operation = mediaAi.images.submit('generate', {
  prompt: 'A minimal blue geometric poster',
  delivery: { kind: 'bytes', format: 'png' },
});

for await (const event of operation.events()) {
  console.log(event.type);
}
```

For deterministic tests, register `MockImageProvider` from `nexus-ai-pro/images/mock`. The current
`submit()` handle is in-process; durable leases, recovery, distributed deduplication, and webhooks remain
future infrastructure work. `nexus-ai-pro/images/assets` includes a bounded, tenant-isolated
`MemoryAssetStore` for local development and single-process workloads; it computes SHA-256 checksums,
copies bytes at its boundaries, enforces retention and capacity, and never silently evicts live assets.

## Asset Stores

`AssetStore` has three implementations. They share one contract, so retention, tenant isolation, and
checksums behave identically:

```ts
import { MemoryAssetStore } from 'nexus-ai-pro/images/assets';
import { FilesystemAssetStore, S3AssetStore } from 'nexus-ai-pro/images/stores';

const store = new FilesystemAssetStore({ directory: '/var/lib/app/assets', defaultTtlSeconds: 86_400 });

const s3 = new S3AssetStore({
  client,                    // structural: S3, R2, MinIO, or a test double
  bucket: 'generated-media',
  prefix: 'nexus-assets/',
});
```

A missing asset and one owned by another tenant are **indistinguishable** — both return `undefined`
rather than a permission error, because a distinguishable error leaks the existence of another
tenant's asset.

Both durable stores write two objects per asset: the bytes, and a JSON sidecar holding the
descriptor, tenant, and expiry. A single shared index would be a write-contention point and a
corruption blast radius; per-asset sidecars let concurrent writers proceed and lose at most one
record. `purgeExpired()` is exact but costs a directory listing, so on a large S3 bucket prefer the
provider's own lifecycle rules and keep this as the fallback.

### Streaming large assets

A video, a long recording, or a scanned archive is too large to hold in memory at once. Stream it:

```ts
import { createReadStream } from 'node:fs';
import { putAssetStream, readAssetStream } from 'nexus-ai-pro/images/assets';

const stat = await putAssetStream(store, { stream: createReadStream('./lecture.mp4'), mimeType: 'video/mp4' }, {
  tenantId: 'acme',
  provenance,
});

// A range, for a resumed download or a player that seeks.
const part = await readAssetStream(store, stat.assetId, 'acme', { range: { start: 0, end: 1_048_575 } });
return new Response(part?.stream, { status: 206 });
```

The filesystem and S3 stores write a stream a chunk at a time and never hold the whole asset. Its
size limit and SHA-256 checksum are checked as the bytes flow, and an asset that fails either leaves
no file or object behind. They read back a stream, whole or by an `AssetByteRange`, as a
`StreamAssetDescriptor`: the descriptor, the stream, and the bytes it holds. In the release gate,
`npm run bench:streams` moves a gigabyte through each and back while memory grows by less than 128 MB.

| Store | `putStream()` | `getStream()` |
| --- | --- | --- |
| `FilesystemAssetStore` | Writes as the disk takes it. | Reads a range from the file. |
| `S3AssetStore` | Streams to the bucket with a client that has `putObjectStream()`; otherwise buffers within `maxAssetBytes` (64 MiB when unset). | Reads a range with `getObjectStream()`, or the object with `getObject()`. |
| `MemoryAssetStore` | Reads the stream in, within `maxAssetBytes`. | Streams the held bytes. |

`putAssetStream()` and `readAssetStream()` work with any `AssetStore`. They use the store's own
`putStream()` and `getStream()` when it has them. Otherwise they read the stream into memory and call
`put()`, at most `PutAssetStreamOptions.maxBufferedBytes` (`DEFAULT_BUFFERED_ASSET_BYTES`, 64 MiB), or
stream what `get()` returns. A `StreamAssetInput` takes an `AssetByteSource`: a web `ReadableStream`,
or any async iterable of bytes, such as a Node.js stream. `AssetStreamOptions` carries the range.

## Requests

Every request shares `ImageRequestBase`. Only `prompt` is required.

| Field | What it sets |
| --- | --- |
| `provider`, `model` | Where it runs. `model: 'auto'` lets the provider choose. |
| `prompt`, `negativePrompt` | What to draw, and what to avoid. |
| `count` | How many images. |
| `dimensions` or `aspectRatio` | Exact size as `ImageDimensions` (width and height), or a ratio. |
| `quality` | An `ImageQuality`: `auto`, `low`, `medium`, `high`, or a provider's own. |
| `outputFormat` | An `ImageOutputFormat`, such as `png`, `jpeg`, `webp`, or `avif`. |
| `delivery` | How results come back. See `ImageDelivery` below. |
| `background` | An `ImageBackground`: `auto`, `opaque`, or `transparent`. |
| `seed` | Makes a run repeatable on providers that honour it. |
| `requestId`, `idempotencyKey`, `signal`, `metadata` | Identity, cancellation, and application data. |

There are two operations, each an `ImageOperation`:

- An `ImageGenerateRequest` is exactly the shared fields.
- An `ImageEditRequest` adds the `input` image, an optional `mask`, and `references` whose style or
  content the edit should follow.

An `ImageMaskInput` is a mask image with its `polarity` stated, because providers disagree about which
colour is editable, and a `resizeMode`. For `submit()`, an `ImageOperationSubmission` pairs an
operation with its request.

`ImageDelivery` has three kinds: `bytes`; a `url` with an expiry; or `stored`, straight into an asset
store.

## Assets

An image going in is an `AssetInput`: where it is, its MIME type, and optionally a file name, a
checksum, and metadata. Where it is — its `AssetLocation` — has one of three kinds, named by
`AssetLocationKind`:

| Location | Holds |
| --- | --- |
| `AssetBytesLocation` | The bytes themselves. |
| `AssetUrlLocation` | A URL, and when it expires. |
| `AssetStoredLocation` | An asset store's URI and the asset's id. |

An image coming out is an `AssetDescriptor`: the same fields, plus width, height, byte length, and an
`AssetProvenance`. The provenance records the provider, model, operation, request, and the assets it
was made from. An `AssetChecksum` is an algorithm and a hex digest.

## Results

An `ImageResult` holds:

| Field | Contents |
| --- | --- |
| `assets` | The images, in order. |
| `usage` | `MediaUsage`: images and bytes in and out, megapixels, and provider units. |
| `safetyFindings` | What moderation found. |
| `warnings` | Each an `ImageWarning`: an option the provider could not honour exactly, its code, and the value asked for. |
| raw response | When you ask for it. |
| `meta` | `OperationMeta`: ids, provider and model, the route taken on failover, times, latency, and cost. Cost is left out rather than guessed when it is unknown. |

## The manager and configuration

`ImageConfig` is the client's `images` option. It sets a `defaultProvider`, `providers` by name, a
`safety` policy, an `inputResolver`, the `tenantId` for stored inputs, an id generator, and a clock.

`ImageManager` is the family behind `ai.images`. Its methods are `registerImageProvider()`,
`hasImageProvider()`, `listImageProviders()`, `generate()`, `edit()`, and `submit()`. `submit()`
returns an in-process handle that emits the operation's lifecycle events.

## Writing a provider

An `ImageProvider` has an `info` and two methods, `generate()` and `edit()`.

`info` is an `ImageProviderInfo`: a name, locality, version, and `ImageProviderCapabilities`. The
capabilities declare what the provider can do, so the manager refuses anything else before calling it:

- the operations it implements, and its models;
- the input locations and delivery kinds it handles;
- input MIME types and output formats;
- fixed dimensions, aspect ratios, qualities, and count limits;
- whether it supports masks, references (and how many), transparency, seeds, and negative prompts.

Each call receives an `ImageProviderCallContext`: the `ProviderCallContext` every provider receives —
request id, signal, deadline, idempotency key, and trace headers — with the operation's id. An
operation runs through the client's [lifecycle](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/lifecycle.md), so it is authorized and audited
like a completion, and a provider that prices in dollars is charged to the shared budget.

## Safety

An `ImageSafetyPolicy` has two checks. `inspectInput()` runs on the prompt and inputs before the
provider sees them. `inspectOutput()` runs on each generated image before you do. Both receive an
`ImageSafetyContext` with the operation, provider, and model.

Each check returns `MediaSafetyFinding` values. A finding has a category, a severity, and an action —
allow, review, or block — plus what triggered it, a confidence, and a message. A blocking finding
raises `ImageSafetyError`.

`createOpenAIVisualModeration()` builds a policy from `VisualModerationOptions`:

| Option | Default |
| --- | --- |
| API key, base URL, `fetch` | — |
| model | `omni-moderation-latest` |
| `blockThreshold` | The provider's own flag. |
| `reviewThreshold` | A lower score that sends the image for review. |
| `categoryThresholds` | Per-category overrides. |
| inspect input, inspect output | Both on. |
| `failOpen` | Off: if moderation itself fails, the request is blocked. |

`combineSafetyPolicies()` runs several policies as one.

## Validating inputs in detail

`createImageInputResolver()` returns an `ImageInputResolver` built from an `ImageInputPolicy`. The
policy includes the web connector's URL rules — pinned DNS, private networks blocked — and these
limits:

| Option | Default | Why |
| --- | --- | --- |
| `maxBytes` | 20 MB | Bounds what is read. |
| `maxPixels` | 40 megapixels | Checked from the header, before decoding. |
| `allowedMimeTypes` | PNG, JPEG, WebP, GIF | Decided by the file's own bytes. |
| `allowUnmeasurableDimensions` | Off | A format whose size cannot be read from its header is what a decompression bomb looks like. |
| `maxRedirects` | 3 | For remote URLs. |
| `timeoutMs` | 15 seconds | For remote URLs. |
| `store`, `tenantId` | — | Where stored inputs are read from. |

A refused input raises `ImageInputError`. Its `reason` is an `ImageInputRejection`: `too-large`,
`too-many-pixels`, `unmeasurable`, `mime-mismatch`, `unsupported-type`, `fetch-failed`, `blocked-url`,
or `not-found`.

## The adapters in detail

**OpenAI.** `OpenAIImageProvider` takes `OpenAIImageProviderConfig`: the API key, base URL,
organization and project headers, default headers, a default model (`gpt-image-2`), `maxInputBytes`
(50 MiB), a `fetch`, `includeRawResponse`, and a `maskTransformer`. When OpenAI's own moderation
refuses, it raises `OpenAIImageProviderError`, with `OpenAIModerationDetails` naming the stage and the
categories.

**Google Imagen.** `GoogleImageProvider` takes `GoogleImageProviderConfig`. Authenticate with an API
key, or with an OAuth `accessToken` and a `baseUrl` for Vertex AI. It also sets the generate model
(`imagen-4.0-generate-001`), the edit model (`imagen-3.0-capability-001`), headers, a `fetch`, a
`maskTransformer`, `personGeneration`, and `includeRawResponse`.

**ComfyUI.** `ComfyUIImageProvider` takes `ComfyUIImageProviderConfig`:

| Option | Default |
| --- | --- |
| `baseUrl` | `http://127.0.0.1:8188` |
| `workflow` | Your graph builder; otherwise the bundled workflows with `checkpoint`. |
| poll interval | 500 ms, backing off to 5 seconds. |
| `timeoutMs` | 10 minutes |
| `dimensions`, `maskTransformer`, headers, `clientId`, `fetch` | — |

A `ComfyWorkflowBuilder` turns a request into a `ComfyWorkflow`: a map of `ComfyNode` values, whose
inputs are literals or links to other nodes. It is given a `ComfyWorkflowContext`: the operation, a
seed, and the uploaded input and mask as `ComfyUploadedImage` values.

**Mock.** `MockImageProvider` returns deterministic images for tests. `MockImageProviderOptions` sets a
name, model, latency, and capabilities.

## Masks and image bytes

An `AssetTransformer` converts a mask for a provider with `prepareMask()`. It is told the `MaskTarget`:
the provider's `MaskSemantics` and the image's size. The semantics are `white-is-editable`,
`black-is-editable`, or `alpha-transparent-is-editable`. It returns a `PreparedMask` of the same size,
with the share of it that is editable as `coverage`.

`PngMaskTransformer` is the bundled transformer. `PngMaskTransformerOptions` sets the luminance
`threshold` (128) and a `maxPixels` guard. `defaultMaskTransformer()` returns a shared one, and
`requireImageDimensions()` reads an input's size or refuses it.

`nexus-ai-pro/images/transform` also exposes the small codec underneath:

| Function | What it does |
| --- | --- |
| `decodePng()` | Decodes into an `RgbaImage`: width, height, and RGBA bytes. `DecodePngOptions.maxPixels` is checked against the header before any pixel memory is allocated. |
| `encodePng()` | Encodes an `RgbaImage` back. |
| `sniffImageType()` | Identifies a `SniffedImage` from the bytes alone: a `SniffedImageFormat` (PNG, JPEG, GIF, WebP, AVIF, or BMP) and its MIME type. |
| `readImageDimensions()` | Reads an `ImageDimensionsInfo` from the header. |

## Media evaluation in detail

`MediaEvalRunner` runs `MediaEvalCase` values through a `MediaEvalTarget`. A case has an id, an
operation, a request, a number of runs, and `MediaEvalExpectations`:

- text the image should contain, with a minimum OCR accuracy (0.8);
- a minimum prompt alignment;
- an image an edit should stay close to;
- whether the request should be blocked;
- latency and cost limits.

`MediaEvalOptions` supplies the `MediaEvalScorers` — an `alignment` judge, an `ocr` function, and a
pixel `decode` — plus a review queue, review bands, and default runs. `MediaEvalRunOptions` names the
experiment and sets a store, metadata, and a signal.

The run returns a `MediaEvalReport`:

| Part | Contents |
| --- | --- |
| `MediaEvalCaseReport` | One per case: each run as a `MediaEvalRunReport` (scores and failures), the pass rate, and `MetricStats` per metric. |
| totals | The overall pass rate and metrics, blocking accuracy, operational totals, and the runs sent to review. |
| experiment | The run as an evaluation experiment. |

A run whose score falls in an uncertain band becomes a `ReviewItem` on a `ReviewQueue`, for a person
to judge. It carries the case, run, reason, scores, image, and findings. `MemoryReviewQueue` keeps
items in memory.

Three helpers do the scoring: `perceptualSimilarity()` compares two images by a difference hash that
survives re-encoding, `textAccuracy()` scores OCR text by edit distance, and `stats()` summarizes
values.

## Asset stores in detail

An `AssetStore` has six methods, and two optional ones for streams. Each of the six may answer
synchronously or with a promise, as `AssetStoreResult` says.

| Method | What it does |
| --- | --- |
| `put()` | Stores a `ByteAssetInput` — bytes only, not a URL — with `AssetPutOptions`: tenant, provenance (`generate`, `edit`, or `upload` for media an application brings in), dimensions, and a `ttlSeconds` or `expiresAt`. Returns an `AssetStat`: the id, tenant, a descriptor pointing back at the store, and its times. |
| `get()` | Returns a `ByteAssetDescriptor`, with the bytes. |
| `stat()` | The `AssetStat`, without the bytes. |
| `delete()` | Removes an asset. |
| `sign()` | Produces a URL through an `AssetSigner`, which receives an `AssetSignerContext`: the stat, the time, and the lifetime asked for in `AssetSignOptions`. |
| `purgeExpired()` | Deletes what has expired. |
| `putStream()` | Optional. Stores a `StreamAssetInput`, checking its size and checksum as the bytes flow. |
| `getStream()` | Optional. Reads an asset, or an `AssetByteRange` of it, as a `StreamAssetDescriptor`. |

Each store has its own options:

| Store | Options |
| --- | --- |
| `MemoryAssetStore` | `MemoryAssetStoreOptions`: `maxEntries` (1,000), `maxTotalBytes` (64 MiB), `maxAssetBytes` (10 MiB), a default lifetime, an id generator, a clock, and a signer. `snapshot()` returns a `MemoryAssetStoreSnapshot` of what it holds against those limits. |
| `FilesystemAssetStore` | `FilesystemAssetStoreOptions`: the `directory`, and the same size, lifetime, id, clock, and signer settings. |
| `S3AssetStore` | `S3AssetStoreOptions`: a `client`, the `bucket`, a `prefix` (`nexus-assets/`), and the rest. The client is an `S3LikeClient` — `putObject`, `getObject`, `deleteObject`, `listObjects`, and optionally `headObject`, `getSignedUrl`, and the streaming `putObjectStream` and `getObjectStream` — so no AWS SDK is a dependency. With the SDK, `putObjectStream` is an `Upload` from `@aws-sdk/lib-storage`, and `getObjectStream` a `GetObjectCommand` with a `Range`. |

Store errors extend `AssetStoreError`:

| Error | When |
| --- | --- |
| `AssetStoreValidationError` | The input is malformed. |
| `AssetStoreCapacityError` | A limit would be exceeded; it names the `AssetCapacityConstraint`. |
| `AssetStoreSigningError` | No URL can be produced. |

## Errors

Every image error extends `ImageError` and has a stable code.

| Error | When |
| --- | --- |
| `ImageValidationError` | The request or an input is invalid. |
| `ImageProviderError` | The provider failed. The next three extend it. |
| `ImageProviderNotFoundError` | No provider serves the request. |
| `ImageCapabilityError` | The provider cannot honour an option. |
| `ImageProviderResponseError` | The response has the wrong shape. |
| `ImageOperationCancelledError` | A `submit()` handle was cancelled. |
| `ImageSafetyError` | A safety check blocked the request or an image. |

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/voice.md

# Voice


Speech-to-text and text-to-speech from `nexus-ai-pro/voice`: transcribe, speak, run a full voice turn through a model, or keep a multi-turn voice session, with OpenAI and Gemini adapters on `nexus-ai-pro/voice/openai` and `nexus-ai-pro/voice/google`, and a contract for your own provider.

## Voice Integrations

Voice support is optional and split into small imports.

Use the provider-neutral core when you bring your own voice service:

```ts
import { VoiceManager } from 'nexus-ai-pro/voice';
```

Use the OpenAI adapter only when your app needs it:

```ts
import { OpenAIVoiceProvider } from 'nexus-ai-pro/voice/openai';
```

This keeps text-only apps small, while voice apps can opt into the larger integration.

### Transcribe

```ts
const ai = new NexusAI({
  providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } },
  voice: {
    defaultTranscriptionProvider: 'openai',
    providers: {
      openai: new OpenAIVoiceProvider({
        apiKey: process.env.OPENAI_API_KEY!,
        transcriptionModel: 'gpt-4o-transcribe',
      }),
    },
  },
});

const transcript = await ai.transcribe({
  audio: { path: './call.wav', mimeType: 'audio/wav' },
});
```

### Speak

```ts
const speech = await ai.speak({
  provider: 'openai',
  text: 'Thanks for calling. I can help with that.',
  voice: 'alloy',
  format: 'mp3',
});

await fs.promises.writeFile('reply.mp3', speech.audio.data);
```

### Full Voice Turn

`ai.voice(...)` runs:

```txt
audio/transcript -> transcribe -> complete -> optional speak
```

The completion still uses the normal Nexus pipeline, so routing, security, context windows, token optimization, cache, tracing, and failover still apply.

```ts
const result = await ai.voice({
  audio: { path: './call.wav', mimeType: 'audio/wav' },
  transcription: { provider: 'openai' },
  completion: {
    model: 'auto',
    messages: [{ role: 'system', content: 'Answer as a helpful support agent.' }],
  },
  transcriptMessage: {
    template: 'Caller said: {{transcript}}',
  },
  speech: {
    provider: 'openai',
    voice: 'alloy',
    format: 'mp3',
  },
});

console.log(result.transcriptText);
console.log(result.response.content);
console.log(result.speech?.audio.data);
```

### Voice Session With App Requests

Use `VoiceSession` when a call has multiple turns, should keep history, and may need to call your app while talking.

The prompt setup is flexible:

- `prompt` describes the assistant or business role.
- `instructions` describe behavior.
- `taskPrompts` add conditional instructions when the caller asks about something specific.
- `tools` let the model call your app, such as booking, CRM, billing, or inventory APIs.

```ts
import { tool } from 'nexus-ai-pro';

const session = ai.createVoiceSession({
  model: 'auto',
  prompt: 'You are a phone booking assistant for a small clinic.',
  instructions: [
    'Keep answers short and natural for a phone call.',
    'Ask one follow-up question at a time.',
  ],
  taskPrompts: [
    {
      name: 'booking',
      when: ['book', 'appointment', 'free slot'],
      instructions: 'When the caller asks about availability, call check_free_slots before offering times.',
      tools: ['check_free_slots'],
    },
  ],
  tools: [
    tool({
      name: 'check_free_slots',
      description: 'Check available booking slots for a date.',
      parameters: {
        type: 'object',
        properties: { date: { type: 'string' } },
        required: ['date'],
      },
      execute: async ({ date }) => bookingApi.freeSlots(String(date)),
    }),
  ],
  toolSelection: 'task',
  speech: { provider: 'openai', voice: 'alloy', format: 'mp3' },
});

const turn = await session.handleTurn({
  transcript: 'Do you have any free slots tomorrow?',
});

console.log(turn.response.content);
console.log(turn.toolSteps);
console.log(turn.speech?.audio.data);
```

You can use only `prompt`, only `instructions`, only `taskPrompts`, or all of them together. For phone calls through Twilio media streams, parse incoming audio/media events in `telephony`, feed complete caller turns into `session.handleTurn(...)`, then send the returned speech audio back through your call transport.

### Custom Voice Provider

Bring any private, local, or hosted voice service:

```ts
ai.registerVoiceProvider('private-voice', {
  info: { name: 'private-voice', isLocal: true },
  async transcribe(request) {
    return {
      text: await myStt(request.audio),
      providerUsed: 'private-voice',
    };
  },
  async speak(request) {
    return {
      audio: {
        data: await myTts(request.text, request.voice),
        format: request.format || 'mp3',
      },
      providerUsed: 'private-voice',
      format: request.format || 'mp3',
    };
  },
});
```

`VoiceSession` remains the batch-oriented path: each turn is transcription -> completion/tools -> optional
speech. For a persistent connection with live microphone audio, streamed remote audio, barge-in, and
provider events, use the [realtime guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/realtime.md). Neither API replaces the other.

## Requests and responses

`TranscriptionRequest` takes the audio as a `VoiceAudioInput`: a file path, a URL, bytes, base64
text, or a readable stream. Each can carry a file name and MIME type, for providers that infer the
format from them.

| Field | What it sets |
| --- | --- |
| provider, model, language | Where and how it is transcribed. |
| `prompt` | Guides spelling and names. |
| temperature | Sampling. |
| `responseFormat` | The provider's format: text, JSON, verbose JSON with timings, or subtitles. |
| `timestampGranularities` | Asks for timings by segment, word, or both. |
| signal, metadata | Cancellation, and application data. |

`TranscriptionResponse` has the text, the provider and model, the language, and the audio's duration.
When asked for, it also has timed `TranscriptionSegment` and `TranscriptionWord` values. The
provider's raw response is included.

`SpeechRequest` takes the text, and optionally:

- the provider, model, and voice;
- a `VoiceAudioFormat`: `mp3`, `wav`, `opus`, `aac`, `flac`, `pcm`, `webm`, or `ogg`;
- the speaking rate, and `instructions` for models that take direction on tone and pace;
- a signal, and metadata.

`SpeechResponse` carries the audio as a `VoiceAudioOutput` — the bytes, their format, and MIME type —
with the provider, model, and voice that produced it.

`VoiceTurnRequest` is a whole turn for `ai.voice()`: listen, answer, and speak.

| Field | What it sets |
| --- | --- |
| audio, or `transcript` | What the user said, as audio or as text already transcribed. |
| transcription settings | How the audio is transcribed. |
| `completion` | The request that answers. |
| `transcriptMessage` | How the transcript joins that request. |
| `speech` | Speech settings, or `false` for a text-only answer. |

`VoiceTranscriptMessageConfig` is the `transcriptMessage` setting. `append: false` leaves the
transcript out, for a prompt that already includes it. `role` makes it a user or system message, and
`template` places it with `{{transcript}}`.

`VoiceTurnResponse` returns the transcription, the text answered, the completion's response, and the
speech.

## Providers and the manager

A `VoiceProvider` is `info` and two optional methods, `transcribe()` and `speak()`. Each receives the
call's `ProviderCallContext` as its second argument, since every transcription and every utterance
runs through the client's [lifecycle](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/lifecycle.md).
`VoiceProviderInfo` gives its name, whether it runs locally, and `supports`, which operations it
implements. `VoiceConfig` is the client's `voice` option: the default transcription and speech
providers and the providers by name.

`VoiceManager` routes each request to the provider it names, else the configured default, else the
first provider that supports the operation. The client uses it, and it also works alone with any
`VoiceCompletionClient` — anything with `complete()`.

| Methods | What they do |
| --- | --- |
| `registerProvider()`, `hasProvider()`, `listProviders()` | Manage providers. |
| `transcribe()`, `speak()`, `runTurn()` | Do the work. |
| `createSession()` | Starts a session. |

Two errors cover failures. `VoiceProviderError` means a provider failed, the named provider is not
registered, or no provider supports the operation; it names the provider and carries the cause.
`VoiceCapabilityError` means the named provider lacks the operation.

`OpenAIVoiceProvider`, from `nexus-ai-pro/voice/openai`, takes `OpenAIVoiceProviderConfig`: the API
key, base URL, organization, `transcriptionModel` (`gpt-4o-transcribe` by default), `speechModel`
(`gpt-4o-mini-tts`), `defaultVoice` (`alloy`), extra headers, and a `fetch`.

`GoogleVoiceProvider`, from `nexus-ai-pro/voice/google`, is Gemini, through `generateContent`. It takes
`GoogleVoiceProviderConfig`: the API key, base URL, `transcriptionModel` (`gemini-3.8-flash` by
default), `speechModel` (`gemini-3.8-flash-tts`), `defaultVoice` (`Kore`, one of Gemini's prebuilt
voices), extra headers, and a `fetch`.
- Speech comes back as WAV, the default, or raw 16-bit PCM, whether the model answers with a WAV file
  or with raw PCM at the rate it names; any other format is refused before the call, since Gemini
  makes none. `instructions`, and a `speed` other than 1, direct the delivery as a system instruction,
  so a model that speaks its text word for word never speaks them.
- A transcript is what a text model hears: the words alone, in the language spoken, guided by
  `language` and by `prompt`, the names and spellings the audio may hold. Audio is sent inline, up to
  14 MB, or by its Files API URI given as the `url`; any other URL is fetched first. The response
  has the audio's length, from its headers. A request for timings or subtitles is refused, since
  Gemini's transcript has none.

```ts
import { GoogleVoiceProvider } from 'nexus-ai-pro/voice/google';

const ai = new NexusAI({
  providers: { google: { apiKey: process.env.GEMINI_API_KEY! } },
  voice: { providers: { gemini: new GoogleVoiceProvider({ apiKey: process.env.GEMINI_API_KEY! }) } },
});
const { audio } = await ai.speak({ provider: 'gemini', text: 'Your order has shipped.', voice: 'Puck', instructions: 'Warm and brief.' });
```

## Sessions

`VoiceSessionConfig` sets up a conversation.

| Field | What it sets |
| --- | --- |
| `systemPrompt`, `prompt`, `instructions` | Prompt text, joined in that order. Each is a `VoicePromptText`: one string, or several joined with blank lines. |
| `taskPrompts` | Prompts that apply only when their task matches. |
| `messages` | The conversation so far. |
| `tools`, `toolSelection` | The tools, and which are offered: `all`, or `task` for only the matched task prompts' tools (every tool when none matched). |
| `maxToolIterations` | Rounds of tool calls per turn. Defaults to 4. |
| sampling, `responseFormat`, `stop`, `userId`, `metadata` | Passed to each completion. |
| transcription, transcript message, speech | Settings every turn shares. |
| `maintainHistory` | Keeps the conversation between turns. On by default. |
| `onToolCall` | Called after each tool call. |

A `VoiceTaskPrompt` has a name, prompt text and instructions added when it applies, the tools it
needs, and `when`, a `VoiceTaskPromptMatcher`: a substring, a pattern, a list of either, or a function
that receives a `VoiceTaskPromptMatcherInput` — the turn's transcript, the conversation, and its
metadata. A task prompt without `when` always applies.

`session.handleTurn()` takes a `VoiceSessionTurnInput`: audio or a transcript, and settings for this
turn only — transcription, speech, completion fields merged over the session's, and extra prompt
text, instructions, task prompts, and tools. It returns a `VoiceSessionTurnResponse`: the session
id, the transcription and the text answered, the response, the speech, each tool call as a
`VoiceSessionToolStep` (its round, call id, tool, arguments, and result or error), the task prompts
that matched, and the conversation after the turn. `getHistory()` copies the conversation, and
`reset()` starts it over.

`VoiceSession`, from `nexus-ai-pro/voice/session`, can be built without a client: it takes the
configuration, a `VoiceSessionRuntime` (anything with `transcribe()` and `speak()`, such as a
`VoiceManager`), and a `VoiceSessionCompletionClient`.

## Limitations

- Each turn is a request and a reply: the user finishes speaking, then the answer is produced. For
  interruption and streamed audio, use realtime.
- `OpenAIVoiceProvider` and `GoogleVoiceProvider` are the bundled adapters; others implement
  `VoiceProvider`. The Gemini adapter is experimental until a live run is recorded.
- Gemini speaks WAV or raw PCM alone, and transcribes without timings.
- Audio preprocessing is limited to what the provider does: no resampling, denoising, or format
  conversion happens here, and providers differ in the formats and languages they accept.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/realtime.md

# Realtime voice agents


Realtime voice agents from `nexus-ai-pro/realtime`: a session over WebRTC in the browser or WebSocket on a server, with barge-in, tools that can require confirmation, reconnection, metrics, and conversation exports. The browser entry points use no Node API.

## Realtime Voice Agents

Realtime is opt-in and is not re-exported from the package root or from `nexus-ai-pro/voice`:

```ts
import {
  createRealtimeAgent,
  defineTool,
  OpenAIRealtimeProvider,
} from 'nexus-ai-pro/realtime';
```

The framework-independent session normalizes connection, speech, transcript, response, tool,
interruption, error, conversation, and metrics events while retaining provider events when configured.
The browser and server transports use structural platform interfaces, so the realtime core does not
require React, a DOM shim, or an OpenAI SDK.

### Browser WebRTC

Give the browser a same-origin `sessionEndpoint`; never put a permanent provider API key in browser
code. The endpoint receives the SDP offer and returns the SDP answer while your server owns the
provider key and authoritative session configuration.

```ts
type BookingInput = { startTime: string; endTime: string; timezone: string };

const createBooking = defineTool({
  name: 'create_calendar_booking',
  description: 'Create a calendar booking after the user confirms it.',
  parameters: {
    type: 'object',
    properties: {
      startTime: { type: 'string', format: 'date-time' },
      endTime: { type: 'string', format: 'date-time' },
      timezone: { type: 'string' },
    },
    required: ['startTime', 'endTime', 'timezone'],
    additionalProperties: false,
  },
  requiresConfirmation: true,
  execute: async (input: BookingInput, context) =>
    calendar.create(input, { idempotencyKey: context.idempotencyKey, signal: context.signal }),
});

const provider = new OpenAIRealtimeProvider({
  sessionEndpoint: '/api/realtime/session',
});

const agent = createRealtimeAgent({
  provider,
  model: 'gpt-realtime',
  modalities: ['audio', 'text'],
  instructions: 'Check availability before booking. Never write without confirmation.',
  tools: [createBooking],
  voice: {
    transport: 'webrtc',
    voice: 'alloy',
    turnDetection: { type: 'server_vad', silenceDurationMs: 450 },
    interruption: {
      enabled: true,
      cancelResponse: true,
      truncateUnheardAudio: true,
    },
  },
  toolExecution: { mode: 'automatic', timeoutMs: 8_000, maxParallelCalls: 3 },
  reconnect: { enabled: true, maxAttempts: 3, initialDelayMs: 500 },
  conversation: { captureTranscripts: true, captureToolCalls: true, maxRawEvents: 2_000 },
  security: {
    maxSessionDurationMs: 30 * 60_000,
    maxAudioDurationMs: 20 * 60_000,
    toolAllowlist: ['create_calendar_booking'],
  },
  telemetry: {
    includeTranscripts: false,
    estimateCost: (usage) => pricing.estimateRealtime(usage),
  },
});

agent.on('tool.confirmation.required', ({ call }) => {
  const approved = window.confirm(`Create booking requested by ${call.name}?`);
  agent.confirmTool(call.callId, approved);
});

agent.on('metrics', (metrics) => {
  console.log('speech end -> first audio', metrics.speechEndToFirstAudioMs);
});

const audioElement = document.querySelector<HTMLAudioElement>('#assistant-audio')!;
await agent.connect({ microphone: true, audioElement });
```

WebRTC attaches the selected microphone track directly to the peer connection and assigns the remote
media stream to `audioElement`. Browsers still require HTTPS (or localhost), user permission, and often
a user gesture before audio playback. `sendAudio()` is intended for transports that accept binary
chunks; normal WebRTC microphone use should stay on the media track.

WebRTC first-audio timing uses the provider's `output_audio_buffer.started` event. WebSocket playback
remains application-owned, so its player must call `markAudioPlayed()` as described below.

A minimal server endpoint can use the focused server helper:

```ts
import { createOpenAIRealtimeCall } from 'nexus-ai-pro/realtime/openai-server';

export async function POST(request: Request) {
  const answerSdp = await createOpenAIRealtimeCall(await request.text(), {
    apiKey: process.env.OPENAI_API_KEY!,
    model: 'gpt-realtime',
    session: {
      instructions: 'You are a concise scheduling assistant.',
      max_output_tokens: 1_024,
    },
  }, request.signal);

  return new Response(answerSdp, { headers: { 'content-type': 'application/sdp' } });
}
```

Authenticate that endpoint, authorize the tenant and requested tools, apply origin/CSRF and rate-limit
controls, and keep its accepted session options allowlisted. The client must not choose arbitrary tools,
instructions, limits, or credentials.

### Server WebSocket

Server-to-server agents can inject their WebSocket implementation and keep provider authorization in
request headers. The package deliberately does not force a WebSocket dependency:

```ts
import {
  OpenAIWebSocketTransport,
  type OpenAIWebSocketFactory,
} from 'nexus-ai-pro/realtime/openai-websocket';
import { createRealtimeSession } from 'nexus-ai-pro/realtime/session';

const webSocketFactory: OpenAIWebSocketFactory = (url, { headers, protocols }) =>
  serverWebSocketClient.connect(url, { headers, protocols });

const session = createRealtimeSession({
  provider: 'openai',
  model: 'gpt-realtime',
  transport: new OpenAIWebSocketTransport({
    apiKey: process.env.OPENAI_API_KEY!,
    webSocketFactory,
  }),
  modalities: ['audio', 'text'],
  reconnect: { enabled: true, maxAttempts: 3 },
  connection: {
    remoteAudioSink: (audio) => audioPlayer.enqueue(audio.data),
  },
});

await session.connect();
```

When an application plays WebSocket audio itself, call `session.markAudioPlayed()` when the first bytes
actually reach playback. That makes the primary latency metric—user speech ended to first assistant
audio played—represent user experience rather than network receipt.

The global browser WebSocket path accepts only an ephemeral token. Supplying a permanent API key to a
browser adapter is rejected; use an injected server factory or an ephemeral-token provider instead.

### Interruption, tools, state, and exports

With interruption enabled, a `speech.started` event can stop local playback, cancel the active response,
and, for WebSocket playback with a known position, truncate unheard assistant audio. Applications can
also call `session.interrupt('manual')`.

Realtime tools support structural `parse`, `safeParse`, or `validate` schemas, execution timeouts,
bounded parallelism, safe-tool retries, duplicate call suppression, idempotency keys, allowlists, and
write confirmation. A tool marked `safe: true` can also opt into cross-call caching with `cache: true`
or `{ ttlMs, key }` when `toolExecution.cache` supplies a structural cache adapter; cache failures are
ignored unless `cacheFailureMode: 'fail'` is configured. `requiresConfirmation` does not replace server
authorization: re-check identity, permissions, business invariants, and idempotency in the tool
implementation.

Session state is available as immutable snapshots and four export formats:

```ts
const json = session.export('json');
const providerEvents = session.export('openai-events');
const transcript = session.export('text');
const analytics = session.export('analytics');

session.on('conversation.updated', (snapshot) => conversationStore.save(snapshot));
session.on('error', (error) => telemetry.recordException(error));
```

`json` contains normalized user/assistant speech, text messages, tool calls/results, interruptions,
errors, and latency/usage metrics. `openai-events` returns retained raw provider events; it is useful for
diagnostics and replay but follows the upstream provider schema. Disable transcript/raw-event retention
for sensitive workloads. `piiHook` redacts normalized conversation/event text, but deliberately does not
rewrite raw provider exports. Telemetry transcript fields are redacted by default; `includeTranscripts`
must be explicitly enabled. `estimateCost` accepts an application-owned pricing callback so model price
changes are not hard-coded into the package. The built-in limits and telemetry hooks are local controls,
not a durable session store, distributed rate limiter, or authorization system.

Focused public imports are available for `realtime`, `realtime/session`, `realtime/tools`,
`realtime/conversation`, `realtime/openai-webrtc`, `realtime/openai-websocket`,
`realtime/gemini-live`, `realtime/openai-sip`, `realtime/openai-server`, and `realtime/mock`. The deterministic mock is suitable for tests and demos;
see [`examples/realtime-scheduling.ts`](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/examples/realtime-scheduling.ts).

## Sessions in detail

`createRealtimeSession()` builds a `RealtimeSession` from a `RealtimeSessionConfig`. Most fields are
optional; the ones you will set first are the provider, the model, and the transport.

| Field | What it sets |
| --- | --- |
| `provider` | A `RealtimeProviderName`. Defaults to `openai`. |
| `lifecycle` | `ai.lifecycle`, to run the session and each of its turns as operations of a client. See [On the client's lifecycle](#on-the-clients-lifecycle). |
| `tenantId`, `userId`, `principal`, `traceContext` | Who the session's operations are for, and the trace they belong to. The tenant and user default to the principal's. |
| `conversationStore`, `resume` | Where the record is kept after each committed turn, and a conversation to carry on. See [Conversations that outlive their process](#conversations-that-outlive-their-process). |
| `video` | `RealtimeVideoOptions`: how often `sendVideoFrame()` sends a frame, and in what format. See [Video and images](#video-and-images). |
| `model` | The realtime model, such as `gpt-realtime`. |
| `transport` | How the session connects: WebRTC, WebSocket, or the mock. |
| `modalities` | Each a `RealtimeModality`: `audio`, `text`, or both. |
| `instructions`, `tools`, `toolChoice`, `toolExecution` | What the model is told, and which tools it may call and how. |
| `audio` | `RealtimeAudioOptions`, below. |
| `interruption`, `reconnect`, `security`, `conversation`, `telemetry` | The option groups below. |
| `connection` | Browser options for `connect()`. |
| `providerSession` | Provider fields passed through as they are. |
| `signal`, `clock`, `idFactory` | Ends the session; replace time and ids in tests. |

The option groups:

| Group | What it controls |
| --- | --- |
| `RealtimeAudioOptions` | Input and output audio. Formats are a `RealtimeAudioFormat` (encoding and sample rate). Turn-taking is `RealtimeTurnDetectionOptions`: `server_vad` waits for silence, `semantic_vad` judges meaning, and `null` leaves it to you. |
| `RealtimeInterruptionOptions` | What barge-in does: `enabled`, `cancelResponse`, `truncateUnheardAudio`, a `stopPlayback` hook, and `getPlaybackPositionMs` for where to cut. |
| `RealtimeReconnectOptions` | `maxAttempts`, and a backoff from `initialDelayMs` growing by `multiplier` up to `maxDelayMs`. |
| `RealtimeSecurityOptions` | Hard limits: `maxSessionDurationMs` and `maxAudioDurationMs` end the session. Also a `toolAllowlist`, `retainTranscripts`, and a `piiHook` that rewrites every transcript first. |
| `RealtimeConversationOptions` | What the record keeps: transcripts, tool calls, raw events (10,000 by default), a `redactTranscript` hook, and `recordImages`, `content` or `digest`. Raw audio events keep the size of their audio, `delta_bytes`, not the audio, unless `retainRawAudio`. `replayItems` caps what a resumed session gives the provider back. |
| `RealtimeTelemetryOptions` | Tracing and metrics without a dependency, so OpenTelemetry plugs in. See the next table. |
| `RealtimeConnectOptions` | What `connect()` takes in a browser: a signal, the microphone, and where audio plays. |

Telemetry and browser playback go through small structural types:

| Type | Role |
| --- | --- |
| `RealtimeTracerLike`, `RealtimeSpanLike` | A tracer and its spans, such as OpenTelemetry's. |
| `RealtimeMeterLike`, `RealtimeMetricInstrumentLike` | A meter and its instruments. |
| `RealtimeTokenUsage` | What `estimateCost` receives: provider, model, token counts, and the raw payload. |
| `RealtimeAudioElementLike` | Anything that plays audio like an `<audio>` element. Or give a `remoteAudioSink`. |

### Session state and methods

A session moves through `RealtimeSessionState`: `idle`, `connecting`, `connected`, `reconnecting`,
`disconnecting`, `disconnected`, and `failed`.

| Method | What it does |
| --- | --- |
| `connect()`, `disconnect()` | Opens and closes the session. |
| `sendText()` | Sends a user message as text. |
| `sendImage()`, `sendVideoFrame()` | Adds an image to the conversation, or streams a frame of video. See [Video and images](#video-and-images). |
| `sendAudio()`, `commitAudio()`, `clearAudio()` | Streams audio by hand, then ends or discards the turn. |
| `createResponse()` | Asks the model to respond now. |
| `updateSession()` | Changes instructions, tools, or audio mid-session; on Gemini Live, by setting up again between turns. |
| `sendEvent()` | Sends a raw `RealtimeClientEvent`. |
| `interrupt()` | Stops the current response. |
| `markAudioPlayed()` | Reports that audio reached the speaker, for accurate latency. |
| `confirmTool()` | Approves or refuses a tool call that needs confirmation. |
| `executeTool()`, `submitToolResult()` | Runs tools yourself in `manual` mode. |
| `getConversation()`, `getMetrics()` | The record and its metrics so far. |
| `getRawEvents()` | Each provider event as a `RealtimeServerEvent`. |
| `whenIdle()` | Resolves when nothing is in flight. |
| `export()` | The record in one of four formats. |

Events arrive through `on()`. Each is a `RealtimeEvent`, and `RealtimeSessionEvents` maps each event
name to its payload. `TypedEventEmitter` is the small emitter underneath; it does not depend on Node's
`events`.

Tests replace time and ids through `RealtimeClock` and `RealtimeIdFactory`. `createRealtimeId()` makes
ids like `<prefix>_<uuid>`, and works where `crypto.randomUUID` is missing.

## Agents and providers

`createRealtimeAgent()` is the friendlier entry point. It takes `CreateRealtimeAgentOptions`, which is
a session configuration with the provider and voice kept separate. `RealtimeAgentVoiceOptions` holds
the voice side: the transport kind, interruption, turn detection, the voice, and audio settings.

A provider creates transports. `OpenAIRealtimeProvider` is a `RealtimeTransportProvider`: a name and a
`createTransport()`. `OpenAIRealtimeProviderOptions` sets the default transport (`webrtc`), the
`sessionEndpoint` for WebRTC, the `apiKey` for server WebSockets, and options for each transport.

`GeminiLiveProvider` is the other bundled provider: Gemini Live, behind the same session, tools,
and conversation record. Its options are those of every `GeminiLiveTransport` it creates
(`GeminiLiveProviderOptions`), and its sessions are recorded as the `google` provider's. One agent
definition runs on either:

```ts
import { createRealtimeAgent, GeminiLiveProvider, OpenAIRealtimeProvider } from 'nexus-ai-pro/realtime';

const definition = { instructions: 'You track orders.', tools: [trackOrder], voice: { interruption: true } };

const onOpenAI = createRealtimeAgent({ ...definition, provider: new OpenAIRealtimeProvider(), model: 'gpt-realtime' });
const onGemini = createRealtimeAgent({
  ...definition,
  provider: new GeminiLiveProvider({ apiKey: process.env.GEMINI_API_KEY! }),
  model: 'gemini-live-2.5-flash-preview',
});
```

In the test suite, one agent definition holds the same conversation on both, each over its own wire
protocol: the person asks where an order is, the model calls the tool, the tool runs once, and the
model answers. The two records match item for item, with the same usage.

Three functions are the building blocks of a custom OpenAI transport:

| Function | What it does |
| --- | --- |
| `createOpenAISessionUpdate()` | Builds OpenAI's `session.update` event from a configuration. |
| `createOpenAIToolResultEvents()` | Builds the events that return a tool result and let the model continue. |
| `normalizeOpenAIRealtimeEvent()` | Turns one OpenAI event into neutral session events. |

## Transports

A `RealtimeTransport` is the connection under a session. It has a `kind`, a `state`, and methods to
connect, send audio and events, interrupt, disconnect, and subscribe with `on()`. A transport whose
provider takes video as realtime input also has `sendVideo()`.

| Type | Meaning |
| --- | --- |
| `RealtimeTransportKind` | `webrtc`, `websocket`, `mock`, or your own. |
| `RealtimeTransportState` | From `idle` to `disconnected` or `failed`. |
| `RealtimeTransportEvents` | What a transport emits, below. |
| `RealtimeTransportConnectedEvent` | The connection is up. |
| `RealtimeTransportDataEvent` | One event from the provider. |
| `RealtimeTransportAudioEvent` | A chunk of model audio. |
| `RealtimeTransportDisconnectedEvent` | The connection closed, and why. |

### Browser WebRTC

`OpenAIWebRTCTransport` takes `OpenAIWebRTCTransportOptions`. Its `OpenAIWebRTCSessionMode` picks how
it authenticates:

- `unified-sdp` sends the browser's offer to your `sessionEndpoint`, which holds the API key. This is
  the default and the safest.
- `ephemeral-token` connects to OpenAI directly with an `OpenAIEphemeralCredential`, given as a value or
  fetched by an `OpenAIWebRTCTokenProvider`.

The other options cover your endpoint's headers, the peer connection (settings, or a
`RealtimePeerConnectionFactory`), the microphone or your own stream or track, a `fetch`, timers, and
timeouts: 15 seconds to connect, and a grace period after an ICE disconnect. You can also name the
data channel and decide whether `sendAudio()` may use it. A session that sends no microphone still
hears the model: the offer asks to receive audio, unless the session's `modalities` are text only.

Every browser API is used through a structural type, so the transport runs without a DOM shim and is
unit-tested without a browser. It also runs in a real one: `npm run test:browser:realtime` drives a
session over WebRTC in headless Chromium, against a second peer connection in the page that plays the
provider. The offer, the answer, ICE, DTLS, and the data channel are real; Chromium's fake microphone
is sent up, a tone comes back as the model's voice and plays, a tool call is answered, and a session
with no microphone still hears the model. That run found two defects the structural tests could not:
the answer's last line lost its line break, which Chromium refuses, and a session without a
microphone offered no audio to receive.

| Browser API | Structural type |
| --- | --- |
| `RTCPeerConnection` | `RealtimePeerConnectionLike` |
| `RTCDataChannel` | `RealtimeDataChannelLike` |
| `RTCSessionDescription` | `RealtimeSessionDescriptionLike` |
| `navigator.mediaDevices` | `RealtimeMediaDevicesLike` |
| `MediaStream`, `MediaStreamTrack` | `RealtimeMediaStreamLike`, `RealtimeMediaTrackLike` |
| `fetch` | `RealtimeFetchLike`, with `RealtimeFetchInitLike` and `RealtimeFetchResponseLike` |
| `AbortController` | `RealtimeAbortControllerLike` |

### Server WebSocket

`OpenAIWebSocketTransport` takes `OpenAIWebSocketTransportOptions`. It authenticates with an `apiKey`,
an ephemeral token, or an `OpenAIEphemeralTokenProvider`. You supply the socket through an
`OpenAIWebSocketFactory`: it receives `OpenAIWebSocketFactoryOptions` (headers and protocols) and
returns a `RealtimeWebSocketLike`. The remaining options set the URL, headers, subprotocols, timeouts
(10 seconds to connect, 1 to disconnect), the largest audio chunk (15 MiB), a text codec, timers, and
a clock. `callId` attaches the transport to a call OpenAI is already carrying, such as an accepted
SIP call, instead of starting a session on a model.

### Gemini Live

`GeminiLiveTransport`, from `nexus-ai-pro/realtime/gemini-live`, speaks Gemini's Live API over a
WebSocket and translates, both ways, between it and the events the session speaks. Its
`GeminiLiveTransportOptions` authenticate with an `apiKey`, sent in the URL, or an ephemeral token,
given as `ephemeralToken` or fetched by `ephemeralTokenProvider`, sent to the constrained endpoint.
A key is refused in a browser. Because the key travels in the URL rather than a header, the runtime's
own `WebSocket` (Node 22 and later) is enough on a server; `webSocketFactory` replaces it. The other
options set the URL, extra `setup` fields, timeouts (10 seconds to connect, 1 to disconnect), the
largest audio chunk (15 MiB), a text codec, timers, and a clock.

What the session sends becomes Gemini's, and what Gemini sends becomes the session's:

| The session | Gemini Live |
| --- | --- |
| Its configuration: model, instructions, tools, voice, language, modalities, turn detection | The `setup` message, at connect. Tool schemas lose `additionalProperties` and `$schema`, which Gemini does not take. |
| A session update, from `updateSession()` | A new `setup`, on a new socket, resumed from the latest handle between turns. |
| Microphone audio, and committing it | `realtimeInput` audio, at the input format's rate (16 kHz by default), and the end of the audio stream. |
| A message, an image, a tool call or result given back on resume | A `clientContent` turn. |
| A request for a response | The end of the person's turn. |
| A tool's result | A `toolResponse`, by the call's id and name. Gemini goes on by itself after it, so the request for a response that follows is dropped. |
| Speech transcribed, the model's audio, text, and transcript | The same events the session reads from OpenAI. |
| A `toolCall` | A finished response that calls the tools. |
| A turn's end, with its usage | A finished response, with its tokens. |
| An interruption | The person's speech starting, then the response cancelled, so the session sees the barge-in. |

Gemini's transcription of the person trails their speech. Words that arrive while the model answers
join the turn it is answering, and that turn's transcript completes when the model's turn does; only
Gemini's own interruption starts a new turn before then, so late words never read as the person
speaking over the model. Gemini does not echo what it is given; the transport does, so the record
keeps typed messages as it does on OpenAI. A tool call Gemini cancels, because the person spoke over it, is not answered.
`sessionResumption`, on by default, asks Gemini for resumption handles: a socket that drops
reconnects with the latest resumable one, and Gemini carries the conversation on where it left it.
Gemini's warning that it will close the socket arrives as a raw `session.go_away` event.

Gemini takes its setup once, at the start of a socket, so `updateSession()` changes it by setting up
again. The update's fields that differ from the setup are folded in: instructions, voice, language,
output modalities, tools, turn detection, and the model, and any other field as a setup field of
Gemini's own. Gemini marks the points it can be resumed from, between turns; at the next one, the
transport closes the socket and opens another with the new setup and that point's handle, and Gemini
carries the conversation on. What the session sends meanwhile waits and goes out in order once the
new socket is set up, and the session sees `session.updated`. An update that changes nothing, as the
session's own after it connects, opens no socket; one made before Gemini has given a handle waits for
it. A new setup that fails is a drop, which the session's reconnect policy decides on.

```ts
session.updateSession({ instructions: 'From now on, answer in French.', audio: { output: { voice: 'Kore' } } });
```

### Server helpers

`nexus-ai-pro/realtime/openai-server` keeps the API key on your server.

| Function | What it does |
| --- | --- |
| `createOpenAIRealtimeCall()` | Exchanges a browser's SDP offer for OpenAI's answer. |
| `createOpenAIRealtimeSessionEndpoint()` | Wraps that as a handler for your route. |
| `createOpenAIRealtimeClientSecret()` | Mints an `OpenAIRealtimeClientSecret` for a browser: a token, its expiry, and OpenAI's response. |
| `acceptOpenAIRealtimeCall()` | Answers a call OpenAI carries with the session that will run it. |
| `rejectOpenAIRealtimeCall()` | Declines it, with a SIP status: 603 by default. |
| `referOpenAIRealtimeCall()` | Transfers an accepted call to another number or SIP address. |
| `hangupOpenAIRealtimeCall()` | Ends it. |
| `verifyOpenAIWebhook()` | Checks a webhook delivery's signature and returns its `OpenAIWebhookEvent`. |

Controlling a call takes `OpenAIRealtimeCallOptions`: the key, the base URL, the safety identifier,
and a `fetch`. `verifyOpenAIWebhook()` takes the delivery's `OpenAIWebhookHeaders`, a `Headers`
object or a plain record, and its raw body, with your endpoint's signing secret. It refuses a
delivery that is unsigned, signed with another secret, altered, or older than
`OpenAIWebhookVerifyOptions.toleranceSeconds` (300), with a `RealtimeError` of code
`invalid_webhook_signature`. It runs on Web Crypto, so anywhere.

They share `OpenAIRealtimeServerOptions`: the key, the model, session settings, a base URL, a
`safetyIdentifier` for the end user, a `fetch`, and a `formDataFactory` for runtimes without
`FormData`. With `credentials`, the resolver the client's lifecycle uses, such as
`tenantCredentials()`, and a `tenantId`, the session an SDP exchange makes and the client secret a
browser connects with are minted with that tenant's key, read for `provider` (`openai`); a tenant the
resolver refuses is refused before anything is sent, and the key is cleaned from every error. The
endpoint `createOpenAIRealtimeSessionEndpoint()` returns takes the tenant per call, as its third
argument, such as the one your route authenticated. Controlling a call OpenAI carries always uses
`apiKey`, the key of the project that received the call. The injected `fetch` is an `OpenAIRealtimeServerFetch` returning an
`OpenAIRealtimeServerFetchResponse`, and the form data a `FormDataLike`.

### The mock

`MockRealtimeTransport` plays a script, for tests and demos. Each step is a `MockRealtimeTransportStep`:
a provider event, audio, an error, or a disconnect. `MockRealtimeTransportOptions` sets the script,
`autoPlay`, a session id, a clock, and `strictState`. Drive it by hand with `enqueue()`, `advance()`,
`flush()`, and the `emit` methods.

## Phone calls over SIP

`answerOpenAISipCalls()`, from `nexus-ai-pro/realtime/openai-sip`, answers phone calls with a
realtime agent. Point a SIP trunk at OpenAI, and OpenAI's `realtime.call.incoming` webhook at the
handler of the `OpenAISipCalls` it returns. For each verified delivery, `answer()` decides from the `OpenAISipCall` (who is
calling, what they called, and every SIP header) and returns an `OpenAISipAnswer`:
- a session's configuration (`OpenAISipSessionConfig`): the call is accepted with it, and a session
  attaches to the call over a WebSocket and runs it, with its tools, conversation record, and
  lifecycle. Its id is the call's;
- `false`, which declines the call with SIP 603, or `{ reject }` with another status, such as 486
  when busy.

```ts
import { answerOpenAISipCalls } from 'nexus-ai-pro/realtime/openai-sip';
import WebSocket from 'ws';

const calls = answerOpenAISipCalls({
  apiKey: process.env.OPENAI_API_KEY!,
  webhookSecret: process.env.OPENAI_WEBHOOK_SECRET!,
  webSocketFactory: (url, { headers }) => new WebSocket(url, { headers }),
  answer: async (call) => {
    const account = await accounts.byPhone(call.from);
    if (!account) return { reject: 486 };
    return { model: 'gpt-realtime', instructions: account.greeting, tools: [bookTable], lifecycle: ai.lifecycle, tenantId: account.id };
  },
  onSession: (session, call) => session.on('error', (error) => log.warn({ callId: call.callId, error })),
});

// Your route, with the body exactly as it arrived:
const { status, body } = await calls.handle({ headers: request.headers, body: rawBody });
```

`handle()` takes an `OpenAISipWebhookRequest` and returns an `OpenAISipWebhookResponse` to reply
with: 200 once the call is accepted or declined, 401 for a delivery your secret did not sign, and
500 when answering failed, so OpenAI delivers it again. Each delivery is claimed by its webhook id
before it is acted on, so a retry or a duplicate answers its call once; a failed one is released, so
its retry is acted on. The claims are remembered in the process, the last 10,000; give `claim` and
`release` backed by a shared store when several replicas take the webhook. The session connects after
the reply; `onSession()` receives it, and `onError()` what fails after. Other verified events go to
`onEvent()`.

An `OpenAISipCall` also transfers the call with `refer()` and ends it with `hangup()`. `sessions`
holds the calls in progress by id, and `close()` hangs each up, for shutdown. `OpenAISipHeader` is one
SIP header, and `OpenAISipOptions` the whole configuration.

In the test suite, a signed delivery of an incoming call is accepted with the agent's session, chosen
from the caller's number. The session attaches by the call's id and books a table through its tool
when the caller asks, and the call is hung up. Forged, altered, and stale deliveries answer nothing. A
retry after a failed accept answers the call, and a duplicate, on the same replica or another, does
not answer it twice.

## Video and images

`sendImage()` adds an image to the conversation as the person's: a photo, a screenshot, a page. It
takes a `RealtimeImageInput` (bytes, a `data:` URL, or base64) with `RealtimeImageOptions`: the `mimeType` (`image/jpeg` by
default), `text` sent in the same message, and `createResponse`, true by default when there is text.
The record keeps it as an `ImageItem`, when it is sent, with a `message.image` event: its type, the
SHA-256 of its bytes, and, unless the conversation's `recordImages` is `digest`, the image itself,
which a resumed session gives back. The words sent with it are a text message after it, once the
provider has them.

`sendVideoFrame()` streams one frame of video, from a camera or a shared screen. On a provider that
takes video as realtime input, such as Gemini Live, the frame goes there; otherwise it goes into the
conversation as an image. `RealtimeVideoOptions` spaces frames out: one sooner than `minIntervalMs`
after the last one sent is dropped (1,000 by default, about what the models take), and the method
returns whether the frame was sent. A frame is live input, not kept in the record, even where it goes
into the conversation as an image.

```ts
const session = createRealtimeAgent({
  provider: new GeminiLiveProvider({ apiKey }),
  model: 'gemini-live-2.5-flash-preview',
  video: { minIntervalMs: 1000 },
});
await session.connect();
camera.onFrame((jpeg) => session.sendVideoFrame(jpeg));
session.sendImage(receiptPng, { mimeType: 'image/png', text: 'What is the total on this receipt?' });
```

## Tools in detail

A `RealtimeTool` is what the model can call.

| Field | Meaning |
| --- | --- |
| `name`, `description` | What the model sees. |
| `parameters` or `schema` | JSON Schema, or a `RealtimeToolSchema` with `parse`, `safeParse` (returning a `RealtimeSchemaResult`), or `validate`. |
| `execute()` | Your implementation. |
| `requiresConfirmation` | A flag, or a function of the input, that makes a person approve the call first. |
| `safe` | The call has no side effects, so it may be retried and cached. |
| `cache`, `metadata` | Caching for safe tools, and application data. |

`defineTool()` checks a tool and, with `DefineRealtimeToolOptions`, infers the input type from its
schema through `InferRealtimeSchema`. `AnyRealtimeTool` is the type-erased shape a mixed list holds.
`toOpenAIRealtimeTools()` converts tools to OpenAI's definitions.

A call flows through three types:

| Type | Contents |
| --- | --- |
| `RealtimeToolContext` | What `execute()` receives: the session id, the call id, an idempotency key stable across retries, the attempt, and a signal. |
| `RealtimeToolCall` | The call as the executor sees it: id, name, parsed arguments, and the raw arguments and parse error when parsing failed. A call that failed to parse is not run. |
| `RealtimeToolResult` | The outcome: the call, whether it succeeded, the result or error, the duration, and the attempts. |

The session's `toolExecution` is a `RealtimeToolExecutionOptions`:

| Option | Meaning |
| --- | --- |
| mode | `automatic` runs tools as the model calls them; `manual` leaves it to you. |
| timeout, parallel calls | A per-attempt timeout, and how many calls run at once. |
| retries | How often, and how long to wait, for `safe` tools only. |
| allowed tools | Intersected with the security allowlist. |
| cache | A `RealtimeToolCache` (`get`, and `set` with a lifetime), and what a cache failure does. |
| confirm | A function that approves calls needing confirmation. |

`RealtimeToolExecutor` is the executor on its own. It takes `RealtimeToolExecutorOptions` — the session
id, a signal, a clock, an id factory — and `RealtimeToolExecutorHooks` for a call starting, needing
confirmation, and completing. Its methods are `register()`, `has()`, `list()`, `execute()`, and
`clearCompleted()`. Each call id runs at most once.

## The conversation record

A `RealtimeConversation` is everything that happened in a session: its id, provider, model, start and
end, a status (`active`, `completed`, or `failed`), the items in order, metrics, and metadata.

`ConversationMetrics` counts turns, tool calls, interruptions, reconnects, errors, audio durations,
tokens, and the estimated cost. `RealtimeLatencyMetrics` times the latest turn: connection setup,
speech end to response, speech end to first audio, tool duration, tool result to first audio, and the
whole turn.

Each item is a `ConversationItem`. They share `ConversationItemBase` — an id, a time, the provider's
item id, the raw event, and metadata — and each is one of these:

| Item | What happened |
| --- | --- |
| `UserSpeechItem` | The user spoke. |
| `AssistantSpeechItem` | The assistant spoke. |
| `TextMessageItem` | A text message, either way. |
| `ImageItem` | The user showed an image: its type, its digest, and, unless the record keeps images by digest, the image. |
| `ToolCallItem`, `ToolResultItem` | A tool was called, and returned. |
| `InterruptionItem` | The user interrupted. |
| `RealtimeConversationErrorItem` | Something failed. |

`nexus-ai-pro/realtime/conversation` holds the record's logic as pure functions. A server uses them to
rebuild a record from stored events:

| Function | What it does |
| --- | --- |
| `createRealtimeConversation()` | Starts a record, from `CreateRealtimeConversationOptions`. |
| `reduceRealtimeConversation()` | Applies one event, with `RealtimeConversationReducerOptions`, to a copy of the record; with `inPlace`, to the record given, for its one owner. |
| `createConversationMetrics()`, `withConversationMetrics()` | Start and update the metrics. |
| `snapshotRealtimeConversation()` | An immutable copy. |
| `exportRealtimeConversation()` | Writes it in a `RealtimeConversationExportFormat`: `json`, `openai-events`, `text`, or `analytics`. |

The `analytics` format is a `RealtimeAnalyticsExport`: durations, item counts, and metrics, without
transcripts.

### On the client's lifecycle

Given `ai.lifecycle`, a session runs through the same lifecycle as every other family. Its
operations carry the session's `tenantId` and `userId` (the principal's by default) and its
`traceContext`:
- the session itself is one operation, `realtime.session`, admitted when it first connects;
- each model turn is one more, `realtime.turn`, admitted as the turn starts: rate-limited,
  authorized, and holding the turns' average cost so far against the budget;
- when the turn finishes, its operation is charged what the turn cost, from `estimateCost`;
- a session's usage and estimated cost are the sum of its turns'.

A turn the lifecycle refuses, because the budget cannot cover it, the rate limit is reached, or
authorization says no, is cancelled before the model runs it. The session then ends with a
`RealtimeError` of code `lifecycle_refused`. A connection that succeeds or fails is reported to the
client's provider health and circuit breaker, as a completion's attempts are.

In the test suite, through a real client: with a five-cent budget and two cents a turn, the first two
turns run, and the third is cancelled as it starts and ends the session. The budget then holds four
cents, and every operation names the session's tenant and user.

## Conversations that outlive their process

Give a session a `conversationStore`, a `RealtimeConversationStore`, and it writes its record after
each committed turn: a finished response, a transcribed utterance, a typed message, and a tool's
result. `MemoryRealtimeConversationStore` keeps records in the process, and
`storeRealtimeConversations()` in any long-term `Store`: the memory, Redis, Postgres, or SQLite store.

Another process carries the conversation on with `resume`: its record, or its id in the store. The
new session takes the conversation's id, items, metrics, and metadata, active again. On connecting,
it gives the provider what was said and what the tools returned, before anything new, so the model
answers with it. `replayRealtimeConversation()` builds those items: speech as text from its
transcript, typed messages, the images the record keeps, tool calls, and their results, the latest
`replayItems` of them. Each
carries an id that begins with `REALTIME_REPLAY_PREFIX`, so the provider's echoes are not recorded
twice. A new transport carries the conversation; the socket and the audio that was in flight are not
recovered.

```ts
import { createRealtimeSession, storeRealtimeConversations } from 'nexus-ai-pro/realtime';

const conversationStore = storeRealtimeConversations(redisStore);
// In the process that takes over, by the id the first one stored:
const session = createRealtimeSession({ model: 'gpt-realtime', transport, conversationStore, resume: conversationId });
await session.connect();
```

In the test suite, a session's process is killed mid-conversation on the memory and SQLite stores.
Another process resumes it by id: the provider is first told what the person said and what the model
answered, and the conversation goes on under its own id, with nothing recorded twice.

## Turn latency and session load

What the library adds to a call is held to a budget, as its size and runtime are.
`npm run bench:realtime` runs sessions over an in-memory transport that delivers a provider's
messages for a spoken turn, each message in a macrotask of its own, as a socket does: the caller's
speech and its transcript, the response, 25 frames of 20 ms audio with its transcript, and the
turn's end. It measures:
- **Added latency**: from a message's arrival to the session's event for it. Held under 5 ms at p99
  for one call, and under 20 ms, one audio frame, with 250 calls at real time.
- **A long call**: 550 turns in one session. A late turn must cost what an early one does, within
  twice, so work that grows with the conversation is caught on any machine.
- **Calls at real time**: 250 sessions, each sent its messages at a call's own pace. The process must
  keep up: event-loop delay under 50 ms at p99, and finished within 100 ms of the calls' length.
- **Throughput and memory**: 1,000 sessions of ten turns, as a ratio to a calibration loop, so a
  budget recorded on one machine holds on a slower one, and the heap each live session holds.
- **Retention**: 1,000 sessions that end leave the heap within 8 MB of where it started.

The budgets are in `realtime-budget.json`, and the release gate runs the check. On the machine that
recorded them, under Node 22:

| Measure | Measured |
| --- | --- |
| A spoken turn, one call | 0.57 ms of the session's time, at p50 |
| Added to a message | 2 µs at p50, 10 µs at p99 |
| 250 calls at real time | 15% of a core: about 1,700 calls per core; event-loop delay 19 ms at p99 |
| A live session | 44 KB, with transcripts and raw events |
| Over WebRTC in Chromium, `sendText()` to the response starting | 0.4 ms at p50, under 1 ms at p95, on loopback |

Writing the method down found what it measures. Before it, a session copied its whole record twice
for every event, once to apply it and once to announce it, so a turn cost 39 ms by the 500th and 250
calls took half a core. It also kept every audio frame it had played, about 13 MB for a 3-minute
call. Now the session applies events to its own record in place, copies it only for a
`conversation.updated` listener, and keeps an audio event's size in place of its audio.

## Errors

Every failure is a `RealtimeError`. It carries a message, the provider's code, the HTTP status, the
provider event that reported it, the cause, and the raw payload. Two flags say what to do: whether a
retry could succeed, and whether the error ends the session.

Its `RealtimeErrorCategory` is one of `authentication`, `permission`, `configuration`, `capability`,
`network`, `timeout`, `rate-limit`, `provider`, `protocol`, `tool`, `confirmation`, `abort`, or
`unknown`. `RealtimeErrorOptions` constructs one, and `toRealtimeError()` wraps any error, inferring
its category.

## Limitations

- OpenAI and Gemini Live are the bundled providers. Another provider implements
  `RealtimeTransportProvider`.
- Gemini Live and SIP calls are verified in the test suite against scripted versions of their
  protocols. The weekly live workflow also runs a Gemini Live session that calls a tool, and OpenAI's
  SIP call control, where the repository's secrets are set; a live call over a SIP trunk is not run.
- Gemini Live changes its setup between turns alone, and only with `sessionResumption`: an update
  waits for the turn to end. Neither `clearAudio()` nor truncating unheard audio has a counterpart;
  Gemini stops its own response when the person speaks.
- On a provider without video input, each frame becomes an image in the conversation, and so part of
  the model's context: space frames out further there.
- Video frames are not kept in the record; an image kept by content makes the record as large as it.
- A resumed conversation gives the provider its transcripts as text, not its audio, so the model
  hears what was said as written, not how it was said.
- A turn is charged from `estimateCost`: a session without one is admitted and rate-limited turn by
  turn, but charges nothing to the budget.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/telephony.md

# Telephony


Phone calls and text messages, from `nexus-ai-pro/telephony`: place and answer calls, transfer
them, put them in conferences, record them and keep the recordings under a retention policy, send
and receive messages, manage phone numbers, and handle media streams. Two providers ship, each on
its own protocol: Twilio, on `nexus-ai-pro/telephony/twilio`, answered with markup, and Telnyx, on
`nexus-ai-pro/telephony/telnyx`, controlled by command. To put a voice agent on a call,
`nexus-ai-pro/telephony/realtime-bridge` connects the call to a realtime session, on either.
A SIP trunk can also send calls to OpenAI directly, with no media stream to relay: see
[Phone calls over SIP](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/realtime.md#phone-calls-over-sip).

## Phone Number Telephony

Phone-number support is optional and separate from the voice layer. Use `voice` for audio transcription/TTS, and use `telephony` when calls, webhooks, TwiML, or media-stream events are involved.

```ts
import { TelephonyManager } from 'nexus-ai-pro/telephony';
import { TwilioTelephonyProvider } from 'nexus-ai-pro/telephony/twilio';
```

Configure only the provider you need:

```ts
const ai = new NexusAI({
  providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } },
  telephony: {
    defaultProvider: 'twilio',
    providers: {
      twilio: new TwilioTelephonyProvider({
        accountSid: process.env.TWILIO_ACCOUNT_SID!,
        authToken: process.env.TWILIO_AUTH_TOKEN!,
      }),
    },
  },
});
```

Generate a Twilio webhook response for an inbound phone number:

```ts
const response = await ai.createTelephonyResponse({
  say: 'Thanks for calling. Connecting you now.',
  stream: {
    url: 'wss://voice.example.com/twilio',
    mode: 'bidirectional',
    parameters: { tenant: 'acme' },
  },
});

return new Response(response.body, {
  headers: { 'content-type': response.contentType },
});
```

Start an outbound call with either a webhook, inline TwiML, an application SID, or a media-stream URL:

```ts
const call = await ai.createCall({
  to: '+15551230000',
  from: '+15557650000',
  mediaStreamUrl: 'wss://voice.example.com/twilio',
});
```

For smaller apps, skip Twilio entirely and register your own `TelephonyProvider`.

### Phone Agents: Bridging a Call to a Realtime Session

A phone agent needs the telephony media stream and a realtime session joined together. The bridge owns
that audio path — caller audio in, assistant audio out, barge-in, playback marks, and stream lifecycle —
while session configuration, tools, and authorization stay in application code.

```ts
import { createTelephonyRealtimeBridge, twilioRealtimeAudioOptions } from 'nexus-ai-pro/telephony/realtime-bridge';
import { OpenAIWebSocketTransport } from 'nexus-ai-pro/realtime/openai-websocket';
import { createRealtimeSession } from 'nexus-ai-pro/realtime/session';

// One socket per call, from your WebSocket server.
function handleCallSocket(socket: AppWebSocket) {
  const session = createRealtimeSession({
    provider: 'openai',
    model: 'gpt-realtime',
    transport: new OpenAIWebSocketTransport({ apiKey: process.env.OPENAI_API_KEY!, webSocketFactory }),
    modalities: ['audio'],
    instructions: 'You are a scheduling assistant.',
    tools: [checkAvailability, createBooking],
    // Telephony audio is 8 kHz G.711 mu-law; this passes it through untranscoded.
    audio: twilioRealtimeAudioOptions({ output: { voice: 'alloy' } }),
  });

  const bridge = createTelephonyRealtimeBridge({
    session,
    telephony,
    send: (message) => socket.send(message.body),
    onStart: (event) => logger.info(`call ${event.callId} started for ${event.parameters?.tenant}`),
    onStop: () => persistTranscript(session.export('json')),
    onError: (error) => logger.error(error),
  });

  socket.on('message', (raw) => bridge.handleMessage(String(raw)));
  socket.on('close', () => bridge.close());
}
```

The bridge feeds only inbound caller audio to the model, so an echoed outbound track never loops back.
On caller speech it clears audio already queued at the provider and cancels the in-flight response, which
is what stops the assistant talking over an interruption. Custom `<Parameter>` values from the stream
instruction are exposed as `bridge.parameters` for tenant and worker routing.

Pass `bargeIn: false` to keep queued audio playing, `autoConnect: false` to connect the session yourself,
and `await bridge.flush()` when you need outstanding sends to settle.

### Call Control and Usage Metering

Meter billable time from the provider's own record rather than from stream lifecycle events, which do not
account for ring time or provider-side teardown:

```ts
// From your status webhook.
const status = ai.parseTelephonyStatusCallback('twilio', requestBody);
if (status?.status === 'completed') {
  await usage.recordCallMinutes({ callId: status.callId, seconds: status.durationSeconds ?? 0 });
}

// Or read it directly, and hang up from a tool.
const details = await ai.getCall({ callId });
await ai.endCall({ callId });
```

### One answer on either provider

Twilio answers a call webhook with TwiML; Telnyx answers it with commands sent to the call, and the
webhook returns nothing. `ai.telephony.answer()` takes an `AnswerCallRequest`, the call's id and a
`TelephonyResponseRequest`, and does the right thing on each: it renders the markup on Twilio, and
runs the commands on Telnyx. Return what it resolves to from the webhook, whichever provider called:

```ts
const reply = await ai.telephony.answer({
  provider,                       // 'twilio' or 'telnyx'
  callId,
  say: 'Connecting you to the assistant.',
  stream: { url: 'wss://voice.example.com/media', mode: 'bidirectional' },
});
return new Response(reply.body, { headers: { 'content-type': reply.contentType } });
```

`ai.telephony` is the client's `TelephonyManager`, for everything the client's own methods leave out:
answering, transfer, conferences, recordings, and messages.

### Transfer and conferences

`transferCall()` takes a `TransferCallRequest` and moves a call to a number or a `sip:` address: the
caller leaves your application for whoever answers. It resolves to a `TelephonyTransfer`.
`joinConference()` takes a `JoinConferenceRequest`, a conference's name and how the call joins it,
as a `TelephonyConferenceConfig`: whether the conference starts when it joins and ends when it
leaves, whether it is muted, and whether a tone plays. Every call that joins the same name meets in
the same conference, created by the first; it resolves to a `TelephonyConferenceJoin`. On Twilio both
replace the live call's TwiML with a `<Dial>`; on Telnyx both are commands.

A webhook answer can dial as well: `dial`, a `TelephonyDialConfig`, connects the caller to a number,
a SIP address, or a conference, with the number shown and how long it rings.

```ts
// A tool the agent calls when the caller asks for a person.
await ai.telephony.transferCall({ provider, callId, to: '+15551230000' });
// Or bring a supervisor into the call.
await ai.telephony.joinConference({ provider, callId, name: `call-${callId}`, endOnExit: true });
```

### Recordings, kept on your terms

`startRecording()` takes a `StartRecordingRequest` (the call, one channel or one per side, and where
Twilio reports the recording) and resolves to a `TelephonyRecordingStarted`; `stopRecording()` takes
a `StopRecordingRequest`. The provider then reports the finished recording to your webhook:
`parseRecordingCallback()` turns that into a `TelephonyRecording`, with where its audio is and how
long it runs. `downloadRecording()` takes a `DownloadRecordingRequest` and fetches its audio as a
`TelephonyRecordingAudio`, and `deleteRecording()` a `DeleteRecordingRequest`.

`recordingKeeper()` does all of that under a retention policy. Given the telephony client, an
`AssetStore` (in memory, on disk, or an S3 bucket), and a `RecordingRetention` in days or seconds, the
`RecordingKeeper` it returns keeps each recording a callback reports: it fetches the audio, stores it
to expire when the policy says, and deletes the provider's copy, so the recording is kept in one
place, for as long as you decided. `sweep()` removes what has expired. A callback the provider retries
keeps its recording once; a recording that fails to keep is kept on the retry.

```ts
import { recordingKeeper } from 'nexus-ai-pro/telephony';

const recordings = recordingKeeper({
  telephony: ai.telephony,
  store: new S3AssetStore({ client, bucket: 'call-recordings' }),
  retention: { days: 30 },
  tenantId: (recording) => tenantOfCall(recording.callId),
});
// The recording webhook:
const kept = await recordings.handle(provider, await request.text());
// Every night:
await recordings.sweep();
```

Each `KeptRecording` names its asset, its tenant, when it expires, and whether the provider's copy
was deleted. `RecordingKeeperOptions` also takes `deleteFromProvider: false`, and `claim` and
`release` backed by a shared store, for replicas that take the same callback.

### Messages

`sendMessage()` sends an SMS, or an MMS with `mediaUrls`, from a `SendMessageRequest`, and resolves to
a `TelephonyMessage`: its id, its numbers, its text and media, how many segments it was billed as,
and its `TelephonyMessageStatus`. A message is its own operation family in the client's lifecycle,
`messaging`, so a budget or a rate limit holds messages apart from calls; one with neither text nor
media is refused before it counts. `parseMessageWebhook()` turns a provider's messaging webhook into
a `TelephonyMessage`: a message `received` by one of your numbers, or a change in one sent's delivery,
from `queued` through `sent` to `delivered`, `undelivered`, or `failed`.

```ts
await ai.telephony.sendMessage({ provider, to: caller, from: ourNumber, body: 'Your table is booked for 7 pm.' });
const message = ai.telephony.parseMessageWebhook(provider, await request.text());
if (message?.direction === 'inbound' && /^stop$/i.test(message.body ?? '')) await unsubscribe(message.from);
```

### Pointing a Number at Your Webhook

```ts
const [number] = await ai.listPhoneNumbers({ phoneNumber: '+15557650000' });
await ai.updatePhoneNumber({
  id: number.id,
  voiceUrl: 'https://api.example.com/voice/incoming',
  statusCallbackUrl: 'https://api.example.com/voice/status',
});
```

## Validating webhooks

A provider signs every webhook request. Skip the check and anyone can drive your call flow.

```ts
const valid = await ai.validateTelephonyWebhook({
  url: 'https://api.example.com/voice/incoming',
  headers: Object.fromEntries(request.headers),
  body: await request.text(),
});
if (!valid) return new Response('Forbidden', { status: 403 });
```

`ai.validateTelephonyWebhook()` takes a `TelephonyWebhookValidationRequest`:

| Field | What to pass |
| --- | --- |
| `url` | The full public URL the provider called, exactly as configured, with its query string. |
| method, `headers` | The request's, including the signature header. |
| `body` | The body, raw or parsed. |
| `params` | Query parameters, when the body was not what was signed. |
| `authToken` | A token to use instead of the configured one. |
| `publicKey` | For Telnyx, a public key to verify with instead of the configured one. |

Twilio signs with an HMAC of the URL and parameters, under the auth token. Telnyx signs the timestamp
and the raw body with Ed25519, under your account's key pair: the provider verifies it against the
public key with Web Crypto, and refuses a webhook more than five minutes old. Pass Telnyx's body
exactly as it arrived.

## Webhook responses

`TelephonyResponseRequest` describes how to answer a call webhook. It is rendered into the provider's
markup. Combine any of these:

| Field | What the caller gets |
| --- | --- |
| `say` | Spoken text. |
| `playUrl` | Audio played from a URL. |
| `gather` | A `TelephonyGatherConfig` that collects speech or keypad digits: what to listen for, where to post it, timeouts, language, the key that ends input, how many digits, and prompts spoken while listening. |
| `stream` | A `TelephonyStreamConfig` that opens a media stream, below. |
| `dial` | A `TelephonyDialConfig` that connects the caller to a number, a SIP address, or a conference. |
| redirect, pause, hangup | Call flow. |

A `TelephonyStreamConfig` has:

- the WebSocket `url`;
- a `TelephonyStreamMode`: `unidirectional` sends the caller's audio to you, and `bidirectional` also
  carries your audio back;
- a `TelephonyStreamTrack`: `inbound`, `outbound`, or `both`;
- a name, and a status callback with its `TelephonyHttpMethod`;
- custom `parameters`, delivered in the stream's start event.

The answer comes back as a `TelephonyWebhookResponse`: the provider, the content type, and the body to
return. `createVoiceTwiML()` renders TwiML directly, without a client, escaping every value.

What a gather collected arrives in a webhook: Twilio posts it to the gather's `actionUrl`, and Telnyx
sends it to the call's webhook. `parseGatherResult()` reads either as a `TelephonyGatherResult`: the
call, what the caller said with how sure the transcription is, the keys they pressed, and whether it is
`final`. On Telnyx a speech gather transcribes the caller's side of the call: each phrase arrives as
the caller speaks it, final once they finish it, until `endGather()` stops listening, with an
`EndGatherRequest` naming the call. A Twilio gather ends by itself, so there `endGather()` has nothing
to stop.

```ts
await telephony.answer({ provider: 'telnyx', callId, gather: { input: ['speech'], prompts: ['How can I help?'] } });
// The call's webhook:
const heard = telephony.parseGatherResult('telnyx', body);
if (heard?.final && heard.speech) {
  await telephony.endGather({ provider: 'telnyx', callId: heard.callId });
  await respondTo(heard.speech);
}
```

## Calls

`CreateCallRequest` places a call. Give `to` and `from` in E.164 form, and one way to handle the call:

| Field | Handles the call with |
| --- | --- |
| `webhookUrl` | Your webhook, asked what to do. |
| `twiml` | Inline TwiML. |
| `applicationSid` | A Twilio application. |
| `mediaStreamUrl` | A media stream to your WebSocket. |

Optional fields add a status callback, recording, a ring timeout, answering-machine detection, and a
signal. `CreateCallResponse` returns the call id and its first status.

A `TelephonyCallStatus` is usually `queued` first, then `ringing`, `in-progress`, and `completed`. A
call can also end `busy`, `failed`, `no-answer`, or `canceled`.

`GetCallRequest` and `EndCallRequest` name a call by id. Ending takes `completed` to hang up, or
`canceled` to drop a call that has not been answered. Both return `TelephonyCallDetails`:

- the status, and the `TelephonyCallDirection` (`inbound` or `outbound`);
- the numbers and times;
- `durationSeconds` and the price, once the call has ended.

`ai.parseTelephonyStatusCallback()` turns a status webhook into a `TelephonyStatusCallback`, with the
same fields and why the call ended.

## Phone numbers

`ListPhoneNumbersRequest` filters to one number and sets a page size. Each `TelephonyPhoneNumber` has:

- the provider-side `id`, used to update it;
- the number and its label;
- where it sends inbound calls, status events, and SMS;
- whether it can do voice, SMS, and MMS.

`UpdatePhoneNumberRequest` changes any of those. That is how you point a number at your agent.

## Media streams

A media stream carries `TelephonyMediaStreamEvent` values, told apart by `event`.
`ai.parseTelephonyMediaEvent()` turns each provider's messages into them:

| Event | Meaning |
| --- | --- |
| `TelephonyConnectedEvent` | The socket opened. |
| `TelephonyStartEvent` | Arrives once, before any audio: the stream and call ids, the tracks, the media format and its `TelephonyAudioEncoding`, and your custom parameters. Twilio streams 8 kHz mu-law. |
| `TelephonyMediaEvent` | A chunk of base64 audio, with its track, sequence number, and timestamp. |
| `TelephonyDtmfEvent` | The caller pressed a key. |
| `TelephonyMarkEvent` | Audio you sent has finished playing, so you know what the caller heard. |
| `TelephonyStopEvent` | The stream ended. |

`ai.formatTelephonyAudioMessage()` builds what you send back, as a `TelephonyOutboundAudioMessage`:
audio, a named mark, or `clear` to drop audio already queued. Its `body` is ready to send on the socket.

## Providers

A `TelephonyProvider` has an `info` and optional methods: create and end calls, render webhook
responses, validate signatures, parse and format stream messages, read calls and status callbacks, and
list and update numbers. `TelephonyProviderInfo.supports` says which methods exist.

`TelephonyConfig` is the client's `telephony` option: a default provider, and providers by name.
`TelephonyManager` routes each request to the named or default provider. It also works without a
client.

The calls that reach the provider's API — creating, reading, and ending calls, and listing and
updating numbers — run through the client's [lifecycle](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/lifecycle.md). Each is authorized,
audited, and counted, and each provider method receives the call's `ProviderCallContext` as its
second argument. Rendering a webhook response and parsing a message are local, so they do not.

`TwilioTelephonyProvider` implements all of it. `TwilioTelephonyProviderConfig` takes the account SID
and auth token (the token also validates webhooks), a REST base URL, and a `fetch`.

`TelnyxTelephonyProvider` implements all of it too, on Telnyx's Call Control API. Its
`TelnyxTelephonyProviderConfig` takes the API key, the Call Control application calls are placed
through, the messaging profile messages are sent through, the public key webhooks are verified with,
the voice and language it speaks in and a speech gather listens for, the `transcriptionEngine` that
transcribes one (Telnyx's own default unless named), a base URL, a `fetch`, and a clock. It runs on any runtime with
`fetch` and Web Crypto. `TelephonyProviderInfo.supports` says, for each provider, whether it
transfers, conferences, records, and messages.

Two errors cover failures. `TelephonyProviderError` means a provider failed, or none was configured
for the operation; it names the provider and carries the cause. `TelephonyCapabilityError` means the
provider lacks the operation.

## The realtime bridge in detail

`TelephonyRealtimeBridgeOptions`:

| Option | Default | What it does |
| --- | --- | --- |
| `session` | — | The realtime session. |
| `telephony` | — | The manager that parses and formats stream messages. |
| provider | `twilio` | Which provider's messages these are. |
| `send` | — | Sends a message on the socket. Awaited, so messages keep their order. |
| `autoConnect` | on | Connects the session on the start event. |
| `bargeIn` | on | Cancels the assistant's speech when the caller talks. |
| `autoDisconnect` | on | Ends the session when the stream stops. |
| `onStart`, `onStop`, `onDtmf`, `onError` | — | Hooks. Errors are reported here, never thrown at the socket. |

The returned `TelephonyRealtimeBridge` exposes the `callId`, `streamId`, and `parameters` once the
stream starts, whether it has `closed`, and `handleMessage()`, `flush()`, and `close()`.

`twilioRealtimeAudioOptions()` sets both audio directions to 8 kHz mu-law, so audio passes through
without transcoding. Telnyx streams the same audio, so `telnyxRealtimeAudioOptions()` is the same
setting, under its name; pass `provider: 'telnyx'` to the bridge and it reads Telnyx's messages.

## Proof

In the test suite, one phone agent runs on both providers against stand-ins for their APIs. A call
comes in and is answered the same way on each, recorded, and streamed through the bridge to a
realtime session: the caller's audio reaches the model, the model's reaches the caller, and the agent
texts a receipt, as a `messaging` operation, and transfers the call. Its recording is then kept in
memory, on disk, and in an S3 bucket, deleted at the provider, and kept once when the callback comes
again; the day before its 30 days end it is there, and the day after a sweep removes it.

## Limitations

- Twilio and Telnyx are the bundled providers; another implements `TelephonyProvider`.
- On Telnyx, a speech gather listens until `endGather()`: its `timeoutSeconds` and `speechTimeout` are
  for keys alone, and the application decides when the caller has said enough. There is no redirect
  to another webhook. A stream carries no custom parameters, so they ride in its URL, which your socket
  server reads.
- On Telnyx, a number's calls go to its Call Control application and its messages to its messaging
  profile, so updating a number's `voiceUrl` or `smsUrl` changes them for every number that shares
  them.
- Two calls joining a Telnyx conference that does not exist yet at the same moment may make two.
- The recording keeper remembers what it kept in its process by default; replicas that take the same
  callback need a shared `claim`.
- Calls and messages are verified against stand-ins for the providers' APIs. The weekly live workflow
  lists each provider's phone numbers, where the repository's secrets are set; it places no call and
  sends no message, which would cost money.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/testing.md

# Testing with recorded traffic


Tests that call a provider need credentials. With recordings, they run everywhere else too.

- `nexus-ai-pro/testing/record` records provider traffic once, redacted, as fixture files you can
  review. It replays them with no network.
- Conformance suites check an adapter against the neutral contract. They run on the recordings in
  every pull request, and live where credentials are.

```ts
import { fixtureFetch } from 'nexus-ai-pro/testing/record';

// Replays by default. NEXUS_FIXTURES=record captures, NEXUS_FIXTURES=live bypasses.
const fetch = fixtureFetch({ directory: 'tests/fixtures/openai' });
const provider = new OpenAIImageProvider({ apiKey: process.env.OPENAI_API_KEY ?? 'replay', fetch });
```

## Recording and replaying

Recording writes one JSON file per exchange. Credential headers and query parameters are removed
before anything is written. A `redact` hook handles the rest, such as personal data in a prompt, with
a PII detector from `nexus-ai-pro/security`.

Replay serves the files back with no network. It matches requests by method, URL, and body, with
object keys sorted and multipart boundaries ignored. A request with no recording throws
`FixtureMissingError`, instead of reaching the live API.

```bash
npm run test:conformance:images       # replays image recordings; needs no credentials
npm run conformance:images:record     # records them; needs OPENAI_API_KEY, GOOGLE_API_KEY, or COMFYUI_URL
```

`fixtureFetch()` picks a `FixtureMode` of `record`, `replay`, or `live`. It reads its `mode` option,
then `NEXUS_FIXTURES`, and defaults to `replay`. So one test file serves all three. The pieces it
chooses between are exported too:

| Function | What it does |
| --- | --- |
| `recordingFetch()` | Wraps a real fetch and writes every exchange to the directory. |
| `replayFetch()` | Serves the recorded files back. |
| `readFixtures()` | Reads every `RecordedExchange` in a directory, to inspect or replay yourself. |
| `installFetch()` | Puts a fetch in place of the global one, for code that calls it directly. Call the function it returns to put the original back. |

`recordingFetch()` returns a `RecordingFetch`: a fetch with a `flush()`. Await `flush()` before a test
ends; it resolves once every fixture is on disk. `RecordOptions` add the real `fetch`, the `redact`
hook, a clock, and `timing`.

**Replaying a stream at its pace.** With `timing: true`, a recording keeps each chunk of the response
and when it arrived, as a `RecordedChunk` in the response's `chunks`. `ReplayOptions.pace`, a
`ReplayPace`, then decides how a replay sends them:

| Pace | Sends |
| --- | --- |
| `instant` (the default) | The whole body at once, as without timing |
| `original` | Each chunk when it arrived, relative to the first |
| A number | The gaps scaled: `0.1` is ten times faster |
| A function | Each chunk once the function resolves, so a test releases them one at a time |

A pacing function is told the chunk's position, how many there are, when it arrived, and the gap
before it, as `ReplayChunkInfo`. Paced replay is what tests idle timeouts, cancellation mid-stream,
backpressure, and a streaming UI against a real provider's rhythm.

```ts
const fetch = replayFetch({ directory, pace: 'original' });
```

`ReplayOptions` add `onMissing`. The default, `throw`, raises `FixtureMissingError` with the request's
method, URL, and match key, so you can find the recording that should have matched. `live` falls
through to the real `fetch` instead.

### Matching options

`FixtureOptions` are shared by recording and replay:

| Option | What it does |
| --- | --- |
| `directory` | Where the fixture files are. |
| `match` | Turns a request into its match key. |
| `ignoreBodyFields` | Top-level body fields that change between runs, such as a request id or a timestamp. |
| `redactHeaders`, `redactQuery` | More names to remove, beyond the credential names always removed. |

A custom `match` function receives a `MatchInput`: the method, the URL with credentials removed and the
query sorted, the redacted headers, and the body as text with multipart boundaries normalized.

### What a fixture holds

Each fixture file is one `RecordedExchange`:

- the match key, and its position among requests that share it. A request made three times replays
  three responses, in order.
- the `RecordedRequest` and the `RecordedResponse`;
- when it was recorded.

Bodies are text, or base64 for binary. Transport headers that `fetch` already applied, such as content
encoding, are left out, so replay never describes a body it no longer has.

## Conformance suites

Each suite checks an adapter against the neutral contract and returns one result per case. A test
asserts on the results; a report prints them. The suites are exported from the root.

### Chat providers

```ts
import { runProviderConformance } from 'nexus-ai-pro/testing';

const results = await runProviderConformance('openai', provider, { testStream: true, testTools: true });
assert.ok(results.every((result) => result.completeOk && result.streamOk !== false), JSON.stringify(results));
```

`runProviderConformance()` takes a provider name, a provider, and `ProviderConformanceOptions`:

| Option | What it does |
| --- | --- |
| `model` | Runs every case on one model. |
| `fixtures` | Replaces the default cases. |
| `testTools` | Adds a tool-calling case. |
| `testStream` | Also streams each case. |
| `testHealth` | Also runs the health check. |

Each `ProviderConformanceCase` is a request and a `validate` function. The defaults are a plain text
completion and a JSON response shape. `PROVIDER_CONFORMANCE_FIXTURES` holds them for each bundled
provider, with a model that provider serves. Each `ProviderConformanceResult` says whether completion,
streaming, and health passed, and what went wrong.

### Embedding providers

`runEmbeddingProviderConformance()` checks that a provider returns one vector per input, in order, all
of one width, with finite components, and that it honors a requested `dimensions`.

- `EMBEDDING_PROVIDER_CONFORMANCE_FIXTURES` has a single input, a batch, and a query-typed input.
- An `EmbeddingProviderConformanceCase` adds an input type, dimensions, and an extra `validate`.
- `EmbeddingProviderConformanceOptions` picks the model and the cases. `testAbort`, on by default,
  checks that an already-aborted signal stops the call before the network. `testDeterminism` embeds
  one input twice and compares.
- Each `EmbeddingProviderConformanceResult` has `ok`, `abortOk`, and the error.

### Image providers

`runImageProviderConformance()` runs `ImageProviderConformanceCase` values. Each is an
`ImageGenerateProviderConformanceCase` or an `ImageEditProviderConformanceCase`: a request and an
optional `validate`.

- `IMAGE_PROVIDER_CONFORMANCE_FIXTURES` is a generation, an edit, and a masked edit. Edits run only when
  the provider declares them, and the masked edit only when it declares mask support.
- `ImageProviderConformanceOptions` sets the model, the cases, and the image and mask the edits start
  from, a portable 1×1 PNG by default. `testAbort`, `testEdit`, and `testMask` turn checks on or off.
- Each `ImageProviderConformanceResult` names the operation, whether it met the contract, and every
  violation found. The contract covers assets, metadata, and the operation and request ids echoed back.

## Limitations

- Replay matches requests exactly, after normalization. A change to a prompt or a request field needs
  a new recording. That is the point: the fixture shows what changed.
- Without `timing`, a streamed response is recorded whole and replayed in one chunk. Recording with
  timing still reads the whole response before handing it to the caller.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/adapter-kit.md

# Adapter kit


How a package outside this repository builds an adapter and proves it works. An adapter can be:
- a chat provider;
- an embeddings backend;
- a vector store;
- a retriever;
- a long-term store.

Each `define*()` function takes four things:
- what the adapter is;
- the releases of `nexus-ai-pro` it supports;
- what it can do;
- how to create it.

It returns an `Adapter`. Every instance the adapter creates normalizes its failures into one
vocabulary and reports each call for telemetry. `verify()` runs the same contract suite this package
runs on its own adapters, plus a small benchmark held to the budgets the adapter declares.

```ts
import { defineVectorStoreAdapter } from 'nexus-ai-pro/adapter-kit';

export const milvus = defineVectorStoreAdapter({
  name: '@acme/nexus-milvus',
  version: '0.3.0',
  nexus: '>=3.0.0 <4',
  capabilities: { filters: 'typed', minScore: true },
  create: (config: MilvusConfig) => new MilvusVectorStore(config),
  normalizeError: (error) => ((error as { code?: string }).code === 'QUOTA' ? { code: 'RATE_LIMIT', retryable: true } : undefined),
  budgets: { search: 150 },
});

// In the adapter's own test suite, against a real or a containerized server:
const report = await milvus.verify(config, { embed });
if (!report.passed) throw new Error(JSON.stringify(report.checks.filter((check) => !check.ok)));
```

## Defining an adapter

| Kind | Builder | Contract | `verify()` options |
| --- | --- | --- | --- |
| A chat provider, extending `BaseProvider` | `defineProviderAdapter()` | Completion, streaming, JSON output, tool calls, health | `ProviderVerifyOptions`: the model and any cases of your own |
| An embeddings backend | `defineEmbeddingsAdapter()` | One vector per input, in order, of one width; abort; determinism | `EmbeddingsVerifyOptions` |
| A `VectorStore` | `defineVectorStoreAdapter()` | `runVectorStoreContract()` | `VectorStoreVerifyOptions`: the embedding function, and a `settle` for an eventually consistent index |
| A `Retriever` or `SparseRetriever` | `defineRetrieverAdapter()` | `runRetrieverContract()` | `RetrieverVerifyOptions`: a `load` for a retriever that does not hold its own passages |
| A long-term `Store` | `defineStoreAdapter()` | `runStoreContract()` | none |

An `AdapterDefinition` names the adapter, its `version`, and `nexus`, the range of releases it
supports. That range reads as npm reads one: `>=2.4.0 <3`, `^2.4.0`, `~2.4.1`, `2.x`, and
alternatives joined by `||`. `satisfiesRange()` and `parseVersion()` are the same check, on their
own. `compatible()` asks it of any release, the installed one by default. `verify()` fails an adapter
whose range leaves out the release it runs against.

`capabilities` declares what the adapter can do, per kind:
- `ProviderAdapterCapabilities`: streaming, JSON output, tools, and a health check;
- `EmbeddingsAdapterCapabilities`: abort and determinism;
- `VectorStoreCapabilities`: `searchVector`, `minScore`, and filters (`none`, `exact`, or `typed`,
  which also tells `3` from `'3'`);
- `RetrieverCapabilities`: whether it takes writes, and filters;
- `StoreCapabilities`: search, filters, and namespace listing.

The contract skips a check the adapter declares it cannot pass, and reports it as skipped. A check it
claims and fails is a failure. `budgets` gives the p95 latency each operation must stay within in the
benchmark.

## What an instance guarantees

`create()` wraps the instance. Each operation of its kind is timed and reported to
`AdapterCreateOptions.onCall`: `search` and `add` for a vector store, `complete` for a provider, and
so on. That hook is where a tracer or a metrics exporter subscribes, with an `AdapterCallEvent` for
every call: the adapter, its `AdapterKind`, the operation, the duration, and how it failed if it did.

Every failure comes out as an `AdapterError`, whose `code` is an `AdapterErrorCode` (`AUTH`,
`RATE_LIMIT`, `NOT_FOUND`, `INVALID`, `CONFLICT`, `TIMEOUT`, `UNAVAILABLE`, `CANCELLED`, or
`UNKNOWN`), with `retryable` and the HTTP `status` when there was one. So code above any adapter
retries and alerts the same way. `classifyAdapterError()` reads a status, a network code such as
`ECONNREFUSED`, or an `AbortError`. An adapter's own `normalizeError` runs first, returning an
`AdapterErrorClass`, for a service with codes of its own. Methods the kit does not instrument, and
private fields, work as they always did.

## Verifying

`verify()` returns an `AdapterReport`:
- whether the adapter is `compatible`;
- whether it `passed`;
- every `AdapterCheck`: its name, whether it held, why not, or that it was skipped;
- the `benchmark`, an `AdapterBenchmarkEntry` per operation with the calls measured and the p50 and
  p95 latency.

The contracts also run on their own, against any instance, so an adapter can put them inside its
own test framework:
- `runVectorStoreContract()` checks, with `VectorStoreContractOptions`, against an empty store:
  - ranking and `topK`;
  - sources and metadata returned whole;
  - filters and typed filters;
  - `minScore`;
  - vector search;
  - replacement by id, deletes, an id added again after its delete, and empty writes.
- `runRetrieverContract()` checks exact terms, ranking, `topK`, filters, and an empty query, plus
  replacement and deletes for a `SparseRetriever`. `RetrieverContractOptions` gives the
  capabilities, the `load` that fills what a retriever without `add()` reads from, and a `settle`.
- `runStoreContract()` checks:
  - values round-trip;
  - a replaced item keeps its creation time;
  - a missing key reads as nothing;
  - deletes;
  - prefix search with filters and paging;
  - namespace listing.

Every contract runs on `ADAPTER_FIXTURE_PASSAGES`, so results compare across adapters. An
`AdapterReport` also carries what the adapter declared, its `nexusRange`, `capabilities`, and
`budgets`, and the `contractVersion` it ran.

## Compatibility records

A compatibility record is what `verify()` proved, kept as a document: which contract version the
adapter passed, against which release, how fast, on what runtime, sealed with a digest. An adapter's
own CI makes one; one page lists them; and before a release ships, every listed adapter is verified
against it again, so a contract change that would break one is caught first.

Each kind's contract is versioned in `ADAPTER_CONTRACTS`: an `AdapterContract` with its current
`version` and its `history`, each `AdapterContractVersion` naming the release it arrived in and the
checks it `added`. The vector-store contract is at version 2, which 2.15 made by adding “adds a
passage again under an id it deleted”, a check a store that marks deletes and never clears the mark
fails. Contract checks are added in a minor release only when they test something the contract
already promised.

The command line makes records from the adapter's own repository, against the installed package:

```bash
# verify.mjs exports { adapter, config, options, record, close }: the adapter, what its verify() takes,
# record options such as its repository, and anything to close after.
npx nexus adapters verify ./verify.mjs --out records/milvus.json --package @acme/nexus-milvus@0.3.0
npx nexus adapters page records --out COMPATIBILITY.md --check
npx nexus adapters recheck records
```

- `nexus adapters verify` loads the module, runs the adapter's `record()`, prints each check, writes
  the record, and exits 1 when the adapter does not pass. `config` and `options` may be functions,
  for a server started first. The record names the module as `verify`, relative to the record, or as
  `--verify` says, such as a package export `@acme/nexus-milvus/verify`, so it can be run again.
- `nexus adapters page` lists records, files or directories of them, as Markdown, or HTML with
  `--html`. `--check` exits 1 on a record that is failing or outside the installed release's range,
  `--strict` on a stale one too, and `--expect` on a page that does not list the records as they are.
- `nexus adapters recheck` verifies each listed adapter again against the installed release, prints
  what it finds, and exits 1 when an adapter that passed its record no longer does, naming each check
  that broke and each new check it fails. `--out` writes the new records; `--update` writes them over
  the listed ones.

In code, `adapter.record()` verifies and seals in one step, and `compatibilityRecord()` seals an
`AdapterReport` with `CompatibilityRecordOptions`: the `package`, the `repository`, the `verify`
module, a `runtime`, and a clock. A `CompatibilityRecord` of format `COMPATIBILITY_RECORD_FORMAT`
(`nexus-compatibility/1`) holds its `CompatibilityRecordAdapter` (name, version, kind, range,
capabilities, budgets, and where it lives), the contract, the release, when, where, its checks, its
benchmark, and its `digest`, which `recordDigest()` computes over everything else.
`readCompatibilityRecord()` reads one back and throws a `CompatibilityRecordError` listing its
`issues`: an unknown format, a missing field, or a digest that does not match an edit.

`recordStatus()` returns a record's `CompatibilityStatus` against a release: its
`CompatibilityState`, `passing`, `failing`, `incompatible` (the release is outside the declared
range), or `stale` (verified against an older contract version), the reasons, and the checks it has
not been verified for. `compareRecords()` returns a `CompatibilityComparison` of a listed record and a
new one: each `CheckChange` that broke, the checks added since, those fixed, the operations whose p95
more than doubled, and whether the new record `regressed`. `compatibilityPage()` writes the page, by
kind and name, with `CompatibilityPageOptions`: a `title`, an `intro`, the release to judge by, and
`markdown` or `html`. The same records make the same page.

This repository's own page, [COMPATIBILITY.md](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/COMPATIBILITY.md), lists records of the adapters it
ships, made the same way from `compatibility/adapters`, and every build verifies each one again.

## Records from an adapter's own CI

An adapter of your own reaches the compatibility page from its own CI, without a pull request here.
Its package carries the keyword `ADAPTER_KEYWORD`, `nexus-ai-pro-adapter`, and names where its CI
publishes its record:

```json
{
  "name": "@acme/nexus-milvus",
  "keywords": ["nexus-ai-pro-adapter"],
  "nexus": { "compatibility": "https://raw.githubusercontent.com/acme/nexus-milvus/compatibility/record.json" }
}
```

Its CI verifies the adapter, on every change and on a schedule against the latest release, and
publishes the record wherever that URL serves from, here a branch of its own:

```yaml
- run: npm install nexus-ai-pro@latest
- run: npx nexus adapters verify ./verify.mjs --out record.json --package "@acme/nexus-milvus@$(node -p "require('./package.json').version")"
  continue-on-error: true # a record that does not pass is published too, so the page shows it red
- run: |
    git switch --orphan compatibility && git add record.json && git commit -m "Compatibility record"
    git push --force origin compatibility
```

Every day, this repository's `Community compatibility` workflow runs `nexus adapters gather`: it finds
every package with the keyword on npm, reads the URL each names, fetches its record, and writes the
page, with the adapters this repository ships, to its `compatibility` branch. A record is listed only
when it reads as `nexus adapters verify` sealed it, its digest matching, and names the package that
points to it, from HTTPS; a record that says the adapter fails turns its row red. A source that is down,
or serves a record that is refused, keeps the last record read, saying why; a package that no longer
names a record, or is no longer found, leaves the page. Nothing an adapter publishes is run here: a
gathered record is listed as its own CI recorded it, and never verified again by this repository's
builds, which run only the code it ships. `compatibility/sources.json` lists, by hand, any adapter that
is not on npm.

```bash
npx nexus adapters gather --out community                            # find adapters on npm by the keyword
npx nexus adapters gather --out community --no-keyword --sources sources.json
npx nexus adapters page compatibility/records community --html --out index.html
```

`nexus adapters gather` writes one record a package into `--out`, keeps the last one for a package
whose record cannot be read now, removes a record no longer published, and prints what became of each.
It exits 1 when npm cannot be searched, rather than list every adapter as gone. In code,
`gatherRecords()` takes `GatherRecordsOptions`: the `keyword`, the `sources` listed by hand, each a
`RecordSource` of a package and its URL, the `registry`, the `previous` records to keep from, and the
limits on a record's size and a request's time. Its `GatherReport` holds the `records` to list and,
for each package, a `GatheredSource`: its URL, its `GatherOutcome` (`gathered`, `kept`, `refused`,
`unreachable`, or `undeclared`), why, and its record.

A record is written onto the page as text, never as markup: an adapter links to its repository only at
an `http` or `https` address, a cell stays on one line, and the HTML page colours each row by its
state, a failing adapter red.

## Proof

A community adapter in a directory of its own, a vector store that marks a deleted id as gone and
never clears the mark, listed with a record that passed contract v1 when 2.14 was current. Verified
again by `nexus adapters recheck` against this release, whose contract v2 adds a check it does not
meet, its record turns red, naming the check, and the command exits 1; fixed, it passes, and its
new record replaces the old. The page shows the old record as stale, and refuses one edited after it
was sealed. In the clean-install test, the installed command records a community adapter from the
consumer project, verifies it again, and lists it.

The same community adapter reaches the page from its own CI, against stand-ins for npm and for where
it publishes: a passing record, published, is gathered and listed passing, with nothing in this
repository changed; a release of it that breaks the contract is recorded by its CI as failing, and
the next gather turns its row red. A record edited after it was sealed is refused and the last one
kept, as it is while its source is down; while npm cannot be searched the gather fails and changes
nothing, and a package npm no longer lists leaves the page.

The clean-install test builds an adapter the way an outside package would. It is a vector store
written in plain JavaScript inside the consumer project, importing nothing but
`nexus-ai-pro/adapter-kit` from the installed tarball, and it passes the full contract with typed
filters and a latency budget. Each kind's contract also runs against this package's own references.
A deliberately broken store fails exactly the checks it breaks.

## Limitations

- The provider and embeddings suites send real requests, so run them against the service or a
  local stand-in, not in a unit test without one.
- Credentials belong in the configuration passed to `create()` and `verify()`, never in the
  definition, which an adapter package publishes.
- A gathered record is trusted to be what its package's own CI recorded: it is checked for its seal and
  its package, and served from the URL the package names, but its checks are not run again here.
- The community page is published to the `compatibility` branch daily, so a record its CI published
  reaches it within a day.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/studio.md

# The studio


A UI for what a nexus-ai-pro application records: traces, threads, approvals, experiments, prompts,
context bundles, issues found in traces, proposed fixes, costs, provider health, and the operation
queue. Local and token-only by default, or shared by a team with accounts and roles, signed in with
your identity provider, and across tenants. It is a separate package,
`nexus-ai-pro-studio`, so the core install never carries a UI. It runs wherever you start it — your
machine, or a server your team reaches — reads your application's own stores through the adapters
it already uses, and needs no hosted service.

## Starting it

```bash
npm install --save-dev nexus-ai-pro-studio
npx nexus-studio --config studio.config.mjs
```

The configuration module's default export is the studio's sources — or a function, possibly async,
that returns them. It is ordinary application code, so it opens the stores the application opens:

```js
// studio.config.mjs
import { PostgresTraceStore } from 'nexus-ai-pro/postgres/traces';
import { PostgresExperimentStore, PostgresDatasetStore } from 'nexus-ai-pro/postgres/evaluate';
import { pool, supportGraph, checkpointer, registry, ai, reviewQueue } from './src/app.js';

export default {
  traces: new PostgresTraceStore(pool),
  experiments: new PostgresExperimentStore(pool),
  datasets: new PostgresDatasetStore(pool),
  graphs: { support: { graph: supportGraph, checkpointer } },
  reviews: { answers: reviewQueue },
  prompts: registry,
  client: ai,
  budgets: [{ name: 'production', limit: 200, period: 'month' }],
};
```

The command prints an address such as `http://127.0.0.1:4747/?token=…`; open it. `--port` picks the
port (`0` for any free one), `--token` fixes the token instead of generating one, and `--host` with
`--allow-host` bind somewhere other than the loopback interface. A TypeScript config works under a
loader: `node --import tsx node_modules/nexus-ai-pro-studio/dist/cli.js --config studio.config.ts`.

To start it from code instead — inside a dev server, say — `startStudio()` takes the same sources and
returns a `RunningStudio` with the `url`, `token`, `port`, and `close()`. `StartStudioOptions` adds the
port and host to the `StudioOptions` every entry point takes. `createStudio()` returns the bare
`Studio` handler, a `Request`-to-`Response` function, for mounting it in a server you already run.

`npm run studio:demo` in the repository starts it on a demo application with something in every view.

## What it shows

Every source is optional. The studio shows a view for each one it is given, and hides the rest.

| View | Source | What you can do |
| --- | --- | --- |
| Traces | `traces` | Filter runs by status, kind, name, model, and time; open a run tree with inputs, outputs, cost, latency, a tool's progress, and each run's events and links; follow a link to the run it names; compare two traces; record feedback on a run |
| Threads | `graphs` | See each graph's diagram with the next nodes highlighted; list threads; read state at any step; fork from a step; edit state; answer an interrupt |
| Inbox | `graphs`, `reviews`, `proposals` | Every thread waiting for a human, every review item waiting for a grade, and every proposed fix waiting for promotion, in one list |
| Issues | `traces` | Failing and slow runs clustered by what went wrong, and what got worse than the week before |
| Experiments | `experiments`, `datasets` | List experiments and datasets; compare two experiments with per-metric intervals and a verdict |
| Prompts | `prompts`, `client` | Versions, labels, and history; diff two versions; promote through the registry's gates; roll back; render and run a version in the playground |
| Bundles | `contexts` | Context bundle versions, labels, and history; diff two versions; promote through the hub's gates; roll back |
| Deployments | `deployments`, `operations`, `tenants` | Each assistant's traffic split and history; each revision's runs, error rate, and latency since the last change; replicas and their health; the queue; tenant usage. Admins start a canary, promote, and roll back |
| Audit | accounts | Every change, and every refused attempt, with who and when — for admins |
| Costs | `traces`, `budgets` | Cost per day and per model, the most expensive runs, and each budget with what it has spent |
| Health | `client`, `circuits` | Provider health scores, circuit states — local and shared across workers — metrics, and cache statistics |
| Operations | `operations`, `assets` | The operation queue by status, with leases and errors; asset store totals |

## The sources

`StudioSources` is the whole configuration. Each field is structural, and the real objects satisfy
it as they are:

- `traces` is any `TraceStore`: memory, JSONL, or Postgres.
- `graphs` maps a name to a compiled graph, or to a `StudioGraphSource` of the graph and how to list
  its threads — its `checkpointer`, or a `threads()` function. `StudioGraphLike` is the part of a
  compiled graph the view uses; `StudioCheckpoint` and `StudioInterrupt` are what it reads.
  `MemoryGraphCheckpointer` and the operation-store checkpointer (Redis, Postgres, SQLite) both list their
  threads; a graph given alone can still open a thread by id.
- `reviews` maps a name to an annotation queue, through `StudioReviewQueue`.
- `datasets` and `experiments` are any `DatasetStore` and `ExperimentStore`.
- `prompts` is a `PromptRegistry`, as `StudioPromptRegistry`.
- `contexts` is a `ContextHub`, as `StudioContextHub`.
- `proposals` is a `ProposalInbox` from `nexus-ai-pro/insights`, as `StudioProposalInbox`: the fixes
  waiting for a person, which admins promote — through the label's gates — and editors reject.
- `client` is a `NexusAI` client, as `StudioClient`: its `complete()` runs the playground, and its
  health, circuit, metrics, and cache methods fill the health view. Each is optional.
- `circuits` is a shared circuit store, as `StudioCircuitStore`, for a deployment whose workers
  share circuit state.
- `operations` is an operation store that can list, as `StudioOperationStore`. Graph checkpoint
  records that live in the same store are left out of the queue. A store with `stats()` also gives
  the deployments view its queue numbers.
- `deployments` is a `Deployments` registry from `nexus-ai-pro/server/deployments`, as
  `StudioDeployments`. Build it on the agent server's state store, with no assistants: the studio
  reads and changes what the servers record, and every change is checked against the version shown.
- `tenants` is the limiter from `tenantLimits()`, as `StudioTenants`, whose `report()` fills the
  tenant table.
- `assets` is an asset store, as `StudioAssetStore`: its `snapshot()` totals, and a listing when it
  has one.
- `budgets` is a list of `StudioBudget` values: a name, a limit in US dollars, a period of a day, a
  week, or a month, and an optional trace filter for which runs count.

The costs view counts model runs, which is where the tracer records cost, so a parent run that sums
its children is not counted twice. `StudioCostReport` is the shape it returns.

`rollups`, a `StudioRollups`, makes the costs view read hourly totals instead of traces: give it the
rollup store behind `rollupTraceStore()` from `nexus-ai-pro/tracing/rollups`. Each `StudioRollupRow`
is an hour of one kind, model, provider, and tenant. A month of a million runs is a few thousand rows,
so the view answers in milliseconds however large the trace store grows. The costliest runs still come
from the traces, from the most recent 1,000 in the window, and a budget whose filter needs more than a
model, provider, kind, or name still reads the traces.

## Access and safety

The studio can change things — answer an interrupt, edit a thread, promote a prompt — so it is
locked down, even on its local default:

- It binds `127.0.0.1` by default, so nothing else on the network can reach it.
- Every request needs the token. It arrives once, in the URL the command prints; the studio moves it
  into an HTTP-only, same-site cookie and takes it out of the address bar. `createToken()` generates
  one, and `tokensMatch()` compares in constant time.
- A change must also carry the token in the `x-studio-token` header (`TOKEN_HEADER`), which a page on
  another site cannot set, so the cookie alone cannot be used against you. The page's own script reads
  the token from a same-origin session route; `SESSION_COOKIE` is the cookie's name.
- Requests whose `Host` is not a loopback name are refused, which closes DNS rebinding: a hostile
  page pointing its own domain at `127.0.0.1`. `hostAllowed()` is the check, and `allowedHosts` adds
  names when you bind elsewhere on purpose.
- Pages are served with a strict content security policy, and every value from a trace, a model, or
  a person is inserted as text, never as HTML. A link from a record is rendered only when it is
  `http` or `https`.
- A request body is at most `maxBodyBytes`, 1 MiB by default. A larger one is refused with `413` as
  it arrives, before anyone is signed in. A change sends its body as `application/json`, a JSON
  object, or it is refused with `415` or `400`.
- A list returns at most 1,000 items, the audit log 10,000 entries, the issues view 90 days, and the
  costs view 366 days. Asking for more is a `400`, so no request makes a store read everything.
- `Studio.routes` lists every `StudioRoute`: its method, path, the role it needs, and the action
  the audit log records for it.
- On the single token, actions are recorded under `actor` — `studio` by default — so a promotion's
  history says who moved the label.

Errors come back as JSON with a stable code, as a `StudioError`. A source that was not configured is a
`404`, a refused promotion a `409` with each gate's verdict, and a malformed request a `400`.

## Sharing it with a team

Give the studio `auth` and each person signs in with a role; everything they do is recorded under
their id. A `StudioAuthenticator` decides who a request is from — a `StudioUser` with an id, a name,
an email, and a `StudioRole` — given the request and a `StudioRequestInfo` with the address it came
from. `STUDIO_ROLES` lists the roles, each including the ones before it, and `hasRole()` compares:

| Role | May also |
| --- | --- |
| `viewer` | Read every view |
| `reviewer` | Answer interrupts, submit reviews, record feedback, and comment |
| `editor` | Edit and fork threads, run the playground, and reject proposals |
| `admin` | Promote and roll back prompts, bundles, and proposals, and read the audit log |

Four authenticators come bundled:

- `personalTokens()` gives each person their own link, for a small team without an identity provider.
  The CLI reads them from a file: `--users users.json`, a list of `{ "id", "name", "role", "token" }`,
  which `loadUsers()` reads.
- `headerAuth()` trusts the identity headers set by a reverse proxy that signs people in, such as an
  OIDC proxy or an access gateway. It trusts them only from the proxy: `HeaderAuthOptions` needs a
  secret header the proxy adds, or the proxy's addresses. It also maps groups to roles.
- `bearerAuth()` accepts an `Authorization: Bearer` token checked by the function you pass — an OIDC
  token verified against the issuer's keys — for scripts and services; `BearerAuthOptions` maps its
  claims to a role.
- `anyOf()` tries several in turn, such as bearer tokens for scripts and a proxy for people.

Or the studio signs people in itself, with no proxy in front: see [Signing people in](#signing-people-in).

```js
// studio.config.mjs
import { headerAuth, FileStudioJournal } from 'nexus-ai-pro-studio';

export default { traces, prompts: registry, contexts: hub, proposals: inbox };
const journal = new FileStudioJournal('/var/lib/nexus-studio');
export const options = {
  host: '0.0.0.0',
  allowedHosts: ['studio.internal.example.com'],
  auth: headerAuth({
    secret: { header: 'x-proxy-secret', value: process.env.PROXY_SECRET },
    role: ({ groups }) => (groups.includes('ml-admins') ? 'admin' : groups.includes('ml') ? 'editor' : 'viewer'),
  }),
  audit: journal,
  comments: journal,
};
```

With accounts, every change must carry the person's page token, or bring its own bearer token in an
`Authorization` header, as a script does. That stops another site from using a signed-in browser
against the studio. A browser adds Basic credentials to another site's request by itself, so they
are not proof, and a change refused for want of the page token is audited. `csrfToken()`
derives the page token from a server `secret` and the user, and the page reads it from its session.
Give several replicas the same `secret`.

People act as themselves: a request body cannot name someone else as the reviewer or promoter.

Every change, and every attempt a role refused, goes into the audit log as a `StudioAuditEntry`: who,
their role, the action, the outcome, and when. Comments attach to runs, review items, proposals,
threads, experiments, prompts, and bundles, each as a `StudioComment`.

The log is a `StudioAuditLog`, and comments a `StudioCommentStore`. Two classes implement both:

- `MemoryStudioJournal`, the default, bounded by `MemoryStudioJournalOptions`: 10,000 audit entries
  and 10,000 comments, the oldest dropped first;
- `FileStudioJournal`, append-only JSON Lines files that survive a restart and can be shipped to a log
  system as they are. On the command line, `--journal <directory>`.

With accounts, the studio can safely listen beyond the loopback interface:

- the host name goes in `allowedHosts`;
- the session cookie is marked `Secure` when the studio is reached over HTTPS;
- a studio listening on the network with only its single token warns at start.

`options.insights.slowMs` sets the latency the issues view reports as slow.

## Signing people in

`oidcSignIn()` signs people in with any OpenID Connect provider, given its issuer, the studio's
client id, and a role for each person. Give the `StudioSignIn` it returns to the studio as `signIn`:

```js
// studio.config.mjs
import { FileStudioJournal, oidcSignIn } from 'nexus-ai-pro-studio';

export default { traces, prompts: registry };
const journal = new FileStudioJournal('/var/lib/nexus-studio');
export const options = {
  host: '0.0.0.0',
  allowedHosts: ['studio.example.com'],
  signIn: oidcSignIn({
    issuer: 'https://login.example.com',
    clientId: 'nexus-studio',
    clientSecret: process.env.STUDIO_CLIENT_SECRET,
    redirectUri: 'https://studio.example.com/auth/callback',
    role: ({ groups = [] }) => (groups.includes('ml-admins') ? 'admin' : groups.includes('ml') ? 'editor' : undefined),
    secret: process.env.STUDIO_SECRET,
  }),
  secret: process.env.STUDIO_SECRET,
  audit: journal,
  comments: journal,
};
```

Someone who opens the studio without a session is sent to `/auth/login`, which sends them on to the
provider: the authorization code flow, with PKCE, a state, and a nonce, kept in a signed cookie for the
ten minutes they have to finish. `/auth/callback` exchanges the code, verifies the ID token against
the provider's keys, its issuer, the client as its audience, and the nonce, and takes them back to the
page they opened. `/auth/logout` signs them out, and out of the provider too when its configuration
has an `end_session_endpoint`. The page shows a link to it.

`OidcSignInOptions`:

- `role` decides each person's role from their ID token's claims, and `undefined` keeps them out;
- `id` picks their id, `sub` by default, and `tenant` their tenant;
- `clientSecret` makes a confidential client, sent in a Basic header, or in the body when the
  provider takes only that. Without it the studio signs in as a public client, by PKCE alone;
- `scopes` replaces `profile email`; `openid` is always asked for;
- `redirectUri` is the callback as registered with the provider. It defaults to `/auth/callback` on
  the address the request came to, so set it behind a proxy;
- `secret` signs the cookies: at least 32 characters, the same on every replica;
- `maxAgeSec` is how long a session lasts, 8 hours by default.

The session is a signed, HTTP-only cookie, `SIGN_IN_COOKIE`, so replicas that share the secret share
sessions, with no store. It is `SameSite=Lax`, so the page the provider sends someone back to opens
signed in; a change still needs the page token. `auth`, when also given, is asked first, so scripts
send bearer tokens to the same studio. Any flow of the same shape works as `signIn`: an
`authenticate` for a request's session, and a `handle` for the paths under `/auth/`.

`oidcSignIn()` reads tokens with nexus-ai-pro's own verifier. Every build signs a person in
through Dex, an OpenID Connect provider run in a container, so the flow is proven against a provider
written by others, not only against the specification.

## Tenants

One studio can serve many tenants. A `StudioUser` with a `tenant` reads the sources
`tenantSources` gives for it, instead of the studio's own:

```js
// studio.config.mjs
import { tenantScope } from 'nexus-ai-pro/tenancy';

export default {}; // Everyone belongs to a tenant, so the studio's own sources are empty.
export const options = {
  signIn: oidcSignIn({ ...provider, tenant: (claims) => claims.org_id }),
  tenantSources: async (tenant) =>
    (await customers.has(tenant)) ? tenantScope(tenant, { traces, prompts, datasets, experiments }) : undefined,
};
```

Each tenant's sources are opened once and kept. A tenant `tenantSources` does not know, by returning
`undefined`, is refused with `403`, as is anyone with a tenant on a studio given no `tenantSources`.
A tenant whose sources fail to open is an error, and is opened again on the next request.

- Comments record their author's tenant, and only that tenant's people see them.
- Audit entries record the tenant too. A tenant's admin reads their tenant's entries alone, whatever
  the audit log does with the filter; an admin without a tenant reads every entry, or one tenant's with
  `?tenant=`. A `StudioAuditQuery` is what `StudioAuditLog.list()` is asked.
- A page token is bound to its tenant, so a page in one tenant cannot carry a change in another.

The authenticators carry the tenant: `oidcSignIn()` and `bearerAuth()` take a `tenant` function of
the claims, and `headerAuth()` a `tenant` header, after which a request without it is refused. Each
person given to `personalTokens()`, and each entry of `--users`, may name a `tenant`.

Someone without a tenant reads the studio's own sources: the operators' view.

## Pieces you can reuse

`layoutGraph()`, re-exported from `nexus-ai-pro/graph/visualize`, lays a graph out in layers for
drawing. It returns a `GraphLayout` of `LaidOutNode` and `LaidOutEdge` values. Edges that point back
up — cycles — are marked, so they can be drawn around the side.

`parseArgs()` and `loadSources()` are the command's own argument parser and config loader.
`loadConfig()` also reads the `options` a config module exports.

## Limitations

- Someone without a tenant reads the studio's own sources. Give a studio where everyone belongs to a
  tenant empty sources of its own.
- Each tenant's sources stay open while the studio runs.
- A session is a signed cookie, not a record, so one person's cannot be ended before it expires.
  Rotate the sign-in `secret` to sign everyone out. A new role applies at the next sign-in.
- Signing out is a plain link, so another site can sign someone out of the studio, though never in.
- A page token stays valid while the `secret`, the user, and their role do. Rotate the secret to
  revoke every page at once.
- Someone without a session who opens another person's personal link is signed in as that person.
  The page shows who is signed in.
- `FileStudioJournal` reads its whole file for each listing, so rotate a long-lived one.
- It reads what the stores can list. An operation store without `list()` hides the queue, and a
  graph given without a way to list threads can only open a thread by id.
- Without `rollups`, the costs view reads up to 100,000 runs in its window, which is slow on a very
  large trace store. Give it rollups.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/server.md

# The agent server


A self-hosted HTTP server for your assistants, from `nexus-ai-pro/server`: threads, background runs,
event streams you can resume, and cron jobs.

It is a thin layer over parts that already exist. A run is a durable operation, and a thread is a
graph thread. What the server adds is the HTTP surface, and the rules for a thread that is already
busy. `nexus-ai-pro/server/remote` is the client, so calling a deployed agent never loads the server.

## A server in fifteen lines

```ts
import { createServer } from 'node:http';
import { createAgentServer, graphAssistant, toNodeListener } from 'nexus-ai-pro/server';
import { jwtAuth } from 'nexus-ai-pro/server/auth';

const server = createAgentServer({
  assistants: { support: graphAssistant(supportGraph, { description: 'Answers support questions' }) },
  authenticate: jwtAuth({ jwks: 'https://login.example.com/.well-known/jwks.json', audience: 'agents-api' }),
  onBusy: 'enqueue',
});

await server.start();
createServer(toNodeListener(server)).listen(8080);
```

`createAgentServer()` returns an `AgentServer`:

| Member | What it does |
| --- | --- |
| `handle(request)` | Answers one `Request`. This is the whole HTTP surface. |
| `runs`, `cron` | The run manager and the scheduler behind it. |
| `start()` | Re-claims runs a crashed worker left, starts the scheduler and the worker loop, and starts heartbeating. |
| `stop()` | Stops all of that. Runs in flight finish. |
| `drain()` | Prepares the replica to shut down. See [Workers and the queue](#workers-and-the-queue). |
| `scaling()` | The numbers an autoscaler reads, as `GET /scaling` reports them. |

`AgentServerOptions` takes:

- the `assistants`, by id;
- everything the run manager takes: the state store, the event log, the operation runner's settings,
  the `queue`, `recoverEveryMs`, the `tenants` limits, the busy policy, and the timeouts;
- the HTTP settings: `basePath`, `authenticate`, `allowAnonymous`, `scopes`, `cron` with the jobs that
  exist from startup, and `heartbeatMs` for live streams;
- `metrics`, which controls `GET /metrics` and `GET /scaling`;
- `deployments` and `replicaMetadata`, for revisions and replica reports;
- `tracer`, which makes each run the root of a trace, as below;
- `region` and `residency`, which place the server in a region and refuse tenants that may not reach
  it, as the [regions guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/regions.md#agent-servers-across-regions) describes.

With a `tracer` from `nexus-ai-pro/tracing`, each attempt of a run is an `agent` run of its own trace,
holding the run's input and output, with everything the assistant traces through the same tracer
beneath it: give the server the tracer its clients use, through `traceModelClient()` and
`traceRetriever()`. The trace is written before the run reads as finished, and the run records it as
`traceId`, so a deployment's evaluation judges the run by what it did. The
[tracing guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/tracing.md#graphs-and-model-calls) says what the root records.

## The HTTP surface

| Route | What it does |
| --- | --- |
| `GET /health` | Readiness, and which worker answered. Answered without credentials; `503` while draining |
| `GET /assistants`, `GET /assistants/:id` | What this server serves, and what each one supports |
| `POST /threads`, `GET /threads`, `GET /threads/:id`, `DELETE /threads/:id` | Conversations |
| `GET /threads/:id/state` | The assistant's state for a thread |
| `POST /threads/:id/runs` | Run on a thread; `{ "resume": … }` answers an interrupt |
| `GET /threads/:id/runs` | That thread's runs |
| `POST /runs`, `GET /runs`, `GET /runs/:id` | Stateless runs |
| `POST /runs/:id/cancel` | Cancel, including a run another replica is executing |
| `POST /runs/:id/feedback` | Leave feedback on a run: a `key`, and a `score`, a `value`, or a `comment` |
| `GET /runs/:id/events` | The event stream, resumable with `Last-Event-ID` |
| `POST /crons`, `GET /crons`, `DELETE /crons/:id` | Scheduled runs |
| `GET /metrics`, `GET /scaling` | The queue and this replica, as Prometheus text and as JSON |
| `GET /usage` | The caller's tenant usage, when `tenants` is set |
| `/deployments`, `/replicas` | Revisions, traffic splits, and replicas, when `deployments` is set |

A list, `GET /threads`, `GET /runs`, or `GET /threads/:id/runs`, holds the caller's tenant's
records, by when each was last written, newest first, then by id. It reads until the page is full, so
another tenant's newer records never crowd it out. `?limit=` sets the page (50), a whole number
above 0. While more follow, the answer has a `nextCursor`, and `?cursor=` asks for the page after it.
A record written while a client pages moves to a fresh first page, rather than appearing twice.
`RunManager.listThreads()` and `listRuns()` return the same `ListPage`.

Mount them under a prefix with `basePath`. A run is accepted with `202` and a `RunRecord`; add
`"stream": true` (or `?stream=true`) to start it and stream it in one request instead. Add
`"revision"` to run one revision of an assistant that has several.

The last three rows are covered in the [deployments guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/deployments.md).

## Assistants

An assistant is anything that streams events for an input. That is the `ServerAssistant` contract, and
its only required member is `stream`. Optional members are what a thread needs: `resume` to answer an
interrupt, `state` to report it, and `step` with `restore` for the rollback policy.

Two helpers build assistants:

- `graphAssistant()` serves a compiled graph. Server threads map to graph threads, so state,
  interrupts, and history are the graph's own. It needs only `GraphLike`, a structural slice, so the
  server entry point never imports the graph runtime. `GraphAssistantOptions` sets a `description`
  for the assistants endpoint, `metadata` recorded on every checkpoint, and `events`: projections
  of the graph's event stream, such as `['messages', 'tools']`, to record on each run, so a client
  following it sees model output and tool calls as they happen. The run's last event still carries
  its state and any question. A graph without a
  checkpointer still serves stateless runs, but cannot resume or roll back.
- `functionAssistant()` serves a plain function, which may return a value or yield events.

An assistant receives an `AssistantRunContext`: the run id, the thread, an abort signal, the principal,
the run's metadata, the attempt, the revision serving it, and a `control` that turns to draining
when the replica drains. Its `recordCost(usd)` records what the
run spent, such as a model call's cost:

```ts
const answer = await ai.complete(request);
await context.recordCost(answer.meta.cost?.amount ?? 0);
```

The amount is added to the run's `cost` and to its tenant's budget. A budget that stops runs in flight
cancels the run there.

An assistant can also have revisions. Its optional `route()` picks the revision for each run, and
`revision()` returns one revision's assistant. `Deployments.assistant()` builds both; see the
[deployments guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/deployments.md).

## Runs are durable operations

`RunManager` submits every run through the operation runner, so leases, heartbeats, retries,
idempotency, dead-lettering, and webhooks are the operations family's, not the server's. It is
available as `server.runs` and usable on its own — `StartRunOptions` and `RunManagerOptions` configure
it, and `RunRecord` and `RunStatus` are what it reports.

- **Crash recovery.** A worker that dies leaves a record whose lease lapses. Another replica's
  `start()`, or `runs.recover()`, claims it as a new attempt. So allow more than one attempt:
  `operations: { retry: { maxAttempts: 2 } }`. A graph or workflow served by `graphAssistant()`
  continues from the last checkpoint that run wrote, so only the step in flight runs again. Any other
  assistant starts over. The assistant sees the attempt in `AssistantRunContext.attempt`; its optional
  `recover()` hook is what continues the run.
- **Idempotency.** `idempotencyKey` on a run replays the existing run instead of starting a second,
  which is what makes a retried request safe.
- **Cancellation.** `POST /runs/:id/cancel` aborts a local run at once and is observed by another
  replica through its heartbeat. A cancelled run stays cancelled even if its executor finishes later.
- **Feedback.** `POST /runs/:id/feedback`, or `RunManager.addFeedback()`, adds feedback to the run
  record. It may be a person's rating or an online evaluator's score, and a run keeps its latest 100.
  The [canary guard](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/deployments.md#judging-with-confidence-and-on-quality) compares it between
  revisions as their quality.
- **Timeouts.** `runTimeoutMs` expires a run that runs too long.

A `RunRecord` also records how the run went:

| Field | What it holds |
| --- | --- |
| `revision` | The revision that served it, its share of traffic, and why it was chosen. |
| `startedAt`, `finishedAt` | When a worker first started it, and when it finished. |
| `durationMs` | Time from the first start to the finish, across every attempt. |
| `cost` | US dollars recorded through `recordCost()`, across every attempt. |
| `worker`, `attempt` | The worker that ran its latest attempt, and which attempt that was. |
| `traceId` | The trace of its latest attempt, when the server has a `tracer`. |
| `region` | The region it was accepted in, when the server has one. |

## Workers and the queue

By default, a run executes on the replica that accepted it. `queue` turns the replicas into a worker
pool instead. `RunQueueOptions` has three fields:

| Field | Default | What it does |
| --- | --- | --- |
| `concurrency` | 10 | Runs this replica executes at once. |
| `claim` | true | Whether this replica claims queued runs. `false` makes an API replica that only accepts them. |
| `pollMs` | 1 s | How often an idle worker looks for queued runs. |

A replica with room starts a new run itself. Otherwise the run waits in the operation store, and the
next free worker claims it. With a queue, each replica also takes over runs whose worker stopped, every
`recoverEveryMs` (30 seconds by default). Without a queue, set `recoverEveryMs` to get the same.

`GET /scaling` returns a `ScalingSnapshot`: queued and running runs across the deployment, their sum
as `load`, the oldest queued run's wait, lapsed leases, and the answering replica's
`ReplicaReport`. `GET /metrics` reports the same as Prometheus text, plus run counts and latency per
revision. Both need a read scope unless `metrics: { public: true }`; `metrics: false` turns them off.

`drain({ timeoutMs })` gets a replica ready to stop. `/health` answers `503`, the replica stops
claiming, and new runs it accepts are queued. Runs in flight get `timeoutMs` (25 seconds by default)
to finish. With a queue, the rest are handed back for another worker to continue. The `DrainResult`
lists the runs that finished and the runs that were handed off.

A client that reads a run's events slowly cannot make the server hold more: events go to the run's
event log, which keeps `maxEventsPerRun` per run, and each reader reads from it at its own pace.

A graph served through `graphAssistant()` does not wait for the timeout. With a queue, a drain sets
the run's `control` to draining, which the graph reads as its `RunControl`. It finishes the superstep
in flight, writes its checkpoint, and stops, and the run goes back to the queue at once. The next
worker continues from that checkpoint, so no step runs twice and none is cut off halfway. Call it on `SIGTERM`:

```ts
process.on('SIGTERM', async () => {
  await server.drain();
  await server.stop();
  process.exit(0);
});
```

## Handing a run off on a drain

When a replica drains with a queue to hand work to, it sets each run's `context.control.draining`. A
graph or a workflow served through `graphAssistant()` stops at its next boundary and is continued by
the next worker from its checkpoint. A function assistant does the same with three pieces of its
context:

- `context.control?.draining`, checked between its own steps;
- `context.saveProgress(details)`, which records how far it got — the last item finished, a cursor —
  and extends the run's lease, as a heartbeat does;
- `context.progress`, which the next attempt reads back.

Throwing `RunHandOffError` while draining puts the run back on the queue as its next attempt, at
once, instead of waiting for the drain's timeout; thrown when the replica is not draining, it fails
the run like any error.

```ts
const batch = functionAssistant(async (input, context) => {
  let next = typeof context.progress === 'number' ? context.progress : 0;
  for (; next < items.length; next += 1) {
    if (context.control?.draining) throw new RunHandOffError();
    await process(items[next]);
    await context.saveProgress?.(next + 1);
  }
  return { processed: next };
});
```

## A busy thread

One thread runs one thing at a time. `onBusy`, set per server or per run, decides what a second
request does, as `ThreadBusyPolicy`:

| Policy | Behaviour |
| --- | --- |
| `reject` | `409` with `THREAD_BUSY`. The default. |
| `enqueue` | Waits its turn behind the runs in flight, up to `queueTimeoutMs`, then runs; still busy then, it is refused with `THREAD_BUSY`. |
| `interrupt` | Cancels the run in flight and starts the new one. |
| `rollback` | Cancels it, puts the thread back to the step it was at before, and starts the new one. |

`rollback` needs an assistant with `restore`; `graphAssistant()` provides it by writing the earlier
checkpoint's state forward, so the history stays intact. An assistant without it is refused with
`AssistantCapabilityError` rather than silently doing something else.

The thread is claimed atomically. The claim is a record in the operation store under a unique
idempotency key, moved from run to run by compare-and-set, so of ten replicas starting runs on one
thread at the same moment exactly one proceeds and the rest meet the policy. Before 2.2 a thread's
busy check read and wrote separately, and replicas racing could all see it free. A custom operation
store without `findByIdempotencyKey` keeps that older behaviour.

## Streaming, and reconnecting

`GET /runs/:id/events` is a server-sent event stream. Every event carries the id it has in the run's
log, so a client that drops sends `Last-Event-ID` and gets exactly what it missed — no gaps, no
repeats. The stream ends on the event that reports a finished or awaiting-input run, and a run that
finished before a client attached still ends with its status.

Where the log lives decides how far that goes. `MemoryRunEventLog` is the default and covers one
replica; `RedisRunEventLog` shares the log, so a client can reconnect to any replica and resume.
`RunEventLog` is the contract, `RunEvent` the event, and `RedisEventLogLikeClient` the handful of
Redis commands the adapter needs. `MemoryRunEventLogOptions`, `RedisRunEventLogOptions`, and the
server's `heartbeatMs` bound memory use and keep idle connections open.

## Cron jobs

`CronScheduler` fires scheduled runs, configured at startup or through `POST /crons`. A schedule is
`{ everyMs }` or `{ cron }` — the five-field syntax, in UTC, parsed by `parseCron()`. `CronRecord` is
a job and `DueCronJob` is one that is due, carrying the `slot` it is due for.

Every replica ticks, and every firing is submitted with the idempotency key `<job>:<slot>`, so the
operation store decides which replica wins and the job runs once however many replicas are up. There
is no lock and no leader election. `CronSchedulerOptions` sets the tick interval, the store, and an
error hook; `tick()` can also be called directly, which is how an external scheduler or a test drives
it.

## Authentication and tenancy

`authenticate` turns a request into a `Principal`, which has these fields:
- `tenantId`, the tenant;
- `userId`, the subject;
- `roles` and `scopes`;
- `method`, how the caller was authenticated;
- `claims`, the verified claims.

Returning nothing refuses the request as unauthenticated; returning a `Response` answers it directly,
which is how a challenge or a redirect is returned. `allowAnonymous: false` refuses everything when
no hook is configured. `GET /health` is always answered, because a probe carries no credentials.

The principal goes everywhere the run goes:
- Its tenant scopes threads, runs, cron jobs, budgets, and the store.
- Its subject is recorded on what it creates and on every checkpoint, as `metadata.userId`, and in
  the run's trace.
- Graph nodes and workflow steps read it as `context.principal`.
- An agent passes it to every tool, middleware, and permission policy. A tool can therefore act as
  that user and no one else.

**Built-in hooks.** `nexus-ai-pro/server/auth` has hooks for tokens, keys, sessions, and a proxy,
each an `AuthHook` that returns the same `Principal`, and the pieces that issue what they read: tokens,
keys, and sessions of the application's own. They are built on Web Crypto, so they need no
dependency.

`jwtAuth()` verifies a JSON Web Token, read from `Authorization: Bearer` unless `token` reads it from
elsewhere, such as a cookie. The key comes from one of three places:
- an identity provider's key set (`jwks`), which is cached; a token that names a new key fetches
  the set again, so a key rotation needs no restart;
- a public key, as PEM, a JWK, or a `CryptoKey`;
- a shared secret.

`JwtAuthOptions` extends `JwtVerifyOptions`:
- the `JwtAlgorithm`s accepted;
- the `issuer` and `audience`;
- clock tolerance;
- `maxAgeSec`;
- required claims;
- `JwtClaimNames`, which says where the tenant, roles, and scopes are. A dot reaches a nested
  claim, as in `realm_access.roles`.

A token must carry an expiry. `alg: none` is never accepted. A secret is never accepted for an
asymmetric algorithm, nor a public key for HMAC, so one cannot be passed off as the other. With a
key set, `audience` is required, because a provider signs tokens for every application it serves.
`audience: false` accepts any audience on purpose.

Responses:
- A request with no token is passed over.
- A refused token answers `401`, with a `WWW-Authenticate` challenge naming why.
- A valid token whose claims make no principal answers `403`.

```ts
import { anyAuth, apiKeyAuth, hashApiKey, jwtAuth } from 'nexus-ai-pro/server/auth';

const server = createAgentServer({
  assistants,
  authenticate: anyAuth(
    jwtAuth({
      jwks: 'https://login.example.com/.well-known/jwks.json',
      issuer: 'https://login.example.com/',
      audience: 'agents-api',
      claims: { tenant: 'org_id', roles: 'realm_access.roles' },
    }),
    apiKeyAuth({ keys: { [await hashApiKey(process.env.CI_KEY ?? '')]: { userId: 'ci', tenantId: 'acme', scopes: ['write'] } } }),
  ),
});
```

`createJwtVerifier()` is the verifier on its own, for a WebSocket handshake or a queue consumer. Its
`JwtVerifier.verify()` returns a `VerifiedJwt`: the `JwtHeader` and the `JwtPayload`, every claim
checked. Otherwise it throws a `JwtError`, whose `JwtErrorCode` says why: malformed, wrong algorithm,
no key, bad signature, expired, not yet valid, too old, another issuer, another audience, or a
missing claim.

`apiKeyAuth()` authenticates services and scripts by key. Keys are compared by their SHA-256 hash,
from `hashApiKey()`, so neither configuration nor a database holds a key itself.
`ApiKeyAuthOptions` takes the hashes with the principal each stands for, a `lookup` by hash, the
header, and a `prefix`. The prefix keeps a token in the same header for another hook. Without one,
anything shaped like a JWT is left alone. An unknown key answers `401`.

`trustedProxyAuth()` takes the caller from headers that an identity-aware proxy, a gateway, or a mesh
sets. Headers are only as trustworthy as the path the request took. So `TrustedProxyAuthOptions`
must give proof that the proxy sent the request: a shared `secret` header, compared in constant
time, a `trust` check of your own, or both. Without that proof the hook refuses to be built.
`TrustedProxyHeaders` names the user, tenant, roles, and scopes headers.

`anyAuth()` tries hooks in order and takes the first principal. When none accepts, the answer is the
first refusal any of them gave, so a caller learns why their credential failed.

`scopes` names the scope each group of routes requires:

| Scope | Routes |
| --- | --- |
| `read` | Reads, streams, `/usage`, and — unless public — `/metrics` and `/scaling`. |
| `write` | Creating threads, runs, and cron jobs, and cancelling runs. |
| `admin` | `/deployments` and `/replicas`. Defaults to a scope named `admin` whenever `authenticate` is set. |

`tenants` enforces per-tenant limits on active runs, rate, and budget. Its contract is `TenantGate`,
and `tenantLimits()` builds one. See the [deployments guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/deployments.md#tenants).

Every thread, run, and cron job records its tenant, and a request only ever sees its own tenant's
resources. Another tenant's thread is a `404`, not a `403`, so the server does not even confirm it
exists.

Errors extend `ServerError` and have a stable code: `BadRequestError`, `UnauthorizedError`,
`ForbiddenError`, `NotFoundError`, `ThreadBusyError`, `AssistantCapabilityError`, and
`TenantLimitError`. An error with a `retryAfterSeconds` is answered with a `Retry-After` header.

### Tokens of your own

`createJwtSigner()` issues tokens that `jwtAuth()`, or any other service, verifies. Its
`JwtSignerOptions` take one key: a `secret` of at least 32 bytes, for HMAC; a `privateKey`, as a
PKCS #8 PEM or a private JWK; or a Web Crypto `keyPair`. The algorithm is the key's own unless
`algorithm` names it, which a PEM key must. A `JwtSigner`:
- `sign()`s claims, adding the signer's `issuer` and `audience`, `iat`, `exp` an hour on by
  default, and a `jti` of its own; a claim given wins, and `JwtSignOptions` set one token's
  lifetime, audience, and `nbf`;
- `issue()`s a token for a `Principal`, its user, tenant, roles, and scopes under the claim names
  `jwtAuth()` reads, so the same `JwtClaimNames` read it back as the same principal;
- gives its `publicKey()`, with a `kid` that defaults to the key's RFC 7638 thumbprint.

`generateJwtKey()` makes a private JWK, ES256 by default, to keep in a secret store. `jwksHandler()`
serves signers' public keys as a key set, cached for five minutes. Give it the signer in use and the
one before it, and a rotation refuses no token issued before it.

```ts
import { createJwtSigner, jwksHandler, jwtAuth } from 'nexus-ai-pro/server/auth';

const signer = createJwtSigner({ privateKey: JSON.parse(process.env.JWT_KEY ?? ''), issuer: 'https://agents.example.com', audience: 'agents-api' });
const jwks = jwksHandler(signer);
const token = await signer.issue({ userId: 'reporting', tenantId: 'acme', scopes: ['read'] });
// Another service verifies it against https://agents.example.com/.well-known/jwks.json.
```

### API keys you keep

`createApiKeyStore()` issues, rotates, and revokes keys for services and scripts, keeping each as an
`ApiKeyRecord` in the server's state store, so every replica reads the same keys. A record holds the
key's SHA-256, never the key: `issue()` returns it once, in an `IssuedApiKey`, with `ApiKeyIssueOptions`
for a `label` and a lifetime. A key begins with the store's `prefix`, `nxk_` by default, so a secret
scanner recognises it and `apiKeyAuth()` leaves a token alone. An `ApiKeyStore` also:
- `rotate()`s a key: a new one for the same principal, while the old works for `graceSec` more,
  none by default, and names its replacement;
- `revoke()`s one at once;
- `get()`s and `list()`s records, a tenant's alone if asked, with when each was last used;
- `lookup()`s a principal by hash, which `auth()` passes to `apiKeyAuth()`.

`ApiKeyStoreOptions` take the `store`, a `MemoryServerStore` by default, the prefix, and
`touchEveryMs`: how often a key's last use is written, once a minute by default. The use is kept
apart from the record, so recording it never undoes a revocation made at the same moment.

```ts
const keys = createApiKeyStore({ store: state });
const { key } = await keys.issue({ userId: 'ci', tenantId: 'acme', scopes: ['write'] }, { label: 'CI' });
createAgentServer({ assistants, state, authenticate: anyAuth(sessions.auth, keys.auth(), jwtAuth({ ... })) });
```

### Browser sessions

`createSessions()` signs browsers in after a sign-in of the application's own: a password, a magic
link, or an identity provider's redirect. Its `Sessions` `start()` one, returning a `StartedSession`: the `Set-Cookie`
value for the sign-in's response, and its `SessionRecord`. The cookie carries a random secret and the
store keeps its SHA-256, so a copy of the store signs no one in. `end()` signs the request's session
out and returns the value that clears the cookie; `revoke()` ends one by id, wherever its browser
is; `list()` shows live sessions, a user's or a tenant's, so "sign out everywhere" is a list and a
revoke; `sweep()` removes those that have ended. `auth` is the hook:
- a request with no cookie is passed over;
- an ended, idle, or forged one answers `401` and clears the cookie;
- a request that changes something, other than `GET`, `HEAD`, or `OPTIONS`, must come from the
  application's own pages. Its `Origin` must be the request's own or one of `origins`, or, with
  none, `Sec-Fetch-Site` must say so. Otherwise it answers `403`;
- the principal carries `method: 'session'` and the session's id as the `sid` claim.

`SessionOptions` take the `store`, `maxAgeSec` (8 hours), `idleSec` (the same), and `origins`,
the application's public origin when a proxy makes the server see another. `SessionCookieOptions`
write the cookie: `HttpOnly`, `Secure`, and `SameSite=Lax` by default, named
`__Host-nexus_session`, which a browser binds to this host alone; `secure: false` for plain HTTP on a
developer's machine.

```ts
const sessions = createSessions({ store: state, origins: ['https://agents.example.com'] });
// The sign-in route, once the user is known:
const { cookie } = await sessions.start({ userId: user.id, tenantId: user.orgId, scopes: ['read', 'write'] });
return new Response(null, { status: 303, headers: { location: '/', 'set-cookie': cookie } });
```

## Where state lives

Threads, runs, and cron jobs are records in a `ServerStateStore`. A `ThreadRecord` is a conversation:
its assistant, tenant, creator, creation and last-run times, the run in flight, and the metadata it
was created with. `MemoryServerStore` is the default;
`fromStore()` puts them in any long-term store — `MemoryStore`, `RedisStore`, `PostgresStore`, or
`SqliteStore` —
which is what lets a second replica see the first replica's threads. `StoreLike` is the slice of the
store contract it uses. Point the operation store at the same backend and the server is stateless:

```ts
import { createAgentServer, fromStore, RedisRunEventLog } from 'nexus-ai-pro/server';
import { RedisStore } from 'nexus-ai-pro/store/redis';
import { RedisOperationStore } from 'nexus-ai-pro/operations/adapters';

const server = createAgentServer({
  assistants,
  state: fromStore(new RedisStore(redis)),
  events: new RedisRunEventLog(redis),
  operations: { store: new RedisOperationStore(redis), retry: { maxAttempts: 2 } },
  cron: { tickMs: 30_000 },
});
```

## Serving it

`toNodeListener()` adapts the server to Node's `http`, and to anything that uses its request and
response objects. Express and Fastify take it as middleware, and a Nest controller can call it from a
route handler. Streaming responses are passed through unbuffered, which keeps an event stream live.

`NodeListenerOptions` sets the origin used to build request URLs, and an error hook. `NodeRequestLike`
and `NodeResponseLike` are the structural slices it needs.

```ts
import express from 'express';
const app = express();
app.use('/agents', toNodeListener(server, { origin: 'https://agents.example.com' }));
```

The repository's `deploy/` folder has a server configured from the environment, and everything to
run it. `deploy/app` is one image that runs as an API, a worker, or both, shared through Redis.
`deploy/compose.yaml` runs API replicas, a worker pool, Redis, and an nginx gateway that leaves event
streams unbuffered. `deploy/kubernetes` and `deploy/helm` run the same on Kubernetes, with the worker
pool autoscaled on the queue. The [deployments guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/deployments.md#kubernetes-and-helm) explains
them.

`ServerDeployments` is the contract the server needs from a deployments registry. `Deployments` from
`nexus-ai-pro/server/deployments` implements it.

## Calling a server

`createRemoteGraph()`, from `nexus-ai-pro/server/remote`, is the client; it returns a `RemoteGraph`,
whose `invoke()` and `stream()` mirror a compiled graph's. `invoke()` runs to
completion, `stream()` yields events and reconnects from the last one it saw, `resume()` answers an
interrupt, and `createThread()`, `state()`, and `cancel()` cover the rest. `RemoteGraphOptions`
configures the URL, assistant, headers, and timeouts; `RemoteRunResult` is what a run returns.

`asNode()` makes a deployed assistant a node in a local graph, so an orchestrating graph can call a
remote one as a subgraph without knowing it is remote:

```ts
const remote = createRemoteGraph({ url: 'https://agents.internal', assistant: 'research' });
const graph = createGraph({ channels })
  .addNode('research', remote.asNode())
  .addNode('write', writeNode)
  .setEntry('research')
  .addEdge('research', 'write');
```

## Limitations

- A minor release may add routes, and fields to what routes return. Existing ones keep their
  shape.
- Recovery repeats the step a run died in — for graphs and workflows — or the whole run, for any
  other assistant. The repeated part should be idempotent — the run id makes a good key — or the run
  should be left at one attempt.
- The `enqueue` busy policy waits in the request that is queued, so a queued run holds a connection.
  The worker `queue` does not: a run waiting for a worker holds nothing.
- Cron resolution is one minute, and schedules are UTC.
- A list's page deep in a long list reads every record before it, as the state store lists from the
  newest. A record a client had not reached when it was written again moves to the first page, so a
  walk of a list that changes does not see it. Two records written by replicas whose clocks disagree,
  closer together than that, may be listed in the order the store wrote them.
- The server signs people in by a session after a sign-in of the application's own: it keeps no
  users or passwords, and runs no identity provider's redirect itself.
- A session's last use is written at most once a minute, so it idles out up to a minute early.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/deployments.md

# Deployments: revisions, canaries, scaling, and tenants


Run the agent server as a service you operate yourself. This guide covers four things:

- **Revisions.** Ship several versions of an assistant in one image. Move traffic between them without a
  redeploy.
- **Canaries.** Send a share of traffic to a new revision. A guard rolls it back when it regresses.
- **Scaling.** Workers claim runs from a shared queue. An autoscaler sizes the pool from the queue.
- **Tenants.** The server holds each tenant to its active runs, its rate, and its budget.

Everything here builds on the [agent server](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/server.md). The Kubernetes manifests, Helm chart, and
Compose file are in the repository's `deploy/` folder.

```ts
import { createAgentServer, graphAssistant } from 'nexus-ai-pro/server';
import { Deployments, watchCanaries } from 'nexus-ai-pro/server/deployments';
import { tenantLimits } from 'nexus-ai-pro/server/tenancy';

const deployments = new Deployments({ state });
const server = createAgentServer({
  assistants: {
    support: deployments.assistant('support', {
      '2026-09-01': graphAssistant(supportV1),
      '2026-09-30': graphAssistant(supportV2),
    }),
  },
  deployments,
  tenants: tenantLimits({ default: { maxActiveRuns: 10, budget: { usd: 50, period: 'day' } } }),
  queue: { concurrency: 4 },
  state,
  events,
  operations: { store: operationStore },
});
await server.start();

await deployments.canary('support', '2026-09-30', 0.1); // a tenth of new traffic
watchCanaries({ deployments, steps: [0.25, 0.5] }); // rolls back or promotes on its own
```

## Revisions

A revision is one version of an assistant's code. `Deployments.assistant()` takes the revisions by id
and returns an assistant, which goes into `createAgentServer()` like any other. The server asks it
which revision should serve each run.

```ts
const support = deployments.assistant(
  'support',
  { '2026-09-01': graphAssistant(v1), '2026-09-30': graphAssistant(v2) },
  { live: '2026-09-01', description: 'Answers support questions' },
);
```

`RevisionedAssistantOptions` has two fields:

| Field | What it does |
| --- | --- |
| `live` | The revision that takes all traffic until a deployment is recorded. Defaults to the last one declared. |
| `description` | What the assistant is, for the assistants endpoint. |

Once a deployment is recorded, the stored record decides, not the code. So a new revision in a new image
takes no traffic until you give it some. The server records each assistant's default when it starts.

### How a run gets its revision

The server calls the assistant's `route()` with a `RevisionRequest`: the run, its thread, the thread's
last revision, and any revision the request named. The answer is a `RevisionChoice`, which is a
`RunRevision` plus the revision's code. `RevisionReason` says why it was chosen:

| Reason | When |
| --- | --- |
| `requested` | The request named a revision, as `"revision": "2026-09-30"` in the run body. |
| `thread` | The run is on a thread whose revision still takes traffic, so the thread stays put. |
| `split` | Everything else: the traffic split chose, by a stable hash of the thread or run id. |

`bucket()` is that hash. It maps a key to a fixed place between 0 and 1. Canaries take the start of the
range and the live revision the rest. So raising a canary's share only adds threads to it, and a
thread never flips back and forth.

How the key becomes a place is the deployment's `BucketStrategy`:

| Strategy | Places |
| --- | --- |
| `fnv1a-v1` | By FNV-1a. Random ids, which the server gives by default, spread evenly. Ids that differ only in their last characters, such as `user-1` to `user-500`, cluster, so a canary's share of them drifts from its weight. What every deployment recorded before 2.5 uses, and 2.x's default. |
| `hash-v2` | By FNV-1a mixed with MurmurHash3's finalizer, so sequential ids spread as evenly as random ones. The default for a deployment first recorded on 3.0. |

A deployment records its strategy, so every replica places a thread the same way, and no thread
moves through an upgrade: one recorded before 3.0 keeps `fnv1a-v1`. `Deployments.assistant()` takes
`bucketStrategy` for a deployment recorded from then on, and `bucketing()` changes a recorded one.
`bucket()` places a key by `fnv1a-v1` unless given the strategy, so pass it the deployment's. A thread already on a revision that takes
traffic stays there; only new threads are placed by the new strategy.

Every run records its `RunRevision`: the revision, its share, the deployment version, and the reason.
The thread records its revision too, which is how a rolled-back canary sends its threads home.

### Revision worker pools

A worker image does not have to carry every revision. Each worker claims only the queued runs routed
to revisions it carries, so a new revision ships in a new image beside the old one.
- **The run's kind names its revision.** A queued run's kind is `assistant:<id>@<revision>`. A worker
  claims with the kinds of every revision it registered, and of each assistant's runs without one.
  A run routed to a revision a worker lacks is left for a worker that has it, and never runs on
  another revision instead.
- **The API replicas route to any revision a worker carries.** A run that will wait in the queue is
  routed over the revisions any fresh worker reports carrying, through `RevisionRequest.queued`. The
  API replicas, with `queue: { claim: false }`, can then send traffic to a revision only the new
  image has. `RevisionChoice.local` says whether the answering replica carries the chosen revision.
  When it does not, the run is queued even when this replica has room.
- **The queue is counted per revision.** `/scaling` reports queued runs per assistant and revision in
  `queuedByRevision`, and `/metrics` as `nexus_server_runs_queued_by_revision`, so an autoscaler
  starts the pool that carries the waiting revision. A run whose pool is scaled to zero waits for it.

```ts
// The API image routes; each worker image carries the revisions its code has.
const api = createAgentServer({ assistants, state, operations, queue: { claim: false }, deployments });
const worker = createAgentServer({ assistants, state, operations, queue: { concurrency: 8 }, deployments });
```

### Changing the split

Every change is a `DeploymentChange`, applied by `change()` or one of its shortcuts:

| Method | What it does |
| --- | --- |
| `canary(assistant, revision, weight)` | Gives one revision a share, from 0 to 1, and the live revision the rest. |
| `split(assistant, traffic)` | Sets several shares at once. They add up to at most 1; the live revision takes the rest. |
| `promote(assistant, revision)` | Makes a revision live, with all the traffic. |
| `rollback(assistant, options)` | Pulls a canary. With no canary, undoes the last promotion. `to` names a revision instead. |
| `bucketing(assistant, strategy)` | Changes how new threads are placed in the split, keeping the split itself. |
| `markUnhealthy(assistant, revision, options)` | Marks a revision unhealthy with a `reason`, and rolls it back at once when it is a canary, unless `rollback` is false. |
| `markHealthy(assistant, revision, options)` | Clears a revision's mark. |

Each shortcut takes `by` and `reason`, recorded in the history. `change()` also takes
`expectedVersion`. It refuses the change with `DEPLOYMENT_CONFLICT` when someone moved the deployment
first, so two people cannot undo each other without noticing.

Changes are atomic across replicas when the state store has `putIfVersion()`. The memory, Redis,
Postgres, and SQLite stores have it, and `fromStore()` passes it on. A change is then written only if
the version it was decided on is still the stored one. A replica that loses the race decides again
on what the winner wrote, so a guard rolling a canary back and an operator promoting at the same
moment both land, in order. A change made on an `expectedVersion` is refused instead.
`Deployments.atomicChanges` and the `atomicChanges` field of `/scaling` say whether a deployment has
this. Without it, changes on one replica still apply in order, but two replicas can lose one.

```ts
await deployments.canary('support', '2026-09-30', 0.1, { by: 'ada', reason: 'new retrieval' });
await deployments.change('support', { action: 'promote', revision: '2026-09-30', expectedVersion: 1 });
await deployments.rollback('support', { reason: 'answers got longer' });
```

A `DeploymentRecord` is what gets stored: the live revision, the shares, the version, when a canary
began, the revisions marked unhealthy, and the last 50 changes. Each change is a
`DeploymentChangeRecord`. A mark is a change of its own, `{ action: 'health', revision, healthy }`,
recorded in the history with the revision it marked, and kept in `unhealthy` as a
`RevisionHealthMark` (the reason, when, and by whom) until the revision is marked healthy again. The
live revision is only marked: rolling it back is a decision for a person. `get()` reads one record,
`list()` reads them all, and `history()` reads one record's changes.

### Reading what each revision did

`stats()` summarizes each revision's runs. Each `RevisionStats` has the run count, successes,
failures, error rate, p50 and p95 latency, and mean cost. It reads one of two sources:

- **Rollups**, with `deployment`, a version. Every replica counts the runs it finishes in a
  `DeploymentRollup`: per revision, a `RevisionRollup` of runs, successes, failures, cancellations,
  latency in buckets 5% apart, and cost. A run is counted under the deployment version it was routed
  with, so one version's rollups are exactly the runs since that change, however many. Each replica
  writes only its own record, every `rollupMs`, so counting never contends, and `rollups()` sums them
  in one read. Percentiles from rollups are within 2.5%, and a run in flight is counted when it
  finishes. A version no replica counted falls back to run records.
- **Run records**, otherwise: the latest 2,000 by default, or `limit`. Pass `since` to limit them to
  a window, such as the time of the last change.

The server counts runs when it is given `deployments`: it calls `recordRun()` as each run finishes.
`flushRollups()` writes what this replica has counted, which `detach()` and every read of rollups do
first. `GET /deployments/:assistant` reports each revision's stats from rollups when replicas keep
them.

`samples()` returns one revision's finished runs in the shape `compareRuns()` from
`nexus-ai-pro/insights` reads. `revisions()` lists the revisions known for an assistant: the ones
registered in this process, and the ones fresh replicas report.

### `DeploymentsOptions`

| Option | Default | What it does |
| --- | --- | --- |
| `state` | memory | Where deployments and heartbeats live. Use the server's state store, so every replica shares them. |
| `heartbeatMs` | 10 s | How often a replica reports itself. |
| `replicaTtlMs` | 3 heartbeats | How long a report counts as fresh. |
| `cacheMs` | 2 s | How long routing reuses a read. A change elsewhere reaches every replica within this time. |
| `historyLimit` | 50 | Changes kept per deployment. |
| `rollupMs` | 1 s | How long a replica gathers finished runs before writing its rollup. |
| `onError` | — | Hears heartbeat and rollup failures, which never stop the server. A rollup that fails to write is kept and written next time. |
| `now` | the clock | Replaces the clock, for tests. |

The same object works without any assistants. Build it on the same store in a studio or a script, and
it reads and changes what the servers record.

## Canary guard

`watchCanaries()` judges every canary on a timer and acts on the result. It returns a `CanaryGuard`
with `check()`, which judges now, and `stop()`.

```ts
const guard = watchCanaries({
  deployments,
  minRuns: 50,
  steps: [0.25, 0.5],
  onDecision: (decision) => console.log(decision.action, decision.assistant, decision.regressions),
});
```

Each check compares a canary with the live revision over the same window: the runs since the split
last changed. It uses the same statistics as the insights' regression detection:

- an error rate counts when its rise passes a two-proportion z-test at 95%;
- p95 latency, and mean cost if you ask for it, count when they rise by a relative margin, a quarter by
  default. A latency rise must also be at least 50 ms.

A regressed canary is rolled back at once. The reason in the history names what got worse. A canary
that holds up moves to its next step, and is promoted after the last one. Without `steps`, it stays
where it is until a person promotes it.

With `from: 'rollups'`, the guard judges a canary on the rollups of the deployment's current version:
every run since the split last changed, counted by every replica, in one read. A busy assistant is
then judged on all of its canary's runs, not the latest 2,000, so a small rise in the error rate that
a few thousand runs cannot show is caught. From rollups, the error rate is judged by its z-test, at
`confidence` when it is set and with `minErrorRateIncrease`, and latency and cost by their margins.
Feedback lives on run records, so a guard over rollups takes neither `samples` nor `feedback`.

```ts
watchCanaries({ deployments, from: 'rollups', minRuns: 500, steps: [0.25, 0.5] });
```

### Judging with confidence, and on quality

Raw rates mislead in both directions. A few unlucky runs can roll back a good canary, and with enough
traffic a trivially small change still clears a significance test. `confidence` judges every metric
the way an evaluation comparison judges a score. Each metric is resampled by bootstrap, and it
regresses only when both of these hold:
- its interval at that confidence does not reach zero;
- the change is at least the metric's minimum effect.

| Metric | Minimum effect |
| --- | --- |
| error rate | `minErrorRateIncrease`, 0.02: two points |
| p95 latency | `latencyIncrease`, a quarter, and at least `minLatencyChangeMs` |
| mean cost | `costIncrease`, a quarter |
| a feedback score | `minFeedbackDrop`, 0.05 |

`resamples` sets the bootstrap's size, 1,000 by default. The resampling is seeded, so the same runs
always get the same verdict, and the rollback reason quotes the interval.

Quality comes from online evaluation. `Deployments.evaluate()` scores each finished run of an
assistant with evaluators from `nexus-ai-pro/evaluate`, such as an LLM judge, and records the scores
as the run's feedback. Name those keys in `feedback`, and a canary that answers worse is rolled back
like one that fails more. A person can leave feedback on a run too, through
`POST /runs/:runId/feedback`.

It runs through the [online-evaluation worker](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/evaluation.md#on-workers), drained once, so it
keeps the worker's guarantees: each run is one operation in `operations`, and replicas that share
that store score each run once between them, even calling at the same moment. An evaluation is
reserved from a `budget` before its evaluators run, and an uncertain one goes to a `reviewQueue`.
Given the server's `traces`, from a server with a [`tracer`](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/server.md#a-server-in-fifteen-lines),
each evaluator sees the run's input as `context.example.inputs` and what it did as `context.trace`:
its tool calls, the documents it retrieved, its model calls, and the media it received or produced,
as the [evaluation guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/evaluation.md#judging-a-run-by-its-trace) describes. An assistant that
traces itself is found by the run id it records, as `metadata: { nexus: { run: context.runId } }`.

```ts
// Every few minutes: score new runs, then judge the canary on quality as well as errors and speed.
await deployments.evaluate('support', {
  evaluators: [helpfulnessJudge, grounded],
  traces: traceStore,
  operations,
  sampleRate: 0.2,
});
watchCanaries({ deployments, confidence: 0.95, minRuns: 50, feedback: ['helpfulness', 'grounded'] });
```

`RunEvaluationOptions` sets:
- the `evaluators`;
- the `traces` the server writes, and the `operations` store evaluations queue in (memory, for one
  process, by default);
- `since`, the start of a canary for instance;
- one `revision`;
- a `sampleRate`, chosen by run id so every replica scores the same runs;
- how many runs are read (`limit`);
- the `source` the scores are recorded under. A run already scored by that source is skipped;
- what the worker takes as it is: `concurrency`, `leaseMs`, `budget`, `estimate`, `costOf`,
  `tenantId`, `reviewQueue`, `reviewWhen`, `onScored`, `onError`, `owner`, and a clock, with
  `pollMs` for how long a drain waits on an evaluation another replica holds.

`RunEvaluationReport` counts the runs scored, skipped, and failed, and lists every score.

For scoring as runs finish rather than on a schedule, `Deployments.evaluationWorker()` returns the
worker itself, an `OnlineEvaluationWorker` to `start()` on every replica, over one shared
`operations` store. `DeploymentEvaluationWorkerOptions` are the same options, with `operations`
required and `maxQueued`. A worker reads the server's runs again each `cacheMs`.

```ts
const worker = deployments.evaluationWorker('support', {
  evaluators: [helpfulnessJudge],
  traces: traceStore,
  operations: new PostgresOperationStore(pool, { table: 'evaluations' }),
  budget: budgetLedger({ limit: 50 }),
  estimate: 0.002,
});
worker.start();
```

`CanaryGuardOptions`:

| Option | Default | What it does |
| --- | --- | --- |
| `deployments` | — | The registry to watch, and where runs are read. |
| `assistants` | all | Only watch these. |
| `everyMs` | 1 minute | How often to judge. |
| `minRuns` | 20 | Finished runs each side needs before a canary is judged. |
| `metrics` | error rate, latency | What is compared. Add `cost` to compare mean cost. |
| `latencyIncrease`, `costIncrease` | 0.25 | The relative rise that counts. |
| `minLatencyChangeMs` | 50 | The smallest latency rise that counts, so noise on fast runs never rolls a canary back. |
| `steps` | none | Shares to move through while the canary holds up, before promotion. |
| `from` | `runs` | What a canary is judged on: run samples, or `rollups` of every run since the split last changed. |
| `samples` | run records | Where runs come from, each with its feedback. Give it traced runs to judge on their feedback instead. |
| `feedback` | none | Feedback keys to compare, such as the ones `Deployments.evaluate()` records. |
| `confidence` | none | Judges each metric by a bootstrap interval and a minimum effect, as below. |
| `minErrorRateIncrease`, `minFeedbackDrop`, `resamples` | 0.02, 0.05, 1,000 | The minimum effects and the bootstrap size under `confidence`. |
| `by` | `guard` | Who the history says made the change. |
| `onDecision`, `onError` | — | Hear each decision, and each failure. |

Each `CanaryDecision` names the assistant, the canary, the live revision, the action, the regressions,
the samples on each side, and the canary's share afterwards. The action is `rollback`, `advance`,
`promote`, or `hold`. A hold means a side is still short of `minRuns`, or the canary holds up without
steps.

A canary marked unhealthy since it began, by a person or an [automation rule](#automation-rules), is
rolled back on the guard's next check however few its runs, and its `CanaryDecision` carries the
`unhealthy` mark it was rolled back on. A mark made before the canary began is ignored, so a revision
fixed and tried again is judged afresh.

A guard on every replica is safe. Each change carries the version it was judged on, so the first guard
to act wins and the others find the deployment already moved.

## Automation rules

An automation rule acts when a score falls. `automationRules()`, from
`nexus-ai-pro/evaluate/automation`, judges each rule's score over the most recent runs and, when it
falls below a threshold over enough samples, runs its actions: open review items for people, send a
webhook, or mark the revision unhealthy, which rolls a canary back.

```ts
import { automationRules, deploymentScores, markUnhealthy, openReview, sendWebhook } from 'nexus-ai-pro/evaluate/automation';
import { mediaJudge } from 'nexus-ai-pro/evaluate/media';

const rules = automationRules({
  scores: deploymentScores(deployments, 'designer'),
  state: store,                  // shared by every replica, so a fall is acted on once
  rules: [
    {
      name: 'banner quality',
      key: 'banner',
      below: 0.6,
      minSamples: 20,
      actions: [markUnhealthy({ deployments }), openReview(queue), sendWebhook({ url, secret })],
    },
  ],
});
const worker = deployments.evaluationWorker('designer', {
  evaluators: [mediaJudge({ client: ai, model: 'gpt-5', rubric: 'A wide, legible launch banner.', key: 'banner' })],
  operations,
  traces,
  onScored: rules.observe,       // judged as the scores arrive
});
worker.start();
```

Each `AutomationRule` takes:

| Field | Default | What it does |
| --- | --- | --- |
| `name`, `key`, `below` | — | The rule, the score key it judges, and the threshold. |
| `statistic` | `mean` | A `RuleStatistic`: the mean score, or `pass-rate`, the share that passed, from `passed` or a score of at least 0.5. |
| `minSamples` | 20 | Runs a scope needs before it is judged. A run scored several times counts once, as its mean. |
| `window`, `windowMs` | 100 runs | The most recent runs judged per scope, and how old a score may be. |
| `confidence` | none | Fires only when the upper bound of the interval at this confidence is below, so a few unlucky samples never fire it. |
| `source`, `where` | all | Narrows the scores, to one source or by a test of your own. |
| `by` | `revision` | What is judged on its own: each revision of each assistant, so a canary is judged apart from the live revision; each `assistant`; `all` as one; or a function of a `ScoreSample`. |
| `actions` | — | `AutomationAction` functions, run in order. One that fails goes to `onError`, and the rest still run. |
| `repeatAfterMs` | never | Fires again after this long while a scope stays below. |

A rule fires once when a scope falls, and again only after it recovers, back at the threshold or
above over `minSamples` runs, or after `repeatAfterMs`; too few runs to judge is not a recovery.
Each `AutomationEvent` is `fired` or `recovered`, for one scope, with the statistic's value, its
interval, the samples, the assistant and revision, the five lowest samples, and a summary sentence.
Recovery runs no action.

**Where scores come from.** A `ScoreSource` gives a key's most recent scores. `deploymentScores()`
reads them from a deployment's run records, where online evaluation and people leave them, through
`Deployments.scores()`, which returns each as a `RunScore`: the run, its revision, the key and score,
the source, and when. `traceScores()` reads the feedback on a trace store's runs. Both are shared by
every replica, so every replica judges the same scores. `memoryScores()` keeps them in one process,
a `MemoryScoreSource` fed by `record()` or `observe()`, and is the default.

**Acting once.** `AutomationRulesOptions.state`, a long-term store with `putIfVersion()`, holds each
rule's firing per scope. A replica claims a firing before it acts, so of several replicas judging
the same fall at once, one acts. `namespace`, `scanLimit` (5,000 samples read per rule), `onEvent`,
`onError`, and a clock are the rest.

**Actions.** `markUnhealthy()` calls `Deployments.markUnhealthy()` for the event's revision, by
`rule:<name>`, with the summary as the reason; `rollback: false` leaves the rollback to a canary
guard. `openReview()` opens an item in a `ReviewOpener`, such as an `AnnotationQueue`, for each of the
lowest-scoring runs, `limit` (5) of them, with its input and output. `sendWebhook()` posts the event
as JSON, signed with HMAC-SHA256 in `x-nexus-signature` when given a `secret`, by its
`AutomationWebhookOptions`; a refusal goes to `onError`.

`AutomationRules` methods: `check()` judges every rule now; `observe()` hears an online
evaluation's scores and judges the rules on their keys at once; `status()` returns an
`AutomationStatus` per rule and scope; `start()` and `stop()` judge on a timer.

In the test suite, a designer assistant draws banners and its canary draws them square. Two replicas
score the runs through the evaluation worker with a media judge shown each banner, and a rule on the
judge's score marks the canary unhealthy: it is rolled back, the five worst banners go to people,
and one signed webhook is sent, once between the replicas. The live revision is never marked, new
runs go to it, and the media catalog finds every one of the canary's banners by its low score.

## Queues and autoscaling

By default, a run executes on the replica that accepted it. With `queue`, the server runs a worker pool
instead:

```ts
createAgentServer({ ..., queue: { concurrency: 4 } });                 // accepts and runs
createAgentServer({ ..., queue: { claim: false } });                   // an API replica: accepts only
createAgentServer({ ..., queue: { concurrency: 8, pollMs: 500 } });    // a worker
```

A replica runs at most `concurrency` runs at once. While it has room, it starts a new run itself, so an
idle deployment adds no latency. The rest wait in the operation store, and the next free worker claims
them. Each claim is a compare-and-set, so no run executes twice. A worker also takes over runs whose
worker stopped, every 30 seconds by default.

### What the autoscaler reads

`GET /scaling` returns the numbers as JSON, and `GET /metrics` returns them as Prometheus text:

| JSON field | Metric | Meaning |
| --- | --- | --- |
| `queued` | `nexus_server_runs_queued` | Runs waiting for a worker. |
| `running` | `nexus_server_runs_running` | Runs executing or retrying. |
| `load` | `nexus_server_queue_load` | Queued plus running: what to size the pool by. |
| `oldestQueuedSeconds` | `nexus_server_queue_oldest_seconds` | How long the oldest queued run has waited. |
| `lapsedLeases` | `nexus_server_leases_lapsed` | Runs a stopped worker left, until another takes them over. |
| `queuedByRevision` | `nexus_server_runs_queued_by_revision` | Queued runs per assistant and revision, for sizing each revision's pool. |
| `atomicChanges` | — | Whether deployment changes are atomic across replicas. |
| `replica` | `nexus_server_worker_*` | The answering replica: its runs, capacity, and whether it is draining. |

Size the pool as `load` divided by each worker's `concurrency`. The deployment-wide numbers read the
same on every replica, so aggregate them with `max()` in PromQL, never `sum()`.

The metrics also count runs per revision and status, run duration, queue wait, hand-offs, and tenant
refusals. Pass `metrics: { public: true }` to answer both routes without credentials, for a scraper
inside the cluster. Keep them off the public ingress then.

### Draining

On `SIGTERM`, call `drain()` and then `stop()`:

```ts
process.on('SIGTERM', async () => {
  await server.drain({ timeoutMs: 25_000 });
  await server.stop();
  process.exit(0);
});
```

A draining replica answers `/health` with `503`, so the load balancer stops sending to it. It stops
claiming, and new runs it accepts go to the queue. Runs in flight get `timeoutMs` to finish. Any still
running are handed back to the queue, and another worker continues them from their last checkpoint.
The hand-off does not count against the retry budget. `drain()` resolves to the runs that finished and
the runs it handed off.

### Kubernetes and Helm

The repository's `deploy/` folder has everything to run it:

| Path | What it is |
| --- | --- |
| `deploy/app` | The server: one image, run as `ROLE=api`, `worker`, or `all`, configured from the environment. |
| `deploy/kubernetes` | An API Deployment, a worker Deployment, a KEDA `ScaledObject`, an HPA alternative, and a demo Redis. |
| `deploy/helm/agent-server` | The same as a Helm chart, with autoscaling by KEDA, HPA, or neither. |
| `deploy/compose.yaml` | Two API replicas, a worker pool, Redis, and an nginx gateway. |

KEDA reads `/scaling` directly, so it needs no Prometheus. Readiness probes use `/health`, which answers
`503` while draining. Liveness probes use a TCP check, so a draining pod is never restarted for it.
Set the worker's `terminationGracePeriodSeconds` longer than its drain timeout.

## Tenants

`tenantLimits()` holds each tenant to its limits. Give the result to `createAgentServer({ tenants })`.

```ts
import { RedisTenantUsage, tenantLimits } from 'nexus-ai-pro/server/tenancy';
import { RedisRateLimitStore } from 'nexus-ai-pro/ops/rate-limit-adapters';

const tenants = tenantLimits({
  default: { maxActiveRuns: 5, rate: { runs: 60, windowMs: 60_000 } },
  tenants: { acme: { maxActiveRuns: 50, budget: { usd: 200, period: 'month', stopRuns: true } } },
  usage: new RedisTenantUsage(redis),
  rates: new RedisRateLimitStore(redis),
});
```

`TenantLimits` has three limits, each optional:

| Limit | What it does |
| --- | --- |
| `maxActiveRuns` | Runs the tenant may have accepted and unfinished at once, queued ones included. |
| `rate` | Runs the tenant may start per window. |
| `budget` | US dollars per period. `stopRuns` also cancels runs in flight once it is spent. |

A `BudgetPeriod` is `hour`, `day`, `week`, or `month` in UTC, or `{ windowMs }`. Weeks start on
Monday. Spending comes from what runs record through `recordCost()` on their `AssistantRunContext`.

### What happens when a run arrives

The server calls the gate when it accepts a run. The gate checks three things in order:

1. The budget. A spent budget refuses the run until the period resets.
2. The rate window. It counts the run, and refuses it past the limit.
3. An active slot. It takes one, and refuses the run when all are taken.

A refusal is a `TenantLimitError`: a `429` with the code `TENANT_BUDGET`, `TENANT_RATE_LIMITED`, or
`TENANT_CONCURRENCY`. `TenantLimit` names which limit refused. `Retry-After` says how long to wait when
that is known. The slot is freed when the run ends, on whichever worker ran it.

`TenantGate` is the contract the server calls: `admit()`, `release()`, `spend()`, and `usage()`.
`tenantLimits()` returns a `TenantLimiter`, which adds `report()` for several tenants at once.

### `TenantLimitsOptions`

| Option | Default | What it does |
| --- | --- | --- |
| `default` | none | Limits for tenants without their own, and for requests without a tenant. |
| `tenants` | none | Limits per tenant id, or a function that looks them up. It runs once per admitted run. |
| `usage` | memory | Where active runs and spending are counted. |
| `rates` | memory | Where rate windows are counted, as a `RateLimitStore`. |
| `slotTtlMs` | 1 hour | How long a slot lasts if its release never comes. Set it above the longest run. |
| `now` | the clock | Replaces the clock, for tests. |

### Where usage is counted

A `TenantUsageStore` has five operations: take a slot, free it, count slots, add to a total, and read a
total. Two are included:

- `MemoryTenantUsage` counts in the process. It suits one replica.
- `RedisTenantUsage` shares the counts across replicas. Each operation is one atomic script, so two
  replicas can never both take a tenant's last slot. It needs only `eval`, the
  `RedisTenantUsageLikeClient`; `RedisTenantUsageOptions` sets the key prefix. A budget reservation
  it records sits in the Cluster slot of the total it charges, so one script moves both.
- `PostgresTenantUsage`, from `nexus-ai-pro/postgres/tenancy`, shares them through Postgres, for a
  deployment that runs Postgres and not Redis. The [Postgres guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/postgres.md#tenant-usage)
  covers it.

`GET /usage` returns the caller's `TenantUsage`: its limits, active runs, and spending this period.

## Replicas

When the server has `deployments`, each replica heartbeats a `ReplicaReport`. The report has its id,
start time, runs in flight, capacity, whether it claims and whether it is draining, the revisions it
has, and any `replicaMetadata` you add, such as the host. `replicas()` returns the fresh ones.

## The HTTP routes

| Route | Scope | What it does |
| --- | --- | --- |
| `GET /deployments` | admin | Every deployment. |
| `GET /deployments/:assistant` | admin | One deployment, with each revision's stats since the last change, from rollups when replicas keep them. |
| `POST /deployments/:assistant` | admin | A `DeploymentChange`, with `expectedVersion` if you like. A `health` change marks a revision. |
| `GET /replicas` | admin | Fresh replica reports. |
| `GET /scaling`, `GET /metrics` | read, or public | The queue and this replica. |
| `GET /usage` | read | The caller's tenant usage. |

The admin scope is `admin` unless `scopes.admin` names another. It applies whenever the server
authenticates callers. The studio's deployments view and `nexus deploy` use these routes. So does a
pipeline that rolls out an image and then moves traffic onto it:

```bash
nexus deploy canary support 2026-09-30 10 --url https://agents.internal --reason "build 412"
nexus deploy status support --url https://agents.internal
nexus deploy promote support 2026-09-30 --url https://agents.internal
```

## Limitations

- With a state store of your own that has no `putIfVersion()`, two replicas changing one deployment
  in the same instant can lose one change: give the server one of the included stores, or pass
  `expectedVersion` and retry a conflict.
- A replica that dies loses the runs it counted in its last `rollupMs`. The run records stay whole.
- A deployment recorded before 3.0 keeps `fnv1a-v1`, so one keyed by sequential ids splits them
  evenly only once `bucketing()` moves it to `hash-v2`.
- A worker of 2.4 or earlier claims a run of any revision, and runs its live revision when it lacks
  the one routed. Finish a rollout from 2.4 before images start carrying different revisions.
- A replica that both serves the API and claims runs routes a run it has room for only among its own
  revisions. Put the API on its own replicas, with `claim: false`, for a canary that only a new
  image carries.
- Rollups are counted by replicas of 2.5 or later that are given `deployments`. During a rollout from
  2.4, the runs older replicas finish are missing from them. A guard over run records, the default,
  reads the latest 2,000, so judge a very busy assistant with `from: 'rollups'`.
- One rollup record is kept per replica per deployment version. They are small, and nothing deletes
  them.
- Automation rules given a `state` store without `putIfVersion()`, or none, claim a firing per
  process: two replicas judging the same fall at once can both act. Give them a shared store with it.
- A rule judges the most recent `scanLimit` scores of its key, 5,000 by default, across every scope.
  Raise it for a key many assistants share.
- The Redis operation store reads every record to find queued work unless it is given `index: true`.
  Postgres and SQLite always use an index. The [operations guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/operations.md) says when to turn it
  on.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/tenancy.md

# Tenancy


One tenant model for everything that stores data. A request, a graph run, and a server run carry a
`tenantId`, and every store has a tenant view that implements the store's own contract: a tenant's
code takes a `TraceStore`, a `PromptStore`, or a `VectorStore` as it always did, and cannot reach
past its tenant.

```ts
import { tenantScope } from 'nexus-ai-pro/tenancy';

// In a request handler, once the principal is known:
const scoped = tenantScope(principal.tenantId, { store, traces, prompts, vectors, cache });
const answer = await scoped.vectors.search(question, { topK: 5 });
```

The views work over the memory, Redis, Postgres, SQLite, and file stores alike, with no schema
change. They are checked by an isolation suite that runs against every storage adapter and fails on
any cross-tenant get, list, search, delete, fork, resume, trace query, vector search, or cache hit.

## Tenant ids

A tenant id is 1 to 128 letters, digits, underscores, and hyphens. `assertTenantId()` returns a valid
one unchanged and throws `TenantIdError` otherwise, and every view checks its id the same way. The
rule is what makes prefixing safe: a scoped name is `tenant/<id>/<name>`, and since no id holds a
`/`, no tenant's prefix is the start of another's — `acme` can never read `acme-x`'s names.

## Scoped views

Each function takes a store and a tenant id and returns that tenant's view of it:

| Function | Scopes | How |
| --- | --- | --- |
| `tenantStore()` | long-term memory | every namespace placed under `['nexus:tenant', tenantId]` (`TENANT_NAMESPACE` is the first part); searches and namespace listings cannot reach past it |
| `tenantTraceStore()` | traces | runs saved through it are stamped `metadata.tenantId`; `get()`, `query()`, `tree()`, and feedback see only stamped runs |
| `tenantOperationStore()` | operations | records carry `tenantId`, idempotency keys are prefixed so two tenants' keys never collide, and queue reads pass the tenant as a filter |
| `tenantPromptStore()` | prompts, and context hub bundles | names prefixed on the way in and taken off on the way out |
| `tenantDatasetStore()`, `tenantExperimentStore()` | datasets, experiments | names, ids, and dataset names prefixed |
| `tenantVectorStore()` | retrieval chunks | ids prefixed, metadata stamped `tenantId`, and every search filtered by it |
| `tenantCache()` | a cache adapter | keys prefixed |
| `tenantAssetStore()` | assets | the tenant bound once, so no call can name another; a `TenantAssetStore` |
| `tenantRollupStore()` | dashboard rollups | rows added and read under the tenant |

`tenantScope()` builds every view at once from a `TenantStores` object, keeping each store's type
and passing through anything it was not asked to scope.

What a view refuses is the same everywhere: another tenant's record reads as missing, a delete or
an update reaches only the caller's own, and a listing or search never returns another tenant's
item. Methods that would act on every tenant at once — `TraceStore.prune()` and `CacheAdapter.clear()`
— are left off the views; run them against the shared store from operations code.

A context hub needs nothing of its own: give a `ContextHub` the tenant's prompt store, and its
bundles, labels, and history are the tenant's.

```ts
const hub = new ContextHub({ store: tenantPromptStore(prompts, tenantId) });
```

## Where the tenant flows

- **Requests.** `CompletionRequest.tenantId` charges a shared budget to the tenant and is recorded on
  the request's trace as `metadata.tenantId`, which hourly rollups count by. The semantic response
  cache only matches entries of the same tenant; see the [caching guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/caching.md) for cache
  namespaces.
- **Graph runs.** `tenantId` is a run option. It is recorded on every checkpoint as
  `metadata.tenantId`, nodes read it as `context.tenantId`, and `context.store` is the tenant's view of
  the graph's store. A thread another tenant owns is not found: resuming, continuing, forking, or
  editing it under a different tenant fails with `GraphThreadNotFoundError` before anything runs, and
  `state()` and `history()` given a `tenantId` read it as absent. A run without a `tenantId` carries
  on its thread's. The [graphs guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/graphs.md) has the details.
- **The agent server.** `graphAssistant()` passes the principal's tenant into every graph run, so a
  server run's checkpoints and store are the tenant's own. The server already scopes threads and runs
  by tenant, and `tenantLimits()` caps each tenant's active runs, rate, and spend; the
  [deployments guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/deployments.md) covers them.
- **Rate limits and budgets.** A client's `rateLimit.key: 'tenantId'` gives each tenant its own
  limit, so one tenant cannot spend another's share, and a shared spend budget charges each
  operation to its `tenantId` by default.
- **Operations.** `OperationSubmitOptions.tenantId` records the tenant on an operation, and
  `OperationStoreFilter.tenantId` narrows `listQueued()` and `stats()`, which Postgres and SQLite
  answer from an index of queued work by tenant.

## Limitations

- A view is a boundary for code that is handed the view. Code holding the shared store can still
  read every tenant; give a tenant's handlers only its views.
- Prefixing changes what is stored: a prompt named `support` is stored as `tenant/acme/support`.
  Data written before a view was used is not the tenant's until it is copied under its prefix.
- A tenant's trace queries in Postgres use the index version 2 of the trace schema adds; other stores
  filter as they always do.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/policy.md

# Policy


What each tenant may reach, its guardrails, routing, budgets, regions, and credentials, as one
versioned file the application supplies: validated when the client starts, compared with the version
before it by `nexus policy check` in CI, and recorded on every operation it decides. A change that
lets a tenant reach more is reviewed like code, and acknowledged where the reviewer reads it.

```json
{
  "name": "production",
  "version": "7",
  "guardrails": { "preset": "enterprise" },
  "routing": {
    "denyModels": ["gpt-4o-mini"],
    "tenants": { "initech": { "allowModels": ["gpt-5*"] }, "globex": { "denyProviders": ["anthropic"] } }
  },
  "budgets": { "period": "month", "default": 100, "tenants": { "acme": 2000 } },
  "residency": {
    "regions": {
      "eu-west": { "providers": ["openai-eu", "mistral"], "twins": { "openai": "openai-eu" } },
      "us-east": { "providers": ["openai", "anthropic"] }
    },
    "tenants": { "acme": "eu-west", "globex": "any", "initech": ["us-east"] },
    "unassigned": "refuse"
  },
  "credentials": { "secretName": "tenants/{tenant}/{provider}" }
}
```

```ts
import { readFile } from 'node:fs/promises';
import { awsSecrets } from 'nexus-ai-pro/credentials';
import { applyPolicy, parsePolicy } from 'nexus-ai-pro/policy';

// Throws a PolicyError naming every issue, so a bad policy fails here, not on a request.
const policy = parsePolicy(await readFile('policy.json', 'utf8'));
const ai = new NexusAI(applyPolicy({ providers }, policy, { secrets: awsSecrets(client) }));
```

## The document

A `PolicyDocument` is plain JSON. Only its `name` and `version` are required; a section it leaves out
leaves the client's own setting in place.

- `guardrails`: the client's `security` settings, as `SecurityConfig`: a `preset`, a `level`, and the
  input and output settings.
- `routing`, a `PolicyRouting`: which providers and models may serve, for every tenant, and in
  `tenants`, for each one. Each is a `PolicyAccess`: `allowProviders` and `allowModels`, where only
  what is listed may serve, and `denyProviders` and `denyModels`, which never do. An entry matches a
  name equal to it, and `*` in it matches anything, as `gpt-5*` or `openai*`. An empty allow list is
  refused as a mistake: leave it out to admit any, or deny `*` to admit none.
- `budgets`, a `PolicyBudgets`: what each tenant may spend in US dollars per `period` (`hour`, `day`,
  `week`, or `month`, the default), with a `default` for a tenant with none of its own and for
  operations with no tenant.
- `residency`, a `PolicyResidency`: its `regions`, each tenant's rule in `tenants`, and the
  `unassigned` and `untenanted` rules, as `residency()` takes them in the
  [regions guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/regions.md#residency).
- `credentials`, a `PolicyCredentials`: where each tenant's provider keys are read, the secret's name
  as `secretName` with `{tenant}` and `{provider}` filled in (`{tenant}/{provider}` by default), what
  `missing` does for a tenant with no secret (`refuse`, or `config` for the provider's configured
  key), and `cacheMs`, `missingCacheMs`, and `staleMs`, as `tenantCredentials()` takes them.
- `acknowledged`: the changes this version makes that a reviewer has accepted, as below.

## Validated when the client starts

`parsePolicy()` reads the document, as JSON text or as an object, and returns a `Policy`: its `name`,
`version`, and `hash`, the document, and the `reference` each operation records. A document that is
not valid throws a `PolicyError` whose `issues` list every `PolicyIssue`, each a `path` and a
`message`: a section or a routing rule it does not know, a region a rule names but no region defines,
a provider listed in two regions, a level or a preset that does not exist, a negative budget, an empty
allow list.

`policyHash()` is the SHA-256 of the content with every object's keys in order, so formatting and key
order never change it and any value does: two files that claim one version are told apart.

## Applied to a client

`applyPolicy()` returns the client's configuration under the policy. Its guardrails go over the
client's `security` settings, keeping the level the client set when the policy sets none. In
`lifecycle`, its residency becomes the residency policy, its credentials a `tenantCredentials()`
resolver over the `secrets` of `ApplyPolicyOptions`, its budgets a budget ledger, kept in `budgetStore`
when every replica should share them, and its routing a `ProviderGate` built by `providerGate()`. A
policy with a `credentials` section and no `secrets` throws a `PolicyError`. Its global model rules
are given to routing as well, so the router chooses among the models the policy admits; a client's own
allow list is kept, and the gate still holds the policy's.

`LifecycleConfig.providers` takes any `ProviderGate`: a `check()` that returns `true` for a provider,
and model when one is named, an operation may reach, or why not. It is checked wherever residency is.
A provider it refuses is refused with an `OperationDeniedError` naming the rule and the policy, such
as `provider openai is denied by routing.tenants.globex.denyProviders (policy production@7)`, before
anything is sent, and audited as `blocked`; a completion's attempt it refuses is dropped from the
route, as one outside the tenant's regions is, and an embedding never falls back to a model it
refuses. Model rules apply to every family the gate sees, so a tenant held to `gpt-5*` embeds with
nothing else either. A residency refusal under a policy names it too: `(tenant acme: eu-west; policy
production@7)`.

`LifecycleConfig.policy` takes the `PolicyReference`, its name, version, and hash, and every operation
records it, as `OperationDescriptor.policy` and on each of its `AuditLogEvent`s as `policy`, so an
audit answers which version of the policy allowed a call. `ai.lifecycle.access()` returns the
`CallResidency` an operation for a tenant would get, for code that decides outside one, and its
`refusal()` says why a call to a provider would be refused once placed on its twin.

## Reviewed in CI

`nexus policy check` validates a policy file, and, given `--base`, compares it with the version it
replaces, a file or `git:<ref>` for the same file at a commit:

```bash
npx nexus policy check policy.json --base git:origin/main
```

```text
Policy production@8 (sha256 4be1c09a7f31) is valid. Compared with production@7 from git:origin/main:
kind      path                    review            summary
--------  ----------------------  ----------------  ------------------------------------------------------------------
narrowed  budgets.tenants.acme                      budgets.tenants.acme lowered: 2000 to 1500
widened   residency.tenants.acme  NOT ACKNOWLEDGED  residency.tenants.acme may now reach us-east: eu-west before, eu-west, us-east now

1 change needs acknowledging: add "residency.tenants.acme" to "acknowledged" in policy.json, or pass --accept residency.tenants.acme.
```

`comparePolicies()` lists each `PolicyChange`: its `path`, its `kind`, a `PolicyChangeKind`, the
values `before` and `after`, and a `summary`.

| Kind | Changes |
| --- | --- |
| `widened` | A tenant's rule gains a region or becomes `any`, as do `unassigned` and `untenanted`; a region gains a provider or a twin; an allow list admits a name no entry before it did, or goes; a deny list lifts a name; a budget rises or goes; the guardrails' level falls, or the section goes; a tenant with no key gets the shared one. |
| `narrowed` | The reverse of each. |
| `changed` | A tenant's home region moves within the same regions; a twin is replaced; other guardrail or credential settings, the budget period, or the name change. |

`checkPolicyChange()` returns a `PolicyCheck`: every change, those `unacknowledged`, whether the
content changed while the version did not (`versionUnchanged`), and `ok`. A widened or changed
change must be acknowledged, by its path or a section above it (`residency` covers
`residency.tenants.acme`), in the new version's `acknowledged`, where the reviewer reads it, or with
`accept`, `--accept` on the command line. A narrowed one never needs to be. The command exits 1 when
the policy is not valid, a change is not acknowledged, or the content changed without a new version,
and prints the check as JSON with `--json`. A file the base commit does not have has nothing to compare
and passes once valid. In a CI checkout, fetch the base branch, as `fetch-depth: 0` does.

In the test suite, version 3 of a policy is committed, and version 4 lets an EU tenant reach the US as
well. `nexus policy check --base git:HEAD` fails, naming the tenant's rule; the client built from
version 4 does send that tenant's calls to the US, and from version 3 does not. Acknowledged in the
file, or accepted on the command line, the check passes; the same change without a new version fails
either way.

## Limitations

- The check compares documents, not clients: a setting the client makes beside the policy, or a
  section the policy leaves to the client, is not compared.
- Lists are compared entry by entry: an entry covered by one before it, as `gpt-5-mini` by `gpt-5*`,
  admits nothing new, but two lists that admit the same names written differently are a change.
- Tenants' regions are a map in the document. A rule that reads a tenant's record belongs in code,
  given to `residency()` directly.
- A budget is per tenant and period. A ledger keyed another way, or with a reservation lease of its
  own, is configured in code.
- `--base git:<ref>` reads the file at the same path; a file that moved has no base there.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/regions.md

# Regions and credentials


Each tenant's requests and data kept in the regions it requires, and each tenant's provider keys,
resolved when a call is made from the secret manager you already run, rotated there without a
restart, and kept out of every error, log, trace, and audit event.

```ts
import { residency } from 'nexus-ai-pro/residency';

const policy = residency({
  regions: {
    'eu-west': { providers: ['openai-eu', 'mistral'], twins: { openai: 'openai-eu' } },
    'us-east': { providers: ['openai', 'anthropic'] },
  },
  tenants: { acme: 'eu-west', globex: 'any' },
  unassigned: 'refuse',
});
const ai = new NexusAI({
  providers: {
    openai: { apiKey: process.env.OPENAI_API_KEY ?? '' },
    custom: [{ name: 'openai-eu', format: 'openai', baseUrl: 'https://eu.api.openai.com/v1', apiKey: process.env.OPENAI_API_KEY }],
  },
  lifecycle: { residency: policy },
});
// Routed to openai, sent to openai-eu: acme's request never leaves the EU.
await ai.complete({ model: 'gpt-5', tenantId: 'acme', messages: [{ role: 'user', content: 'Hello' }] });
```

## Residency

`residency()` builds a `ResidencyPolicy` from `ResidencyOptions`:
- `regions`, each region's `RegionDefinition`: the `providers` that run in it, by the name each is
  registered under, and its `twins`, a provider outside the region mapped to one inside it that serves
  the same models, such as the EU endpoint of the same service. A provider is listed in one region.
- `tenants`, each tenant's `RegionRule`, as a map or a function, such as one that reads the tenant's
  record: `any`, `refuse`, a region, or a list of regions, the first its home. A region a rule names
  must be defined; a map is checked when the policy is built.
- `unassigned`, the rule of a tenant `tenants` says nothing of, and `untenanted`, the rule of an
  operation with no tenant, both `any` by default.

`LifecycleConfig.residency` applies it to every operation. When an operation starts, the lifecycle
decides its `ResidencyDecision`, the regions its tenant may reach and the rule that says so, and every
provider it reaches must run in one of them. A completion or a stream is placed after routing: each
attempt, the first and every fallback, is sent to the provider routing chose when that runs inside the
regions, to its twin inside them when there is one, and dropped when neither, so a fallback never
leaves the regions either; the context-window summary is placed the same way. In every other family,
the provider an operation reaches, named, chosen by default, or chosen for what it supports, is
checked, and an embedding never falls back to a provider outside the regions. A provider that runs in
no region is outside every rule but `any`.

What runs outside the regions is refused with a `ResidencyError` before anything is sent. It carries
the `code` `RESIDENCY_DENIED`, the `target` it would have reached (a `ResidencyTarget`: its kind, name,
and region), the tenant, the regions, and the `rule`, and its message quotes them:
`provider anthropic runs in us-east, outside the regions tenant acme may reach (tenant acme: eu-west)`.
The audit log records it as `blocked`. A provider of your own reads the operation's `CallResidency`
from `ProviderCallContext.residency` to choose among providers the same way: `allows()`, `place()`,
which returns a `ResidencyPlacement`, and `check()`.

`Residency.regions()` lists the regions; `regionsFor()`, `regionOf()`, `place()`, `allows()`, and
`check()` are the policy's own, for code that decides outside an operation.

### Traces and assets in their regions

`regionalTraceStore()` writes traces to a trace store per region, by `RegionalTraceStoreOptions`: the
`stores`, the `residency` policy, how to read a run's tenant (`tenantOf`, by default its
`metadata.tenantId` or the `metadata.nexus.tenant` an agent server records), the region of a trace
with no tenant (`defaultRegion`), and how many traces' regions to remember (`maxTraces`, 10,000).
Each run goes to its tenant's home region, the first it may reach that has a store, and a run with no
tenant goes where its trace went, so a trace stays in one region. A run whose tenant may reach no
region with a store is refused with a `ResidencyError` and written nowhere; a tracer reports it
through its `onError`. Queries read every region, newest first across them. A `RegionalTraceStore`
also has `forTenant()`, a view that reads and writes one tenant's regions only, and
`regionOfTrace()`.

`regionalAssetStore()` does the same for assets, by `RegionalAssetStoreOptions`: each asset is stored
in its tenant's home region and read, signed, and deleted only through the regions the tenant may
reach, so an asset is never served outside them, even after the tenant's rule changes.

In the test suite, a provider server per region records every request that reaches it. A tenant
pinned to the EU is answered by the EU twin of the provider routing chose, for completions and
streams, and nothing of it reaches the US; a provider with no twin there is refused, with the rule,
before anything is sent. An embedding whose EU provider fails does not fall back to the US, and each
tenant's traces, every run of a trace, and assets land in its home region.

```ts
import { NexusAI } from 'nexus-ai-pro';
import { awsSecrets, tenantCredentials } from 'nexus-ai-pro/credentials';
import { SecretsManager } from '@aws-sdk/client-secrets-manager';

const ai = new NexusAI({
  providers: { openai: { apiKey: process.env.OPENAI_API_KEY ?? '' } },
  lifecycle: { credentials: tenantCredentials({ secrets: awsSecrets(new SecretsManager({})) }) },
});

// Sent with acme's own key, read from the secret `acme/openai`.
await ai.complete({ model: 'gpt-5', tenantId: 'acme', messages: [{ role: 'user', content: 'Hello' }] });
```

## Agent servers across regions

Run one agent server per region, each with its own state store, queue, traces, and assets, and put
`regionalGateway()`, from `nexus-ai-pro/server/regions`, in front of them. A server given
`AgentServerOptions.region` records it on every `ThreadRecord`, `RunRecord`, and `ReplicaReport`, says
it in `GET /health`, and starts every thread and run id with it, as `eu-west~thread-…`, so the gateway
sends every later request for one to the region that holds its data. Given `residency` as well, it
refuses a thread or a run for a tenant that may not reach its region with a 403, `RESIDENCY_DENIED`,
whoever sends it.

```ts
import { createAgentServer } from 'nexus-ai-pro/server';
import { regionalGateway } from 'nexus-ai-pro/server/regions';

// In each region:
const server = createAgentServer({ assistants, state, region: 'eu-west', residency: policy, authenticate });

// In front of them:
const gateway = regionalGateway({
  regions: {
    'eu-west': { url: 'https://eu-west.agents.internal' },
    'eu-central': { url: 'https://eu-central.agents.internal' },
    'us-east': { url: 'https://us-east.agents.internal' },
  },
  residency: policy,
  tenantOf: async (request) => (await verify(request)).tenantId,
});
gateway.start();
```

The gateway answers each request by `RegionalGatewayOptions`: the `regions`, each a `RegionTarget`
with the server's `url` or a `handle` in this process; the `residency` policy; `tenantOf`, which
reads a request's tenant, such as from a verified token; `health`; `lists`; a `fetch`; and an
`onError`.

- **A thread or a run** is sent to the region its id names, and only there. A tenant that may not reach
  that region is refused with a 403; while the region is down, the request is answered 503,
  `REGION_DOWN`, with a `Retry-After`, since the thread's data is there and nowhere else.
- **New work**, a thread, a stateless run, or any other request, goes to the tenant's home region, the
  first one its rule names, and, when that is down or answers 502 or 504, to the next region the rule
  names, with the same body; never to another. A tenant every region of which is down is answered 503;
  one that may reach no region the gateway serves is refused with a 403 naming the rule.
- **A list of the tenant's threads or runs**, `GET /threads` or `GET /runs`, is gathered from every
  region the tenant may reach. Each is asked for the same page, and their answers are merged newest
  first and cut to the `limit` (50). The answer names the `regions` that answered and, under
  `incomplete`, each `RegionListGap`: a region that was down, or that failed or answered an error, and
  why. A refusal every region gives, such as a missing credential, is the answer; with no region
  answering, it is 503, `REGION_DOWN`. `lists: 'home'` asks the home region alone, as before 2.18.
  While more follow, the answer has a `nextCursor`, and `?cursor=` asks for the page after it. A
  cursor names a place in the order every region lists by, last written then id, so each region is
  asked for its own page after that place, and the merged page is exact: each record once, in order,
  however the records fall between regions. A region that does not page by the cursor, as one before
  2.18, contributes only what comes after it.
- Every answer carries `x-nexus-region`, the region that gave it, or, for a merged list, each that
  did.

A region is down after `failuresToDown` (1) failures in a row, of the checks `start()` runs every
`everyMs` (10 seconds) against `path` (`/health`, which a draining server answers with 503) within
`timeoutMs` (2 seconds), or of the requests it forwards; a check that passes brings it back. These are
the `RegionHealthOptions`. A `RegionalGateway` also has `check()`, `status()`, which returns each
region's `RegionHealth`, and `markDown()` and `markUp()`, for an operator taking a region out: a
region marked down is neither probed nor brought back until it is marked up.

In the test suite, three regions each run an agent server with its own state, traces, and provider,
behind a gateway, every tenant's key from one secret manager. A run of four model calls has its key
rotated and revoked after the second, and the third and fourth are answered on the new key. Then the
EU tenant's home region goes down: its new threads and stateless runs go to the other EU region, its
existing thread waits for its own region, a tenant held to that region waits too, a US tenant never
notices, and nothing of either EU tenant reaches the US: no thread, run, trace, or provider call. A
tenant working in both EU regions lists every thread and run of both, newest first, through the
gateway, and while one region is down, those of the other, with the missing one named.

## Credentials per tenant

`LifecycleConfig.credentials` takes a `CredentialResolver`: something that turns a
`CredentialRequest` (the provider, by the name it is registered under, the tenant, the family, and
the operation) into a `ProviderCredential`, an `apiKey` and any `headers`. Every family runs through
the lifecycle, so the resolver is asked for each provider a call reaches. An adapter sends what it
returns in place of the key it was configured with: OpenAI and every OpenAI-compatible provider as a
bearer token (in `api-key` for Azure, as its `credentialHeader` says), Anthropic in `x-api-key`,
Gemini in the URL, Cohere, embeddings, and images in their own headers, and every other family as
below. A resolver that returns nothing leaves the configured key; one that throws refuses the call.

`tenantCredentials()` is the resolver over a `SecretSource`, any object with `get(name)`. It reads
the secret named `<tenant>/<provider>`, or `<provider>` for a call with no tenant, or whatever `name`
returns, and `parseSecret()` reads it into a credential: a plain string is the key, and a JSON object
carries `apiKey` and `headers`, such as an organization. `TenantCredentialsOptions`:

| Option | Default | What it does |
| --- | --- | --- |
| `secrets` | — | Where the secrets are. |
| `name`, `parse` | as above | The secret's name for a request, and how it becomes a credential. |
| `missing` | `refuse` | A tenant with no secret is refused with a `CredentialError` before anything is sent, or, with `config`, sent the configured key. |
| `cacheMs` | 5 minutes | How long a credential is reused before the source is read again. |
| `missingCacheMs` | 30 seconds | How long a missing secret is remembered as missing. |
| `staleMs` | 1 hour | While the source fails, how long a credential past its time still carries calls. |
| `maxEntries` | 1,000 | Credentials cached at once, the least recently read dropped first. |
| `onError` | — | Hears each failure of the source. |

Calls that need one credential at the same moment share one read of the source. A `TenantCredentials`
also has `clear()`, which forgets every cached credential or one tenant's, and `stats()`, which
returns `TenantCredentialStats`: reads, hits, stale credentials served, refusals, and what is cached.

**Rotation.** A key changed in the secret manager is used from the next call that reads the source:
within `cacheMs`, or at once after `clear()`. When the provider has already revoked the old key, it
is sooner still. A completion or a stream refused for its credential is resolved again and made once
more on the new key, without counting as a retry. In any other family, the operation fails and the
next one reads the source again. A key the source still holds, and the provider still refuses, fails
the call after that one more attempt.

**Never written down.** Every key and header value an operation resolves is removed from the errors it
raises, before anything records them: the message, the stack, the cause, and the fields they hold,
so a provider that says which key it refused is quoted with `[redacted]` in its place. The audit log,
the logs, traces, and the provider health report see the cleaned error, and a credential is never
part of a request, so it never reaches a trace or a cache.

### In every family

Each family takes the tenant as its requests do elsewhere, `tenantId`, and sends the tenant's key in
place of the configured one:

| Family | Requests that take `tenantId` | Sent as |
| --- | --- | --- |
| Video | `VideoGenerateRequest`, as before | Runway's bearer token; Veo's `x-goog-api-key`, in place of a configured key or access token |
| Media understanding | `MediaReadRequest`, `MediaDescribeRequest`, `MediaAnswerRequest` | Mistral OCR's bearer token; a vision model through `modelMediaProvider()` is asked through the client with the tenant, so the client resolves its key |
| Batches | `BatchSubmitRequest`, `EmbeddingBatchSubmitRequest` | Anthropic's `x-api-key`, OpenAI's bearer token, Gemini's `x-goog-api-key` |
| Voice | `TranscriptionRequest`, `SpeechRequest` | OpenAI's bearer token |
| Telephony | Every call, number, recording, and message request | Telnyx's bearer token; for Twilio, `<accountSid>:<authToken>` of the tenant's subaccount, which is used in the request's path too, or an auth token for the configured account |
| Realtime | `RealtimeSessionConfig.tenantId`, as before | The OpenAI WebSocket's bearer token, Gemini Live's `key` |

A video job or a batch outlives its operation: it is polled, collected, or cancelled later, by its
ref, from any process. Its ref records the tenant it was started for, so every later call on it,
`status()`, `resume()`, `cancel()`, `download()`, uses the tenant's key and is held to the tenant's
regions; a ref recorded before 2.14 is resumed as the `tenantId` `resume()` names. These calls run
outside an operation, through `OperationLifecycle.callContext()`: it gives a `ContinuedCallContext`,
the tenant's credentials and residency and a `scrub()` for the call's errors, without authorizing,
budgeting, or auditing another operation. `OperationLifecycleLike.callContext` is optional, so a
lifecycle of your own without it leaves those calls on the configured key.

A realtime session resolves its tenant's key for its provider when it connects and reconnects, as
`RealtimeSessionConfig.credential`, which an application can also set itself. A key the application
mints into an ephemeral token is the application's to choose, and an OpenAI SIP call is attached
with the key of the project that received it, since the call belongs to that project. Every error the
connection raises is cleaned of the key before the session reports it: to its listeners, its span,
and the provider health report.

### Secret sources

Each source reads a secret that does not exist as none, and throws when the store cannot be read.
None needs a dependency: each takes a client you build with the vendor's own SDK, through the
methods it already has.

| Source | Reads from |
| --- | --- |
| `awsSecrets(client, { prefix })` | AWS Secrets Manager, through an `AwsSecretsClient`, such as `SecretsManager` from `@aws-sdk/client-secrets-manager`. |
| `gcpSecrets(client, { project, version })` | Google Secret Manager, through a `GcpSecretsClient`, such as the `@google-cloud/secret-manager` client. `/` in a name becomes `-`. |
| `azureSecrets(client)` | Azure Key Vault, through an `AzureSecretsClient`, such as `SecretClient` from `@azure/keyvault-secrets`. Anything but letters, digits, and `-` becomes `-`. |
| `vaultSecrets(options)` | HashiCorp Vault's KV version 2 engine, over HTTP, by `VaultSecretsOptions`: the `address`, a `token` or a function that returns the current one, the `mount` (`secret`), the `field` (`value`), and a `namespace`. |
| `envSecrets({ prefix, env })` | Environment variables: `acme/openai` is read from `ACME_OPENAI`, after `prefix`. |
| `memorySecrets(initial)` | This process, as `MemorySecrets`, whose `set()` and `delete()` change it: for tests, and for keys an application rotates itself. |

`credentialRequest()` builds the request a resolver receives, for code that resolves a credential
outside a call.

In the test suite, a provider of the test's own takes only the keys it is told are live. Each tenant's
calls carry its own key through completions, streams, and embeddings, and, against a stand-in for each
provider's API, through video, media understanding, completion and embedding batches, voice,
telephony, and realtime: a video job and a batch keep the tenant's key on every later call by their
ref, from a manager built after a restart, a Twilio call goes out on the tenant's subaccount, and a
SIP call keeps its project's key. In a run of five calls whose
key is rotated after the second, and revoked by the provider, the third is refused, resolved again,
and answered on the new key, and every call after it uses that key. A key the provider quotes back in
its refusal is found in no error, stack, cause, audit event, log, trace, or health report.

## Limitations

- `ai.plan()` shows the route before placement: the provider routing chose, not its twin.
- A region missing from one page of a merged list, as `incomplete` names it, is not caught up on the
  pages after: its records newer than the cursor are on a fresh first page.
- The gateway decides by its own health checks and the requests it forwards; a region that answers but
  cannot do the work behind it is up until a check or a request says otherwise.
- A cache, a checkpointer, or any other store the application gives the client is wherever the
  application put it. The response cache keeps each tenant's entries apart; to keep them in a region,
  give the region its own cache.
- A credential is resolved for the adapter's own name in the families that took it in 2.14, such as
  `runway` or `twilio`: name each tenant's secret after it, as `acme/runway`.
- A Gemini Live ephemeral token is minted by the application, with the key it chooses; the OpenAI
  realtime server helpers mint with the tenant's, as the
  [realtime guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/realtime.md#server-helpers) describes.
- A resolved key is removed from errors when it is at least four characters long, and only where it
  appears whole.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/workflows.md

# Workflows


Ready-made workflows from `nexus-ai-pro/workflows`, for common jobs you would otherwise prompt by hand.

- **Chains** answer from sources, extract structured data, classify, compare options, and summarize
  with verification.
- **Domain templates** triage support tickets, qualify sales leads, review contracts, and review code.

Each is a function over any client with a `complete()` method. It returns the final content, the
last response, and every step taken.

```ts
import { ragAnswer, supportTriageWorkflow } from 'nexus-ai-pro/workflows';

const answer = await ragAnswer(ai, { model: 'gpt-5.4-mini', question, chunks });
const triage = await supportTriageWorkflow(ai, { model: 'gpt-5.4-mini', input: ticket, customerTier: 'enterprise' });
console.log(triage.content, triage.steps.length);
```

## The shape every workflow shares

A workflow needs one thing from a client: a `complete()` method. That is the `WorkflowClient`
contract. A `NexusAI` client satisfies it, and so does a test double that returns scripted responses.

Every workflow returns a `WorkflowResult`:

- `content`, the final answer;
- `response`, the final response, with its usage and cost;
- `steps`, one `WorkflowStepResult` per completion — its name and its response — so you can see how
  the answer was reached.

The client also offers `ai.summarizeVerifyFormat()` directly.

## Chains

| Chain | What it does | Its options |
| --- | --- | --- |
| `ragAnswer()` | Answers a question from retrieved passages, citing them by id. | `RagAnswerOptions`: `model`, `question`, `chunks`, and `verify`, which repairs claims the passages do not support. A repair costs a second completion. |
| `extractStructured()` | Pulls data matching a JSON Schema out of text. The client validates the output. | `ExtractStructuredOptions`: `model`, `input`, `schema`, and an optional `instruction`. |
| `classifyRoute()` | Picks exactly one label, at temperature 0, answering `{ label, confidence }`; with `decisions`, a typed decision's choice, with every label's `probabilities`. | `ClassifyRouteOptions`: `labels`, an optional `instruction`, and `decisions`, a decision manager and its provider and model. |
| `compareAndDecide()` | Weighs options against criteria and chooses one, reasoning privately first. | `CompareOptions`: `input`, `options`, and `criteria`. |
| `summarizeVerifyFormat()` | Summarizes, checks the summary against sources, and formats the result. | `SummarizeVerifyFormatOptions`, below. |

`SummarizeVerifyFormatOptions` takes the `input`, and optionally:

- `verifyContext`, the sources the summary must agree with. The verify step repairs any claim they do
  not support. Without sources, that step is skipped.
- `responseFormat`, applied only to the final answer;
- instructions that replace each step's default.

So it costs two completions, or three or four with verification.

```ts
import { summarizeVerifyFormat } from 'nexus-ai-pro/workflows';

const brief = await summarizeVerifyFormat(ai, {
  model: 'gpt-5.4-mini',
  input: meetingNotes,
  verifyContext: [decisionLog],
  responseFormat: { type: 'json_schema', schema: briefSchema },
});
```

## Domain templates

The domain workflows answer in JSON with a fixed set of fields, so their output can be parsed and
routed. Each takes `DomainWorkflowOptions`: the `model`, the `input`, and optional `context` passages
for the prompt. Each also has one option of its own:

| Workflow | Its option | Required fields in the answer |
| --- | --- | --- |
| `supportTriageWorkflow()` | `SupportWorkflowOptions.customerTier` | `severity`, `category`, `nextAction`, `reply` |
| `salesQualificationWorkflow()` | `SalesWorkflowOptions.product` | `fitScore`, `painPoints`, `recommendedOffer`, `followUpEmail` |
| `legalReviewWorkflow()` | `LegalReviewWorkflowOptions.jurisdiction` | `risks`, `suggestedQuestions`, `safeSummary` |
| `codeReviewWorkflow()` | `CodeReviewWorkflowOptions.language` | `findings`, `riskLevel`, `testSuggestions` |

`legalReviewWorkflow()` adds an explicit fallback for text it cannot judge, reasons privately, and is
not legal advice. Each template is one completion, so it is a starting point to copy and adapt, not
a fixed product.

## Limitations

- The templates name their required fields but not their types; add your own schema through
  `extractStructured()` when a field's shape matters.
- Verification compares claims with the sources as text; see the [grounding guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/grounding.md)
  for how, and for what it cannot catch.
- Without `decisions`, `classifyRoute()`'s `confidence` is what the model says of its own answer, not
  a probability. With them, it is read from every label's probability, as the
  [decisions guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/decisions.md) describes.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/cli.md

# The command line


The package installs a `nexus` command. It is a thin layer over the library, for the terminal and for
CI scripts. No library entry point imports it.

```bash
nexus scan src --json
nexus eval run eval.mjs --out candidate.json --baseline baseline.json --fail-on-regression
nexus traces list --store runs.jsonl --status error --since 2026-09-20T00:00:00Z
nexus deploy canary support 2026-09-30 10 --url https://agents.internal
```

## The commands

| Command | What it does |
| --- | --- |
| `nexus scan` | Checks files for secrets, PII, and prompt-injection patterns. |
| `nexus models` | Lists the bundled model registry. |
| `nexus models export`, `check`, `refresh` | Write the bundled registry as a file, check a file, and compare one with the providers' own model lists. |
| `nexus optimize` | Previews token optimization for a request or a prompt file. |
| `nexus eval` | Runs JSON or JS eval cases. |
| `nexus eval run`, `compare`, `gate` | Run an evaluation module to an experiment file, compare two, and fail a build only on a regression beyond noise. |
| `nexus traces list`, `show`, `export` | Read a JSONL trace file, or any trace store a module exports. |
| `nexus db sql`, `status`, `migrate` | Print the Postgres schema, show which migrations a database has, and apply the rest. |
| `nexus deploy` | Reads and changes an agent server's deployments over HTTP. |
| `nexus migrate` | Moves imports to the subpaths the 2.0 root import keeps them on. |
| `nexus graph lint` | Checks a compiled graph's shape for designs that fail in production. |
| `nexus adapters verify`, `page`, `recheck`, `gather` | Record an adapter's contract suite and benchmark, list records on one page, verify each listed adapter again against the installed release, and gather the records adapters' own CI publishes. |
| `nexus policy check` | Validates a policy file, and fails a build on a change that widens what a tenant may reach without an acknowledgment. |
| `nexus doctor` | Checks a deployment: runtime, credentials, database and migrations, Redis, the queue, the registry, health, and settings that only work in one process. |

`nexus help` prints every flag.

## Scanning, models, and optimization

```bash
nexus scan src --json
nexus models --provider deepseek
nexus optimize prompt.txt --model gpt-5.4-mini --max-input-tokens 4000
```

`nexus scan` exits 1 when it finds something, unless you pass `--no-fail`. It never prints a matched
value unless you ask with `--reveal-values`, in an interactive terminal.

## Registry data

```bash
nexus models export --out registry.json
nexus models check registry.json
nexus models refresh --file registry.json --providers openai,anthropic,google
```

`export` writes the bundled registry as a file to keep and update. `check` validates a file, lists
every problem with its path, and exits 1 when there is one; for a valid file it shows what differs
from the bundled data. `refresh` reads each provider's model list with the key in its usual variable,
such as `OPENAI_API_KEY`, and reports models the data lacks and models the provider no longer lists.
A provider without a key is skipped, and `--strict` exits 1 when anything differs. Prices are in no
provider's list, so they stay a check against its pricing page.

## Evaluation

```bash
nexus eval examples/cli-eval.json
nexus eval run eval.mjs --out candidate.json --baseline baseline.json --fail-on-regression
nexus eval gate baseline.json candidate.json
```

- An eval file exports `{ cases, client }` or `{ cases, config }`. JSON cases may use `expected`,
  `contains`, and a `match` of `includes`, `exact`, or `regex`.
- A module for `eval run` exports `{ target, dataset, evaluators }`, and optionally a summary and options.
- `eval gate` exits 1 when a metric got worse beyond noise, a new failure appeared, or the datasets
  differ.

## Traces and the database

```bash
nexus traces list --store runs.jsonl --status error --since 2026-09-20T00:00:00Z
nexus traces show trace-123 --store store.mjs
nexus db sql --adapters operations,traces | psql "$DATABASE_URL"
nexus db status --client db.mjs --check
nexus db migrate --client db.mjs --dry-run
```

`--store` takes a JSONL file, or a module that exports a trace store as `{ store }`, such as a
Postgres store over your own pool. `nexus db sql` prints the schema rather than applying it, so it
needs no database driver and goes through your own migration tooling; `--record` adds the rows that
mark each migration applied.

`nexus db status` and `nexus db migrate` reach the database through a module you write, which exports
your own client, so the CLI still needs no driver:

```js
// db.mjs
import pg from 'pg';
export const client = new pg.Pool({ connectionString: process.env.DATABASE_URL });
export const close = () => client.end();
```

The module may also export `dialect` (`postgres` or `sqlite`, detected from the client otherwise), a
`transaction` function for Postgres (see the [Postgres guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/postgres.md)), and `migrations`, a
list that replaces the bundled one, for stores on custom tables. `--adapters` narrows the bundled list
and `--table` renames the migrations table. `status` prints each migration as applied, pending,
changed, or written by a newer release, and with `--check` exits 1 while one is pending or changed,
which suits a deploy step. `migrate --dry-run` prints the statements it would run.

## Checking a deployment

`nexus doctor` runs every check it can and exits 1 when one fails, or with `--strict` when one warns:

| Check | Fails or warns when |
| --- | --- |
| Node.js | the version is older than 22 |
| Optional packages | never; lists which optional peers are installed |
| Provider credentials | no provider's API key is in the environment (warns; only names are printed, never values) |
| Model registry | the bundled registry is older than its window (warns) |
| Database | it cannot be reached |
| Schema migrations | a migration is pending or changed after it ran |
| Redis | `PING` fails |
| Operation queue | a running operation's lease lapsed, or the oldest queued one waited past `--max-queue-age-ms` (warns) |
| Provider health | a provider is stale or unhealthy (warns) |
| One-process settings | with more than one replica, a store keeps its state in process memory: a memory checkpointer, operation store, server state, or tenant usage fails; a memory rate-limit, circuit, trace, prompt, cache, or rollup store warns |
| Opt-in settings | a store lacks a setting that stays opt-in through 2.x but that a deployment needs: a `RedisOperationStore` without `index: true`, or a `Tracer` writing to a persistent store without `incremental` (warns, with the setting that fixes it) |
| What 3.0 changes | with `--upgrade 3.0`, a store whose behaviour 3.0's defaults change: a tracer that writes at the end, a Redis operation store without its index, or an assistant whose deployment is not yet recorded (warns, saying what to do first and how to keep 2.x's behaviour), when [2.x's defaults](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/core.md#30s-defaults-and-keeping-2xs) are kept; `ok` under 3.0's |

Without `--module` it checks what a shell can see. A module reaches the rest, exporting any of
`database` (or `client`, so a `nexus db` module works unchanged), `redis`, `operations`, `ai`,
`deployment` (`{ replicas, stores }`), and `close`. `--replicas` sets the replica count. Put the
application's tracer among the `stores` for the tracing check: `Tracer.store` and
`Tracer.incremental` are what it reads, and `RedisOperationStore.indexed` for the queue's; with
`--upgrade 3.0`, a `Deployments` among them is read for the assistants it has not yet recorded.

```bash
nexus doctor --module doctor.mjs --strict
```

`diagnose()`, on `nexus-ai-pro/doctor`, runs the same checks from code — at boot, or behind a health
endpoint. `DoctorOptions` takes each input, including a `DoctorDatabase`, a `DoctorDeployment`, and
the release to check readiness for as `upgrade`, and it resolves to a `DoctorReport`: every `DoctorCheck` with its `DoctorStatus`
(`ok`, `warn`, `fail`, or `skip`), detail, and hint, and whether the deployment is `ok`. It never
throws: a check that cannot run is a failure with the reason, and anything heavy is loaded only when
its check runs.

## Deployments

`nexus deploy` talks to a running [agent server](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/server.md). A pipeline uses it to move traffic
onto a new revision after it rolls out the image:

```bash
export NEXUS_SERVER_URL=https://agents.internal NEXUS_SERVER_TOKEN=…
nexus deploy canary support 2026-09-30 10 --reason "build 412"
nexus deploy status support
nexus deploy promote support 2026-09-30
nexus deploy rollback support
```

| Command | What it does |
| --- | --- |
| `deploy status [assistant]` | Every deployment, or one with its revisions' runs and its history. |
| `deploy canary <assistant> <revision> <percent>` | Gives a revision that share of new traffic. |
| `deploy promote <assistant> <revision>` | Makes a revision live, with all the traffic. |
| `deploy rollback <assistant> [--to revision]` | Pulls the canary, or undoes the last promotion. |

`--url` and `--token` override the environment variables. `--reason` is recorded in the history.
`--version` refuses the change when someone moved the deployment first. The server's admin scope
guards every one of these. See the [deployments guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/deployments.md) for what they do.

## Migrating to 2.0

`nexus migrate` moves a 1.x codebase onto 2.0. Most of what changes is where names are imported
from: the 2.0 root keeps only the core client, so a family imported from the root moves to its
subpath.

```bash
npx nexus migrate src            # reports what would change
npx nexus migrate src --write    # rewrites the files
npx nexus migrate src --check    # exits 1 while anything is left
```

It reads TypeScript and JavaScript files, and the code blocks in Markdown. It skips `node_modules`,
build output, and version control.

| It rewrites | It reports, for you to finish |
| --- | --- |
| `import`, `import type`, and `export … from` of the root | A namespace import of the root |
| A destructured `require()` of the root | A dynamic `import()` or a whole `require()` of the root |
| A name that moved under another name, keeping your local name | A read of a response's `estimatedCost` |
| A removed alias, such as `ImageManagerConfig` | An option 2.0 removed, such as `projectId` or `latencyHalfLife` |
| | A model entry that still lists `modalities` |
| | A model name the 2.0 registry dropped, such as `deepseek/deepseek-chat`, with what to use instead |

A second run changes nothing. Comments inside an import list that it splits are not kept. It works on
text, so an import statement inside a string is rewritten too. The
[migration guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/MIGRATING.md) covers what 2.0 and 3.0 change beyond imports. It also renames 3.0's
removed `GraphCheckpointV2`, and reports reads of the OpenAPI tools' `skipped` and a `useDefaults('3.0')`
that 3.0 makes redundant.

## Linting a graph

`nexus graph lint` loads a module and runs `lintGraph()` on the graph it exports: the export named
after `#`, or `graph`, or the default. The export can be a compiled graph or a description saved as
JSON.

```bash
npx nexus graph lint ./dist/billing.js#checkout --deployed
```

It prints each finding and its fix, and exits 1 when a finding is an error. `--strict` fails on
warnings too, `--deployed` treats the in-process checkpointer as an error, and `--ignore` skips codes,
such as `--ignore DYNAMIC_ROUTE`. The [graph guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/graphs.md#linting-a-graph) lists every rule.

## Adapter compatibility records

`nexus adapters verify ./verify.mjs --out records/x.json` runs an adapter's contract suite and
benchmark from its own repository and writes its compatibility record; `nexus adapters page records`
lists records on one page; `nexus adapters recheck records` verifies each listed adapter again against
the installed release and exits 1 when one that passed no longer does; `nexus adapters gather --out
community` fetches the records adapters' own CI publishes, found on npm by their keyword, keeps those
sealed and naming their package, and never runs them. The
[adapter kit guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/adapter-kit.md#compatibility-records) describes the module each takes and the
record each writes.

## Checking a policy

`nexus policy check` validates a policy file, and with `--base`, a file or `git:<ref>`, compares it
with the version it replaces. It exits 1 when the policy is not valid, when a change that widens what
a tenant may reach, or that needs a reviewer, is not acknowledged in the file or with `--accept`, or
when the content changed without a new version.

```bash
npx nexus policy check policy.json --base git:origin/main --accept budgets.tenants.acme
```

The [policy guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/policy.md#reviewed-in-ci) lists what widens, what narrows, and what changes.

## Exit codes

Every command takes `--json`. A usage mistake exits 2, and a failed check or a refused request exits
1, so a CI script can tell them apart. The newer commands load their code on demand, so `nexus scan`
starts no slower.

## Limitations

- The doctor sees only what it is given. A store left out of `deployment.stores` is not checked, and
  it knows a store's kind by its class name, so a store of your own is checked only for what it
  reports.

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/packaging.md

# Packaging and install weight


Every capability has its own entry point, and CI holds each one to a size budget. So an application
pays only for what it imports. This guide lists the entry points and what each one costs.

## Import Surface

The root import holds the core client and nothing else: the client, its config builders, its types,
the errors it throws, and the lifecycle every call runs through. Code written for 1.x imported
families from the root too; `nexus migrate` moves those imports to their subpaths, and the
[migration guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/MIGRATING.md) has the details, for 2.0 and for 3.0.

```ts
import { NexusAI, createNexus, createNexusConfig } from 'nexus-ai-pro';
```

Every family is on a subpath of its own:

```ts
import { NexusAI } from 'nexus-ai-pro/core';
import { createNexusConfig } from 'nexus-ai-pro/config';
import { budgetLedger } from 'nexus-ai-pro/lifecycle';
import { TokenOptimizer } from 'nexus-ai-pro/optimizer';
import { estimateCost } from 'nexus-ai-pro/optimizer/cost';
import { Router } from 'nexus-ai-pro/router';
import { PipelineRunner } from 'nexus-ai-pro/pipeline';
import { MetricsCollector, ProviderHealthMonitor } from 'nexus-ai-pro/ops';
import { withRagContext, completeVerified } from 'nexus-ai-pro/grounding';
import { ingestFilesAfterScan } from 'nexus-ai-pro/rag/files';
import { createFetchUrlTool } from 'nexus-ai-pro/connectors';
import { runProviderConformance } from 'nexus-ai-pro/testing';
import { createNexusRouteHandler } from 'nexus-ai-pro/next';
import { ContextWindowManager } from 'nexus-ai-pro/context';
import { negotiateCompletionRequest } from 'nexus-ai-pro/capabilities';
import { guardrailPolicy } from 'nexus-ai-pro/security';
import { RedisCacheAdapter } from 'nexus-ai-pro/cache';
import { DeepSeekProvider } from 'nexus-ai-pro/providers/deepseek';
import { OperationRunner } from 'nexus-ai-pro/operations';
import { RedisOperationStore } from 'nexus-ai-pro/operations/adapters';
import { verifyOperationWebhook } from 'nexus-ai-pro/operations/webhooks';
import { EmbeddingManager } from 'nexus-ai-pro/embeddings';
import { OpenAIEmbeddingProvider } from 'nexus-ai-pro/embeddings/adapters';
import { MockEmbeddingProvider } from 'nexus-ai-pro/embeddings/mock';
import { KNOWN_EMBEDDING_MODELS } from 'nexus-ai-pro/embeddings/models';
import { ImageManager } from 'nexus-ai-pro/images';
import { MemoryAssetStore } from 'nexus-ai-pro/images/assets';
import { MockImageProvider } from 'nexus-ai-pro/images/mock';
import { OpenAIImageProvider } from 'nexus-ai-pro/images/openai';
import { GoogleImageProvider } from 'nexus-ai-pro/images/google';
import { ComfyUIImageProvider } from 'nexus-ai-pro/images/comfyui';
import { PngMaskTransformer } from 'nexus-ai-pro/images/transform';
import { createImageInputResolver } from 'nexus-ai-pro/images/inputs';
import { createOpenAIVisualModeration } from 'nexus-ai-pro/images/moderation';
import { MediaEvalRunner } from 'nexus-ai-pro/images/evals';
import { createRealtimeSession } from 'nexus-ai-pro/realtime/session';
import { OpenAIWebRTCTransport } from 'nexus-ai-pro/realtime/openai-webrtc';
import { TelephonyManager } from 'nexus-ai-pro/telephony';
import { TwilioTelephonyProvider } from 'nexus-ai-pro/telephony/twilio';
import { createTelephonyRealtimeBridge } from 'nexus-ai-pro/telephony/realtime-bridge';
```

| Fact | Detail |
| --- | --- |
| Required dependencies | None. |
| Provider SDKs | Optional peer dependencies. |
| Validators | `zod`, `ajv`, and `ajv-formats` are optional peers: a zod shape brings its own zod, and only a JSON Schema response format needs ajv. |
| Node's types | `@types/node` is an optional peer. A Node project installs it, as it would for any other code; the client, graphs, and the lifecycle compile without it. |
| Module formats | ESM and CommonJS builds. |
| Node.js | 22 or newer. |
| Tree shaking | Marked `sideEffects: false`. |
| Public surface | Only the entry points in the package's export map. Deep imports into `dist`, `dist-cjs`, or `src` are unsupported. |

### What each import actually costs

Importing one piece loads one piece. The table below measures each entry point's whole import graph.
`npm run size:check` generates it from the build, and fails CI when an entry point grows past its
budget. So the numbers stay true.

The last column is what a size table usually hides: what an entry point forces you to install.

No entry point forces one. The root, `/graph`, `/server`, `/images/*`, and the rest import no
third-party package, and the validators load only when a JSON Schema has to be checked.

A production install is about 5.3 MB of `node_modules`, and all of it is this package. Until 2.0 it
was 12.3 MB, because `zod`, `ajv`, and `@types/node` came with it. The clean-install test holds the
total to a ceiling, and checks that none of the optional peers is installed.

The package itself is most of what remains: an ESM and a CommonJS build, each with its type
declarations, which carry every doc comment to your editor.

| Import | Size | Share of root | Third-party install |
| --- | --- | --- | --- |
| `nexus-ai-pro` | 366 KB | 100% | none |
| `nexus-ai-pro/config` | 359 KB | 98% | none |
| `nexus-ai-pro/core` | 353 KB | 96% | none |
| `nexus-ai-pro/realtime` | 228 KB | 62% | none |
| `nexus-ai-pro/deep-agent` | 213 KB | 58% | none |
| `nexus-ai-pro/server/deployments` | 153 KB | 42% | none |
| `nexus-ai-pro/realtime/openai-sip` | 149 KB | 41% | none |
| `nexus-ai-pro/server` | 126 KB | 34% | none |
| `nexus-ai-pro/batch` | 118 KB | 32% | none |
| `nexus-ai-pro/realtime/session` | 115 KB | 31% | none |
| `nexus-ai-pro/agent` | 114 KB | 31% | none |
| `nexus-ai-pro/protocols/a2a/grpc` | 111 KB | 30% | none |
| `nexus-ai-pro/decisions` | 110 KB | 30% | none |
| `nexus-ai-pro/protocols/a2a` | 96 KB | 26% | none |
| `nexus-ai-pro/providers/groq` | 90 KB | 25% | none |
| `nexus-ai-pro/providers/mistral` | 90 KB | 25% | none |
| `nexus-ai-pro/postgres` | 90 KB | 25% | none |
| `nexus-ai-pro/providers/azure-openai` | 89 KB | 24% | none |
| `nexus-ai-pro/providers/deepseek` | 89 KB | 24% | none |
| `nexus-ai-pro/providers/llamacpp` | 89 KB | 24% | none |
| `nexus-ai-pro/providers/lmstudio` | 89 KB | 24% | none |
| `nexus-ai-pro/providers/openai` | 89 KB | 24% | none |
| `nexus-ai-pro/providers/openrouter` | 89 KB | 24% | none |
| `nexus-ai-pro/evaluate/online` | 89 KB | 24% | none |
| `nexus-ai-pro/policy` | 85 KB | 23% | none |
| `nexus-ai-pro/router` | 83 KB | 23% | none |
| `nexus-ai-pro/graph` | 82 KB | 22% | none |
| `nexus-ai-pro/images` | 81 KB | 22% | none |
| `nexus-ai-pro/providers/anthropic` | 79 KB | 22% | none |
| `nexus-ai-pro/providers/google` | 74 KB | 20% | none |
| `nexus-ai-pro/decisions/typesafe` | 71 KB | 19% | none |
| `nexus-ai-pro/embeddings/batch` | 61 KB | 17% | none |
| `nexus-ai-pro/agent/sandbox/container` | 60 KB | 16% | none |
| `nexus-ai-pro/providers/cohere` | 59 KB | 16% | none |
| `nexus-ai-pro/providers/ollama` | 59 KB | 16% | none |
| `nexus-ai-pro/security` | 59 KB | 16% | none |
| `nexus-ai-pro/adapter-kit` | 59 KB | 16% | none |
| `nexus-ai-pro/operations` | 58 KB | 16% | none |
| `nexus-ai-pro/video` | 58 KB | 16% | none |
| `nexus-ai-pro/evaluate/media` | 54 KB | 15% | none |
| `nexus-ai-pro/models/data` | 54 KB | 15% | none |
| `nexus-ai-pro/mcp` | 53 KB | 14% | none |
| `nexus-ai-pro/evaluate` | 52 KB | 14% | none |
| `nexus-ai-pro/batch/openai` | 52 KB | 14% | none |
| `nexus-ai-pro/batch/anthropic` | 51 KB | 14% | none |
| `nexus-ai-pro/telephony` | 50 KB | 14% | none |
| `nexus-ai-pro/realtime/gemini-live` | 49 KB | 13% | none |
| `nexus-ai-pro/agent/sandbox` | 48 KB | 13% | none |
| `nexus-ai-pro/ops` | 48 KB | 13% | none |
| `nexus-ai-pro/batch/mock` | 46 KB | 13% | none |
| `nexus-ai-pro/realtime/openai-webrtc` | 46 KB | 13% | none |
| `nexus-ai-pro/agent/openapi` | 44 KB | 12% | none |
| `nexus-ai-pro/embeddings` | 43 KB | 12% | none |
| `nexus-ai-pro/sqlite` | 43 KB | 12% | none |
| `nexus-ai-pro/voice` | 42 KB | 11% | none |
| `nexus-ai-pro/tracing` | 40 KB | 11% | none |
| `nexus-ai-pro/optimizer/cost` | 40 KB | 11% | none |
| `nexus-ai-pro/context-hub` | 40 KB | 11% | none |
| `nexus-ai-pro/server/auth` | 39 KB | 11% | none |
| `nexus-ai-pro/agent/workspace` | 38 KB | 10% | none |
| `nexus-ai-pro/tracing/otlp-grpc` | 38 KB | 10% | none |
| `nexus-ai-pro/images/stores` | 38 KB | 10% | none |
| `nexus-ai-pro/models` | 37 KB | 10% | none |
| `nexus-ai-pro/prompts/registry` | 36 KB | 10% | none |
| `nexus-ai-pro/lifecycle` | 35 KB | 10% | none |
| `nexus-ai-pro/mcp/registry` | 34 KB | 9% | none |
| `nexus-ai-pro/protocols/acp/editor` | 34 KB | 9% | none |
| `nexus-ai-pro/media/rag` | 34 KB | 9% | none |
| `nexus-ai-pro/graph/functional` | 33 KB | 9% | none |
| `nexus-ai-pro/agent/middleware` | 33 KB | 9% | none |
| `nexus-ai-pro/evals` | 32 KB | 9% | none |
| `nexus-ai-pro/insights` | 31 KB | 8% | none |
| `nexus-ai-pro/telephony/telnyx` | 31 KB | 8% | none |
| `nexus-ai-pro/tracing/otlp` | 30 KB | 8% | none |
| `nexus-ai-pro/telephony/twilio` | 30 KB | 8% | none |
| `nexus-ai-pro/realtime/openai-websocket` | 29 KB | 8% | none |
| `nexus-ai-pro/images/inputs` | 27 KB | 7% | none |
| `nexus-ai-pro/evaluate/generate` | 27 KB | 7% | none |
| `nexus-ai-pro/server/tenancy` | 27 KB | 7% | none |
| `nexus-ai-pro/loaders/web` | 26 KB | 7% | none |
| `nexus-ai-pro/runtime` | 25 KB | 7% | none |
| `nexus-ai-pro/voice/google` | 25 KB | 7% | none |
| `nexus-ai-pro/images/assets` | 25 KB | 7% | none |
| `nexus-ai-pro/protocols/acp` | 25 KB | 7% | none |
| `nexus-ai-pro/testing` | 25 KB | 7% | none |
| `nexus-ai-pro/video/google` | 24 KB | 7% | none |
| `nexus-ai-pro/realtime/openai-server` | 24 KB | 7% | none |
| `nexus-ai-pro/images/evals` | 23 KB | 6% | none |
| `nexus-ai-pro/media/mistral` | 22 KB | 6% | none |
| `nexus-ai-pro/operations/adapters` | 21 KB | 6% | none |
| `nexus-ai-pro/images/transform` | 20 KB | 5% | none |
| `nexus-ai-pro/media/catalog` | 20 KB | 5% | none |
| `nexus-ai-pro/images/openai` | 19 KB | 5% | none |
| `nexus-ai-pro/embeddings/adapters` | 19 KB | 5% | none |
| `nexus-ai-pro/connectors` | 19 KB | 5% | none |
| `nexus-ai-pro/realtime/conversation` | 19 KB | 5% | none |
| `nexus-ai-pro/workflows` | 19 KB | 5% | none |
| `nexus-ai-pro/evaluate/emulate` | 18 KB | 5% | none |
| `nexus-ai-pro/doctor` | 18 KB | 5% | none |
| `nexus-ai-pro/realtime/mock` | 18 KB | 5% | none |
| `nexus-ai-pro/agent/interpreter` | 17 KB | 5% | none |
| `nexus-ai-pro/rag/retrievers` | 17 KB | 5% | none |
| `nexus-ai-pro/rag/pipeline` | 17 KB | 5% | none |
| `nexus-ai-pro/images/comfyui` | 16 KB | 4% | none |
| `nexus-ai-pro/testing/record` | 16 KB | 4% | none |
| `nexus-ai-pro/embeddings/batch/google` | 15 KB | 4% | none |
| `nexus-ai-pro/sqlite/operations` | 15 KB | 4% | none |
| `nexus-ai-pro/evaluate/automation` | 15 KB | 4% | none |
| `nexus-ai-pro/grounding` | 15 KB | 4% | none |
| `nexus-ai-pro/prompts/client` | 15 KB | 4% | none |
| `nexus-ai-pro/video/runway` | 15 KB | 4% | none |
| `nexus-ai-pro/rag/weaviate` | 15 KB | 4% | none |
| `nexus-ai-pro/context` | 14 KB | 4% | none |
| `nexus-ai-pro/images/google` | 14 KB | 4% | none |
| `nexus-ai-pro/protocols/ag-ui` | 14 KB | 4% | none |
| `nexus-ai-pro/server/regions` | 14 KB | 4% | none |
| `nexus-ai-pro/postgres/operations` | 14 KB | 4% | none |
| `nexus-ai-pro/rag/pinecone` | 14 KB | 4% | none |
| `nexus-ai-pro/voice/session` | 13 KB | 4% | none |
| `nexus-ai-pro/sqlite/vectors` | 13 KB | 4% | none |
| `nexus-ai-pro/residency` | 13 KB | 4% | none |
| `nexus-ai-pro/postgres/store` | 13 KB | 4% | none |
| `nexus-ai-pro/media` | 13 KB | 4% | none |
| `nexus-ai-pro/realtime/tools` | 13 KB | 4% | none |
| `nexus-ai-pro/rag/redis` | 13 KB | 4% | none |
| `nexus-ai-pro/postgres/traces` | 12 KB | 3% | none |
| `nexus-ai-pro/postgres/rollups` | 12 KB | 3% | none |
| `nexus-ai-pro/rag/files` | 12 KB | 3% | none |
| `nexus-ai-pro/tenancy` | 12 KB | 3% | none |
| `nexus-ai-pro/evals/judge` | 12 KB | 3% | none |
| `nexus-ai-pro/providers` | 11 KB | 3% | none |
| `nexus-ai-pro/providers/base` | 11 KB | 3% | none |
| `nexus-ai-pro/optimizer` | 11 KB | 3% | none |
| `nexus-ai-pro/images/mock` | 11 KB | 3% | none |
| `nexus-ai-pro/embeddings/models` | 11 KB | 3% | none |
| `nexus-ai-pro/embeddings/cache` | 11 KB | 3% | none |
| `nexus-ai-pro/sqlite/store` | 11 KB | 3% | none |
| `nexus-ai-pro/rag/qdrant` | 11 KB | 3% | none |
| `nexus-ai-pro/rag/chroma` | 11 KB | 3% | none |
| `nexus-ai-pro/postgres/vectors` | 10 KB | 3% | none |
| `nexus-ai-pro/postgres/migrations` | 10 KB | 3% | none |
| `nexus-ai-pro/prompts` | 10 KB | 3% | none |
| `nexus-ai-pro/postgres/tenancy` | 10 KB | 3% | none |
| `nexus-ai-pro/loaders/json` | 10 KB | 3% | none |
| `nexus-ai-pro/voice/openai` | 9 KB | 2% | none |
| `nexus-ai-pro/images/moderation` | 9 KB | 2% | none |
| `nexus-ai-pro/embeddings/batch/openai` | 9 KB | 2% | none |
| `nexus-ai-pro/sqlite/migrations` | 9 KB | 2% | none |
| `nexus-ai-pro/graph/visualize` | 9 KB | 2% | none |
| `nexus-ai-pro/graph/lint` | 9 KB | 2% | none |
| `nexus-ai-pro/tracing/spool` | 9 KB | 2% | none |
| `nexus-ai-pro/ops/circuit-breaker` | 9 KB | 2% | none |
| `nexus-ai-pro/agent/permissions` | 8 KB | 2% | none |
| `nexus-ai-pro/store/redis` | 8 KB | 2% | none |
| `nexus-ai-pro/credentials` | 8 KB | 2% | none |
| `nexus-ai-pro/evaluate/threads` | 8 KB | 2% | none |
| `nexus-ai-pro/capabilities` | 8 KB | 2% | none |
| `nexus-ai-pro/store` | 7 KB | 2% | none |
| `nexus-ai-pro/ops/rate-limit-adapters` | 7 KB | 2% | none |
| `nexus-ai-pro/loaders` | 7 KB | 2% | none |
| `nexus-ai-pro/loaders/csv` | 7 KB | 2% | none |
| `nexus-ai-pro/rag/elasticsearch` | 7 KB | 2% | none |
| `nexus-ai-pro/embeddings/batch/mock` | 6 KB | 2% | none |
| `nexus-ai-pro/postgres/fulltext` | 6 KB | 2% | none |
| `nexus-ai-pro/postgres/evaluate` | 6 KB | 2% | none |
| `nexus-ai-pro/server/remote` | 6 KB | 2% | none |
| `nexus-ai-pro/postgres/prompts` | 6 KB | 2% | none |
| `nexus-ai-pro/postgres/circuits` | 6 KB | 2% | none |
| `nexus-ai-pro/ops/circuit-store` | 6 KB | 2% | none |
| `nexus-ai-pro/telephony/realtime-bridge` | 6 KB | 2% | none |
| `nexus-ai-pro/loaders/html` | 6 KB | 2% | none |
| `nexus-ai-pro/providers/errors` | 5 KB | 1% | none |
| `nexus-ai-pro/embeddings/mock` | 5 KB | 1% | none |
| `nexus-ai-pro/sqlite/embedding-cache` | 5 KB | 1% | none |
| `nexus-ai-pro/tracing/rollups` | 5 KB | 1% | none |
| `nexus-ai-pro/postgres/embedding-cache` | 5 KB | 1% | none |
| `nexus-ai-pro/prompts/file` | 5 KB | 1% | none |
| `nexus-ai-pro/video/mock` | 5 KB | 1% | none |
| `nexus-ai-pro/cache/semantic-cache` | 5 KB | 1% | none |
| `nexus-ai-pro/rag` | 5 KB | 1% | none |
| `nexus-ai-pro/loaders/text` | 5 KB | 1% | none |
| `nexus-ai-pro/loaders/markdown` | 5 KB | 1% | none |
| `nexus-ai-pro/rag/rerankers` | 5 KB | 1% | none |
| `nexus-ai-pro/pipeline` | 4 KB | 1% | none |
| `nexus-ai-pro/prompts/redis` | 4 KB | 1% | none |
| `nexus-ai-pro/loaders/pdf` | 4 KB | 1% | none |
| `nexus-ai-pro/loaders/git` | 4 KB | 1% | none |
| `nexus-ai-pro/operations/webhooks` | 3 KB | 0.8% | none |
| `nexus-ai-pro/cache/memory-cache` | 3 KB | 0.8% | none |
| `nexus-ai-pro/media/mock` | 2 KB | 0.5% | none |
| `nexus-ai-pro/decisions/mock` | 2 KB | 0.5% | none |
| `nexus-ai-pro/cache` | 2 KB | 0.5% | none |
| `nexus-ai-pro/cache/adapters` | 2 KB | 0.5% | none |
| `nexus-ai-pro/jobs` | 2 KB | 0.5% | none |
| `nexus-ai-pro/jobs/durable-adapters` | 2 KB | 0.5% | none |
| `nexus-ai-pro/jobs/queue` | 2 KB | 0.5% | none |
| `nexus-ai-pro/streaming` | 1 KB | 0.3% | none |
| `nexus-ai-pro/providers/type-guards` | 1 KB | 0.3% | none |
| `nexus-ai-pro/next` | 1 KB | 0.3% | none |
| `nexus-ai-pro/jobs/batch` | 1 KB | 0.3% | none |

A capability that only works by importing the whole runtime is treated as a design problem, not an
acceptable cost.

## Runtime budgets

Speed and memory are held the way import size is. `npm run bench:runtime` times graph creation, a
1,000-node compile, `Send` throughput, checkpoint latency in memory and SQLite, routing over 10, 100,
and 1,000 models, a 20-middleware agent, trace writes, the event stream, server event fan-out,
operation claims, and cache lookups. Each timing is divided by a calibration loop on the same
machine and compared with `runtime-budget.json`, so a budget recorded on one machine holds on a
slower one; the check fails when a ratio grows past three times its budget. The routing claim is
absolute: a warm route over 1,000 models stays under 100 µs at p50 and 1 ms at p99.

Memory is checked by retention: 100,000 checkpoints through one thread, and 10,000 threads run one
after another with a long-lived signal, must each leave the heap within 8 MB of where it started
after a collection. That check found 2.1 keeping about 5 KB per run whenever runs shared a signal.
The release gate runs it; `npm run bench:runtime -- --update` records new budgets after a change
that is meant to cost more. Realtime has budgets of its own, for turn latency and session load, in
`realtime-budget.json`: see [Turn latency and session load](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/realtime.md#turn-latency-and-session-load).

---

Source: https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/runtimes.md

# Runtimes


Most of this package runs on any JavaScript runtime: Node.js, Deno, Bun, edge runtimes such as
Cloudflare Workers and Vercel's, and browsers. That includes:
- the client;
- every provider adapter;
- graphs and agents;
- the agent protocols.

What needs Node.js is what needs a file system, a socket, a child process, or Node's crypto: file
stores, the server, operations, telephony, and the sandbox. The table at the end lists each of
those entry points and exactly what it needs.

## The kernel

`nexus-ai-pro/runtime` gathers the contracts every model call and tool call is built on into one
small import. Nothing it reaches uses `fs`, `net`, `child_process`, Node's `crypto`, or Node's
globals. The check that guards this fails the build if that ever changes. `runtimeInfo()` reports
which runtime the code is on, as a `RuntimeInfo` with a `RuntimeName` and a version.

```ts
import { runtimeInfo, tool, collectStream } from 'nexus-ai-pro/runtime';
import { GoogleProvider } from 'nexus-ai-pro/providers/google';

export default {
  async fetch(request: Request, env: { GEMINI_KEY: string }): Promise<Response> {
    const provider = new GoogleProvider({ apiKey: env.GEMINI_KEY });
    const stream = provider.stream({ model: 'gemini-3.8-flash', messages: [{ role: 'user', content: await request.text() }] });
    return new Response(`${await collectStream(stream)} (${runtimeInfo().name})`);
  },
};
```

The kernel holds these.

**Messages.** A `Message` has a `MessageRole` and content made of `ContentPart` values:
- `TextContent`;
- `ImageContent`, `AudioContent`, and `VideoContent`, whose bytes are a `BinaryBuffer`;
- `DocumentContent`, a document such as a PDF, as bytes, base64 text, a URL, or a path;
- `ToolReferenceContent`, a tool a search found;
- `AssetContent`, for media held in an asset store.

A `CacheHint` marks where a cacheable prefix ends, and `PromptCacheConfig` sets how a request
caches. A `CompletionRequest` carries the messages, the tools, a `ToolChoice`, and a
`ReasoningConfig`.

**Responses.** A `NexusResponse` has content, any `ToolCall` values, and its `ResponseMeta`:
- `TokenUsage`;
- `ResponseCost`;
- the `CacheOutcome`.

A stream is a `NexusStream` of `StreamChunk` values. `collectStream()` reads one to its text, and
`mapStream()` transforms each chunk. `createTextStream()` makes a stream from text, for tests and
cached answers.

**The provider contract.** An adapter extends `BaseProvider`: it states its `ProviderInfo`, then
completes and streams. Each call receives a `ProviderCallContext` with the operation's id, deadline,
trace headers, and idempotency key.

Failures are a `NexusProviderError` with a `NexusProviderErrorCategory`, built from
`NexusProviderErrorOptions`:
- `createProviderHttpError()` turns a failed HTTP response into one;
- `toNexusProviderError()` turns anything thrown into one;
- `categorizeProviderError()` reads the category;
- `isRetryableProviderError()` says whether trying again can help;
- `isAbortError()` recognizes a cancelled request.

**The tool contract.** A `ToolDefinition` names a tool, describes it for the model, and declares
its JSON Schema. `tool()` defines one with typed arguments. When an agent runs it, the tool
receives a `ToolContext`. A tool returns any value, or a `ToolOutput` from `toolOutput()` that
carries images back to the model. `isToolOutput()` recognizes one, and `toolMessageContent()` turns
a result into the message the model reads. `ToolExecutor` runs tools by name outside an agent, and
a `ToolCallResult` pairs a result with the call it answers.

**Capabilities.** A tool declares what it does to the world as capabilities, read by
`parseCapability()` into a `ParsedCapability`. `capabilitiesOf()` resolves the capabilities of a
call. `isSensitiveCapability()` says which need a person's approval by default: anything but a read.

Models have capabilities too. `negotiateCompletionRequest()` fits a request to what the routed
model supports, by `NegotiateOptions`. It returns a `NegotiationResult` with each change it made,
or raises a `NexusCapabilityError` under a strict policy.

## Provider adapters on any runtime

Each provider adapter keeps its own entry point, so an application loads only the providers it
uses. The OpenAI, Anthropic, Google, and Cohere adapters take a `fetch` option. Every request then
goes through it, for:
- a platform's bound fetch;
- a proxy that adds credentials;
- a stub in a test.

The OpenAI and Anthropic adapters call their official SDKs, which run on every runtime. Each SDK
is an optional peer, loaded only when a request is sent.

```ts
import { OpenAIProvider } from 'nexus-ai-pro/providers/openai';

const provider = new OpenAIProvider({
  apiKey: env.OPENAI_KEY,
  fetch: (input, init) => env.EGRESS.fetch(input, init),
});
```

The client, graphs, agents, and the agent protocols need nothing more. A graph's node cache keys
its entries with SHA-256 in plain JavaScript. It uses the runtime's native digest where one is
available synchronously, and the digest is the same either way. Ids come from Web Crypto.

## Proof

The kernel's tests are written against web-standard APIs only, with no test framework and nothing
of Node's. One file, `tests/portable/kernel.mjs`, runs against the built package on four runtimes
and in three browser engines:
- **Node.js and an edge runtime** run on every build. The edge runtime is Vercel's, which exposes
  no `process`, `Buffer`, or `require`, and the tests are bundled first, as a platform's build would
  bundle them.
- **Deno and Bun** run in CI beside it.
- **A browser** runs on every build too. `tests/browser.mjs` serves the build to headless Chromium,
  the browser the machine has or `CHROME_BIN`, with nothing installed for it. The page imports every
  kernel entry point and runs the kernel's tests, then a graph, an agent with a streaming tool against
  a mock model, and the AG-UI adapter. It fails if the page ever sees a Node.js `process`.
- **Firefox and WebKit**, Safari's engine, run the same page in CI beside Deno and Bun, in
  Playwright's builds of each: `npm run test:browser:firefox` and `npm run test:browser:webkit`,
  after `npx playwright-core install firefox webkit`.

On each runtime, the tests check the following:
- tools and capabilities;
- streams;
- a provider over `fetch`, completing and streaming against a stub that splits the stream mid-frame;
- the client routing a request to a provider;
- a graph with a cached node;
- an agent calling a tool;
- an agent answering over A2A and rendering over AG-UI.

`scripts/check-portability.mjs` walks every entry point's imports, static and dynamic, and fails
the build in two cases:
- the client, the kernel, a provider adapter, graphs, agents, or a protocol reaches anything
  Node-only;
- the table below falls out of date.

## Limitations

- **WebKit is Playwright's build, not Safari itself.** It is the engine Safari runs, built for Linux,
  so a difference that lives in Safari's shell rather than its engine is not seen.
- **A provider key in a browser is visible to every user.** Call providers from a server or an edge
  function, or through a proxy that holds the key. Some providers also refuse cross-origin requests.
- **SHA-256 in plain JavaScript is slower on large inputs.** It takes about 7 µs for a typical
  cache key, and about ten times native time for megabytes of image bytes. It is used only where
  the runtime has no synchronous native digest.

## What needs Node.js

The table is generated from the built package's import graph, so it is exact. A runtime with Node
compatibility, such as Deno or Bun, or Workers with Node compatibility enabled, runs these too.


Of 200 entry points, 161 run on any runtime. These need Node.js, or a runtime with Node compatibility, and what they need:

| Entry point | Needs |
| --- | --- |
| `nexus-ai-pro/voice/openai` | the `Buffer` global |
| `nexus-ai-pro/images` | `node:fs`, `node:fs/promises`, `node:path`, `node:stream` |
| `nexus-ai-pro/images/openai` | the `Buffer` global |
| `nexus-ai-pro/images/google` | the `Buffer` global |
| `nexus-ai-pro/images/comfyui` | the `Buffer` global |
| `nexus-ai-pro/images/transform` | `node:zlib`; the `Buffer` global |
| `nexus-ai-pro/images/inputs` | `node:dns/promises`, `node:http`, `node:https`, `node:net`; the `Buffer` global |
| `nexus-ai-pro/images/moderation` | the `Buffer` global |
| `nexus-ai-pro/operations` | `node:crypto`; the `Buffer` global |
| `nexus-ai-pro/operations/webhooks` | `node:crypto`; the `Buffer` global |
| `nexus-ai-pro/agent/sandbox` | the `process` global |
| `nexus-ai-pro/agent/sandbox/container` | the `process` global |
| `nexus-ai-pro/deep-agent` | the `process` global |
| `nexus-ai-pro/mcp` | the `process` global |
| `nexus-ai-pro/mcp/registry` | the `process` global |
| `nexus-ai-pro/protocols/a2a/grpc` | `node:http2`, `node:zlib`; the `Buffer` global |
| `nexus-ai-pro/protocols/acp/editor` | the `process` global |
| `nexus-ai-pro/tracing` | `node:fs/promises`, `node:path`, `node:async_hooks` |
| `nexus-ai-pro/tracing/otlp` | `node:async_hooks` |
| `nexus-ai-pro/tracing/otlp-grpc` | `node:http2`, `node:zlib`, `node:async_hooks`; the `Buffer` global |
| `nexus-ai-pro/evaluate` | `node:fs/promises`, `node:path`; the `process` global |
| `nexus-ai-pro/evaluate/generate` | `node:fs/promises`, `node:path` |
| `nexus-ai-pro/evaluate/online` | `node:crypto`; the `Buffer` global |
| `nexus-ai-pro/insights` | the `process` global |
| `nexus-ai-pro/connectors` | `node:dns/promises`, `node:http`, `node:https`, `node:net`; the `Buffer` global |
| `nexus-ai-pro/server` | `node:crypto`; the `Buffer` global |
| `nexus-ai-pro/server/deployments` | `node:crypto`; the `Buffer` global |
| `nexus-ai-pro/doctor` | `node:module`, `node:path`; the `process` global |
| `nexus-ai-pro/prompts/registry` | `node:crypto`; the `Buffer` global |
| `nexus-ai-pro/prompts/file` | `node:fs/promises`, `node:path`; the `process` global |
| `nexus-ai-pro/testing/record` | `node:fs/promises`, `node:path`; the `Buffer` global, the `process` global |
| `nexus-ai-pro/batch` | `node:crypto`; the `Buffer` global |
| `nexus-ai-pro/images/stores` | `node:fs`, `node:fs/promises`, `node:path`, `node:stream` |
| `nexus-ai-pro/telephony` | the `Buffer` global |
| `nexus-ai-pro/telephony/realtime-bridge` | the `Buffer` global |
| `nexus-ai-pro/telephony/twilio` | `node:crypto`; the `Buffer` global |
| `nexus-ai-pro/loaders/web` | `node:dns/promises`, `node:http`, `node:https`, `node:net`; the `Buffer` global |
| `nexus-ai-pro/loaders/git` | the `process` global |
| `nexus-ai-pro/rag/redis` | the `Buffer` global |

These run anywhere, and load a Node module only when a feature that needs it is used:

| Entry point | Loads, when used |
| --- | --- |
| `nexus-ai-pro/voice/google` | `node:fs/promises` |
| `nexus-ai-pro/images/evals` | `node:zlib`; the `Buffer` global |
| `nexus-ai-pro/embeddings/batch` | `node:crypto`; the `Buffer` global |
| `nexus-ai-pro/agent/workspace` | `node:fs/promises`, `node:path` |
| `nexus-ai-pro/agent/interpreter` | `node:worker_threads` |
| `nexus-ai-pro/video` | `node:crypto`; the `Buffer` global |
| `nexus-ai-pro/loaders/text` | `node:fs/promises`, `node:fs`, `node:path` |
| `nexus-ai-pro/loaders/markdown` | `node:fs/promises`, `node:fs` |
| `nexus-ai-pro/loaders/html` | `node:fs/promises`, `node:fs` |
| `nexus-ai-pro/loaders/csv` | `node:fs/promises`, `node:fs` |
| `nexus-ai-pro/loaders/json` | `node:fs/promises`, `node:fs` |
| `nexus-ai-pro/loaders/pdf` | `node:fs/promises`, `node:fs` |