# Caveman Typescript middleware

Native framework adapters for the Caveman compression runtime. Your framework keeps its inference client, tools, retries, streams, and original conversation. Caveman projects eligible tool-result text into a copied outbound request. Inference stays with your provider.

This quickstart targets middleware `1.0.1` and SDK `1.2.0`. Read the [release notes and limitations](https://github.com/JuliusBrussee/caveman/blob/main/packages/middleware/typescript/CHANGELOG.md) before upgrading.

## Run a complete example

Follow the [AI SDK quickstart](https://docs.caveman.so/integrations/frameworks) for a fresh environment and [runtime installation](https://docs.caveman.so/reference/faq). Start the local runtime separately; the client package does not include it.

```sh
npm install --save-exact @caveman-ai/sdk@1.2.0 @caveman-ai/middleware@1.0.1 ai@7.0.94 @ai-sdk/provider@4.0.11 zod@4.4.3
```

The framework guide above is the runnable walkthrough. A separate downloadable quickstart script is not published on the docs site.

## Choose an integration

- [Framework guide](https://docs.caveman.so/integrations/frameworks): public entrypoints, native APIs, recovery ownership, transports, and limitations.
- [Compatibility matrix](https://docs.caveman.so/integrations/frameworks): resolver ranges versus accepted ranges versus exact validation evidence. A range is not an exhaustive test result.
- [Deployment](https://docs.caveman.so/reference/faq): process/container lifecycle, remote TLS/authentication, persistence, session affinity, deadlines, and rollback.
- [Recovery and scope](https://github.com/JuliusBrussee/caveman/blob/main/docs/technical/middleware-protocol.md): namespace/session/branch/cache epoch, exact originals, excerpts, and expiry.
- [Troubleshooting](https://docs.caveman.so/reference/troubleshooting): final reason codes and strict readiness versus normal inference fallback.
- [Measurement](https://docs.caveman.so/reference/faq): quality, latency, retries, recovery calls, cache effects, and provider usage.

## Entry points: which ones compress

Compression needs a recovery tool the adapter registered itself, so the model can always fetch an exact original. Entry points that cannot register one only record: in `compress` mode they pass content through unchanged, report `recovery_unbound`, and log once which entry point to use instead.

| Adapter | Tier | Compresses | Records only (`recovery_unbound` in compress mode) |
|---|---|---|---|
| `ai-sdk` | certified | `withCaveman` | `createCavemanMiddleware` (`wrapLanguageModel`) |
| `openai` | certified | `withCavemanOpenAITools`, `chat.completions.runTools` on `withCavemanOpenAI` | plain `create` calls on `withCavemanOpenAI` |
| `anthropic` | certified | `beta.messages.toolRunner` on `withCavemanAnthropic` | `messages.create` / `messages.stream` |
| `langchain` | certified | `withCavemanAgent`, `createCavemanLangChain` | `withCavemanModel`, `CavemanChatModel` |
| `google` | experimental | `CavemanGoogleGenAI` with callable tools (automatic function calling) | calls without callable tools |
| `strands` | experimental | `withCavemanStrands` | `withCavemanStrandsModel` |
| `mastra` | experimental | `withCavemanMastra` | `createCavemanMastraProcessor` |
| `mcp` | experimental | `CavemanMCPHost` with `register()`ed tools | |

Certified adapters are gated by the conformance suite. Experimental ones get the same fail-open guard, logging and tests, but may change in a minor release. `CavemanGoogleGenAI` hooks the Google SDK's internal `ApiClient`; if that moves, the client falls back to the native one (`adapter_error`). `CavemanDocumentCompressor` (LangChain RAG) compresses only with a `sourceExpansion` reader. `@caveman-ai/middleware/langchain` loads `langchain`; if you only wrap a chat model or compress documents, import `withCavemanModel`, `CavemanChatModel`, `CavemanDocumentCompressor` and `scopeFromConfig` from `@caveman-ai/middleware/langchain-model` instead, which needs only `@langchain/core`. Every compressing entry point disables recovery if your tools already include one named `caveman_retrieve`; calls then report `recovery_name_conflict`.

## Framework versions

The package declares its frameworks as optional peers with no version range (`*`), so strict resolvers such as Yarn PnP let it import them. Ranged peers made a plain `npm install` fail with `ERESOLVE` (0.1.0-alpha.1), so the version check happens at run time instead, against the copy the adapter actually runs: for `openai` and `@anthropic-ai/sdk`, the version of the client you pass in; for every other framework, the copy this package resolves, whatever the working directory. When your application resolves a different copy of a framework than this package does, the gate still reads this package's copy and logs one warning naming both versions.

In a workspace (npm, pnpm, Yarn), `@caveman-ai/middleware` can be hoisted to the repository root next to an older copy of a framework that another package pulled in (`ai@6`, say) while your app uses its own `ai@7`. The adapter then runs `ai@6` and passes through with `unsupported_version`. Make both resolve one copy: find the other one (`npm ls ai`, `pnpm why ai`), then dedupe it (`npm dedupe`, `pnpm dedupe`) or pin one version for the repository (`overrides` in npm, `pnpm.overrides`). Under pnpm with `hoist=false`, add the frameworks you use to `public-hoist-pattern`. `@strands-agents/sdk` itself declares peers on `openai` 6, `@ai-sdk/provider` 3 and `@anthropic-ai/sdk` 0.109, so npm needs `--legacy-peer-deps` to install it next to newer versions of those.

`inspectFrameworkCompatibility(adapter)` from `@caveman-ai/middleware/compatibility` reports the version of the copy this package resolves, supported range, tested releases and tier without importing a framework (it has no client, so for `openai` and `anthropic` it can differ from what the wrapped client reports). `package.json` lists the same data in `supportedFrameworkVersions` and `testedFrameworkVersions`.

- **Out of range** (including every prerelease, such as `7.1.0-canary.3`): calls pass through unchanged, reporting `unsupported_version`, with one warning. After testing that version, set `acceptFrameworkVersion: true` in the adapter options.
- **Unreadable** (a deploy bundle with no `node_modules`): the adapter checks that the framework hooks it needs exist and runs, with one `version_unverified` warning. If a hook is missing it passes through with `version_unavailable`.
- Nothing throws when you wrap a client. With `strict: true`, the version decision surfaces from `runtime.ready()` / `preflight()`.

## Scope per request

Every adapter takes `scope` as a value or a function. A function is called for each request, inside that request's async context, so one module-level client can serve many users, and `runtime.deleteSession(scope)` removes exactly one user's originals:

```ts
const client = withCavemanOpenAI(new OpenAI(), { runtime, fetch, scope: () => ({ namespace: 'app', session_id: currentUser().id }) });
```

LangChain passes the `RunnableConfig` (`config => scopeFromConfig(config, 'app')` reads `configurable.thread_id`), Mastra its `RequestContext`, Strands the agent. Any ID text works: values outside `[A-Za-z0-9._:/-]` (emails, `user 42 / chat #7`) become a stable `h-` hash. A missing scope (no `thread_id`) runs the call recovery-free with `recovery_unbound`. An unusable one reports `invalid_scope`. Neither throws, except `invalid_scope` in strict mode.

## Budgets and logs

A large history never skips the whole call. Each tool result over the runtime's `segment_bytes` is skipped on its own (`payload_budget`), and images or bytes enter the context manifest as hashes. `manifestBytes` (default 2 MiB) bounds how much history is hashed, and `wireBytes` (default 16 MiB, OpenAI and Anthropic) bounds the request body the adapter parses. OpenAI Responses turns pass through with `provider_state_retained` unless `store: false`, because OpenAI would store the compressed turn; `allowStoredResponses: true` opts in.

Each pass-through reason logs once per process through `console.warn`, as `adapter=<id> reason=<code>`, never content, scope values or credentials. Any exception inside adapter code becomes a pass-through with `adapter_error`; with `strict: true` the call raises `MiddlewareError('adapter_error')` instead.

## Runtimes and module systems

Node.js 22.12 or later, ESM `import` or CommonJS `require()`. TypeScript resolves the types with `moduleResolution` `bundler`, `nodenext`, `node16` or `node10`. Edge runtimes (Cloudflare `workerd`, Vercel `edge-light`) are not supported: adapters rely on `node:async_hooks` call ownership, and importing one under those conditions throws an explanatory error. Run the adapter in a Node.js function instead.

## Contracts to keep

Keep original stored history. Register the actual recovery executor through the native helper; a tool schema alone does not attest recovery. Handles are scope-bound and expire according to runtime retention. A recovery the runtime refuses (unknown or expired handle, runtime down) answers the model `{"error": "<code>"}` (an `isError` result in MCP) and warns once, so the tool loop keeps running; caller cancellation still propagates. Recoverability does not guarantee model quality.

Client modes are `off`, `record`, and `compress`. The client defaults to compression; the standalone runtime defaults to recording. Set both deliberately. Runtime unavailability normally retains original inference input. Strict mode, startup `ready()`, cancellation, and requested recovery failures have different error contracts.

Final decision reports contain status, reason, transform IDs, replacement/reuse counts, and call IDs. **They contain no token counters.** Local segment estimates are inferred; provider usage and billed savings are separate evidence. Nothing in this local example verifies billing savings.

Close the runtime client and native framework/provider resources at shutdown. Closing the client does not stop the runtime process. See the deployment guide before sharing a runtime across workers or tenants.

## Licence and support

The client, adapters, and Engine runtime are all Apache-2.0. Read [LICENSING.md](https://github.com/JuliusBrussee/caveman/blob/main/LICENSING.md). This package is separate from Caveman Agent SDK. File sanitized reproducible issues in [Caveman](https://github.com/JuliusBrussee/caveman/issues).