# AGENTS.md

## Project Overview

Tako MCP Server — an MCP (Model Context Protocol) server that provides AI agents access to Tako's knowledge base of 100K+ data visualizations.

The implementation is a **single Cloudflare Workers TypeScript server** (`workers/`) deployed at `mcp.tako.com` (prod) and `mcp.staging.tako.com` (staging). This is the canonical Tako MCP — all tool work lands here.

## Build & Run

### Hosted Worker (TypeScript)

```bash
cd workers

# Install
npm ci

# Local dev (in-memory, no Cloudflare deploy)
npm run dev

# Deploy (requires CF auth)
npm run deploy:staging
npm run deploy:production

# Verification
npm run typecheck
npm test
npm run registry:check     # validate registry/server.json + _registry.ts have no drift
SMOKE_BASE_URL=https://mcp.staging.tako.com TAKO_SMOKE_API_TOKEN=... npm run smoke
```

## Architecture

### Hosted (Cloudflare Workers) — `workers/`

- **Framework**: `@modelcontextprotocol/sdk` with `StreamableHTTPServerTransport`
- **Runtime**: Cloudflare Workers (TypeScript, `nodejs_compat` flag)
- **Endpoints**: two path-selected MCP surfaces — `POST /mcp` (generic, anonymous tier allowed) and `POST /mcp/chatgpt` (OAuth-only, the surface submitted to OpenAI) — plus `GET /health`. The request path picks the surface; nothing reads the client User-Agent (`workers/src/surface.ts`).
- **Auth**: `Authorization: Bearer <TAKO_API_TOKEN>` extracted at request boundary, forwarded to Django as `X-API-Key`; OAuth 2.1 flow for Claude.ai / ChatGPT
- **Tool registry**: auto-generated from `workers/src/tools/*.ts` via `workers/scripts/gen-registry.ts`; outputs `workers/src/tools/_registry.ts`, `registry/server.json`, `registry/lhm.plugin.json`, `docs/TOOLS.md` and `workers/src/generated/skills.ts` in lockstep (CI checks all five for drift)
- **Skills**: `/mcp` serves the `skills/` directory through the MCP Skills extension (SEP-2640): `skills/list`, `skills/get` and `skill://<name>/SKILL.md` resources. A connection gets only the skills whose every named tool it registers, so a default `/mcp` gets `tako-search`. A connection declares the extension only when it serves a skill, and declares `resources` when it serves a skill or a widget tool. `/mcp/chatgpt` serves no skills; ChatGPT gets them from the plugin zip (`chatgpt-plugin/build.sh`).
- **CI**: `.github/workflows/workers-ci.yml` (typecheck + tests on PRs), `workers-deploy.yml` (staging on push to `main`; production on published GitHub Release, gated by the `production` environment; manual `workflow_dispatch` for either env), `workers-smoke.yml` (auto-smoke after successful staging deploys)

### Key Files

| File | Purpose |
|------|---------|
| `workers/src/index.ts` | Worker entrypoint — resolves the MCP surface from the request path, routes `/health` |
| `workers/src/surface.ts` | `MCP_PATHS` + `surfaceForPath` — the only thing that decides which surface a request gets |
| `workers/src/mcp_session.ts` | Reads MCP Apps support from `initialize`, and mints and classifies the `ui-` and `noui-` `Mcp-Session-Id` that decides widget or PNG on `/mcp` |
| `workers/src/tools/_surface.ts` | Per-surface tool membership (`isToolOnSurface`) and annotation overrides |
| `workers/src/tools/_input_pins.ts` | `?sources=` URL pins: parsing, the 400 message, and how each tool's schema, input, and description change on a pinned connection |
| `workers/src/mcp.ts` | MCP server wrapper, tool dispatch, `djangoErrorToToolResult` |
| `workers/src/django.ts` | Typed HTTP client with `DjangoError` hierarchy |
| `workers/src/auth.ts` | Bearer token extraction |
| `workers/src/tools/*.ts` | One file per tool (`ToolModule` contract from `types.ts`) |
| `workers/src/tools/_registry.ts` | Auto-generated barrel — DO NOT edit by hand; run `npm run registry:gen` |
| `workers/src/skills.ts` | Skills extension on `/mcp`: `skillsForTools` filter, `skills/list`, `skills/get` and `skill://` resources |
| `workers/src/generated/skills.ts` | Auto-generated skill records (text, frontmatter, digest, size, named tools) — DO NOT edit by hand; run `npm run registry:gen` after you change `skills/` |
| `workers/scripts/gen-registry.ts` | Codegen for `_registry.ts`, `registry/server.json` and `src/generated/skills.ts` (through `gen-skills.ts`) |
| `workers/scripts/smoke.ts` | Post-deploy MCP smoke test (TAKO-2611) |
| `workers/wrangler.jsonc` | Cloudflare deploy config (per-env names, custom-domain routes, env vars) |
| `registry/server.json` | Public MCP registry discovery card (auto-generated from Workers tools) |

### Tools

Source of truth: `workers/src/tools/*.ts`. Tools are discovered at runtime via the MCP `tools/list` handshake.

Two path-selected surfaces, no User-Agent sniffing (`workers/src/surface.ts`): `/mcp` is the generic surface every client shares (anonymous tier allowed; charts render in the MCP Apps widget for a client whose `initialize` declares MCP Apps support, and as an inline PNG for any other client. The Worker mints a `ui-` or `noui-` `Mcp-Session-Id` at `initialize`, so a client that doesn't echo that header on later calls gets the PNG, and the `[mcp] session` log line records each request's session kind); `/mcp/chatgpt` is the OAuth-only surface submitted to OpenAI's app directory (MCP Apps widget, top-level `securitySchemes`, Apps-review annotation overrides via `annotationsBySurface`). Membership per surface: `workers/src/tools/_surface.ts` — two constant name sets plus `resolveToolSet(surface, requested)`. `?tools=` is an allowlist of tool names that REPLACES the generic default listing (`workers/src/tools/_tools_param.ts`; the `tako_` prefix is optional) and is ignored on the chatgpt surface, whose listing is fixed. `?sources=` pins a tool input for the whole `/mcp` connection (`workers/src/tools/_input_pins.ts`): a tool opts in through `ToolModule.inputPins`, the pinned input leaves its schema, and every call sends the pinned value. A tool's `describeForPins` gives its description on a pinned connection, including tools that pin nothing but name a source the pin removes. An invalid value is an HTTP 400 before auth; `/mcp/chatgpt` ignores the param. The tool LISTING never varies by auth state; anonymous EXECUTION is gated at dispatch in `mcp.ts` to `FREE_TIER_TOOL_NAMES` (`tako_search` alone). No tool declares an `anonymousInputRejects` gate any more: `tako_search`'s `include_contents` was the only one, and rows moved to `tako_contents`, which is auth-required outright.

1. `tako_search` — Fast retrieval of a **list of structured cards** (every card renders as a chart in the widget on `/mcp/chatgpt` and for a `/mcp` client that declares MCP Apps support; any other `/mcp` client gets the first card as a PNG). Four parameters — `query`, `sources`, `country_code`, `locale`, with `sources` dropped on a `?sources=` connection — and no defaults of its own, so an omitted field takes the v3 API's. It finds results; `tako_contents` fetches them. Reach for it when fanning out queries in parallel to see what exists, or when the chart is the deliverable. Default.
2. `tako_contents` — Fetch what's behind a result url in full: a web page's text, or an exportable card's data rows as one projected `rows` shape. Eight inputs on `/mcp` — `urls` (required), `max_rows`, `max_chars`, `query`, `delivery`, `format`, `offset` and `premium` — and six on `/mcp/chatgpt`, which has no `delivery` or `format` and returns rows inline only. `delivery: "link"` returns a file that expires 5 minutes after the call (CSV for `rows`, JSON for `card`), and is the path to analyze rows in code. `format: "card"` returns the card's own structure. Requires a signed-in connection. Default.
3. `tako_available_data` — Find **what proprietary, structured data exists** on an entity or metric, for questions about what Tako covers. Lists the available metrics in one call. Each match carries its canonical name — search on that name, since `tako_search` takes no `node_ids` pin — plus an `id` for `tako_graph_related` traversal. Ambiguous `q` returns the top entity and top metric as candidates; `metric` is the substring browse filter; `limit` (max 20) widens the list. Default.
4. `tako_graph_related` — Explore a node's relations (metrics, entities, `rel:competes_with`, `part_of`, `members`, sources) as a map or one paged relation. The map previews each group's first names only; a drilled item carries `id`, `name` and `kind`. `q` is a substring filter. Drill into a node `tako_available_data` already resolved. Default.
5. `tako_search_advanced` — The WHOLE v3 `SearchRequest` body plus `include_answer` and `output_schema`, derived from `workers/src/generated/schemas.ts` so the set cannot drift (parity is tested, not asserted in a header). `include_answer: true` runs `POST /api/v1/answer` instead of `/api/v3/search`: one synthesized, citation-backed answer with the retrieval as its citations. Every level is `.strict()`, and `mcp.ts` registers the full schema object, so an unknown key is a -32602 on the wire. It is the only tool that takes a `node_ids` pin, so the license-gated values path (METRIC node id alone + `strict: true` + `include_answer` + `data.include_contents`) lives here. Opt-in via `?tools=search_advanced`.
6. `tako_agent` — Answer Agent for multi-step data questions. Start-or-resume: a call takes `query` (start) or `run_id` (resume), exactly one. It waits up to 45 s (`AGENT_CALL_WINDOW_MS`) and returns the finished run, or `status: "running"` with an `answer_run_…` `run_id` to send back. Neither agent tool cancels a run; an aborted call leaves it going. Both channels carry the projected run: `answer`, `citations`, the `cards` it built, and the definitions/assumptions/methodology maps. On `/mcp`, agent `tools/call` requests answer over SSE, so `notifications/progress` reach a client that sends a `progressToken`; every other request, and every request on `/mcp/chatgpt`, stays buffered JSON. On `/mcp/chatgpt`, and on `/mcp` for a client that declares MCP Apps support, every card renders in the chart widget. Opt-in on the generic surface; always listed on the chatgpt surface.
7. `tako_company_agent` — Company Agent for lists of companies or people: companies by location, industry, status, founding date, funding stage, or size, and the officers, directors, owners, or employees of those companies. Same start-or-resume contract, 45 s window, SSE progress, and chart widget as `tako_agent`, with a `company_run_…` `run_id`. It sends `cards: true` and leaves `locale`, `timezone`, and `output_settings` unset. Opt-in via `?tools=company_agent` on the generic surface; always listed on the chatgpt surface and in `chatgpt-app-submission.json`.
8. `tako_visualize` — Create a card from your own structured data. The one tool here that WRITES: the card is public and permanent, and anyone with the returned url can open it without signing in. Opt-in on `/mcp`, listed by default on `/mcp/chatgpt`. On `/mcp/chatgpt`, and on `/mcp` for a client that declares MCP Apps support, the card renders in the chart widget.

9. `tako_company_search` and `tako_people_search` — Screen companies or people by filters through `POST /api/v1/companies/search` and `POST /api/v1/people/search`. Each input is the generated request schema without `include_contents`, `max_rows` and `content_format`: the tools never inline rows, so the model fetches them with `tako_contents` on the card's url, where rows and premium cells are billed. Each returns the response's own fields plus the list card projected to title, description (a markdown table of the first 10 rows), url, `exportable`, `total_rows` and source. Opt-in via `?tools=contents,company_search,people_search`; never on `/mcp/chatgpt`.
10. `tako_company_enrich` and `tako_person_enrich` — Match up to 25 known companies or people through `POST /api/v1/companies/enrich` and `POST /api/v1/people/enrich`. Each result carries its `index` and the item as sent, so it lines up with the input. Opt-in via `?tools=contents,company_enrich,person_enrich`; never on `/mcp/chatgpt`.

Each enrichment tool description lists the card's columns and marks the premium ones. The lists in `workers/src/tools/_enrich_fields.ts` come from the enums the spec publishes (`CompanyListField`, `PersonListField` and their preview, card-only and premium subsets), so a spec sync that adds or drops a field changes the descriptions. Only the preview table's labels are written by hand, and typecheck fails until each preview field has one. `workers/scripts/smoke.ts` also fails when the API's columns differ from the lists. `?tools=` replaces the defaults, so every example names `contents` beside these tools: without `tako_contents` a connection gets the card and no way to read its rows. `_enrich.ts` adds a 400's field, code and suggestions to the tool result, and `llms-full.txt` carries each description's field sentences verbatim, which `registry:check` enforces.

**Enrichment field lists: finishing a spec sync that changes one.** When a sync adds or drops a list field, the field sentences of the four descriptions change, and `llms-full.txt` must carry them verbatim. `registry:gen` and `registry:check` fail until it does, so the bot's `auto/sdk-sync` PR carries no regenerated code, including unrelated `schemas.ts` changes. To finish it, from `workers/`:

1. Check out `auto/sdk-sync` and run `npm run schemas:gen`.
2. If the field is a preview field, add its label to `COMPANY_PREVIEW_LABELS` or `PERSON_PREVIEW_LABELS` in `_enrich_fields.ts`; `npm run typecheck` names the missing one. No other field needs a hand edit there.
3. Run `npm run registry:gen`. It fails and prints, for each tool, the sentences its `llms-full.txt` section must contain. Replace each section's old field sentences in the root `llms-full.txt` with them.
4. Run `npm run registry:gen` again. All four enrichment descriptions sit a few characters under the 1,000-character cap, so a new field in either list usually fails `[prose-budget]` here: shorten that tool's own prose in its module (`tako_company_search.ts`, `tako_company_enrich.ts`, `tako_people_search.ts`, `tako_person_enrich.ts`), not the field sentences.
5. Run `npm run registry:check`, and commit the regenerated files with `llms-full.txt`.

On `/mcp`, items 5-10 are opt-in: name the tool in `?tools=` (an allowlist that replaces the defaults; see `workers/src/tools/_tools_param.ts` and `_surface.ts`). `/mcp/chatgpt` lists items 6-8 by default. The generated reference is `docs/TOOLS.md`; `registry:check` fails when it is stale.

### Endpoints

#### Workers (`mcp.tako.com`)

| Path | Method | Description |
|------|--------|-------------|
| `/mcp` | POST | MCP JSON-RPC over streamable HTTP — the generic surface every client shares. Also serves the skills its tools support through the Skills extension |
| `/mcp/chatgpt` | POST | Same protocol, ChatGPT app surface. OAuth required: an anonymous request gets 401 + `WWW-Authenticate`, never the free tier |
| `/health` | GET | Simple `200 ok` |

## Code Conventions

### Workers (`workers/`)

- TypeScript 5+, ESM, Node 22 (matches Cloudflare Workers runtime)
- Strict TS — `strict`, `noUncheckedIndexedAccess`, `exactOptionalPropertyTypes`
- One file per tool under `workers/src/tools/`; each exports a `ToolModule` (see `types.ts`)
- Vitest for unit tests, `@cloudflare/vitest-pool-workers` for Worker-context tests
- Never edit `workers/src/tools/_registry.ts` or `registry/server.json` by hand — run `npm run registry:gen`
- OpenAI rescans `/mcp/chatgpt` daily and updates each tool after its automated checks pass. Until a changed tool passes, ChatGPT keeps its earlier definition. So a production deploy, not a resubmission, is what moves a tool's name, description, or schema in ChatGPT. `chatgpt-app-snapshot.json` is the sha256 parity record of that surface as last ACCEPTED here; `registry:check` compares the live tools against it, and `npm run registry:gen -- --accept-chatgpt-snapshot` is the only thing that moves it. Accept a change only when you mean it to reach ChatGPT at the next production deploy. Keep a changed input schema compatible with the approved one until the new definition is live. `chatgpt-app-submission.json` holds each tool's annotation justifications and the test cases for OpenAI's review; `chatgpt-plugin/build.sh` writes the test cases into the package. A `chatgpt-plugin/plugin.json` change reaches the listing only through a new package upload; see the README. `chatgpt-plugin/mcp.json` declares the one connected server, `https://mcp.tako.com/mcp/chatgpt`. An upload can't change that URL, so keep `/mcp/chatgpt` in `MCP_PATHS`; a new URL goes through OpenAI support.
- Errors: handlers throw `DjangoError`; `mcp.ts` catches and maps via `djangoErrorToToolResult`

## PR Guidelines

- Run `cd workers && npm run typecheck && npm test && npm run registry:check` before submitting
- New tools: add a `workers/src/tools/<name>.ts` exporting a `ToolModule`, then `npm run registry:gen` to refresh the registry
- Smoke locally: `SMOKE_BASE_URL=https://mcp.staging.tako.com TAKO_SMOKE_API_TOKEN=... npm run smoke`
- Keep tool descriptions agent-optimized: lead with what the tool DOES in one
  line, then a `Best for:` line naming when to reach for it. (Not "Use this
  when..." — no tool has led with that phrasing for a long time, and the
  generated `docs/TOOLS.md` is the place to check what the model actually reads.)
- Before you change a bundled skill, the server instructions, a tool
  description, or what a result says about cost, read
  `docs/WRITING_FOR_MODELS.md`. The skill, not the descriptions, decides
  whether a model calls Tako, and prices stay out of descriptions.
- Never commit API keys or tokens

## Safety Rules

- Never commit API keys, tokens, or credentials
- Never modify production infrastructure configs without an explicit ticket
- All tool functions must validate inputs before making API calls
- Error responses must never expose internal URLs or stack traces

## Releases

Releases are automated with [release-please](https://github.com/googleapis/release-please).

- **Conventional Commits are required.** Merges are **squash-only** and the squash
  commit uses the **PR title**, which `pr-title-lint` validates. Use `feat:` (minor),
  `fix:` (patch), `chore:`/`docs:`/`refactor:`/etc. Below 1.0, breaking changes bump
  the minor, not the major.
- **Do not hand-edit the version.** release-please owns it across `version.txt` (the anchor) and the seven tracked files:
  `server.json`, `agent.json`, `workers/package.json`, `workers/src/mcp.ts`
  (`SERVER_VERSION`), `registry/metadata.json`, `registry/server.json`, and
  `registry/smithery.yaml`. (`registry/server.json` is generated from
  `registry/metadata.json`; release-please bumps its `version` directly so
  `npm run registry:check` stays green.)
- **How a release flows:** release-please keeps a "release: X.Y.Z" PR open on `main`.
  Merging it bumps the versions + `CHANGELOG.md`, tags `vX.Y.Z`, and publishes a
  GitHub Release. That merge also: publishes to the MCP registry (`publish-mcp.yml`,
  on the `server.json` bump) and deploys the Worker to **staging**
  (`workers-deploy.yml`). The GitHub Release then triggers the **production** deploy,
  which waits for one approval on the `production` environment.