# RFC-0037: cloudhal.dev, the developer site: docs from the repo, a link contract, docs for agents - Status: Accepted (2026-10-21, by Halil; ADR-0039). Open questions decided as recommended: all ADRs and RFCs are published with their status shown, error codes live in one registry in `packages/core-contracts`, and the docs MCP waits for a registry. - Amendment (2026-10-21, phase 3): no docs tools in `cloudhal mcp`; the standalone `cloudhal.dev/mcp` is the only docs MCP, with a build-time index instead of Pagefind's. See §4 and Implementation notes. - Author: Halil (Madraka), drafted with Claude · Date: 2026-10-21 - Related: - ADR-0037 (domains): `cloudhal.dev` is the developer domain; `cloudhal.dev/docs` is the only copy of the docs; `/schemas/v1/*` is permanent. - #33 (the stopgap now live: schemas, `/fetch`, `/llms.txt`, `/docs` redirects to GitHub, all from `apps/docs`). - RFC-0004 §5.7 (the fetcher page), RFC-0019 and RFC-0029 (site search), ADR-0011 (MCP tools and transports). - Prior art: Cloudflare's developer docs ([docs for agents](https://developers.cloudflare.com/docs-for-agents/): `/index.md` per page, `Accept: text/markdown`, `/llms.txt` and `/llms-full.txt` per product), Sanity and Strands (`.md` per page, `llms.txt`), Microsoft Learn's docs MCP server, Astro Starlight with `starlight-llms-txt`. ## Summary cloudhal.dev becomes an Astro Starlight site in `apps/docs`, built from the Markdown in the monorepo's `docs/` folder and deployed as the `cloudhal-dev` Worker that already serves the schemas. Every URL it publishes is a contract: it never breaks, moves get a redirect, and CI checks every link that Cloudhal's code, packages and sites point at. Every page is also published for agents, as Markdown at `.md` and through content negotiation, indexed in `/llms.txt`, and searchable through a read-only MCP endpoint at `cloudhal.dev/mcp`. ## Motivation - **Links already point here.** The 13 npm packages' `homepage`, seven links on cloudhal.com, 89 schema `$id`s, the fetcher's User-Agent and the playground Worker's documented `deployUrl` all name cloudhal.dev URLs. Until #33 none of them resolved. Today `/docs/*` lands on GitHub's file view, which is fine for contributors but poor for users: no navigation, no search, no Markdown for agents. - **Agents read docs before people do.** Cloudhal's users are agents as much as people. Agents need small, structured Markdown, a stable index and a way to search, not HTML pages. Cloudflare, Sanity and Microsoft already serve docs this way; Cloudhal claims to be agent-native and should do no less. - **The docs must not drift from the code.** Today guides live next to the code and change in the same PR. Any docs site that moves them into a CMS or a separate repo loses that, which is how docs go stale. - **Why now:** 0.3 is public, the Deploy button and the playground send people to `/docs`, and the reference (CLI flags, MCP tools, `cloudhal.json`) exists only as source code. ## Proposal ### 1. Stack and source - `apps/docs` is an Astro 7 + Starlight site. Content comes from the monorepo's `docs/` through an Astro content-collection `glob` loader rooted at `../../docs`. Nothing is copied by hand; `docs/` stays the single source. - The stopgap's `scripts/build.ts` keeps producing `/schemas/v1/*`, `/fetch` and the `_headers`/`_redirects` files. Starlight renders everything else into the same `dist/`. - Reference pages are **generated** at build time, never written by hand: - CLI: from `packages/cli/src/cli/help.ts`; - MCP tools: from the server's tool list (name, description, input schema, policy class); - `cloudhal.json` and the contracts: from the JSON Schemas; - changelog: from the packages' `CHANGELOG.md` (one page; the packages share one version). - Cloudhal's own product site, cloudhal.com, stays a Cloudhal site. cloudhal.dev does not dogfood Cloudhal, because docs must change in the same PR as code (alternatives below). ### 2. Information architecture | Path | Content | Source | |---|---|---| | `/` | What Cloudhal is, install, links | `apps/docs` | | `/docs/start/*` | Install, first site, connect an agent, deploy | `docs/start/` (new) | | `/docs/guides/*` | Task guides (deploy button, search, schedules, comments, multi-site, translations, hardening) | `docs/guides/` | | `/docs/concepts/*` | Site graph, snapshots, policy and approvals, surfaces, publishing | `docs/concepts/` (new; drawn from the ADRs and RFCs) | | `/docs/reference/cli`, `/mcp`, `/config`, `/schemas`, `/api` | Generated reference | §1 | | `/docs/adr/*`, `/docs/rfcs/*` | Decisions and proposals, rendered as they are | `docs/adr/`, `docs/rfcs/` | | `/changelog` | Release notes | `CHANGELOG.md` | | `/schemas/v1/*` | Permanent JSON Schemas | `packages/core-contracts/schemas` | | `/fetch` | The fetcher page | `apps/docs/static` | | `/llms.txt`, `/llms-full.txt`, `/docs/
/llms.txt` | Indexes for agents | generated | | `/mcp` | Read-only docs MCP endpoint (§4) | Worker | `/docs/deploy` (used by the playground) stays as a permanent alias of `/docs/start/deploy`. ### 3. The link contract Invariants: 1. A URL that cloudhal.dev has served with a 200 never returns a 404. A moved page gets a `301` in `apps/docs/redirects.ts`, which the build turns into `_redirects`. 2. `/schemas/v1/*` is never moved or deleted (ADR-0037). A breaking schema change adds `/schemas/v2/`. 3. Code, packages and sites link only to cloudhal.dev URLs, never to GitHub file paths, for anything a user reads: CLI errors and help, Studio help links, package `homepage`, the template's README, cloudhal.com and the playground. GitHub links are for source and issues. 4. CLI and API errors that need an explanation link to `https://cloudhal.dev/e/`, a stable page per error code (like Rust's `E0277` or React's error decoder). The first codes are the ones users have actually hit: `deploy_token_invalid`, `setup_token_missing`, the Workers Builds name override, D1 migration failures. 5. CI checks the contract: - every cloudhal.dev URL in the repo (code, READMEs, fixtures excepted) resolves in the built site; - every URL in the previous deploy's sitemap still resolves or redirects; - internal links in `docs/` resolve. Who links where: | From | To | |---|---| | cloudhal.com (nav, footer, "Docs") | `cloudhal.dev/docs` | | npm `homepage` (all packages) | `cloudhal.dev`; `repository` and `bugs` stay on GitHub | | GitHub README | `cloudhal.dev/docs` for users, `CONTRIBUTING.md` for contributors | | CLI help and errors | `cloudhal.dev/docs/reference/cli#`, `cloudhal.dev/e/` | | Studio help icons | `cloudhal.dev/docs/guides/*` | | try.cloudhal.com | `cloudhal.dev/docs/deploy` and the template | ### 4. Docs for agents - **Markdown per page.** Every page is also built as Markdown at `.md` (Sanity's convention) and `/index.md` (Cloudflare's). The `.md` keeps frontmatter `title`, `description` and `source` (the file in the repo). - **Content negotiation.** A small Worker script in front of the assets answers `Accept: text/markdown` on any page with its Markdown, sets `Vary: Accept`, and adds `x-markdown-tokens` (an estimate), as Cloudflare does. Everything else is served by the assets binding unchanged. - **Indexes.** `/llms.txt` lists every page with its one-line description, grouped by section. `/llms-full.txt` is the whole docs in one file. Each section has its own `llms.txt` and `llms-full.txt`. - **Docs MCP.** `https://cloudhal.dev/mcp` is a stateless, read-only MCP endpoint with no authentication. It has two tools: - `search_docs(query, section?)` returns up to 8 hits with title, URL and snippet, from a build-time index; - `read_doc(url)` returns one page's Markdown. *Amended 2026-10-21:* `cloudhal mcp` does **not** gain these tools. The site MCP server is for agents working on sites (content, design, generated UI, comments, schedules), and its how-to knowledge lives in skills, tool descriptions and server instructions. Docs tools there would add tools that narrow-scoped keys never need, a second knowledge source that can contradict the first, and an outbound dependency on cloudhal.dev. Microsoft Learn, AWS, Mintlify and GitBook ship docs as a standalone, unauthenticated MCP server; Supabase and Sanity put docs tools in their product server, but those servers are developer tools, while Cloudhal's is content-operational and policy-governed. Developer agents connect `cloudhal.dev/mcp` (or read `llms.txt`); the site server's instructions name `llms.txt` and nothing more. - **AGENTS.md for users.** `/docs/agents/docs-for-agents` explains how to point an agent at Cloudhal: the MCP server, the docs MCP, the `llms.txt`. The page is written for agents to read. ### 5. Search, versions, languages - Search: Pagefind (Starlight's default; static, no service) for people. The docs MCP has its own index, built from the same page list (Implementation notes). - Versions: while Cloudhal is 0.x, only the latest release is documented, with an "applies to 0.3.x" note in the header taken from the packages' version. At 1.0, older majors move to `/docs/v1/` and so on. - Languages: English only (the default language rule). Starlight's i18n and the Weblate setup (`docs/guides/translations-weblate.md`) can add languages later without URL changes, under `/docs//`. ### 6. Build and deploy - CI builds `apps/docs` on every PR that touches `docs/**`, `apps/docs/**`, the schemas, `help.ts` or the MCP tool list, and runs the link checks (§3.5). - Every PR gets a preview: `wrangler versions upload` with a preview alias, posted as a PR comment. - A push to `main` deploys `cloudhal-dev`. This needs a Cloudflare API token, limited to Workers Scripts edit on this one Worker, stored as a GitHub Actions secret. Since #236 this is `.github/workflows/docs-deploy.yml`, with the token in the `docs-deploy` environment. - `cloudhal.dev` sends no mail. DNS gets `v=spf1 -all`, DMARC `p=reject` and a null MX (ADR-0037). ### 7. Phases 1. Starlight in `apps/docs` over the existing `docs/`, with the IA above, Pagefind, `.md` per page, `llms.txt`/`llms-full.txt`, and the redirects that replace today's GitHub 302s. 2. Generated reference (CLI, MCP tools, config, schemas, changelog), `/e/` pages, link checks in CI, deploy from CI with PR previews. 3. Content negotiation Worker and `https://cloudhal.dev/mcp` (no docs tools in `cloudhal mcp`, amended). 4. New content: `docs/start/*` and `docs/concepts/*`. Then point the CLI, Studio, cloudhal.com and the template at the new URLs. ## Alternatives considered | Option | Pros | Cons | Why not | |---|---|---|---| | cloudhal.dev as a Cloudhal site (dogfood) | Shows the product; Studio editing | Docs leave the repo, so they stop changing in the same PR as code; needs a "Markdown from git" import feature first | Drift is the main way docs fail. cloudhal.com already dogfoods | | Mintlify or GitBook (hosted) | Fast, polished, built-in AI features | Vendor lock-in, a paid plan for a custom domain at our size, docs not served from our Worker or our schemas' domain | Contradicts open source and self-hosted | | Docusaurus | Mature, versioning built in | React and Webpack/Rspack, a second stack next to Astro | Starlight is Astro, which the renderer and starters already use | | Docs on GitHub only (today's stopgap) | Zero work | No search, no navigation, no Markdown endpoints, no stable URLs we control | Fine for a week, not for users | | Docs in a separate repo | Clean history | Code and docs change in different PRs | Same drift problem | ## Compatibility & migration - No contract or `schemaVersion` change. The schema URLs stay as they are. - Today's `/docs/*` 302s to GitHub become 301s to the new pages, once phase 1 ships. `/docs/deploy` and `/docs/guides/` keep working throughout. - Moving files inside `docs/` from now on needs a redirect entry; CI enforces it (§3.5). - Rollback: the Worker's previous version (`wrangler rollback`). The stopgap stays deployable from `scripts/build.ts` alone. ## Security & policy impact - The docs MCP endpoint is read-only, unauthenticated and stateless. It is rate limited per IP with a Workers rate-limiting binding. It reads only the built docs, never a site. - The site MCP server sends nothing to cloudhal.dev (amended: no docs tools there). - The CI deploy token can edit only the `cloudhal-dev` Worker. - No cookies, no analytics beyond Cloudflare's aggregate Web Analytics, no third-party scripts. ## Open questions 1. ADRs and RFCs on the public site: render all of them, or only Accepted ones, with drafts staying on GitHub? Recommendation: all, with the status shown in the header. 2. Error codes: one registry in `packages/core-contracts` (shared by TS and Rust), or per package? Recommendation: one registry, since the API, CLI and Studio report the same codes. 3. Should the docs MCP also be offered as an MCP registry entry (`registry.cloudhal.dev`, ADR-0037), or wait until there is a registry? Recommendation: wait. ## Implementation notes - **Phase 1** (`apps/docs`): a custom content loader (`src/lib/repo-docs.ts`) reads `docs/` and the site's own pages (`src/lib/site-pages.ts`). The H1 becomes the title; the description is the first text under `## Summary` or `## Context`; relative links become site paths, or GitHub links for files outside `docs/`. Folders without a README get a generated index (`/docs/rfcs`). `scripts/postbuild.ts` writes the schemas, the `.md` copies, `llms.txt`/`llms-full.txt` (site and per section) and `_redirects` from `src/lib/redirects.ts` from the same page list. The `starlight-llms-txt` plugin is not used: one list feeds the HTML, the Markdown and the indexes, so they cannot disagree. - **Phase 3** (`apps/docs/src/worker`, amended 2026-10-21): the `cloudhal-dev` Worker now has a script, run before the assets (`run_worker_first`). `/mcp` is the docs MCP: Streamable HTTP, stateless (no `Mcp-Session-Id`, no event stream; GET and DELETE answer 405), one JSON answer per POST, protocol versions 2024-11-05 to 2025-11-25, tools `search_docs` and `read_doc` (read-only annotations), 100 requests a minute per IP through the `MCP_RATE_LIMITER` binding. `scripts/postbuild.ts` writes the search index (`/_mcp/search-index.json`: plain text and an inverted index with title, description and path boosts, from the same page list as the HTML) and the Worker ranks with BM25 plus prefix matches. Pagefind's index is built for the browser (fragments fetched on demand), so it was not reused. `read_doc` takes only cloudhal.dev URLs or site paths and answers the page's `.md` copy. `Accept: text/markdown` on a page answers that copy with `Vary: Accept` and `x-markdown-tokens` (characters / 4); every other request goes to the assets binding unchanged, and page HTML gains `Vary: Accept`.