# Working with Malloy Publisher Publisher is the analytics engine for [Malloy](https://malloydata.dev), created and maintained by [Credible](https://www.credibledata.com). It serves one or more Malloy model packages over a REST API and a single MCP endpoint. If you are an AI agent working in this repo, here is what you can do with it and how to start. **The fast path.** Everything below is detail on these four steps: 1. Start a server (section 1) and poll `GET http://localhost:4000/api/v0/status` until `operationalState` is `"serving"`. 2. Make sure your MCP client is connected to `http://localhost:4040/mcp` (section 2). If you started the server yourself in this session, your Malloy tools will not appear until the user reconnects the client; unattended, use REST (section 7). 3. Call `list_packages` for the environment and package names, then `get_context` scoped to one package with your question (section 4). 4. Run `execute_query` with the names it returned, verbatim. ## What you can do - Discover what data exists: environments, packages, models, sources, and fields, without knowing any names in advance. - Answer plain-English questions by running Malloy queries, which Publisher compiles to SQL and runs against the connected database. - Build and change Malloy models: validate an edit with `compile_model`, save it, then `reload_package` to run it by name. The `malloy-modeling` skill covers the workflow. - Build a data app: a hand-authored HTML page in a package's `public/` directory, backed by that package's models and served by Publisher with no build step. The `malloy-html-data-apps` skill covers it. - Review Malloy for correctness with the `malloy-review` skill. - Measure whether an agent can actually answer questions against a model, with the `eval-loop` skill: it scores answers against verified goldens, reports which entities retrieval delivered, diagnoses each failure and who owns it, and builds a servable report. [`examples/storefront/evals/storefront-tour`](examples/storefront/evals/storefront-tour) is a worked example over the bundled `storefront` package: twelve questions and an answer key, each golden derived twice. Its [README](examples/storefront/evals/storefront-tour/README.md) has the exact commands. The set is the questions and the key; running it, and whatever the run produces, is yours. - Build a report: a `.malloy` file with an `## artifact` tag is a report, a page of query and text tiles behind filter controls with drill click-through, no code and no build step. Its `layout=` is the one choice: `grid` for a dashboard scanned at a glance, `linear` for a story read top to bottom. A report can live anywhere in the package; write a new one at the package root unless asked otherwise, and point `# drill { to="path/without-extension" }` at another report by its path. People who want the classic drag-and-drop feel can build the same file in the Console instead — drag a tile to move it, drag its right edge to set its width — and the two are interchangeable, because the builder splices the file rather than owning a layout document of its own. The `malloy-reports` skill covers it; [docs/reports.md](docs/reports.md) is the reference, and [docs/choosing-a-surface.md](docs/choosing-a-surface.md) says when to pick a report, which layout, or a data app. - Govern access: [givens](docs/givens.md) declare runtime parameters that drive filter widgets, [row-level access](docs/row-level-access.md) decides which rows a caller sees, [`#(authorize)`](docs/authorize.md) decides whether a caller may query a source at all, and [discovery curation](docs/discovery-and-access.md) — a package's `index.malloy` — decides what is visible, and queryable, in the first place. The bundled `governed-analytics` package shows all four together. - Materialize for cost and speed: one `#@ persist` annotation turns an expensive source into a table, rebuilt on demand, from the `malloy-pub` CLI, or on a cron with the opt-in scheduler; `#@ preaggregate` rolls a measure up to a coarse grain. The `malloy-materialization` skill covers it; [docs/materialization.md](docs/materialization.md) and [docs/preaggregation.md](docs/preaggregation.md) are the references. - Model a warehouse from scratch: add a [connection](docs/connections.md) (BigQuery, Snowflake, Postgres, MySQL, SQL Server, Trino, Databricks, MotherDuck, DuckLake, and more), then `search_database_schema` ranks its tables against a plain-English description and hands back the `source:` line for each. DuckDB is built in for CSV, Parquet, JSON, and Excel files. - Browse it all in the Publisher Console, the built-in web UI on the REST port ([docs/console.md](docs/console.md)), where charts and dashboards defined in the model render. All of it runs against a local server you start in step 1 and reach over MCP in step 2, or over REST when you work unattended (section 7). ## 1. Start the server first The MCP tools talk to a running server, so nothing works until it is up. **Requirements.** Node.js 20 or newer for `npx` and for a clone. Building from a clone also needs [Bun](https://bun.sh/) 1.3.13 or newer. The Docker image carries its own runtime and needs neither. The bundled example packages are all DuckDB-backed, so no database credentials are needed for anything in this file. **Package names.** Everything here is scoped: the server is `@malloy-publisher/server`, the scaffolder `@malloy-publisher/create-malloy-package`, and the language itself `@malloydata/malloy`. The unscoped `malloy` on npm is an unrelated logging library, so `npm install malloy` installs nothing you want. The fastest way, with nothing cloned and no Bun installed: ```bash npx @malloy-publisher/server@latest --port 4000 # REST on :4000, MCP on :4040 ``` From a clone of this repo: ```bash bun install bun run build && bun run start # same ports ``` In Docker, with a `publisher.config.json` in the current directory (copy [`packages/server/publisher.config.example.duckdb.json`](packages/server/publisher.config.example.duckdb.json) to serve the same three example packages): ```bash docker run -d -p 4000:4000 -p 4040:4040 \ -v $(pwd)/publisher.config.json:/publisher/publisher.config.json:ro \ ms2data/malloy-publisher ``` Once it is serving, the Publisher Console is at **http://localhost:4000** and the MCP endpoint is at `http://localhost:4040/mcp`. The REST port is `--port` (env `PUBLISHER_PORT`, default 4000) and the MCP port is `--mcp_port` (env `MCP_PORT`, default 4040). The REST port binds `0.0.0.0` by default; the MCP port binds `127.0.0.1` (env `MCP_HOST`, falling back to `PUBLISHER_HOST`). Pass `--host 127.0.0.1` to keep both loopback-only, or set `MCP_HOST=0.0.0.0` to expose MCP on the network. The server is stateless and unauthenticated on both ports and can read any data the models connect to, so keep it on localhost, or put an authenticating gateway in front before exposing it further ([docs/security-posture.md](docs/security-posture.md) lists what it defends against and what it leaves to the gateway). [docs/deployment.md](docs/deployment.md) covers npx, Docker, and Compose in full. To re-initialize the sample storage on a later run, build first and then start with `--init`: `bun run build && bun run start:init`. Start one npx server at a time: concurrent first runs can race in the shared npx cache and corrupt the install ([docs/deployment.md](docs/deployment.md#run-with-npx) has the recovery step). Keep the `@latest`. `npx` resolves through a shared cache and will happily re-run a build it downloaded weeks ago, so a bare `npx @malloy-publisher/server` can serve an old version while looking like a fresh start. To see which build answered, read `version` from `GET /api/v0/status` (or `get_status`); a fixed bug that appears to still be there is the usual first sign of a stale one. On startup the server creates a `.mcp.json` in the directory it was run in, naming the MCP port it bound, which is why a session started in that directory finds the Malloy tools with no registration step. **It does not always create one**, so do not promise a user the file exists without checking: it skips an existing file, git working trees, the home directory, and a few other cases ([the full list](docs/configuration.md#the-mcpjson-the-server-writes)). Read the startup log rather than assuming, and `ls -a` if you need certainty. Whenever it skips, it prints the `claude mcp add` command that connects an agent anyway; use it as printed, because it is deliberately local scope and `-s user` would be shadowed by the very file that caused the message. The file also outlives the server and is never corrected, so a stale one does not merely fail: another process may hold that port and answer from the wrong data. Comparing URLs does not settle that, since two Publishers on one port give the same URL; call `list_packages`, which names what you are actually talking to. Never delete a `.mcp.json` you did not create. `--no-mcp-config` turns it off, and the Docker image sets `PUBLISHER_NO_MCP_CONFIG=1`. Poll until the server reports `serving` rather than assuming a fixed wait. From a clone the sample packages are read straight from `examples/`, so this is usually seconds. A first `npx` run has to download the published package and then fetch the samples from GitHub, which is network-bound and can push it to a minute or two: ```bash curl -s http://localhost:4000/api/v0/status | jq .operationalState # -> "serving" ``` The server also prints one `PUBLISHER_READY` line to stderr at the moment it reaches `serving`, carrying environment, package, and load-error counts, so a script can watch for that line instead of polling; if initialization fails, `PUBLISHER_INIT_FAILED` is printed in its place. A first-run download reports its clone progress on stderr too. Publisher needs Node.js 20 or newer: on anything older it prints `PUBLISHER_UNSUPPORTED_NODE required=>=20 detected=` and exits non-zero without binding a port, so if you are waiting on `PUBLISHER_READY` treat that line as terminal rather than continuing to poll. The only fix is to run Publisher on a newer Node. If you are changing the runtime yourself, switch it and then start the server as a separate command: `mise use -g node@20 && npm start` chained in one shell still runs the old Node, because PATH is not re-evaluated mid-chain, and the second failure looks identical to the first. The full startup-signal contract is in [docs/configuration.md](docs/configuration.md#startup-signals). `serving` does not mean everything loaded. A package that fails to load is skipped, not fatal, so the server serves whatever did load and the package is simply absent. If data you expect is missing, check `curl -s http://localhost:4000/api/v0/status | jq .loadErrors`, which is absent when everything loaded and otherwise names each environment or package that did not load, and why. The same array reports the other failure, the one you cannot see by looking at what is present: an entry with `stale: true` is a package that IS listed and IS answering queries, but whose most recent reload failed to compile, so it answers from the model compiled before that save rather than from the files on disk. Nothing else says so. The package's entry under `environments`, and its own package resource, both read as serving, so join on the package name to tell a current package from a stale one. Fix the model and reload to clear it. ### Starting from your own data To serve the user's data rather than the bundled examples, scaffold a package — in a fresh directory, because the scaffolder writes a workspace (start/reset scripts, an MCP config, agent instructions, the skills) into the current directory as well as the package: ```bash mkdir my-data && cd my-data npm create @malloy-publisher/malloy-package@latest sales -- --data ./orders.csv npm start ``` - Keep the `@latest`; without it npm may reuse a cached, older scaffolder. - The `--` before `--data` is required, or `npm create` swallows the flag and the scaffolder stops. - `--data` takes CSV, Parquet, JSON, newline-delimited JSON, or Excel `.xlsx`, by a path relative to the current directory; the file is copied into the package. Omit it for a small sample dataset. - A seeded package starts as a row count and an overview — the modelling is yours to do next. - With the workspace in place, `npm start` (and a bare `npx @malloy-publisher/server` from that directory) serves this package, not the examples. `npm start` serves it in watch mode, so a saved edit to the model recompiles without a reload (section 6 explains watch mode and its limits). - The workspace also writes a `.mcp.json`, so an agent session started in that directory finds the Malloy tools with no registration step (section 2). Details — caching, workspace layout, the bare `npx` form — are in [docs/scaffolding.md](docs/scaffolding.md). **Connecting a database.** A package is just Malloy, so it is not limited to local files. Add a [connection](docs/connections.md) to the package, point the model at it, and the same workspace serves a warehouse. If the warehouse exists but no model does yet, `search_database_schema` ranks its tables against a plain-English description and returns the `source:` line to start each one from. Ranking works with no API key; the optional embedding-backed mode is in [docs/configuration.md](docs/configuration.md#semantic-ranking-for-search_database_schema). ## 2. Connect your agent Publisher exposes one MCP endpoint: `http://localhost:4040/mcp` (streamable HTTP, stateless, unauthenticated; put it behind a gateway if you expose it beyond localhost). Connect the client after the server is up. An MCP client discovers a server's tools when it connects, so if the client was already running when you started the server (for example you asked the agent to start it), its Malloy tools stay missing until it reconnects. In Claude Code: run `/mcp`, select `malloy`, then choose **Reconnect**. That panel reports `Auth: not authenticated` and offers `Authenticate` as its first option, which does not apply here because the endpoint has no auth; Reconnect is the one that works. Restarting Claude also works. The simplest path is to start the server first, then launch the agent. If you are the agent and you started the server during this session, your Malloy tools will not show up however long you wait: your tool list was fixed when you connected. You cannot reconnect yourself. When a user is present, say so and ask them to run `/mcp`, select `malloy`, and choose Reconnect (the panel offers `Authenticate` first, which is not it), or to restart Claude. Do not quietly fall back to calling the REST API with curl instead: it hides a fixable problem the user can clear in seconds, and it gives up the grounded discovery, compile checks, and reload that the tools exist to provide. Running unattended, with nobody to reconnect you, is the other case: there the REST API is the supported interface, not a workaround. See section 7. There is a third case, and it is the one that costs the most time because it looks exactly like the first: **a project's `.mcp.json` is only discovered from the directory the agent session _started_ in.** A session launched from a parent directory, or from anywhere else, never sees the server, however long it waits and however many times it reconnects. Reconnecting cannot fix it, because the server was never in the client's list to reconnect to. Skills are the near miss here, and they behave differently: `.claude/skills/` is rescanned as the working directory changes, so a session started further up picks them up on its own once work moves into this directory. That asymmetry is worth knowing precisely because the two symptoms look identical from the outside: same "the agent has nothing", different cause, different fix. Tell them apart by what is missing: | Symptom | Cause | Fix | | ------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `malloy` is listed in `/mcp` but disconnected | client connected before the server existed | Reconnect (or relaunch the agent) | | `malloy` is not listed in `/mcp` at all, and `.mcp.json` is here | session started outside this directory | relaunch the agent from here. A session that starts here reads this file, so a user-scoped entry would be shadowed by it; user scope is for sessions started elsewhere | | `malloy` is not listed in `/mcp` at all, and there is no `.mcp.json` here | the server skipped writing one, or the write failed; its startup log says which | run the `claude mcp add` line the server printed, from the directory you start the agent in. Relaunching alone cannot help when there is no config to find, and note a config may exist at the repository root instead of here | | tools present, no skills auto-invoked | session started outside this directory, the same cause as the second row | relaunch the agent from here; skills are rescanned as the working directory changes | The registration that stops the directory mattering, because it is stored per user rather than per project. Use the port this server actually bound, which its startup log prints; `4040` below is only the default: ```bash claude mcp add --transport http malloy http://127.0.0.1:4040/mcp -s user ``` This one is persistent and global: it outlives the session and the server. Undo it with `claude mcp remove malloy -s user`. Prefer the command the server printed, at its default local scope, unless the point is specifically to stop the directory mattering. Note the two symptoms of that one cause do **not** share a fix: the MCP server list is fixed at session boot, so it needs either a relaunch or the user-scoped registration above, while skills need the session actually rooted in the directory. There is no user-scope equivalent for project skills short of symlinking them into `~/.claude/skills/`. Claude Code: this repo ships a project `.mcp.json`, so from a clone Claude Code offers to connect on first run. Approve it once. To add it elsewhere: ```bash claude mcp add --transport http malloy http://localhost:4040/mcp ``` Cursor: add to `.cursor/mcp.json` or global settings: ```json { "mcpServers": { "malloy": { "url": "http://localhost:4040/mcp" } } } ``` Codex: add to `~/.codex/config.toml`: ```toml [mcp_servers.malloy] url = "http://localhost:4040/mcp" ``` stdio-only clients (older Claude Desktop) bridge through mcp-remote: ```json { "mcpServers": { "malloy": { "command": "npx", "args": ["-y", "mcp-remote", "http://localhost:4040/mcp", "--allow-http"] } } } ``` ### The trust gate Claude Code asks, once per directory, whether to trust the folder — and this is a second gate, separate from connecting the server. In a workspace nobody has trusted yet it lists the Malloy tools and then refuses every call, and a `.claude/settings.json` allowlist is discarded rather than merged. Start Claude Code interactively in the directory once and answer the prompt; a headless run is never asked, so it cannot clear the gate. You know it cleared when a query returns data. ## 3. The MCP tools - `get_context`: discovery and grounding, and the tool to reach for. Describe what the question needs as typed `search_targets` (`{target_type, search_text}`, where target_type is `source`, `dimension`, `measure`, `view`, `join` or `dimensional_value`) and name the package in `scopes`. **One call answers**: the response is a `sources` array, best first, with the entities that matched nested inside each source's card, so there is no second call to drill into a source. Omit a target's `search_text` to enumerate that type instead of ranking it; a `source` target with no text is the catalog browse, which pages with `offset`/`next_offset` and returns thin cards (a one-line summary rather than the full doc). `scopes` is required and takes exactly one entry naming an environment and a package, because retrieval is indexed per package; `list_packages` gives you those names. Each card's `source_info.resource_id` holds `environment`, `package`, `model_path` and `source`, which `execute_query` takes as `environmentName`, `packageName`, `modelPath` and `sourceName`. `source_info` also carries that source's doc, its **complete** join list (empty means it declares none, so write the relationship inline instead of probing for one), `givens`, `filter_params`, and `authorize` — the gate expressions in force with the givens each reads, so a source with `authorize` denies the query unless you supply them. A field reached through a join is its own entity named by its full dotted path, with `join_path` naming the traversal and `relationship` saying whether aggregating through it fans out. Use the names it returns verbatim. - `list_packages`: what this server serves — every environment and the packages in each, which is where `get_context`'s `scopes` names come from. It is also the only place a package that FAILED to load appears; a plain listing just omits it, which reads as "does not exist" rather than "is broken". - `execute_query`: run a Malloy query (a named view or query, or ad-hoc code) against a model and get JSON back. - `compile_model`: compile-check Malloy source and get structured diagnostics back (severity, message, model, line and column) without running a query. Use it to validate a model or a change while authoring, instead of firing a throwaway query. Pick the `scope` that matches the change: `append` (default) for a NEW definition, `file` for an EDIT (the source compiles as the file, at true coordinates), `package` for a dry-run of every file as saved — including a what-if edit importers must survive. `append` text is checked against the model's published surface, so it may not declare its own data roots (`import`, `connection.table(...)`, `connection.sql(...)`), declare a `given:` (reading the model's own givens as `$NAME` is fine), or set a `##!` compiler flag; validate a model that declares those at `file` or `package` scope. `append` text that carries a model-level `## artifact` tag is a document, which other viewers run, so it is also refused for `# image`, `# link`, markup in a `# label`, `@env.` in an annotation and the other render-tag checks (docs/reports.md); plain `append` text keeps those tags. `append` text that carries a model-level `## artifact` tag is also read as a report, and the response gains a `document` carrying its `layout` (see [docs/reports.md](docs/reports.md#documents-held-as-text)). - `reload_package`: recompile a package from its on-disk model files so a source or view you added or changed after boot becomes queryable by name, without restarting the server. Use it to close the edit-and-run loop: validate with `compile_model`, save, `reload_package`, then `execute_query` the new view. - `get_status`: the server's health. Its operational state, each environment's loaded packages, and every configured package that failed to load or is serving a stale model. This is the only MCP surface where a load failure is visible on its own, so call it before concluding a package is empty or missing, and after a model edit whose reload you did not run yourself. - `search_malloy_docs`: search the Malloy language documentation when you need syntax. - `search_database_schema`: discover what tables a database connection holds, and find the right ones for a plain-English description. Use it when building a model from a database rather than exploring one that already exists. Each table it returns carries the `source:` line to start from. It returns names and types only: no row value is returned. - `get_skill`: fetch a task guide, or list the guides in force. With a `scopes` entry it includes the guides the package ships in its own `skills/` directory, and a package guide of the same name replaces the bundled one for that package (`origin` says which you are reading). - `get_agent`: list the agents a package declares in `publisher.json`, or fetch one agent's resolved definition (instructions, its own skills, declared schedules, and the `sourceContentSha` and `definitionSha` it came from). **Adopt an agent only when the user names it**: listing is not a reason to apply one, and nothing in `get_context` advertises them. A definition adds a brief and skills to your session and grants no tool or permission; its schedules are declared, not run. The `malloy-run-agent` skill is the recipe, and `GET …/packages/{pkg}/agents/{name}` is the REST form. ## 4. A first run, end to end Ask "what can I explore here?" From a cold start, knowing no names at all: 1. `list_packages`, then pick an environment (the bundled one is `examples`) and a package (the bundled packages are `storefront`, `governed-analytics`, and `html-data-app`). 2. `get_context` with a target per concept your question needs and that environment and package in `scopes`, to get the source, view, and field names. This one call returns each matching source with its own matched fields nested in it, so there is no follow-up call to drill into a source. 3. `execute_query` with those names, to get the answer. Charts and dashboards defined in the model render in the UI at http://localhost:4000. Step 1 is only for a cold start. Once you know the environment and package, whether from your system prompt, a previous call, or the user, go straight to step 2. ## 5. Skills The [`skills/`](skills/) directory holds task-specific guides. They are symlinked into `.claude/skills/`, so Claude Code auto-discovers them, and other hosts can pull the same content as MCP prompts from the endpoint above. Start with `malloy-getting-started`. Use `malloy-modeling` to build or change a model, `malloy-analysis` to explore and answer questions, and `malloy-review` to check Malloy for correctness. Most of these are shared, open-source Malloy skills that Credible also serves, from the published `@malloy-publisher/skills` package; [`skills/README.md`](skills/README.md) explains what is shared, why `credible-*` skills never appear here, and how the bare tool names in shared skills line up with the ones this server registers. ## 6. Iterating on a model (watch mode) For your own fast checks while authoring, use `compile_model`; it validates a change and returns diagnostics with no server restart. When you save a model edit and want its new sources and views queryable by name, call `reload_package` (or `GET /api/v0/environments//packages/?reload=true`), which recompiles just that package from disk in place, with no restart. Publisher compiles each configured package at boot and serves that cached model, so a source you add afterwards is not resolvable by name until the package is reloaded. That closes the edit-and-run loop without watch mode. A reload that fails to compile is safe: your files are left alone, the previously compiled model keeps serving, and the compile errors come back in the response. Reach for `compile_model` on the part you are unsure of, not the whole file. Both tools return the same diagnostics, and a failed reload is non-destructive, so once you are editing a saved file, write-then-reload is as safe as compiling first and one call cheaper. What `compile_model` uniquely buys you is checking something that is _not_ on disk yet — an unfamiliar bit of syntax, a view body you are drafting — so send that fragment on its own. It has to stand alone as top-level Malloy, though: a bare `view:` / `dimension:` / `measure:`, or anything that only parses as a continuation of the model's last statement, is refused with a 400 rather than compiled. Wrap a view body in a top-level `query:`, or a field in a throwaway `source: check is extend { … }`. Pasting a whole redefined source into it is the expensive way to learn what a reload would have told you, and a source the model already declares reports "Cannot redefine" rather than checking anything. If a reload appears to succeed but your edit never takes effect, check that the package name is in the path: `…/packages/?reload=true` reloads, `…/packages?reload=true` is the collection and cannot (it now answers 400; older servers answered 200 with the package list, which reads as success). Spell the value `true` or `false` and nothing else. `?reload=1`, `?reload=yes`, `?reload=TRUE` and a repeated `?reload=true&reload=1` now answer 400 quoting the value back; older servers read every one of them as `false` and answered 200 without recompiling. The same rule holds for the other boolean query params, `dropTables` and `bypass_filters`. Both tools read the copy under `publisher_data///`, which sits in the server root: the directory the server was launched from, unless `--server_root` set another. For an env that is not in watch mode, that is a copy Publisher made of the configured source, so editing the original source directory does nothing until you re-copy it. To tell which mode you are in, `GET /api/v0/watch-mode/status` reports whether watching is enabled and for which environment; a watch-mounted package also shows as a symlink in `publisher_data/`. Editing the `publisher_data/` copy is fine for a quick iteration, but keep the source of truth outside it: nothing there is version-controlled and `--init` wipes the whole tree. Watch mode mounts your own source directory in place, which is the durable way to iterate. Watch mode is a separate, optional thing for a human: it is how someone launches the server so that they and any open browser tab see model edits live. It is a launch-time choice for whoever starts the server, not something to turn on by restarting a server that is already running. To use it, start the server with `--watch-env ` (or `PUBLISHER_WATCH=`), which names an environment whose packages Publisher mounts in place (as symlinks) and watches, so edits to the source recompile. Requirements: - The environment's packages must be LOCAL directories, not `github`, `gcs`, or `s3` URLs. From a clone, the bundled `examples` env is local and therefore watch-eligible. Under `npx` it is not: the published server has no repo to read, so its bundled default fetches the same packages from GitHub. - The in-place mount is set up when the environment is first loaded from config: the first boot on a fresh server root (empty `publisher_data/` storage), or any boot with `--init`. If you previously started the env WITHOUT `--watch-env`, its packages were copied into `publisher_data/` and edits to your source do nothing; run once with both flags together, `--watch-env --init`, to re-mount them in place (`--init` alone re-copies, it does not symlink). You do NOT need `--init` on every boot: once a package is mounted as a symlink it stays one, and later boots keep watching. - Only the first environment in the watch list auto-reloads. A save that fails to compile is skipped: the package keeps serving the model it compiled last, so a change that does not appear is the symptom, and confident numbers from the previous model are the trap. The server does not push that at you, but it does record it: `get_status` (or `GET /api/v0/status`) reports the package as a `loadErrors` entry with `stale: true` and the compile error, `get_context` marks it in the package listing and attaches a note to anything it returns for that package, and both clear on the next save that compiles. Compile-check with `compile_model` first anyway; it is faster feedback than noticing afterwards. ## 7. Working unattended: the REST API If you started the server yourself and there is no user to reconnect your MCP client (a one-shot task, a cloud sandbox), MCP is out of reach for the whole session. Use the REST API on port 4000 instead; discovery, query, compile, reload and schema discovery are all there. For schema discovery the endpoints are `GET /api/v0/environments/{env}/connections`, then `.../connections/{conn}/schemas`, then `.../schemas/{schema}/tables`, with a `.../packages/{pkg}/connections/...` form for the per-package `duckdb` sandbox; you rank the table list yourself, since that ranking is MCP-only. (`search_malloy_docs` and `get_context`'s plain-English ranking stay MCP-only; for syntax, read the bundled [`skills/`](skills/) markdown). Like MCP it is unauthenticated, so keep it on localhost. The playbook with worked examples is [docs/ai-agents.md](docs/ai-agents.md), and the running server serves its complete OpenAPI spec at http://localhost:4000/api-doc.yaml. A complete query against the bundled `storefront` package, to copy from: ```bash curl -s -X POST \ http://localhost:4000/api/v0/environments/examples/packages/storefront/models/storefront.malloy/query \ -H 'content-type: application/json' \ -d '{"query":"run: order_items -> by_category","compactJson":true}' | jq -r .result ``` The same query from Python, with only the standard library. There is no Python SDK on PyPI to install: `packages/python-client` is generated from the OpenAPI spec but not published, and an unrelated `malloy-publisher-client` there is a third-party project. ```python import json import urllib.error import urllib.request URL = ("http://localhost:4000/api/v0/environments/examples/packages/storefront" "/models/storefront.malloy/query") def query(malloy: str) -> list[dict]: body = json.dumps({"query": malloy, "compactJson": True}).encode() req = urllib.request.Request( URL, data=body, headers={"content-type": "application/json"}) try: with urllib.request.urlopen(req, timeout=60) as resp: payload = json.load(resp) except urllib.error.HTTPError as err: # The body carries the Malloy diagnostics; print it, don't swallow it. raise SystemExit(f"{err.code}: {err.read().decode()}") # `result` is a JSON string, not an object: parse it a second time. return json.loads(payload["result"]) for row in query("run: order_items -> by_category"): print(row) ``` The map: - `GET /api/v0/status`: poll until `operationalState` is `"serving"`, then check `loadErrors` (absent when everything loaded, and the REST equivalent of `get_status`). Re-check `loadErrors` after every edit-and-reload: an entry with `stale: true` names a package that is still answering, from the model it compiled before your last save. An `emptyReason` means the server found no config at startup (or the `--config` path was missing) and is serving nothing; it names the path it checked. `version` is the server release. - `GET /api/v0/environments`: the environment names every other path needs (the bundled one is `examples`). - `GET /api/v0/environments/{env}/packages`, then `…/packages/{pkg}/models`: what exists. - `GET …/models/{path}`: the discovery step. The response's `sources` (each with its `views`), `queries`, and `givens` are the names you can run. Use them verbatim; never guess. - `POST …/models/{path}/query`: run a query. The body is either `{"query": "run: …"}` (ad-hoc) or `{"queryName": "…", "sourceName": "…"}` (a named view; `queryName` alone runs a model-level named query). Add `"compactJson": true` and parse the `result` string to get plain row objects. - `POST …/models/{path}/compile`: body `{"source": "…", "scope": "append" | "file" | "package"}`; structured diagnostics without running anything. `"append"` (default) validates NEW definitions against the model (text carrying a model-level `## artifact` tag is a document and is also refused for render tags that make a value a URL or markup, `@env.` and the like); `"file"` compiles the source AS the file, which is how to validate an edit (append collides with "Cannot redefine"); `"package"` (source optional) runs reload's worker compiler over every `.malloy` and `.malloynb` file without touching the served model, plus the render-tag and dashboard checks a reload runs after it (not the package's other warnings: storage, persist, materialization, explores, surface and discovery warnings, among others). Its diagnostics may name files hidden from discovery; a replacement path that does not exactly match a file is warned and treated as new. - `GET …/packages/{pkg}/reports`, then `…/reports/{path}`: a package's reports (every `.malloy` file with an `## artifact` tag, wherever it lives, plus legacy `.malloynb`), each with its `path` and `layout` (`grid` or `linear`), and one report's manifest. The `path` is a report's identity; two reports may share a basename in different directories. Every tile carries a `kind`: `query` (it has a `query`) or `text` (it has `markdown`, a `name`, and no `query`). There is no run endpoint for a report: run its queries through `…/models/{path}/query` with `givens`, the same governed path every other query takes, and skip the `kind` `text` tiles, which have nothing to run. The two query fields take DIFFERENT request shapes and are not interchangeable: a single-query report's manifest `query` is a NAME, so send `{"queryName": ""}`; a tile's `query` is an expression, so send `{"query": "run: "}`. Use the report's `path` as the model in both. The older `…/dashboards` and `…/notebooks` routes are deprecated aliases for the same reports; do not build on them. - `PUT …/models/{path}`: body `{"source": "…", "expectedHash": ""}`; write a report into the package and reload it in place. Omit `expectedHash` to **create** a report at any `.malloy` path that does not exist yet (put it at the package root unless asked otherwise; missing directories are created, never under `public/`, never at the root `index.malloy`), which answers 201 and refuses with 409 if the file already exists. With `expectedHash` it **replaces** only a file the package already serves as an editable report, so it never overwrites a model, an untagged file, a `.malloynb` or `index.malloy`; a file that changed since you read it answers 409 and nothing is merged. The text must carry an `## artifact` tag, always with `layout=`; it is compiled first and refused with its problems when it does not compile, and the previous text is restored if the reload does not take it. 403 under `frozenConfig`. [docs/reports.md](docs/reports.md#the-write-path) has the full rule. - `GET …/packages/{pkg}?reload=true`: recompile a package after editing its files (the REST form of `reload_package`). - `POST /api/v0/environments/{env}/packages` with `{"name": "…", "location": "/absolute/path"}`: serve a package of your own on a running server. [docs/packages.md](docs/packages.md) is the package format. Reading this file without a clone? Every doc referenced here resolves at `https://raw.githubusercontent.com/malloydata/publisher/main/`, for example `docs/ai-agents.md`. ## 8. Going deeper - [`docs/`](docs/) is the reference hub, see its [index](docs/README.md). Start with [docs/ai-agents.md](docs/ai-agents.md) for per-client MCP config and the MCP tool reference. Then, by task: [docs/packages.md](docs/packages.md) for the package format (`publisher.json`, models, data); [docs/connections.md](docs/connections.md) for databases; [docs/reports.md](docs/reports.md), [docs/html-data-apps.md](docs/html-data-apps.md), and [docs/choosing-a-surface.md](docs/choosing-a-surface.md) for surfaces; [docs/givens.md](docs/givens.md), [docs/row-level-access.md](docs/row-level-access.md), [docs/authorize.md](docs/authorize.md), and [docs/discovery-and-access.md](docs/discovery-and-access.md) for governance; [docs/materialization.md](docs/materialization.md) and [docs/preaggregation.md](docs/preaggregation.md) for cost and speed; [docs/configuration.md](docs/configuration.md) for every flag and env var; [docs/deployment.md](docs/deployment.md) for Docker and Compose; [docs/architecture.md](docs/architecture.md) and [docs/api-overview.md](docs/api-overview.md) for how it fits together. - [`examples/`](examples/) holds the three served packages: [`storefront`](examples/storefront) (ecommerce model + reports), [`governed-analytics`](examples/governed-analytics) (givens, authorize, row-level access), and [`html-data-app`](examples/html-data-app) (a no-build HTML dashboard). [`data-app`](examples/data-app) is a standalone React SDK app, not a served package.