--- vcs: system: github remote: https://github.com/christianclaudio/mcp-server-sigma owner: christianclaudio repo: mcp-server-sigma default_branch: main branch_policy: pr_only merge_method: squash delete_branch_on_merge: true --- # AGENTS.md Instructions for AI coding agents (Antigravity, Claude Code, Copilot, Cursor, Windsurf) working on this repository or integrating Sigma Computing cloud analytics capabilities. --- ## 🎯 Project Overview This is `mcp-server-sigma` — an enterprise Python Model Context Protocol (MCP) server for **Sigma Computing**, built on its public REST API (v2 and v3alpha); `scripts/check_openapi_drift.py` lists the spec endpoints the client does not cover. Bulk-destructive tools are listed in `full` and refused at call time unless enabled; the expected per-profile tool counts live in `scripts/check_tool_contract.py`. **Primary Purpose**: Expose deep cloud business intelligence, embedded analytics, workbook lineage, SQL data modeling, user/team administration, and tenant token exchange to AI agents with strict enterprise safety gates, offline testing, and multi-tenant token isolation. --- ## 🏗️ Key Paths - `src/sigma_mcp/server.py` — root gateway: `create_server` mounts the domain sub-servers (`workbooks`, `datasets`, `elements`, `workspace`, `admin`) with matching namespaces, then applies the job allowlist, the read-only filter, and (on `full` only) Tool Search or Code Mode; prompts, resources. - `src/sigma_mcp/profiles.py` — `PROFILES` (`full`, `readonly`, and the job profiles `analyst`, `author`, `modeler`, `embed`, `access_admin`), their tool allowlists, `FULL_ONLY_TOOLS`, `BULK_DESTRUCTIVE_TOOLS`, and the `readOnlyHint` filter. - `src/sigma_mcp/tools/.py` — domain sub-servers (`admin`, `datasets`, `elements`, `workbooks`, `workspace`); `tools/common.py` holds the `ANNOTATION_*` constants and shared helpers. - `src/sigma_mcp/client.py` — async HTTP client (`SigmaClient`, `_encode_segment()`, RFC 8693 token exchange). `SSRFSafeAsyncTransport` resolves each request's host once (the ASCII/IDNA name, off the event loop), checks every address, and connects only to a checked one through a pinned httpcore network backend, trying the next checked address if a connect fails while the request's connect timeout lasts (one deadline shared by all the addresses, not a fresh timeout for each); the URL keeps the hostname for the Host header, TLS SNI, certificate verification and per-hostname connection pooling. A host with no checked address and Unix sockets are refused. The API client, the documentation fetches and `scripts/check_openapi_drift.py` all use it. httpx ignores env proxies whenever a transport is passed (`allow_env_proxies = trust_env and transport is None`), so connections always go direct through the pin; egress-proxy deployments need a proxy-aware transport, which isn't provided; `trust_env=False` on the drift fetch is belt-and-braces. `errors.py` — exceptions and secret redaction. `auth.py` — `SharedTokenVerifier`, the FastMCP `TokenVerifier` that `create_server()` attaches at build whenever `SIGMA_MCP_AUTH_TOKEN` is set (stripped, non-blank), so `main()`, `fastmcp run` and `http_app()` all enforce it (constant-time compare; stdio ignores it); `main()` also attaches it at serve time if the env was set after import, and refuses (exit 2) a tokenless HTTP bind to a host other than `127.0.0.1`, `::1` or `localhost` unless `SIGMA_MCP_ALLOW_UNAUTHENTICATED_BIND=1`. `webhooks.py` — HMAC verification. `middleware.py` — read-only gate (`readOnlyHint` only, unwraps the Tool Search `call_tool` proxy) and the admin bulk-destructive gate. `config.py` — settings. - `scripts/check_tool_contract.py` — source of truth for expected tool counts and annotations. `README.md` and four test files repeat some of these counts (`SIGNED_OFF` in `tests/test_profiles.py`, the 263 in `tests/test_unit.py`, `tests/test_enterprise_assertion.py` and `tests/test_layered.py`), so change them together; do not add new copies. - `scripts/check_openapi_drift.py`, `scripts/write_ops_check.py` (live create-and-teardown against a Sigma org; needs `SIGMA_CLIENT_ID`, `SIGMA_CLIENT_SECRET`, and `SIGMA_API_BASE_URL`), `scripts/check_conformance.sh` + `conformance-baseline.yml`. - `scripts/release_notes.py` — release body from squash commits since the previous `v*` tag. `scripts/check_version.py` — runs after the build and reads the version from the single wheel in `dist/` (the file that ships, as release.yml's tag check does); fails on `0.0.0` (no git metadata) or `0.0.1.devN` (no reachable tag, a shallow checkout). - `tests/` — unit tests are offline; live network tests live in `tests/test_e2e_live.py` (marked `pytest.mark.e2e`, deselected by default pytest `addopts` `-m 'not e2e'`, run with `-m e2e`). That test sets no skip and no env check; tool calls use `get_client`, which requires `SIGMA_CLIENT_ID` and `SIGMA_CLIENT_SECRET`. The unmarked `test_dispatch_tool_call_offline` in that file stays in the default suite. `tests/test_integration_live.py` skips unless `SIGMA_LIVE_TESTS=1` and `SIGMA_CLIENT_ID` are set. Security-hardening, webhook, and protocol tests are `tests/test_security_hardening.py`, `tests/test_webhooks.py`, and `tests/test_protocol.py`. Profile, read-only, and composition tests are `tests/test_profiles.py` and `tests/test_layered.py`. Version and release-tooling tests are `tests/test_version.py`, `tests/test_release_notes.py`, and `tests/test_check_version.py`. - `.github/workflows/` — `ci.yml`, `release.yml` (on a `v*` tag: build with full history, check the wheel version matches the tag, build the release notes, publish to PyPI, create the GitHub Release from `scripts/release_notes.py`, then push the Docker image only after the publish job succeeds, then stamp the tag version into `server.json` and publish to the MCP Registry in a last `registry` job that needs the Docker job), `dependabot-automerge.yml` (squash auto-merge only for Dependabot PRs whose highest update is minor or patch; major updates wait for a human review). - `server.json` (MCP Registry metadata), `Dockerfile`, `pyproject.toml`. --- ## ⚡ The Canonical Workflow: Building Tools from API / llm.txt When translating an API documentation page or OpenAPI specification into an MCP tool, follow these 4 steps in exact order: ### 1. Client Method (`client.py`) - Implement a dedicated `async def` method on `SigmaClient`. - Type all arguments strictly. Never use bare `dict` or `Any` when a concrete schema or literal is known. - URL path parameters are encoded per segment by `_encode_segment()` inside `_request()`, preserving colons on custom methods (e.g. `{id}:materialize`) while eliminating path traversal vulnerabilities (`..` $\rightarrow$ `%2E%2E`). Pass raw IDs in the path; do not pre-quote them, or they are encoded twice. - Call `await self._request("METHOD", path, params=..., json_data=...)`. - For a `multipart/form-data` endpoint, pass `files={"field": (filename, content, content_type)}` instead of `json_data`; `_request` then drops the JSON `Content-Type` so httpx sets the multipart boundary (`add_dbt_metadata` is the one user today). ### 2. Tool Handler (`tools/.py`) - Register the tool on its domain sub-server with `@_server.tool(name=..., annotations=...)` and wrap with `@sigma_tool` (e.g. `tools/workbooks.py`). The root gateway in `server.py` mounts the domain with its namespace. - Provide an explicit, agent-friendly docstring describing capabilities, parameters, and return shape. - Destructive tools (delete, archive, deactivate, and bulk removal) **must** accept `confirm: bool = False`. Not every `POST`, `PUT`, or `PATCH` does (for example `workbooks_create_workbook` has no `confirm`). ### 3. Tool Annotations & Gating - Pass MCP `ToolAnnotations` at registration with the `ANNOTATION_*` constants from `tools/common.py`: - `readOnlyHint`: `True` for inspection/GET; `False` for mutations. Every tool sets it explicitly; the read-only gate reads nothing else (a missing hint is treated as a write). - `destructiveHint`: `True` for delete/archive/deactivate actions; `False` otherwise. - `idempotentHint`: among the constants, `True` only on `ANNOTATION_IDEMPOTENT`; `ANNOTATION_READ_ONLY`, `ANNOTATION_WRITE_SAFE` and `ANNOTATION_DESTRUCTIVE` leave it unset. The Tool Search and Code Mode discovery tools get `READ_ONLY_ANNOTATIONS` from `profiles.py` (`readOnlyHint` and `idempotentHint` `True`, `openWorldHint` `False`). - `openWorldHint`: `True` when interacting with external networks/APIs. - Gating: - Read-only mode (`SIGMA_MCP_READONLY=1` or profile `readonly`) keeps only tools annotated `readOnlyHint=True` and refuses every other call, directly or through `call_tool`, with a `SafetyViolationError` (a FastMCP `ToolError`, so `isError: true`). - Bulk protection: `admin_bulk_deactivate_members` and `admin_bulk_remove_team_members` are listed in `full` and refused at call time unless `SIGMA_MCP_ALLOW_BULK_DESTRUCTIVE=1`. - Profiles (`SIGMA_MCP_PROFILE` or `--profile`): `full`, `readonly`, `analyst`, `author`, `modeler`, `embed`, `access_admin`. A new tool goes into the job profiles in `profiles.py` or into `FULL_ONLY_TOOLS`; unknown names fail at build. Tool Search and Code Mode attach on `full` only and never together. Code Mode's sandbox is the public `code-mode` extra (`fastmcp[code-mode]`, same floor as `dev`); Code Mode tests call `pytest.importorskip("pydantic_monty")`, and `tests/conftest.py` aborts a CI session (`CI` set) without it, so CI syncs `--extra dev --extra code-mode`. ### 4. Pure Offline Testing & Contract Sync (`tests/`) - Add unit tests in `tests/` mocking responses via `unittest.mock.AsyncMock`. - **Zero live network calls in the default suite.** Tests must run 100% offline in CI. Opt-in live modules are `tests/test_e2e_live.py` (`-m e2e`) and `tests/test_integration_live.py` (`SIGMA_LIVE_TESTS=1` plus `SIGMA_CLIENT_ID`). - Keep `tests/test_client_surface.py` (mandatory house standard; conformance is not a substitute): it builds `create_server` under every profile in `PROFILES` and, through an in-memory `fastmcp.Client`, lists every tool and reads every resource and prompt; add fixture URIs or prompt arguments in its tables, never skip a component. - Update expected tool counts (per profile) in `scripts/check_tool_contract.py`, the `README.md` tables, and the copies in `tests/test_profiles.py` (`SIGNED_OFF`), `tests/test_unit.py`, `tests/test_enterprise_assertion.py` and `tests/test_layered.py`. A full-only tool also goes in `EXPECTED_FULL_ONLY` in `scripts/check_tool_contract.py`. - Ensure test statement coverage remains at **100.0%** (`--cov-fail-under=100`). Branch coverage is not enabled. --- ## 🛡️ Non-Negotiable Safety & Security Rules 1. **Destructive Confirmation Gate**: - Delete, archive, deactivate, and bulk-removal tools must accept `confirm: bool = False`. If `False`, do not execute the side-effect: return `_confirm_required(...)`, a normal result (`isError: false`) telling the caller to re-call with `confirm=true` (or a dry-run preview on `admin_bulk_deactivate_members`). - Input validation failures call `_invalid_request(...)`, which raises `ToolError` with the redacted payload, so the result carries `isError: true`. Other failures the tool detects itself (not found, timeout, a missing ID in an upstream response) call `_tool_failure(type, message, **details)`, which raises the same way. Never return an `{"error": ...}` dict as a normal result, and never put a raw upstream body in the message. A batch tool raises `_tool_failure("batch_failed", "All N failed; nothing was .", ..., failed_count=N, errors=[...])` when every item failed, and otherwise returns `status` (`success` / `partial_success`), a `_count`, `failed_count`, `results` and `errors`. Every item entry has `{"id", "status", "result"}` or `{"id", "status": "failed", "error"}`, and keeps any field names and entries the tool returned before alongside them (prefer additive changes over renames; a failed item that earlier releases listed in `results` stays there and is also listed in `errors`); use `_batch_outcome(items, verb, results, errors, context=...)` to build the result or raise. Catch `Exception` (never `BaseException`) per item, build its `error` with `_item_error(exc, index=i)`, which always carries a non-empty, redacted `message` and logs only the item's zero-based index and its error type (the exception class name, or the `error_type` for a reason string), never the message (no `logger.exception`, no `exc_info`). Raise every redacted `ToolError` after the `except` block ends (capture what you need inside it), so the original exception is on neither `__cause__` nor `__context__`. Do not use either helper for the confirm prompt. 2. **Secret Redaction**: - Error messages, logs, and tracebacks must pass through regex redaction (`_redact_secrets`) stripping Bearer tokens, client secrets, access tokens, subject tokens, raw JWTs, `ghs_` tokens, and the fleet token set copied verbatim from the template (API, access, refresh, auth, id and session tokens as `key=value`, `key: value`, JSON or URL-encoded `%3D`; `X-Auth-Token` and `Authorization: Token` headers; JSON `"token"`; a bare `token` key not preceded by a letter, digit or underscore, followed by `=` or `:` with optional whitespace on either side and an optional single or double quote before the value, as in `token=x`, `token: x`, `token = x` and `token: "x"`; API keys, passwords, AWS `secretAccessKey`, and a connection's `privateKeyPassphrase` and `pythonSecrets`; a bare `value` only inside an API credential's `apiKey` object, marked by its `isQueryParam` sibling), while leaving words such as `tokenizer`, `page_token=` and `max_tokens=` alone. A PEM block runs from `-----BEGIN