# How CALM works This is the technical deep-dive the main [README](../README.md) only summarizes — multi-tier indexing, the SCIP/LSP overlay system, search, the edit safety net, concurrency, self-grading, memory, and the sanitization layer. If you just want to know what CALM does for you and how to install it, the README is a shorter, better place to start; come back here once you're deciding whether to depend on it, or you want to know exactly how a specific guarantee is implemented. ## Design philosophy CALM isn't a pile of MCP tools bolted together — it's designed as a **map and an active co-pilot for the agent actually holding the wheel**, not a dashboard for a human watching from the sidelines. - **Every response carries `suggested_next`.** The agent is rarely left guessing what step comes next — the tool that just ran tells it. - **The genuinely risky steps are hard-gated, not just recommended.** `edit_context` before any edit, `diff_impact` before any commit — these are enforced, not suggested. Everything lower-stakes just nudges; the agent keeps its own judgment where the cost of being wrong is low. - **The signals are proactive, not something the agent has to ask for.** `fitness_report`, `session_context`'s `pending_diff_impact` / `possibly_stuck`, `repo_overview`'s `memory_notes_count` — the agent never has to remember "did I already check impact?" or notice on its own "am I going in circles?". CALM answers before it's asked. The end goal is reduced cognitive load: the agent spends its budget on the work that actually creates value, not on managing its own navigational bookkeeping. ## Multi-tier indexing - **6 Tier-0 languages** — Python, TypeScript, JavaScript, Java, Rust, Go — get full `tree-sitter` AST parsing, a real call graph, an import graph, and multi-tier resolution, always on, no feature flag required. - **18 Tier-0.5 languages** — full `tree-sitter` AST parsing with call-graph and import resolution, gated behind Cargo features: - **On by default** (`tier0-5` feature bundle): C, C++, C#, Ruby, PHP, Shell, R. - **Opt-in, one feature flag each**: Kotlin, Swift (`lang-kotlin`/`lang-swift`), plus Scala, Dart, Lua, Elixir, Haskell, OCaml, Zig, PowerShell, Groovy — the 9 languages added in the 25-language expansion's Phase C batch. Falls back to regex/line-scan symbol extraction (no call graph) only when the matching grammar feature isn't compiled in. - Dart is the one exception worth calling out by name: it parses symbols cleanly but produces zero call edges, because its tree-sitter grammar has no call-expression node — a documented limitation, not a bug. - **SQL gets its own standalone indexer** (`sqlparser`, real grammar parsing, not regex) — extracts tables/views/procedures accurately across Postgres/MySQL/SQL Server dialects, but stops short of a call graph, since "calls" isn't a coherent concept across SQL dialects the way it is for the languages above. - **Incremental watcher** — only changed files get re-parsed (FNV-1a content hash diff); the call graph rebuilds incrementally, parallelized with `rayon`. `calm serve` picks incremental reindex automatically whenever an index already exists. ## A call graph you can actually trust - **Every edge carries a confidence label** — `resolved` / `inferred` / `formal` / `textual` (plus `ambiguous`/`unresolved` fallback tiers when a call site's target genuinely can't be pinned down) — so an agent knows when it's looking at a sure thing versus a best guess. - **SCIP overlay — formal, compiler-grade ground truth, 9 providers spanning 12 languages**: Rust (`rust-analyzer`), Go (`scip-go`, including multi-module `go.work` workspaces — indexed once per member module, then rebased into one graph, since `scip-go` itself has no native workspace flag), Python (`scip-python`), JavaScript + TypeScript (`scip-typescript`), Java (`scip-java` — also indexes Kotlin in the same pass, since `scip-java` bundles a `kotlinc` plugin for mixed Gradle/Maven modules), C# (`scip-dotnet`), PHP (`scip-php`), Ruby (`scip-ruby`/Sorbet), and C/C++ (`scip-clang`, live-verified 2026-07-15). Each language is a data-driven `ScipProvider` entry (`crates/calm-core/src/scip/provider.rs`), not a copy-pasted module — adding another language is one table row. Every provider auto-detects its own binary and silently sits out if it isn't there (zero behavior change on a machine missing that toolchain), runs under a hard timeout, and caches against a per-language fingerprint (lockfile/build-file hash + toolchain + dirty source keys) so an unchanged project never re-pays the cost. Verification maturity varies honestly by language: `.github/workflows/scip-nightly.yml` runs one independent job per provider (restructured 2026-07-15 from a single monolithic job, so one provider's install failure can no longer suppress test evidence for the others) — all 9 providers (Rust/Go/Python/JavaScript/Java/C#/Ruby/C++/PHP) were live-verified against a real installed binary during the 2026-07-15 restructure, but as of that same day the automated nightly cron had not yet completed a post-restructure run (check `gh run list --workflow=scip-nightly.yml` for current status) — "works against a real binary" and "nightly CI is green" are two separate claims, don't conflate them until a run confirms the latter. PHP's own path was the last one fixed, and not with a security-posture workaround: Packagist's only `scip-php` release (v0.0.2, 2023) has two independent problems — a Composer advisory block on its `google/protobuf` range, and a separate `RuntimeException: Invalid scip-php vendor directory` crash (github.com/davidrjenni/scip-php#235) that's already fixed upstream (PR #797, merged 2026-03-28) but never cut into a release or rebuilt into the `davidrjenni/scip-php` Docker image (reported: github.com/davidrjenni/scip-php/issues/862). Both disappear installing from that post-fix commit directly instead of from Packagist — `main`'s own `composer.json` already requires a patched `google/protobuf` range, so the advisory question doesn't even arise. Pinned to an exact commit (not floating `main`) for reproducibility; bump it once upstream ships a real release. - **LSP overlay — a second, complementary path to formal edges**: for Go (`gopls`) and C/C++ (`clangd`) — plus Rust's `rust-analyzer` on a live-session path, distinct from its batch SCIP export — a real language server resolves ambiguous/textual call sites interactively, on demand. This doesn't run automatically (`policy` defaults to `on_demand`, and always will unless per-save latency is measured safe — see ADR-0004); trigger it explicitly with the `lsp_refresh` tool or `calm scip-run`/equivalent CLI. - Trigger either overlay on demand with `calm scip-run --lang ` or the `scip_refresh`/`lsp_refresh` MCP tools; `calm index --scip-file --sub-root ` ingests a pre-built SCIP index instead, for CI/sandboxed runs with no network access to install an indexer. - **Trace output stays readable at scale.** A real hub symbol can have dozens of callers — `callers`/`callees`/`edit_context` put non-test callers first and cap the visible list at a configurable size (the true total is still reported alongside it), so the one production call site isn't buried behind sixty near-identical test fixtures. A repeat call on an unchanged symbol can skip resending the list entirely via `if_none_match`/etag conditional-fetch, the same pattern `source()` already used — measured up to a ~60% token reduction on a real hub symbol in this repo's own graph, with the pre-edit risk assessment always computed from the full list before it's ever truncated. - **Graph metrics — `coreness` (k-core) and `is_hub`** — flag the symbols central enough that touching them is inherently higher-risk. `repo_overview.core_symbols` reuses the same metric to sketch the architecture's "skeleton" on the very first call (inspired by Aider's PageRank repo-map, but built on a metric CALM already computes rather than a separate pass). ## Search that actually finds things - **Full-text + semantic search, fused** — FTS5 (BM25) combined with semantic embeddings (`model2vec-rs`, pure Rust, no ONNX) via a 3-way Reciprocal Rank Fusion (text + symbol-identity vector + code-body-chunk vector) — finds relevant code even when the query doesn't share a token with the symbol name. KNN is a brute-force cosine scan in pure Rust with an in-RAM cache — no C vector-search extension, so it behaves identically on every release platform (the previous `sqlite-vec` dependency didn't compile on musl libc, which silently killed semantic search on Linux/Docker builds). The default model (`minishlab/potion-code-16M`, MIT-licensed — from MinishLab, the same lab behind Semble, one of the tools benchmarked in the README's [Measured against the tools that came before it](../README.md#measured-against-the-tools-that-came-before-it)) is vendored straight into the binary: `build.rs` fetches the weights from Hugging Face and SHA256-verifies them at compile time (cached in the working tree afterward — not Git LFS, that mechanism was removed 2026-07-12), then embeds them via `include_bytes!` — no network needed at runtime for the default case; a failed compile-time fetch falls back to a placeholder plus a one-time Hugging Face download at runtime instead, unless you explicitly opt out (`allow_network_fallback`) to keep a strict zero-network guarantee. - **Real grep/glob, straight off disk** — `search(kind="grep")` uses actual regex + glob filtering through a `.gitignore`-respecting walker, bypassing the index entirely — so it reaches files the indexer never parses (`Cargo.toml`, `docs/*.md`) too, each match enriched with its surrounding symbol when one exists. - **Scores you can actually read** — `search`/`locate` round RRF/similarity scores to 4 decimal places before returning them (`0.01639344262295082` → `0.0164`) — purely representational, doesn't change ranking. - **Noise-penalty ranking** — results living in test/generated/example files are scored down when an equivalent real-implementation result exists, so the actual code surfaces first instead of getting buried under a same-named test fixture. ## Editing with an actual safety net - **`edit_lines`/`edit_symbol`** — the one write path, working on any tracked file (not just parsed symbols). A content-hash conflict guard (FNV-1a) on the exact line range rejects stale writes and hands back the current hash/content to re-read; multiple hunks in one call apply bottom-up so offsets never drift between them. - **Syntax-validated before it ever touches disk** — `tree-sitter` checks the result parses cleanly; a write that would introduce a syntax error is refused outright, nothing gets written. - **Hub and high-fan-in symbols need three things, not just one** — `edit_context` must have actually been called for that exact symbol *this session* (not a prior session, not a stale review), `confirm:true`, and a `reason` string that cites a real caller name `edit_context` itself returned, not a generic phrase like "this looks safe". A policy only a tool with a real call graph — and a memory of what it just showed you — can enforce. - **Atomic writes, immediate reindex** — temp file + fsync + rename, then reindexed synchronously (not waiting on the file watcher); the response comes back with post-edit risk/callers, like a miniature `diff_impact`. - **Hook-enforced, not just documented** — under Claude Code, `.claude/hooks/calm-nudge.sh` actually blocks the first `Edit` of a session until `edit_context` has been called, and blocks `git commit`/`git push` if files changed since the last `diff_impact`. `session_context`'s `pending_diff_impact` gives the same signal on any other MCP client. ## Concurrency & reliability Running CALM from more than one editor session on the same repo used to mean N independent indexers, N SQLite connections, and N copies of the embedding model in RAM — real pain found by pointing CALM's own instrumentation at this repo mid-session. That's been closed from multiple directions: - **Cross-process edit lock** — an OS `flock` on `.calm/edit.lock`, layered underneath the in-process hash check, closes a narrow TOCTOU window where two separate processes could both pass a stale-hash check and the second write would silently discard the first. - **Single-instance indexing lock, with promotion** — only one `calm serve` process per project root ever runs the background indexer/watcher; a losing process auto-promotes to owner if the current owner exits mid-session, instead of a second editor session being stuck read-only forever. - **A real SIGTERM watchdog** — a raw kernel `alarm()` guarantees the process exits even when the MCP transport's stdio-read thread is blocked in an uncancellable OS read, a bug that a purely async watchdog silently never caught. - **A shared-daemon model, on by default** — `calm serve --listen unix:PATH` runs one owning daemon; `calm connect` gives every other session a lightweight forwarder over the same socket instead of its own full process, with automatic stale-build detection (`daemon.meta`) that respawns the daemon after a rebuild instead of silently serving an old binary. `scripts/mcp-launcher.sh` (and therefore the npm/plugin distribution) defaults to this on Unix whenever no extra launcher args are in play — falls back to the original one-process-per-session `calm serve` for any custom invocation, or set `CI_MCP_LAUNCHER_NO_DAEMON=1` to opt out entirely. - **Concurrent-agent awareness** — under the shared daemon, `session_context.other_active_sessions` reports every other connection sharing that socket right now (the file it last touched, when, how many tool calls), so an agent can notice "someone else is already working in this file" before stepping on the same area — not full multi-agent coordination, just making concurrent sessions visible instead of invisible. ## The codebase grading itself - **`calm fitness-check` / `fitness_report`** — 9 metrics (hub concentration, dead code, hotspot risk, edge coverage, cyclomatic complexity, architecture-boundary violations, doc-drift) checked against thresholds in `thresholds.toml`, queryable mid-session or as a CI gate. - **Coverage-aware dead-code detection** — auto-detects lcov / `.coverage` / Go `coverage.out` / Cobertura XML at startup and folds real runtime coverage into `dead_code_confidence`, so code a test actually exercises at runtime doesn't get flagged just because the static call graph missed the call site. `scripts/gen-coverage.sh` generates one on demand for this repo itself. - **Architecture boundaries — `[[boundaries]]`** — declare "module A must not import module B" directly in `thresholds.toml`, matched by path prefix against the real import graph; every violation is reported with the actual offending file pair, not just a count. - **Doc-drift detection — `[config_drift]`** — flags file-path references inside declared docs that no longer point at anything real, so a design doc doesn't quietly keep describing a file that was deleted three refactors ago. ## An agent that remembers, and knows when it's stuck - **`remember`/`recall`** — durable, interpretive notes (an architecture decision, a gotcha) keyed by topic, surviving restarts — distinct from `session_context`, which only tracks in-session navigation and resets on restart. - **Notes surface themselves, without a separate `recall` call** — `edit_context`/`locate` automatically attach any `remember`d note that references the file in play (`related_notes`). On a hub file a note only qualifies if its text names the exact symbol (`specificity: "symbol"`); a plain file-level match is used only on smaller, non-hub files, so one old note doesn't bury every symbol in a large file forever. A note that trips the same prompt-injection heuristic `source`'s `content_warning` uses is left out of this automatic surface — still fully readable via an explicit `recall`. - **`pattern_debt_register`/`pattern_debt_status`** — found the same bug in more than one place? Anchor it by the symbol's qualified_name (survives line-shifting edits elsewhere in the file, unlike a raw path+line) and baseline the duplicate count with `search(kind="similar")`; re-check later and get back `open`, `resolved`, or `anchor_lost` (the anchor symbol itself was renamed/removed — reported honestly, never silently counted as fixed). - **Git co-change mining** — `edit_context` mines `git log` for files that historically change alongside the one being edited despite no import/call relationship (a model and its migration, say) — a coupling signal the static graph can't see on its own. - **Session progress signal** — `session_context.possibly_stuck` flags 10+ tool calls with no new file/symbol touched; informational only, the decision to break the loop stays with the host (e.g. Claude Code's `/goal`). - **MCP Prompts** — `review_symbol`, `debug_symbol`, `onboard_area` package a full multi-step workflow into one slash-command-style call. ## Honest about its own freshness - **Index state machine surfaced everywhere** — `scanning → parsing → building_edges → ready`, so an agent never mistakes stale data for current. - **Build-freshness check** — `calm doctor` compares the commit the running binary was built from against the repo's current `HEAD`; `scripts/mcp-launcher.sh` checks source mtimes before trusting an existing `target/{debug,release}/calm`, rebuilding rather than silently serving a stale binary. ## Safe by default - **Output sanitization** — `source`/`understand` redact credential-shaped text (PEM keys, GitHub/AWS/Slack tokens, JWTs, password assignments) before it's ever returned, and flag a `content_warning` when code contains prompt-injection-shaped text — flagged, never silently altered, since a false positive there would corrupt real code. The heuristic (`calm_core::sanitize`, 19 labeled patterns) covers plain-English phrasing (`"ignore previous instructions"`), chat-template role-marker spoofing (ChatML `<|im_start|>`, `[INST]`/`[SYS]` brackets, fake `system:`/Markdown-heading role markers), fake tool/turn-boundary tags (``, ``), jailbreak/persona-override phrasing, exfiltration phrasing (prompt/secret-reveal requests), zero-width Unicode obfuscation, and a first pass at Vietnamese-language equivalents — deliberately excludes anything with real false-positive risk (e.g. generic tag-density scoring, homoglyph detection) rather than guess. - **`scan_text` — the same detection, on demand, for anything that didn't come through the index.** `source`/`understand`'s `content_warning` only covers indexed source; every tool's own output is separately scanned (advisory-logged, not surfaced) at one central choke point (`timed_tool`). Neither covers content an agent fetches itself — a WebFetch/WebSearch result, a subagent's report. `scan_text` closes that gap: point it at any text and get the same labeled hits back, entirely local and regex-based — no dependency on a hosted LLM safety classifier being available, so it keeps working even when that classifier isn't. - **Local-only** — no outbound calls for the code/data path. The one narrow exception is the semantic-search default model download, which is a single public, static file fetch, opt-out-able, and unrelated to your repo's contents ever leaving the machine. --- Back to the [README](../README.md) for install instructions, the MCP tool reference, and the benchmark summary.