Four pictures. Each one is a single principle; together they explain why a resident entity graph changes what an agent can afford to do — not just how fast it does it.
Text search re-reads the corpus on every question, so its cost grows with repo size. An index answers in near-constant time — but only if it’s already in memory. A fresh process paid ~800 ms to hydrate before its first answer, so scanning often won anyway. A resident, prewarmed graph removes that floor: the constant-time line drops below the scan line at every repo size.
Model curves anchored to measured points (●): warm sem_context <1 ms on an 85K-LOC repo
(faster than a ripgrep scan of the same repo); fresh process + SQLite hydrate ≈800 ms. Both axes logarithmic.
Humans can’t perceive delays under roughly 100 ms, so anything below that line is free to run on every event — every prompt, every save — without anyone noticing. The old paths sat well above the line, so sem had to be an explicit, occasional call. Every new path sits below it.
Measured values from the changelog; “extra model turn” is the illustrative cost of answering a retrieval question with another inference round-trip. Logarithmic axis.
An agent’s real costs are inference turns — seconds of thinking and a re-read of the whole context each time. At this scale, tool time is already an invisible sliver (the gaps between segments below). So the only retrieval optimization that still matters is removing turns: one-call lookup removed one, prompt-time prefetch removed the other.
Illustrative timeline with ~6 s inference turns; tool segments drawn to true scale — grep ≈0.4 s,
file read ≈0.2 s, sem_context 26 ms, prefetch 10 ms (before the first token).
Every retrieval turn also pays in context: grep hits plus two raw file dumps land in the window whether or not they’re relevant. An entity answer carries only the graph neighborhood, and the compact tree rendering cut that payload to ~15% of the JSON it replaced. Cheaper turns and fewer of them.
Grep-era and JSON figures are typical/illustrative; the tree-vs-JSON ratio (~15%) is measured.
Read together: indexes beat scans (1), residency made the index answer imperceptibly fast (2), which made speculation free, which removed whole turns from the loop (3), and each remaining answer is a fraction of the tokens (4). That’s why ambient context is now possible and wasn’t before.