# Distil — compression with a quality contract

Distil is a cache-aware, causally-pruned context compressor for LLM agentic
runtimes. It compresses input and output context, certifies that the agent's
decisions are statistically unchanged (non-inferiority test), and is the only
compressor that proves its guarantees rather than assuming them. Zero runtime
dependencies. Four ways to use it, and they do not all reach the same tier:

  - Agent wrap: `distil wrap -- <agent>` for claude, codex, gemini, aider,
    opencode, qwen, goose, grok, openhands, copilot (GitHub Copilot CLI, BYOK
    mode), kimi (Kimi CLI). Zero config. (openhands additionally needs its own
    --override-with-envs flag before it will read the environment.)
  - HTTP proxy for Anthropic Messages, OpenAI (Chat Completions + the Responses
    API), Azure OpenAI, and Google Gemini generateContent. Point any base_url
    client at it. The same provider coverage is in all three servers: the sync
    proxy, the async proxy, and the standalone gateway.
  - Python library: `from distil import compress_messages, expand_handle`.
  - TypeScript library: `compress(messages)` from the `distil-llm` npm package,
    plus `distilMiddleware()` for the Vercel AI SDK's wrapLanguageModel.

Framework hooks (in-process, no network hop): LiteLLM, LangChain, LangGraph,
Agno, Strands, AutoGen, LlamaIndex. A generic ASGI middleware (distil.integrations.asgi)
compresses provider-shaped POST bodies for apps that host their own LLM-facing
endpoint. An MCP server exposes distil_compress / distil_expand /
distil_savings to any MCP client.

IMPORTANT for anyone integrating: the reversible DIGEST tier is reached by the
proxy, wrap, and MCP paths. The in-process LIBRARIES are LOSSLESS-tier only, on
purpose — the digest mints restore handles that must share one store with the
proxy, and it is the tier the decision-equivalence certificate measures. The
TypeScript port is held byte-identical to the Python engine by a conformance
suite; where it cannot match Python's bytes it declines rather than emitting
output the certificate does not cover.

Graded on SWE-bench Verified: 42.0% task success
vs 39.2% full context (paired non-inferiority certified; superiority not
significant). 83.2% savings at 0% decision-change rate on a live head-to-head
(2026-07-05, distil 1.10.1 vs llmlingua 0.2.2 and headroom-ai 0.27.0; that run's Headroom figures are 0.27.0-era). Re-run 2026-09-04
against headroom-ai 0.37.0: distil-causal 52.9% tokens / 58.7% dollars at 100%
decision-equivalence (PASS) vs Headroom 1.7% / 2.0% / 81% (FAIL) on the warm
corpus gate; on the read->edit->re-read codebench workload Headroom takes 35.6%
tokens for +4.9% dollars.

Two reporting commands read the same content-free local records and answer
different questions. `distil dissect <session>` is backward-looking and
per-session: what happened to this one. `distil discover` is forward-looking and
cross-session: where savings are still being left on the table across the last N
sessions, as a ranked list of actions, each carrying the tokens and dollars per
week it would recover, the derivation of that number, and the one command to act
on it. `discover` prints the median and p10/p90 of per-session savings beside the
best session so a best case is never read as typical, prefers the rate the
machine itself measured over any published ratio, and stays silent where it
cannot measure.

Five properties are checked rather than asserted, and each is separately
runnable:

  - Cache contract: compression is suffix-only and cache-monotonic — it may
    never rewrite bytes the provider has already cached. Enforced as an
    invariant, after shipping the bug once and measuring it at 2x the cost of
    compressing nothing.
  - Exact-quote guarantee: a block whose bytes are a verbatim slice of a file is
    never digested, so a later Edit(old_string=...) still matches. Provenance is
    read from the shell command (cat, head, sed -n), not just the tool name.
    Byte-exact quote loss on real coding traffic: 39.3% -> 16.2%.
  - Adversarial validation: `distil validate --adversarial` runs a COMA-class
    battery through the same public path the proxy uses. No keep budget is
    shared between blocks anywhere, asserted as an equality in CI. Two findings
    are published rather than smoothed over: dedup-baiting does fold the genuine
    error line (reversibility recovers it), and decoy-verdict flooding is a real,
    unmitigated denial of savings.
  - Degradation curve: `distil bench --curve` reports savings, fact recall,
    visible recall, facts lost and reversibility at every rung of the ladder,
    offline and free.
  - Paired live estimator: shadow mode replays a sampled request three times (A
    and A' on the original, B on the compressed context) and reports the paired
    difference 1{A=B} - 1{A=A'} with a bootstrap 95% CI, unclipped, so it can
    report harm. One reporting floor (50 A/B + 30 A/A) gates every surface, and
    below it every surface prints "below reporting floor" rather than a number.
    The maintainer's live sample cleared that floor on 2026-09-15: paired
    equivalence 97.5% with a 95% CI of [95.5, 99.5] over n=398 A/B and 399 A/A,
    paired difference -0.025 [-0.045, -0.005], digest and lossless-only mixed.
    Replays run hot (399 of 399, temperature not pinned), so the paired
    difference is the statistic, not the 53.0% raw agreement. Artifact:
    benchmarks/results/shadow-live-2026-09-15.json

## Key sources

README (canonical overview, install, head-to-head table, security):
https://github.com/dshakes/distil/blob/main/README.md

CHANGELOG (per-version feature list):
https://github.com/dshakes/distil/blob/main/CHANGELOG.md

Changelog (the same CHANGELOG.md, rendered on the site so it can never drift):
https://dshakes.github.io/distil/changelog.html

THREAT_MODEL.md (what distil protects against and explicitly does not):
https://github.com/dshakes/distil/blob/main/THREAT_MODEL.md

docs/EVALUATION.md (evaluation methodology, what the numbers mean):
https://dshakes.github.io/distil/EVALUATION.md

benchmarks/README.md (reproducible head-to-head harness):
https://github.com/dshakes/distil/blob/main/benchmarks/README.md

## Site pages

Getting started (install, quickstart, wrap presets, proof ledger):
https://dshakes.github.io/distil/getting-started.html

Architecture (module map, tier model, cert loop, OTel, Proof Ledger):
https://dshakes.github.io/distil/architecture.html

CLI reference (every subcommand, flags, sample output):
https://dshakes.github.io/distil/cli.html

Deploy & Security (topologies, encrypt-at-rest, gateway keys, OTel session id):
https://dshakes.github.io/distil/deploy-security.html

Security (where to report a vulnerability, the full data-handling whitepaper,
and the deploy security guide — one page, three links):
https://dshakes.github.io/distil/security.html

Metrics & Observability (Prometheus /distil/metrics exposition and its admin
gate, OpenTelemetry counters + GenAI spans, what to alert on):
https://dshakes.github.io/distil/metrics.html

FAQ (decision-equivalence, nightly gate, OpenAI/Gemini adapters, troubleshooting):
https://dshakes.github.io/distil/faq.html

Benchmark (certified compression frontier, live head-to-head):
https://dshakes.github.io/distil/benchmark.html

Provider Compaction (pre-registered A/B of Anthropic context editing & OpenAI
server-side compaction: does the provider's own context manipulation change
agent decisions? clearing changed 92.5% of agent decisions at keep=0 and 95-100% at the
shipped keep=3 default, where the failure shifts from stalling to acting on the
wrong facts; OpenAI compaction 12.5-20%):
https://dshakes.github.io/distil/provider-compaction.html

Reproduce the numbers (all CI gates, real-model grading, scorecard):
https://dshakes.github.io/distil/benchmarks.html

Compare (distil vs LLMLingua-2 vs Headroom):
https://dshakes.github.io/distil/compare.html

Techniques (tiers, causal pruning, query-aware salience, learned keep-model):
https://dshakes.github.io/distil/techniques.html

Concepts (DERC certificate, conformal risk control, trajectory certificate):
https://dshakes.github.io/distil/concepts.html

Library API (embed distil in your own agent — Python and TypeScript, the tier
boundary, and why the in-process path is lossless-only):
https://dshakes.github.io/distil/library.html

Which mode am I? (proxy vs hook vs library — the one page that answers "which
one applies to me" before any jargon; start here if unsure):
https://dshakes.github.io/distil/which-mode.html

Subscription mode (why the proxy stays lossless-only on a flat-rate plan, what
the PostToolUse hook does instead, and what "savings" means when the bill is
fixed):
https://dshakes.github.io/distil/subscription.html

Evals (every probe distil runs on itself, including the three bugs these probes
found in distil's OWN harness — published, not hidden):
https://dshakes.github.io/distil/evals.html

Independent benchmark (75 agent runs by an outside engineer, graded from the
API's own usage fields: the defect that finished 6 of 15 tasks, why it happened,
and what 1.49.0 changed):
https://dshakes.github.io/distil/benchmark-independent.html

Research (the negative results too: where compression did NOT survive real
end-to-end execution, and what that bounds):
https://dshakes.github.io/distil/research.html

Cache contract (the invariant: compression may never rewrite bytes the provider
has already cached, and what happens when it does):
https://dshakes.github.io/distil/cache-contract.html

Threat model (compression as an attack surface: the COMA-class battery, what is
mitigated by construction, what is only detected, and the two cases that do not
come back clean):
https://dshakes.github.io/distil/threat-model.html

Cache (prompt-prefix economics: why a compressor that removes tokens can still
cost more, and how distil stays cache-monotonic):
https://dshakes.github.io/distil/cache.html

Integrations (framework hooks and SDK adapters — LiteLLM, LangChain, LangGraph,
Agno, Strands, AutoGen, LlamaIndex, an ASGI middleware, Vercel AI SDK, OpenAI,
Anthropic, Gemini):
https://dshakes.github.io/distil/integrations.html
https://dshakes.github.io/distil/autogen.html
https://dshakes.github.io/distil/llamaindex.html
https://dshakes.github.io/distil/asgi.html

Adoption (install counts and community savings, and why an opt-in census can
never attribute them to individual users):
https://dshakes.github.io/distil/adoption.html

IDE agents and config-file agents (Cursor, Cline, Kilo, Roo, Warp, Cortex Code
via the proxy since `wrap` cannot reach them; the config key each config-file
agent needs pointed at `distil proxy`'s printed URL):
https://github.com/dshakes/distil/blob/main/docs/IDE-AGENTS.md

Enterprise (Helm chart, OIDC + RBAC, audit receipts, and an honest commercial
status: no SOC 2, no SLA, no SAML today):
https://github.com/dshakes/distil/blob/main/docs/ENTERPRISE.md

Security whitepaper (data residency, encrypt-at-rest, supply chain, what is
explicitly out of scope):
https://github.com/dshakes/distil/blob/main/docs/SECURITY-WHITEPAPER.md