# squeez
[](https://github.com/claudioemmanuel/squeez/actions/workflows/ci.yml)
[](https://www.npmjs.com/package/squeez)
[](https://crates.io/crates/squeez)
[](LICENSE)
[](CONTRIBUTING.md#license--contributor-sign-off)
[](CHANGELOG.md)
End-to-end token optimizer for seven AI CLI hosts β Claude Code, GitHub Copilot CLI, OpenCode, Gemini CLI, OpenAI Codex CLI, Pi, and Hermes. Compresses bash output up to **95%**, collapses redundant calls, preserves exact identifiers through every summary, refuses net-loss compressions, and injects a terse prompt persona β automatically, with zero new runtime dependencies.
---
## π° What's new
> Latest first. Full history in [CHANGELOG.md](CHANGELOG.md).
### v1.46.0 β shipped filter pack + native `git status`
The filter DSL could always cover the long tail `GenericHandler` can't compress β it just shipped empty, so every user started from a blank `filters.ini`. This release fills it, adds a native `git status` path, and makes two guards that were only ever measured in benchmarks run for real.
- **41 built-in filter rules** (`assets/filters_builtin.ini`) β `pip`/`uv`/`poetry`, `bundle`/`composer`/`rubocop`/`phpstan`, `dotnet`/`swift`/`gcc`, `mypy`/`basedpyright`/`golangci-lint`, `shellcheck`/`hadolint`/`yamllint`/`markdownlint`/`pre-commit`, `systemctl`/`journalctl`/`rsync`/`ping`/`ansible-playbook` and more. Every rule ships an inline self-test that CI gates on. Project and user rules still shadow any built-in; `builtin_filters = false` turns the pack off.
- **`git status` reads as porcelain** β re-run as `git status --porcelain=v1 -b` and rendered as a working-tree summary. Exact paths are never collapsed to counts, raw porcelain codes never reach the model, and in-progress rebase/merge/bisect (which porcelain omits) is read from `.git/` markers instead of a second `git status`.
- **The preservation guard actually runs.** `economy::preservation` used to execute only in `benchmark`. It now scores anchor survival at wrap time on β₯90%-reduction calls; below the floor the verbatim original is stashed and the header carries `[anchors: N%]`.
- **The net-win gate counts its own marker.** The ~40-token retrieve marker was appended *after* the gate decided the call was a win, so a call saving 25 tokens could ship a 40-token marker and still claim a win. It is now inside the arithmetic.
- **Dispatch blind spots closed** β `TF_LOG=debug terraform plan` and `npx terraform plan` reach the terraform branch; wrappers peel in a loop (`sudo npx vitest`); a pipeline ending in a transforming stage (`cargo build | grep error`) dispatches on that stage.
- **Benchmarks grew to 46 scenarios**, six of them covering the new rule pack, so the pack's contribution is measured rather than asserted.
---
## Install
Three methods β all produce the same result (binary at `~/.claude/squeez/bin/squeez`, hooks registered).
### curl (recommended)
```bash
curl -fsSL https://raw.githubusercontent.com/claudioemmanuel/squeez/main/install.sh | sh
```
> **Windows:** requires [Git Bash](https://git-scm.com/downloads). Run the command above inside Git Bash β PowerShell/CMD are not supported.
### npm / npx
```bash
# Install globally
npm install -g squeez
# Or run once without installing
npx squeez
```
Downloads the correct pre-built binary for your platform (macOS universal, Linux x86_64/aarch64, Windows x86_64). Requires Node β₯ 16.
### cargo (build from source)
```bash
cargo install squeez
```
Builds from [crates.io](https://crates.io/crates/squeez). Requires Rust stable. On Windows you also need [MSVC C++ Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/).
---
### Supported hosts
`squeez setup` auto-detects every CLI present on disk and registers the hooks. `squeez uninstall` removes them. Session data and `config.ini` are preserved so reinstall is lossless.
| Host | Memory file | Bash wrap | Session memory | Budget inject (Read/Grep) | Notes |
|---|---|---|---|---|---|
| **Claude Code** | `~/.claude/CLAUDE.md` | β
native | β
native | β
native | Restart Claude Code to pick up hooks |
| **Copilot CLI** | `~/.copilot/copilot-instructions.md` | β
native | β
native | β
native | Restart Copilot CLI after setup |
| **OpenCode** | `~/.config/opencode/AGENTS.md` | β
native | β
native | β
native | Plugin at `~/.config/opencode/plugins/squeez.js`; MCP tool calls skip hooks (upstream sst/opencode#2319) |
| **Gemini CLI** | `~/.gemini/GEMINI.md` | β
native | β
native | π‘ soft via `GEMINI.md` | `BeforeTool` rewrite schema pending upstream docs ([google-gemini/gemini-cli#25629](https://github.com/google-gemini/gemini-cli/issues/25629)) |
| **Codex CLI** | `~/.codex/AGENTS.md` | β
native | β
native | π‘ soft via `AGENTS.md` | `apply_patch` hooks landed in 0.123.0 ([#18391](https://github.com/openai/codex/pull/18391)); `updatedInput` + `read_file`/`grep` hook surface still pending ([openai/codex#18491](https://github.com/openai/codex/issues/18491)) |
| **Pi** | `~/.pi/agent/skills/squeez/SKILL.md` | β
native | β
via skill | β
native | TypeScript extension at `~/.pi/agent/extensions/squeez/index.ts`; restart Pi after setup |
| **Hermes** | `~/.hermes/profiles/default/SOUL.md` | β
native | β
native | β no budget surface | Python plugin (`__init__.py` + `plugin.yaml`) auto-discovered by Hermes; detected via `~/.hermes/` |
### Manage
```bash
squeez setup # register into every detected host
squeez setup --host= # register into one host
squeez uninstall # remove squeez entries from every detected host
squeez uninstall --host=
```
Slugs: `claude-code` / `copilot` / `opencode` / `gemini` / `codex` / `pi` / `hermes`.
After install, restart the CLI you use to pick up the new hooks.
### Uninstall
```bash
squeez uninstall # preserves session data + config.ini
bash ~/.claude/squeez/uninstall.sh # (legacy) full wipe, if the script exists
```
### Self-update
```bash
squeez update # download latest binary + verify SHA256
squeez update --check # check for update without installing
squeez update --insecure # skip checksum (not recommended)
```
---
## What it does
| Feature | Description |
|---------|-------------|
| **Bash compression** | Intercepts every command via `PreToolUse` hook, applies smart filter β dedup β log-template β relevance-truncation. Up to 95% reduction. (Per-dir grouping stays reserved for `git status`, where the path list is the payload.) |
| **Reversible compression** | When a large output is compressed, the verbatim original is stashed in a content-addressed blob and a `[squeez: β¦ call squeez_retrieve with key=""]` marker is emitted. The model recovers any dropped detail via the `squeez_retrieve` MCP tool β so compression can be aggressive without losing information. TTL-pruned, zero-dep. |
| **Log-template compaction** | Collapses near-identical log lines that differ only by a timestamp, id, hex hash, or `1ms`-style value into one `[ΓN] ` line β what `dedup` (exact-only) leaves on the table. |
| **Relevance-aware truncation** | When the generic handler must truncate, it keeps the highest-relevance lines (error/signal words + terms drawn from the command) instead of a blind head β so a buried error survives. |
| **Context engine** | Cross-call redundancy with two paths: exact-hash match (FNV-1a, fast) **and** fuzzy trigram-shingle Jaccard β₯0.85 (whitespace, timestamps, single-line edits no longer defeat dedup). |
| **Summarize fallback** | Outputs exceeding 500 lines are replaced with a β€40-line dense summary (top errors, files, test result, tail). **Benign outputs get 2Γ the threshold** so successful builds stay verbatim. |
| **Identifier factsheet** | Exact identifiers in the region a summary drops β git SHAs, UUIDs, ticket codes, versions, large ids β ride along in every dense summary (`ids_preserved:` / `"ids":[...]`). Deterministic, budget-capped (16 facts / 256 chars), so lossy summarization never silently loses a hash or ticket number. |
| **Net-win gate** | The `# squeez` header itself costs ~15β25 tokens. When applied compression saves less than `net_win_min_tokens` (default 24), the call becomes a verbatim passthrough with zero savings recorded β no net-loss invocations. |
| **Adaptive intensity** | Truly adaptive: **Full** (Γ0.6 limits) below 80% of token budget, **Ultra** (Γ0.3) above. Used to be always-Ultra; now actually responds to session pressure. |
| **MCP server** | `squeez mcp` runs a JSON-RPC 2.0 server over stdio exposing 17 tools (16 read-only session-memory queries + `squeez_retrieve` to expand a compressed output) so any MCP-compatible LLM can query session memory directly. Hand-rolled, no `mcp.server` dependency. |
| **Built-in filter pack** | 41 shipped filter-DSL rules (`assets/filters_builtin.ini`) covering the long tail with no dedicated handler β `pip`/`uv`/`poetry`, `bundle`/`composer`/`rubocop`/`phpstan`, `dotnet`/`swift`/`gcc`, `mypy`/`basedpyright`/`golangci-lint`, `shellcheck`/`hadolint`/`yamllint`/`markdownlint`/`pre-commit`, `systemctl`/`journalctl`/`rsync`/`ping`/`ansible-playbook`, and more. Every rule carries an inline self-test that `squeez filter-test` and CI gate on. A project or user rule of the same name shadows a built-in; `builtin_filters = false` turns the pack off. |
| **`git status` porcelain** | `git status` is re-run as `git status --porcelain=v1 -b` and rendered as a working-tree summary (branch, ahead/behind, staged/unstaged/untracked/conflicted). Exact paths are preserved, never collapsed to counts, and raw porcelain codes never reach the model. In-progress rebase/merge/bisect/cherry-pick β which porcelain omits β is read from `.git/` markers rather than a second `git status` run. Yields to any output flag you passed yourself; degrades to raw output on an unrecognized shape. |
| **Preservation guard** | On calls that reduce by β₯90%, the surviving fraction of navigation anchors (file paths, `file:line` refs, error markers, test verdicts) is scored at wrap time. Below `preservation_floor` (0.70) the verbatim original is stashed even outside the usual size gates and the header carries `[anchors: N%]` β so over-compression costs a marker, not a re-investigation. |
| **Config CLI + `/squeez`** | `squeez config get/set/list/reset/path` reads and writes `config.ini` safely (schema validation, comment-preserving writes). `squeez setup` installs a `/squeez` slash command that drives it in natural language from inside the session. |
| **Post-compact re-injection** | After `/compact`, the `SessionStart` hook (`source: compact`) re-injects squeez's tracked session state (recent files, error snippets, git refs, retrievable blob ids), capped at 4000 chars β so concrete state survives compaction instead of being re-discovered. |
| **Bash-wrap safety** | Risky commands (`rm -rf`, `git push --force`, `npm publish`, β¦ β configurable `bash_risk_patterns`) and bypassed commands run **unwrapped**, so the host's native permission rules evaluate the original command. `wrap_bash = false` disables wrapping entirely. See [SECURITY.md](SECURITY.md). |
| **Token estimate** | Compression-timing decisions use a content-class calibrated estimate: output is classified Dense/Prose/Mixed and counted at chars/2.0, chars/3.7, or the code- and CJK-aware char-class estimator β dense tool output really runs ~1.9 chars/token in production, not chars/4. Flat legacy path stays available via `class_density = false`. |
| **Auto-teach payload** | `squeez protocol` (or the `squeez_protocol` MCP tool) prints a 2.4 KB self-describing payload β the LLM learns squeez's markers and protocol on first call. |
| **Caveman persona** | Injects an ultra-terse prompt at session start so the model responds with fewer tokens. |
| **ADHD focus mode** | `focus = adhd` shapes output *structure* rather than length: next action first, numbered steps, one-line state restatement, advisories capped at 5, `squeez doctor` sorted failures-first with a single closing command. Stacks with any persona. |
| **Memory-file compression** | `squeez compress-md` compresses CLAUDE.md / AGENTS.md / copilot-instructions.md in-place β pure Rust, zero LLM. i18n-aware: set `lang = pt` (or `--lang pt`) for pt-BR article/filler/phrase dropping and Unicode-correct matching. |
| **Session memory** | On `SessionStart`, injects a structured summary of the previous session: files investigated, learned facts (errors + git events), completed work (builds, test passes), and next steps (unresolved errors, failing tests). Summaries carry temporal validity (`valid_from`/`valid_to`). |
| **Token tracking** | Every `PostToolUse` result (Bash, Read, Grep, Glob, Monitor, SubagentStop) feeds a `SessionContext` so squeez knows what the agent has already seen. Read/Grep/Glob/Monitor outputs are also rewritten via `updatedToolOutput` (Claude Code v2.1.119+) when content is redundant or oversized. |
| **Token economy** | Sub-agent cost tracking (~200K tokens/spawn), burn rate prediction (`[budget: ~N calls left]`), session efficiency scoring, tool result size budgets. |
| **Auto-calibration** | `squeez calibrate` runs benchmarks on install and generates an optimized `config.ini` (aggressive / balanced / conservative profiles). |
---
## How squeez compares
There are now several token-reduction tools targeting AI coding CLIs. They make different bets β the right one depends on what you care about: zero deps, lossless filtering, structural reformatting, or task-conditioned ML.
| Tool | Approach | Hosts | Deps | Key wins | Trade-off |
|------|----------|-------|------|----------|-----------|
| **squeez** (this project) | Hook + filter pipeline + context engine (MinHash dedup, log-template, relevance truncation, summarize + identifier factsheet, adaptive intensity, net-win gate) + **reversible compression** (retrieve) + MCP server | Claude Code, Copilot CLI, OpenCode, Gemini CLI, Codex CLI, Pi, Hermes | **Zero runtime deps** (`libc` only on Unix) | Up to 95% on bash; cross-call dedup; reversible `squeez_retrieve`; exact ids (SHAs/UUIDs/tickets) survive every summary; signature-mode for source files; TOON re-encoder (incl. nested JSON); 17 MCP tools; post-compact state re-injection; cache-aware savings proof; enterprise (Bedrock/Vertex) USD-saved estimate | Heuristic, not ML β no per-task understanding |
| [chopratejas/headroom](https://github.com/chopratejas/headroom) | Library + **HTTP proxy** + MCP; compresses tool output, logs, RAG chunks **and conversation history** at the API layer with real tokenizers and ML (Kompress/Magika) | Any (OpenAI/Anthropic/Bedrock/Vertex via proxy) | Python + Rust; PyTorch/HF models | 60-95%; reaches conversation history (the biggest sink) via the proxy; reversible CCR; image compression | Heavier (proxy + ML deps); not a zero-dep drop-in hook. squeez adopts its reversible-retrieve and post-compact ideas within the zero-dep hook model. |
| [teamchong/pxpipe](https://github.com/teamchong/pxpipe) | **Local API proxy** that renders bulky request context (system prompt, tool docs, old history, big tool_results) as dense **PNG images** β image tokens are priced by pixel area, so dense text packs ~3.1 chars/image-token vs ~1 as text | Claude Code (`/v1/messages`), Codex (`/v1/responses`) | Node/pnpm; canvas at build time | ~59β70% end-to-end bill cut on Fable 5 traffic; reaches surfaces hooks can't (system prompt, tool docs, history); rigorous per-request `count_tokens` counterfactual measurement | **Lossy**: exact-string recall from images fails on most models (0/15 verbatim on Opus 4.8; silent confabulation, not errors) β default scope is Fable 5 only. Complementary to squeez, not competing: squeez compresses losslessly at the hook layer *before* content enters context; pxpipe cheapens what remains on the wire. They stack β squeez v1.35.0's factsheet, class-density and cache-aware accounting are ported from its measurement discipline. |
| [rtk-ai/rtk](https://github.com/rtk-ai/rtk) | Hook proxy that **rewrites bash commands** (`git status` β `rtk git status`), then compresses 100+ command outputs | Claude Code, Cursor | Zero deps (Rust) | 60-90% on 100+ commands; `rtk read -l aggressive` for signature mode | Rewriting a command changes what runs, not just how its output is printed. Compression also has a floor: on low-entropy content there is little to remove, and a transformed format can cost more than it saves. |
| [KRLabsOrg/squeez](https://github.com/KRLabsOrg/squeez) | **Task-conditioned ML** (Qwen 2B / ModernBERT 150M) β pipe tool output + task description, get back only relevant lines | Any (CLI tool) | Python, PyTorch / vLLM server | 92% compression, F1 0.80; task-aware (same log slices kept differently per query) | Requires running an LLM locally; not zero-dep. Same project name, different design. |
| [ojuschugh1/sqz](https://github.com/ojuschugh1/sqz) | CLI context compressor | Any | Python | Single-command compression | Lower coverage than the others. |
| [LLMLingua-2](https://github.com/microsoft/LLMLingua) (Microsoft) | Neural prompt compressor that removes 50-80% of a prompt while preserving meaning | API / library | Python, transformers | Strong on long static prompts | Latency + model dep; not a CLI hook. |
| [TOON](https://github.com/toon-format/toon) | Schema-aware JSON replacement (`users[100]{id,name,role}:`) β ~40% fewer tokens on arrays of uniform objects | Library, not a CLI | TypeScript SDK | Lossless on the right shape; squeez [embeds a TOON encoder](https://github.com/jhonatanjunio/squeez/pull/4) for `gh`/`kubectl`/`aws`/`gcloud`/`az` JSON outputs | Only helps on uniform JSON shapes. |
If you want a CLI hook that just works, never needs a Python runtime, and never silently inflates your output tokens, squeez is the safe default. If you can run an LLM next to your shell and want task-aware filtering, KRLabsOrg/squeez is worth a look as a complement. The two squeez projects share a name but are independent.
---
## Scope & Limits
squeez optimizes what it can reach β the surfaces exposed by each host's hook API. It cannot fix token leaks outside those surfaces.
### Coverage table
| Surface | How | When | Supported hosts |
|---|---|---|---|
| **Bash stdout/stderr** | `PreToolUse` wraps command w/ the filter pipeline (smart-filter β dedup β log-template β relevance-truncation; original stashed for `squeez_retrieve`). Risky/bypassed commands run unwrapped under native permission rules. | Every Bash invocation | all 5 |
| **Read / Grep / Glob limits** | `PreToolUse` injects `limit` / `head_limit` per `read_max_lines` / `grep_max_results` | Every Read/Grep/Glob call | Claude Code, Copilot, OpenCode (hard); Gemini + Codex soft via GEMINI.md / AGENTS.md |
| **Read / Grep / Glob / Monitor output rewrite** | `PostToolUse` runs `squeez compress-output` and returns `updatedToolOutput` when content is redundant or oversized | Claude Code v2.1.119+ | Claude Code |
| **Agent / Task prompt** | `PreToolUse` compresses `tool_input.prompt` (markdown-aware, via `compress-prompt`) | When prompt > `agent_prompt_max_tokens` | Claude Code (postβv1.8.0) |
| **Sub-agent output** | `SubagentStop` hook feeds `last_assistant_message` into SessionContext for cross-call dedup | On every sub-agent completion | Claude Code |
| **Compaction lifecycle** | `PreCompact` / `PostCompact` log the event; `SessionStart` (`source: compact`) re-injects tracked session state (files, errors, git refs, retrievable blob ids) so it survives compaction | On context compaction | Claude Code |
| **Session memory** | `SessionStart` injects prior session summary + file-access cache | Once per session start | all 5 |
| **Markdown viewing** | Bash handler routes `.md` reads through `compress-md` when `auto_compress_md=true` | Viewer commands on .md paths | all 5 |
### What squeez CANNOT compress
**Agent/Task returned output.** No hook API surface exists to rewrite an Agent's return value. `PostToolUse updatedToolOutput` (Claude Code v2.1.119+) covers built-in tools (Read, Grep, Glob, Monitor) but not the Agent/Task result. Workaround: keep agent prompts compact (squeez compresses at dispatch time via PreToolUse), and use `squeez_agent_costs` MCP tool to monitor spawn overhead.
**Skills & slash-command files.** Claude Code loads these into the system prompt before any hook fires. squeez has no visibility into session-start system prompt construction.
**User's top-level prompt.** squeez runs per tool call, not on user turns.
**Tools whose host doesn't expose PreToolUse / BeforeTool.** E.g. Codex `apply_patch` hooks landed in 0.123.0, but `updatedInput` is explicitly unsupported and `read_file`/`grep` still have no hook surface ([openai/codex#18491](https://github.com/openai/codex/issues/18491)) β so Read/Grep caps for Codex are soft hints in AGENTS.md, not hard injections.
### Secondary wins (not compression, but token-saving)
- **Cross-call redundancy dedup** β exact-hash and fuzzy-trigram collapsing across 16 recent calls (see [Context engine](#what-it-does))
- **Skill re-injection dedup** β when the same skill body is injected by the Skill tool more than once in a session, the repeat collapses to `[squeez: identical to Skill #N]`. Keyed by body hash in a session-long store (not the 16-call window), so it fires even when injections recur far apart
- **File-access cache** β subsequent Bash commands trimmed when re-reading a file squeez has already fingerprinted
- **Burn-rate warnings** β `[budget: ~N calls left]` nudges so the user changes behavior before context pressure spikes
### Reducing overall session cost
squeez cannot automate these, but you can:
- Fewer Agent/Task dispatches per session β use `squeez_agent_costs` to track, then refactor tasks to batch work
- Smaller prompts injected into agents β squeez compresses them at dispatch, but smaller is better
- Shorter CLAUDE.md / AGENTS.md files β run `squeez compress-md --ultra` to drop abbreviations and filler
---
## Benchmarks
Measured on macOS (Apple Silicon). Token count = `chars / 4` (matches Claude's ~4 chars/token). Run `squeez benchmark` to reproduce.
### Per-scenario results β 46 scenarios Γ 5 iterations
| Scenario | Before | After | Reduction | Latency |
|----------|--------|-------|-----------|---------|
| `curl_json` | 18,904 tk | 36 tk | **-100%** | 623 Β΅s |
| `az_json` | 23,479 tk | 74 tk | **-100%** | 349 Β΅s |
| `summarize_huge` | 82,257 tk | 467 tk | **-99%** | 73.0 ms |
| `xcode_build` | 1,881 tk | 17 tk | **-99%** | 182 Β΅s |
| `go_test_ndjson_failures` | 7,176 tk | 106 tk | **-99%** | 700 Β΅s |
| `read_reread_distant` | 717 tk | 17 tk | **-98%** | 8.8 ms |
| `rsync_transfer` | 912 tk | 26 tk | **-97%** | 128 Β΅s |
| `repetitive_output` | 4,692 tk | 134 tk | **-97%** | 236 Β΅s |
| `pytest_failures` | 3,402 tk | 108 tk | **-97%** | 180 Β΅s |
| `jest_json_failures` | 5,643 tk | 218 tk | **-96%** | 472 Β΅s |
| `systemctl_status` | 732 tk | 41 tk | **-94%** | 124 Β΅s |
| `ps_aux` | 40,373 tk | 2,338 tk | **-94%** | 619 Β΅s |
| `cargo_test_failures` | 1,934 tk | 157 tk | **-92%** | 145 Β΅s |
| `git_log_200` | 2,692 tk | 275 tk | **-90%** | 264 Β΅s |
| `tsc_errors` | 731 tk | 101 tk | **-86%** | 137 Β΅s |
| `eslint_json_failures` | 1,553 tk | 233 tk | **-85%** | 239 Β΅s |
| `pip_install` | 407 tk | 62 tk | **-85%** | 117 Β΅s |
| `high_context_adaptive` | 4,418 tk | 729 tk | **-84%** | 918 Β΅s |
| `cargo_build_noisy` | 2,106 tk | 439 tk | **-79%** | 283 Β΅s |
| `bundle_install` | 121 tk | 28 tk | **-77%** | 104 Β΅s |
| `docker_logs` | 665 tk | 181 tk | **-73%** | 143 Β΅s |
| `curl_html_response` | 2,181 tk | 626 tk | **-71%** | 147 Β΅s |
| `git_status` | 50 tk | 16 tk | **-68%** | 114 Β΅s |
| `ruff_json_failures` | 1,261 tk | 494 tk | **-61%** | 230 Β΅s |
| `verbose_app_log` | 4,957 tk | 1,978 tk | **-60%** | 488 Β΅s |
| `npm_install` | 524 tk | 218 tk | **-58%** | 142 Β΅s |
| `crosscall_redundancy_3x` | 486 tk | 222 tk | **-54%** | 51.9 ms |
| `ls_la` | 1,782 tk | 872 tk | **-51%** | 159 Β΅s |
| `mypy_errors` | 650 tk | 349 tk | **-46%** | 117 Β΅s |
| `shellcheck_run` | 335 tk | 187 tk | **-44%** | 101 Β΅s |
| `agent_directory_output` | 3,348 tk | 1,937 tk | **-42%** | 575 Β΅s |
| `env_dump` | 441 tk | 287 tk | **-35%** | 115 Β΅s |
| `agent_heavy` | 2,306 tk | 1,514 tk | **-34%** | 412 Β΅s |
| `git_copilot` | 640 tk | 421 tk | **-34%** | 173 Β΅s |
| `find_deep` | 424 tk | 279 tk | **-34%** | 124 Β΅s |
| `adversarial_tiny_output` | 4 tk | 3 tk | **-25%** | 94 Β΅s |
| `md_prose` | 187 tk | 142 tk | **-24%** | 76 Β΅s |
| `md_claude_md` | 316 tk | 270 tk | **-15%** | 152 Β΅s |
| `claude_md_overhead` | 717 tk | 635 tk | **-11%** | 109 Β΅s |
| `next_build_output` | 902 tk | 884 tk | **-2%** | 145 Β΅s |
| `git_diff` | 502 tk | 497 tk | **-1%** | 139 Β΅s |
| `jest_failures` | 451 tk | 448 tk | **-1%** | 126 Β΅s |
| `state_first_simulation` | 182 tk | 181 tk | **-1%** | 97 Β΅s |
| `kubectl_pods` | 1,513 tk | 1,513 tk | **-0%** | 131 Β΅s |
| `adversarial_dense_json` | 485 tk | 485 tk | **-0%** | 107 Β΅s |
| `adversarial_reread_tiny` | 4 tk | 4 tk | **-0%** | 6.0 ms |
### Aggregate
| Metric | Value |
|--------|-------|
| **Total token reduction** | **91.2%** β 229,443 tk β 20,249 tk |
| Bash output | **-88.7%** |
| Markdown / context files | **-18.1%** |
| Wrap / cross-call engine | **-99.2%** |
| Quality (signal terms preserved) | **46 / 46 pass** |
| Latency p50 (filter mode) | **3.3 ms** |
| Latency p95 (incl. wrap/summarize) | **9 ms** |
### Estimated cost savings β Claude Sonnet 4.6 Β· $3.00 / MTok input
| Usage | Baseline / month | Saved / month |
|-------|-----------------|---------------|
| 100 calls / day | $18.00 | **$16.41 (91%)** |
| 1,000 calls / day | $180.00 | **$164.11 (91%)** |
| 10,000 calls / day | $1800.00 | **$1641.06 (91%)** |
### Independently verified with a real tokenizer
The table above uses `chars / 4` for reproducibility (no vocab, no deps). A fair
objection: *"that's a made-up unit β show me the reduction under a real byte-pair
tokenizer."* So `bench/verify_tokens.py` re-tokenizes every fixture β the raw
output **and** the squeez-compressed output β with `cl100k_base` (the real GPT-4
family BPE tokenizer, via `tiktoken`) and compares against `chars/4`:
| Model | Aggregate reduction (22 filter/markdown fixtures) |
|-------|--------------------------------------------------|
| **Real BPE (`cl100k_base`)** | **83.5%** β 112,557 tk β 18,590 tk |
| `chars / 4` (benchmark unit) | 83.0% |
| **Divergence** | **0.5 pts** β the reported reduction is *not* a token-model artifact |
Reduction ratios are near model-invariant, so a real tokenizer confirms the
claim rather than inflating it. Reproduce end-to-end:
```bash
cargo build --release
pip install tiktoken
python3 bench/verify_tokens.py # human-readable table
python3 bench/verify_tokens.py --json # machine-readable (see bench/verify_tokens.json)
```
Nothing is cherry-picked: fixtures that squeez can't help (e.g. `kubectl_pods`,
a 61-line output below the truncation threshold) stay in the aggregate at 0%.
---
## Commands
```bash
squeez wrap # compress a command's output end-to-end
squeez filter # compress stdin (piped usage)
squeez config # inspect/change config.ini (also via the /squeez skill)
squeez compress-md [--ultra] [--dry-run] [--all] ... # compress markdown files
squeez benchmark [--json] [--output ] [--scenario ] [--iterations ]
squeez mcp # JSON-RPC 2.0 MCP server over stdin/stdout
squeez protocol # print the auto-teach payload (markers + protocol)
squeez update [--check] [--insecure] # self-update
squeez init [--copilot] # session-start hook (called by hook, not manually)
squeez calibrate # auto-tune config from benchmarks
squeez budget-params # output JSON budget patch for tool
squeez compact-summary [--session-start] # re-inject session state after /compact (called by SessionStart hook)
squeez --version
```
### Escape hatch β bypass compression for one command
```bash
--no-squeez git log --all --graph
```
Prefix any command with `--no-squeez` to run it raw without squeez touching it.
### `squeez wrap`
Runs a command, compresses its output, and prints a savings header:
```
# squeez [git log] 2692β289 tokens (-89%) 0.2ms [adaptive: Ultra]
```
`wrap` re-executes the command through a shell: `sh -c` on Unix, and on
Windows `bash -c`, then `sh -c`, when one of them resolves on `PATH` β every
agent host writes its terminal commands for bash there, so re-running them
under `cmd.exe` corrupts quoting, `$(β¦)`, backticks and `;`. `cmd /C` remains
the fallback when no POSIX shell is present, and is also retried automatically
if the preferred shell cannot be spawned. `%SystemRoot%\System32\bash.exe` is
skipped: that is the WSL launcher, which would run the command inside a Linux
distro rather than against the host filesystem. `CLAUDE_CODE_GIT_BASH_PATH` is
honoured when set. Set `SQUEEZ_SHELL` to force a specific shell:
```bash
SQUEEZ_SHELL="C:/Program Files/Git/bin/bash.exe" squeez wrap 'ls -la'
```
### `squeez filter`
Reads from stdin. Use for manual pipelines:
```bash
git log --oneline | squeez filter git
docker logs mycontainer 2>&1 | squeez filter docker
```
### `squeez compress-md`
Pure-Rust, zero-LLM compressor for markdown files. Preserves code blocks, inline code, URLs, headings, file paths, and tables. Compresses prose only. Always writes a backup at `.original.md`.
```bash
squeez compress-md CLAUDE.md # Full mode (English default)
squeez compress-md --ultra CLAUDE.md # + abbreviations (withβw/, fn, cfg, etc.)
squeez compress-md --lang pt CLAUDE.md # pt-BR locale (articles, fillers, phrases)
squeez compress-md --dry-run CLAUDE.md # preview, no write
squeez compress-md --all # compress all known locations automatically
```
When `auto_compress_md = true` (default), `squeez init` runs `--all` silently on every session start.
### `squeez benchmark`
Reproducible measurement of token reduction, cost, latency, and quality across 19 scenarios:
```bash
squeez benchmark # human-readable report
squeez benchmark --json # JSON to stdout
squeez benchmark --output report.json # save JSON report
squeez benchmark --scenario git # run only git scenarios
squeez benchmark --iterations 5 # more iterations per scenario
squeez benchmark --list # list all scenarios
```
Quality is scored by checking that **signal terms** (words from error/warning/failed lines in the baseline) survive compression. 19/19 pass at β₯ 50% threshold.
`squeez benchmark --efficiency-proof` additionally reports **cache-aware effective costs**: list-price ratios (input Γ1.0, cache write Γ1.25, cache read Γ0.1, output Γ5) applied identically to both sides of every row under the same cache state β so the provider's caching discount is never miscounted as compression savings. Negative savings are representable, never floored; JSON output is `schema_version: 2`. Savings are reported per compressed slice; end-to-end depends on workload.
### `squeez mcp`
Runs a Model Context Protocol JSON-RPC 2.0 server over stdin/stdout. Hand-rolled, no `mcp.server` / `fastmcp` dependency β keeps the `libc`-only constraint intact. Wire it into Claude Code:
```bash
claude mcp add squeez -- /path/to/squeez mcp
```
Eighteen read-only tools become available to the LLM:
| Tool | Returns |
|------|---------|
| `squeez_recent_calls` | Last N bash invocations with hash + length + cmd snippet β check before re-running |
| `squeez_seen_files` | Files this session has touched, with the call number where each was last seen, sorted by recency |
| `squeez_seen_errors` | Distinct error fingerprints observed this session (FNV-1a hashes of normalized errors) |
| `squeez_seen_error_details` | Error fingerprints with the first 128 chars of message text β find *what* the error was |
| `squeez_session_summary` | Token accounting + call counts (tokens_bash / tokens_read / tokens_other / seen_files / seen_errors / seen_git_refs) |
| `squeez_session_stats` | Dedup hit counts (exact + fuzzy), summarize triggers, Ultra-mode calls, tokens saved per category |
| `squeez_agent_costs` | Sub-agent usage: spawn count, cumulative estimated tokens, per-call breakdown |
| `squeez_session_efficiency` | Session efficiency scores: compression ratio, tool choice, context reuse, budget conservation (basis points) |
| `squeez_prior_summaries` | Last N finalized prior-session summaries with structured fields: investigated / learned / completed / next_steps |
| `squeez_search_history` | Full-text search across all session summaries β find when you last saw an error or touched a file |
| `squeez_file_history` | Sessions where a given file path was touched, with token-savings and commit status |
| `squeez_session_detail` | Full structured view of a past session by date: calls, files, errors, git events, test summary |
| `squeez_protocol` | Auto-teach payload β read once per session to learn squeez's markers + memory protocol |
| `squeez_context_pressure` | Budget used %, calls remaining, tokens saved, and a recommendation (ok / compact_soon / use_state_first) |
| `squeez_handler_stats` | Per-handler cumulative compression across sessions; flags under- (<10%) and over-performers (β₯90%) at β₯5 calls |
| `squeez_enterprise_savings` | Enterprise transport mode (Bedrock/Vertex/OTEL) + USD saved this session at public Sonnet rates |
| `squeez_retrieve` | Expands a stashed output back to its verbatim original by `key`; optional `start_line`/`count` to page a slice |
| `squeez_stash_search` | Finds a stashed output by meaning when the key is lost β term overlap + fuzzy shingle match, returns key + preview |
All read-only. Backed by `SessionContext::load()`, `memory::read_last_n()`, and `memory::search_history()`. No side effects.
### `squeez protocol`
Prints the auto-teach payload β a 2.4 KB self-describing block covering:
- The 5-rule **memory protocol** (what to do with `[squeez: ...]` markers, when to call the MCP tools)
- The **output marker spec** (`# squeez [...]`, `[squeez: identical to ...]`, `[squeez: ~95% similar to ...]`, `squeez:summary`, `# squeez hint:`)
Same content the MCP `squeez_protocol` tool returns. Pipe it into a `system` prompt or paste it into a one-shot session that doesn't have the MCP server connected.
---
## Configuration
Optional config file β all fields have defaults, none are required.
| Platform | Config path |
|----------|------------|
| Claude Code / default | `~/.claude/squeez/config.ini` |
| Copilot CLI | `~/.copilot/squeez/config.ini` |
```ini
# ββ Compression ββββββββββββββββββββββββββββββββββββββββββββββββ
max_lines = 200 # generic truncation limit
dedup_min = 3 # collapse lines appearing β₯N times
git_log_max_commits = 20
git_diff_max_lines = 150
docker_logs_max_lines = 100
find_max_results = 50
bypass = docker exec, psql, mysql, ssh # never compress these
# ββ Context engine βββββββββββββββββββββββββββββββββββββββββββββ
adaptive_intensity = true # truly adaptive: Full <80% budget, Ultra β₯80%
context_cache_enabled = true # track seen files/errors across calls
redundancy_cache_enabled = true # collapse identical OR fuzzy-similar recent outputs
read_dedup_session_long = true # also collapse a re-read of an unchanged file past the 16-call window
summarize_threshold_lines = 500 # outputs above this trigger summarize fallback (Γ2 if benign)
compact_threshold_tokens = 120000 # session token budget β drives adaptive intensity
# ββ Session memory βββββββββββββββββββββββββββββββββββββββββββββ
memory_retention_days = 30
# ββ Output / persona βββββββββββββββββββββββββββββββββββββββββββ
persona = ultra # off | lite | full | ultra
focus = off # off | adhd β output *structure*, orthogonal to persona
auto_compress_md = true # run compress-md on every session start
lang = en # compress-md locale: en | pt (pt-BR) β more languages extensible
# ββ Advanced tuning (rarely needed) βββββββββββββββββββββββββββ
max_call_log = 32 # rolling call log depth (also caps redundancy window)
recent_window = 16 # how many recent calls are eligible for redundancy lookup
similarity_threshold = 0.85 # Jaccard threshold for fuzzy dedup (0.0β1.0)
ultra_trigger_pct = 0.80 # fraction of context budget at which Full β Ultra
class_density = true # content-class token estimate: dense chars/2.0, prose chars/3.7 (false = legacy flat path)
net_win_min_tokens = 24 # skip compression + header when it saves less than this (0 = gate off)
mcp_prior_summaries_default = 5 # default n for squeez_prior_summaries
mcp_recent_calls_default = 10 # default n for squeez_recent_calls
# ββ Token economy βββββββββββββββββββββββββββββββββββββββββββββ
agent_warn_threshold_pct = 0.50 # warn when agent cost > 50% of budget
burn_rate_warn_calls = 20 # warn when < 20 calls remaining
agent_spawn_cost = 200000 # estimated tokens per Agent/Task spawn
read_max_lines = 0 # max lines injected into Read tool_input (0 = off)
grep_max_results = 0 # max results injected into Grep tool_input (0 = off)
# ββ Auto-curation nudges ββββββββββββββββββββββββββββββββββββββ
nudge_enabled = true # emit [squeez: hint ...] markers on recurring patterns
nudge_error_threshold = 3 # fingerprint repeats before a nudge fires
nudge_file_mod_threshold = 5 # writes/creates to same path before nudge fires
nudge_cmd_repeat_threshold = 4 # expensive-command repeats before nudge fires
# ββ Continuous handler calibration ββββββββββββββββββββββββββββ
handler_stats_enabled = true # accumulate per-handler savings across sessions
```
### Adaptive intensity β Full / Ultra split
When `adaptive_intensity = true` (default), squeez **actually adapts** to session pressure rather than always running Ultra:
| Used / budget | Tier | Scaling |
|---|---|---|
| `< 80%` | **Full** | Γ0.6 limits, dedup_min Γ0.66 (floor 2) |
| `β₯ 80%` | **Ultra** | Γ0.3 limits, dedup_min Γ0.5 (floor 2) |
| `adaptive_intensity = false` | **Lite** | passthrough β no scaling |
Floors are enforced so we never reduce to zero: `max_lines β₯ 20`, `git_diff_max_lines β₯ 20`, `dedup_min β₯ 2`, `summarize_threshold_lines β₯ 50`.
The active level is shown in every bash header: `[adaptive: Full]` or `[adaptive: Ultra]`.
Pre-0.3 squeez was effectively always-Ultra. The new behavior preserves more verbatim text in the common case (empty / mid-session) and only graduates to aggressive compression when the context budget is genuinely under pressure.
### Caveman persona
Three intensity levels (`lite`, `full`, `ultra`) and `off`. Default is `ultra`. The persona prompt is injected into:
- The Claude Code session banner (printed at `SessionStart`)
- The `β¦` block in `~/.copilot/copilot-instructions.md` for Copilot CLI
### ADHD focus mode
`focus = adhd` is a second, independent axis. Persona decides *how terse* the
prose is; focus decides *how it is ordered*. They stack β `persona = ultra` plus
`focus = adhd` keeps maximum compression while forcing action-first structure.
Turning it on changes three surfaces:
1. **The model's prose** β a 10-rule block (EN or pt-BR, per `lang`) rides
alongside the persona block: lead with the next action, number multi-step
work, restate state in one line, concrete time estimates, cap lists at 5, no
preamble or closing pleasantries.
2. **The session banner** β the pending next step becomes line 1, remaining
steps are numbered, one prior session is shown instead of three, and the
stats line sinks to the bottom (state, not an action).
3. **Advisories and doctor** β `[squeez: β¦]` bursts are capped at 5 per command
with an honest `+N more advisories suppressed` tail, and `squeez doctor`
sorts failures first and ends with one command to run.
```bash
squeez config set focus adhd # then `squeez init` to rewrite the memory block
squeez config set focus off # back to the default ordering
```
The ruleset is adapted from the MIT-licensed
[i-have-adhd](https://github.com/ayghri/i-have-adhd) skill by ayghri.
---
## How it works
### Compression pipeline
Each bash command passes through four strategies in order:
1. **smart_filter** β strips ANSI codes, progress bars, spinner chars, timestamps, and tool-specific noise (npm download lines, stack frame noise, etc.)
2. **dedup** β lines appearing β₯ `dedup_min` times are collapsed to one entry annotated `[ΓN]`
3. **grouping** β files in the same directory (β₯5 siblings) are collapsed to `dir/ N modified [squeez grouped]`
4. **truncation** β `Head` (keep first N) or `Tail` (keep last N) depending on handler; truncated portion noted
### Supported handlers
| Category | Commands |
|----------|----------|
| Git | `git` |
| Docker / containers | `docker`, `docker-compose`, `podman` |
| Package managers | `npm`, `pnpm`, `bun`, `yarn` |
| Build systems | `make`, `cmake`, `gradle`, `mvn`, `xcodebuild`, `cargo` (build), `next build/dev/start` |
| Test runners | `cargo test`, `jest`, `vitest`, `pytest`, `nextest`, `playwright`, `bun test` |
| TypeScript / linters | `tsc`, `eslint`, `biome` |
| Cloud CLIs | `kubectl`, `gh`, `aws`, `gcloud`, `az`, `wrangler` |
| Databases | `psql`, `prisma`, `mysql`, `drizzle-kit` |
| Filesystem | `find`, `ls`, `du`, `ps`, `env`, `lsof`, `netstat` |
| JSON / YAML / IaC | `jq`, `yq`, `terraform`, `tofu`, `helm`, `pulumi` |
| Text processing | `grep`, `rg`, `awk`, `sed` |
| Network | `curl`, `wget` |
| Runtimes | `node`, `python`, `ruby` |
| Generic fallback | everything else |
### Hooks (Claude Code & Copilot CLI)
Six hooks work together automatically after install on Claude Code (three on Copilot CLI):
- **`PreToolUse`** β rewrites safe Bash calls: `git status` β `squeez wrap git status` (risky/bypassed commands pass through to native permission rules); injects Read/Grep/Glob limits; compresses Agent/Task prompts
- **`SessionStart`** β runs `squeez init`: finalizes previous session into a memory summary, injects the persona prompt
- **`PostToolUse`** β tracks every tool result; rewrites Read/Grep/Glob/Monitor output via `updatedToolOutput` when content is redundant or oversized (Claude Code v2.1.119+)
- **`SubagentStop`** *(Claude Code only)* β feeds `last_assistant_message` into SessionContext so the parent agent can dedup against what the sub-agent saw
- **`PreCompact`** *(Claude Code only)* β logs compaction events for session efficiency metrics; allows compaction to proceed
- **`PostCompact`** *(Claude Code only)* β logs the compaction. Session state (files, errors, git refs, retrievable blob ids) is re-injected by `SessionStart` with `source: compact` (`squeez compact-summary --session-start`), since Claude Code does not deliver PostCompact output to the model
### Cross-call redundancy
Two-path dedup across the last 16 calls:
**Exact match** β FNV-1a hash of the compressed output. When a subsequent call produces the same bytes, it collapses to:
```
[squeez: identical to 515ba5b2 at bash#35 β re-run with --no-squeez]
```
**Fuzzy match** β bottom-k MinHash over whitespace-token trigrams (k=96, Jaccard β₯ 0.85, length-ratio guard β₯ 0.80). Survives timestamp changes, added/removed blank lines, and single-line edits. Collapses to:
```
[squeez: ~92% similar to 515ba5b2 at bash#35 β re-run with --no-squeez]
```
Minimum 6 lines to attempt fuzzy match (below that, exact-only).
### Summarize fallback
When raw output exceeds `summarize_threshold_lines` (default 500), the full pipeline is bypassed and replaced with a β€40-line dense summary:
```
squeez:summary cmd=docker logs app
total_lines=5003
top_errors:
- error: connection refused on tcp://10.0.0.1:5432
top_files:
- /var/log/app/error.log
test_summary=FAILED: 3 of 248
ids_preserved:
- 3f2ec81ea3e4ce24
- 550e8400-e29b-41d4-a716-446655440000
- PROJ-1482
tail_preserved=20
[last 20 lines verbatim...]
```
**Identifier factsheet:** the `ids_preserved:` block carries exact identifiers found in the region the summary drops β git SHAs / hex ids (β₯7 chars with a digit), UUIDs, ticket codes, versions, and large integers β so the most dangerous failure mode of lossy summarization (a silently lost hash) can't happen. Extraction is deterministic and budget-capped at 16 facts / 256 chars; bulk generated sequences (hundreds of version strings in a build log) are recognized as noise and dropped wholesale while opaque ids survive. The Structured summary shape carries the same list as `"ids":[...]`.
**Benign-aware threshold:** before summarizing, squeez scans for error markers (`error:`, `panic`, `traceback`, `FAILED`, `EXCEPTION`, `Fatal`). If none are found, the threshold is doubled (1,000 lines default) so successful builds, clean test runs, and uneventful logs stay verbatim unless they are genuinely huge.
---
## Platform notes
### OpenCode
Plugin installed at `~/.config/opencode/plugins/squeez.js`. OpenCode auto-loads plugins on startup. All Bash commands are automatically compressed via `squeez wrap`.
### Hermes
Directory plugin installed at `$HERMES_HOME/plugins/squeez-fallback/` β
`~/.hermes/plugins/` when `HERMES_HOME` is unset (a leading `~` is expanded).
Set `HERMES_HOME` if Hermes lives elsewhere (e.g. `%LOCALAPPDATA%\hermes` on
Windows), otherwise
`squeez setup --host=hermes` will not detect it. Directory plugins are opt-in;
enable it once with `hermes plugins enable squeez-fallback`.
### GitHub Copilot CLI
Hooks registered in `~/.copilot/settings.json`. Session memory written to `~/.copilot/copilot-instructions.md` (Copilot CLI reads this automatically). State stored separately at `~/.copilot/squeez/`.
Refresh memory manually:
```bash
SQUEEZ_DIR=~/.copilot/squeez ~/.claude/squeez/bin/squeez init --copilot
```
### Pi
TypeScript extension installed at `~/.pi/agent/extensions/squeez/index.ts`. Pi auto-discovers extensions from that directory β no settings patching needed. Session memory is injected via a skill at `~/.pi/agent/skills/squeez/SKILL.md`; Pi includes the skill description in every system prompt and loads full instructions on demand. Unlike other hosts, Pi achieves `BUDGET_HARD` output compression via the `tool_result` event (return-patch API), not just soft hints.
---
## Local development
Requires Rust stable. Windows requires Git Bash.
```bash
git clone https://github.com/claudioemmanuel/squeez.git
cd squeez
cargo test # run all tests (356 tests, 37 suites)
cargo build --release # build release binary
bash bench/run.sh # filter-mode benchmark (14 fixtures)
bash bench/run_context.sh # context-engine benchmark (3 wrap scenarios)
./target/release/squeez benchmark # full 19-scenario benchmark suite
bash build.sh # build + install to ~/.claude/squeez/bin/
```
---
## Contributing
```bash
git checkout -b feature/your-change
cargo test
cargo build --release
bash bench/run.sh
git push -u origin feature/your-change
gh pr create --base main --title "Short title" --body "Description"
```
CI runs `cargo test`, `bench/run.sh`, `bench/run_context.sh`, and `squeez benchmark` on every push and pull request.
See [CONTRIBUTING.md](CONTRIBUTING.md) for coding standards.
---
## Similar projects
There is an unrelated project with the same name at [KRLabsOrg/squeez](https://github.com/KRLabsOrg/squeez). This project (claudioemmanuel/squeez) is a hook-based token compressor for AI coding CLIs.
---
## License
Licensed under the **Apache License 2.0** β see [LICENSE](LICENSE) + [NOTICE](NOTICE).
Contributions require a DCO sign-off (`git commit -s β¦`) rather than a CLA. You keep copyright on what you contribute; sign-off is a lightweight affirmation that you have the right to submit it under Apache 2.0. See [CONTRIBUTING.md](CONTRIBUTING.md#license--contributor-sign-off) for details.