# Misceo **English** · [简体中文](README.zh-CN.md) [![npm version](https://img.shields.io/npm/v/%40misceo%2Fcli?label=npm)](https://www.npmjs.com/package/@misceo/cli) [![license](https://img.shields.io/badge/license-FSL--1.1--ALv2-green)](LICENSE) ![platforms](https://img.shields.io/badge/platforms-macOS%20%7C%20Linux%20%7C%20Windows-blue) > [!IMPORTANT] > This is the official public documentation, issue tracker, and release home > for Misceo. The product is distributed as prebuilt binaries through npm. > Source code is not included in this repository. **Cut AI-agent costs without breaking conversations or tool loops.** Misceo is a local Anthropic-compatible gateway. It routes eligible requests through a lower-cost model, checks the completed candidate, and escalates to a stronger model when policy requires it. Unlike a request-only router, Misceo also protects agent state: session continuity, cross-model transcript compatibility, and ownership of active tool loops. ## Why Misceo Strong-only routing is simple but pays strong-model cost for every request. Cheap-only routing lowers spend but cannot recover when the answer is empty, malformed, or inadequate. A static router decides before generation. Misceo can also inspect the first completed candidate before choosing which answer the client receives. | Strategy | Decision point | Trade-off | |---|---|---| | Strong-only | Before generation | Consistent quality posture; strong-model cost on every request. | | Cheap-only | Before generation | Lowest routing overhead; no automatic recovery. | | Static router | Before generation | Uses request metadata but cannot inspect the generated answer. | | Misceo | Before and after the first attempt | Accepts a checked candidate or regenerates with a stronger model. | Misceo does not make a cheaper model equivalent to a stronger model. It makes the cost-versus-quality policy explicit, observable, and configurable. ## How a request flows ```mermaid flowchart LR A["Anthropic-compatible client"] --> B["Misceo local gateway"] B --> C{"Protected route?"} C -->|"Yes"| S["Strong backend"] C -->|"No"| L["Lower-cost attempt"] L --> G{"Structural gate"} G -->|"Fail"| S G -->|"Pass"| J{"Judge policy"} J -->|"Accept or skip"| R["Return candidate"] J -->|"Reject or fail closed"| S S --> R2["Return regenerated answer"] B --> O["Local history and dashboard"] ``` 1. Protected routes, open tool loops, active pins, and explicit rules run before the cascade. 2. Eligible traffic first reaches the configured lower-cost backend. 3. A structural gate rejects upstream failures, empty output, and malformed tool calls. 4. Depending on the selected mode, an optional judge scores the visible candidate against the latest user turn. 5. A passing candidate is returned. A rejected candidate is discarded and a stronger backend generates the served answer. 6. Escalated conversations remain on the stronger backend for a configurable number of fresh turns. See [Routing and handoffs](docs/routing-and-handoff.md) for decision precedence, session identity, tool-loop protection, and mode-specific failure behavior. ## Agent-safe routing ### Conversation continuity Misceo groups requests by structured session identity when the client provides it. Bootstrap, title-generation, and visible requests from one launch remain part of the same conversation. ### Cross-model handoff Provider-specific thinking signatures, cache markers, model names, and message invariants are handled at model-family boundaries. The original local audit record remains available for inspection. ### Tool-loop ownership When a model opens a tool loop, that backend keeps ownership until the loop is complete. Misceo does not move an active `tool_use` / `tool_result` exchange to another model family halfway through. ### First-visible-reply protection For eligible structured sessions, known launch bootstrap traffic is separated from the user's first visible request. The first visible reply uses the strong backend before later requests enter normal routing. ## Where data goes The proxy, embedded dashboard, configuration, and traffic logs run locally. Local does not mean offline: inference requests go to the providers selected by your routing policy. For an eligible cascade, the first provider receives the request. If the candidate is rejected, the escalation provider also receives the request. A configured judge receives the capped latest user turn, visible candidate, and tool names. Read [Privacy and data](docs/privacy-and-data.md) before using Misceo with sensitive prompts or exposing either listener beyond loopback. ## Highlights - One Anthropic-compatible endpoint for Claude, GLM, and custom backends. - Three explicit postures: `quality-first`, `balanced`, and `savings-first`. - Structural validation plus an optional LLM judge. - Session-aware model-family handoffs and tool-loop protection. - Embedded dashboard with live traffic, history, cost, and deletion controls. - Observable serving backend, judge result, latency, usage, and route reason. - Manual backend handoff from the CLI or dashboard. ## Requirements - Node.js 18 or newer and npm for installation and the npm launcher. - Provider credentials for the backends you enable. - No separate Bun, Docker, or UI installation. Published platform packages currently cover: | Operating system | Architectures | |---|---| | macOS | arm64, x64 | | Linux | glibc arm64, glibc x64 | | Windows | x64 | See [Compatibility](docs/compatibility.md) for client and platform boundaries. ## Start in five minutes Install the CLI: ```bash npm install --global @misceo/cli ``` Do not install with `--omit=optional`; the platform executable is delivered as an optional package. Create a private configuration: ```bash mkdir my-misceo cd my-misceo misceo init --mode balanced ``` Add the provider credentials you use to `.env`: ```dotenv ANTHROPIC_API_KEY=your-anthropic-key ZAI_API_KEY=your-zai-key ``` Validate and start: ```bash misceo doctor misceo start ``` `doctor` performs static checks. It does not contact providers, validate live credentials, or prove model availability. Connect Claude Code on macOS or Linux: ```bash ANTHROPIC_BASE_URL=http://127.0.0.1:4141 \ ANTHROPIC_AUTH_TOKEN=misceo \ claude ``` For Windows PowerShell and a complete verification procedure, read [Getting started](docs/getting-started.md). Default local addresses: - Proxy: `http://127.0.0.1:4141` - Dashboard: `http://127.0.0.1:5141` ## Verify routing in 60 seconds Send a normal prompt in Claude Code, followed by a second small request. In an eligible structured session, the first visible response is protected and a later request can exercise the cascade. Open the dashboard and inspect **Live traffic**. Confirm the serving backend, route or cascade note, status, latency, and judge score when a judge ran. In another terminal: ```bash curl http://127.0.0.1:4141/healthz misceo status ``` A healthy response, a traffic row, and a visible session confirm that the client is using Misceo instead of contacting a provider directly. ## Routing modes | Mode | Judge behavior | If the judge is unavailable | |---|---|---| | `quality-first` | Required | Escalate to the strong backend. | | `balanced` | Used when configured | Serve a structurally valid lower-cost candidate. This is the default. | | `savings-first` | Skipped | Continue from the structural gate without a judge call. | Select a mode during initialization: ```bash misceo init --mode quality-first misceo init --mode balanced misceo init --mode savings-first ``` ## Dashboard The management UI is embedded in the installed executable. It includes: - live routing state, sessions, handoffs, and pins; - traffic detail with request, response, judge, status, and latency; - token and cost fields with contextual explanations; - searchable, paginated conversation history; - deletion for one conversation or all recorded history. The dashboard has no user login. Keep it on loopback unless trusted authentication and network controls are placed in front of it. Read [Dashboard](docs/dashboard.md) for listener options and data controls. ## Evidence and limitations Misceo does not publish a universal savings or quality percentage. Results vary with tasks, models, providers, prices, region, and sampling behavior. The public benchmark contract compares four policies on the same versioned task set: 1. `strong-only` 2. `cheap-only` 3. `cheap-first + structural gate` 4. `cheap-first + judge` Reports must include task success, quality loss, cost, latency, escalation rate, judge error rates, and handoff retention. No public result is included until the corresponding task set and raw report can be published together. See [Benchmark methodology](docs/benchmark.md). ## CLI quick reference | Command | Purpose | |---|---| | `misceo init` | Create a private starter `.env`. | | `misceo doctor` | Perform static configuration checks. | | `misceo start` | Start the proxy and embedded dashboard. | | `misceo status` | Show current routing and session state. | | `misceo handoff BACKEND` | Switch the active backend. | | `misceo cost` | Report token cost and configured savings. | | `misceo audit` | Run explicit paid compatibility probes. | | `misceo logs` | Prune, reindex, or forget local history. | Detailed flags and configuration examples are in [Getting started](docs/getting-started.md) and [Configuration](docs/configuration.md). ## Documentation | Guide | Purpose | |---|---| | [Getting started](docs/getting-started.md) | Install, configure, connect, verify, update, and uninstall. | | [Routing and handoffs](docs/routing-and-handoff.md) | Cascade decisions, sessions, judges, and tool loops. | | [Configuration](docs/configuration.md) | Environment variables, endpoints, pricing, and listeners. | | [Compatibility](docs/compatibility.md) | Supported platforms, clients, and protocol boundaries. | | [Dashboard](docs/dashboard.md) | UI behavior, history, cost fields, and exposure safety. | | [Privacy and data](docs/privacy-and-data.md) | Provider flow, local logs, retention, and deletion. | | [Troubleshooting](docs/troubleshooting.md) | Common installation, endpoint, port, and history problems. | | [Benchmark methodology](docs/benchmark.md) | Reproducibility rules, metrics, and claim boundaries. | ## Source availability, license, and support This repository does not contain Misceo source code. Published binaries are licensed under the [Functional Source License 1.1 with the Apache 2.0 Future License](LICENSE). Review the license before redistribution or competing use. Use [GitHub Issues](https://github.com/MaySudo/Misceo/issues) for reproducible bugs and documentation problems. Read [SUPPORT.md](SUPPORT.md) before filing. Report security vulnerabilities privately as described in [SECURITY.md](SECURITY.md).