# agent-trace

[![PyPI](https://img.shields.io/pypi/v/agent-strace)](https://pypi.org/project/agent-strace/)
[![Python](https://img.shields.io/pypi/pyversions/agent-strace)](https://pypi.org/project/agent-strace/)
[![CI](https://github.com/Siddhant-K-code/agent-trace/actions/workflows/test.yml/badge.svg)](https://github.com/Siddhant-K-code/agent-trace/actions/workflows/test.yml)
[![GitHub Marketplace](https://img.shields.io/badge/Marketplace-agent--trace%20eval-blue?logo=github)](https://github.com/marketplace/actions/agent-trace-eval)
[![Open VSX](https://img.shields.io/open-vsx/v/Siddhant-K-code/agent-strace)](https://open-vsx.org/extension/Siddhant-K-code/agent-strace)
[![VS Marketplace](https://img.shields.io/badge/VS%20Marketplace-v0.3.0-blue?logo=visual-studio-code)](https://marketplace.visualstudio.com/items?itemName=Siddhant-K-code.agent-strace)
[![License](https://img.shields.io/github/license/Siddhant-K-code/agent-trace)](LICENSE)
[![PyPI Downloads](https://static.pepy.tech/personalized-badge/agent-strace?period=total&units=INTERNATIONAL_SYSTEM&left_color=BLACK&right_color=GREEN&left_text=downloads)](https://pepy.tech/projects/agent-strace)

**Local evidence and replay for coding agents.**

`agent-strace` records what a coding agent did in a session (prompts, tool calls, file operations, commands, failures) as local files, then does three jobs:

1. **Replay what happened.** [`replay`](docs/commands.md#replay) shows the session in order. [`explain`](docs/commands.md#explain) summarizes it in phases.
2. **See what the trace could not prove.** [`health`](docs/evidence-health.md) rates the evidence `healthy`, `partial`, `unknown`, or `invalid`, with observed gaps and known provider capture limits.
3. **Attach safe evidence to review.** [`review`](docs/review.md) checks a PR's reviewability from its authoring session. [`share`](docs/commands.md#share) writes a privacy-minimized HTML evidence packet; `share --dry-run` previews what it would disclose.

```bash
uv tool install agent-strace
agent-strace setup --cli copilot   # or claude, codex, gemini, cursor; then use the agent
agent-strace replay                # latest session
agent-strace health                # what the capture missed
agent-strace review                # reviewability of this branch
```

Works with Claude Code, Codex, Copilot CLI, Gemini CLI, Cursor, and any MCP server, plus partial [Copilot cloud agent capture](docs/copilot-cloud-agent.md). Python 3.10+ standard library only. Secret redaction is on by default. Start with the [docs index](docs/index.md), [provider guides](docs/capture-matrix.md), and [PR evidence workflow](docs/ai-agent-pr-evidence.md).

![demo](assets/demo.svg)

> [!WARNING]
> Experimental. Do not treat it as a production security boundary without your own threat model, controls, and testing.

## Try it without setup: which skills never load

```bash
uvx agent-strace skills --from-claude
```

This reads the Claude Code transcripts already on your machine (`~/.claude/projects`), writes no persistent trace store (it uses a temporary directory deleted on exit), and reports each skill as available, loaded, followed, and what happened after the load. It reports counts, not a score, and only for what the sessions recorded. For other agents, record sessions first, then run `agent-strace skills`. A never-loaded skill is a lead, not a verdict: no task may have needed it. Test it with [`skills eval`](docs/skills-eval.md). Guide: [docs/skills.md](docs/skills.md).

Sample output (139 recorded sessions, trimmed):
```text
Skills across 139 sessions since 2025-11-09. Availability reflects the filesystem now, not session time.
skill                   avail loaded load rate user/model median t pre-edit  follow  tests
------------------------------------------------------------------------------------------
frontend-design           139      2     2/139    0/3         3.1h      1/2     0/2    0/2
grilling                  139      0     0/139    0/0            -        -       -      -
web-design-guidelines     139      0     0/139    0/0            -        -       -      -

Never loaded (available in >= 5 sessions, loaded in 0):
  grilling (available in 139 sessions)
  web-design-guidelines (available in 139 sessions)
```

## Quick start

**Option 1: CLI hooks.** Captures prompts, responses, and hook-visible tool calls

```bash
agent-strace setup             # Claude Code hooks for ~/.claude/settings.json
agent-strace setup --cli claude --verify # check Claude Code hooks + CLI, not live capture
agent-strace setup --cli codex # OpenAI Codex hooks in ~/.codex/hooks.json
agent-strace setup --cli codex --verify # check Codex user hooks + CLI, not live capture
agent-strace setup --cli gemini # Gemini CLI extension under ~/.gemini/extensions
agent-strace setup --cli cursor # Cursor project hooks in .cursor/hooks.json
agent-strace setup --cli copilot # GitHub Copilot CLI hooks in ~/.copilot/hooks
agent-strace list              # list sessions
```

Provider coverage and known gaps: [docs/capture-matrix.md](docs/capture-matrix.md)

### Claude Code plugin

After installing `agent-strace`, install the repository as a native Claude Code
plugin to enable tracing hooks plus the shared trace analyst agent and skill:

```bash
claude plugin marketplace add Siddhant-K-code/agent-trace
claude plugin install agent-strace@agent-trace
```

See the [Claude Code plugin guide](docs/claude-code-plugin.md) for verification,
updates, uninstall steps, privacy boundaries, provider gaps, and release
publication.

### GitHub Copilot CLI plugin

After installing `agent-strace`, install the repository as a Copilot CLI plugin to enable tracing hooks plus the trace analyst agent and skill:

```bash
copilot plugin marketplace add Siddhant-K-code/agent-trace
copilot plugin install agent-strace@agent-trace
```

Restart Copilot CLI, then use `/agent` to select
`agent-strace:trace-analyst`. Traces are written to `.agent-traces/` in the
repository you opened. Do not also run `agent-strace setup --cli copilot`,
which would register duplicate hooks.

See the [Copilot CLI plugin guide](docs/copilot-plugin.md) for verification,
updates, and usage.

**Option 2: MCP proxy.** Wraps any MCP server, works with Cursor, Windsurf, and Copilot Desktop MCP servers

```bash
agent-strace record -- npx -y @modelcontextprotocol/server-filesystem /tmp
agent-strace replay
```

**Option 3: Python decorator.** No MCP required.

```python
from agent_trace import trace_tool, start_session, end_session

start_session(name="my-agent")

@trace_tool
def search_codebase(query: str) -> str:
    return search(query)

end_session()
```

Sessions go to `$AGENT_TRACE_DIR` when set, else `.agent-traces`; an explicit `start_session(trace_dir=...)` wins.

Full setup guide: [docs/setup.md](docs/setup.md). Machine-wide install for platform teams: [docs/platform-install.md](docs/platform-install.md)

## What you can do

### Understand a session

| Command | What it does |
|---|---|
| [`agent-strace replay <id>`](docs/commands.md#replay) | Replay a session in the terminal or as HTML |
| [`agent-strace replay <id-a> --diff <id-b>`](docs/commands.md#replay) | Side-by-side session comparison with tool args and output delta |
| [`agent-strace health <id>`](docs/commands.md#health) | Evidence health with observed gaps and capture-matrix provider limitations |
| [`agent-strace explain <id>`](docs/commands.md#explain) | Plain-English phase summary, no LLM required |
| [`agent-strace timeline <id>`](docs/commands.md#timeline) | Phase-by-phase view with costs and retries |
| [`agent-strace why <id> <event>`](docs/commands.md#why) | Causal chain for a specific decision |
| [`agent-strace diff <id-a> <id-b>`](docs/commands.md#diff) | Structural or semantic session comparison |
| [`agent-strace compare <id-a> <id-b>`](docs/commands.md#compare) | Regression report with verdict |
| [`agent-strace compaction <id>`](docs/compaction.md) | Inspect context loss and behavior changes after compaction |

### Control and protect

| Command | What it does |
|---|---|
| [`agent-strace watch`](docs/commands.md#watch) | Live monitor with kill-switch rules |
| [`agent-strace watch --timeout 30m --budget 5`](docs/commands.md#watch) | Watchdog mode: kills on limit and heartbeats sessions for postmortems |
| [`agent-strace mcp-scan`](docs/commands.md#mcp-scan) | Scan runtime MCP poisoning indicators |
| [`agent-strace audit <id>`](docs/commands.md#audit) | Audit tool calls against a policy file |
| [`agent-strace policy backtest`](docs/commands.md#policy) | Test policy impact against historical sessions |
| [`agent-strace approval list`](docs/commands.md#approval) | Human-in-the-loop approval queue |
| [`agent-strace rbac assign`](docs/commands.md#rbac) | Org and workspace-scoped role assignments |
| [`agent-strace auth login`](docs/commands.md#auth) | SSO/OIDC login to a hosted collector |
| [`agent-strace telemetry status`](docs/telemetry.md) | Inspect, disable, or re-enable anonymous CLI telemetry, which is on by default |
| [`agent-strace apply`](docs/commands.md#apply) | Apply `.agent-strace.yaml` config to local store or collector |
| [`agent-strace workspace new`](docs/commands.md#workspace) | Create an isolated workspace |
| [`agent-strace compliance export`](docs/commands.md#compliance) | Export compliance reports (EU AI Act, SOC 2, HIPAA) |
| [`agent-strace record`](docs/commands.md#record) | Strip secrets from traces before storage by default |
| [`agent-strace export --anonymize`](docs/commands.md#export) | Remove PII at export time |

### Analyse across sessions

| Command | What it does |
|---|---|
| [`agent-strace dashboard`](docs/commands.md#dashboard) | Multi-session overview |
| [`agent-strace budget-report`](docs/commands.md#budget-report) | Weekly spend digest |
| [`agent-strace team-report`](docs/commands.md#team-report) | Team spend by author, branch, or PR |
| [`agent-strace org-report`](docs/org-report.md) | Monthly organization usage and estimated-cost digest |
| [`agent-strace compliance-report`](docs/compliance-report.md) | Privacy-minimized framework evidence crosswalk |
| [`agent-strace cost --breakdown provider`](docs/commands.md#cost) | Offline spend by provider and model |
| [`agent-strace tenant report`](docs/tenancy.md) | Per-customer cost, export, and erasure workflows |
| [`agent-strace backfill-meta`](docs/commands.md#backfill-meta) | Write missing metadata for older subagent session dirs (dry run by default) |
| [`agent-strace cognitive-debt`](docs/commands.md#cognitive-debt) | Unreviewed agent-written code by session |
| [`agent-strace context-score`](docs/commands.md#context-score) | Compare efficiency stats (cost, tool calls, lint, scope) across AGENTS.md/CLAUDE.md versions. Not an outcome score |
| [`agent-strace skills`](docs/skills.md) | Skill adoption: available → loaded → followed → outcome (no score) |
| [`agent-strace skills eval`](docs/skills-eval.md) | Experimental: run should/shouldn't-trigger tasks and count skill loads |
| [`agent-strace lint <id>`](docs/commands.md#lint) | Flag bad behaviour patterns (loops, spirals, waste) |
| [`agent-strace drift`](docs/commands.md#drift) | Detect behavioural drift over time |
| [`agent-strace fingerprint`](docs/commands.md#fingerprint) | Baseline an agent's behavioural profile |
| [`agent-strace tree`](docs/commands.md#tree) | Show parent/child session hierarchy |
| [`agent-strace freeze`](docs/commands.md#freeze) | Freeze a tool-call sequence for regression checks |
| [`agent-strace standup`](docs/commands.md#standup) | Plain-English summary of one session (latest by default) |
| [`agent-strace eval <id>`](docs/commands.md#eval) | Gate a session on named quality criteria and baselines |
| [`agent-strace eval ci`](docs/commands.md#eval) | Run the legacy normalized-score CI workflow |

### Export and integrate

| Command | What it does |
|---|---|
| [`agent-strace export --format otlp-genai`](docs/production.md) | Export to Datadog, Honeycomb, Grafana, Jaeger |
| [`agent-strace export --format temporal`](docs/integrations.md#temporal-workflows) | Nest agent spans under Temporal activities |
| [`agent-strace export --format eu-ai-act`](docs/commands.md#export) | Generate Article 12/13 audit packages |
| [`agent-strace export --metrics`](docs/production.md#behavioral-metrics) | Export per-session behavioral metrics as OTLP gauges |
| [`agent-strace identity show`](docs/commands.md#identity) | Machine identity: sign and verify sessions |
| [`agent-strace mcp`](docs/mcp.md) | MCP server: query sessions, health, and skill reports from Claude Code, Copilot CLI, Codex, or Gemini CLI |
| [`agent-strace server`](docs/server.md) | Server-side collector for multi-agent, multi-machine |
| [`agent-strace share <id>`](docs/commands.md#share) | Generate a shareable HTML replay |
| [`agent-strace share --assignment <id>`](docs/hiring.md) | Create a privacy-minimized assignment submission ZIP |
| [`agent-strace score submission.zip`](docs/hiring.md) | Apply a deterministic process-telemetry rubric |
| [`agent-strace pr-comment`](docs/commands.md#pr-comment) | Post or update a session summary on the current PR |
| [`agent-strace review`](docs/review.md) | Check PR reviewability: rationale, scope, validation, stewardship, evidence health |
| [`agent-strace sample`](docs/commands.md#sample) | Export worst sessions as JSONL for eval datasets |

Full flag reference: [docs/commands.md](docs/commands.md)

## VS Code extension

Install **agent-strace** from the [Extensions panel](https://open-vsx.org/extension/Siddhant-K-code/agent-strace) to see live session activity without leaving the editor.

Anonymous extension usage telemetry is enabled by default and can be disabled
from the Command Palette or `agentTrace.telemetry.enabled` setting. No prompts,
trace contents, paths, session IDs, command arguments, or repository data are sent.

| Feature | Description |
|---|---|
| Status bar | Live cost, tool call count, and active tool name. Click to open the event stream. |
| Gutter annotations | Blue border on files the agent read, amber on files it modified. |
| Evidence review | Ordered events, evidence health, gaps, relationships, annotations, raw fields, and reviewer filters. |
| Local review actions | Compare sessions, save a privacy-minimized HTML review, or download OTLP JSON without uploading. |
| Pause button | Stops the agent mid-session via SIGSTOP. |

```bash
pip install agent-strace   # 1. install
agent-strace setup         # 2. add hooks to Claude Code; use --cli codex, gemini, cursor, or copilot for other CLIs
# 3. open project in VS Code; extension activates when .agent-traces/ exists
# 4. start Claude Code; status bar appears immediately
```

Full docs: [docs/vscode.md](docs/vscode.md)

## Production

**OTLP export**: sessions become traces, tool calls become spans:

```bash
agent-strace export <session-id> --format otlp-genai \
  --endpoint http://localhost:4318
```

Per-backend setup (Datadog, Honeycomb, Grafana, New Relic, Splunk, Langfuse): [docs/production.md](docs/production.md)

**Server-side collector**: for containers, CI, and multi-machine setups:

```bash
agent-strace server --port 4317 --storage ./traces
export AGENT_STRACE_ENDPOINT=http://collector:4317   # read by hook commands and Python integrations
```

Full guide: [docs/server.md](docs/server.md)

**Auto-instrumentation**: no code changes required:

```python
from agent_trace.integrations import instrument_langchain
instrument_langchain()
```

Supported: OpenAI Agents SDK, LangChain, LangGraph, CrewAI, LiteLLM, Anthropic SDK, OpenAI SDK, AWS Strands. Guide: [docs/integrations.md](docs/integrations.md)

**GitHub Actions**: run evals in CI, post results to the step summary, fail on regression:

```yaml
- uses: Siddhant-K-code/agent-trace@v0.93.1
  with:
    config: .agent-evals.yaml
    baseline: .agent-evals-baseline.json
    tolerance: "0.05"
```

[Marketplace listing](https://github.com/marketplace/actions/agent-trace-eval) · [Action reference](action.yml)

## How it works

**Claude Code hooks**: Claude Code fires hook events at every stage of its agentic loop. agent-strace registers as a handler, reads JSON from stdin, and writes trace events. Each hook runs as a separate process; session state in `.agent-traces/.active-session` correlates PreToolUse and PostToolUse for latency measurement.

**MCP stdio proxy**: sits between the agent and the MCP server, reads JSON-RPC messages (Content-Length framed or newline-delimited), classifies each one, and writes a trace event. Messages are forwarded unchanged. The agent and server do not know the proxy exists.

**MCP HTTP/SSE proxy**: same idea, different transport. Listens on a local port, forwards POST and SSE requests to the remote server, captures every JSON-RPC message in both directions.

**Python decorator**: `@trace_tool` logs a `tool_call` event before execution and a `tool_result` after. Errors and timing are captured automatically. `@trace_llm_call` does the same for LLM calls.

## Running tests

```bash
python -m unittest discover -s tests -v
```

## License

MIT. Use it however you want.

---

<!-- mcp-name: io.github.Siddhant-K-code/agent-trace -->
[Sponsor](https://github.com/sponsors/Siddhant-K-code) · [ADRs](ADRs/) · [Security](SECURITY.md) · [Contributing](CONTRIBUTING.md) · [Release guide](docs/releases.md) · [PyPI](https://pypi.org/project/agent-strace/)