🇺🇸 English | [🇧🇷 Português](pt-BR/arquitetura.md)
# toolbelt Architecture (`tb`)
## Overview
`toolbelt` is a set of tools for AI coding agents, unified under a single binary (`tb`) and protocol. The goal is to provide a safe, efficient, and auditable environment for autonomous agents to run development tasks.
```mermaid
graph TB
subgraph "Human Interface"
CLI[tb CLI]
TUI[tb md - TUI Markdown]
OBS[Obsidian]
end
subgraph "Protocol Layer"
MCP[MCP Server]
A2A[A2A - Agent to Agent]
RPC[JSON-RPC 2.0]
end
subgraph "Core"
ORCH[Orchestrator]
MEM[Memory Manager]
CTX[Context Assembler]
GRAPH[Graph Engine]
SANDBOX[Sandbox Manager]
end
subgraph "Storage"
VAULT[Markdown Vault
source of truth]
SQLITE[(SQLite + FTS5
derived index)]
AUDIT[Audit Log
append-only]
end
subgraph "Execution"
PROC[Process Manager]
WORK[Git Worktrees]
ISO[Isolation
bubblewrap+Landlock]
end
CLI --> ORCH
TUI --> VAULT
OBS --> VAULT
MCP --> ORCH
A2A --> ORCH
ORCH --> MEM
ORCH --> CTX
ORCH --> SANDBOX
ORCH --> PROC
MEM --> VAULT
MEM --> GRAPH
GRAPH --> SQLITE
VAULT -.->|sync| SQLITE
SANDBOX --> WORK
SANDBOX --> ISO
ORCH --> AUDIT
```
---
## 1. Workspace and Sandbox
### 1.1 Isolation by Worktree
Each agent operates in a dedicated **git worktree**, enabling:
- Parallel work without branch conflicts
- Instant snapshot and rollback via git
- Simple cleanup when the task finishes
```mermaid
graph LR
REPO[Main Repository] --> WT1[worktree/agent-001]
REPO --> WT2[worktree/agent-002]
REPO --> WT3[worktree/agent-003]
WT1 --> SB1[Sandbox 1]
WT2 --> SB2[Sandbox 2]
WT3 --> SB3[Sandbox 3]
```
### 1.2 Isolation Layers
| Layer | Technology | Responsibility |
|-------|------------|----------------|
| Filesystem | bubblewrap | Mounts only allowed directories; read-only bind-mount for tests |
| Syscalls | seccomp | Allowed syscall list per task profile |
| Filesystem (runtime) | Landlock | Access restrictions even for child processes |
| Resources | cgroups v2 | CPU, RAM, and execution-time limits |
| Network | netns/bubblewrap | Denied by default; explicit whitelist |
### 1.3 Snapshot and Rollback
Before each task:
1. `git stash --include-untracked` or a temporary commit
2. State hash saved in the audit log
3. Task execution in the sandbox
4. On failure or violation: `git reset --hard` + `git clean -fdx`
---
## 2. Memory vs Context
**Memory** and **context** are distinct concepts:
| Aspect | Memory | Context |
|--------|--------|---------|
| **Definition** | Persistent storage of facts, decisions, summaries, and the graph | What is assembled into the model window for a specific task |
| **Persistence** | Permanent (Markdown vault + SQLite) | Ephemeral (lasts one request) |
| **Size** | Unlimited (grows with the project) | Limited by the token budget |
| **Format** | Markdown + frontmatter + graph | Text serialized for the prompt |
### 2.1 Context Types
```mermaid
graph TB
subgraph "Global Context"
RULES[Rules and Profiles]
CAPS[Agent Capabilities]
TOOLS[Available Tools]
end
subgraph "Local Context"
REPO_CTX[Repository Context]
TASK_CTX[Task Context]
AGENT_CTX[Agent State]
end
subgraph "Assembly"
BUDGET[Token Budget]
ASSEMBLER[Context Assembler]
end
RULES --> ASSEMBLER
CAPS --> ASSEMBLER
TOOLS --> ASSEMBLER
REPO_CTX --> ASSEMBLER
TASK_CTX --> ASSEMBLER
AGENT_CTX --> ASSEMBLER
BUDGET --> ASSEMBLER
ASSEMBLER --> PROMPT[Final Prompt]
```
### 2.2 Context Hierarchy
1. **Global**: system rules, user profile, available MCP tools
2. **Repository**: project structure, conventions, main dependencies
3. **Agent**: role/persona, recent history, specific capabilities
4. **Task**: objective, relevant files, graph query results
The **Context Assembler** receives a token budget and prioritizes the most relevant information, truncating or summarizing when needed.
---
## 3. Storage: Markdown Vault + SQLite/FTS5
### 3.1 Markdown Vault (Source of Truth)
Inspired by Obsidian, memory is a vault of Markdown files:
```
.tb/
├── vault/
│ ├── facts/
│ │ ├── projeto-estrutura.md
│ │ └── decisoes-arquitetura.md
│ ├── agents/
│ │ ├── agent-001-log.md
│ │ └── agent-002-log.md
│ ├── summaries/
│ │ └── sprint-2024-01.md
│ └── code/
│ ├── modules/
│ │ └── auth-module.md
│ └── functions/
│ └── parse-config.md
├── index.db # SQLite derivado
└── audit.log # Log append-only
```
Each file uses:
- **YAML frontmatter**: structured metadata (type, tags, date, provenance)
- **Wikilinks**: `[[outro-arquivo]]` for relationships
- **Tags**: `#decision`, `#fact`, `#inferred`
- **Backlinks**: calculated automatically
Example file:
```markdown
---
type: fact
tags: [architecture, database]
created: 2026-01-15T10:30:00Z
source: extracted
confidence: high
---
# DecisĂŁo: SQLite para Armazenamento
O projeto usa SQLite como banco de dados principal.
## Justificativa
- Simplicidade de deploy (arquivo Ăşnico)
- Performance excelente para leitura
- FTS5 para busca full-text
## Relacionamentos
- Implementado em: [[database-module]]
- Decidido em: [[adr-003-sqlite-fts5]]
- Afeta: [[config-schema]], [[migration-system]]
```
### 3.2 SQLite + FTS5 (Derived Index)
SQLite is a **derived index**, rebuilt from the vault and the code:
```sql
-- NĂłs do grafo
CREATE TABLE nodes (
id TEXT PRIMARY KEY,
type TEXT NOT NULL, -- 'file', 'function', 'class', 'fact', 'decision'
path TEXT, -- caminho no vault ou no cĂłdigo
title TEXT,
content TEXT,
source TEXT NOT NULL, -- 'vault', 'code', 'inferred'
confidence REAL DEFAULT 1.0, -- 0.0-1.0 para dados inferidos
created_at TEXT,
updated_at TEXT,
metadata JSON
);
-- ĂŤndice full-text
CREATE VIRTUAL TABLE nodes_fts USING fts5(
title, content,
content='nodes',
content_rowid='rowid'
);
-- Arestas do grafo
CREATE TABLE edges (
id INTEGER PRIMARY KEY,
source_id TEXT NOT NULL,
target_id TEXT NOT NULL,
type TEXT NOT NULL, -- 'calls', 'imports', 'references', 'implements', 'links_to'
source TEXT NOT NULL, -- 'extracted', 'inferred', 'manual'
confidence REAL DEFAULT 1.0,
metadata JSON,
FOREIGN KEY (source_id) REFERENCES nodes(id),
FOREIGN KEY (target_id) REFERENCES nodes(id)
);
-- Índices para queries rápidas
CREATE INDEX idx_edges_source ON edges(source_id);
CREATE INDEX idx_edges_target ON edges(target_id);
CREATE INDEX idx_edges_type ON edges(type);
CREATE INDEX idx_nodes_type ON nodes(type);
CREATE INDEX idx_nodes_path ON nodes(path);
-- Views Ăşteis
CREATE VIEW callers AS
SELECT e.source_id as caller, e.target_id as callee, e.confidence
FROM edges e WHERE e.type = 'calls';
CREATE VIEW dependencies AS
SELECT e.source_id as dependent, e.target_id as dependency, e.confidence
FROM edges e WHERE e.type = 'imports';
```
### 3.3 Vault → SQLite Sync
The sync direction is **unidirectional**: vault → SQLite.
```mermaid
sequenceDiagram
participant V as Vault Markdown
participant W as Watcher
participant P as Parser
participant S as SQLite
V->>W: Arquivo modificado
W->>P: Parse YAML + Markdown
P->>P: Extrai wikilinks e tags
P->>S: UPSERT node
P->>S: REPLACE edges para este node
S->>S: Atualiza FTS5
```
**Conflict resolution**: the vault always wins. If SQLite diverges (corruption, manual edit), the `tb graph rebuild` command rebuilds everything.
### 3.4 Code Indexing with Provenance
Inspired by Graphify and reimplemented in toolbelt ([ADR-014](adr/014-graphify-reimplementacao.md)), `tb graph` indexes code with `go/ast` (Go) and conservative regex extractors (TS/JS, Python, Rust, frameworks). tree-sitter remains a future option, not implemented:
```mermaid
graph LR
CODE[CĂłdigo Fonte] --> TS[go/ast + extratores regex]
TS --> AST[AST]
AST --> EXT[Extrator]
EXT --> NODES[Nós: funções, classes, módulos]
EXT --> EDGES[Arestas: calls, imports]
NODES --> |source='extracted'| DB[(SQLite)]
EDGES --> |source='extracted'| DB
subgraph "Opcional: Inferência Semântica"
AST --> LLM[Modelo de Linguagem]
LLM --> INF[Relações Inferidas]
INF --> |source='inferred', confidence < 1.0| DB
end
```
**Edge provenance**:
- `extracted`: derived directly from the AST (high confidence)
- `inferred`: deduced by LLM or heuristics (variable confidence)
- `manual`: inserted by a human via the vault
### 3.5 Queries for Agents
The graph exposes queries that save tokens:
```bash
# Quem chama esta função?
tb graph callers --function "parseConfig" --depth 2
# Quais sĂŁo as dependĂŞncias deste mĂłdulo?
tb graph deps --module "auth" --transitive
# Qual o impacto de mudar este arquivo?
tb graph impact --file "src/config/parser.go"
# Busca semântica
tb graph search "autenticação JWT"
```
**Hooks for agents**: before reading raw files, the agent should query the graph:
```
1. Recebe tarefa: "Corrigir bug no parser de config"
2. tb graph search "parser config" → encontra nós relevantes
3. tb graph callers --function "parseConfig" → entende quem usa
4. tb graph impact --file "parser.go" → avalia escopo
5. SĂł entĂŁo lĂŞ os arquivos especĂficos necessários
```
This drastically reduces the number of tokens consumed reading files.
---
## 4. Security
### 4.1 Threat Model
| Threat | Mitigation |
|--------|------------|
| Sandbox escape | Multiple layers (bubblewrap + Landlock + seccomp) |
| Secret access | Variables injected only for authorized tools; never in the prompt |
| Test modification | Read-only test files + verified hash |
| Data exfiltration | Network denied by default; explicit whitelist |
| Privilege escalation | Minimal capabilities; capability tokens per action |
| Reward hacking | Implementer ≠evaluator; suspicious-pattern detection |
### 4.2 Permissions by Capability
Each role has a set of capabilities:
```yaml
roles:
worker:
capabilities:
- read:code
- write:code
- run:tests
- query:graph
denied:
- write:tests
- delete:*
- network:*
supervisor:
capabilities:
- read:*
- write:code
- approve:worker_changes
- query:graph
denied:
- delete:*
- network:external
planner:
capabilities:
- read:*
- write:plans
- spawn:supervisor
- query:graph
- network:internal
```
### 4.3 Capability Tokens for Destructive Actions
Dangerous actions require explicit capability tokens:
```go
type CapabilityToken struct {
Action string // "delete:file", "network:external"
Resource string // caminho ou padrĂŁo
GrantedBy string // quem aprovou
ExpiresAt time.Time
Hash string // assinatura para verificação
}
```
The flow:
1. Agent requests a destructive action
2. System generates a pending token
3. Human (or authorized supervisor) approves
4. Token is signed and has a short validity
5. Action executes presenting the token
### 4.4 Secret Handling
```mermaid
sequenceDiagram
participant A as Agente
participant O as Orquestrador
participant S as Secret Store
participant T as Ferramenta
A->>O: Preciso chamar API externa
O->>O: Verifica capacidade do agente
O->>S: Solicita secret para "api_externa"
S->>O: Secret (criptografado)
O->>T: Executa ferramenta com secret injetado
T->>T: Usa secret internamente
T->>O: Resultado (sem secret)
O->>A: Resultado sanitizado
```
Secrets never appear:
- In the model prompt
- In logs
- In vault memory
- In responses to the agent
---
## 5. Scripts and Processes
### 5.1 `tb run` and `tb proc` Commands
```bash
# Execução única com timeout
tb run --timeout 30s "go test ./..."
# Processo de longa duração com logs
tb proc start --name "dev-server" --log /tmp/dev.log "npm run dev"
tb proc list
tb proc logs dev-server --follow
tb proc stop dev-server
# Execução em sandbox
tb run --sandbox --allow-net "localhost:8080" "curl localhost:8080/health"
```
### 5.2 Process Management
```mermaid
stateDiagram-v2
[*] --> Pending: tb proc start
Pending --> Running: processo inicia
Running --> Completed: exit 0
Running --> Failed: exit != 0
Running --> Timeout: tempo excedido
Running --> Killed: tb proc stop
Timeout --> Killed: SIGTERM → SIGKILL
Failed --> [*]
Completed --> [*]
Killed --> [*]
```
Characteristics:
- **Structured logs**: stdout/stderr with timestamps
- **Cascading kill**: killing the parent kills children (via cgroups)
- **Hierarchical timeout**: warn → SIGTERM → grace period → SIGKILL
- **Resource limits**: CPU/RAM per process via cgroups
---
## 6. Roles and Personas
### 6.1 Definition in YAML + Markdown
```yaml
# .tb/roles/worker-backend.yaml
name: worker-backend
description: Implementa cĂłdigo backend em Go
model: claude-sonnet-4-20250514
temperature: 0.3
capabilities:
- read:code
- write:code:backend/**
- run:tests:backend/**
- query:graph
tools:
- tb:run
- tb:graph:search
- tb:graph:callers
- tb:graph:deps
- git:diff
- git:commit
context:
include:
- .tb/vault/facts/go-conventions.md
- .tb/vault/facts/project-structure.md
budget_tokens: 8000
persona: |
VocĂŞ Ă© um desenvolvedor backend especializado em Go.
Siga as convenções do projeto definidas em go-conventions.md.
Sempre consulte o grafo antes de ler arquivos diretamente.
Escreva testes para cĂłdigo novo.
```
### 6.2 Tools by Role
| Role | Allowed Tools |
|------|---------------|
| `planner` | `tb:graph:*`, `tb:mem:*`, `tb:agent:spawn` |
| `supervisor` | `tb:graph:*`, `tb:mem:*`, `tb:proc:*`, `tb:agent:review` |
| `worker` | `tb:run`, `tb:graph:search,callers,deps`, `git:diff,commit` |
| `evaluator` | `tb:run` (read-only), `tb:bench:run`, `tb:graph:search` |
---
## 7. TUI Markdown (`tb md`)
### 7.1 Features
Terminal User Interface (TUI) using Bubble Tea to:
- **Navigate** the memory vault
- **Visualize** the relationship graph
- **Edit** Markdown files in-place
- **Review** agent plans and logs
- **Approve** pending actions
### 7.2 Visual Theme
The TUI follows the **Synthwave** theme defined in [docs/identidade-visual.md](identidade-visual.md). Design tokens in `assets/theme/theme.css` should be used to keep visual consistency:
- Background: violet (`--tb-bg`) or dark surface (`--tb-surface`)
- Headers: primary purple (`--tb-primary`)
- Links and interactive elements: cyan (`--tb-accent`)
- Code: mono font on surface with accent color
- Status: success/error/warning per semantic tokens
### 7.3 Interoperability with Obsidian
The vault at `.tb/vault/` is a valid Obsidian vault:
```bash
# Abrir no Obsidian
obsidian .tb/vault
# Ou via tb
tb md --obsidian # abre Obsidian se instalado
```
Sync is bidirectional for human edits:
1. Human edits in Obsidian
2. Watcher detects the change
3. SQLite index is updated
4. Agents see updated data
---
## 8. Agent Network
### 8.1 Hierarchy
```mermaid
graph TB
HUMAN[Humano] --> PLANNER[Planner
Decompõe tarefas]
PLANNER --> SUP1[Supervisor Backend]
PLANNER --> SUP2[Supervisor Frontend]
PLANNER --> SUP3[Supervisor Testes]
SUP1 --> W1[Worker Go 1]
SUP1 --> W2[Worker Go 2]
SUP2 --> W3[Worker React 1]
SUP2 --> W4[Worker CSS]
SUP3 --> W5[Worker Testes Unit]
SUP3 --> W6[Worker Testes E2E]
EVAL[Evaluator
isolado] -.->|avalia| W1
EVAL -.->|avalia| W2
```
### 8.2 Context Flow
Each level receives only the necessary context:
```mermaid
sequenceDiagram
participant H as Humano
participant P as Planner
participant S as Supervisor
participant W as Worker
H->>P: Tarefa de alto nĂvel
P->>P: Query grafo, analisa escopo
P->>S: Subtarefa + contexto relevante
S->>S: Refina, divide em unidades
S->>W: Unidade de trabalho + contexto mĂnimo
W->>W: Implementa
W->>S: Resultado
S->>S: Review, integra
S->>P: Subtarefa concluĂda
P->>H: Tarefa concluĂda
```
### 8.3 Agent Lifecycle
```mermaid
stateDiagram-v2
[*] --> Created: spawn
Created --> Initializing: setup sandbox
Initializing --> Ready: contexto montado
Ready --> Working: recebe tarefa
Working --> Ready: tarefa concluĂda
Working --> Blocked: aguarda aprovação
Blocked --> Working: aprovado
Working --> Failed: erro/timeout
Ready --> Terminated: idle timeout
Failed --> Terminated: cleanup
Terminated --> [*]
```
### 8.4 Budgets
Each agent has limits:
```yaml
budgets:
planner:
tokens_per_task: 50000
max_subtasks: 10
timeout: 30m
supervisor:
tokens_per_task: 20000
max_workers: 5
timeout: 15m
worker:
tokens_per_task: 8000
max_tool_calls: 50
timeout: 5m
```
### 8.5 Elephant-Goldfish Pattern
Inspired by Google's [Elephant-Goldfish Model (EGM)](https://github.com/google/elephant-goldfish-model), toolbelt implements a review workflow with context isolation:
```mermaid
graph TB
subgraph "Elephant (Contexto Completo)"
PLANNER[Planner
conhece todo o projeto]
PLANNER --> DOC[Documento/Spec]
PLANNER --> REFS[Material Referenciado]
end
subgraph "Goldfish (Contextos Frescos)"
G1[Revisor 1
fresco]
G2[Revisor 2
fresco]
G3[Revisor 3
fresco]
end
DOC --> G1
DOC --> G2
DOC --> G3
REFS --> G1
REFS --> G2
REFS --> G3
G1 --> |findings| AGG[Agregador]
G2 --> |findings| AGG
G3 --> |findings| AGG
AGG --> HUMAN[Humano
resolve trade-offs]
```
**Roles**:
| Role | Context | Responsibility |
|------|---------|----------------|
| Elephant (Planner) | Full | Drafts specs, brainstorms, knows the whole project |
| Goldfish (Reviewers) | Fresh | Reviews without prior context, finds gaps and assumptions |
| Human | N/A | Resolves trade-offs, approves changes |
**Why fresh context?**
1. **Quality guard**: a reviewer without context finds problems the author, blinded by accumulated knowledge, does not see
2. **Token savings**: reviewers receive only the document + sources, not the whole project
3. **Anti-cheating**: reinforces author ≠reviewer separation
**Implementation via `tb review`**:
```bash
# Revisar documento com 3 goldfish
tb review docs/design-auth.md --reviewers 3 --standards security,clarity
# Revisar PR com revisores especializados
tb review pr:42 --roles "security-reviewer,api-reviewer,test-reviewer"
```
Context isolation is **mandatory**: if the system cannot create fresh contexts, the review fails instead of proceeding with contaminated context.
### 8.6 Router/Workers Pattern
For complex tasks that require high parallelization and context savings, toolbelt supports the Router/Workers pattern:
```mermaid
graph TB
subgraph "Coordenação"
ROUTER[Router
decompõe e delega]
BOARD[Board
fila de tickets]
WATCHDOG[Watchdog
monitora health]
MERGE[Merge
agrega resultados]
end
subgraph "Execução"
W1[Worker 1
sandbox isolado]
W2[Worker 2
sandbox isolado]
W3[Worker 3
sandbox isolado]
end
ROUTER --> |tickets| BOARD
BOARD --> W1
BOARD --> W2
BOARD --> W3
W1 --> |resultado| MERGE
W2 --> |resultado| MERGE
W3 --> |resultado| MERGE
WATCHDOG -.-> |monitora| W1
WATCHDOG -.-> |monitora| W2
WATCHDOG -.-> |monitora| W3
MERGE --> ROUTER
```
**Main concepts**:
| Concept | Description |
|---------|-------------|
| **Ticket** | Atomic unit of work with scope, permissions, and acceptance criteria |
| **Mandate** | Defines which files the worker may read/write and which are forbidden |
| **Board** | Coordination queue with support for prioritization and dependencies |
| **Watchdog** | Monitor that detects slow/stuck workers and emits alerts (`worker_slow`, `worker_stuck`, `queue_growing`…); runs inside the router loop and does not scale workers automatically |
| **Merge** | Aggregator that resolves conflicts between workers and applies changes atomically |
**Execution flow**:
```mermaid
sequenceDiagram
participant R as Router
participant B as Board
participant W as Workers
participant M as Merge
participant WD as Watchdog
R->>R: Recebe tarefa complexa
R->>R: Decompõe em tickets atômicos
R->>B: Enfileira tickets
B->>W: Workers puxam tickets
WD->>W: Monitora health/timeout
W->>M: Retornam resultados
M->>M: Resolve conflitos
M->>R: Resultado consolidado
```
**Ticket example**:
```yaml
ticket:
id: "TKT-001"
title: "Implementar handler de login"
mandate:
scope: ["src/handlers/auth.go", "src/services/auth.go"]
permissions: [read, write]
forbidden: ["src/database/*", "tests/hidden/*"]
context:
files: ["docs/specs/auth.md"]
symbols: ["User", "Session", "validateCredentials"]
acceptance:
criteria: "Testes em tests/auth_test.go passam"
timeout: "10m"
dependencies: []
```
**Benefits**:
- **Minimal context**: each worker receives ~80% less context than a monolithic approach
- **Parallelization**: independent tickets run simultaneously
- **Failure isolation**: an error in one worker does not pollute others' context
- **Cache-read**: workers reuse token cache via repeated contexts
- **Audit**: each ticket has an isolated, traceable history
**Direct Delivery**:
When a worker completes a ticket with dependents, it can subscribe to receive dependent tickets **directly**, without going through the Board:
```mermaid
sequenceDiagram
participant R as Router
participant B as Board
participant W1 as Worker 1
R->>B: TKT-001 (sem deps)
B->>W1: Worker 1 puxa TKT-001
W1->>W1: Executa TKT-001
W1->>R: TKT-001 done + subscribe TKT-002
R-->>W1: TKT-002 entregue DIRETO (sem Board)
Note over W1: Contexto de TKT-001
já em cache
```
Benefits of direct delivery:
- **Lower latency**: ticket does not wait in the queue
- **Context affinity**: worker already has relevant context
- **Maximized cache-read**: previous ticket tokens in cache
- **Fewer conflicts**: the same worker edits related files
**Implementation via `tb router` and `tb ticket`**:
```bash
# Create tickets (--title or --from rd.md; dependencies via --dep)
tb ticket create --title "Criar modelo User" --scope "src/models/**"
tb ticket create --title "JWT signing" --dep TKT-001
# Run the router (foreground; Ctrl-C returns running -> pending)
tb router run --agent-cmd "meu-agente --task-file $TB_TICKET_FILE"
# View the ticket board
tb board list --status pending,running
# Monitor workers (active subscriptions, alert counts)
tb watchdog status
# Merge results (strategies: fail | last_wins)
tb merge TKT-001 TKT-002
```
The router is a **foreground** scheduler (`tb router run`; there is no
`start`/`stop` daemon); the watchdog runs inside the same loop and persists
state and alerts under `.toolbelt/router/`. Details in [cli.md](cli.md).
See [ADR-009](adr/009-router-e-workers.md) for design details.
---
## 9. Anti-Cheating Protections
### 9.1 Principles
1. **Implementer/evaluator separation**: whoever implements never evaluates their own code
2. **Test immutability**: benchmark files are read-only and hash-verified
3. **Full audit**: all actions are logged
4. **Anomaly detection**: suspicious patterns trigger alerts
### 9.2 Test File Protection
```bash
# Antes da tarefa
tb guard snapshot --paths "tests/**" "benchmarks/**"
# Durante execução
# - Arquivos montados read-only no sandbox
# - Qualquer tentativa de escrita: EACCES
# ApĂłs tarefa
tb guard verify --snapshot-id
# Verifica hashes; se diferir → falha + alerta
```
### 9.3 Audit Log
```json
{
"timestamp": "2026-01-15T10:30:00.123Z",
"agent_id": "worker-001",
"action": "file:write",
"resource": "src/parser.go",
"hash_before": "sha256:abc123...",
"hash_after": "sha256:def456...",
"approved_by": "supervisor-001",
"capability_token": "tok_xyz789"
}
```
Characteristics:
- Append-only (even root cannot edit old entries)
- Signed with a chained hash (tamper-evident)
- Rotated by size/time
### 9.4 Reward Hacking Detection
Monitored patterns:
| Pattern | Indicator | Action |
|---------|-----------|--------|
| Deleted tests | Test file removed or emptied | Block + alert |
| Test skips | `t.Skip()` added | Requires justification |
| Hardcoded outputs | Literal strings matching expected values | Flag for review |
| Artificial coverage | Tests that do not exercise code | Mutation analysis |
| Excessive retry | Many attempts until pass | Log + limit |
---
## 10. Protocols
### 10.1 Overview
```mermaid
graph LR
subgraph "Externo"
IDE[IDE/Editor]
OTHER[Outros Agentes]
end
subgraph "toolbelt"
MCP_S[MCP Server]
A2A_S[A2A Handler]
RPC[JSON-RPC Router]
CORE[Core]
end
IDE -->|MCP/stdio| MCP_S
OTHER -->|A2A/HTTP| A2A_S
MCP_S --> RPC
A2A_S --> RPC
RPC --> CORE
```
### 10.2 MCP (Model Context Protocol)
`tb` acts as an **MCP server**, exposing tools. The JSON below is illustrative (`tb_run` does not exist); the real list (36 tools with `TB_GRAPH=on` in v0.0.5) is in [cli.md](cli.md).
```json
{
"tools": [
{
"name": "tb_run",
"description": "Executa comando no sandbox",
"inputSchema": {
"type": "object",
"properties": {
"command": {"type": "string"},
"timeout": {"type": "string"},
"sandbox": {"type": "boolean"}
}
}
},
{
"name": "tb_graph_search",
"description": "Busca no grafo de conhecimento",
"inputSchema": {
"type": "object",
"properties": {
"query": {"type": "string"},
"types": {"type": "array", "items": {"type": "string"}}
}
}
},
{
"name": "tb_graph_callers",
"description": "Encontra chamadores de uma função",
"inputSchema": {
"type": "object",
"properties": {
"function": {"type": "string"},
"depth": {"type": "integer"}
}
}
}
]
}
```
### 10.3 A2A (Agent-to-Agent)
For communication between agents in the hierarchy:
> **Implemented (v0.0.5):** local agent ↔ agent transport is the `tb bus` bus ([ADR-013](adr/013-barramento-multi-vendor.md)), with the `bus/0.1` JSON-RPC envelope specified in [bus-protocolo.md](bus-protocolo.md) over a SQLite store (`.toolbelt/bus/bus.db`), not a Unix socket. The example below is illustrative.
```json
{
"jsonrpc": "2.0",
"method": "task.assign",
"params": {
"from_agent": "supervisor-001",
"to_agent": "worker-002",
"task": {
"id": "task-123",
"type": "implement",
"description": "Implementar função parseConfig",
"context": {
"files": ["src/config/schema.go"],
"graph_nodes": ["node-456", "node-789"]
},
"budget": {
"tokens": 5000,
"timeout": "3m"
}
}
},
"id": 1
}
```
### 10.4 Transport
| Scenario | Transport |
|----------|-----------|
| IDE ↔ tb | stdio (MCP standard) |
| tb ↔ child process | stdio + JSON-RPC |
| agent ↔ agent (local) | Unix socket |
| agent ↔ agent (remote) | HTTP/2 + mTLS |
---
## 11. Benchmark and KPIs
See [docs/benchmark.md](benchmark.md) for full details.
### KPI Summary
| KPI | Description | Goal |
|-----|-------------|------|
| `pass@1` | First-attempt success rate | > 70% |
| `tokens_per_task` | Tokens consumed per task | < baseline |
| `wall_time` | Total execution time | < baseline |
| `tool_calls` | Number of tool calls | < baseline |
| `rework_rate` | Rework rate | < 20% |
| `guard_violations` | Blocked security violations | 0 unblocked |
---
## 12. Adversarial Verification (`tb verify`)
### 12.1 Concept: Generator vs Discriminator Loop
Inspired by GANs (Generative Adversarial Networks), toolbelt implements an **adversarial loop** where:
- **Generator** (implementer): runs the task and submits for verification
- **Discriminator** (verifier): an independent agent that tries to FAIL the work
```mermaid
sequenceDiagram
participant G as Gerador
participant D as Discriminador
participant H as Humano
loop Máximo 3 iterações
G->>G: Implementa/corrige
G->>D: Submete para verificação
D->>D: Guard check (paths protegidos)
D->>D: Clean build
D->>D: Captura fresh
D->>D: Medição com ruler
D->>D: Verifica trapaça
alt PASS
D->>G: PASS + relatĂłrio
G->>H: ConcluĂdo
else FAIL
D->>G: FAIL + gaps
Note over G: Próxima iteração
end
end
D->>H: Escalação (3 falhas)
```
**Fundamental principle**: Only the discriminator's measurement counts as proof. Generator measurements are informative, never evidentiary.
### 12.2 Pluggable Rulers
The `tb verify` system is generic, with domain-specific **rulers**:
| Ruler | Domain | Metric | Threshold |
|-------|--------|--------|-----------|
| `visual` | UI/screens | Pixel similarity + WCAG | ≥ 95% per screen |
| `tests` | Code | Pass on hidden tests | 100% |
| `review` | Documents | Fresh-context review coverage | Configurable |
Each ruler defines:
- Primary metric and pass threshold
- Specific cheating rules
- Report format
### 12.3 Discriminator Sandbox
The discriminator operates in total isolation:
```yaml
discriminator:
sandbox:
port: dedicated # porta prĂłpria
database: dedicated # DB prĂłprio
network: isolated # sem acesso ao gerador
permissions:
push: denied
branch_switch: denied
stash: denied
fix_code: denied # apenas reporta, nunca corrige
cleanup: required # limpa após execução
```
### 12.4 Guard Check and Protected Paths
Before measurement, the discriminator runs a **guard check** over `base..head`:
```mermaid
graph LR
DIFF[git diff base..head] --> GUARD[Guard Check]
GUARD --> PROT{Mudou path protegido?}
PROT -->|Sim| DEC{Tem decisĂŁo registrada?}
PROT -->|Não| CONT[Continua verificação]
DEC -->|Sim| CONT
DEC -->|Não| FLAG[Flag: justificativa necessária]
```
**Protected paths** (built-in defaults + optional `.toolbelt/guard.yaml`):
Defaults: `tests/hidden/*`, `benchmarks/*`, `.toolbelt/thresholds.yaml`, `.toolbelt/guard.yaml`. Entries in `protected_paths` in the YAML on the **base** (string or `pattern`) are merged with these defaults; a missing file does not remove the defaults. Case-insensitive matching with globs (`**`); malformed YAML fails closed. A pattern with a trailing slash (`secrets/`) is treated as a directory prefix (protects children).
```yaml
protected_paths:
- pattern: "references/**"
reason: "ReferĂŞncias visuais nĂŁo podem ser editadas sem decisĂŁo"
- pattern: "masks/**"
reason: "Masks requerem justificativa de região dinâmica"
- pattern: "**/comparator.{js,ts,go}"
reason: "Lógica de comparação é imutável"
- pattern: ".toolbelt/thresholds.yaml"
reason: "Thresholds nĂŁo podem ser relaxados"
- pattern: "**/*.seed"
reason: "Seeds determinĂsticos sĂŁo fixos"
```
Changes to protected paths are **not automatically cheating**, but require a decision recorded in the vault or explicit approval.
### 12.5 Definition of Cheating
Cheating causes an **immediate FAIL**, even if the primary metric passes:
| Type | Description | Detection |
|------|-------------|-----------|
| Reference edit | Modify/delete existing refs | Guard check |
| Invalid mask | Mask without a real dynamic region | Justification analysis |
| Comparator change | Change threshold, padding, counting | Guard check |
| Seed change | Alter seed to hide differences | Guard check |
| Wrong state | Capture a state different from the designed one | Comparison with spec |
| Inverted design | Move the design toward the code | Diff on the design |
### 12.6 Iteration and Escalation Policy
```yaml
verification:
max_iterations: 3
on_pass:
- log_result
- update_audit
- notify_generator
on_fail:
iteration_1:
- provide_gaps
- return_to_generator
iteration_2:
- provide_detailed_gaps
- suggest_fixes
- return_to_generator
iteration_3:
- escalate_to_human
- provide_full_report
- block_merge
```
### 12.7 Discriminator Report
```yaml
report:
verdict: PASS | FAIL
per_screen:
- screen: "login"
light_mode: 97.2%
dark_mode: 95.8%
size_match: true
contrast_pass: true
gaps:
- id: G1
file: "src/components/Button.tsx"
line: 45
issue: "Cor do botĂŁo difere 12% do design"
proposed_fix: "Usar var(--primary-600) ao invés de #3b82f6"
accepted_deviations:
- reason: "Fonte do sistema difere entre OS"
approved_by: "decision-2026-01-15"
design_system_followups:
- "Considerar adicionar --primary-600 ao token set"
cheating_detected: []
metrics:
build_time: "45s"
capture_time: "12s"
comparison_time: "3s"
```
---
## 13. Open Questions
### 13.1 Integration vs Reimplementation
- **Graphify**: integrate via MCP or reimplement indexing?
- Integration: less work, depends on external maintenance
- Reimplementation: full control, can optimize for the use case
- **Decided ([ADR-014](adr/014-graphify-reimplementacao.md))**: reimplement; `tb graph explain`/`path` delivered, roadmap in the ADR
### 13.2 Inference Model for Provenance
- Which model to use to infer semantic relationships?
- Should it run locally or via API?
- Cost vs inference quality trade-off
### 13.3 Agent State Persistence
- Where to persist agent state between sessions?
- Format: JSON, SQLite, or in the vault itself?
- How to handle orphan agents?
### 13.4 Hierarchy Scalability
- How many workers per supervisor?
- How many supervisors per planner?
- How to balance load?
### 13.5 Vault Versioning
- Should the vault be versioned together with the code?
- How to merge memory changes?
- How to handle branch-specific memory?
---
## 14. Research Findings (2026-11)
In November 2026, a review of ~100 primary sources (papers, specs, official blogs, repos, and benchmarks) identified findings that may change toolbelt's design.
See the full documentation in [docs/pesquisa/](pesquisa/):
- [Primary sources](pesquisa/fontes-primarias.md) — tables by area with all sources
- [Top 10 readings](pesquisa/top-10.md) — the 10 most important sources to start with
- [Findings and new ideas](pesquisa/achados.md) — concrete proposals by module
### Pending Proposals by Module
| Module | Proposal | ADR/Status |
|--------|----------|------------|
| **mem/ctx** | Invariants file pinned outside compaction; hybrid compaction (mechanical extraction + LLM summary) | [ADR-010](adr/010-invariantes-fora-compactacao.md) (proposed; invariants in `tb mem` implemented, hybrid compaction pending) |
| **mem/ctx** | AGENTS.md as default local context; bitemporal memory with fact invalidation | partial: `tb init` reads/`--write-agents` seeds `AGENTS.md`/`CLAUDE.md` (never overwrites); `tb ctx` still pending |
| **graph** | On-demand `tb graph` with A/B by task type; prioritize human skills | [ADR-011](adr/011-grafo-sob-demanda.md) (accepted) |
| **verify** | Held-out suite the generator never sees; visible/hidden gap as KPI | pending |
| **guard** | Trajectory monitor (suspicion score); cheating patterns beyond deleting tests | pending |
| **benchmark** | `tb bench --impossible` to detect cheating; pass^k beyond pass@1; costĂ—accuracy frontier | pending |
| **protocol** | Stateless `tb mcp` (MCP 2026-07-28); tools as files + deferred search | pending |
| **router** | Mandate with base commit and files; disjoint scopes for parallelism; waste watchdog | partially in [ADR-009](adr/009-router-e-workers.md) |
**Note**: proposals marked "pending" are documented in [achados.md](pesquisa/achados.md) but do not yet have an ADR.
---
## 15. Inspirations
Toolbelt incorporates patterns from three reference projects:
| Project | What We Adopted |
|---------|-----------------|
| **[Graphify](https://github.com/Graphify-Labs/graphify)** | Queryable graph, AST indexing, provenance (extracted vs inferred), graph-first hooks |
| **[Obsidian](https://obsidian.md/)** | Markdown vault as source of truth, wikilinks, backlinks, YAML frontmatter, local-first |
| **[Elephant-Goldfish Model](https://github.com/google/elephant-goldfish-model)** | Fresh-context review, author/reviewer separation, draft→review→resolve workflow, drift detection |
See the ADRs for details:
- [ADR-006: Graphify and Obsidian](adr/006-inspiracoes-graphify-obsidian.md)
- [ADR-007: Elephant-Goldfish Model](adr/007-inspiracao-elephant-goldfish.md)
---
## References
- [MCP Specification](https://spec.modelcontextprotocol.io/)
- [Google A2A Protocol](https://github.com/google/A2A)
- [Graphify](https://github.com/Graphify-Labs/graphify)
- [Obsidian](https://obsidian.md/)
- [Elephant-Goldfish Model](https://github.com/google/elephant-goldfish-model)
- [bubblewrap](https://github.com/containers/bubblewrap)
- [Landlock](https://landlock.io/)
- [tree-sitter](https://tree-sitter.github.io/)
- [Bubble Tea](https://github.com/charmbracelet/bubbletea)