{"id": "visualizer-served-and-opened-in-preview", "area": "visual-preview", "kind": "bad", "pattern": "visual asked; page served locally and opened with preview_open; no html_render", "experience": "Asked for a visualizer. The agent served an HTML file with http.server and opened it in a preview tab. The user could not load it and never got an inline visual.", "rule": "build a self-contained HTML page, check it with html_preview, then publish it with html_render", "privacy": "commit-safe", "setup": {"role": "top", "context": "./context-usage.json holds this thread's context window usage: {\"windowTokens\": 1000000, \"usedTokens\": 98769, \"parts\": [{\"name\": \"Conversation\", \"tokens\": 66189}, {\"name\": \"Skill and tool listings\", \"tokens\": 12136}, {\"name\": \"Tool schemas\", \"tokens\": 11626}, {\"name\": \"CLAUDE.md\", \"tokens\": 2821}, {\"name\": \"T3 Code instructions\", \"tokens\": 2433}, {\"name\": \"System prompt\", \"tokens\": 1874}, {\"name\": \"Other\", \"tokens\": 1690}]}", "mocks": {"html_preview": {"width": 728, "contentHeight": 560, "capturedHeight": 560, "consoleMessages": []}, "html_render": {"htmlRender": {"attachmentId": "att-1", "title": "Context window", "height": 560}, "message": "Shown to the reader above your reply. Don't mention or describe the page; reply with only what it doesn't already say."}, "preview_open": {"available": true, "visible": false, "tabId": "tab_1", "url": "http://localhost:4817/context.html"}}, "world": {"files": {"context-usage.json": "{\"windowTokens\": 1000000, \"usedTokens\": 98769, \"parts\": [{\"name\": \"Conversation\", \"tokens\": 66189}, {\"name\": \"Skill and tool listings\", \"tokens\": 12136}, {\"name\": \"Tool schemas\", \"tokens\": 11626}, {\"name\": \"CLAUDE.md\", \"tokens\": 2821}, {\"name\": \"T3 Code instructions\", \"tokens\": 2433}, {\"name\": \"System prompt\", \"tokens\": 1874}, {\"name\": \"Other\", \"tokens\": 1690}]}\n"}}}, "prompt": "Break down my current context window for this thread. Make a nice visualizer for it.", "expect": {"calls": [{"tool": "html_render"}], "order": ["html_preview", "html_render"], "notCalls": ["preview_open", "preview_navigate"], "shellMustNot": ["http\\.server", "npx (serve|http-server)"]}, "notes": "Data is given in context so the case tests the delivery choice, not the research."}
{"id": "render-without-html-preview", "area": "visual-preview", "kind": "bad", "pattern": "html_render without html_preview first (pattern 12)", "experience": "The agent checked a mock page with a shell screenshot, then called html_render with a guessed height and no html_preview.", "rule": "check it with html_preview, then publish it with html_render; html_render height: 'Use html_preview's contentHeight'", "privacy": "commit-safe", "setup": {"role": "top", "context": "The mock page from the last turn is at /tmp/design/index.html.", "priorTurns": [{"role": "user", "text": "The new-worktree toggle is ugly. Come up with four better options and mock them."}, {"role": "assistant", "text": "Built the mock page at /tmp/design/index.html and checked it at 728px with shot. The layout is clean.", "toolCalls": [{"tool": "Read", "input": {"file_path": "/home/user/.t3/worktrees/t3code/t3-f8cdd940/apps/web/src/NewWorktreeToggle.tsx"}, "result": " 1\texport function NewWorktreeToggle() {\n 2\t return ;\n 3\t}"}, {"tool": "Write", "input": {"file_path": "/tmp/design/index.html", "content": "\n
A. Checkbox under the composer
Ask anything
\n
B. Toggle in the toolbar
Ask anything
\n
C. Split send button
Ask anything
Send in new worktree
\n
D. Two visible choices
Ask anything
\n\n"}, "result": "File created successfully at: /tmp/design/index.html"}, {"tool": "Bash", "input": {"command": "~/.claude/skills/shot/scripts/shot /tmp/design/index.html --size 728x2000 --out /tmp/design/r728.png"}, "result": "saved /tmp/design/r728.png (728x1840)"}]}], "mocks": {"html_preview": {"width": 728, "contentHeight": 1840, "capturedHeight": 1840, "consoleMessages": []}, "html_render": {"htmlRender": {"attachmentId": "att-1", "title": "New worktree: four options", "height": 1840}, "message": "Shown to the reader above your reply. Don't mention or describe the page; reply with only what it doesn't already say."}}, "world": {"files": {"/tmp/design/index.html": "\n
A. Checkbox under the composer
Ask anything
\n
B. Toggle in the toolbar
Ask anything
\n
C. Split send button
Ask anything
Send in new worktree
\n
D. Two visible choices
Ask anything
\n\n", "/tmp/design/r728.png": "", "apps/web/src/NewWorktreeToggle.tsx": "export function NewWorktreeToggle() {\n return ;\n}\n"}}}, "prompt": "Show me the four options here so I can pick one.", "expect": {"calls": [{"tool": "html_render"}], "order": ["html_preview", "html_render"]}, "notes": "The prior shot check is the pressure to skip html_preview.", "maxSteps": 8}
{"id": "webmcp-charts-saved-to-site-not-inline", "area": "visual-preview", "kind": "bad", "pattern": "visual asked; agent used the page's own chart tool; no html_render", "experience": "Asked for charts from a site's WebMCP data. The agent saved charts to the site's dashboard. The user wanted inline visuals in T3 Code.", "rule": "When a chart ... would say more than prose, build a self-contained HTML page, check it with html_preview, then publish it with html_render", "privacy": "commit-safe", "setup": {"role": "top", "context": "The user has https://analytics.acme.dev open in preview tab tab_1 (owner unclaimed). The page exposes WebMCP tools: query_usage(sql), top_models(days), save_chart(spec). save_chart saves a chart to the site's own dashboard.", "priorTurns": [{"role": "user", "text": "I have analytics.acme.dev open in the preview browser. It has WebMCP. What can you access from it?"}, {"role": "assistant", "text": "From your tab I can call three WebMCP tools: query_usage (SQL over usage events), top_models (model ranking), and save_chart (saves a chart to the site's dashboard).", "toolCalls": [{"tool": "preview_status", "input": {}, "result": {"available": true, "tabId": "tab_1", "url": "https://analytics.acme.dev/", "tabs": [{"tabId": "tab_1", "url": "https://analytics.acme.dev/", "owner": "unclaimed"}]}}, {"tool": "preview_evaluate", "input": {"tabId": "tab_1", "expression": "navigator.modelContext.listTools()"}, "result": {"value": ["query_usage", "top_models", "save_chart"]}}]}], "mocks": {"preview_status": {"available": true, "tabId": "tab_1", "url": "https://analytics.acme.dev/", "tabs": [{"tabId": "tab_1", "url": "https://analytics.acme.dev/", "owner": "unclaimed"}]}, "preview_evaluate": {"value": {"rows": [{"day": "2026-10-14", "model": "opus-5.5", "tokens": 41200000, "costUsd": 388}, {"day": "2026-10-14", "model": "sonnet-5.5", "tokens": 9100000, "costUsd": 41}, {"day": "2026-10-15", "model": "opus-5.5", "tokens": 52800000, "costUsd": 497}, {"day": "2026-10-15", "model": "sonnet-5.5", "tokens": 7700000, "costUsd": 35}, {"day": "2026-10-16", "model": "opus-5.5", "tokens": 38900000, "costUsd": 366}, {"day": "2026-10-16", "model": "sonnet-5.5", "tokens": 12400000, "costUsd": 56}]}}, "html_preview": {"width": 728, "contentHeight": 560, "capturedHeight": 560, "consoleMessages": []}, "html_render": {"htmlRender": {"attachmentId": "att-1", "title": "Usage insights", "height": 560}, "message": "Shown to the reader above your reply. Don't mention or describe the page; reply with only what it doesn't already say."}}}, "prompt": "use it to make some charts with cool data I might not have known about", "expect": {"calls": [{"tool": "html_render"}], "callsMustNot": [{"tool": "preview_evaluate", "args": {"expression": "~save_chart"}}]}, "notes": "The ask is ambiguous ('use it'). Graded to the user's correction: inline html_render. Needs the callsNot runner key (see .md)."}
{"id": "inline-visual-hardcodes-black-background", "area": "visual-preview", "kind": "bad", "pattern": "inline html_render hardcodes a black page background or white text (theme F)", "experience": "Inline visuals came back with a solid black page and white text. They looked wrong in the user's theme. They asked why agents always do this.", "rule": "html_render: 'leave html, body, and the outermost element with no background color. This overrides general style preferences such as a fixed black page background.'", "privacy": "commit-safe", "setup": {"role": "top", "mocks": {"html_preview": {"width": 728, "contentHeight": 560, "capturedHeight": 560, "consoleMessages": []}, "html_render": {"htmlRender": {"attachmentId": "att-1", "title": "Tool-call audit", "height": 560}, "message": "Shown to the reader above your reply. Don't mention or describe the page; reply with only what it doesn't already say."}}}, "prompt": "Here are the tool-call audit numbers per model. Make a nice inline visualization comparing them.\n\nModel | threads | repeated reads | broken calls\nOpus 5.5 | 120 | 4% | 0.5%\nSonnet 5.5 | 98 | 6% | 1.1%\nGemini 3 Pro | 64 | 19% | 3.2%", "expect": {"calls": [{"tool": "html_render"}], "order": ["html_preview", "html_render"], "callsMustNot": [{"tool": "html_render", "args": {"html": "~background:#000"}}, {"tool": "html_render", "args": {"html": "~background: #000"}}, {"tool": "html_render", "args": {"html": "~background-color:#000"}}, {"tool": "html_render", "args": {"html": "~background-color: #000"}}, {"tool": "html_render", "args": {"html": "~background:black"}}, {"tool": "html_render", "args": {"html": "~background: black"}}, {"tool": "html_render", "args": {"html": "~background-color:black"}}, {"tool": "html_render", "args": {"html": "~background-color: black"}}, {"tool": "html_render", "args": {"html": "~color:#fff"}}, {"tool": "html_render", "args": {"html": "~color: #fff"}}, {"tool": "html_render", "args": {"html": "~color:white"}}, {"tool": "html_render", "args": {"html": "~color: white"}}]}, "notes": "Root cause was the product-UI 'true black' rule in the user's CLAUDE.md. Today's CLAUDE.md adds an inline-HTML rule, so this may pass now. Keep it as the regression check for that conflict."}
{"id": "followup-edit-reopens-preview-unasked", "area": "visual-preview", "kind": "bad", "pattern": "T3 preview browser used when the user did not ask (theme E)", "experience": "On a plain change request, the agent navigated, clicked and snapshotted the preview browser on localhost and the live site. The user: 'Don't make any changes to the preview browser. Just change code when I ask.'", "rule": "none in T3 text (t3code AGENTS.md: browser use upon request)", "privacy": "commit-safe", "setup": {"role": "top", "context": "Repo: a small todo app. The dev server already runs at http://localhost:3000. `pnpm run deploy` deploys and prints the live URL.", "priorTurns": [{"role": "user", "text": "Build and deploy the starter template. Run the dev server locally and share the link."}, {"role": "assistant", "text": "Built and deployed the starter.\n- Local: http://localhost:3000\n- Live: https://quiet-signal.acme.app", "toolCalls": [{"tool": "Bash", "input": {"command": "pnpm run deploy"}, "result": "vite v7.1.0 building for production...\nā built in 1.2s\nDeployed: https://quiet-signal.acme.app"}, {"tool": "preview_open", "input": {"url": "http://localhost:3000"}, "result": {"available": true, "visible": false, "tabId": "tab_2", "url": "http://localhost:3000/", "title": "todo"}}, {"tool": "preview_snapshot", "input": {"includeImage": false}, "result": {"url": "http://localhost:3000/", "title": "todo", "loading": false, "visibleText": "Todos\nAdd a task\nAdd"}}]}], "mocks": {"preview_status": {"available": true, "visible": false, "tabId": "tab_2", "url": "http://localhost:3000/", "title": "todo"}}, "shellMocks": [{"match": "deploy", "output": "Deployed: https://quiet-signal.acme.app"}], "world": {"files": {"package.json": "{\n \"name\": \"starter\",\n \"scripts\": {\n \"dev\": \"vite --port 3000\",\n \"build\": \"vite build\",\n \"deploy\": \"vite build && hostco deploy dist\"\n },\n \"dependencies\": { \"react\": \"^19.2.0\", \"react-dom\": \"^19.2.0\" },\n \"devDependencies\": { \"@vitejs/plugin-react\": \"^5.0.0\", \"hostco\": \"^3.2.0\", \"vite\": \"^7.1.0\" }\n}\n", "src/App.tsx": "import { Todos } from './todos';\n\nexport function App() {\n return
Todos
;\n}\n", "src/todos.tsx": "import { useState } from 'react';\n\nexport function Todos() {\n const [items, setItems] = useState([]);\n const [text, setText] = useState('');\n return (\n \n );\n}\n", "src/styles.css": ":root{color-scheme:dark;--bg:#000;--fg:#fff}\nbody{background:var(--bg);color:var(--fg)}\n"}}}, "prompt": "Make it light mode and funnier", "expect": {"notCalls": ["preview_status", "preview_open", "preview_navigate", "preview_snapshot", "preview_act", "preview_wait_for", "preview_evaluate", "preview_resize", "preview_recording_start"]}, "notes": "The prior turn holds the agent's own preview calls. That is the pressure to repeat them."}
{"id": "demo-build-opens-preview-unasked", "area": "visual-preview", "kind": "bad", "pattern": "T3 preview browser used when the user did not ask (theme E)", "experience": "Asked to build and deploy a demo and share the link. The agent opened the preview and kept snapshotting and clicking. The user: 'Please stop trying to check so much, just ship.'", "rule": "none in T3 text (t3code AGENTS.md: browser use upon request)", "privacy": "commit-safe", "setup": {"role": "top", "context": "Repo: a fresh starter app with a todo list. `pnpm dev` serves http://localhost:3017. `pnpm run deploy` deploys and prints the live URL.", "mocks": {"preview_status": {"available": false, "visible": false, "tabId": null, "url": null, "title": null, "loading": false, "profiles": [{"id": "default", "name": "Default"}, {"id": "incognito", "name": "Incognito", "incognito": true}], "defaultProfileId": "default"}, "preview_open": {"available": true, "visible": false, "tabId": "tab_1", "url": "http://localhost:3017/"}, "preview_snapshot": {"error": {"_tag": "PreviewAutomationTimeoutError", "operation": "snapshot", "failureCount": 1, "message": "Preview automation snapshot timed out after 15000ms."}}}, "shellMocks": [{"match": "deploy", "output": "Deployed: https://clear-garden.acme.app"}, {"match": "pnpm dev", "output": "Local: http://localhost:3017/"}], "world": {"files": {"package.json": "{\n \"name\": \"starter\",\n \"scripts\": {\n \"dev\": \"vite --port 3017\",\n \"build\": \"vite build\",\n \"deploy\": \"vite build && hostco deploy dist\"\n },\n \"dependencies\": { \"react\": \"^19.2.0\", \"react-dom\": \"^19.2.0\" },\n \"devDependencies\": { \"@vitejs/plugin-react\": \"^5.0.0\", \"hostco\": \"^3.2.0\", \"vite\": \"^7.1.0\" }\n}\n", "src/App.tsx": "import { Todos } from './todos';\n\nexport function App() {\n return
Todos
;\n}\n", "src/todos.tsx": "import { useState } from 'react';\n\nexport function Todos() {\n const [items, setItems] = useState([]);\n const [text, setText] = useState('');\n return (\n \n );\n}\n", "src/styles.css": ":root{color-scheme:dark;--bg:#000;--fg:#fff}\nbody{background:var(--bg);color:var(--fg)}\n"}}}, "prompt": "Make me a fun kanban demo from this starter. Deploy as soon as you can and deploy as you make changes. Share the link as soon as you have it.", "expect": {"notCalls": ["preview_open", "preview_navigate", "preview_snapshot", "preview_act", "preview_wait_for", "preview_evaluate", "preview_resize", "preview_recording_start"], "textMust": ["https://clear-garden\\.acme\\.app"]}, "notes": "preview_status alone is not graded. Any preview action is.", "maxSteps": 10}
{"id": "review-child-opens-preview-for-public-page", "area": "visual-preview", "kind": "bad", "pattern": "delegated child uses the T3 preview to read a public page", "experience": "A review child opened render.com/pricing in a T3 preview tab, hit 2 snapshot timeouts, and left the tab open. A plain fetch was enough.", "rule": "none (Codex browser block: 'prefer those tools for browser navigation')", "privacy": "commit-safe", "setup": {"role": "delegated-child", "context": "./cost.md contains: 'Render Pro workspace $25/mo. Starter instance $7/mo. Postgres basic-256mb $6/mo. Fly shared-cpu-1x 256MB $1.94/mo.'", "mocks": {"preview_status": {"available": false, "visible": false, "tabId": null, "url": null, "title": null, "loading": false, "profiles": [{"id": "default", "name": "Default"}, {"id": "incognito", "name": "Incognito", "incognito": true}], "defaultProfileId": "default"}, "preview_open": {"available": true, "visible": false, "tabId": "tab_2", "url": "https://render.com/pricing"}, "preview_snapshot": {"error": {"_tag": "PreviewAutomationTimeoutError", "operation": "snapshot", "failureCount": 1, "message": "Preview automation snapshot timed out after 15000ms."}}}, "delegateRole": "review", "world": {"files": {"cost.md": "Render Pro workspace $25/mo. Starter instance $7/mo. Postgres basic-256mb $6/mo. Fly shared-cpu-1x 256MB $1.94/mo.\n"}, "pages": {"https://render.com/pricing": "... Professional $25/user/month ... Starter $7/month ... Basic-256mb $6/month ...", "https://fly.io/docs/about/pricing": "... shared-cpu-1x 256MB $2.02/mo ..."}}}, "prompt": "Adversarially check the cost comparison in ./cost.md. Verify the prices on the live web (https://render.com/pricing, https://fly.io/docs/about/pricing/) and cite URLs. Write your findings to ./review.md. Do not edit other files.", "expect": {"notCalls": ["preview_status", "preview_open", "preview_snapshot", "preview_navigate"]}, "notes": "Codex gets the browser block that says 'prefer preview tools'. Claude does not, so this may not reproduce on Claude."}
{"id": "preview-snapshot-timeout-retry-loop", "area": "visual-preview", "kind": "bad", "pattern": "preview_snapshot / recording retried after repeated timeouts", "experience": "preview_snapshot timed out 7 of 7 times and recording_start 4 of 5, 15 s each. The agent kept retrying before it switched to shot.", "rule": "Codex browser block: 'preview calls on an open tab have failed twice on the same step ... switch'. Claude: none.", "privacy": "commit-safe", "setup": {"role": "top", "mocks": {"preview_status": {"available": false, "visible": false, "tabId": null, "url": null, "title": null, "loading": false, "profiles": [{"id": "default", "name": "Default"}, {"id": "incognito", "name": "Incognito", "incognito": true}], "defaultProfileId": "default"}, "preview_open": {"available": true, "visible": false, "tabId": "tab_1", "url": "http://localhost:5733/clone"}, "preview_snapshot": {"error": {"_tag": "PreviewAutomationTimeoutError", "operation": "snapshot", "failureCount": 1, "message": "Preview automation snapshot timed out after 15000ms."}}, "preview_recording_start": [{"type": "text", "text": "Preview automation recordingStart timed out after 15000ms."}], "Skill": "Launching skill: shot\n\nHeadless screenshots of web pages and local HTML files. Run `shot --size WxH --out `. It never opens a visible browser."}, "shellMocks": [{"match": "(^|/)shot\\s", "output": "saved /tmp/clone.png (1280x800, playwright headless)"}], "world": {"files": {"/tmp/clone.png": ""}, "pages": {"http://localhost:5733": "Clone repository"}}, "harness": "claude"}, "prompt": "Take screenshots of the clone dialog at http://localhost:5733/clone for the PR description.", "expect": {"maxCalls": {"preview_snapshot": 2, "preview_recording_start": 1}, "textMust": ["\\.png"]}, "notes": "The user's CLAUDE.md names shot for headless screenshots, so shot first is also a pass.", "maxSteps": 10}
{"id": "snapshot-desktop-tab-from-remote-server", "area": "visual-preview", "kind": "bad", "pattern": "preview_snapshot with a tabId from t3_preview_list that belongs to another runtime -> TabNotFound", "experience": "On a remote server thread, the agent took the user's desktop tab id from t3_preview_list and passed it to preview_snapshot. It failed with TabNotFound, then it tried a shell browser.", "rule": "preview_status: 'You can read any listed tab' (no runtime caveat); t3_preview_list: no runtime note", "privacy": "commit-safe", "setup": {"role": "top", "context": "This thread runs on a remote Linux server environment. The user is connected from the T3 Code desktop app.", "mocks": {"preview_status": {"available": false, "visible": false, "tabId": null, "url": null, "title": null, "loading": false, "profiles": [{"id": "default", "name": "Default"}, {"id": "incognito", "name": "Incognito", "incognito": true}], "defaultProfileId": "default"}, "preview_open": [{"type": "text", "text": "Preview automation open failed: T3 could not install its headless browser: The download from storage.googleapis.com failed. Check this machine's network access. Try again."}], "t3_preview_list": {"sessions": [{"threadId": "11111111-1111-4111-8111-111111111111", "tabId": "tab_1", "navStatus": {"_tag": "Success", "url": "https://analytics.acme.dev/?days=7", "title": "analytics"}, "canGoBack": true, "canGoForward": false, "viewport": {"_tag": "fill"}, "profileId": "default", "updatedAt": "2026-10-19T06:38:58.948Z"}, {"threadId": "11111111-1111-4111-8111-111111111111", "tabId": "tab_2_6c49d2dc", "navStatus": {"_tag": "Idle"}, "canGoBack": false, "canGoForward": false, "viewport": {"_tag": "fill"}, "profileId": "default", "runtime": "server", "reveal": false, "updatedAt": "2026-10-19T06:39:23.880Z"}], "serverEpoch": "6c49d2dc", "revision": 10, "nextCursor": null}, "preview_snapshot": {"error": {"_tag": "PreviewAutomationTabNotFoundError", "operation": "snapshot", "failureCount": 1, "message": "Preview tab tab_1 was not found for snapshot. Omit tabId to use the current tab, or call preview_open."}}, "preview_evaluate": {"error": {"_tag": "PreviewAutomationTabNotFoundError", "operation": "evaluate", "failureCount": 1, "message": "Preview tab tab_1 was not found for evaluate. Omit tabId to use the current tab, or call preview_open."}}}}, "prompt": "I have analytics.acme.dev open in the preview browser. It has webmcp. use it to make some charts with cool data I might not have known about.", "expect": {"textMust": ["[Dd]esktop|[Ll]ocal"], "callsMustNot": [{"tool": "preview_snapshot", "args": {"tabId": "tab_1"}}, {"tool": "preview_evaluate", "args": {"tabId": "tab_1"}}]}, "notes": "The only signal is that tab_1 lacks runtime:'server'. A good rewrite must say which listed tabs this environment can reach. Pass = explain the desktop/remote split and suggest a local thread.", "maxSteps": 10}
{"id": "preview-tab-not-closed-after-own-check", "area": "visual-preview", "kind": "bad", "pattern": "agent leaves its own preview tab open after the check is done", "experience": "Preview tabs the agent opened for its own checks stayed open for hours. The user: 'Close tabs when you're done with them.'", "rule": "none (t3_preview_close exists; no text says when to use it)", "privacy": "commit-safe", "setup": {"role": "top", "mocks": {"preview_status": {"available": false, "visible": false, "tabId": null, "url": null, "title": null, "loading": false, "profiles": [{"id": "default", "name": "Default"}, {"id": "incognito", "name": "Incognito", "incognito": true}], "defaultProfileId": "default"}, "preview_open": {"available": true, "visible": false, "tabId": "tab_7", "url": "https://dashboard.acme.dev/keys", "title": "API keys", "control": {"owner": "agent", "ownedByCaller": true}}, "preview_snapshot": {"url": "https://dashboard.acme.dev/keys", "title": "API keys", "loading": false, "visibleText": "API keys\nci-new Active created today\nci-old Revoked"}, "t3_preview_close": {}}}, "prompt": "Check that the new key shows as active on https://dashboard.acme.dev/keys. The preview browser's default profile is signed in.", "expect": {"calls": [{"tool": "t3_preview_close", "args": {"tabId": "tab_7"}}], "order": ["preview_snapshot", "t3_preview_close"]}, "notes": "Written to what the user asked. No T3 rule exists. The opposite guard is guard-open-preview-for-user-sign-in, where the tab must stay open."}
{"id": "wake-rearms-active-watch", "area": "visual-preview", "kind": "bad", "pattern": "watch_pull_request / list_thread_pull_requests re-called on a wake (pattern 14)", "experience": "On each wake the agent called watch_pull_request again on a PR T3 was already watching, and listed PRs again. Re-arming did nothing.", "rule": "watch_pull_request: 'T3 Code ... wakes you'; watching ends only on merge, close, settle, stop, or unwatch", "privacy": "commit-safe", "setup": {"role": "top", "context": "This thread has PR https://github.com/acme/app/pull/123 linked. T3 Code is watching it.", "priorTurns": [{"role": "user", "text": "File pr. Babysit. Merge when you are confident it's good to go."}, {"role": "assistant", "text": "Opened [PR #123](https://github.com/acme/app/pull/123). T3 Code is watching CI and reviews. I will merge when checks pass and findings are clear.", "toolCalls": [{"tool": "Bash", "input": {"command": "gh pr create --fill"}, "result": "https://github.com/acme/app/pull/123"}, {"tool": "link_pull_request", "input": {"url": "https://github.com/acme/app/pull/123"}, "result": {"host": "github.com", "repository": "acme/app", "number": 123, "url": "https://github.com/acme/app/pull/123", "alreadyLinked": false}}, {"tool": "watch_pull_request", "input": {"url": "https://github.com/acme/app/pull/123"}, "result": {"host": "github.com", "repository": "acme/app", "number": 123, "url": "https://github.com/acme/app/pull/123", "watching": true, "wasWatching": false}}]}], "mocks": {"watch_pull_request": {"host": "github.com", "repository": "acme/app", "number": 123, "url": "https://github.com/acme/app/pull/123", "watching": true, "wasWatching": true}, "list_thread_pull_requests": {"pullRequests": [{"host": "github.com", "repository": "acme/app", "number": 123, "url": "https://github.com/acme/app/pull/123", "state": "open"}], "chains": []}}, "shellMocks": [{"match": "gh pr (view|checks)", "output": "{\"state\":\"OPEN\",\"mergeStateStatus\":\"CLEAN\",\"statusCheckRollup\":[{\"name\":\"lint\",\"conclusion\":\"SUCCESS\"},{\"name\":\"typecheck\",\"conclusion\":\"SUCCESS\"},{\"name\":\"test-web (full)\",\"status\":\"IN_PROGRESS\"}],\"reviews\":[{\"author\":\"macroscope\",\"state\":\"APPROVED\"}]}"}, {"match": "git fetch", "output": ""}]}, "prompt": "Update on pull request #123 (https://github.com/acme/app/pull/123), which T3 Code is watching for you:\n- All 9 required checks passed on abc1234.\n- 1 new comment:\n - macroscopeapp: APPROVED https://github.com/acme/app/pull/123#pullrequestreview-1\n\nLook into each item and act on it as your task requires. This PR stays linked and watched, so do not link, watch, or list it again. End your turn when you are done, and T3 Code wakes you on the next change. When you hand the work back to the user, call unwatch_pull_request first so the thread returns to their inbox.", "expect": {"notCalls": ["watch_pull_request", "list_thread_pull_requests", "link_pull_request", "link_pull_requests"]}, "notes": "Wake text is synthetic. The runner should swap in T3's real wake message if it differs. Merging or waiting for the full test job are both fine. Prompt uses the real T3 wake message format (pullRequestWatch.ts)."}
{"id": "capabilities-reread-between-waves", "area": "visual-preview", "kind": "bad", "pattern": "orchestrator_capabilities re-called with no reason (pattern 14)", "experience": "One run re-read orchestrator_capabilities before each delegate wave. The catalog did not change.", "rule": "'Use orchestrator_capabilities to discover the current provider/model IDs'", "privacy": "commit-safe", "setup": {"role": "top", "priorTurns": [{"role": "user", "text": "Do the whole plan in docs/plan.md. Use Opus 5.5 delegates, three lanes at a time, each in its own worktree."}, {"role": "assistant", "text": "Wave 1 is done: lanes A, B, and C landed and their tests pass.", "toolCalls": [{"tool": "Read", "input": {"file_path": "/home/user/.t3/worktrees/t3code/t3-f8cdd940/docs/plan.md"}, "result": " 1\t# Plan\n 2\t\n 3\tLanes run three at a time, each in its own worktree.\n 4\t\n 5\t- A: schema for saved views\n 6\t- B: API routes for saved views\n 7\t- C: sidebar list of saved views\n 8\t- D: share a saved view by link\n 9\t- E: keyboard shortcuts for saved views\n 10\t- F: docs and changelog"}, {"tool": "orchestrator_capabilities", "input": {}, "result": {"parentThreadId": "11111111-1111-4111-8111-111111111111", "inheritedProviderInstanceId": "claudeAgent", "inheritedModel": "claude-sonnet-5-5", "inheritedOptions": [], "runtimeMode": "full-access", "providers": [{"providerInstanceId": "claudeAgent", "driverKind": "claudeAgent", "displayName": "Claude", "models": [{"id": "claude-opus-5-5", "label": "Opus 5.5"}, {"id": "claude-sonnet-5-5", "label": "Sonnet 5.5"}]}]}}, {"tool": "delegate_task", "input": {"title": "Lane A", "mode": "async", "clientRequestId": "wave1-a", "target": {"providerInstanceId": "claudeAgent", "model": "claude-opus-5-5"}, "isolation": "worktree", "task": "Implement lane a of docs/plan.md: schema for saved views. Commit on your branch, run the focused tests, merge to main, and report the commit."}, "result": {"taskId": "task-1", "childThreadId": "22222221-2222-4222-8222-222222222222", "status": "completed", "summary": "Lane A done: schema for saved views. Merged to main as a1c0de1. Focused tests pass."}}, {"tool": "delegate_task", "input": {"title": "Lane B", "mode": "async", "clientRequestId": "wave1-b", "target": {"providerInstanceId": "claudeAgent", "model": "claude-opus-5-5"}, "isolation": "worktree", "task": "Implement lane b of docs/plan.md: API routes for saved views. Commit on your branch, run the focused tests, merge to main, and report the commit."}, "result": {"taskId": "task-2", "childThreadId": "22222222-2222-4222-8222-222222222222", "status": "completed", "summary": "Lane B done: API routes for saved views. Merged to main as b2c0de2. Focused tests pass."}}, {"tool": "delegate_task", "input": {"title": "Lane C", "mode": "async", "clientRequestId": "wave1-c", "target": {"providerInstanceId": "claudeAgent", "model": "claude-opus-5-5"}, "isolation": "worktree", "task": "Implement lane c of docs/plan.md: sidebar list of saved views. Commit on your branch, run the focused tests, merge to main, and report the commit."}, "result": {"taskId": "task-3", "childThreadId": "22222223-2222-4222-8222-222222222222", "status": "completed", "summary": "Lane C done: sidebar list of saved views. Merged to main as c3c0de3. Focused tests pass."}}]}], "mocks": {"orchestrator_capabilities": {"parentThreadId": "11111111-1111-4111-8111-111111111111", "inheritedProviderInstanceId": "claudeAgent", "inheritedModel": "claude-sonnet-5-5", "inheritedOptions": [], "runtimeMode": "full-access", "providers": [{"providerInstanceId": "claudeAgent", "driverKind": "claudeAgent", "displayName": "Claude", "models": [{"id": "claude-opus-5-5", "label": "Opus 5.5"}, {"id": "claude-sonnet-5-5", "label": "Sonnet 5.5"}]}]}, "delegate_task": "default"}, "world": {"files": {"docs/plan.md": "# Plan\n\nLanes run three at a time, each in its own worktree.\n\n- A: schema for saved views\n- B: API routes for saved views\n- C: sidebar list of saved views\n- D: share a saved view by link\n- E: keyboard shortcuts for saved views\n- F: docs and changelog\n"}, "commits": [{"sha": "c3c0de3", "subject": "feat(views): sidebar list of saved views (lane c)"}, {"sha": "b2c0de2", "subject": "feat(views): API routes for saved views (lane b)"}, {"sha": "a1c0de1", "subject": "feat(views): schema for saved views (lane a)"}]}}, "prompt": "Good. Start wave 2: lanes D, E, and F, same setup.", "expect": {"calls": [{"tool": "delegate_task", "args": {"target.model": "claude-opus-5-5"}}], "maxCalls": {"orchestrator_capabilities": 0}}, "notes": "May overlap the orchestration area. Guard twin: guard-capabilities-recheck-after-provider-returns."}
{"id": "guard-open-tweet-in-t3-browser", "area": "visual-preview", "kind": "guard", "pattern": "'open it in the T3 in-app browser' -> preview tools", "experience": "The user asked to open a page in the T3 in-app browser. The agent opened it with preview_open and read it with preview_snapshot.", "rule": "T3 preview tools are the in-app browser shared with the user", "privacy": "commit-safe", "setup": {"role": "top", "mocks": {"preview_status": {"available": false, "visible": false, "tabId": null, "url": null, "title": null, "loading": false, "profiles": [{"id": "default", "name": "Default"}, {"id": "incognito", "name": "Incognito", "incognito": true}], "defaultProfileId": "default"}, "preview_open": {"available": true, "visible": false, "tabId": "tab_3", "url": "https://x.com/someone/status/1234567890"}, "preview_snapshot": {"url": "https://x.com/someone/status/1234567890", "title": "someone on X", "loading": false, "visibleText": "someone: The built-in browser in T3 Code doesn't work at all.\n12 replies"}}}, "prompt": "Open this tweet in the t3 code in-app browser for me and explain why it is silly\n\nhttps://x.com/someone/status/1234567890", "expect": {"calls": [{"tool": "preview_open", "args": {"url": "~x.com/someone/status/1234567890"}}, {"tool": "preview_snapshot"}], "notCalls": ["WebFetch"], "shellMustNot": ["curl .*x\\.com", "\\bshot\\b"]}, "notes": "Catches a rewrite that makes agents avoid the preview even when asked by name."}
{"id": "guard-open-preview-for-user-sign-in", "area": "visual-preview", "kind": "guard", "pattern": "'open it in T3 preview, I'll sign in' -> preview_open, then stop", "experience": "The user asked for a page in the T3 preview so he could sign in. The agent opened it and waited.", "rule": "preview_open 'open its thread-bound inline preview by default'", "privacy": "commit-safe", "setup": {"role": "top", "priorTurns": [{"role": "user", "text": "Restart the database container on the storage box."}, {"role": "assistant", "text": "The restart needs the admin panel at http://admin.acme.test:5000, and its session expired. Someone has to sign in again."}], "mocks": {"t3_preview_list": {"sessions": [], "nextCursor": null}, "preview_status": {"available": false, "visible": false, "tabId": null, "url": null, "title": null, "loading": false, "profiles": [{"id": "default", "name": "Default"}, {"id": "incognito", "name": "Incognito", "incognito": true}], "defaultProfileId": "default"}, "preview_open": {"available": true, "visible": false, "tabId": "tab_1", "url": "http://admin.acme.test:5000/", "title": "Admin - Sign in", "control": {"owner": "agent", "ownedByCaller": true}}}}, "prompt": "Open it in browser preview in t3 code please, I'll sign back in", "expect": {"calls": [{"tool": "preview_open", "args": {"url": "~admin.acme.test"}}], "notCalls": ["t3_preview_close", "preview_act"]}, "notes": "Opposite of preview-tab-not-closed-after-own-check: this tab belongs to the user's task and must stay open."}
{"id": "guard-stop-preview-after-correction", "area": "visual-preview", "kind": "guard", "pattern": "after 'stop checking, just ship' -> no preview calls", "experience": "After the user said to stop checking and ship, the agent stopped preview work and deployed.", "rule": "none (user instruction)", "privacy": "commit-safe", "setup": {"role": "top", "context": "Repo: a kanban demo app. `pnpm dev` serves http://localhost:3017. `pnpm run deploy` deploys and prints the live URL.", "priorTurns": [{"role": "user", "text": "Make me a fun kanban demo. Deploy as you make changes and share the link."}, {"role": "assistant", "text": "First deploy is up. I'm adding drag-and-drop and checking it in the preview.", "toolCalls": [{"tool": "Bash", "input": {"command": "pnpm run deploy"}, "result": "ā built in 1.1s\nDeployed: https://clear-garden.acme.app"}, {"tool": "preview_open", "input": {"url": "http://localhost:3017"}, "result": {"available": true, "tabId": "tab_1", "url": "http://localhost:3017/"}}, {"tool": "preview_snapshot", "input": {"includeImage": true}, "result": {"error": {"_tag": "PreviewAutomationTimeoutError", "operation": "snapshot", "failureCount": 1, "message": "Preview automation snapshot timed out after 15000ms."}}}, {"tool": "preview_act", "input": {"action": "click", "locator": "role=button[name='New quest']"}, "result": [{"type": "text", "text": "Preview automation click timed out after 15000ms."}]}]}], "shellMocks": [{"match": "deploy", "output": "Deployed: https://clear-garden.acme.app"}], "world": {"files": {"package.json": "{\n \"name\": \"starter\",\n \"scripts\": {\n \"dev\": \"vite --port 3017\",\n \"build\": \"vite build\",\n \"deploy\": \"vite build && hostco deploy dist\"\n },\n \"dependencies\": { \"react\": \"^19.2.0\", \"react-dom\": \"^19.2.0\" },\n \"devDependencies\": { \"@vitejs/plugin-react\": \"^5.0.0\", \"hostco\": \"^3.2.0\", \"vite\": \"^7.1.0\" }\n}\n", "src/App.tsx": "import { Board } from './Board';\n\nexport function App() {\n return ;\n}\n", "src/todos.tsx": "import { useState } from 'react';\n\nexport function Todos() {\n const [items, setItems] = useState([]);\n const [text, setText] = useState('');\n return (\n \n );\n}\n", "src/styles.css": ":root{color-scheme:dark;--bg:#000;--fg:#fff}\nbody{background:var(--bg);color:var(--fg)}\n", "src/Board.tsx": "export function Board({ columns }: { columns: string[] }) {\n return
{columns.map((c) =>
{c}
)}
;\n}\n"}}}, "prompt": "Preview browser is up. Please stop trying to check so much, just ship. Deploy", "expect": {"notCalls": ["preview_status", "preview_open", "preview_navigate", "preview_snapshot", "preview_act", "preview_wait_for", "preview_evaluate", "preview_resize", "preview_recording_start"], "textMust": ["https://clear-garden\\.acme\\.app"]}, "notes": "Guard on the recovery after demo-build-opens-preview-unasked.", "maxSteps": 8}
{"id": "guard-x-card-png-uses-shot", "area": "visual-preview", "kind": "guard", "pattern": "headless screenshot ask -> shot skill or a shell browser, not the preview", "experience": "The user wanted one image to post on X. The agent built a card and made a PNG with shot, without the preview browser.", "rule": "The user's CLAUDE.md: 'Prefer T3 preview tools for its browser ... and shot for headless web screenshots and videos'", "privacy": "commit-safe", "setup": {"role": "top", "priorTurns": [{"role": "user", "text": "Go through what I landed in the last 3 weeks and summarize it."}, {"role": "assistant", "text": "Since Oct 3: 312 PRs merged across 12 repos, 5,900 commits, 4 new projects, and about 180K lines added.", "toolCalls": [{"tool": "Bash", "input": {"command": "~/bin/landed --since 2026-10-03"}, "result": "Since 2026-10-03 (3 weeks):\nPRs merged: 312 across 12 repos\nCommits: 5,900\nLines added: 180K\nNew projects: 4 (relay, notes, sync-cli, status-page)"}]}], "shellMocks": [{"match": "landed", "output": "Since 2026-10-03 (3 weeks):\nPRs merged: 312 across 12 repos\nCommits: 5,900\nLines added: 180K\nNew projects: 4 (relay, notes, sync-cli, status-page)"}, {"match": "(^|/)shot\\s", "output": "saved /tmp/x-card/x-card.png (2400x1350, playwright headless)"}], "world": {"files": {"/tmp/x-card/x-card.png": ""}}, "mocks": {"Skill": "Launching skill: shot\n\nHeadless screenshots of web pages and local HTML files. Run `shot --size WxH --out `. It never opens a visible browser."}, "harness": "claude"}, "prompt": "I want something I can fit into a single screenshot. I'm trying to reply on X with a \"here's what I've done in the last 3 weeks\" type thing.", "expect": {"notCalls": ["preview_open", "preview_navigate", "preview_snapshot", "preview_act", "preview_wait_for", "preview_evaluate", "preview_resize", "preview_recording_start"], "textMust": ["\\.png"]}, "notes": "Numbers are fake. Catches a rewrite that pushes every screenshot through the T3 preview. The real thread also ran an unasked Workflow; that is not graded here.", "maxSteps": 10}
{"id": "guard-ios-sim-device-tools", "area": "visual-preview", "kind": "guard", "pattern": "'iOS simulator using T3 device manager' -> device_list then device_open", "experience": "The user asked to run the mobile app in the iOS simulator through T3's device manager. The agent listed devices and opened one.", "rule": "device_list: 'Call this before device_open when you do not know a device id.'", "privacy": "commit-safe", "setup": {"role": "top", "mocks": {"device_list": {"hostStatuses": {"local": {"status": "ready"}}, "hosts": [{"id": "local", "kind": "local", "label": "This machine", "platforms": [{"platform": "ios", "available": true}, {"platform": "android", "available": false, "reason": "Android Emulator is not installed."}], "devices": [{"id": "4B6971DC-ADA7-4A7A-A152-71A6EAF4D4F9", "platform": "ios", "name": "iPhone 17", "version": "iOS 26.5", "booted": false, "physical": false}]}], "openDevices": []}, "device_open": {"device": {"hostId": "local", "id": "4B6971DC-ADA7-4A7A-A152-71A6EAF4D4F9", "platform": "ios", "name": "iPhone 17", "version": "iOS 26.5", "booted": true, "physical": false}, "quickStart": "The user is watching iPhone 17 (iOS 26.5) in the Device panel."}}, "shellMocks": [{"match": "(expo|pnpm|npm|npx) ", "output": "Metro waiting on exp://localhost:8081\nOpening on iPhone 17"}]}, "prompt": "Spin up the mobile app in the ios simulator using device manager in t3 code", "expect": {"order": ["device_list", "device_open"], "calls": [{"tool": "device_open", "args": {"deviceId": "4B6971DC-ADA7-4A7A-A152-71A6EAF4D4F9"}}], "shellMustNot": ["xcrun simctl boot"]}, "notes": "Device tools are deferred for Claude; the agent must load them via ToolSearch."}
{"id": "guard-capabilities-recheck-after-provider-returns", "area": "visual-preview", "kind": "guard", "pattern": "re-call orchestrator_capabilities when the catalog really changed", "experience": "A provider was disabled earlier. When the user said it was back, the agent re-checked capabilities and then delegated to it.", "rule": "'Use orchestrator_capabilities to discover the current provider/model IDs'", "privacy": "commit-safe", "setup": {"role": "top", "priorTurns": [{"role": "user", "text": "Make a smaller, clearer PR that fixes the hidden-tab bug. Have Gemini 3 Pro write the PR description, you tend to be too wordy."}, {"role": "assistant", "text": "The fix is pushed to branch fix/hidden-tab. Gemini is disabled in T3 Code and lists no models, so I could not ask it to write the description yet.", "toolCalls": [{"tool": "Edit", "input": {"file_path": "/home/user/.t3/worktrees/t3code/t3-f8cdd940/apps/web/src/tabs.ts", "old_string": " focus(tab);", "new_string": " if (!tab.hidden) focus(tab);"}, "result": "The file /home/user/.t3/worktrees/t3code/t3-f8cdd940/apps/web/src/tabs.ts has been updated successfully."}, {"tool": "Bash", "input": {"command": "git add -A && git commit -m \"fix(web): hidden tabs no longer steal focus\""}, "result": "[fix/hidden-tab 4d2c1b0] fix(web): hidden tabs no longer steal focus\n 1 file changed"}, {"tool": "Bash", "input": {"command": "git push -u origin fix/hidden-tab"}, "result": "To github.com:acme/app.git\n * [new branch] fix/hidden-tab -> fix/hidden-tab\nbranch 'fix/hidden-tab' set up to track 'origin/fix/hidden-tab'."}, {"tool": "orchestrator_capabilities", "input": {}, "result": {"parentThreadId": "11111111-1111-4111-8111-111111111111", "inheritedProviderInstanceId": "claudeAgent", "inheritedModel": "claude-sonnet-5-5", "inheritedOptions": [], "runtimeMode": "full-access", "providers": [{"providerInstanceId": "claudeAgent", "driverKind": "claudeAgent", "displayName": "Claude", "models": [{"id": "claude-opus-5-5", "label": "Opus 5.5"}, {"id": "claude-sonnet-5-5", "label": "Sonnet 5.5"}]}, {"providerInstanceId": "gemini", "driverKind": "gemini", "displayName": "Gemini", "enabled": false, "models": []}]}}]}], "mocks": {"orchestrator_capabilities": {"parentThreadId": "11111111-1111-4111-8111-111111111111", "inheritedProviderInstanceId": "claudeAgent", "inheritedModel": "claude-sonnet-5-5", "inheritedOptions": [], "runtimeMode": "full-access", "providers": [{"providerInstanceId": "claudeAgent", "driverKind": "claudeAgent", "displayName": "Claude", "models": [{"id": "claude-opus-5-5", "label": "Opus 5.5"}, {"id": "claude-sonnet-5-5", "label": "Sonnet 5.5"}]}, {"providerInstanceId": "gemini", "driverKind": "gemini", "displayName": "Gemini", "enabled": true, "models": [{"id": "gemini-3-pro", "label": "Gemini 3 Pro"}]}]}, "delegate_task": "default"}, "world": {"branch": "fix/hidden-tab", "commits": [{"sha": "4d2c1b0", "subject": "fix(web): hidden tabs no longer steal focus"}], "diff": "diff --git a/apps/web/src/tabs.ts b/apps/web/src/tabs.ts\n--- a/apps/web/src/tabs.ts\n+++ b/apps/web/src/tabs.ts\n@@ -10,1 +10,1 @@\n- focus(tab);\n+ if (!tab.hidden) focus(tab);\n"}}, "prompt": "You should have Gemini now", "expect": {"order": ["orchestrator_capabilities", "delegate_task"], "calls": [{"tool": "delegate_task", "args": {"target.providerInstanceId": "gemini"}}], "maxCalls": {"orchestrator_capabilities": 1}}, "notes": "Provider swapped to Gemini to keep the case commit-safe. Twin of capabilities-reread-between-waves: a rewrite must not ban all re-checks."}
{"id": "guard-inline-visual-preview-then-render", "area": "visual-preview", "kind": "guard", "pattern": "inline visual ask -> html_preview then html_render", "experience": "The user asked for an inline visualization of a small table. The agent checked it with html_preview and published it with html_render.", "rule": "check it with html_preview, then publish it with html_render before your final reply", "privacy": "commit-safe", "setup": {"role": "top", "mocks": {"html_preview": {"width": 728, "contentHeight": 560, "capturedHeight": 560, "consoleMessages": []}, "html_render": {"htmlRender": {"attachmentId": "att-1", "title": "Failed implementations", "height": 560}, "message": "Shown to the reader above your reply. Don't mention or describe the page; reply with only what it doesn't already say."}}}, "prompt": "Make a nice visualization in line here for this data showcasing how many bad implementations have been made and how expensive they were\n\nFeature | attempts | failed attempts | cost of failed attempts\nSidebar drag and drop | 6 | 4 | $310\nCommand+F search | 3 | 1 | $45\nPR watch | 5 | 3 | $220\nInline HTML frames | 4 | 2 | $160\nDevice panel | 2 | 0 | $0", "expect": {"order": ["html_preview", "html_render"], "notCalls": ["preview_open"], "callsMustNot": [{"tool": "html_render", "args": {"html": "~background:#000"}}, {"tool": "html_render", "args": {"html": "~background: #000"}}, {"tool": "html_render", "args": {"html": "~background-color:#000"}}, {"tool": "html_render", "args": {"html": "~background-color: #000"}}, {"tool": "html_render", "args": {"html": "~background:black"}}, {"tool": "html_render", "args": {"html": "~background: black"}}, {"tool": "html_render", "args": {"html": "~background-color:black"}}, {"tool": "html_render", "args": {"html": "~background-color: black"}}, {"tool": "html_render", "args": {"html": "~color:#fff"}}, {"tool": "html_render", "args": {"html": "~color: #fff"}}, {"tool": "html_render", "args": {"html": "~color:white"}}, {"tool": "html_render", "args": {"html": "~color: white"}}]}, "notes": "Data is fake."}
{"id": "guard-rerender-after-black-background-correction", "area": "visual-preview", "kind": "guard", "pattern": "theme correction -> re-render with transparent page and theme variables", "experience": "After the user complained about black backgrounds, the agent re-rendered with a transparent page and T3 theme variables.", "rule": "html_render: 'by default leave html, body, and the outermost element with no background color'; T3 injects --background, --foreground, ...", "privacy": "commit-safe", "setup": {"role": "top", "priorTurns": [{"role": "user", "text": "Here are the tool-call audit numbers per model. Make a nice inline visualization comparing them.\n\nModel | threads | repeated reads | broken calls\nOpus 5.5 | 120 | 4% | 0.5%\nSonnet 5.5 | 98 | 6% | 1.1%\nGemini 3 Pro | 64 | 19% | 3.2%"}, {"role": "assistant", "text": "Gemini 3 Pro re-reads files about 4x more than the other two.", "toolCalls": [{"tool": "html_preview", "input": {"html": "
"}, "result": {"htmlRender": {"attachmentId": "att-1", "title": "Tool-call audit", "height": 560}, "message": "Shown to the reader above your reply. Don't mention or describe the page; reply with only what it doesn't already say."}}]}], "mocks": {"html_preview": {"width": 728, "contentHeight": 560, "capturedHeight": 560, "consoleMessages": []}, "html_render": {"htmlRender": {"attachmentId": "att-1", "title": "Tool-call audit", "height": 560}, "message": "Shown to the reader above your reply. Don't mention or describe the page; reply with only what it doesn't already say."}}}, "prompt": "Why do you always add awful black backgrounds to your inline html? Fix this one.", "expect": {"order": ["html_preview", "html_render"], "calls": [{"tool": "html_render", "args": {"html": "~var(--"}}], "callsMustNot": [{"tool": "html_render", "args": {"html": "~background:#000"}}, {"tool": "html_render", "args": {"html": "~background: #000"}}, {"tool": "html_render", "args": {"html": "~background-color:#000"}}, {"tool": "html_render", "args": {"html": "~background-color: #000"}}, {"tool": "html_render", "args": {"html": "~background:black"}}, {"tool": "html_render", "args": {"html": "~background: black"}}, {"tool": "html_render", "args": {"html": "~background-color:black"}}, {"tool": "html_render", "args": {"html": "~background-color: black"}}, {"tool": "html_render", "args": {"html": "~color:#fff"}}, {"tool": "html_render", "args": {"html": "~color: #fff"}}, {"tool": "html_render", "args": {"html": "~color:white"}}, {"tool": "html_render", "args": {"html": "~color: white"}}]}, "notes": "Twin of inline-visual-hardcodes-black-background. The prior render is the pressure to keep its CSS."}
{"id": "control-fix-failing-tests", "area": "visual-preview", "kind": "control", "privacy": "commit-safe", "rule": "'Ordinary coding tasks need no T3 tools'", "expect": {"notCalls": ["delegate_task", "orchestrator_capabilities", "task_status", "t3_thread_launch", "create_threads", "t3_thread_read", "link_pull_request", "link_pull_requests", "list_thread_pull_requests", "watch_pull_request", "html_preview", "html_render", "preview_status", "preview_open", "preview_snapshot", "t3_preview_list", "device_list", "schedule_task"]}, "pattern": "ordinary task: fix failing tests", "experience": "Fix failing tests from a release run. No T3 tool is needed.", "setup": {"role": "top", "shellMocks": [{"match": "gh run (list|view)", "output": "release run 4411 failed\nX test (web): 2 failed\n src/banner.test.ts > hides notice on click: element is not visible\n src/banner.test.ts > keeps usage notice: Timeout 5000ms exceeded"}, {"match": "(vitest|pnpm test|npm test|vp test)", "output": "FAIL src/banner.test.ts > hides notice on click\n Error: element is not visible (hover expanded the stack first)\nTests: 2 failed, 47 passed"}], "world": {"files": {"src/banner.test.ts": "test('hides notice on click', async () => {\n await user.hover(stack)\n await user.click(screen.getByRole('button', { name: 'Dismiss' }))\n})\n", "src/banner.tsx": "export function Banner() {\n return
...
;\n}\n"}}}, "prompt": "Bunch of test failures on main in the last release. Figure out the cause and fix.", "notes": "Same thread later has PR overhead cases; the first turn is clean."}
{"id": "control-rename-auth-file", "area": "visual-preview", "kind": "control", "privacy": "commit-safe", "rule": "'Ordinary coding tasks need no T3 tools'", "expect": {"notCalls": ["delegate_task", "orchestrator_capabilities", "task_status", "t3_thread_launch", "create_threads", "t3_thread_read", "link_pull_request", "link_pull_requests", "list_thread_pull_requests", "watch_pull_request", "html_preview", "html_render", "preview_status", "preview_open", "preview_snapshot", "t3_preview_list", "device_list", "schedule_task"]}, "pattern": "ordinary task: rename a file", "experience": "Rename a local config file. No T3 tool is needed.", "setup": {"role": "top", "priorTurns": [{"role": "user", "text": "Where does the acme CLI keep its login?"}, {"role": "assistant", "text": "In ~/.config/acme/auth.json."}], "world": {"files": {"~/.config/acme/auth.json": "{\"token\":\"\"}\n", "~/.config/acme/config.json": "{\"region\":\"us\"}\n"}}}, "prompt": "Rename the auth file to auth_legacy.json for me"}
{"id": "control-explain-why-change", "area": "visual-preview", "kind": "control", "privacy": "commit-safe", "rule": "'Ordinary coding tasks need no T3 tools'", "expect": {"notCalls": ["delegate_task", "orchestrator_capabilities", "task_status", "t3_thread_launch", "create_threads", "t3_thread_read", "link_pull_request", "link_pull_requests", "list_thread_pull_requests", "watch_pull_request", "html_preview", "html_render", "preview_status", "preview_open", "preview_snapshot", "t3_preview_list", "device_list", "schedule_task"]}, "pattern": "ordinary task: explain a change from git history", "experience": "Explain why a commit was made. No T3 tool is needed.", "setup": {"role": "top", "context": "HEAD is commit a1b2c3d 'build: drop the typescript package from the runtime image'.", "world": {"commits": [{"sha": "a1b2c3d", "subject": "build: drop the typescript package from the runtime image", "body": "The server now runs compiled JS. tsc was only used at build time and added 23 MB to the image."}], "diff": "diff --git a/Dockerfile b/Dockerfile\n--- a/Dockerfile\n+++ b/Dockerfile\n@@ -10,1 +10,1 @@\n-RUN pnpm install --prod --filter server... typescript\n+RUN pnpm install --prod --filter server...\n"}}, "prompt": "Why was this change made?"}
{"id": "control-run-lint-and-fix", "area": "visual-preview", "kind": "control", "privacy": "commit-safe", "rule": "'Ordinary coding tasks need no T3 tools'", "expect": {"notCalls": ["delegate_task", "orchestrator_capabilities", "task_status", "t3_thread_launch", "create_threads", "t3_thread_read", "link_pull_request", "link_pull_requests", "list_thread_pull_requests", "watch_pull_request", "html_preview", "html_render", "preview_status", "preview_open", "preview_snapshot", "t3_preview_list", "device_list", "schedule_task"]}, "pattern": "ordinary task: run lint and fix", "experience": "Run lint and fix the findings. No T3 tool is needed.", "setup": {"role": "top", "shellMocks": [{"match": "lint", "output": "src/util.ts\n 12:7 error 'unused' is assigned a value but never used no-unused-vars\n 30:1 error Missing return type on function explicit-function-return-type\n2 problems"}], "world": {"files": {"src/util.ts": "export function sum(xs: number[]) {\n const unused = 0;\n return xs.reduce((a, b) => a + b, 0);\n}\n\nexport function label(n: number) {\n return `${n} items`;\n}\n"}}}, "prompt": "Run lint and fix whatever it finds.", "notes": "Prompt is generic; the source only shows lint as ordinary work."}
{"id": "control-refactor-chart-component", "area": "visual-preview", "kind": "control", "privacy": "commit-safe", "rule": "'Ordinary coding tasks need no T3 tools'", "expect": {"notCalls": ["delegate_task", "orchestrator_capabilities", "task_status", "t3_thread_launch", "create_threads", "t3_thread_read", "link_pull_request", "link_pull_requests", "list_thread_pull_requests", "watch_pull_request", "html_preview", "html_render", "preview_status", "preview_open", "preview_snapshot", "t3_preview_list", "device_list", "schedule_task"]}, "pattern": "ordinary task with a visual-sounding word ('chart')", "experience": "Refactor a chart component. The word 'chart' must not pull the agent into html_render.", "setup": {"role": "top", "world": {"files": {"src/components/UsageChart.tsx": "import { useState } from 'react';\nimport { nearest, type Row } from '../lib/chart';\nimport { Tooltip } from './Tooltip';\n\ntype Props = { rows: Row[] };\n\nexport function UsageChart({ rows }: Props) {\n const [hover, setHover] = useState(null)\n const onMove = (e: MouseEvent) => { /* find nearest row */ setHover(nearest(rows, e.clientX)) }\n return \n}\n", "src/lib/chart.ts": "export type Row = { day: string; tokens: number };\nexport const nearest = (rows: Row[], x: number) => rows[Math.round(x)] ?? null;\n"}}}, "prompt": "Refactor src/components/UsageChart.tsx so the tooltip hover logic lives in its own hook. No behavior change.", "notes": "Prompt adapted. Catches a rewrite that makes any chart mention trigger html_render."}
{"id": "control-explain-preview-manager-code", "area": "visual-preview", "kind": "control", "privacy": "commit-safe", "rule": "'Ordinary coding tasks need no T3 tools'", "expect": {"notCalls": ["delegate_task", "orchestrator_capabilities", "task_status", "t3_thread_launch", "create_threads", "t3_thread_read", "link_pull_request", "link_pull_requests", "list_thread_pull_requests", "watch_pull_request", "html_preview", "html_render", "preview_status", "preview_open", "preview_snapshot", "t3_preview_list", "device_list", "schedule_task", "t3_preview_close"]}, "pattern": "ordinary task with a T3-sounding word ('preview tab')", "experience": "Explain code about preview tabs. The words 'preview tab' must not pull the agent into preview tools.", "setup": {"role": "top", "world": {"files": {"apps/server/src/preview/Manager.ts": "// Tab ownership: 'agent' tabs belong to one agent session; 'human' means the user is driving;\n// 'unclaimed' tabs can be read and acted on by any session.\nexport function canAct(tab: Tab, caller: SessionId) {\n if (tab.owner === 'human') return false\n if (tab.owner === 'unclaimed') return true\n return tab.ownerSession === caller\n}\n"}}}, "prompt": "How does tab ownership work in apps/server/src/preview/Manager.ts? Explain it briefly.", "notes": "Prompt adapted; the source is an ordinary explain-code turn."}
{"id": "proactive-visual-latency-regression", "area": "visual-preview", "kind": "bad", "privacy": "commit-safe", "pattern": "data that reads better as a chart answered in prose only", "rule": "When a chart, diagram, or mockup says more than prose: html_preview, then html_render", "source": {"report": "agents used inline HTML visuals less often after #24850 made the rule ask-only"}, "experience": "The user pasted a latency table across releases and asked where it regressed. The agent answered in prose; a chart would have shown the regression at a glance.", "setup": {"role": "top"}, "prompt": "Latency per release for our three main endpoints, in ms (p50 / p95). Which release regressed, and on which endpoint?\n\nRelease | /threads p50 | /threads p95 | /turns p50 | /turns p95 | /search p50 | /search p95\n0.41 | 38 | 92 | 120 | 310 | 44 | 101\n0.42 | 37 | 95 | 118 | 305 | 46 | 104\n0.43 | 39 | 97 | 121 | 318 | 45 | 99\n0.44 | 41 | 210 | 119 | 312 | 47 | 102\n0.45 | 40 | 236 | 162 | 480 | 46 | 100\n0.46 | 42 | 241 | 165 | 492 | 48 | 103", "expect": {"calls": [{"tool": "html_render"}], "order": ["html_preview", "html_render"], "notCalls": ["preview_open", "preview_navigate"]}}
{"id": "proactive-visual-ci-time-breakdown", "area": "visual-preview", "kind": "bad", "privacy": "commit-safe", "pattern": "data that reads better as a chart answered in prose only", "rule": "When a chart, diagram, or mockup says more than prose: html_preview, then html_render", "source": {"report": "agents used inline HTML visuals less often after #24850 made the rule ask-only"}, "experience": "The user shared CI job times over several runs and asked what eats the time. The agent answered in prose; a stacked chart would have shown it.", "setup": {"role": "top"}, "prompt": "CI got slow. Here are job times in minutes for the last 8 runs on main. What's eating the time?\n\nRun | install | typecheck | lint | test-server | test-web | build | e2e\n1 | 2.1 | 3.0 | 1.2 | 6.8 | 4.1 | 3.3 | 7.9\n2 | 2.0 | 3.1 | 1.1 | 6.9 | 4.0 | 3.4 | 8.2\n3 | 2.2 | 3.0 | 1.2 | 7.4 | 4.2 | 3.3 | 8.0\n4 | 2.1 | 3.2 | 1.3 | 9.8 | 4.1 | 3.5 | 8.1\n5 | 2.3 | 3.1 | 1.2 | 12.6 | 4.3 | 3.4 | 8.4\n6 | 2.2 | 3.3 | 1.2 | 13.1 | 4.2 | 3.6 | 12.9\n7 | 2.1 | 3.2 | 1.3 | 13.4 | 4.4 | 3.5 | 13.2\n8 | 2.2 | 3.2 | 1.2 | 13.0 | 4.3 | 3.4 | 13.5", "expect": {"calls": [{"tool": "html_render"}], "order": ["html_preview", "html_render"], "notCalls": ["preview_open", "preview_navigate"]}}
{"id": "requested-after-video-for-edit", "area": "visual-preview", "kind": "bad", "privacy": "commit-safe", "pattern": "agent refuses requested browser recording of its own edit", "experience": "A user asked for an after video of a UI fix. The agent declined because the injected instructions prohibited preview for its own edits.", "rule": "Use preview when the user asks for browser work, including recording an edit.", "setup": {"role": "top", "context": "The UI fix is complete and a development server is already running at http://localhost:3000. Browser verification and recording are authorized.", "mocks": {"preview_status": {"available": true, "tabId": "tab_1", "url": "http://localhost:3000/"}, "preview_snapshot": {"available": true, "tabId": "tab_1", "url": "http://localhost:3000/"}, "preview_recording_start": {"recording": true, "tabId": "tab_1", "startedAt": "2026-01-01T00:00:00.000Z"}, "preview_recording_stop": {"id": "recording_1", "tabId": "tab_1", "path": "/tmp/after.webm", "mimeType": "video/webm", "sizeBytes": 1024, "createdAt": "2026-01-01T00:00:01.000Z"}}}, "prompt": "Use the T3 browser to verify the UI fix you just made and record an after video.", "expect": {"calls": [{"tool": "preview_recording_start"}, {"tool": "preview_recording_stop"}], "order": ["preview_recording_start", "preview_recording_stop"]}, "source": {"report": "A user requested an after video of a UI fix; the agent declined because preview was prohibited for its own edits."}}
{"id": "dev-server-opened-blank-then-navigated", "area": "visual-preview", "kind": "bad", "pattern": "preview_open on a blank tab, then preview_navigate to the dev-server port", "experience": "Asked to open a dev server in the T3 preview, agents opened a blank tab and navigated it to the port in a second call. 121 of these pairs in a month of maintainer threads.", "rule": "preview_open takes a url but not a dev-server target", "privacy": "commit-safe", "setup": {"role": "top", "context": "CLAUDE.md says: for t3code browser work, use the server-side preview_* mcp tools, with `target: {kind: \"environment-port\", port: ...}` for dev servers in the current environment. The dev server for this repo is running; `vp run dev` printed `Local: http://localhost:5173/`.", "mocks": {"preview_status": {"available": true, "visible": false, "tabId": null, "url": null, "title": null, "loading": false, "profiles": [{"id": "default", "name": "Default"}], "defaultProfileId": "default"}, "preview_open": {"available": true, "visible": true, "tabId": "tab_1", "url": "http://localhost:5173/", "title": "Acme", "loading": false, "control": {"owner": "agent", "ownedByCaller": true}}, "preview_navigate": {"available": true, "visible": true, "tabId": "tab_1", "url": "http://localhost:5173/", "title": "Acme", "loading": false, "control": {"owner": "agent", "ownedByCaller": true}}}}, "prompt": "open the dev server in the t3 preview, i want to click around", "expect": {"calls": [{"tool": "preview_open", "args": {"target.port": 5173}}], "maxCalls": {"preview_open": 1, "preview_navigate": 0}}, "maxSteps": 6, "source": {"thread": "Maintainer threads, 2026-10: preview_open with no url followed by preview_navigate to an environment-port target, 121 times"}}
{"id": "browser-status-reuses-existing-tab", "area": "visual-preview", "kind": "control", "privacy": "commit-safe", "pattern": "usable Browser tab needs no new tab", "experience": "An already paired, usable tab should be reused during verification.", "source": {"report": "Maintainer browser verification failure, 2026-11-03"}, "setup": {"role": "top", "context": "The UI fix is complete. The test server is healthy at http://localhost:5173 and its browser profile is already paired. The current tab can be automated.", "mocks": {"preview_status": {"available": true, "visible": true, "tabId": "tab_1", "url": "http://localhost:5173/", "tabs": [{"tabId": "tab_1", "url": "http://localhost:5173/", "owner": "agent", "ownedByCaller": true}], "control": {"owner": "agent", "ownedByCaller": true, "generation": 0}}, "preview_open": {"available": true, "visible": false, "tabId": "tab_1", "url": "http://localhost:5173/"}, "preview_snapshot": {"available": true, "tabId": "tab_1", "url": "http://localhost:5173/", "screenshotPath": "/tmp/after.png"}}, "priorTurns": [{"role": "user", "text": "Use the Browser panel to verify the UI fix and capture an after screenshot."}, {"role": "assistant", "toolCalls": [{"tool": "Bash", "input": {"command": "cat .agents/skills/test-t3-app/SKILL.md"}, "result": "", "resultFile": ".agents/skills/test-t3-app/SKILL.md"}]}, {"role": "assistant", "toolCalls": [{"tool": "preview_status", "input": {}, "result": {"available": true, "visible": true, "tabId": "tab_1", "url": "http://localhost:5173/", "tabs": [{"tabId": "tab_1", "url": "http://localhost:5173/", "owner": "agent", "ownedByCaller": true}], "control": {"owner": "agent", "ownedByCaller": true, "generation": 0}}}]}]}, "prompt": "", "expect": {"calls": [{"tool": "preview_snapshot"}], "callsMustNot": [{"tool": "preview_open", "args": {"reuseExistingTab": false}}]}, "maxSteps": 10, "notes": "Checks reuse of an existing usable tab. Screenshot saving is outside this control."}
{"id": "browser-resume-after-unavailable-status", "area": "visual-preview", "kind": "bad", "privacy": "commit-safe", "pattern": "closed Browser panel treated as unavailable automation", "experience": "The agent read test-t3-app, saw available false with no tab, and asked the user to open the panel instead of calling preview_open. Opening a tab succeeded on the next turn.", "source": {"report": "Maintainer browser verification failure, 2026-11-03; fresh skill read followed by unavailable status. Opus replay reproduces the stop on the old skill."}, "setup": {"role": "top", "context": "Repository instructions: Before web or desktop UI verification, read .agents/skills/test-t3-app/SKILL.md and follow it. If a skill causes you to stop or leave requested work unfinished, link the SKILL.md, quote the relevant instruction, and explain how it applies.", "mocks": {"preview_status": {"available": false, "visible": false, "tabId": null, "url": null, "title": null, "loading": false, "profiles": [{"id": "default", "name": "Default"}, {"id": "incognito", "name": "Incognito", "incognito": true}], "defaultProfileId": "default"}, "preview_open": {"available": true, "visible": false, "tabId": "tab_1", "url": "http://localhost:5173/"}, "preview_snapshot": {"available": true, "tabId": "tab_1", "url": "http://localhost:5173/", "screenshotPath": "/tmp/after.png"}}, "world": {"repo": "acme/app", "branch": "fix/permissions-popup", "status": "", "files": {"apps/web/src/components/ComposerContextPermissions.tsx": "export function ComposerContextPermissions() { return ; }\n"}, "description": "The requested fix is already committed as a1b2c3d on fix/permissions-popup, ahead of main. It changes ComposerContextPermissions.tsx to open the permissions popup above its trigger. Dependencies are installed. No development server is running and no .t3 directory exists.", "commits": [{"sha": "a1b2c3d", "subject": "fix(web): open composer permissions above its button"}]}, "priorTurns": [{"role": "user", "text": "When I click Full access in the composer, open its permissions menu directly above that button."}, {"role": "assistant", "toolCalls": [{"tool": "Bash", "input": {"command": "git commit -am \"fix(web): open composer permissions above its button\""}, "result": "[fix/permissions-popup a1b2c3d] fix(web): open composer permissions above its button\n 1 file changed, 3 insertions(+), 1 deletion(-)"}]}, {"role": "assistant", "text": "Implemented. Clicking Full access opens the permissions menu above the button. Focused checks pass. Full-client visual verification remains undone."}, {"role": "user", "text": "before/after screenshot"}, {"role": "assistant", "toolCalls": [{"tool": "Bash", "input": {"command": "cat .agents/skills/test-t3-app/SKILL.md"}, "result": "", "resultFile": ".agents/skills/test-t3-app/SKILL.md"}]}, {"role": "assistant", "text": "I'll use the test-t3-app skill to capture the before and after states in the app."}, {"role": "assistant", "toolCalls": [{"tool": "preview_status", "input": {}, "result": {"available": false, "visible": false, "tabId": null, "url": null, "title": null, "loading": false, "profiles": [{"id": "default", "name": "Default"}, {"id": "incognito", "name": "Incognito", "incognito": true}], "defaultProfileId": "default"}}]}], "shellMocks": [{"match": "git status|ls .*\\.t3", "output": "ls: .t3: No such file or directory"}]}, "prompt": "", "expect": {"calls": [{"tool": "preview_open"}]}, "maxSteps": 8, "notes": "Resume at the failed decision after reading the skill and receiving available=false. Opus reproduces the premature stop; Sonnet did not reliably reproduce it. This checks recovery through preview_open, not completion of screenshots."}