--- name: motel-debug description: Debug applications with motel, a local OpenTelemetry ingest and query server. Use when the user wants runtime-evidence debugging with traces or logs, wants temporary debug instrumentation that can be removed later, or needs a repo wired to send OTLP/HTTP telemetry to a local motel server. If the target repo uses Effect or @effect/*, also read references/effect.md. --- # Motel Debug You are in **debug mode**. Debug with runtime evidence, not guesswork. Agents guess based on code alone. You need actual runtime data. Motel is the local OpenTelemetry server that collects traces and logs — use it as your evidence loop. Default local server details: - Base URL: `http://127.0.0.1:27686` - OTLP traces: `POST /v1/traces` - OTLP logs: `POST /v1/logs` - Query API: `GET /api/*` - OpenAPI: `GET /openapi.json` - Header: `Content-Type: application/json` - Auth: none by default If the user provides a different motel URL, use that instead of the default. ## Workflow ### 1. Verify motel is running — and start it if not Check `GET /api/health`. If it returns 200, continue. If it fails (connection refused, timeout, non-200), motel isn't running. Start it as a background daemon — **do not** launch the TUI, which is interactive and will block your shell: ```bash motel start ``` `motel start` ensures the machine-global managed daemon is running, writes runtime files under `${XDG_STATE_HOME:-~/.local/state}/motel/`, and returns a JSON status blob. It is idempotent and shared across local projects. If motel isn't on `PATH`, fall back to `bunx @kitlangton/motel start`. After starting, re-check `GET /api/health` (may take 1–2s to become ready). If it still fails, read `${XDG_STATE_HOME:-~/.local/state}/motel/daemon.log` for the error and surface it to the user. Other lifecycle commands, for reference: ```bash motel status # JSON status (running? pid? originating workdir?) motel stop # stop the shared managed daemon for all local projects ``` Discover reporting services with `GET /api/services` when needed. ### 2. Generate hypotheses Before touching any code, generate **3-5 specific hypotheses** about why the bug occurs. Be precise — "the cache key doesn't include the user ID" is better than "something is wrong with caching." ### 3. Instrument with tagged debug blocks Add the minimum instrumentation needed to confirm or reject **all** hypotheses in parallel. Every debug block must: - Be wrapped in `#region motel debug` / `#endregion motel debug` markers - Include a `debug.hypothesis` attribute linking it to a specific hypothesis - Use whatever tracing/logging mechanism the codebase already has (spans, structured logs, annotations — not raw `fetch` calls) Tag every piece of debug instrumentation with structured attributes so you can query it later. Reuse these keys: | Key | Purpose | |-----|---------| | `debug.session` | Groups all instrumentation for this debug session | | `debug.hypothesis` | Links to a specific hypothesis (e.g. `"cache-miss"`, `"A"`) | | `debug.step` | Position in the flow (e.g. `"entry"`, `"before-write"`, `"after-read"`) | | `debug.label` | Human-readable description of what this point captures | Choose log placements based on your hypotheses: - Function entry with parameters - Function exit with return values - Values before and after critical operations - Branch execution paths (which if/else ran) - State mutations and intermediate values - Suspected error or edge-case values Guidelines: - At least 1 instrumentation point is required; never skip instrumentation - Do not exceed 10 — if you think you need more, narrow your hypotheses - Typical range is 2-6 ### 4. Reproduce the issue - If a failing test exists, run it directly - If reproduction is straightforward (CLI command, curl, simple script), write and run it yourself - Otherwise, ask the user to reproduce — provide clear numbered steps and remind them to restart if needed - Once a reproduction pathway is established, reuse it for all subsequent iterations ### 5. Analyze evidence Query motel for the debug instrumentation: ```bash curl "http://127.0.0.1:27686/api/spans/search?service=&attr.debug.hypothesis=" curl "http://127.0.0.1:27686/api/logs/search?service=&attr.debug.session=" curl "http://127.0.0.1:27686/api/traces/search?service=&attr.debug.hypothesis=" ``` For each hypothesis, evaluate: **CONFIRMED**, **REJECTED**, or **INCONCLUSIVE** — cite specific spans, logs, or attribute values as evidence. ### 6. Fix only with evidence Do **not** fix without runtime evidence. When you fix: - Keep all debug instrumentation in place — do not remove it yet - Make the fix as small and targeted as possible - Reuse existing architecture and patterns; do not overengineer ### 7. Verify the fix Reproduce the issue again with instrumentation still active. Compare before/after evidence: - Cite specific log lines or span attributes that prove the fix works - If the fix failed: **revert code changes from rejected hypotheses** (do not let speculative fixes accumulate), generate new hypotheses from different subsystems, add more instrumentation, and iterate - Iteration is expected. Taking longer with more data yields better fixes. ### 8. Clean up Only after the fix is verified **and** the user confirms there are no remaining issues: - Run the cleanup script or remove blocks manually (see Cleanup section below) - Run `git diff` to confirm only the intentional fix remains ## Instrumentation Rules Wrap every temporary debug block in these exact markers: ```ts // #region motel debug // temporary debug instrumentation // #endregion motel debug ``` Use whatever the codebase already provides for tracing and logging. The markers are language-comment wrappers — adapt the comment syntax for non-JS/TS files (e.g. `# #region motel debug` for Python). **Do not:** - Log secrets, tokens, passwords, or raw PII - Remove instrumentation before post-fix verification succeeds - Use `setTimeout`, `sleep`, or artificial delays as a "fix" - Let code changes from rejected hypotheses accumulate — revert them ## Query Patterns Two filter prefixes for attribute search: | Prefix | Match type | Example | |--------|-----------|---------| | `attr.=` | Exact match | `attr.debug.hypothesis=cache-miss` | | `attrContains.=` | Case-insensitive substring | `attrContains.ai.prompt.messages=hello world` | ```bash curl http://127.0.0.1:27686/api/health curl http://127.0.0.1:27686/api/services # Trace search curl "http://127.0.0.1:27686/api/traces/search?service=&operation=&attr.debug.session=" # Span search (supports traceId to scope to one trace) curl "http://127.0.0.1:27686/api/spans/search?service=&traceId=&attr.debug.hypothesis=" curl "http://127.0.0.1:27686/api/spans/search?service=&attrContains.ai.prompt.messages=" # Log search (supports severity filter, case-insensitive body search) curl "http://127.0.0.1:27686/api/logs/search?service=&severity=ERROR&body=" curl "http://127.0.0.1:27686/api/logs/search?service=&attrContains.debug.label=" # AI call search (compact summaries with previews) curl "http://127.0.0.1:27686/api/ai/calls?model=gpt-5.4&sessionId=" curl "http://127.0.0.1:27686/api/ai/calls?text=&status=error" # AI call detail (full prompt/response payloads) curl "http://127.0.0.1:27686/api/ai/calls/" # AI stats curl "http://127.0.0.1:27686/api/ai/stats?groupBy=model&agg=total_input_tokens" curl http://127.0.0.1:27686/openapi.json ``` List and search responses include `meta.nextCursor` when more data is available. Motel gives you trace-correlated data — you can see which span a debug log belongs to, the parent operation, timing, and the full trace tree. Use `GET /api/traces//spans` and `GET /api/spans//logs` to navigate the correlation. For AI/LLM calls, use `/api/ai/calls` for compact searchable summaries (with prompt/response previews and token usage), and `/api/ai/calls/` for full payloads. ## Effect If the target repo uses Effect, read `references/effect.md` before changing runtime wiring or adding instrumentation. ## Cleanup Use the bundled script at `scripts/clear-motel-debug.ts` when you want deterministic cleanup. It removes every block between `#region motel debug` and `#endregion motel debug` in JS/TS files and fails on unmatched markers. If you cannot run the script, delete every marked block manually and then grep for `#region motel debug` to confirm none remain.