# Lifecycle, distribution, and testing

How scouts get discovered, scheduled, and dispatched; the two distribution paths and their exact mechanics; and how to test a scout in each.

## How a scout runs

- **Discovery.** A scout is a skill that holds a `SignalScoutConfig`, and the coordinator dispatches from those config rows.
  The `signals-scout-` name prefix is optional; it controls only auto-registration, below.
- **Config.** Each scout has one `SignalScoutConfig` per `(project, skill_name)` carrying its schedule (`run_interval_minutes`, default 1440, or a project-local `run_cron_schedule` that takes precedence when set), `enabled`, `emit`, `network_access` (`trusted` default, `full` for scouts that read arbitrary external sites), the rest of the run posture (`output_destinations`, `structured_output_schema`, `write_scopes`, `mcp_gateway_server_ids`, `model`, `tags`, `auto_pause_exempt`, `display_name`), and a `last_run_at` stamp.
  A config is **auto-registered** the first time the coordinator sees a `signals-scout-*` skill without one, so authoring a prefixed skill is enough to get a scout.
  A skill named anything else needs its config created with it.
  Create a fresh per-team scout and its config together with `posthog:scout-create`; the nested `config` object sets its schedule, emit posture, and destinations before it can run, and `files` bundles reference files in the same call.
  The lower-level `posthog:scout-config-create` remains available when a skill already exists without a config.
  Config responses also carry the scout's `description`, read live from the skill's frontmatter (not a config field you set), plus `scout_origin` (`canonical` or `custom`) and `owners`.
- **Coordinator.** A periodic Temporal workflow ticks (~every 30 min).
  Each tick it bounds candidates to projects enrolled via the `signals-scout` feature-flag allowlist, then dispatches every **enabled** scout whose schedule is **due**, most-overdue first, capped per tick.
  On a rolling interval, due means `last_run_at is None` (a never-dispatched scout is maximally overdue; manual runs do not set the stamp) or `now - last_run_at ≥ run_interval_minutes`.
  On a cron schedule, due means the first slot after the latest of `last_run_at`, the last schedule edit, and the config's creation has passed, so a fresh or re-scheduled cron scout waits for its next slot instead of firing at once.
  There is no sampling — every due scout runs.
  `last_run_at` advances for everything dispatched.
- **Run.** Each dispatched scout becomes one sandboxed agent run with a hard budget of 15 minutes; a run still going at the wall is killed and its row marked failed.
  The body is the system prompt; the agent orients, explores, files reports or remembers, and writes a one-paragraph summary to the run row.
  A streak of consecutive **scheduled** failures longer than a twelve-hour outage could explain trips a breaker that pauses the scout (`pause_reason=repeated_failures`): the threshold is one more than the runs the schedule fits in twelve hours, clamped to 5–25 (five for a daily scout, thirteen for a rolling hourly interval, fifteen for an hourly cron, whose slots are counted over a window padded for daylight-saving shifts). Manual (`scout-run-now`) and workflow-triggered failures never count toward the streak, while a clean run from any trigger clears it. The coordinator then probes a paused scout once a day and resumes it on a clean run, unless the project is at its enabled-scout cap, in which case it stays paused until another scout is paused or deleted; enabling it by hand is refused at the cap too.

Pausing a scout = `enabled=false`.
That records `status=paused_by_user`, which automatic lifecycle sweeps never resume or re-pause; `enabled=true` resumes from any pause, including a system-applied one (`status=paused_by_system`, cause in the read-only `pause_reason`).
Config responses expose `status` and `pause_reason` read-only; writes flow through `enabled`.
Slowing it = a larger `run_interval_minutes` (or, on a cron scout, a sparser `run_cron_schedule`; the cron wins while it is set).
Dry-running it = `emit=false`.
Letting it reach sites outside the trusted-domain allowlist = `network_access="full"`.
All of these via `posthog:scout-config-update` (get the `id` from `-config-list`), or set at creation time in the nested `config` object passed to `posthog:scout-create`.

## Path A — per-team (skills store)

The common path for a user customizing scouts for their own project.
A scout is just an `LLMSkill` row named `signals-scout-*`; create or edit it with the skills-store tools, and the harness globs it in on the next tick.

```text
# List existing scouts and other skills
posthog:skill-list {"search": "signals-scout"}

# Read a canonical scout to use as a template
posthog:skill-get {"skill_name": "signals-scout-error-tracking"}

# New scout from scratch: create the complete definition and config.
posthog:scout-create {"name": "signals-scout-<scope>", "description": "...", "body": "...", "config": {"run_interval_minutes": 120}}

# Adapt an existing per-team scout — use the SMALLEST primitive (find/replace, not full-body)
posthog:skill-get {"skill_name": "signals-scout-<scope>"}          # get current version first
posthog:skill-update {"skill_name": "signals-scout-<scope>", "base_version": N, "edits": [{"old": "...", "new": "..."}]}

# Duplicate a canonical scout into a new per-team scout you then edit (keeps the canonical intact)
posthog:skill-duplicate {"skill_name": "signals-scout-general", "new_name": "signals-scout-<scope>"}

# Bundle a reference file onto a per-team scout
posthog:skill-file-create {"skill_name": "signals-scout-<scope>", "path": "references/cookbook.md", "content": "...", "content_type": "text/markdown", "base_version": N}
```

Notes:

- Prefer `edits` (find/replace) over a full `body` rewrite for tweaks — a full rewrite forces you to reproduce the whole body and risks silently dropping unrelated content.
  Each `old` must match exactly once.
  Every write bumps an immutable `version`; chain further edits via `base_version`.
- **Divergence:** once you edit a canonical scout's row for your team, canonical sync treats it as **diverged** and stops force-updating it — you keep your edits but lose upstream improvements to that scout.
  To customize _without_ diverging, `duplicate` the canonical scout into a new `signals-scout-<your-scope>` row and edit that; leave the original alone.
- Writing reports needs the `signal_scout_report:write` scope, and the scratchpad needs `signal_scout_internal:write` (the sandbox has both).
  Authoring a scout doesn't require either — only the harness writes.

## Path B — canonical (in-repo, for PostHog contributors)

Improving a scout for **every** enrolled project.
Disk under `products/signals/skills/signals-scout-*/` is the source of truth; `lazy_seed` mirrors changes onto each enrolled team's `LLMSkill` rows on the next coordinator tick (or immediately via `python manage.py sync_signals_scout_skills --all-enabled`).
Teams that hand-edited a row are diverged and left alone.

```sh
hogli init:skill            # scaffold a new skill directory
hogli lint:skills           # validate frontmatter / syntax / binaries — fast, no Django
hogli build:skills          # render + package into dist/skills.zip
hogli sync:skill -- --name signals-scout-<scope>   # build + sync to .agents/skills/ for local agent testing
hogli unsync:skill -- --name signals-scout-<scope>
```

Authoring a new canonical scout is just creating `signals-scout-<scope>/SKILL.md` and merging — the next tick discovers it, seeds it onto enrolled teams, and auto-registers an enabled config on the default every-24-hours schedule.
**If you change the fleet shape (add/rename a scout, change the SKILL.md schema), update `products/signals/skills/AGENTS.md`.** On master, CI builds and publishes `dist/skills.zip` to the downstream distribution repos (the `ai-plugin` bundle and the standalone skills repo) automatically.

## Testing

**Dogfood the scout yourself first — before spending any real run.** The authoring agent has the same PostHog MCP tools a scout uses at runtime (`execute-sql`, `read-data-schema`, the per-product list tools, `scout-project-profile-get`), so the cheapest iteration is to walk the scout's own logic against the live project by hand: confirm the watched entity exists and has the assumed shape, run the **discriminator** to check it separates signal from noise on this project's data, and run each **explore pattern**'s queries.
Free and instant — refine the body, re-run the queries, repeat, until the logic holds on real data.

Only once you're happy do you spend a real run.
`posthog:scout-run-now {"id": <config_id>}` dispatches one run of the scout immediately, regardless of its schedule (get the `id` from `-config-list`) — the **initial real run**, the scout executing end-to-end in the harness.
An optional `note` steers that run alone (read next to the durable notes and never delivered to a later run as a note, though it stays visible in that run's metadata; needs `llm_skill:write` and skill-editor access, like a durable note), so you can aim the first run at the case you dogfooded.
The run is **asynchronous**: the call returns a workflow id right away; poll `-runs-list` (pass `skill_name` to scope to this scout) / `-runs-retrieve` for the result.
A disabled scout can still be run this way (test before enabling), and a manual run doesn't touch the schedule or `last_run_at`.
It inherits the scheduled path's guards (403 not enabled, 429 over quota / daily run budget, 409 a run already in progress) and draws from the **same daily run budget** as scheduled runs — a dry-run (`emit=false`) counts too.
There's no free test run, and it's slow (async, one run per call): firing the same scout repeatedly in a short window burns the project's daily allowance (and can starve its scheduled scouts).
**Don't iterate via `-run-now`** — dogfood the queries by hand to get the body right, and reserve `-run-now` for the initial real run and the odd re-check after a genuinely meaningful change.
The loop is **dogfood → run once ready → inspect**:

1. Dogfood the discriminator + explore patterns yourself against the live project (above), refining the body until the logic holds — the cheap, iterable part.
2. Create the scout and its config together via `posthog:scout-create` (the default `emit=true` goes in the nested `config`), leaving `run_interval_minutes` at a sustainable value — no short-interval trick needed.
   Then spend one `-run-now` to watch the whole scout execute end-to-end, and inspect once it finishes:
   - `posthog:inbox-reports-list` — the reports it actually wrote.
   - `posthog:scout-runs-list` — run summaries.
   - `posthog:scout-runs-retrieve` — the full reasoning for one run.
   - `posthog:scout-scratchpad-search` — the durable memory it wrote.
3. If it needs work, go back to dogfooding the queries by hand for the iteration, re-edit via `skill-update`, and spend another `-run-now` only once you've batched a meaningful change.

**Extra-careful variant — dry-run first.** For a scout you expect to be chatty, expensive, or high-stakes, set `emit=false` so it runs and logs what it _would_ have written (visible in `-runs-list` / `-runs-retrieve`) without writing to the inbox.
Trigger it with `-run-now`, inspect, refine, then `scout-config-update` to `emit=true`.
For most scouts, writing straight away and watching the inbox is the faster calibration.

Repo contributors additionally get `hogli sync:skill` to run the scout against the local harness for a tighter loop before merging.