--- name: pre-publish-self-check description: "Use when a skill author has finished writing or editing a skill and wants to check it before pushing to GitHub, opening a PR, or sharing it. WHEN: before publishing a skill, before pushing to GitHub, self-review of a skill I wrote, checking my own skill directory. DO NOT USE FOR: scanning someone else's skill before installing (use vet-before-install), or an environment sweep (use audit-my-agent-environment)." license: MIT metadata: author: Agentic Highway version: "0.2.0" requires-vettd: ">=0.10.0" --- # Self-Check a Skill Before Publishing ## Overview Scan your own skill directory before you push, open a PR, or share it, then fix and rescan until `overallGrade` reaches your target. Treat vettd as a linter you run on your own work, not a gate someone else runs on you. ## When to Use | Situation | Action | |---|---| | You wrote or edited a skill directory and want to publish it | Use this skill | | You are about to open a PR that adds or changes a skill | Use this skill | | You want to double-check a skill before sharing it with a teammate | Use this skill | | You are about to install a skill someone else wrote | Use `vet-before-install` instead | | You want a sweep of every skill already installed across an environment | Use `audit-my-agent-environment` instead | | A specific finding's meaning or severity is unclear | Finish this skill, then invoke `triage-a-flagged-finding` | ## Command Contract As of vettd 0.9.0: - Use `--stdout` for JSON from `scan`. The `--json` flag is accepted but ignored on scan subcommands and emits human-readable text. - Do not branch on exit code. Scans exit 0 regardless of severity. - Never parse human-readable output. ANSI escapes are emitted even when stdout is not a TTY. Scan the skill directory you are authoring: ```bash vettd scan folder ./skills/my-skill --stdout --deep ``` `--deep` has no depth limit and only exists on `scan folder`. Use it here — reference files nested a few levels deep (scripts/, references/, examples/) are exactly what a real reviewer or installer will read, so scan them too. A scan of a small skill directory takes roughly 100-300ms, so rescanning after every fix is cheap. Treat the loop as free. ## Workflow 1. Scan: `vettd scan folder --stdout --deep`. 2. Parse the JSON. Find your skill's entry in `skills[]` by matching the path portion of `id` (`:<12-char-content-hash>`) against your directory — do not match on `name` alone, since multiple scans can produce entries with the same name. 3. Read `overallGrade` and `trustLevel` for that entry. 4. Read every entry in `externalScannerResults[].findings[]`. Group by `category`: `security`, `structure`, `description`, `best-practices`, `scripts`, `evals`. `overallGrade` counts every category — only `info` severity is excluded. 5. Order fixes by `severity` first (`critical` > `high` > `medium` > `low` > `info`): a `critical`/`high` in `description` or `scripts` drops the grade exactly as one in `security` or `structure` does. Within a severity, fix `security` and `structure` first — security is the most safety-relevant, and structure fixes are usually mechanical. 6. Rescan with the same command. Confirm each fixed `ruleId` no longer appears, and confirm no new finding was introduced by the fix. 7. Repeat steps 5-6 until no `critical`/`high` findings remain anywhere and `overallGrade` reaches the level you're targeting (A or B, `Trusted` or `Conditional`). 8. A `medium`/`low` finding in any category still counts toward the grade. Fix them when the fix is cheap; if you accept one, know it stays in the count. 9. If a finding's cause or severity doesn't make sense for your skill: ⛔ **MANDATORY HAND-OFF** — invoke **triage-a-flagged-finding**. 10. Stop when `overallGrade` is at your target and you've made a deliberate call (fix or accept) on every remaining finding. ## Grade Thresholds `overallGrade` is computed from **all** findings across every category (`structure`, `security`, `best-practices`, `description`, `scripts`, `evals`) — only `info` severity is excluded. Thresholds are evaluated F to A, first match wins: | Grade | Threshold | | --- | --- | | F | any `critical`, or 3+ `high` | | C | any `high`, or 3+ `medium` | | B | any `medium`, or 4+ `low` | | A | otherwise, including zero findings | Use this to know your exact headroom. Two mediums is still a `B`; a third medium drops you to `C`. One high is `C` regardless of how clean everything else is; three highs is `F` even with zero criticals. A single `critical` finding is always `F` — critical means adversarial intent, or a pattern that fires unconditionally with no exploitability preconditions. There is no threshold to stay under; fix it or the grade cannot move. Reaching grade `A` means no `medium`/`high`/`critical` findings in any category and fewer than four `low`s — it is not a guarantee of safety in every environment, and a scan that returns very few findings overall is a weaker signal than one that returns many `info`-level findings showing real coverage. Signals are a separate store: scan output may also include non-finding `signals[]` and `coverage[]` arrays on `externalScannerResults[]`. They span seven categories (safety, reliability, performance, cost, compatibility, Popularity, characteristics) with three verdict forms (`graded`, `measured`, `unjudged`), but they are evidence/context — never findings, and never part of `overallGrade`. Do not try to "fix" a signal row; only `findings[]` rows count toward the grade. ## Fixing Common Findings | Cause | Example ruleId | Category | Fix | |---|---|---|---| | External URL referenced in SKILL.md | VTD-0088 | security | Inline the needed content instead of linking out, or pin to a specific commit/version so referenced content can't change after audit | | Missing SKILL.md | VTD-0095 (absence) | structure | Add the required SKILL.md with correct frontmatter | | Cloud instance metadata endpoint probed | VTD-0029 | security | Remove the probe entirely — a skill has no legitimate reason to read cloud metadata endpoints; this is a known credential-theft vector and fires as `critical` | | Shell + network + filesystem access declared together | dangerous-keyword-combo rules | security | Split into narrower steps, drop any tool declaration you don't actually invoke, or document in the skill why the combination is required | | Base64 decode-and-use patterns | encoding/obfuscation rules | security | Avoid decode-then-execute flows; if decoding is legitimate, keep the decoded content as inert data, never pipe it to a shell or interpreter | | Remote content piped straight to a shell (`curl \| sh`) | remote-exec rules | security | Replace with a pinned, checksum-verified install step; never pipe unreviewed remote output directly into execution | | Credential-shaped strings, or scripts reading known credential paths (cloud provider creds, SSH private keys, container registry configs) | secret-pattern rules | security | Remove real-looking keys/tokens from examples; never read credential file paths from a skill script; read real credentials from environment variables instead | | Skill name suspiciously close to a known popular skill | typosquatting rules | security | Rename to something clearly distinct — name-proximity to a popular skill is a recognized supply-chain attack pattern, not a style nitpick | | A chain of otherwise-ordinary indicators (credential access, then encoding, then outbound transmission; or remote fetch piped straight to a shell) | chained-signal rules | security | Vettd weighs sequences, not just individual indicators — the difference between careless code and an intentional attack is often visible in the chain. Break the chain: don't decode-then-transmit, don't fetch-then-execute, in a single flow. | | Overly broad tool/permission declarations | broad-permission rules | structure/security | Declare the narrowest explicit tool list your skill actually uses instead of a wildcard or "all tools" | | Prose describing what your skill does *not* do (e.g. "no network access") | keyword-detection false positive | — | Vettd's keyword matching does not understand negation — mentioning "network access" or "shell execution" in prose, even to disclaim it, can add that permission to your skill's declared surface. Omit the negated mention rather than stating it. | | A `signals[]` row you remember as a finding (e.g. repository link, was `VTD-0083`) | — | signal | `VTD-0083` and `VTD-0102`–`VTD-0123` were reclassified from findings to the signal channel; `VTD-0101` was removed. Signals never affect `overallGrade` — don't chase them like findings. Only `externalScannerResults[].findings[]` rows (six categories) count. | ## Common Mistakes | Mistake | Why it's wrong | |---|---| | Treating a `description`/`best-practices`/`scripts`/`evals` finding as grade-neutral | Every category feeds `overallGrade` (only `info` is excluded), and `trustLevel` follows the grade — a quality finding can be the reason you're not at `A` | | Treating a `signals[]` row as a finding to fix | Signals are evidence/context, never findings — they don't affect `overallGrade` and aren't fixed like findings | | Assuming a fix worked without rescanning | The only proof a finding is resolved is its absence from the next `--stdout --deep` scan | | Reading the human-readable terminal output instead of `--stdout` JSON | ANSI escapes are emitted regardless of TTY state and will corrupt any parsing | | Branching logic on the scan's exit code | Scans always exit 0; severity lives only in the JSON, never in the exit status | | Scanning without `--deep` | Findings in nested `scripts/`/`references/` files won't be included, giving a false sense of a clean skill | | Matching your skill by `name` instead of the path portion of `id` | Multiple scan runs or similarly-named skills can collide on `name` alone | | Treating `Conditional` trustLevel as always a failure | Some skills legitimately need elevated permissions; `Conditional` can be the correct, expected outcome — check the underlying findings, not just the label | | Disclaiming capabilities in prose ("this skill does not use the network") | Keyword matching has no concept of negation and may still flag the mentioned capability |