---
name: verify-vuln-issues
description: Triage the repo's open "[Vulnerability]" GitHub issues — for every issue lacking a true-positive/false-positive label, fetch the referenced commit, judge whether it is a genuine security fix or a false positive, post an analysis comment, and apply the verdict label. Pure dependency bumps get a `dependency` label and are closed. Use when asked to "verify", "triage", or "review" the vulnerability issues / findings in the tracker.
---

# Verify vulnerability issues

This repo's monitoring action files a GitHub issue for every commit it thinks is a
security patch. Many are genuine; some are false positives (routine dependency
bumps, feature commits, build-tooling changes, AI-misclassified commits). This
skill verifies the unverified ones, comments, and labels each.

## Scope & disposition

- **Target:** **open** issues authored by `github-actions[bot]` that carry **none**
  of the verdict labels — `true-positive`, `false-positive`, or `dependency`.
  An issue carrying any of those, or one already **closed** (e.g. a dependency
  bump closed in an earlier run, or a duplicate), has already been dispositioned
  — skip it. `prepare.mjs` applies exactly this filter.
- For every targeted issue: post one analysis comment, then apply the disposition:

  | Finding | Action |
  |---|---|
  | Genuine security fix | comment + add `true-positive` label |
  | False positive | comment + add `false-positive` label |
  | **Pure dependency bump** | comment + add `dependency` label + **close** (`not planned`) |

- **Pure dependency bump** = the commit's entire diff is package manifests,
  lockfiles, or version changelogs — no hand-written fix code. Close it regardless
  of whether the upstream advisory is real; dependency bumps are out of scope for
  this tracker.
  - **Exception — keep it open:** if the commit *also* carries actual
    vulnerability-fix code (e.g. it backports/cherry-picks specific fix code into
    vendored sources, not merely a version number/lockfile entry), it is a real
    finding — treat it as Verified (`true-positive`), do **not** close it.
- **Duplicate issues:** the monitor sometimes files more than one issue for the
  *same commit*. `prepare.mjs` detects these (writes `/tmp/vsa/duplicates.json`).
  Keep and verify the canonical one (lowest issue number = first filed); the
  rest are duplicates — close them as duplicates, or delete with `gh issue
  delete` **only after the user confirms**, since deletion is permanent.
- Labels: **add** the verdict label. Never remove a label someone *else* applied
  — if an issue carries a conflicting one, stop and surface it. You *may* correct
  a verdict this skill applied earlier (e.g. `true-positive` → `false-positive`)
  when re-analysis warrants: remove the stale label, add the right one, and post
  a correction comment explaining why.
- Relay a final summary (counts per repo, false positives, issues closed as
  dependency bumps, duplicates found).

## Prerequisites

`gh` must be authenticated. `gh auth status` reporting "not logged in" inside the
sandbox is normal for git operations, **but `gh api`/`gh issue` calls still need a
real token.** If `gh` calls fail with `HTTP 401`/`Bad credentials`, ask the user
to run `! gh auth login` in the prompt (the dummy-`GH_TOKEN` trick does not work —
the proxy rejects it).

Ensure the `dependency` label exists (create once if missing):

```bash
gh label create dependency --repo <owner/repo> \
  --description "Issue is only a dependency/version bump — out of scope, ignored" \
  --color ededed
```

## Procedure

### 1. Fetch & prepare

```bash
node .claude/skills/verify-vuln-issues/prepare.mjs
```

It lists issues, filters to unverified ones, parses each body for its repo + full
commit SHA (via `scripts/lib/parser.mjs`), downloads every commit diff and
metadata, and writes `/tmp/vsa/`:
- `todo.json` — `[{number, state, repository, commitSha, ...}]`
- `ctx/<n>.txt` — per-issue: the AI claim + commit message + **file list** + diff
  (lockfiles trimmed, huge files truncated)
- `duplicates.json` — groups of issues sharing one commit SHA (see "Duplicate
  issues" above)
- `results.json` — starts `{}`; record verdicts here as you go.

The `--- FILES CHANGED ---` list in each context file is what you use to spot a
pure dependency bump (only `package.json` / `*.lock` / `go.mod` / `go.sum` /
`CHANGELOG*` / `doc/changelogs/*` / toolchain files).

### 2. Analyze, in batches

Read `ctx/<n>.txt` files (~10–15 per turn; read large ones individually). For each
issue decide the finding using the rubric below, and record into `results.json`
(`verified` / `false-positive` / `dependency`) so progress survives compaction.

### 3. Comment + label (+ close)

Write each comment to `/tmp/vsa/comments/<n>.md`, then per issue:

```bash
gh issue comment <n> --repo <owner/repo> --body-file /tmp/vsa/comments/<n>.md
# verified:
gh issue edit <n> --repo <owner/repo> --add-label true-positive
# false positive:
gh issue edit <n> --repo <owner/repo> --add-label false-positive
# pure dependency bump:
gh issue edit <n> --repo <owner/repo> --add-label dependency
gh issue close <n> --repo <owner/repo> --reason "not planned"
```

Loop over a batch; report any failures.

## Verdict rubric

The commit diff is ground truth — the issue body is an LLM guess and is often
wrong about the mechanism, severity, or even the vuln class.

### Verified — a genuine security fix (`true-positive`)
- Commit message cites a **CVE-ID**, GHSA advisory, HackerOne report, an internal
  vuln tracker (`GL-Vuln:`), or a `[security]` tag.
- Hand-written code change that removes a real, **demonstrably exploitable** flaw
  (memory safety, auth/authz check added, injection/CRLF/XSS sink fixed, DoS
  guard), ideally with a regression test. A guard that closes a *demonstrated*
  bypass still counts even when low-severity — but a change that only addresses a
  theoretical concern does not; see "Preventive hardening" under False positive.
- A vendored-dependency update that **backports/cherry-picks specific fix code**
  for a vulnerability (not just a version number) — even though it touches
  `deps/`, it carries the actual patch, so keep it open.
- Still verify even if the issue mislabeled the class — say so and correct it
  (e.g. "this is request smuggling, not SSRF"). Note overstated severity.

### Pure dependency bump (`dependency`, then close)
- The whole diff is package manifests / lockfiles / version changelogs — no fix
  code in the project itself. The fix, if any, lives entirely upstream.
- Applies whether the issue is otherwise "verified" (real upstream advisory) or a
  false positive — either way it is dependency noise, label `dependency` + close.
- Includes vendored-dependency *version* bumps (re-vendoring a bundled dep to a
  new release). But see the Verified exception: a vendored *security backport*
  that patches specific code stays open.

### False positive — not a substantiated vulnerability (`false-positive`)
- **Build/dev tooling**: indirect or dev dependency edits under `tools/`, `hack/`,
  `devDependencies`, or a maintainer-only script — no runtime/attacker surface.
- **Feature, not a fix**: the "vulnerable" code and its validation/guard are
  introduced *in the same commit* (no prior released version was vulnerable); or
  the commit just wires up / enables an in-development feature.
- **Mischaracterized**: the issue's described mechanism contradicts the diff
  (e.g. claims "no cert validation" but pre-patch used standard TLS; claims
  prototype pollution but `__proto__` was already handled).
- **Correctness bug with no attacker**: internal concurrency/overflow/data-
  integrity bugs (build engines, ORM dedup, migration races) with no security
  boundary or attacker-controlled input.
- **Preventive / defense-in-depth hardening**: the change hardens against a
  *theoretical* or *downstream* weakness with no demonstrated exploit in a
  released version — e.g. giving an object a null prototype to address
  prototype-pollution *concerns*, or a guard the commit frames as precautionary.
  Tell-tale: the commit message hedges ("harden", "defense-in-depth",
  "concerns") and cites no CVE / advisory / PoC showing the prior version was
  actually exploitable. (Contrast: a guard that closes a *demonstrated* bypass is
  Verified, even if low-severity.)
- **Docs-only** change — unless it reflects a real default-hardening shipped
  elsewhere.

When unsure, state the caveat explicitly (dependency bump → flaw is upstream;
dev-only reach; severity overstated) rather than forcing a binary call.

## Comment format

Start with a bold verdict line, reference the short commit SHA, reason from the
diff, then a one-line `Verdict:`. Keep it substantive but tight:

```markdown
## Verification analysis — **Verified** (genuine <class> fix)

Reviewed commit `<sha>`. <what the diff actually does and why it is/ isn't a
real fix; cite advisory refs, tests, caveats>.

Verdict: verified — <one line>.
```

If an issue already has a CVE in its commit message but no `cve:` label, mention
it (it would graduate the finding from "verified" to "confirmed").