// SPDX-License-Identifier: CC-BY-SA-4.0
= D199 handoff: automated post-squash stack recovery
:toc:
:toclevels: 3

Status: handoff only. Not implemented or deployed by the `.github` session.
Target repository: https://github.com/hyperpolymath/cicd-squabbler[`hyperpolymath/cicd-squabbler`].
Target service: `squabble-app` webhook leg, with a new stack-recovery move in
the squabble core/forge layers. Related owner rollout:
https://github.com/hyperpolymath/cicd-squabbler/issues/122[cicd-squabbler#122]
(D266/D267). Policy source:
https://github.com/hyperpolymath/.github/issues/13[.github#13] (D199) and
`metadatastician/standards/docs/SIGNING-POLICY.adoc`.

== Why this repository/service

D199 is estate-wide PR branch recovery, not a community-health file. The
owner has already ruled that squabble rolls out as a GitHub App installed on
all repositories of both accounts, hosted on owned compute, with no per-repo
workflow churn. `squabble-app` on `main` currently exposes only loopback
`/health`, `/api/v1/diagnose`, and `/api/v1/fight`; it has no webhook route or
App-auth leg. Implementing D199 here would put estate mutation logic in the
wrong source and would not satisfy the rollout decision.

Where `gitbot-fleet/robot-repo-automaton` is present and working, respect the
estate precedence directive and avoid duplicate writers. The squabbler App
must either dispatch to the primary or hold a per-repo lease before mutating
branches. It must remain standalone when the fleet is absent.

== Non-negotiable policy constraints

* No unsigned bot pushes.
* No force-pushes, including `git push --force-with-lease` to a PR branch.
* No protection bypass, admin merge, required-context removal, or check rename.
* No silent conflict skip (`git rebase --skip` is not an autonomous remedy).
* Every new commit must be GitHub-verified, preferably web-flow GPG through
  `createCommitOnBranch`.
* A recovered stack is not "green" until the downstream PR's own required
  checks run and pass on the new head.

Because a normal rebase updates a branch by non-fast-forward, D199 must be
implemented as *replacement-branch recovery*, not in-place force-push.

== Event and detection

Subscribe to `pull_request` `closed`. Act only when all of the following are
established from fresh API reads, not from the webhook payload alone:

. `merged == true`;
. the base is the repository default branch or a known stack base;
. the merge is positively classified as a squash merge;
. the merged head branch has open downstream PRs targeting it.

Squash classification must be fail-closed. GitHub does not expose a stable
post-merge `mergeMethod` field on `PullRequest`, and `merge_commit_sha` alone
does not identify the method. The implementation must record the exact
checks it uses. The minimum evidence set is:

* repository merge settings at event time permit squash and forbid merge
  commits and rebase merges (`allow_squash_merge == true`,
  `allow_merge_commit == false`, `allow_rebase_merge == false`);
  These settings must come from persisted event-time webhook or audit
  evidence. If that evidence is unavailable, classify the event as
  `ambiguous_merge_method` and take no mutating action.
* the PR is merged and has a non-null `merge_commit_sha`;
* the merge commit is reachable from the base branch and has exactly one
  parent, ruling out an ordinary two-parent merge commit;
* the PR timeline contains the corresponding `merged` event and no
  merge-queue entry;
* the commit message/tree metadata is consistent with GitHub's squash commit
  for that PR, not merely a title containing `(#N)`.

If any evidence is missing, unreadable, or conflicting, record
`ambiguous_merge_method` and take no mutating action. Merge commits, rebase
merges, merge-queue merges, and ambiguous events are negative cases, not
silent skips.

== Idempotency and resume state

Record a stack event key: repository id + merged PR number + merge commit SHA.
Also record a separate recovery row for each downstream PR:

`repository id + downstream PR number + stack event key`.

Each row must persist a state machine, lease owner, lease expiry, attempt
count, last error, captured downstream head/base SHAs, replay tree hash,
replacement branch name, replacement PR number/URL, and original-PR closure
status. Suggested states:

* `observed`
* `replayed`
* `branch_created`
* `pr_created`
* `original_closed`
* `completed`
* `escalated`
* `failed_retryable`

Persist the row before mutation. Webhook redelivery must resume non-terminal
rows rather than suppress the only retry. A stack-level key alone is not
sufficient because siblings can partially succeed. If a replacement PR
already exists for the same row and tree hash, report it instead of creating
another.

== Recovery algorithm

For each downstream PR in topological order (base PR before children):

. Fetch the downstream head, the merged base, and the old stack base into an
  isolated temporary repository. Never operate on a user's working tree.
. Compute the post-squash replay: `git rebase --onto <new-base> <old-base>
  <downstream-head>` in the temporary repository, or an equivalent three-way
  cherry-pick sequence. Capture the exact commit range and tree hashes.
. If any conflict occurs, stop that branch. Do not push a partial result.
  Post a structured report naming the PR, conflicting paths, old/new base
  SHAs, and the manual command a human can run. Label it `stack:conflict` or
  the repo's equivalent. Continue independent sibling stacks only if their
  bases are unaffected.
. Immediately before creating the replacement branch, re-fetch the original
  downstream PR and its base. If the PR is no longer open, or its head/base
  repository, ref, or SHA differs from the captured values, discard the stale
  replay and recompute or escalate. Compare ref identities as well as SHAs:
  a retarget to another base branch at the same SHA is still a different PR
  relationship. `expectedHeadOid` protects the new recovery branch; it does
  not protect the original downstream head.
. If clean and still current, create a new branch in the same repository, for
  example `stack-recovery/pr-<number>-<merge-sha-prefix>`, from the new base.
. Recreate the downstream commits on that branch through GitHub's signed
  commit API (`createCommitOnBranch`). This API authors commits as the
  credential owner; it cannot preserve the original primary author. Preserve
  original author/committer metadata in the recovery report and add
  `Co-authored-by` trailers where appropriate, but do not claim primary
  authorship is preserved. If exact primary authorship is a hard requirement,
  escalate or use a separately approved verified signing path with explicit
  author metadata; never push unsigned local commits.
. Verify each returned commit before proceeding. In GraphQL, require
  `signature.isValid == true` and `signature.wasSignedByGitHub == true`;
  a missing signature is a failure. As a cross-check, REST
  `commit.verification.verified` must also be true. If the API cannot preserve
  required file modes/binary changes, escalate rather than pushing unsigned
  local commits.
. Compare the final replacement-branch tree with the captured replay tree
  before opening the replacement PR. Compare every path's type, mode, and
  object ID. `createCommitOnBranch` accepts file contents and paths but not
  file-mode metadata, so a mode/type/object mismatch is a hard stop. If any
  entry differs, do not open the replacement PR or close the original PR;
  report the mismatch and escalate.
. Open a replacement PR against the correct new base. Link the original PR,
  merge commit, recovery run, and tree-equivalence evidence. Copy relevant
  labels/review requests only where the API permits and the semantics are
  safe. Do not copy approvals as if they apply to the new head.
. Immediately before closing the original downstream PR, re-fetch it again.
  Close it as superseded only if it is still open, its head/base repository,
  ref, and SHA still match the recovered values, the tree-equivalence check
  passed, and the replacement PR exists with the new head visible. If closing
  is not permitted, the PR changed, or the trees differ, leave a comment and
  report the partial state.
. Re-target or recover nested children against the replacement branch/PR in
  the same topological pass. A child must never be replayed onto a base that
  has not itself been recovered or merged.

== Forks

* Never push to a fork without explicit permission and a verified signing
  path. Default: create an upstream replacement branch from the fork head if
  the App has read access and the repository permits it, then open an
  upstream replacement PR crediting the fork author.
* If the fork head cannot be read or upstream replacement is not permitted,
  do not silently skip. Post a recovery report to the original PR with the
  exact conflict-free patch/branch instructions and mark it needing human
  action.
* Private forks and deleted source branches are explicit escalation cases.

== Concurrency and rate limits

* Queue work per repository and per stack root. Use a lease or equivalent
  compare-and-swap on the per-downstream-PR recovery row.
* Use `expectedHeadOid` on every commit API call. On mismatch, re-fetch and
  recompute; never overwrite a moving branch.
* Debounce by downstream head SHA. One installation token can cover hundreds
  of repositories; pace requests and treat secondary-limit 403 as retryable
  infrastructure failure, not a verdict.
* Make redelivery safe: if a replacement PR already exists for the same
  recovery row and tree hash, report it instead of creating another.
* Crash recovery must resume `replayed`, `branch_created`, and `pr_created`
  rows. A row is terminal only when `completed` or explicitly `escalated`.

== Required tests and evidence

Unit/fixture tests:

* squash merge with one downstream PR, clean replay;
* merge commit, rebase merge, and merge-queue merge: no replacement PR;
* ambiguous or conflicting merge evidence: no mutation, escalation recorded;
* nested stack A -> B -> C, topological recovery;
* sibling stacks, independent failure isolation;
* conflict in a middle PR: no push, no silent skip, precise report;
* duplicate webhook delivery: one replacement PR;
* crash after persistence but before replacement PR creation: resume succeeds;
* concurrent downstream head update: stale replay discarded and recomputed;
* concurrent original-PR closure/merge: no stale closure;
* `expectedHeadOid` refusal on the recovery branch: recompute;
* fork PR: no unauthorized fork push;
* deleted fork/base branch: explicit escalation;
* binary file, executable bit, rename, and delete preservation;
* unsigned API response, missing GraphQL signature, or failed REST verification:
  stop before closing old PR;
* rate-limit/secondary-limit response: retry/escalate, never green.

Integration tests:

* Mock GitHub GraphQL/REST with recorded fixtures for `createCommitOnBranch`,
  PR creation, PR close, labels, and check-rollup reads.
* A disposable canary repository with a real squash merge and a two-level
  stack. Evidence must include old/new SHAs, replacement PR URLs, verified
  commit badges/API verification, required-check runs on the new heads, and
  the final merge state.
* A paired conflict canary proving the service refuses to push and reports
  the conflicting paths.

Deployment evidence:

* App registration/installation ids and permission set, without secrets.
* Webhook signature verification: bad signature -> 401.
* Ingress/TLS and loopback origin configuration on owned compute.
* Health/readiness and queue/lease observability.
* A redelivered production webhook that is idempotent.
* Explicit statement that the squabble advisory check is not required.

== Acceptance

D199 is complete only when a squash merge automatically produces verified,
conflict-free replacement downstream PRs (or explicit conflict/fork
escalations), nested stacks recover in order, duplicate/concurrent deliveries
are safe, and the downstream required checks pass on the recovered heads.
Closing the tracking issue or seeing the App online is not sufficient.