name: CI # Guardrails as automated gates (CLAUDE.md "The guardrails as automated gates"): # - release job: configure (portable default, no RIPWIRE_NATIVE — CI hardware is not "this dev machine") # + build + the full gate suite (170+ gates incl. determinism/G4/quality) on SIX legs: # build-flavour (Release / plain) x front end (macOS AppleClang, Linux gcc, Linux # clang) — see the job comment for why each axis exists — run in parallel via # test/pargates.py -j 3 (same authoritative list as regression.sh), each leg's suite # SHARDED across 4 runner jobs on every leg (--shard K/N, cost-balanced by # .github/pargates-shard-weights.json) so the workflow's wall clock is one shard, not one suite. # ubuntu-24.04 here is also the FIRST real proof of L1's portability fix: real x86-64 # Linux hardware, not the local RIPWIRE_PRETEND_LINUX proxy test/portablebuildcheck.sh # uses on the (Apple Silicon) dev machines this repo is normally built on. # - rhel job: the platform this project ships a Linux prebuilt for, checked at PR time instead of only # at release. RHEL 9 userland (ubi9) via `container:` on a hosted Ubuntu runner, built with # Red Hat's own gcc-toolset (RHEL 9's default gcc 11 cannot do C++23). Build + fixture # self-run + the two headline guarantees, NOT the full suite — see the job comment for why. # - asan job: configure -DRIPWIRE_ASAN=ON (G1: address,undefined,integer,float-divide-by-zero, # float-cast-overflow, fail-fast) + build + the fixture self-run + the 5 heaviest verbs # (--pack-task/--from-trace/--quality-delta/--merge-scout/--edit-check) # via their own gate scripts pointed at the ASan binary. NOT the full regression.sh under # ASan — that would run 170+ mostly-non-heavy gates at sanitizer speed for little extra # signal; the release job already proves those gates pass on a normal binary per OS. # - both jobs end with an explicit 2-run byte-diff det-gate + xmllint --noout, even though # test/regression.sh already covers both, so a CI log reader sees the two headline product # guarantees (determinism, well-formed XML / G4) called out as their own named steps. # # There is no dependency-download step and no dependency cache: tree-sitter core, all 15 grammars and # doctest are VENDORED under third_party/deps (see CMakeLists.txt), so `actions/checkout` is the whole # provisioning story. That also means CI exercises the same hermetic build a stranger's clone gets — # a cache step would have hidden a re-introduced network fetch behind a warm runner. # # LIGHT SET vs. FULL MATRIX (owner decision, 2026-09-17: CI was the bottleneck — a merge to main re-ran all # 33 jobs on a tree its PR had already tested, and per-lane full CI on the landing queue found almost # nothing). The `plan` job below computes ONE `full` output from the event name, the pull request's labels # and the ref; every heavy job reads that output through its own `if:` instead of repeating the same # event/label logic. # LIGHT (style + the one ubuntu-24.04/Release/clang release leg, all 4 shards — that leg already runs the # det-gate and G4 xmllint checks): a push to main, or a pull_request carrying the `train-member` label. # Labelling is maintainer-only by construction — a fork PR cannot add a label to itself. # FULL (all 35 jobs): every other pull_request (integration/* train PRs, direct-land PRs, contributor # PRs), workflow_dispatch — the owner requires a full build and test on the EXACT commit before any # release tag, so dispatch this manually against the commit you are about to tag rather than trusting an # earlier green nightly — and a nightly `schedule` against main. See the report-failure/report-green jobs # at the bottom for where a red scheduled run is reported: its OWN tracking issue (label # nightly-full-matrix, alongside the shared nightly-failure), separate from nightly.yml's TSan issue. on: push: branches: [main] pull_request: # The GitHub default (opened/synchronize/reopened) plus labeled/unlabeled, so adding or removing # `train-member` re-evaluates the plan job's decision on the SAME pull request without a new push. types: [opened, synchronize, reopened, labeled, unlabeled] workflow_dispatch: schedule: # Full-matrix proof against main, independent of nightly.yml's own 07:17 UTC ThreadSanitizer run — off # :00 and a different minute from that cron so the two scheduled workflows don't start together. - cron: "41 5 * * *" # 05:41 UTC daily permissions: contents: read # least privilege at the top; only report-failure/report-green below widen, and only for themselves concurrency: group: ci-${{ github.workflow }}-${{ github.ref }} # Cancel a superseded run on a PULL REQUEST only. On main every push is a distinct commit that deserves its # own verdict: on 2026-09-07 three consecutive main runs were cancelled by unrelated pushes inside the # ~26-min window, with zero failures among the 45 jobs that had completed — the signal was thrown away, not # superseded. A queued main run costs runner minutes; a cancelled one costs the answer. cancel-in-progress: ${{ github.event_name == 'pull_request' }} # Never RIPWIRE_NATIVE in CI: -march=native would bake in whatever ISA the CI runner's host happens to # expose that week, defeating the entire point of testing the portable default. jobs: # ─── plan: light set or full matrix — computed ONCE, so every heavy job reads one output via `if:` ─────── # # No extra permission needed: a pull_request event's own webhook payload already lists its labels # (github.event.pull_request.labels), so this reads the trigger event instead of calling the API. plan: name: plan (light set or full matrix?) runs-on: ubuntu-24.04 outputs: full: ${{ steps.decide.outputs.full }} release_matrix: ${{ steps.decide.outputs.release_matrix }} steps: # release_matrix is computed here, not job-level `if:` on `release`, because the matrix context is # not available to a job-level `if:` (only to steps already inside an expanded matrix job — actionlint # catches this if it regresses: "context matrix is not allowed here"). Generating it also means there # is exactly ONE source of truth for the full 24-row matrix instead of a second hand-typed copy that # can drift from it: six legs {macos-26 AppleClang, ubuntu gcc, ubuntu clang} x {Release, plain}, each # sharded 4 ways — the same axes `release`'s own job comment documents. LIGHT keeps only the one leg # the light set always runs, ubuntu-24.04/Release/clang, all 4 shards (it already carries the # det-gate and G4 xmllint checks). - name: Decide, from event name, PR labels and ref — see the header comment for the rule id: decide env: EVENT: ${{ github.event_name }} REF: ${{ github.ref }} PR_LABELS: ${{ toJSON(github.event.pull_request.labels.*.name) }} run: | full=true case "$EVENT" in push) # on.push.branches is already scoped to [main]; the ref check is belt-and-suspenders so this # job's own decision does not silently depend on that trigger filter never changing. if [ "$REF" = refs/heads/main ]; then full=false fi ;; pull_request) if jq -e 'index("train-member") != null' <<<"$PR_LABELS" >/dev/null; then full=false fi ;; workflow_dispatch|schedule) full=true ;; esac echo "event=$EVENT ref=$REF labels=$PR_LABELS -> full=$full" echo "full=$full" >> "$GITHUB_OUTPUT" if [ "$full" = true ]; then legs='[ {"os":"macos-26","flavor":"Release","cc":"appleclang"}, {"os":"macos-26","flavor":"plain","cc":"appleclang","skiplist":".github/pargates-macos-plain-skip.txt"}, {"os":"ubuntu-24.04","flavor":"Release","cc":"gcc"}, {"os":"ubuntu-24.04","flavor":"plain","cc":"gcc"}, {"os":"ubuntu-24.04","flavor":"Release","cc":"clang"}, {"os":"ubuntu-24.04","flavor":"plain","cc":"clang"} ]' else legs='[{"os":"ubuntu-24.04","flavor":"Release","cc":"clang"}]' fi release_matrix="$( jq -c '{ include: [ .[] as $leg | range(1;5) as $shard | ($leg + {shard:$shard, shards:4}) ] }' <<<"$legs" )" echo "release_matrix=$release_matrix" >> "$GITHUB_OUTPUT" # ─── style: one gating clang-format check, one advisory clang-format report, one advisory clang-tidy ─── # # CLANG_VERSION is a PIN, not a floor. clang-format's output changes across major releases, so an # unpinned checker reports "unformatted" on a tree that was formatted correctly by a different release. # .clang-format was authored and verified against clang-format 22 on the dev machine; scripts/formatcheck.sh # refuses to run under any other major, so bumping this number without re-running `clang-format -i` over the # gated file set fails loudly instead of silently re-styling the repo. # # The clang-format GATE covers only the files listed in scripts/formatcheck.sh, which are the ones that # already match .clang-format byte for byte. It cannot cover the tree: the house style in CONTRIBUTING.md §3 # is hand-formatted in ways clang-format has no option to preserve (multi-statement one-liners, # `for( … ) if( … ) return i;`, packed initialiser rows and `case` labels, hand-chosen 160-200 column wrap # seams), and reformatting all 98 first-party C++ files changes 11837 lines that survive `git diff -w` — # real joins and splits — across 89 of them. A whole-tree --Werror check would therefore be red on a # correctly-styled tree, which is a gate that punishes the documented style. The --advisory step below # prints that gap on every run so it stays visible rather than forgotten. # # clang-tidy is ADVISORY and must stay that way: continue-on-error, and .clang-tidy's WarningsAsErrors is # empty. Its default catalogue argues for a different C++ than the data-oriented one G2 mandates (POD/SoA, # C arrays, 32-bit handles, ASSUME instead of exceptions), so the config is curated down to bugprone-* / # clang-analyzer-* / performance-* / misc-dangling-* and even those only report. Baseline at the commit # that added it: 312 unique sites, of which 126 are bugprone-easily-swappable-parameters and 85 are # bugprone-exception-escape. Re-measured 2026-09-10 at d90acb2c, unique file:line:check: 358 while main.cpp's # TU failed to parse (see the configure step below), 396 once it parsed, 359 after triage — 204 of them # bugprone-easily-swappable-parameters and 30 bugprone-exception-escape. Every remaining class was read and # refused with evidence; the record is in the commit that wrote these numbers. Read it as a to-triage list, # not a queue of defects. style: name: style (clang-format gate + advisory clang-tidy) runs-on: ubuntu-24.04 env: CLANG_VERSION: "22" steps: # L4 (Linux probe): checkout@v4 defaults to a --depth 1 clone, which leaves ONE commit of history # in the tree. The churn / co-change / ownership gates (churnjoincheck, hotspots, --owners, the # quality short-horizon-churn kind) mine `git log` for real, so on a shallow checkout they do not # error — they measure zero and fail, or worse, pass while measuring nothing. Full history it is. - uses: actions/checkout@v4 with: fetch-depth: 0 # churn/co-change gates read real git history — a shallow clone reddens them persist-credentials: false # no step in this job pushes or fetches with the token - name: Install clang-format / clang-tidy (PINNED major — see the job comment) run: | # Third-party apt sources ship in the runner image and are outside our control. TWICE now # one has been mid-publish when a job ran and taken every Linux job on every branch down # with it: `apt-get update` exits 100 and the leg dies in setup, before a single gate runs, # with a failure that looks nothing like a package problem. google-chrome on 2026-09-09 # ("Hash Sum mismatch"; deb822 .sources on 24.04, not .list) and packages.microsoft.com on # 2026-09-10 ("403 Forbidden ... is no longer signed"). Naming vendors one at a time only # ever fixes the last outage, so: we install ubuntu-archive packages ONLY, and every # non-ubuntu source goes. -print names what went, so a future reader can tell whether a # missing package traces back to here. ls /etc/apt/sources.list.d/ || true sudo find /etc/apt/sources.list.d -type f \( -name '*.list' -o -name '*.sources' \) \ ! -name 'ubuntu*' -print -delete || true wget -qO /tmp/llvm.sh https://apt.llvm.org/llvm.sh chmod +x /tmp/llvm.sh sudo /tmp/llvm.sh "$CLANG_VERSION" all clang-format-"$CLANG_VERSION" --version clang-tidy-"$CLANG_VERSION" --version # GATING. Runs clang-format --dry-run --Werror over scripts/formatcheck.sh's file list. # Local equivalent, proven on the dev machine before this step was written: # CLANG_FORMAT=/opt/homebrew/opt/llvm/bin/clang-format scripts/formatcheck.sh - name: clang-format — gate (scripts/formatcheck.sh) run: CLANG_FORMAT=clang-format-"$CLANG_VERSION" bash scripts/formatcheck.sh # NON-GATING by construction: --advisory always exits 0. It names every first-party C++ file # clang-format would rewrite, so the size of the ungated remainder is on the record every run. - name: clang-format — advisory report over the whole first-party set (non-gating) run: CLANG_FORMAT=clang-format-"$CLANG_VERSION" bash scripts/formatcheck.sh --advisory # embedded_queries.h is written at CONFIGURE time (file(WRITE) in CMakeLists.txt). version.h is NOT: it is # the BYPRODUCT of the ripwire_version_stamp custom target, which only runs at build time. src/cli.h # includes it, so a configure-only database handed clang-tidy a fatal "'version.h' file not found" in # every TU and the report was taken over a broken parse. Building that one target runs a cmake -P script, # not the compiler, so there is still no compile needed to lint. - name: Configure a compile database for clang-tidy run: | cmake -S . -B build-tidy -DCMAKE_EXPORT_COMPILE_COMMANDS=ON cmake --build build-tidy --target ripwire_version_stamp # GATING, and deliberately narrow: only the silent-wrong-answer checks that sit at ZERO rows on these five TUs # (the list lives in scripts/tidycheck.sh, the counts in .clang-tidy's header). --warnings-as-errors='*' # makes any new row red. CLANG_TIDY is named, so a wrong or missing major is an error here, never a SKIP. # Local equivalent: CLANG_TIDY=/opt/homebrew/opt/llvm@22/bin/clang-tidy bash scripts/tidycheck.sh - name: clang-tidy — gate (scripts/tidycheck.sh, the zero-row subset) run: CLANG_TIDY=clang-tidy-"$CLANG_VERSION" bash scripts/tidycheck.sh build-tidy - name: clang-tidy — ADVISORY, never gates continue-on-error: true run: | clang-tidy-"$CLANG_VERSION" -p build-tidy --quiet \ src/main.cpp src/ingest.cpp src/pagerank.cpp src/tsprobe.cpp src/infra/diagnostics.cpp release: needs: plan # LIGHT set still runs this job — just ONE leg of it, ubuntu-24.04/Release/clang (all 4 shards): see # `plan`'s own step, which computes the matrix below to exactly that one leg when light and to all six # legs when full, instead of this job carrying an `if:` (job-level `if:` cannot read the matrix context # — only a step already inside an expanded matrix job can, so pruning legs needs the matrix ITSELF to # be conditional, not a condition alongside it). # Two independent flavours per OS, each on its OWN runner — they used to run stacked in one job, # doubling wall time for no coverage gain: # Release — defines NDEBUG, which compiles the DISCLOSE( msg ) trace out; the optimizer-visible build. # plain — NDEBUG off, so the DISCLOSE( msg ) trace compiles in and the degrade-path gates can observe # the alert they assert. The e7405e7 qsnap fix was invisible to CI before this flavour # existed. estchargecheck #14 / qualitystalecheck arms 7/8c probe the flavour (an # already-gated degrade path + --version's build-type token) and SKIP honestly on the # Release leg, pointing at the plain leg as where the alert is proven; a missing alert # on the plain flavour still FAILS. Neither flavour alone is enough. # # And BOTH FRONT ENDS on Linux, not just the platform default. gcc is the one this job used to have, # and it is a feature — a second front end over the same tree, which found three real portability bugs # on its first run. But it is not the whole story either: RIPWIRE_OPT_REMARKS and RIPWIRE_PGO are # Clang-only by construction (-Rpass=/-fsave-optimization-record, .profdata/llvm-profdata) and # CMakeLists FATAL_ERRORs on anything else, so on a gcc-only Linux leg optremarkscheck's configure arms # cannot be expressed and can only skip. macOS covers Clang but not Linux's libc, headers or linker. # Six legs = {macos-26 AppleClang, ubuntu gcc, ubuntu clang} x {Release, plain}: every guardrail is # observed by at least one leg on the OS it matters for, and no leg has to assert the impossible. name: release (${{ matrix.os }}, ${{ matrix.flavor }}, ${{ matrix.cc }}, shard ${{ matrix.shard }}/${{ matrix.shards }}) strategy: fail-fast: false # There is no gcc on the macOS runners (`g++` there is an AppleClang shim, so a `cc: gcc` macOS leg # would silently be a second AppleClang leg claiming to be gcc), and a leg that lies about its # toolchain is worse than a leg that is missing — `plan`'s generator uses an explicit include list # for exactly that reason, rather than crossing three axes and excluding the impossible cell. # Each leg's gate suite is SHARDED across runner jobs (2026-09-07): at 560+ gates the suite is ~150 # CPU-minutes, ~60 min wall at -j 3 on a 4-vCPU runner, and that one number was the wall clock of the # whole workflow. test/pargates.py --shard K/N cuts the list into N cost-balanced, deterministic slices # (weights in .github/pargates-shard-weights.json); every shard job builds its own binary (~3 min), stages the # HEAD comparison binary in its own step (a second build, outside every gate budget -- see that step), and # runs its slice. Every leg takes 4 shards. macOS took 2 until PR #110, on the grounds that GitHub caps # concurrent macOS jobs far lower than Linux; 2 put ~300 gates in each macOS job, doubling the -j 3 contention # every gate there ran under. The cost of 4 is stated rather than hidden: when the eight macOS shard jobs # (plus asan's) exceed the account's concurrent macOS cap, the excess waits for a free runner. # # The matrix itself is `plan`'s computed output, not a literal list here (see that job's step): its # SIZE is conditional on the light/full decision, which a static `include:` cannot express, and a # second hand-typed copy of the full 24-row list would drift from the one `plan` generates. matrix: ${{ fromJSON(needs.plan.outputs.release_matrix) }} runs-on: ${{ matrix.os }} # The macOS legs test the toolchain release.yml's macos-arm64 leg ships: the same runner, Xcode 26.6 and the pinned # minimum macOS 14.0 (test/portablebuildcheck.sh #2i holds all three against release.yml). Xcode 26.6's Apple clang # 21 is LLVM 18+, so its loop vectorizer reads ASSUME_NO_ALIAS's separate-storage promise (test/noaliascheck.sh arm # 6); its libc++ still defines __cpp_lib_print at a 14.0 deployment target, and it decides what else it declares # available by that target, so the legs build at the one the release ships. Through 0.6.1 these legs ran Xcode # 16.2 (LLVM 17) on macos-14, which retires 2026-11-02. Both variables are empty, and inert, on the Linux legs. env: DEVELOPER_DIR: ${{ matrix.os == 'macos-26' && '/Applications/Xcode_26.6.app/Contents/Developer' || '' }} MACOSX_DEPLOYMENT_TARGET: ${{ matrix.os == 'macos-26' && '14.0' || '' }} steps: # L4 (Linux probe): checkout@v4 defaults to a --depth 1 clone, which leaves ONE commit of history # in the tree. The churn / co-change / ownership gates (churnjoincheck, hotspots, --owners, the # quality short-horizon-churn kind) mine `git log` for real, so on a shallow checkout they do not # error — they measure zero and fail, or worse, pass while measuring nothing. Full history it is. - uses: actions/checkout@v4 with: fetch-depth: 0 # churn/co-change gates read real git history — a shallow clone reddens them persist-credentials: false # no step in this job pushes or fetches with the token # clang is installed on BOTH Linux legs, not only the clang one. optremarkscheck's Clang-only # configure arms pin `clang++` when the default front end is not Clang, so the gcc leg keeps that # coverage instead of skipping it — the same remedy the asan job applies for G1's Clang-only # sanitizer members. It is a package install, not a change of the leg's own compiler: the gcc leg # still BUILDS ripwire with gcc, which is the point of having it. - name: Install tooling (Linux) if: runner.os == 'Linux' run: | # Third-party apt sources ship in the runner image and are outside our control. TWICE now # one has been mid-publish when a job ran and taken every Linux job on every branch down # with it: `apt-get update` exits 100 and the leg dies in setup, before a single gate runs, # with a failure that looks nothing like a package problem. google-chrome on 2026-09-09 # ("Hash Sum mismatch"; deb822 .sources on 24.04, not .list) and packages.microsoft.com on # 2026-09-10 ("403 Forbidden ... is no longer signed"). Naming vendors one at a time only # ever fixes the last outage, so: we install ubuntu-archive packages ONLY, and every # non-ubuntu source goes. -print names what went, so a future reader can tell whether a # missing package traces back to here. ls /etc/apt/sources.list.d/ || true sudo find /etc/apt/sources.list.d -type f \( -name '*.list' -o -name '*.sources' \) \ ! -name 'ubuntu*' -print -delete || true sudo apt-get update # poppler-utils is pdftotext: ripwirepubliccheck arm 1b FAILS a tracked PDF deck it cannot read. sudo apt-get install -y --no-install-recommends libxml2-utils ripgrep bc clang poppler-utils - name: Install tooling (macOS) if: runner.os == 'macOS' # poppler is pdftotext, which ripwirepubliccheck arm 1b needs: it FAILS a tracked deck it cannot read. Only the # Release legs install it, because the plain legs skip that gate (.github/pargates-macos-plain-skip.txt). run: brew install ripgrep ${{ matrix.flavor == 'Release' && 'poppler' || '' }} # CC/CXX: the clang leg names clang; the gcc leg names gcc-14 (the std::print floor, 2026-09-08 — the # ubuntu-24.04 image ships GNU C++ 12.4 / 13.3 / 14.2, and the SYSTEM default `g++` is 13.3, whose # libstdc++ has but not ); the macOS leg is left to its platform default under the # DEVELOPER_DIR pinned at the job level. The stranger's-clone question this used to answer — does # the stock ubuntu g++ 13 still build? — is answered by the `fallback-emitter` job below instead, # which builds with g++-13 on purpose and proves the fallback path emits the same bytes. - name: Configure (portable — no RIPWIRE_NATIVE; empty build type = the plain flavour) env: CC: ${{ matrix.cc == 'clang' && 'clang' || matrix.cc == 'gcc' && 'gcc-14' || '' }} CXX: ${{ matrix.cc == 'clang' && 'clang++' || matrix.cc == 'gcc' && 'g++-14' || '' }} run: cmake -S . -B build ${{ matrix.flavor == 'Release' && '-DCMAKE_BUILD_TYPE=Release' || '' }} # The front end is recorded per leg so a log reader never has to infer it from the job name alone — # a `cc: clang` leg whose CMake cache says GNU is a broken matrix, not a passing run. # A macOS leg also proves the release toolchain pin reached it. The job env's matrix.os condition falls back to an # empty string on any other runner label, so a runner bump without the condition would otherwise build silently # with the image's default Xcode at the runner's own minimum macOS. - name: Confirm the front end this leg actually configured with run: | grep -h 'CMAKE_CXX_COMPILER_ID\|CMAKE_CXX_COMPILER_VERSION' build/CMakeFiles/*/CMakeCXXCompiler.cmake if [ "$RUNNER_OS" = macOS ]; then if [ -z "$DEVELOPER_DIR" ] || [ -z "$MACOSX_DEPLOYMENT_TARGET" ] \ || ! grep -qx "CMAKE_OSX_DEPLOYMENT_TARGET:STRING=$MACOSX_DEPLOYMENT_TARGET" build/CMakeCache.txt; then echo "ci: macOS leg without the release toolchain pin: DEVELOPER_DIR='$DEVELOPER_DIR' MACOSX_DEPLOYMENT_TARGET='$MACOSX_DEPLOYMENT_TARGET' cache='$( grep '^CMAKE_OSX_DEPLOYMENT_TARGET:' build/CMakeCache.txt )'" >&2 exit 1 fi echo "DEVELOPER_DIR=$DEVELOPER_DIR MACOSX_DEPLOYMENT_TARGET=$MACOSX_DEPLOYMENT_TARGET (in CMakeCache.txt)" fi - name: Build run: cmake --build build -j # The floor this matrix exists to hold, read off the BINARY rather than inferred from the compiler # version: emit= names the emitter that compiled in (src/infra/emit.h). A leg whose standard library # lacks still builds — the emitter falls back to std::format+fputs by feature test — so # without this line such a leg would build green and read as "the std::print floor holds". - name: The emitter this leg compiled in is std::print run: | ./build/ripwire --version ./build/ripwire --version | grep -q 'emit=std::print' # The Linux half of the cross-platform byte-identity check (xplat-diff below; the Windows half runs in # windows-package.yml on the packaged exe). ONE leg — ubuntu-24.04/Release/clang, shard 1, which the light set # also runs — and BEFORE the suite, so the outputs exist even when an unrelated gate goes red. About 6 s. - name: Cross-platform verb set, Linux half (scripts/ci-xplat-outputs.sh) if: matrix.os == 'ubuntu-24.04' && matrix.cc == 'clang' && matrix.flavor == 'Release' && matrix.shard == 1 run: bash scripts/ci-xplat-outputs.sh ./build/ripwire xplat-linux - uses: actions/upload-artifact@v4 if: matrix.os == 'ubuntu-24.04' && matrix.cc == 'clang' && matrix.flavor == 'Release' && matrix.shard == 1 with: name: xplat-linux path: xplat-linux if-no-files-found: error retention-days: 7 # The HEAD comparison binary, built ONCE per job in its own step: before the suite, outside every gate budget. # Six gates (crossdirinclude, nestedimport, preproccond, py/rust/tsimportprecise; binoverridecheck re-runs them) # compare this tree's binary against a second ripwire built from git HEAD. test/lib/headbinlib.sh used to build # it INSIDE whichever of them ran first: a parallel cmake build inside a wall-clock budget, beside -j 3 # neighbours, whose cost is super-linear in that contention. On macos-14 Release shard 2/2 the xmlwellformed # runner-speed probe moved 4.6x across five draws while crossdirincludecheck moved more than 15x, and it was # killed at 900 s and then at 1200 s with the budget floor applied (runs 34479806177, 34495265793; again in # 34514091873). No budget survives the next slow draw, so the build moved here, where it gets the idle runner # the Build step above gets. RIPWIRE_HEADBIN puts headbinlib.sh in STAGED mode for every gate in this job: it # never builds and never waits, and a missing or wrong binary FAILS the gate instead of rebuilding inside its # budget. test/headbinstagecheck.sh gates that this step exists, precedes the suite, verifies and exports. - name: Stage the HEAD comparison binary (once per job, before the suite, outside every gate budget) run: | export RIPWIRE_HEADBIN_BUILD_LOG="$RUNNER_TEMP/headbin-build.log" . test/lib/headbinlib.sh hb="$( ripwire_head_binary "$PWD" "$RUNNER_TEMP" )" || { echo "HEAD binary build failed; last 80 lines of its log:"; tail -n 80 "$RIPWIRE_HEADBIN_BUILD_LOG"; exit 1; } ripwire_headbin_verify "$hb" "$( git rev-parse HEAD )" "$hb" --version echo "RIPWIRE_HEADBIN=$hb" >> "$GITHUB_ENV" # test/pargates.py runs the SAME authoritative gate list regression.sh holds (manifestcheck.sh # keeps that list honest) in parallel, with the timing-sensitive edit-check budget isolated via # its `exclusive` group — the exact runner CLAUDE.md prescribes locally. Sequential regression.sh # cost ~26 min per suite pass at 170 gates; at 560+ gates the whole suite measured ~62 min wall per # leg at -j 3 (run 34109497781, 2026-09-07), so each job now runs ONE shard of it — see the matrix # comment. The shard's own gates are still LPT-ordered by the scratch timings; -j 3 stays because # the 4-vCPU runners saturate (many gates are internally multi-threaded). - name: "gate suite, parallel, this job's shard (560+ gates: determinism, cache transparency, G4 XML, quality, …)" # --budget-scale 4: the harness's flat 300 s default is a hang tripwire calibrated on an idle dev machine; # here -j 3 on a 4-vCPU runner is a 3-8x multiplier (two gates hit rc=124 at exactly 300 s in the first # sharded runs). The scale applies to the default only; GATE_BUDGET_SEC's explicit entries stay as declared. # `skiplist` (macOS plain only): a committed list of platform-independent gates this leg does not run — # see .github/pargates-macos-plain-skip.txt for the measurement and the per-gate reason. run: python3 test/pargates.py . build/ripwire -j 3 --shard ${{ matrix.shard }}/${{ matrix.shards }} --budget-scale 4 ${{ matrix.skiplist && format('--exclude-list {0}', matrix.skiplist) || '' }} - name: det-gate — 2-run byte-identical diff run: | ./build/ripwire test/fixture --no-cache >/tmp/run_a.xml ./build/ripwire test/fixture --no-cache >/tmp/run_b.xml diff -q /tmp/run_a.xml /tmp/run_b.xml - name: G4 — xmllint --noout run: ./build/ripwire test/fixture --no-cache | xmllint --noout - # ─── fallback-emitter: the stock ubuntu compiler still builds, and emits the SAME bytes ─────────────── # # Every leg above pins a toolchain with . A stranger's `apt install g++` on ubuntu-24.04 is still # 13.3, whose libstdc++ has no ; src/infra/emit.h then compiles the std::format+fputs arm. This job # builds that arm ON PURPOSE (CC/CXX = gcc-13/g++-13, both on the image), asserts the binary discloses it, # and runs the printf-family parity fence against the manifest the std::print legs pinned — so the two # arms are proven byte-identical on every CI run, not assumed. Without this job the fallback would rot # unobserved until the next stranger hit it. fallback-emitter: needs: plan if: needs.plan.outputs.full == 'true' # not part of the light set — see the header comment name: fallback-emitter (ubuntu gcc-13, std::format+fputs) runs-on: ubuntu-24.04 steps: - uses: actions/checkout@v4 with: fetch-depth: 0 persist-credentials: false # no step in this job pushes or fetches with the token - name: Install tooling run: | # Third-party apt sources ship in the runner image and are outside our control. TWICE now # one has been mid-publish when a job ran and taken every Linux job on every branch down # with it: `apt-get update` exits 100 and the leg dies in setup, before a single gate runs, # with a failure that looks nothing like a package problem. google-chrome on 2026-09-09 # ("Hash Sum mismatch"; deb822 .sources on 24.04, not .list) and packages.microsoft.com on # 2026-09-10 ("403 Forbidden ... is no longer signed"). Naming vendors one at a time only # ever fixes the last outage, so: we install ubuntu-archive packages ONLY, and every # non-ubuntu source goes. -print names what went, so a future reader can tell whether a # missing package traces back to here. ls /etc/apt/sources.list.d/ || true sudo find /etc/apt/sources.list.d -type f \( -name '*.list' -o -name '*.sources' \) \ ! -name 'ubuntu*' -print -delete || true sudo apt-get update sudo apt-get install -y --no-install-recommends libxml2-utils ripgrep bc - name: Configure with the image's stock gcc 13 (plain flavour) env: CC: gcc-13 CXX: g++-13 run: cmake -S . -B build - name: Confirm the front end this leg actually configured with run: grep -h 'CMAKE_CXX_COMPILER_ID\|CMAKE_CXX_COMPILER_VERSION' build/CMakeFiles/*/CMakeCXXCompiler.cmake - name: Build run: cmake --build build -j - name: The emitter this leg compiled in is the fallback, and says so run: | ./build/ripwire --version ./build/ripwire --version | grep -q 'emit=std::format+fputs' - name: Byte parity — the fallback arm reproduces the manifest the std::print legs pinned run: bash test/printffmtparitycheck.sh - name: versioncheck — the disclosure arm on this arm run: bash test/versioncheck.sh - name: det-gate — 2-run byte-identical diff run: | ./build/ripwire test/fixture --no-cache >/tmp/run_a.xml ./build/ripwire test/fixture --no-cache >/tmp/run_b.xml diff -q /tmp/run_a.xml /tmp/run_b.xml # ─── rhel: the platform this project actually ships a prebuilt for, checked at PR time ─────────────── # # GitHub hosts no RHEL runner image, but a job can run inside RHEL userland ON a hosted Ubuntu runner via # `container:`. UBI (Universal Base Image) pulls from registry.access.redhat.com with no subscription, # which is what makes this work on public CI — release.yml's `smoke-rhel` job already relies on it. # # WHY IT IS WORTH A LEG. release.yml builds the Linux artifact in a manylinux_2_28 container and smoke-runs # it on ubi9 precisely because a binary built on stock ubuntu-24.04 needs GLIBCXX_3.4.31 and DIES on RHEL 9 # (verified, and recorded in that workflow's own comment). But that check only runs when a release is cut: # every ordinary PR was tested on glibc 2.39 Ubuntu and macOS only, so a RHEL-incompatible change stayed # invisible until publish time. This leg moves that signal to the pull request. # # SCOPE, STATED HONESTLY. It builds and runs the two headline product guarantees (determinism, well-formed # XML) plus the fixture self-run — NOT the 361-gate suite. That is deliberate, not laziness: the suite needs # ripgrep, which is not in the RHEL repositories (it lives in EPEL), and the RHEL-specific signal here is # "does this toolchain compile it and does this userland run it", which the ubuntu legs cannot give and # which these steps do give. The full suite stays on the four ubuntu legs and the two macOS ones. rhel: needs: plan if: needs.plan.outputs.full == 'true' # not part of the light set — see the header comment name: rhel (ubi9, ${{ matrix.flavor }}) runs-on: ubuntu-24.04 container: registry.access.redhat.com/ubi9/ubi strategy: fail-fast: false matrix: flavor: [Release, plain] steps: - uses: actions/checkout@v4 with: fetch-depth: 0 persist-credentials: false # no step in this job pushes or fetches with the token # RHEL 9's default gcc is 11, which does not implement C++23 — CMakeLists sets CXX_STANDARD 23 with # STANDARD_REQUIRED ON, so the configure would fail outright. gcc-toolset-N is Red Hat's own supported # answer (a parallel-installable newer GCC in /opt/rh). Probe newest-first rather than pinning one exact # version: which toolsets a given UBI9 AppStream carries changes over the RHEL 9.x lifecycle, and a hard # pin on one N turns a routine repo refresh into a red leg the day that exact package is dropped. # Whichever is found is ECHOED, so the log always records which compiler actually built this leg rather # than leaving it to be inferred. # # Floor is 14, not merely "whichever toolset exists": the house emitter (src/infra/emit.h) is # std::print, which libstdc++ implements starting at GCC 14 ( arrived at 13, at 14). # A lower toolset still BUILDS — the emitter falls back by feature test — which is exactly why the # probe must not accept one: the leg would report "toolset found, leg is fine" while the binary it # built took the fallback. The assertion step after Build reads the truth off the binary. - name: Install the RHEL build toolchain run: | set -eu dnf install -y cmake git libxml2 python3 make diffutils findutils found="" for v in $( seq 20 -1 14 ); do if dnf install -y "gcc-toolset-$v-gcc-c++"; then found="$v" break fi done [ -n "$found" ] || { echo "no gcc-toolset-N (N>=14) in this UBI9 AppStream — the house emitter is std::print, which needs libstdc++ 14+ (); gcc-toolset-13 and older would silently take the std::format+fputs fallback"; exit 1; } echo "TOOLSET=$found" >> "$GITHUB_ENV" - name: Record the toolchain this leg is using run: | echo "gcc-toolset-$TOOLSET" "/opt/rh/gcc-toolset-$TOOLSET/root/usr/bin/g++" --version cmake --version | head -1 cat /etc/redhat-release # CC/CXX point straight at the toolset's binaries rather than `source .../enable`, because every `run:` # step is its own shell — a sourced environment would not survive to the next step. - name: Configure (portable — no RIPWIRE_NATIVE; empty build type = the plain flavour) env: CC: /opt/rh/gcc-toolset-${{ env.TOOLSET }}/root/usr/bin/gcc CXX: /opt/rh/gcc-toolset-${{ env.TOOLSET }}/root/usr/bin/g++ run: cmake -S . -B build ${{ matrix.flavor == 'Release' && '-DCMAKE_BUILD_TYPE=Release' || '' }} - name: Build run: cmake --build build -j - name: The emitter this leg compiled in is std::print (gcc-toolset-14's libstdc++ has ) run: | ./build/ripwire --version ./build/ripwire --version | grep -q 'emit=std::print' - name: Self-run on the fixture (it parses and ranks under RHEL userland) run: ./build/ripwire test/fixture --no-cache | head -c 400; echo - name: det-gate — 2-run byte-identical diff run: | ./build/ripwire test/fixture --no-cache >/tmp/run_a.xml ./build/ripwire test/fixture --no-cache >/tmp/run_b.xml diff -q /tmp/run_a.xml /tmp/run_b.xml - name: G4 — xmllint --noout run: ./build/ripwire test/fixture --no-cache | xmllint --noout - asan: needs: plan if: needs.plan.outputs.full == 'true' # not part of the light set — see the header comment name: asan (${{ matrix.os }}) strategy: fail-fast: false matrix: os: [macos-26, ubuntu-24.04] runs-on: ${{ matrix.os }} env: # Darwin's arm64 runtime rejects the standalone leak sanitizer at startup (CMakeLists.txt's own # comment on this, `ripwire_asan_fixture`); Linux gets full leak detection. # matrix.os, not runner.os: the runner context does not exist at job-level env — GitHub's # parser rejects the whole workflow (found by the first real Actions run; local YAML lint # cannot see it). Step-level `if: runner.os` below is legal and stays. # log_path: a sanitizer abort inside a gate that pipes 2>/dev/null leaves NO report in the log # (CI round 3: an asan/x86-64 abort read as "pin no longer resolves"). With log_path the report # lands in a file the failure artifact below preserves. ASAN_OPTIONS: ${{ matrix.os == 'macos-26' && 'detect_leaks=0:halt_on_error=1:abort_on_error=1:log_path=/tmp/asanlog' || 'detect_leaks=1:halt_on_error=1:abort_on_error=1:log_path=/tmp/asanlog' }} UBSAN_OPTIONS: halt_on_error=1:print_stacktrace=1:log_path=/tmp/asanlog LSAN_OPTIONS: suppressions=${{ github.workspace }}/lsan_suppressions.txt # The release toolchain on the macOS leg, as in the build matrix: Xcode 26.6 at the pinned minimum macOS 14.0 # (test/portablebuildcheck.sh #2i). Inert (empty) on Linux. DEVELOPER_DIR: ${{ matrix.os == 'macos-26' && '/Applications/Xcode_26.6.app/Contents/Developer' || '' }} MACOSX_DEPLOYMENT_TARGET: ${{ matrix.os == 'macos-26' && '14.0' || '' }} steps: # L4 (Linux probe): checkout@v4 defaults to a --depth 1 clone, which leaves ONE commit of history # in the tree. The churn / co-change / ownership gates (churnjoincheck, hotspots, --owners, the # quality short-horizon-churn kind) mine `git log` for real, so on a shallow checkout they do not # error — they measure zero and fail, or worse, pass while measuring nothing. Full history it is. - uses: actions/checkout@v4 with: fetch-depth: 0 # churn/co-change gates read real git history — a shallow clone reddens them persist-credentials: false # no step in this job pushes or fetches with the token - name: Install tooling (Linux) if: runner.os == 'Linux' run: | # Third-party apt sources ship in the runner image and are outside our control. TWICE now # one has been mid-publish when a job ran and taken every Linux job on every branch down # with it: `apt-get update` exits 100 and the leg dies in setup, before a single gate runs, # with a failure that looks nothing like a package problem. google-chrome on 2026-09-09 # ("Hash Sum mismatch"; deb822 .sources on 24.04, not .list) and packages.microsoft.com on # 2026-09-10 ("403 Forbidden ... is no longer signed"). Naming vendors one at a time only # ever fixes the last outage, so: we install ubuntu-archive packages ONLY, and every # non-ubuntu source goes. -print names what went, so a future reader can tell whether a # missing package traces back to here. ls /etc/apt/sources.list.d/ || true sudo find /etc/apt/sources.list.d -type f \( -name '*.list' -o -name '*.sources' \) \ ! -name 'ubuntu*' -print -delete || true sudo apt-get update sudo apt-get install -y --no-install-recommends libxml2-utils ripgrep bc clang - name: Install tooling (macOS) if: runner.os == 'macOS' run: brew install ripgrep # CLANG, not the default cc. G1's `integer` group is Clang-only and gcc rejects the whole option # ("unrecognized argument to '-fsanitize=' option: 'integer'" — the first public CI run). CMakeLists.txt # now filters the Clang-only members out under GCC, but that is the HONEST-DEGRADE path for a # contributor, not something public CI should be running: pinning clang here keeps the COMPLETE G1 # stack on Linux, so the gates below mean the same thing on both matrix legs. # # Deliberately NOT applied to the release job: gcc there is a FEATURE. It is a second front end over # the same tree and it found three real portability bugs on its first run. - name: Configure (G1 sanitizer stack — clang, see the step comment) if: runner.os == 'Linux' env: CC: clang CXX: clang++ run: cmake -S . -B asan -DRIPWIRE_ASAN=ON # Fails, as the build matrix's front-end step does, when the release toolchain pin did not reach this leg. - name: Configure (G1 sanitizer stack) if: runner.os == 'macOS' run: | cmake -S . -B asan -DRIPWIRE_ASAN=ON if [ -z "$DEVELOPER_DIR" ] || [ -z "$MACOSX_DEPLOYMENT_TARGET" ] \ || ! grep -qx "CMAKE_OSX_DEPLOYMENT_TARGET:STRING=$MACOSX_DEPLOYMENT_TARGET" asan/CMakeCache.txt; then echo "ci: macOS asan leg without the release toolchain pin: DEVELOPER_DIR='$DEVELOPER_DIR' MACOSX_DEPLOYMENT_TARGET='$MACOSX_DEPLOYMENT_TARGET' cache='$( grep '^CMAKE_OSX_DEPLOYMENT_TARGET:' asan/CMakeCache.txt )'" >&2 exit 1 fi - name: Build run: cmake --build asan -j - name: The emitter this leg compiled in is std::print run: | ./asan/ripwire --version ./asan/ripwire --version | grep -q 'emit=std::print' - name: G1 fixture self-run (ripwire_asan_fixture) run: cmake --build asan --target ripwire_asan_fixture - name: Heavy verb — --pack-task (test/packtaskcheck.sh) run: RIPWIRE_BIN=asan/ripwire bash test/packtaskcheck.sh - name: Heavy verb — --from-trace (test/tracecheck.sh) run: RIPWIRE_BIN=asan/ripwire bash test/tracecheck.sh - name: Heavy verb — --quality-delta (test/qualitycheck.sh) run: RIPWIRE_BIN=asan/ripwire bash test/qualitycheck.sh - name: Heavy verb — --merge-scout (test/mergescoutcheck.sh) run: RIPWIRE_BIN=asan/ripwire bash test/mergescoutcheck.sh - name: Heavy verb — --edit-check (test/editcheckcheck.sh) run: RIPWIRE_BIN=asan/ripwire bash test/editcheckcheck.sh # The TS/JS import resolver (tsconfig/package.json/workspace readers) under the G1 sanitizers. Train 21's # JsonNode::get wrapped its unsigned index on every key miss (-fsanitize=integer, exit 134); editcheckcheck # arm (g) only caught it because this repo's own package.json happens to lack `workspaces`. This gate's # fixtures drive every reader deterministically (72 aborts before the fix). - name: TS/JS import resolution — --deps (test/depsprecisecheck.sh) run: RIPWIRE_BIN=asan/ripwire bash test/depsprecisecheck.sh # cachefuzzcheck's two mutation tables ARE sanitizer sweeps: they drive corrupt/truncated/hostile cache # blobs and corrupt qsnap blobs through the readers and assert that no sanitizer report fires. That needs # an instrumented binary — RIPWIRE_ASAN_BIN is the whole point of the gate — and the `release` jobs, which # configure only build/, cannot supply one. Until now the gate hard-FAILED there for the missing binary # (PR #1, run 30732976779: "absorb gate (cachefuzzcheck.sh failed)" on BOTH release legs, that one line # its only failure) while NO job anywhere ran the sweeps it exists for. The gate now skips those two arms # with a named reason on a leg that has no ASan build, and this step is where they actually run: both # variables point at the instrumented binary, so the ground-truth arms and the sanitizer arms agree by # construction. Adding it here is what makes that skip reason true rather than an excuse. - name: Sanitizer sweep — corrupt cache / qsnap blobs (test/cachefuzzcheck.sh) run: RIPWIRE_BIN=asan/ripwire RIPWIRE_ASAN_BIN=asan/ripwire bash test/cachefuzzcheck.sh # NOT a heavy verb — here because it is the ONLY gate whose subject is an integer-overflow class that # a plain binary cannot observe. cppqualcheck §9 drives an unbalanced `operator>` frame through # --from-trace; before the H4 W2b fixup that aborted the G1 build (rc=134) while the plain build ran # it in silence. The suite job above runs the whole suite TWICE and both times PLAIN, so this gate was # green-by-construction in CI until it appeared here (V3-H-2). - name: Sanitizer-only class — cppqualcheck §9 operator-frame scan (test/cppqualcheck.sh) run: RIPWIRE_BIN=asan/ripwire bash test/cppqualcheck.sh # crashsweepcheck B3 is the other sanitizer-only class: a file dated past 2262 overflowed the stat readers' # nanosecond product, silent UB in the plain build and an abort here. The job's Linux leg is the one whose # filesystem stores such a date (APFS clamps at 2262, and the arm says so on macOS). B2's hostile # fixed-name files ride along on the instrumented binary; B1's preload shim skips a sanitizer build by name. - name: Sanitizer-only class — crashsweepcheck far-future timestamp and fixed-name shapes (test/crashsweepcheck.sh) run: RIPWIRE_BIN=asan/ripwire RIPWIRE_ASAN_BIN=asan/ripwire bash test/crashsweepcheck.sh # A lifetime class only a sanitizer shows: one long-lived `--mcp` session rebuilt in place (a chmod, a new non-source # file, an edit, a root switch, a memory release) and queried again each time. A cached ValueRefIndex once outlived # the index it pointed into (heap-use-after-free) and no single-query gate could see it. The gate drives that session # on this binary for 20 cycles and checks every answer against a one-shot CLI run; one binary plays both roles. # LeakSanitizer is off in this step on BOTH legs (the gate sets detect_leaks=0 for its own processes; its subject is # lifetime, not leaks, and the Linux leg has not yet shown a clean exit-time leak set for it). # A failure leaves the gate's sanitizer reports in /tmp/asanlog-mcpreload.* for the upload step below. - name: Sanitizer-only class — MCP index reloads under ASan (test/mcpreloadasancheck.sh) run: RIPWIRE_BIN=asan/ripwire bash test/mcpreloadasancheck.sh - name: det-gate — 2-run byte-identical diff (ASan binary) run: | ./asan/ripwire test/fixture --no-cache >/tmp/run_a.xml ./asan/ripwire test/fixture --no-cache >/tmp/run_b.xml diff -q /tmp/run_a.xml /tmp/run_b.xml - name: G4 — xmllint --noout (ASan binary) run: ./asan/ripwire test/fixture --no-cache | xmllint --noout - # Self-dogfood, STILL NON-GATING — but for a smaller and better-measured reason than before. # # The verb now separates the historical floor from the rot: a failed anchor the AUTHOR dated is # kind="dated-record" and counts in dated=, leaving drift= for live claims. On this repo's own docs # that splits 104 failed anchors into 63 rot + 41 records, and BOTH constants this comment used to # name (`kMcpVerbCount = 22`, `kCacheVersion=7`) are now records — their authors had already written # the "as of " hedge the lane reads. # # `|| true` stays, and note first that it was never suppressing an exit code: --doc-drift always # exits 0 by design (a report, not a gate), so this only guards a crash. Gating would mean asserting # drift="0", and the residual 63 will not go to zero by any rule change — they are genuine stale # file:line maps in undated DESIGN studies, plus two audits that are manifestly artifacts-of-a-date # to a human but never write that date where a machine can read it. The lane refuses to guess genre # (measured: a bare date in the opening prose mis-dates three LIVE documents here), and git history # cannot help either — 90 of 98 stale anchors were CORRECT at their own doc's last commit, records # and rot alike. So the honest options are a zero-gate that is red on day one, or a ratchet ceiling # that goes red whenever a doc is edited in a shared tree. Neither earns its noise. # # What DOES gate is test/docdriftcheck.sh, which pins every lane and every rec= kind on a labelled # fixture, including the negative controls. Re-open this decision when the DESIGN docs' anchors are # fixed and the review notes carry a self-date: at that point drift= can be gated at 0. - name: Self-report (non-gating) — --doc-drift on our own docs run: ./asan/ripwire . --doc-drift --with-history || true # Sanitizer reports written via log_path survive gates that swallow stderr; keep them when a # step failed so the log tells us WHICH abort happened, not just that one did. - name: Upload sanitizer reports on failure if: failure() uses: actions/upload-artifact@v4 with: name: sanitizer-reports-${{ matrix.os }} path: /tmp/asanlog* if-no-files-found: ignore # ─── windows: the port's build proof, on both MSVC-ABI front ends ────────────────────────────────────── # # Nobody on this side of the port has a Windows machine, so "it builds on Windows" can only ever be a green # job here — @lennix1337's local run against #44 is what told us it did NOT, and this job is what keeps the # answer from going stale between contributor runs. Two legs, because the owner's 2026-09-19 ruling is that # clang-cl stays the primary toolchain and plain cl.exe must ALSO build: a fix that pleases one front end # and breaks the other is the failure mode this matrix exists to catch (it caught `-Werror=switch`, which # clang-cl takes and cl.exe reads as a numeric argument to /W). # # WHAT THIS JOB IS NOT. It is a BUILD proof plus a smoke test, not the gate suite — the bash gate scripts # need the Windows harness that is still to land, and running a subset of them here would report a green # that means less than it looks. What runs instead is the part that needs no harness: the unit tests # (ctest — including ripwire.oswin32logic, the Windows port's own pure-logic suite), `--version`/`--help`, # a real crawl of test/fixture, the two-run byte-identical determinism contract, and G4 well-formedness. # Everything beyond that is still a contributor's run on real hardware; see the checklist on #44. Both legs # run every step below; only the ASan flavour's compile is clang-cl-only. # # Ninja, and the VC environment from vcvars64.bat rather than the Visual Studio generator: single-config is # what every flavour rule in CMakeLists.txt assumes (an empty CMAKE_BUILD_TYPE IS the plain flavour), and it # is the toolchain spelling the contributor's report used, so an error here is comparable to one there. # vcvars only exports into its own process, so configure+build live in ONE cmd step per leg. windows: needs: plan if: needs.plan.outputs.full == 'true' # not part of the light set — as rhel/asan/fallback-emitter name: windows (${{ matrix.toolchain }}, ninja) strategy: fail-fast: false # the two front ends fail differently and both answers are wanted in one run matrix: toolchain: [clang-cl, cl] runs-on: windows-latest # cl.exe builds: the GCC/Clang language extensions the tree is written in (`[[gnu::…]]` attributes, # `__attribute__((…))`, `__builtin_*`, the `asm volatile` barriers) go through the compiler-extension seam # in src/infra/platform.h. Until that seam landed this leg asserted its own documented failure; it now # builds, links, runs the binary and ctest, and holds the determinism and well-formedness contracts, the # same steps as clang-cl. This job does NOT carry continue-on-error: a break on either front end is red. defaults: run: shell: bash # Git Bash: the smoke steps below are POSIX shell; the build steps override to cmd steps: - uses: actions/checkout@v4 with: fetch-depth: 0 # churn/co-change gates read real git history — a shallow clone reddens them persist-credentials: false # no step in this job pushes or fetches with the token # One step, because vcvars64.bat exports the INCLUDE/LIB/PATH the compilers need into its own process # only. vswhere locates the install rather than a hard-coded "2022\Enterprise" path, so a runner-image # edition bump does not silently red this leg. The standalone LLVM on the image is preferred over the # one bundled with Visual Studio when it is present (it is the newer clang, and the one the contributor # reported against); either way vcvars has already supplied the MSVC headers and import libraries it # compiles against, which is what makes clang-cl an MSVC-ABI compiler and not a second toolchain. - name: Configure and build (${{ matrix.toolchain }}, Ninja, plain flavour) shell: cmd run: | @echo on for /f "usebackq tokens=*" %%i in (`"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath`) do set "VSINSTALL=%%i" if not defined VSINSTALL ( echo ci: no Visual Studio install with the x64 C++ tools & exit /b 1 ) echo VSINSTALL=%VSINSTALL% call "%VSINSTALL%\VC\Auxiliary\Build\vcvars64.bat" || exit /b 1 set "RW_LLVM=%VSINSTALL%\VC\Tools\Llvm\x64\bin" if exist "C:\Program Files\LLVM\bin\clang-cl.exe" set "RW_LLVM=C:\Program Files\LLVM\bin" set "RW_TC=-DCMAKE_C_COMPILER=cl -DCMAKE_CXX_COMPILER=cl" if "${{ matrix.toolchain }}"=="clang-cl" set "RW_TC=-DCMAKE_C_COMPILER=clang-cl -DCMAKE_CXX_COMPILER=clang-cl" if "${{ matrix.toolchain }}"=="clang-cl" set "PATH=%RW_LLVM%;%PATH%" cmake --version || exit /b 1 cmake -S . -B build -G Ninja %RW_TC% -DCMAKE_BUILD_TYPE= -DRIPWIRE_TESTS=ON || exit /b 1 cmake --build build -j || exit /b 1 # The front end this leg actually configured with, read off the cache rather than inferred from the # matrix label — a `clang-cl` leg whose cache says MSVC is a broken matrix, not a passing run. - name: Confirm the front end this leg actually configured with run: | grep -h 'CMAKE_CXX_COMPILER_ID\|CMAKE_CXX_COMPILER_VERSION\|CMAKE_CXX_COMPILER_FRONTEND_VARIANT' build/CMakeFiles/*/CMakeCXXCompiler.cmake # The flavour is PINNED by the configure above (-DCMAKE_BUILD_TYPE= is the plain one), not inherited: # this runner's environment hands an unasked-for Debug to a bare configure, and a flavour that depends # on the machine is not one deterministic build step (G3). Read it back rather than assume it took. grep -x 'CMAKE_BUILD_TYPE:STRING=' build/CMakeCache.txt \ || { echo "ci: windows leg did not configure the plain flavour: $( grep '^CMAKE_BUILD_TYPE:' build/CMakeCache.txt )" >&2; exit 1; } id="$( grep -h 'set(CMAKE_CXX_COMPILER_ID' build/CMakeFiles/*/CMakeCXXCompiler.cmake )" case "${{ matrix.toolchain }}" in clang-cl) case "$id" in *Clang*) ;; *) echo "ci: clang-cl leg configured with $id" >&2; exit 1 ;; esac ;; cl) case "$id" in *MSVC*) ;; *) echo "ci: cl leg configured with $id" >&2; exit 1 ;; esac ;; esac # The SHAPE of the answer, not just the exit code. `--version` printing nothing at all would satisfy # a bare invocation, and a binary that cannot find its own provenance is exactly what a broken # Windows build looks like. emit= and built_from= are the two tokens a bug report is useless without. - name: The binary runs — --version and --help run: | ./build/ripwire.exe --version | tee version.txt grep -qE '^ripwire [0-9]+\.[0-9]+\.[0-9]+ \(' version.txt grep -q 'emit=' version.txt grep -q 'built_from=' version.txt test "$( ./build/ripwire.exe --help | wc -c )" -gt 1000 # --no-tests=error: `ctest` exits 0 on an EMPTY test set, so without it this step passes when the # RIPWIRE_TESTS block registers nothing — which is a plausible Windows failure, not a hypothetical # (measured: empty CTestTestfile.cmake -> rc 0 bare, rc 8 with the flag). - name: Unit tests (ctest — csr, pagerank, radix, strkern, and the Windows port's own oswin32logic) run: ctest --test-dir build --output-on-failure --no-tests=error # A real crawl, then the two contracts that hold for the map on every platform: byte-identical across # two runs, and well-formed XML (G4). xmllint is not on the Windows image, so Python's own parser is # the reader here — it refuses exactly what xmllint --noout refuses. - name: det-gate and G4 — two runs byte-identical, well-formed, and NOT empty run: | ./build/ripwire.exe test/fixture --no-cache > a.xml ./build/ripwire.exe test/fixture --no-cache > b.xml cmp a.xml b.xml python -c "import xml.etree.ElementTree as E,sys; E.parse(sys.argv[1]); print('well-formed', sys.argv[1])" a.xml # `test -s` is not the assertion this step means. An EMPTY crawl — the crawler enumerating zero # entries, which is the precise failure class a Windows path-spelling bug produces — still emits a # 951-byte legend plus one bare , is deterministic across two runs, and parses as well-formed # XML. Measured on this fixture: 7 `= 7" >&2; exit 1; } test "$syms" -ge 15 || { echo "ci: the fixture crawl found $syms symbol(s), expected >= 15" >&2; exit 1; } case "$iters" in pr_iters=\"0\"|'') echo "ci: PageRank ran 0 iterations — the graph was empty ($iters)" >&2; exit 1 ;; esac # #326: cacheDirLadder() (src/quality.h) is the one function every cache consumer builds a path from — # --doctor's cache-dir row, resolveCacheBlobPath (nearly every cache-blob path), cache eviction, --slice/ # edit-preview temp roots, crossref, the doc-bridge cache, and the remote-clone cache all took its # un-rebased return, real only through os::mkdir's own silent Windows rebase. Fixed at the source: # cacheDirLadder() now returns the already-rebased path, so every consumer is correct by construction. # Reproduces the filed condition, then proves --doctor's report AND a warm-run cache hit # (resolveCacheBlobPath) end to end; eviction shares the fix by construction (same dir=, see CHANGELOG). # --doctor exits 1 on ANY non-ok row (binary-path included): the helper records that code instead of # aborting on it under `bash -eo pipefail`, and asserts on the cache-dir row alone. - name: "--doctor cache-dir and cache reuse must be TRUE (#326): no false unwritable/empty, no missed cache hit" run: | unset TMPDIR XDG_CACHE_HOME ./build/ripwire.exe test/fixture > populate.xml || { echo "ci: the populating crawl itself failed" >&2; exit 1; } row="$( bash scripts/ci-windows-doctor-row.sh ./build/ripwire.exe doctor.xml )" cat doctor.xml blobs1="$( printf '%s' "$row" | grep -o 'blobs="[0-9]*"' | grep -o '[0-9]*' || true )" [ -n "$blobs1" ] && [ "$blobs1" -ge 1 ] \ || { echo "ci: #326 regressed — cache-dir row reports blobs=$blobs1 right after a crawl that just wrote there: $row" >&2; exit 1; } # a second identical crawl must reuse the SAME blob (resolveCacheBlobPath), not write a new one. ./build/ripwire.exe test/fixture > populate2.xml || { echo "ci: the warm crawl itself failed" >&2; exit 1; } cmp populate.xml populate2.xml || { echo "ci: cold and warm crawls of the same tree produced different bytes" >&2; exit 1; } row2="$( bash scripts/ci-windows-doctor-row.sh ./build/ripwire.exe doctor2.xml )" blobs2="$( printf '%s' "$row2" | grep -o 'blobs="[0-9]*"' | grep -o '[0-9]*' || true )" [ -n "$blobs2" ] && [ "$blobs2" -eq "$blobs1" ] \ || { echo "ci: #326 regressed — a warm, identical crawl changed blobs= from $blobs1 to $blobs2 (a cache miss/duplicate write, not a reuse)" >&2; exit 1; } # #326's sibling: the eviction sweep walked cacheDirLadder()'s un-rebased "/tmp/ripwire-" with # std::filesystem and so evicted nothing on Windows, ever. The script seeds the REAL cache directory (from # --doctor's dir=) with stale blobs, forces a cache write, and asserts they are gone; loud on the pre-fix code. - name: "the cache-eviction sweep must really evict on Windows (sibling of #326): scripts/ci-windows-evict-smoke.sh" run: bash scripts/ci-windows-evict-smoke.sh ./build/ripwire.exe # The ASan flavour's own configure+build. It is here because it is a BUILD failure this round fixed — # clang-cl refuses `-fsanitize=address` together with the debug CRT a plain configure selects — and a # fix to a build failure that no build exercises is not fixed. ripwire_probe alone, not the whole # flavour: it carries the same sanitizer compile options as every other instrumented target, so it # proves the CRT choice reaches the compile, at a fraction of the runtime. Running the sanitizer # suite on Windows is a later round's work, not this one's. - name: ASan flavour configures and compiles (clang-cl only — the MSVC ABI's only sanitizer front end) if: matrix.toolchain == 'clang-cl' shell: cmd run: | @echo on for /f "usebackq tokens=*" %%i in (`"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath`) do set "VSINSTALL=%%i" call "%VSINSTALL%\VC\Auxiliary\Build\vcvars64.bat" || exit /b 1 set "RW_LLVM=%VSINSTALL%\VC\Tools\Llvm\x64\bin" if exist "C:\Program Files\LLVM\bin\clang-cl.exe" set "RW_LLVM=C:\Program Files\LLVM\bin" set "PATH=%RW_LLVM%;%PATH%" cmake -S . -B build-asan -G Ninja -DCMAKE_C_COMPILER=clang-cl -DCMAKE_CXX_COMPILER=clang-cl -DCMAKE_BUILD_TYPE= -DRIPWIRE_ASAN=ON || exit /b 1 cmake --build build-asan --target ripwire_probe || exit /b 1 # ─── windows-package: the windows-x64 release zip, built exactly as release.yml builds it, then verified ─── # # The job IS release.yml's build-windows job: both call .github/workflows/windows-package.yml, so the zip this # run uploads (artifact ripwire--windows-x64) is what a tag would publish, and a Windows tester can take it # from the PR run with no release. That file's header lists what it proves on the UNZIPPED exe and its design # choices (clang-cl, Release/LTO, static CRT, zip). About 10-12 runner-minutes, in parallel with the release # shards, so no wall-clock cost; FULL set only, like the windows job above. windows-package: needs: plan if: needs.plan.outputs.full == 'true' # not part of the light set — as windows/rhel/asan name: windows-package uses: ./.github/workflows/windows-package.yml with: release_tag: "" # ─── xplat-diff: the packaged Windows exe and the Linux binary print the same bytes ───────────────────────── # # The strongest check we have on a platform none of us runs: the fixed verb set (scripts/ci-xplat-outputs.sh — # map, --for, --callers, --impact, --expand over three copies of test/fixture, plus one MCP session) from the # windows-package job and from the ubuntu-24.04/Release/clang shard-1 job, diffed byte for byte under the three # rules scripts/ci-xplat-diff.sh names (identity; a '\'-typed root prints '/'; CRLF changes only --expand's # source bytes). It waits for the release shards it needs one artifact from; the job itself takes under a minute. xplat-diff: needs: [plan, release, windows-package] if: ${{ !cancelled() && needs.plan.outputs.full == 'true' }} name: xplat-diff (Windows packaged exe vs Linux, same verbs, same bytes) runs-on: ubuntu-24.04 steps: - uses: actions/checkout@v4 with: persist-credentials: false - uses: actions/download-artifact@v4 with: name: xplat-linux path: xplat/linux - uses: actions/download-artifact@v4 with: name: xplat-windows path: xplat/windows - name: Windows vs Linux, byte for byte (scripts/ci-xplat-diff.sh) run: bash scripts/ci-xplat-diff.sh xplat/linux xplat/windows # ─── report-failure / report-green: this schedule's OWN tracking issue, separate from nightly.yml's ───── # # ci.yml's 05:41 UTC schedule and nightly.yml's 07:17 UTC schedule are two independent full runs against # main. They used to report to the SAME issue, and that was a bug: nightly.yml's report-green closes on a # green TSan run alone, which is not evidence THIS workflow's full matrix is also clean, so a green TSan # night could close an issue this workflow opened while its own matrix was still red — and the reverse, # a green full-matrix night closing an issue TSan opened while TSan was still red. Neither green run has # any way to know the other workflow's current state. # # THE FIX: two labels, not one. `nightly-failure` stays as the SHARED label (so "every nightly-scale # failure on main" is still one query), but every open/comment/close here also requires # `nightly-full-matrix` — the label ONLY this workflow's report-failure applies, and ONLY this workflow's # report-green matches on. nightly.yml's own pair does the same with `nightly-tsan` instead, on ITS own # issue titled "Nightly checks failing on main (TSan)". The two issues, two titles, two label pairs: a # green run in one workflow can now only ever touch the issue carrying ITS OWN second label. Neither job # here runs for push, pull_request or workflow_dispatch — only the scheduled full-matrix run reports, # matching the "scheduled full-matrix failures" scope this pair exists for; a workflow_dispatch run is # watched live by whoever triggered it. See CONTRIBUTING.md §5 for the paragraph-length version. report-failure: name: report-failure (open or update the tracking issue) needs: [plan, style, release, fallback-emitter, rhel, asan, windows, windows-package, xplat-diff] if: failure() && github.event_name == 'schedule' && github.ref == 'refs/heads/main' runs-on: ubuntu-24.04 permissions: issues: write steps: - name: Open "Nightly checks failing on main (full matrix)", or comment on the open one env: GH_TOKEN: ${{ github.token }} REPO: ${{ github.repository }} HEAD_SHA: ${{ github.sha }} RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} NEEDS_JSON: ${{ toJSON(needs) }} run: | label=nightly-failure ownLabel=nightly-full-matrix body="$RUNNER_TEMP/body.md" failed_jobs="$( jq -r 'to_entries[] | select(.value.result == "failure") | .key' <<<"$NEEDS_JSON" | paste -sd, - )" { echo "The ci.yml full-matrix schedule failed on \`main\` at $HEAD_SHA." echo echo "- run: $RUN_URL (schedule)" echo "- failing jobs: ${failed_jobs:-none recorded}" echo echo "This is ci.yml's full 35-job matrix (release/fallback-emitter/rhel/asan/style/windows/windows-package/xplat-diff), not the" echo "nightly.yml ThreadSanitizer run — read this run's own job logs for the failing gate." } > "$body" if ! gh api "repos/$REPO/labels/$label" --silent 2>/dev/null; then gh label create "$label" --repo "$REPO" --color B60205 \ --description "A nightly-scale workflow run failed on main; see the workflow-specific label for which one" fi if ! gh api "repos/$REPO/labels/$ownLabel" --silent 2>/dev/null; then gh label create "$ownLabel" --repo "$REPO" --color 1D76DB \ --description "The full-matrix nightly failed on main; opened and closed by .github/workflows/ci.yml only" fi open="$( gh issue list --repo "$REPO" --label "$label,$ownLabel" --state open --json number --jq '.[0].number // empty' )" if [ -n "$open" ]; then gh issue comment "$open" --repo "$REPO" --body-file "$body" else gh issue create --repo "$REPO" --title "Nightly checks failing on main (full matrix)" --label "$label,$ownLabel" --body-file "$body" fi report-green: name: report-green (close the tracking issue on this workflow's own green schedule run) needs: [plan, style, release, fallback-emitter, rhel, asan, windows, windows-package, xplat-diff] if: success() && github.event_name == 'schedule' && github.ref == 'refs/heads/main' runs-on: ubuntu-24.04 permissions: issues: write steps: - name: Comment "green again" on the open tracking issue and close it env: GH_TOKEN: ${{ github.token }} REPO: ${{ github.repository }} HEAD_SHA: ${{ github.sha }} RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} run: | numbers="$( gh issue list --repo "$REPO" --label nightly-failure,nightly-full-matrix --state open --json number --jq '.[].number' )" if [ -z "$numbers" ]; then echo "no open nightly-full-matrix issue; nothing to close" exit 0 fi for n in $numbers; do gh issue comment "$n" --repo "$REPO" --body "green again at $HEAD_SHA ($RUN_URL, ci.yml full matrix)" gh issue close "$n" --repo "$REPO" done