version: "3" chore_min_version: 0.6.0 # THE DEPENDENCY MODEL THIS FILE OWNS. # # This repo depends on two different KINDS of thing, and they get two # different mechanisms. Getting that split right is the whole point: # # FAMILY REPOS -> siblings, checked out next to this one. # rust-fs-core, rust-fs-ext4, winfsp-fs-skeleton, fs-windows-test-harness. # These release together and are expected to agree on a version at # any given moment. A submodule pins a copy PER CONSUMER, so the # same crate ends up on disk several times at several versions with # nothing reporting the divergence -- each repo being internally # consistent with its own stale pin. A sibling has one copy, so # that drift is not merely detected, it is unrepresentable. # # It had already happened here. Before this file existed, this repo # pinned am-fs-core at v0.1.0 while the rest of the family was on # v0.2.4, and pinned am-fs-ext4 and winfsp-fs-skeleton to # `heads/main` -- floating refs, which the dependency-pinning policy # forbids outright, and which mean the build was not reproducible. # # THIRD-PARTY FORKS -> siblings as well, pinned to a COMMIT. # winfsp-rs is our fork of SnowflakePowered/winfsp-rs on the # fix/guard-strategy-constant-type branch, carrying a patch that is not ours to # release and not on a schedule we control. It used to be a # submodule at vendor/winfsp-rs; it is now checked out at # ../winfsp-rs like everything else, so there are no submodules # left. Because a fork has no release cadence of ours, it is pinned # to a bare commit (WINFSP_RS_REF) rather than a tag, and both CI # workflows patch a header inside that checkout. vars: # The family pins. One place, read by both `chore siblings` and the # CI workflow -- CI parses them out of this file rather than # restating them, so bumping one is still a one-line edit. # URL as well as ref. The family is in one org now, but a URL rebuilt # from a repo name is wrong the moment one moves, and the failure is an # interactive credential prompt on a private-looking URL rather than a # clean 404, so it reads as an auth problem. FS_CORE_URL: https://github.com/antimatter-studios/rust-fs-core.git FS_CORE_REF: v0.3.12 FS_EXT4_URL: https://github.com/antimatter-studios/rust-fs-ext4.git FS_EXT4_REF: v0.8.0 # The matrix's ext4 images are made by rust-fs-ext4's _vm-builder.sh, # which that repository removed at v0.6.0 (#257): v0.5.1 is the last # release that ships it. Pinned apart from the driver so a driver bump # cannot take the builder away again: `chore siblings` checks it out as # ../rust-fs-ext4-image-builder, the checkout every script that starts or # reaches the builder uses (#37). EXT4_IMAGE_BUILDER_REF: v0.5.1 # Pinned to the tag, not to `main`. A floating ref was one of the two # things wrong with the submodule setup this file replaces, and # swapping a floating submodule for a floating sibling would have # fixed nothing. Bump this when the skeleton next releases -- its # `translate` and `reader` modules are what this driver will move to. # v0.2.2: logical partitions inside an MBR extended partition are # listed (#11), and disks already attached when the service or the # watcher starts are mounted without a replug (#23). SKELETON_URL: https://github.com/antimatter-studios/winfsp-fs-skeleton.git SKELETON_REF: v0.2.2 # Pinned to a COMMIT, not a branch. This is a fork of a third-party # crate, so there is no release cadence of ours to track, and upstream # 0.13.0 does not compile for *-pc-windows-gnullvm -- it declares the # guard-strategy constant signed while bindgen emits it unsigned for # that target. WINFSP_RS_URL: https://github.com/antimatter-studios/winfsp-rs.git WINFSP_RS_REF: 0e94163771bcfd3ec1265d3ab35eb2a871136940 HARNESS_URL: https://github.com/antimatter-studios/fs-windows-test-harness.git # v4.2.0: a scenario must pass on its first attempt (no more silent # retries), and one matrix run holds the VM workdir at a time. Both # change what a run means, so a bump is read, not just typed. # v4.3.0: every VM command runs under a bound and a timeout names the # command, and the VM-lock helper no longer streams on remote stdin. # Under v4.2.0 the CI matrix stopped silently after taking the lock and # was killed by the step's 30-minute timeout. v4.3.1: a host-side op's # double quotes reach it intact on a Windows orchestrator. HARNESS_REF: v4.3.1 tasks: matrix:check: desc: Validate fs-windows-test-harness.toml and test-matrix.json against the harness's schemas # The runner accepts unknown scenario keys, so a matrix the schema # rejects still runs; this holds the matrix to the schema instead. # Needs the harness sibling (`chore siblings`) and `jsonschema`. cmds: - python3 scripts/check-matrix-schema.py siblings: desc: Check out the sibling repos this crate builds against, at their pinned refs # REFUSES TO TOUCH A DIRTY SIBLING. A checkout is destructive to # uncommitted work, and a task that runs in CI must not be a task # that eats someone's afternoon when they run it locally. It # reports and fails; the human decides. cmds: - | set -eu # nameurlref printf '%s\t%s\t%s\n' \ rust-fs-core '{{.FS_CORE_URL}}' '{{.FS_CORE_REF}}' \ rust-fs-ext4 '{{.FS_EXT4_URL}}' '{{.FS_EXT4_REF}}' \ rust-fs-ext4-image-builder '{{.FS_EXT4_URL}}' '{{.EXT4_IMAGE_BUILDER_REF}}' \ winfsp-fs-skeleton '{{.SKELETON_URL}}' '{{.SKELETON_REF}}' \ fs-windows-test-harness '{{.HARNESS_URL}}' '{{.HARNESS_REF}}' \ winfsp-rs '{{.WINFSP_RS_URL}}' '{{.WINFSP_RS_REF}}' \ | while IFS="$(printf '\t')" read -r name url ref; do dir="../$name" if [ ! -d "$dir/.git" ]; then echo "siblings: cloning $name at $ref" # init + fetch-by-ref, not clone. One of these refs is a # bare commit, and `git clone --branch` takes a ref NAME # only. A plain clone does not help either: that commit is # reachable from no published branch, so it simply is not # in the result. `git fetch origin ` asks the server # for it directly, which GitHub serves. git init --quiet "$dir" git -C "$dir" remote add origin "$url" git -C "$dir" fetch --quiet --depth 1 origin "$ref" git -C "$dir" checkout --quiet FETCH_HEAD continue fi if [ -n "$(git -C "$dir" status --porcelain)" ]; then echo "siblings: $name has uncommitted changes -- refusing to move it." >&2 echo " Commit or stash them in $dir, then re-run." >&2 exit 1 fi current=$(git -C "$dir" rev-parse HEAD) # Fetch the ref itself rather than everything: this also has # to work for a bare commit that no branch contains, which # `fetch --tags` would never bring down. git -C "$dir" fetch --quiet origin "$ref" wanted=$(git -C "$dir" rev-parse FETCH_HEAD) if [ "$current" = "$wanted" ]; then echo "siblings: $name already at $ref" continue fi echo "siblings: moving $name to $ref" git -C "$dir" checkout --quiet FETCH_HEAD done siblings:check: desc: Report whether the siblings are at their pinned refs, changing nothing cmds: - | set -eu rc=0 printf '%s\t%s\n' \ rust-fs-core '{{.FS_CORE_REF}}' \ rust-fs-ext4 '{{.FS_EXT4_REF}}' \ rust-fs-ext4-image-builder '{{.EXT4_IMAGE_BUILDER_REF}}' \ winfsp-fs-skeleton '{{.SKELETON_REF}}' \ fs-windows-test-harness '{{.HARNESS_REF}}' \ | while IFS="$(printf '\t')" read -r name ref; do dir="../$name" if [ ! -d "$dir/.git" ]; then echo "siblings: $name is not checked out (want $ref)" >&2 rc=1 continue fi have=$(git -C "$dir" describe --tags --always 2>/dev/null || echo unknown) printf ' %-24s %s (want %s)\n' "$name" "$have" "$ref" done exit $rc # THE TIERS. Each runs through ../rust-fs-core/scripts/tier.sh: QUIET BY DEFAULT. The # whole run goes to tmp/logs/.log and a pass prints one verdict # line; a passing run that printed more than the tier's measured budget # fails with status 65. `-- --verbose` (or OUTPUT_BUDGET_VERBOSE=1) # streams the run as well, still under the budget. ci.yml runs the same # lines, so a developer and CI are held to the same numbers. check: desc: Type-check the parts that build off Windows cmds: - bash ../rust-fs-core/scripts/tier.sh --refuse-skips check -- cargo check --locked --no-default-features test: desc: Run the tests that do not need Windows cmds: - bash ../rust-fs-core/scripts/tier.sh --refuse-skips test -- cargo test --locked --no-default-features build:host: desc: Build the host-side CLI the matrix's verifiers run (target/release/ext4) cmds: - cargo build --release --locked --no-default-features --quiet matrix: desc: 'Run test-matrix.json on the Windows VM, quietly: `chore matrix -- [filter] [--verbose]`' # scripts/run-matrix.sh starts the Alpine builder VM, then runs the # harness: ssh to the Windows VM, a PowerShell step per recipe op, a # mount and unmount each. All of it -- every echoed command line, every # verifier's output -- goes to tmp/logs/matrix.log; what is printed is # the verdict, and the scenario count against its floor. # # THE FLOOR IS 40 on an unfiltered run: run-matrix.sh runs the 5-scenario # smoke group as a gate first, then the 35 scenarios that have a recipe # (the other 2 of the 37 are markers, reported as ignored). Both runs # report a `test result:` line and the floor counts both. A filtered run # selects its own count, so it has no floor. deps: [build:host] cmds: - | set -eu filter="" for arg in {{.CLI_ARGS}}; do case "$arg" in --verbose|-v) continue ;; esac filter="$filter $arg" done # shellcheck disable=SC2086 # the words are the point bash ../rust-fs-core/scripts/tier.sh --refuse-skips matrix -- bash scripts/run-matrix.sh $filter if [ -z "$filter" ]; then bash ../rust-fs-core/scripts/test-floor.sh matrix 40 fi