Cross-build evidence — image/cross-build ======================================== Host: solomon, x86_64-linux, 32 cores, 125 GiB RAM Nix: nix (Nix) 2.35.2 Date: 2026-09-20 nixpkgs: github:NixOS/nixpkgs/nixos-unstable (see flake.lock for the pin) Task 1.1 — the cross toolchain resolves --------------------------------------- $ nix build --no-link --print-out-paths .#checks.x86_64-linux.cross-hello /nix/store/8dlbjqlyh82xc1h02klzrgfbi9pw2sv4-hello-riscv64-unknown-linux-gnu-2.12.3 $ file /nix/store/8dlbjqlyh82xc1h02klzrgfbi9pw2sv4-hello-riscv64-unknown-linux-gnu-2.12.3/bin/hello ELF 64-bit LSB pie executable, UCB RISC-V, RVC, double-float ABI, version 1 (SYSV), dynamically linked, interpreter /nix/store/syg6xw4x296w82267qnvrd3nn0cp6j31-glibc-riscv64-unknown-linux-gnu-2.42-84/lib/ld-linux-riscv64-lp64d.so.1, for GNU/Linux 4.15.0, not stripped Task 1.2 — what a cross-build costs on this host ------------------------------------------------ cross-hello, cold (toolchain fetched from cache.nixos.org): 1m15s wall. Almost all of that was substituting the riscv64 cross toolchain, which is itself an x86_64 package and therefore IS cached. What is not cached is anything built FOR riscv64. Warm rebuild (everything substituted or already built): 19s wall. Derivations that would not cross-compile ----------------------------------------- (filled in by task 2.3) Cache coverage — measured, and it corrects an assumption --------------------------------------------------------- The proposal assumed riscv64 has no binary cache at all. That is true of NATIVE riscv64-linux builds and false of CROSS-compiled ones, because a pkgsCross.riscv64 derivation is built on x86_64 and upstream Hydra builds many of them. Measured on the k230 closure build, 2026-09-20: substituted from cache.nixos.org : 636 paths of which riscv64 outputs : 174 built locally : 164 So the cross path gets substantial cache coverage. Examples of riscv64 paths that were substituted, not compiled: acl-riscv64-unknown-linux-gnu-2.4.0-man attr-riscv64-unknown-linux-gnu-2.6.0-man bash-interactive-riscv64-unknown-linux-gnu-5.3p15-man image/cross-build's cache requirement was rewritten to say this rather than the thing we guessed. Stage-1 seam (task 4.1) ----------------------- $ nix eval --json .#stage1 vendored = true, builtFromSource = false $ grep -rniE 'u-?boot|opensbi' flake.nix nix/ no matches outside explanatory comments Nothing in the flake compiles U-Boot or OpenSBI. Emulation fallback — available in principle, not enabled in practice --------------------------------------------------------------------- Nothing in this build needed emulation, and nothing COULD have been emulated: $ nix config show extra-platforms i686-linux x86_64-v1-linux x86_64-v2-linux x86_64-v3-linux riscv64-linux is absent, so Nix will not build riscv64-linux derivations natively on this host regardless of binfmt. The OS-level binfmt handler IS registered -- the cross-built hello runs directly on the build host -- but that is a property of the OS, not of Nix. So image/cross-build's "emulation remains available as a fallback" is true only after a one-line change: nix.settings.extra-platforms = [ "riscv64-linux" ]; # NixOS extra-platforms = riscv64-linux # /etc/nix/nix.conf That change has NOT been made, because nothing has needed it. Recorded here so the fallback is understood as one config line away rather than as something already in place. A minimal NixOS closure needs TWO cross toolchains --------------------------------------------------- Unexpected, and the largest single cost in the first build. The glibc cross toolchain (riscv64-unknown-linux-gnu-gcc 15.3.0) substitutes from cache.nixos.org in seconds. But NixOS's initrd wants a statically linked busybox, which is built against musl -- and no musl riscv64 toolchain is cached, so the whole bootstrap is compiled locally: riscv64-unknown-linux-musl-nolibc-gcc-15.3.0 musl-static-riscv64-unknown-linux-musl-1.2.6 riscv64-unknown-linux-musl-gcc-15.3.0 riscv64-unknown-linux-musl-binutils-wrapper-2.46 That is a second GCC bootstrap, compiled in parallel with the kernel, and it is why "minimal NixOS" is not a short build here even with good cache coverage on the glibc side. It is a one-time cost per nixpkgs pin -- the results land in the local store and subsequent builds reuse them. Recorded so the first build's duration is not mistaken for the steady-state cost, and so anyone trimming the closure knows the initrd is where the expensive part hides. A note on parallelism: -j 32 was the wrong flag ------------------------------------------------ The closure build was launched with `nix build -j 32`. That is max-jobs -- up to 32 concurrent DERIVATIONS -- and each derivation separately defaults to using every core. Measured during the build: load average : 101 compiler processes : 156 cores : 32 So the machine was roughly 3x oversubscribed. Work still completes and it was left running rather than restarted (45 minutes of GCC bootstrap already invested), but it is not the fast configuration. For a rebuild, prefer letting nix pick, or set both explicitly so the product is bounded: nix build -j 8 --cores 4 # 8 derivations x 4 cores = 32 Recorded because "the build is slow" and "the build is oversubscribed" look identical from outside, and the next person timing this should not inherit a bad number as the baseline. TASK 2.2 — the closure built ----------------------------- $ nix build .#nixosConfigurations.k230.config.system.build.toplevel /nix/store/lzg30bab6kqkgsa54jaql1ixfxgizxv3-nixos-system-nixos-26.11.20260919.20b1ddd EXIT=0 ELAPSED=4413s (73m33s) 2026-09-20 Final tallies for the hardware closure: derivations built locally : 353 paths substituted : ~970 closure size : 1.2 GiB store paths in closure : 474 Remember the timing is from an oversubscribed run (-j 32, load 101). It is an upper bound, not a clean baseline. TASK 2.3 — nothing needed emulation ------------------------------------ The list is empty, and stated explicitly as the task requires: no derivation refused to cross-compile, and none could have been emulated anyway, because nix's extra-platforms does not include riscv64-linux on this host. Every one of the 353 local builds was a cross build or a native host tool. Minimality (system/nixos-config) --------------------------------- Checked against the built closure rather than asserted: NetworkManager 0 paths xserver 0 paths ModemManager 0 paths mesa 0 paths The QEMU variant IS heavier -- netboot-minimal imports installation-device, which brings NetworkManager, ModemManager and a python3 environment. That is the test harness, not the product, and it never reaches the board. It is also why tools/qemu-k230.sh gives the guest 6G: a 1.2 GiB closure has to be resident in a ramdisk there, where on hardware it is an ext4 root on the card.