# Changelog All notable changes are documented here. The project follows this SemVer contract: - TUI layout changes are NOT covered (minor). - CLI commands + exit codes are strictly SemVer. - `--format json` output is additive-only (removing/renaming fields → major bump). - Config schema is backwards compatible. - MCP tool registry is append-only. - **Rust library API** — the crate is linked by the proxima desktop UI. The supported surface (`api`, `config`, `app::preflight`, `state::model`) is additive within a minor; everything else is an implementation detail. Enumerated in `src/lib.rs`. ## [Unreleased] ## [0.17.0] - 2026-10-24 Headline: **the verification-discipline milestone, closed, and approved MCP operations get the risk preflight.** What the documentation says about the CLI, the library surface and the architecture is now held by tests, mutation testing and a coverage ratchet guard the approval, audit and incident code, two advisories in the SSH signing path are fixed, and `patch plan` reads the apt inventory of nodes with pending updates again. Two changes reach existing setups: an MCP client that stopped an HA-managed guest now gets a refusal (#507, under Security), and `metric-servers create` and `update` no longer accept `--comment`, which PVE's metric-server schema does not have (#494, under Fixed). ### Security - **Approved MCP operations get the risk preflight the CLI runs, at request and again at execution (#507).** `stop_guest`, `restart_guest`, `delete_guest` and `migrate_guest` ran no risk preflight: an approver saw the action and the target only, and an approved stop of an HA-managed guest was executed (the HA manager then started it again). Now the MCP gate runs `assess_deep` before asking: a SEVERE risk (a lock, HA management on stop or delete, a running guest on delete) refuses the call and nobody is asked, and the other risks are added to the Telegram approval message. When `proxxx hitl serve` executes an approved stop, restart, delete or migrate (from MCP or from a CLI batch command), it checks the guest as it is then and refuses a SEVERE risk. **Migration:** an MCP client that relied on stopping an HA-managed guest gets a refusal; change the HA state first, as `proxxx stop` already required. `ARCHITECTURE.md` now also says that CLI start, stop, restart, suspend and resume do go through Telegram approval when a policy matches (its table said "none"). - **`crypto-bigint` 0.7.5 → 0.7.6 (RUSTSEC-2026-0387).** Several of its operations meant to be constant-time (subtraction, zero counting, division, Montgomery multiplication, `UintRef` equality and `BoxedMontyForm::invert`) could branch on secret values, depending on the compiler and target CPU. It is in the build graph through russh (rsa, and elliptic-curve for the ECDSA keys), the code that signs with the SSH client key. Lockfile-only; `cargo audit` reports it on the old lockfile and passes on the new one. - **`ctutils` 0.4.2 → 0.4.5 (RUSTSEC-2026-0385).** Before 0.4.4 its constant-time mask generation could compile to a branch, and its `u128` comparisons branched on 32-bit targets. It is in the build graph through russh (rsa and elliptic-curve via `crypto-bigint`), the code that signs with the SSH client key. Lockfile-only; `cargo audit` reports it on the old lockfile and passes on the new one. ### Fixed - **`patch plan` reads the apt inventory of a node with upgradable packages (#517).** Its parser mapped the package name to both `Package` and `Title`; PVE sends both in every entry of `/nodes/{node}/apt/update`, so the whole response was refused as a duplicate field, and every node with pending updates was reported under `status.failed` with no packages, no pending kernel and no reboot flag. The name is now read from `Package` only. On the lab (PVE 9.1.1) the plan goes from two failed nodes to 257 upgradable packages each, with a pending kernel and a required reboot. The live `test_run.sh` now fails when a node's inventory is under `status.failed`, since `patch plan` still exits 0 in that case. - **The daemon skips an unmanned converge during a global freeze for a named profile too (#504).** Its converge gate read only the profile's own lock, so with `[profiles.]` a global `incident freeze` was not seen: each tick ran the converge, every write was refused by the mutation gate (which reads both locks), and the failure alert fired. The gate now applies the same rule as the mutation gate, global freeze or the profile's own, with an unreadable lock counting as frozen. Nothing reached the cluster before either; the change removes the failed runs and their alerts. - **Every subcommand has a description in `--help` (#494).** 64 nested ones had none (for example `firewall-cluster alias list`, `acme plugin show`, `node-system dns get`), so their parent's help listed them blank. A test now checks every level. - **Arguments PVE requires are required by the CLI, and a flag PVE does not accept is gone (#494).** Measured on PVE 9.1.1, each call below failed with a 400 "property is missing and it is not optional" unless the argument was given: `node-system dns set` now requires `--search`, `firewall-cluster alias update` and `firewall-guest alias update` require `--cidr`, `cluster-mapping usb update` requires `--map`, and `metric-servers update` requires `--server` and `--port`. `metric-servers create` and `update` no longer offer `--comment`, which PVE's metric-server schema does not have. An invocation that worked before still works. ### Documentation - **The MCP registry checksum examples show what the command prints.** Three pages showed `proxxx mcp tools --checksum` returning `{ "checksum": "8467de772787baa0" }`, 16 hex characters. The command prints a KEY/VALUE table unless `--format json` is given, and the value is the full SHA-256 of the registry, 64 lowercase hex characters. The examples now use `--format json` and `jq -r .checksum`, and the registry test checks that the hash is lowercase hex as well as its length. - **The README risk-gate demo is a recording of the real binary (#511).** The previous `assets/demo.svg` was captioned "every line verbatim from a live Proxmox VE 9.1 cluster", but its typed command `proxxx stop 101 --allow-risk SEVERE` does not parse, the preflight has no risk score and no database-connection check, and a CLI stop whose policy matches does not wait for Telegram: it records the request and exits 9, and `proxxx hitl serve` executes it after approval. Its manifest was never in this repository. The new demo is a terminal recording against the lab cluster: `proxxx delete 7777 --yes` on a running container refused by the pre-flight (exit 6), and `proxxx stop 7777` under an approval policy with no Telegram section, not run (exit 2). `assets/demo-record.sh` records it with asciinema into `assets/demo.cast`, and svg-term-cli renders the SVG; nothing is edited in between. The approval itself is not shown: recording it needs a Telegram bot on the lab. ### Tests and tooling - **CONTRIBUTING states a surface freeze (#295).** No new gateway method group or declarative state family lands until line coverage meets its floor (#298), the family is under its size ceiling (#293) and someone outside the repository asked for it; until those numbers exist the freeze holds. The README non-goals link to it. - **The HITL approval store and the TUI refresh fan-out are tested under real concurrency (#294).** Eight approvers race `process_callback` on one request, through one shared store and through one store each on the same file (as the daemon and the TUI do), for a quorum of 1 and of 2: exactly one call authorises execution. Splitting the read and the update into two transactions makes two calls approve the same request, and the test fails. A second test holds the refresh's 16-request ceiling on a multi-threaded runtime (a ceiling of 17 fails it). CONTRIBUTING now says which code needs such a test. - **Mutation testing runs on the approval, audit and incident modules (#297).** `.cargo/mutants.toml` scopes cargo-mutants to `src/hitl`, `src/audit` and `src/incident` and gives the reason for each mutant it excludes; a weekly workflow (`mutants.yml`, also run by hand) tests them in 8 shards and writes the survivors to the run summary. The first run found controls that no test held, now each covered: `incident freeze` refuses a mutation through the real lock lookup until `thaw`, and a profile freeze refuses only that profile (new `tests/incident_freeze_e2e.rs`); an edited row in the middle of the audit chain blocks the next mutation; `create_guest` refuses a node or guest type that differs from the approved row; a callback id outside `[A-Za-z0-9_-]{1,64}` is rejected, and a callback tag that is not 16 hex characters is answered as unsigned rather than as a failed signature; an unreadable HMAC key is an error and is left in place, while a key of the wrong length is replaced; a CLI action refuses a missing or zero VMID; a row whose retired key is gone counts as a verification failure, an edited legacy v1 row fails verification, and a pre-v2 audit database and a pending store without the `upid` column are migrated. Smaller gaps are closed too: the 7-day retention boundary of approval rows, MarkdownV2 escaping, the `editMessageText` outcomes, the long-poll HTTP timeout margin, the UTC date conversion behind audit timestamps (checked against known dates through 2100), and the operator label, now checked against an independent uid source and for the host it reads from `HOSTNAME`, then `HOST`. - **Rotating the audit key evicts only that log's cached writer (#297).** It cleared the writers of every audit directory in the process; the next write of each reopened its log. - **Documentation is bound to the code (#296).** The `--help` of all 336 commands is snapshotted (`src/snapshots/proxxx__tests__cli_help.snap`), so a renamed flag or a dropped subcommand fails the test. The four modules under the library compatibility promise (`api`, `config`, `app::preflight`, `state::model`) carry doctests that `cargo test` runs; there were none. `ARCHITECTURE.md` is rewritten from the code and checked by `tests/architecture_claims.rs`: every module path it names exists, every mutating request in `src/api` runs the read-only guard, the freeze check and the write-ahead audit in that order, the reducer does no I/O, and the commands in its process table parse. The rewrite corrects claims that no longer held: the delete data flows named functions and actions that do not exist, and the page said every caller runs the same risk preflight and approval gates, which is not the case (the page now has a table of what each caller runs). The secret-handling docs name `SecretString` instead of `Zeroizing`, and the gateway trait no longer cites a `MockGateway` type. CONTRIBUTING states the rule: a documentation claim about behaviour has a test, or it is not written. - **Line coverage is a ratchet (#298).** `.coverage-floor` holds the last measured line coverage (70.87% on main at the time) minus a 0.5pp tolerance, and the CI coverage job fails below it, naming the files whose uncovered lines grew. A pull request that raises coverage by more than the tolerance is asked to raise the floor in the same change; the run's `coverage-lcov` artifact carries the new file. The per-file summary is unchanged. The check is `scripts/coverage-ratchet.py`. ## [0.16.0] - 2026-10-22 Headline: **the 2026-10-18 audit milestone, closed.** Hints, help and table output now say what the commands do, lookups tell a mistyped node from an offline one, the bundled images exist again, and a TUI refresh no longer grows with three requests per node. A few changes reach scripts and setups; they are listed first, each with its migration. ### Breaking (SSH keys, exit codes, output) **SSH** - **An RSA client key under 2048 bits, or any DSA key, is refused before connecting (#485).** **Migration:** create an Ed25519 key (`ssh-keygen -t ed25519`), install it on the nodes and point `key_path` at it. **Exit codes** - **A `--node` that is not a cluster member exits 5 (was 1); an offline member stays 1 with a message that says so (#383).** - **`ha preview --node ` exits 5 (was 0 with empty previews) (#383).** - **`patch changelog --package ` exits 5 (was 0, printing apt's error as the changelog) (#383).** - **`find ` for a guest on an offline node exits 1 (was 5, "not found") (#383).** A teardown script that treats 5 as "already gone" no longer reads a guest that still exists as deleted. - **`metrics` refuses a series the RRD does not have (exit 1, was 0 with all nulls), and `watch` refuses a condition or `--since` it cannot read (#385).** - **`feature --feature` and `cluster-resources --kind` refuse values outside their lists (exit 2) (#385).** PVE refused most of them already. - **Migration:** a script that branches on these exit codes: re-check the cases above. **Output** - **`audit export --export-format csv` prints RFC 4180 CSV, not a JSON object holding it (#383).** **Migration:** read the output as CSV. - **`--format table` prints an object as key/value rows instead of pretty JSON (#385).** **Migration:** use `--format json` for machine-readable output (`version` keeps printing JSON). - **`search` reports `"stopped"` / `"qemu"` instead of `"Stopped"` / `"Qemu"`; `accounting` labels guests without a tag `(none)` instead of `(no-tag)`; `--format json mcp tools` prints the registry object, not a one-element array (#385).** ### Security - **An SSH client key below the NIST minimum through 2030 is refused before proxxx connects (#485).** proxxx used any key the configuration named: a 1024-bit RSA key loaded and signed (measured). It now refuses an RSA key under 2048 bits and any DSA key, naming the file and pointing to `ssh-keygen -t ed25519`. Ed25519, ECDSA and RSA of 2048 bits or more are unchanged. **Breaking** for a setup that uses such a key. **Migration:** create an Ed25519 key (`ssh-keygen -t ed25519`), install its public key on the nodes and point `key_path` at it. - **The operator name in the audit log is looked up with the reentrant `getpwuid_r` (#476).** `getpwuid` returns a pointer into one static buffer shared by every thread, and the lookup runs from the audit hooks inside the multi-threaded runtime, so two concurrent lookups made the copy a data race (undefined behaviour) on the value sealed into the audit actor. The lookup now uses a buffer of its own, grown on `ERANGE`; a test keeps the non-reentrant passwd and group calls out of `src/`. ### Performance - **A TUI refresh costs N+4 requests instead of 3N+3, and leaves offline nodes alone (#493).** Every 5 s cycle listed qemu, lxc and storage on every node, offline ones included; against the default 10 requests/s it saturated around 16 nodes, and since #491 each offline node also cost a `/nodes` lookup per failed request (measured on the lab, 3 nodes with 1 offline: 15 requests per cycle). Guests now come from one `/cluster/resources` call and storage from the online nodes only (6 per cycle on the same lab; saturation moves to about 46 nodes). Guest status in the dashboard can trail the node by a few seconds (measured 3-5 s, pvestatd's cycle); the checks before a mutation still read the node itself. Storage stays per node because `/cluster/resources` has no exact `avail` (it overstated free space on a `dir` storage by its 5% root reserve). If `/cluster/resources` fails, the nodes are walked as before. An offline node keeps its last-known guests, marked stale; on a first load its guests are shown as PVE reports them. ### Fixed - **`iso list` / `iso download` and `cloud-img download` point at images that exist (#492).** On 2026-10-22, 4 of the 6 bundled ISO-library URLs answered 404 (Ubuntu 22.04 and 24.04, Fedora 41, Rocky 9; measured), because mirrors delete old builds and the weekly re-pin covered only `cloudimg.rs`. Both registries are re-pinned from the official manifests (two of them checked by downloading the image and hashing it). Ubuntu now comes from `releases//release-YYYYMMDD/`, which Ubuntu keeps since GA, and the end-of-life Fedora 41 from the permanent Fedora archive. `scripts/repin-cloudimg.py` now re-pins `iso_library.rs` too, refreshes the ISO view snapshot in the same bump, and the weekly workflow fails when a pinned URL stops answering (`--check-urls`). This supersedes the pending bot PR #244. - **A node name outside the cluster is "not found" (exit 5), and an offline member says it is offline (#383).** PVE answers a request for a node it cannot route with the same HTTP 500 "hostname lookup failed" for a typo and for an offline member (measured on PVE 9.1.1), and proxxx passed it on with exit 1. proxxx now asks `/nodes` which one it is. `ha preview --node ` refuses a name that is not a member instead of printing a preview in which nothing is affected (it exited 0). **Breaking** for a script that tests the exit code: a mistyped `--node` now exits 5. - **`find`, MCP and the HITL daemon no longer report a guest on an offline node as missing (#383).** The lookup walked online nodes only, so such a guest came back "not found" and `find ` exited 5, the code a teardown script reads as "already gone". It is now an error naming the offline node (exit 1), as `docs/reference/exit-codes.md` already stated. - **`patch changelog --package ` fails with exit 5 (#383).** PVE returns apt's error text with HTTP 200 and a trailing `RC: 100`; proxxx printed it as the changelog and exited 0. Any other non-zero `RC` is an error with apt's message. - **`audit export --export-format csv` prints CSV (#383).** It printed a JSON object holding the CSV text, with fields unquoted, so a comma in a result message shifted the later columns. Fields are now quoted per RFC 4180. **Breaking** for a consumer that parsed the JSON wrapper: read the output as CSV. - **`ct interfaces` on a stopped container returns an empty list (#383).** PVE answers `{"data": null}` (measured), which proxxx reported as schema drift. - **`gpu-inspect` reports the IOMMU as enabled only when IOMMU groups exist (#383).** Any dmesg line mentioning the IOMMU counted, including "DMAR: IOMMU disabled". The PCI class name keeps its first word ("VGA compatible controller", not "compatible controller"), and the vendor:device pair is read from the last hex `[xxxx:yyyy]` bracket even when the device name has brackets. - **`play-cast` accepts asciicast v2 files without `title` or `timestamp` (#383).** Both are optional in the format. - **`events stream` no longer grows without bound (#383).** The map of seen tasks kept every task it had ever seen; it now holds the tasks of the latest poll. - **`ls --all-profiles` and `find ` report a panicked profile query under its profile name (#383),** instead of under a generic "unknown" label. - **TUI queue (#383):** ops queued in the same microsecond no longer share an id; an op that was running when proxxx exited comes back as "interrupted, outcome unknown" with a pointer to the task log, instead of staying Running forever (it was never re-dispatched, and still is not); a queued op finds its guest through `/cluster/resources` instead of walking every node, and reports a failed lookup as itself instead of "Node not found"; saving the queue no longer holds key handling while SQLite is busy. - **`state export` help no longer claims byte-stable output (#383).** Two exports of an unchanged cluster differ in `meta.exported_at`; the help, README and docs now say so. - **Hints and `explain` no longer point at commands that do not exist (#385).** `explain unauthorized` recommended `proxxx auth login` / `auth show`, `explain parse` recommended `proxxx api GET`, `explain preflight-refusal` recommended `proxxx preflight`, and the parse-error hint suggested `proxxx -vvv`; none of them exist. They now point at `proxxx init --interactive`, `proxxx doctor`, `pvesh get` on a node and `RUST_LOG=proxxx=debug`. A test checks every cited `proxxx` command against the real command tree. - **The 403 hint distinguishes a root-only endpoint (#385).** PVE reserves some endpoints, such as `/cluster/backup-info` (`backup-jobs info`), to the root@pam user itself and refuses every API token, root@pam's included (measured on PVE 9.1.1); the hint said to fix the ACL. The generic 403 hint now gives the full `proxxx perms --node ` form. - **PVE "does not exist" errors get a not-found hint (#385),** and the generic hint for other PVE errors no longer refers to an `Other { status }` field that is never shown. - **`qga` on a container points at commands that work on one (#385):** `ct interfaces`, `serial --node` and `ssh`, not `exec`, which is QEMU only too. - **Help text matches the commands (#385).** `init` has its description back and `profiles --help` no longer opens with the `init` text; `node-system syslog` says it pages from the oldest line and points at `journal --lastentries` for the newest; third-level usage lines carry their ancestors' positionals (`firewall-guest alias list`). - **`feature --feature` takes `snapshot`, `clone` or `copy`, and `cluster-resources --kind` takes `vm`, `storage`, `node`, `sdn`, `pool` or `network` (#385).** Both were free text; PVE 9 refuses `migrate` and `replicate` for features, and `pool` and `network` as a resource filter (measured). `pool` and `network` are now filtered by proxxx. - **`watch` validates its request before waiting (#385).** `-u status=banana` waited out the full timeout; an unsupported target, key or value is now an error at once. `--since` takes s, m, h, d and w (`1d` was read as one hour) and refuses anything else. The help no longer offers a task UPID as a target. - **`search` matches and reports the labels `ls` prints (#385).** `search 0` matched every QEMU VM through the guest type's hidden number; JSON rows now say `"stopped"` / `"qemu"` instead of `"Stopped"` / `"Qemu"`. **Breaking** for a consumer that compared those values. - **`--format table` renders objects as tables (#385).** An object prints as `KEYVALUE` rows plus one table per array of objects (`config`, `incident status`, `ha preview`); a nested object in a row expands into columns, so `find` and `ls --all-profiles` no longer print a JSON blob in a `DATA` column; the columns are the union of the rows' keys. `--format plain` is unchanged: one compact JSON value per line. - **`mcp tools --json` prints the registry as is (#385).** The flag was ignored, so the default printed the table form and `--format json` wrapped the registry in a one-element array. - **`accounting --include-unassigned` labels guests without a tag `(none)`, as it does guests without a pool (#385).** It used `(no-tag)`. **Breaking** for a consumer keyed on `(no-tag)`. - **`metrics` labels a guest by its real type and refuses series that do not exist (#385).** `metrics vm` on a container printed "VM" (PVE serves the RRD under either path); `--field loadavg` or `iowait` on a guest, `diskread`/`diskwrite` on a node and anything but `used`/`total` on a storage returned all nulls with exit 0 and are now errors. `metrics node --field used|total` reads the root filesystem, as its help says, instead of returning nulls. ### Tests and tooling - **The live harness runs again and cannot pass by accident (#479, #480, #489).** The bash scripts build a private config from the curl target and never read or rewrite the user's config; an enabled cargo live suite with an incomplete env fails instead of passing in 0.00 s; the state and reconcile scripts were repaired; `setup_demo.sh` reads its VM from `PROXXX_E2E_DEMO_VMID`. - **The weekly image re-pin covers both registries and fails on a dead URL (#492).** - **CONTRIBUTING: a pull request comes with tests that fail without the change, or says why it has none (#484).** ## [0.15.0] - 2026-10-22 Headline: **the second audit backlog, closed.** This release is mostly security and correctness fixes. Several of them change what a v0.14.0 config, script or library consumer sees; they are listed first, each with its migration. Read them before upgrading an unattended deployment (daemon, cron job, CI gate, MCP server). ### Breaking (config, CLI flags and exit codes, JSON output, library API) **Configurations that no longer load (exit 3)** - **Unknown keys are refused in every TOML section, including `[[policies]]` (#326).** v0.14.0 ignored them (measured: a flat config with `verfy_tls = false` loaded and ran). **Migration:** run any command once; the error names the key. Fix the spelling or remove the key. `[[alerts]]` now accepts `trigger` as an alias of `when`. - **A profile-only config with a top-level section that no profile reads is refused (#440).** With `[profiles.]` tables and no top-level `url`, a top-level `[[alerts]]`, `[telegram]`, `[ssh]`, `[pbs]` or `[[policies]]` used to be dropped silently (measured: a top-level `[[alerts]]` next to `[profiles.lab]` loaded). **Migration:** move the section to `[defaults.*]` (shared) or `[profiles..*]`. Configs that also have a top-level `url` are unchanged. - **`ssh.strict_host_key_checking` is validated, and `"yes"` now means strict (#375).** Accepted: `tofu`, `strict`, `off`, and the OpenSSH spellings `accept-new` (tofu), `yes` (strict), `no` (off). Any other value used to mean TOFU (measured: `"ask"` loaded); it is now refused. A config with `"yes"` used to trust a new node's key on first use and now refuses a node that is not already in `known_hosts`. The `/tmp/proxxx` fallback for `known_hosts` is gone. **Migration:** write `tofu` if you relied on trust on first use; set an explicit `known_hosts` path where the per-user config directory cannot be resolved. - **`pbs.fingerprint` is enforced (#414).** v0.14.0 parsed and ignored it (measured: `fingerprint = "not-a-fingerprint"` loaded). A malformed value is now refused at load; a well-formed but stale value fails the TLS handshake, and the pin takes precedence over `verify_tls = false`. **Migration:** set it to the current SHA-256 fingerprint of the PBS certificate (`proxmox-backup-manager cert info`), or remove the key. **Mutations that are now refused** - **Every PVE mutation and SSH `patch apply` needs a writable, verifiable audit log (#344).** Before any `POST`, `PUT` or `DELETE` leaves the process, proxxx verifies the HMAC chain in `audit.db` and writes a write-ahead row; if the database or key cannot be opened, verified or written, the mutation is refused with exit 8. This applies to the CLI, the TUI, `daemon serve` and the MCP server. An existing chain that v0.14.0 broke with concurrent writers (#368) refuses every mutation. **Migration:** before upgrading, run `proxxx audit verify` as each user that runs proxxx (CLI, daemon, MCP); make sure the audit directory is writable for that user (`PROXXX_AUDIT_DIR`, `PROXXX_AUDIT_KEY`); if verify reports tampered or unverifiable rows, archive `audit.db` and `audit.key` before the upgrade so a new chain starts. - **`--secure` is enforced on the CLI (#338, #399).** v0.14.0 accepted `--secure` on the CLI and ignored it, so `proxxx --secure delete 100 --yes` ran. Every destructive CLI command (the 64 actions in `hitl::DESTRUCTIVE_ACTIONS`: stop, delete, migrate, `node stopall`, `snapshot delete`, `pool delete`, `state apply`, `reconcile converge`, `patch apply`, `pbs restore`, `ssh --cmd`, and the others listed in the Security section) now exits 8 under `--secure` before any API call, and `schedule run-due` passes `--secure` to the jobs it starts. **Migration:** drop `--secure` from scripts and schedules that must run destructive commands unattended; keep it where refusal is the point. - **`node stopall` and `node suspendall` require `--yes` (#399).** Without it they exit 1 before any request. **Migration:** add `--yes`. - **`--token-secret` / `--token-secret-file` are refused by fan-out commands (#374).** `ls --all-profiles`, `find` and `fleet` exit 3 when the flag is given (measured: v0.14.0 accepted it and sent the secret to every profile). **Migration:** give each profile its own secret (`token_secret_file`, environment variable or keychain) and drop the flag from fan-out calls. - **`state apply`, `reconcile converge` and the daemon's `auto_converge` refuse a state file whose `[meta] profile` differs from the resolved profile (#346).** Exit 3; `state diff`, `reconcile run` and `reconcile watch` only warn. v0.14.0 wrote `profile = "default"` on every export made without `--profile`, also when a named profile was selected through `default = ""` or a single `[profiles.]`, so those files now mismatch. The daemon never overrides the check. **Migration:** set `[meta] profile` to the profile name (or re-export with v0.15.0); on the CLI, `--allow-profile-mismatch` overrides. - **A state file that declares no resource family is refused by `state diff`, `state apply` and `reconcile` (#371).** Exit 1 for an empty, comment-only or `[meta]`-only file, which used to report "no changes" with exit 0. **Migration:** declare the families the file manages, e.g. `pools = []`. `state export` now writes empty families that way. - **`state apply --dry-run` and `reconcile converge --dry-run` run the pre-flight gate (#369).** A rehearsal can now exit 6 where it exited 0. **Migration:** treat exit 6 from a dry-run as "the real run would be refused". Without `--prune`, skipped deletes no longer trip the gate. - **A storage update refuses a change to an immutable field (#370).** A declared change to `type`, `path`, `export`, `datastore`, `vgname` or `thinpool` used to be reported as Applied without effect, on every tick; it is now an error before any request. **Migration:** align the declared value with the cluster, or recreate the storage. - **`mcp serve-http` checks `Host` and `Origin` on every route (#342).** On a non-loopback bind only `:` and each `--allowed-host` value are accepted, so a `0.0.0.0` bind answers 403 to clients that use the machine's name or address; on a loopback bind only `127.0.0.1`, `localhost` and `[::1]` are accepted, which refuses a reverse proxy that forwards the public `Host`. A cross-origin `Origin` is refused. **Migration:** add `--allowed-host ` for every name clients use. - **HITL approval requests expire after 120 s, and keyboards minted before the upgrade are refused (#341).** v0.14.0 kept no expiry. Requests now live in `/hitl_pending.db`, shared by the sender and the daemon. **Migration:** approve within two minutes, or request again; re-request anything pending at upgrade time. **Exit codes** - **A batch operation waiting for approval exits 9, not 3 (#360).** When the approval request cannot be sent (no `[telegram]`, or Telegram refused it), the guest is reported as an error and the batch exits 2. **Migration:** treat 9 as "pending approval"; 3 now only means a configuration error. - **Configuration errors exit 3, not 1 (#359):** unknown `--profile`, no default profile, missing required fields, invalid enum-like values, `proxxx pbs ...` without a PBS block (measured: an unknown profile exited 1 on v0.14.0). **Migration:** treat 3 as a configuration error. - **A VMID absent from the cluster exits 5, not 1 (#358)**, for guest-targeted commands, `metrics vm|ct`, `watch -t vm-`, `snapshot` and `console vnc`. An unreachable node still exits 1 (or 7). **Migration:** treat 5 as "guest does not exist". - **`ls --all-profiles` exits 1 when any profile fails, and `find ` exits 1 when a profile failed (#322).** Both exited 0 (measured with one unreachable profile). `find` exits 5 only when every profile answered and none owns the VMID. **Migration:** treat 1 as partial output; the rows are still printed. - **`upgrade-check` no longer passes when it could not check (#355).** A node that cannot be read gives `indeterminate` and exit 1, a target with no checks (e.g. `banana`, `10`) is a usage error (exit 2), and every node's version is checked, so a node behind the target is a block finding. **Migration:** use `--target 9`; treat 1 as "not proven ready". - **`heatmap` exits 1 when a node does not answer (#353)**, and each row now times the node itself. v0.14.0 always exited 0. - **`anomaly` exits 1 when a node is skipped (offline or too little RRD history), and judges each node against its own history (#356).** It used to exit 1 only on an anomaly, computed across the nodes of one snapshot. New `--min-cpu-delta` (10) and `--min-mem-delta` (5) defaults gate what counts. **Migration:** read `skipped` in the JSON before alerting on exit 1. - **`logs tail` exits 1 when a node fails, and reports the failure on stderr instead of as a log record (#364).** The `[ssh error]` records no longer appear on stdout (JSON or text). **Migration:** check the exit code and stderr instead of parsing error records. - **`perms ` exits 5 for a user that does not exist (#347)**, instead of printing `paths: []` with exit 0. **Changed defaults and behaviour** - **`logs tail --no-follow` without `--since` prints the last 200 lines per node (#392).** It used to print the whole journal. **Migration:** pass `--since` or the new `--lines N` to choose the window. - **Mutating requests are not retried after a timeout, a `502`, a `504` or a `503` without `Retry-After` (#348).** They used to be retried up to 4 times, which could run the change twice. The command now fails (exit 1 for a timeout, 7 for the status codes) and says PVE may have run it. **Migration:** after such a failure, check `proxxx tasks` or the PVE task log before retrying. - **The global `--format json` now applies to commands with their own output flag (#362):** `explain`, `describe`, `cloud-img list`, `schedule list`, `backup-verify`, `heatmap`, `anomaly`, `upgrade-check`, `accounting`, `import`, `gpu-inspect`, `state export|diff|apply`, `reconcile run|converge`, `mcp tools`. They printed text. **Migration:** a wrapper that always passes `--format json` and wants text must pass the command's own flag (`--output`, `--print`, `--json`), which still wins. - **The default data directory is narrowed to `0700` on every start, and the log directory is created `0700` (#384).** It is also the default audit directory (on macOS the same directory holds the config). A log shipper running as another user loses read access, and proxxx fails to start if it cannot narrow a directory it does not own. **Migration:** run log readers as the same user, or point `PROXXX_LOG_DIR` at a directory you manage (a directory set there is created `0700` but not narrowed). - **Node-level SSH reaches the API node through the profile `url` host when `ssh.hosts` has no entry for it (#328).** Host keys are recorded per connection host, so under `strict` a `known_hosts` entry made for the node name does not cover the URL host. **Migration:** add `hosts = { = ":22" }` under `[ssh]` to keep the previous target, or record the URL host's key. **CLI flags** - **`node-system journal`: `--service` is removed and `--since` / `--until` take UNIX epoch seconds (#351).** PVE's journal endpoint has no unit filter (use `syslog --service`) and accepts only epoch seconds. In v0.14.0 every call failed on PVE 9.1.1 (gzip body). - **`watch -t storage-` refuses a node-local storage present on several nodes (#352).** It used to watch whichever node's row came first. **Migration:** use `storage-@`. **JSON output** - **`ha status` has the shape PVE returns (#350):** `manager_status` (`master_node`, `node_status`, `service_status`), `lrm_status` per node and `quorum` replace the top-level `master`, `mode` and `node_status`, which v0.14.0 always printed empty. A response without `manager_status` is now a parse error. - **Option `get` commands omit keys PVE did not return (#349):** `firewall-cluster options get`, `firewall-guest options get` and `cluster-config get` used to print unset keys as `false`, `0` or `""`. `registered-tags` and `tag-style` are passed through as PVE sends them, `allowed-tags` and `policy_forward` are new, and `acme plugin list` reads the real `validation-delay`. **Migration:** treat a missing key as "PVE default" (for example `jq '.ebtables // true'`). - **`backup-verify --output json` (#357):** a `missing` row no longer carries `age_days` (it was `0.0`), the new status `unknown` has no `age_days`, and a node that could not be read gives one `unknown` row without `vmid`. - **Fatal errors under `--format json-envelope` are wrapped in the envelope (#361):** `{"proxxx": "", "data": [ ... ]}` instead of a bare array (measured on v0.14.0). Under both JSON formats `error` now carries the whole cause chain, and `causes` and `exit_code` are added. **Rust library API (`api`, `config`, `state::model`)** - **Type and signature changes for library consumers:** `ClusterOptions`, `FirewallOptions`, `GuestFirewallOptions` and `state::model::FirewallOptionsDecl` fields become `Option<_>`; `AcmePlugin::validation_delay` is `Option`; `HaManagerStatus` is reshaped (see `ha status`); `StoragePool` gains a public `node` field; `ProxmoxGateway` gains the required methods `get_node_version` and `get_user`; `api::tls_pin::pinned_cert_path` and `load_pinned_cert` are removed; `ApiError::actionable_hint` is no longer `const`; `SshConfig::resolve_host` takes the API URL and local node and returns `ResolvedNodeSsh`; `SshConfig::known_hosts_path` returns `Result`; `ConfigError` gains variants (`UnknownProfile`, `NoDefaultProfile`, `ProfileParse`, `InvalidValue`, `MissingPbsConfig`, `ProfileMismatch`, `DefaultsParse`, ...); `PbsConfig` and `TelegramConfig` gain `profile_name`. **Migration:** rebuild proxima against v0.15.0 and adjust; implementors of `ProxmoxGateway` must add the two methods. ### Security - **Six smaller security fixes (#384).** `mcp serve-http` takes its bearer token from `--token-file` or `PROXXX_MCP_TOKEN`, so it no longer has to sit in the process listing; the file wins over `--token`, which wins over the variable, which wins over `mcp_token`. An open SSE stream re-checks its token before every event and every 15 s and ends once a `SIGHUP` reload has rotated it out; it used to keep receiving events. An MCP integer parameter outside 0 to 4294967295 is refused; it was narrowed with `as u32`, so `guest_id` 4294967396 acted on VMID 100 and HITL policies matched the wrapped id. A PBS datastore name is percent-encoded as one path segment, and PBS requests pass the same relative-segment check as PVE ones; `../` in a name used to reach other API paths. A webhook URL is logged and reported as scheme, host and port only, since hosted webhooks keep their secret in the path; it was logged at info on every alert. `audit.db` is created `0600` (an older, wider one is narrowed on open), and the log directory is created `0700`, with the default data directory narrowed on start: it is also the default audit directory, and `main` created it `0755` before the audit layer could create it `0700`. The panic message names the real log file instead of a fixed `~/.local/share/proxxx (proxxx.log)`. - **Release builds are pinned, checked and signed end to end (#384).** The aarch64 musl build pins the cross image by digest in a new `Cross.toml` (the `0.2.5` tag alone is mutable, and the image holds the linker that builds the released binary). A tag whose version differs from `Cargo.toml` fails before any build, so tarballs named `proxxx-X` can no longer report version Y. The CycloneDX SBOM is signed with the same keyless cosign bundle as the tarballs. `--version` and the startup log line name the commit a release was built from (`proxxx X.Y.Z (commit <12 hex>)`), and the release job checks it on the targets it can run. The RUSTSEC-2023-0071 rationale in `.cargo/audit.toml`, `deny.toml` and `THREAT_MODEL.md` names the current path (russh 0.64.1 to rsa 0.10.0-rc.18, directly and through ssh-key 0.7.0-rc.11) instead of rc.16 through a forked crate. - **`--format table` and `--format plain` no longer pass terminal control sequences from API strings (#365).** Table, the default CLI format, printed every string as received, so a value such as a PBS snapshot comment or a user comment synced from LDAP could carry escape sequences that move the cursor, retitle the terminal or write the clipboard (OSC 52). Every cell and header now has its control characters removed, and a tab, newline or carriage return inside a value is printed as `\t`, `\n` or `\r` so it cannot split the row a script reads. Plain output and the pretty JSON that table prints for a single object drop DEL and C1 controls (JSON already escapes the rest). The shared display filter, also used by the TUI, now removes the C1 controls U+0080 to U+009F as well: some terminals read U+009B as CSI and U+009D as OSC. Output with no control characters is unchanged. - **A tag publishes nothing unless the tagged commit passes CI, and every build is pinned to `Cargo.lock` (#381).** `release.yml` assumed the tagged commit had already passed CI, so a tag on a red commit, or on one with a fresh advisory, still produced signed artefacts and a Homebrew bump. Its first job now calls `ci.yml` (tests, clippy, fmt, cargo audit, cargo deny, secret scan, musl build) on the tagged commit, and the build matrix needs it. No cargo command passed `--locked`, so a stale lock was re-resolved on the runner and the shipped binary could contain crates that audit, deny and the SBOM never saw; every `cargo build`, `test`, `clippy`, `llvm-cov` and `cross build` in the workflows now uses `--locked`. `tests/release_gates.rs` fails if the gate or a `--locked` is dropped. - **The CLI token secret is no longer sent to other clusters or to PBS (#374).** `--token-secret` / `--token-secret-file` hold one profile's secret, but `ls --all-profiles`, `find` and `fleet` passed it to every profile, the TUI kept it after an in-app profile switch, and the `delete` pre-flight passed it to the PBS client. Fan-out commands now refuse the flag (exit 3); after a TUI profile switch the next profile resolves its own secret; the PBS client in the `delete` pre-flight resolves its own secret (`proxxx pbs ...` commands still use the flag). When PBS is configured but its client cannot be built, the pre-flight reports the PBS backup check as skipped instead of silently leaving PBS out. The docs no longer mention a `--pbs-token-secret` flag that does not exist. - **Private files are never written with wider permissions first (#376).** The HMAC key, `config.toml` from `init` and the wizard, `schedules.toml`, the freeze lock and console recordings were written with the default umask and only then chmod-ed to 0600, so another local user watching the directory could open the file in between and keep the descriptor. All of them, plus token secret files and TLS pins, now go through one helper that creates the parent directory 0700 if missing, writes a temp file that is 0600 from the first byte, fsyncs it and renames it durably; TLS pins and console recordings were also not fsync-ed before. A guard test fails if a write-then-chmod comes back. - **An unrecognised `ssh.strict_host_key_checking` value no longer degrades to TOFU (#375).** Any value other than `tofu`, `strict` or `off` used to mean "trust on first use", so the OpenSSH spelling `"yes"` trusted the first key presented and a man-in-the-middle on a new or reinstalled node was pinned. The value is now validated at config load (exit 3 with the accepted list); the OpenSSH spellings are accepted as aliases (`accept-new` = tofu, `yes` = strict, `no` = off), and an unknown value seen at runtime fails safe to strict. The `known_hosts` file no longer falls back to the shared `/tmp/proxxx` when the per-user config directory cannot be resolved: proxxx asks for an explicit `known_hosts` path instead, and the TUI checks host keys strictly. - **Exhaustive CLI `--secure` destructive-command classification, `--secure` forwarding in `schedule run-due`, and `--yes` confirmation on `node stopall` / `node suspendall` (#399).** `src/cli/mod.rs::cli_hitl_action` previously matched only 9 top-level variants (`start`, `stop`, `restart`, `delete`, `migrate`, `exec`, `disk move`, `disk resize`, `snapshot create`) and fell back to `_ => None`, `src/hitl/mod.rs::DESTRUCTIVE_ACTIONS` listed only the 7 TUI `SideEffect` action names (`stop`, `delete`, `restart`, `migrate`, `exec`, `move_disk`, `resize_disk`), and `src/cli/schedule.rs::run_due` spawned scheduled child commands without forwarding `--secure`. As a result, destructive CLI commands outside those 7 — including `node stopall`, `node suspendall`, `snapshot delete`, `template`, `task-stop`, `backup --mode stop|suspend`, `patch apply`, `state apply`, `reconcile converge`, `storage delete`, `storage-defs update` / `delete`, `pool delete`, `pool remove-members`, `access user-update` / `user-delete` / `group-delete` / `acl-unset` / `password`, `token revoke` / `token bootstrap --rotate`, `firewall-cluster` / `firewall-guest` deletions and policy/alias/ipset updates (`options set`, `alias update`, `ipset add-cidr`), `cluster-mapping` deletions, `cluster-bootstrap` node/qdevice changes (`nodes add` / `remove`, `qdevice setup` / `update` / `delete`, `join join`), `node-system` cert/subscription deletion, `notifications` / `metric-servers` / `backup-jobs` / `acme` / `ha` / `schedule` deletions, `vm raw-set`, `vm sendkey`, `vm unlink`, `ct raw-set`, `qga write`, `pbs restore`, and non-interactive `ssh --cmd` — bypassed the `--secure` gate, as did any destructive command invoked via `proxxx --secure schedule run-due`. `DESTRUCTIVE_ACTIONS` now documents the canonical classification rule (an action is destructive if it is irreversible, removes data or a resource, or interrupts availability) and lists all 64 destructive action names, `cli_hitl_action` matches exhaustively on `Command` and every mutating nested subcommand enum without any `_ =>` wildcard (so adding a new CLI subcommand fails to compile until classified), `schedule::run_due` forwards `--secure` to every spawned child process when the parent runs under `--secure`, and `node stopall` and `node suspendall` (`src/cli/node.rs`) now require `--yes` via `cli::common::require_yes` before invoking the gateway. - **TLS TOFU (`tls_pin_mode = "tofu"`) and PBS `pbs.fingerprint` now enforce true SHA-256 leaf-certificate pinning via a custom `rustls` `PinnedLeafVerifier` across REST, WebSocket serial console (`wsterm`), `PbsClient`, and `doctor` probes (#320, #414).** `PxClient` previously loaded `/known_certs/_.der` via `reqwest::Certificate::from_der` and passed it to `ClientBuilder::add_root_certificate`, which WebPKI treats as a CA trust anchor (failing with `InvalidCertValidity` / `UnknownIssuer` / `NotValidForName` against Proxmox VE's default `pve-root-ca.pem` → `pve-ssl.pem` non-CA leaf or IP URLs), while `PbsConfig.fingerprint` (`#414`) was deserialized and silently ignored by `PbsClient::new`. `src/api/tls_pin.rs` now provides `PinnedLeafVerifier` implementing `rustls::client::danger::ServerCertVerifier`: `verify_server_cert` compares `SHA-256(end_entity DER)` against the pinned 32-byte digest in constant time (`subtle::ConstantTimeEq`), and `verify_tls12_signature` / `verify_tls13_signature` verify the handshake `CertificateVerify` signature via `rustls::crypto::verify_tls12_signature` / `verify_tls13_signature` backed by `ring`'s `signature_verification_algorithms`. On mismatch, the handshake fails immediately without retry (`src/api/transport.rs::is_retryable_error` now excludes TLS/certificate errors) and reports both pinned and presented SHA-256 fingerprints, the `/known_certs/_.der` path, and instructions to verify out of band before deleting the file to re-trust. `PxClient`, `wsterm::connect` (`src/cli/console.rs`), `PbsClient::new`, and `doctor`'s anonymous version probes all honour the pinned verifier via `use_preconfigured_tls`, and malformed `pbs.fingerprint` values are rejected at config load (`ConfigError::InvalidValue`, exit code 3). - **`proxxx doctor` no longer prints the plaintext Telegram bot token on transport or proxy failures (#410).** `src/cli/doctor.rs` previously constructed `https://api.telegram.org/bot{token}/getMe` directly and formatted any `reqwest::Error` with `{e}`, whose `Display` implementation appends `for url (https://api.telegram.org/bot/getMe)`. `doctor` now routes its Telegram check through `TelegramGateway::new_for_probe` and `TelegramGateway::get_me`, which strip the request URL via `reqwest::Error::without_url()` and redact both the configured token and secret patterns via `panic_hook::scrub_secrets`, ensuring no `doctor` stdout or stderr output (JSON or table) contains the bot token on any path. - **The MCP HTTP server (`proxxx mcp serve-http`) now enforces fail-closed `Host` and `Origin` validation on every route before Bearer auth (#342) and keeps `--token` across `SIGHUP` config reloads (#343).** `src/mcp/http_server.rs` previously accepted any `Host` and `Origin` header on `POST /mcp`, `GET /mcp` (SSE), and `GET /health`, leaving loopback listeners open to browser DNS-rebinding attacks, and `src/cli/mod.rs` wrote the `--token` CLI argument directly into the reloadable `ProfileConfig` (`cfg.mcp_token = Some(...)`) so a `SIGHUP` config reload from a file without `mcp_token` overwrote the CLI token (dropping authentication on loopback or locking out all callers with `401` on an exposed bind). A router-wide `require_allowed_host_and_origin` layer now runs before `require_authorized` on every route: missing or empty `Host` headers return `400 Bad Request`; `Host` must match one of the allowed authorities (`127.0.0.1:`, `localhost:`, and `[::1]:` on loopback binds, or `:` plus each repeatable `--allowed-host ` entry on exposed binds) or is refused with `403 Forbidden` naming the rejected host; if an `Origin` header is present it must be `http://` or `https://` followed by an allowed authority or is refused with `403 Forbidden` (requests without `Origin` remain allowed and no CORS headers are emitted). For `#343`, the CLI `--token` is no longer written into `ProfileConfig`: it is stored as `cli_token: Option` in `McpState` at startup and takes precedence over the reloadable `config.mcp_token`, so `SIGHUP` reloads can still rotate `mcp_token` from disk when `--token` was not given but can never clear or replace a CLI-supplied token. - **Every PVE REST mutation and SSH `patch apply` now writes a fail-closed write-ahead `"sent"` entry and a post-call outcome entry to the HMAC-chained `audit.db` (#344).** `src/audit/mod.rs` previously called `AuditLogger::log()` only from `incident freeze`, `incident thaw`, and `state apply`, leaving guest lifecycle operations (`start`, `stop`, `shutdown`, `restart`, `suspend`, `resume`, `delete`, `clone`, `migrate`), snapshots, disk move/resize, firewall changes, user/role/ACL/token mutations, ISO downloads, and SSH `patch apply` unaudited despite documentation claiming every mutation was recorded. `PxClient::post`, `PxClient::put`, `PxClient::delete`, `PxClient::upload_to_storage` (with `PxClient::resize_disk` routed through `PxClient::put`), and `app::patch::Orchestrator::run_upgrade` now pass through a mandatory write-ahead audit gate (`MutationAuditCtx::write_ahead` / `write_ahead_custom` and `record_outcome`) backed by a process-wide lazy `AuditLogger` with a 5-second SQLite `busy_timeout`. Before any mutation leaves the process, the gate verifies the existing HMAC chain and appends a `"sent"` row (`action` = `" "` or `"SSH patch apply"`, `user` = authenticated PVE user or SSH user, `node`/`vmid` parsed from `/nodes/{node}/(qemu|lxc|openvz)/{vmid}/...`, and `params_json` with secret keys matching `password|secret|token|ticket|key|cookie` replaced by `"[REDACTED]"` and non-secret values scrubbed via `panic_hook::scrub_secrets`), then appends a second row with the outcome (`"ok "`, `"error "`, `"ok 0"`, `"error "`, or `"error transport: "`). If opening, verifying, or writing `audit.db` or `audit.key` fails, the mutation is refused before any network call with `AuditWriteRefusal` (exit code 8). - **The HITL daemon now executes all 15 policy-gated CLI and MCP operations from persisted `params_json` with live Guest Binding and strict parameter decoding (#340).** `src/hitl/daemon.rs::execute_approved_action` previously implemented only `stop`, `delete`, `restart`, and `start`, dropped stored parameters such as `force`, `timeout_secs`, `snapname`, `newid`, and `target_node`, re-resolved `(node, guest_type)` from the live cluster by `vmid` alone without checking the approved row, and marked every other policy-gated operation (`shutdown`, `stop_guest`, `restart_guest`, `delete_guest`, `create_snapshot`, `delete_snapshot`, `suspend_guest`, `clone_guest`, `migrate_guest`, `create_guest`, `clone_with_cloudinit`) as `Unknown HITL action`. The daemon now decodes each row into an exhaustive `ApprovedOp` enum across all 5 CLI batch operations and 10 destructive MCP tools (`#[serde(deny_unknown_fields)]` plus `Shape::Name` validation on path-bearing fields), verifies Guest Binding against `get_cluster_resources()` before execution (existing guests must still reside on `(row.node, row.guest_type)` and target VMIDs for `create_guest` / `clone_guest` / `clone_with_cloudinit` must still be free), calls the shared `cli::common::exec_cli_batch_op` and `mcp::dispatch::exec_*` executors with the exact stored parameters, and records the returned UPID or identifier in `hitl_pending_requests.upid`. - **HITL pending approvals are now persisted in `/hitl_pending.db` with 64-bit random request IDs and a 120-second TTL (#341).** CLI and MCP policy intercepts previously constructed a deterministic `{action}:{vmid}` transaction ID while `src/hitl/pending.rs` kept the consumed set only in process memory with no `issued_at` or `expires_at`, allowing an old approval button to replay after a daemon restart and causing a second legitimate request for the same `(action, vmid)` within a single daemon session to be rejected as stale. Senders and the HITL daemon now share `/hitl_pending.db` (SQLite, `0600` in a `0700` parent directory alongside `telegram_hmac.key`, with 7-day retention pruning of terminal rows on open), storing `id` (16 lowercase hex chars from `getrandom`), `action`, `vmid`, `node`, `guest_type` (`qemu`/`lxc`), `params_json`, `require`, `issued_at`, `expires_at` (120 s default TTL), `status` (`pending`, `approved`, `denied`, `expired`, `executed`, `failed`), `approvers_json`, and `error`. Telegram inline keyboards now emit `approve::` (41 bytes) and `deny::` (38 bytes), leaving 23 bytes of headroom under Telegram's 64-byte `callback_data` cap regardless of action length, and legacy 3/4-segment buttons are refused with a prompt to request again. - **CLI `--secure` now refuses destructive commands with exit code 8 instead of silently ignoring the flag (CLI-surface behaviour change, #338).** `src/cli/mod.rs::execute` previously accepted `--secure` as an unused `_secure: bool` parameter while `--help` and `docs/integrations/hitl.md` advertised `proxxx --secure delete 100 --yes` as requiring Telegram 2FA, causing destructive CLI commands under `--secure` to execute immediately without human approval. Both the TUI and the CLI now consult the shared `hitl::DESTRUCTIVE_ACTIONS` set (`stop`, `delete`, `restart`, `migrate`, `exec`, `move_disk`, `resize_disk`), and `cli::enforce_cli_secure_mode` refuses any destructive CLI command before any API call with `hitl::SecureModeRefusal` (exit code 8, in the same local-lock refusal family as `read_only` and `incident freeze`), while non-destructive CLI commands continue to run normally. - **All mutating TUI side effects and queued batch operations now pass through the HITL gate (#339).** `src/tui/mod.rs` previously called `check_hitl` only inside the `StartGuest`, `StopGuest`, and `RestartGuest` arms of `dispatch_side_effect`, leaving `DeleteGuest`, `MigrateGuest`, `ExecuteGuestCommand`, `CreateSnapshot`, `DownloadIso`, `MoveDisk`, `ResizeDisk`, and `ExecuteQueue` ungated even when `--secure` was enabled or a matching `[[policies]]` rule was configured. `tui::hitl_action` now exhaustively classifies every `SideEffect` variant with no wildcard arm so new variants fail to compile until classified, a single `check_hitl` gate runs before the `match effect` dispatch for all mutating variants (`start`, `stop`, `restart`, `delete`, `snapshot`, `migrate`, `exec`, `download_iso`, `move_disk`, `resize_disk`), and `ExecuteQueue` gates each queued operation individually while preserving write-ahead persistence and immediate dispatch for non-gated operations. - **Unified HITL Telegram callback verification and quorum enforcement across the daemon and TUI (#336, #337).** `TelegramGateway::request_approval` mints approve callbacks with a trailing `:r` quorum segment (`approve::r:`), while `src/tui/mod.rs` previously split only at the first `:` after stripping the HMAC tag, leaving `:r` attached to `txn_id` so every TUI approval looked up a non-existent pending key and timed out (#336). In addition, the TUI poller verified only the HMAC tag and skipped the originating `chat.id` check, the `allowed_approvers` allowlist, and `require > 1` quorum counting (#337). Both the daemon and the TUI now validate callbacks through `hitl::daemon::verify_callback` and `parse_callback_payload` (HMAC tag, `chat.id`, fail-closed `allowed_approvers`, and `:[:r]` with backward compatibility for 3-segment keyboards), enforce `require` distinct approver votes via `PendingApprovals`, and answer the Telegram callback with `Approved` or `Denied` only after a matching pending entry is resolved (or `Expired or unknown request` when none matches). - **The panic hook scrubs `PVEAPIToken=` headers before writing to stderr and the log file (#377).** `src/util/panic_hook.rs` redacted `PVEAuthCookie=`, `PBSAPIToken=`, `tokenid=`, `password=`, `token_secret=`, `csrfpreventiontoken=`, `vncticket=`, and `Authorization: Bearer `, but omitted `PVEAPIToken=`, the `Authorization` header format sent on PVE API requests (`PVEAPIToken=!=`). Adding `"PVEAPIToken="` to `KEY_VALUE_MARKERS` masks the entire `!=` value run with `[REDACTED]` (case-insensitively). - **Upgraded `russh` from `0.62.5` to `0.64.1` (#315).** Closes five GitHub Security Advisories against `russh`: `GHSA-47hw-gvq5-r2gm` (high, client-side channel-scoped `Handler` callbacks firing for channel IDs the client never opened), `GHSA-w3jg-pjxf-73p4` (medium, missing X25519 low-order point validation in hybrid ML-KEM key exchange), `GHSA-35g8-35p8-c8fw` (medium, unbounded memory growth via `CHANNEL_OPEN` flood during a stalled rekey), `GHSA-p8qx-h547-fjw9` (low, strict-kex enforcement before initial KEX completion), and `GHSA-g6xm-f9xp-qq35` (low, server/agent handling). `SshHandler::check_server_key` now adapts to `PublicKeyOrCertificate` while keeping `host_key_certificates` empty (plain `Ed25519` keys only) and rejecting public-key serialization failures instead of falling back to an empty key blob. - **The docs workflow moved off the self-hosted runner.** `docs.yml` declared `runs-on: [self-hosted, Linux, X64]` — labels that name no particular machine, so the job matched *any* self-hosted runner attached to the repo. It builds on `pull_request`, and proxxx is public, so a fork PR touching `docs/**` could execute on whichever runner happened to be registered. Both jobs now run on `ubuntu-latest`; `npm ci` + `npx vitepress build` never needed a private host, and the deploy job additionally holds the `github-pages` credential. - **Live-cluster credentials now live outside the working tree.** The `tests/live/*.sh` harnesses resolve `~/.config/proxxx/live-env` before the gitignored in-repo `env.local` (`PROXXX_E2E_ENV_FILE` still overrides both). A real PVE token inside a checkout is one `git add -f` or one world-readable clone away from leaking, and the in-repo copy was mode 0644 in practice. ### Changed - **Configuration typos now fail loudly (#326).** All TOML configuration sections reject unknown keys; configs that contain unknown keys now fail to load; previously the keys were ignored. Stale documentation examples (such as `[ssh].key`, `threshold`, and `password_file`) have been corrected and are now drift-proofed in CI. In `[[alerts]]` rules, `trigger` is now accepted as an alias of `when`: the documented examples used `trigger` and previously failed to load with a missing `when` field. `[[policies]]` entries also reject unknown keys; the HITL policy examples in the docs used fields that do not exist (`require_approval`, `timeout_secs`, `when = {...}`, `require = "telegram"`) and now show the real schema (`action`, `target`, `channel`, `require`), and `telegram.chat_id` is shown as a string, which is what the loader requires. - **The live tier moved off GitHub Actions and onto a LAN-only box.** v0.14.0 added `.github/workflows/live-tier.yml`, a `workflow_dispatch` job on a self-hosted runner, to close the audit finding that the `#[ignore]`d suites and the `tests/live/` harnesses never ran. That shape was wrong for a **public** repo on two counts, and both are removed rather than mitigated: - A self-hosted runner attached to a public repo can be named by a fork's own workflow file (`runs-on: [self-hosted,