# Proof Every marketing sentence in the README, with the evidence behind it. Last full pass: **2026-08-10**, against the maintainer's production install (166,578-game RomM library, ten live indexers, qBittorrent + SABnzbd), at commit `a56a72f`. Regenerate the screenshots any time with [docs/screenshots.py](screenshots.py) pointed at your own install. Since that pass, the streaming-host rows were closed separately on **2026-08-11** against a live Wolf and a live Sunshine built for the purpose — see [design/streaming-hosts.md](design/streaming-hosts.md) §7 for the rig. "Tested against fakes" below means the protocol conversation is asserted in CI but no live server was in the loop — those rows are exactly where a field report is worth the most. | Claim | How it is proven | Evidence | |---|---|---| | 1,187 tests, green on every push | CI runs the whole suite; the README badge is the workflow, not a promise | [`.github/workflows/docker.yml`](../.github/workflows/docker.yml) runs pytest before any image builds; run it yourself: `python -m pytest tests/ -q` | | DAT verification: verified / bad-dump / unknown, copier headers, per-track discs | 28 dedicated tests + daily production use | [`tests/test_dat.py`](../tests/test_dat.py), [`tests/test_verified_import.py`](../tests/test_verified_import.py); the `[!]` and `+50 verified good dump` in the [search screenshot](img/interactive-search-live.png) is the live scorer reading a live DAT index | | Scoring explains itself, release by release | Every reason string in the WHY column is generated by `Judgement.why()`, asserted in tests | [screenshot](img/interactive-search-live.png): 51 releases, hack releases at −120 with the phrase that matched, a Switch port rejected as "names another platform"; [`tests/test_selection.py`](../tests/test_selection.py) (51 tests) | | The clock: auto-import 1min, wanted search with backoff, RSS sync, list sync, update check | Scheduler registry + backoff ladder tested; jobs visible live with last-run and result | [`tests/test_scheduler.py`](../tests/test_scheduler.py), [`tests/test_scheduling_integration.py`](../tests/test_scheduling_integration.py); [Tasks screenshot](img/tasks.png) shows the five jobs on the production install | | RSS can never grab what a search would refuse | RSS matches go through `best_release` — the same scorer — asserted directly | `test_rss_sync_grabs_a_wanted_match_through_the_scorer` in [`tests/test_scheduling_integration.py`](../tests/test_scheduling_integration.py) | | Import lists: paste/URL/Steam/GOG/Xbox/PSN/itch.io, added once ever | Parser, ledger and every account fetcher tested; a pasted top-100 synced live | [`tests/test_lists.py`](../tests/test_lists.py), [`tests/test_store_lists.py`](../tests/test_store_lists.py), [`tests/test_discover_logs_accounts.py`](../tests/test_discover_logs_accounts.py); [Lists screenshot](img/lists.png) shows a synced list with its ledger count. Steam/GOG/Xbox/PSN/itch.io are tested against protocol fakes — live-account field reports welcome | | Collections: full sets and 1G1R from DATs, resumable batches | 20 tests over planning/batching; page live | [`tests/test_collections.py`](../tests/test_collections.py); [screenshot](img/collections.png) | | Eight download clients | qBittorrent/SAB/NZBGet: production use (860 grabs — [stats](img/stats.png)). **Transmission/Deluge/rTorrent: proven against live daemons** — real Transmission 4.1.0, Deluge 2.2.0 and rTorrent 0.9.8, driven through `build_client` exactly as the Settings page builds them | [`scripts/live_proof.py`](../scripts/live_proof.py), run 2026-08-10: **9/9** — the 409 handshake, `auth.login`+`web.connect`, both rTorrent dialects over XML-RPC, adds landing in each live queue with the label applied. Plus the fakes: [`tests/test_more_clients.py`](../tests/test_more_clients.py) (34), [`tests/test_new_clients.py`](../tests/test_new_clients.py) (16) | | Playnite extension | **Runtime-proven against the real SDK and a live server**: `Playnite.SDK` 6.11 from NuGet, config pointed at the production install, only dialogs and the game database stubbed | [`scripts/playnite_proof.ps1`](../scripts/playnite_proof.ps1), run 2026-08-10: 200 live-export games imported as real `Playnite.SDK.Models.Game` objects with play actions; second sync added 0 (dedupe). The proof also caught a live 500 in the export endpoint, fixed the same hour — see `frontend_rows` in app.py | | LaunchBox plugin | **Compiled (first time ever) and executed**: the unmodified plugin source built against a reconstruction of the eight API members it touches, `Import()` reflection-invoked with live export XML | [`scripts/launchbox_proof/`](../scripts/launchbox_proof/), run 2026-08-10: 5/5 — 200 games, 2 platforms auto-created, `Save()` called once, re-import added 0. Stated gap: `Unbroken.LaunchBox.Plugins.dll` is not redistributable, so real-assembly signatures remain unverified | | The shelf: status/ratings/notes | Store round-trip + case-insensitivity + husk-cleanup tested; API exercised live | [`tests/test_game_meta.py`](../tests/test_game_meta.py); the live API round-trip is in this file's history: set → read → clear returned `{}` | | Live log tail | Ring semantics (cursor, level floor, eviction-safe sequence) tested; page live | [`tests/test_discover_logs_accounts.py`](../tests/test_discover_logs_accounts.py); [screenshot](img/logs.png) | | Auth: password, TOTP, API key, ForwardAuth; every route gated | HTTP-level: unauthenticated requests to each route must 401 | [`tests/test_auth_http.py`](../tests/test_auth_http.py) (36), [`tests/test_sso.py`](../tests/test_sso.py) (30), [`tests/test_totp.py`](../tests/test_totp.py) (21) | | Secrets never reach a browser | Redaction asserted for clients, indexers, libraries, lists; Prowlarr keys stripped from display strings | `test_a_keyed_url_is_kept_for_the_download_client`, `sanitise_for_display` tests in [`tests/test_library.py`](../tests/test_library.py); list credentials masked in [`romarr/app.py`](../romarr/app.py) (`LIST_SECRETS`) | | Zip-slip cannot escape the library | Traversal entries dropped before any write, tested with hostile archives | [`tests/test_library.py`](../tests/test_library.py) `test_archive_entries_escaping_the_root_are_dropped`, `test_traversal_entry_never_reaches_the_filesystem` | | Multi-library platform routing | 21 tests over routing rules, defaults, per-platform overrides | [`tests/test_multi_library.py`](../tests/test_multi_library.py) | | Questarr feature + roadmap parity | Item-by-item audit in the design doc, each row implemented and tested above | [design doc](design/2026-08-10-questarr-absorption-design.md); their P0–P6 mapped in the commit messages of `76eaf1e`, `b92ea04`, `a56a72f` | | Local launcher connect (Steam/Epic/GOG/Battle.net/EA) | **Proven on real hardware**: the scanner found a live Epic library and a Blizzard install on the maintainer's PC, then pushed them to the production server over the network | 17 tests in [`tests/test_launchers.py`](../tests/test_launchers.py); `scripts/connect_launchers.py` run 2026-08-10 against 192.168.0.182 — "Local launchers" appears in the [Lists screenshot](img/lists.png). Correctly found **nothing** for an empty EA folder and a Steam library on an unplugged drive, which matters more than the positives: a scanner that invents games is worse than none | | The screenshots are real | Taken by script against the production instance, same session as this file's date | [`docs/screenshots.py`](screenshots.py) — point it at your install and diff | | Every page renders its actions | A browser signs in and visits all 27 pages the way a user navigates, asserting each shows its action control with no JS error | [`scripts/prove_pages.py`](../scripts/prove_pages.py), run 2026-08-11: **27/27**. Building it caught four latent late-write bugs (a fetch resolving after navigation), all fixed | | Per-file player capability: EmulatorJS / Ruffle / js-dos / Emularity | Every capability is upstream's own declaration, not ours: core extensions from each core's `supported_extensions` in `libretro/libretro-super`, EmulatorJS's archive handling from `data/src/compression.js`, Ruffle's coverage percentages from ruffle.rs/compatibility, its projector gap from [ruffle#11539](https://github.com/ruffle-rs/ruffle/issues/11539), Emularity's engine list from its own README, RomM's Ruffle gate from `isRuffleEmulationSupported`. Then **measured against the live library**: all 166,548 rows walked through `routes_for_file` | [`tests/test_players.py`](../tests/test_players.py) (50 tests, most of them refusals); the walk, 2026-08-11 against the production RomM 5.1.0: 72,120 rows with files, 94,428 with none, **49,890 playable by EmulatorJS today**, 17,125 by Archive.org, **0 by Ruffle** — because every one of the 94,415 `.swf` rows is a catalogue entry with no bytes behind it | | "No file on the library server" is said, not guessed | `missing_from_fs` comes from RomM's own bookkeeping and its content endpoint 404s; the model emits **no** download route for such a row rather than naming one that fails | `test_a_row_with_no_file_plays_on_nothing_and_says_why` and `test_missing_and_unsupported_are_different_answers` in [`tests/test_players.py`](../tests/test_players.py); the live split is `Romm.counts()`, 166,548 = 72,120 + 94,428 | | The ecosystem credits are real links, not fabrications | Every upstream URL confirmed against the live project before it shipped; the page renders 17 cards with working links on the production install | [`romarr/ecosystem.py`](../romarr/ecosystem.py), [`tests/test_ecosystem.py`](../tests/test_ecosystem.py); [screenshot](img/ecosystem.png) — 23 external links, ROMarr marked "you are here" | | Moonlight hosts (Wolf/Sunshine) report honestly | **Proven against live servers**: a real Wolf (`stable`) and a real Sunshine (`2026.516.143833`) stood up on the maintainer's cluster, driven through the same `MoonlightHost` the Settings page builds. 60 tests still assert every refusal — that a RetroArch or a Steam app grants no platform anything, that a reachable host with an unreadable app list grants nothing, that a PIN submission never reports "paired" | [`scripts/moonlight_proof.py`](../scripts/moonlight_proof.py), run 2026-08-11: **39/39** — `/serverinfo` on both, Wolf's **UNIX socket** and its nginx proxy, Sunshine's self-signed TLS, and eleven real app titles refused while `PCSX2` granted PS2. [design doc](design/streaming-hosts.md) §6–§7, [`tests/test_moonlight.py`](../tests/test_moonlight.py) | | Moonlight pairing: ROMarr relays a real PIN and a real client pairs | **Proven end to end on both hosts** with the official moonlight-qt 6.1.0. Wolf listed the waiting client, ROMarr rebuilt its PIN-page URL and relayed the PIN; Sunshine was told the PIN with no way to list waiters, as documented | Same run: Wolf logged `Succesfully paired` and its client list went 1 → 2; Sunshine's `named_certs` went 0 → 1 with the name `ROMarr` that `submit_pin` sends. The wrong-PIN bug ([Sunshine#3944](https://github.com/LizardByte/Sunshine/issues/3944)) was **reproduced**, not cited: `{"status": true}` for a wrong PIN, and nothing paired | ## The install paths Added 2026-08-12. Each row was run against a live Docker host and a throwaway Proxmox container on the maintainer's cluster, not read. | Claim | How it is proven | Evidence | |---|---|---| | The published image pulls, boots, and answers | `ghcr.io/blizzhacker/romarr:latest` pulled on a real Docker host, started, healthy, `/api/health` 200 and `/api/v1/system/status` 401 unauthenticated / 200 with a key | Run 2026-08-12 on Docker 29.1.1. The multi-arch manifest carries all three platforms; the armv7 leg is in ["what is NOT proven"](#what-is-not-proven-in-the-same-breath) below | | The documented commands are the ones that were run | Every command in [INSTALL.md](INSTALL.md) executed verbatim — only the published port differs so it does not fight the RomM stack on that host — including the compose fail-fast error, the first-run claim, the API-key lookup, the backup, the upgrade and the rollback | The compose refusal message in the docs is the string compose actually printed. `POST /api/v1/setup` returned 200 then 409. A pin to `sha-6dc32c7` and back to `latest` both came up healthy with the state file intact | | An unreadable state file cannot silently reset an install | The failure was reproduced first: a root-owned `romarr.json` under `PUID=1000` came back with a **new API key, zero history, defaults for every setting, and an unauthenticated `POST /api/v1/setup` answering 200**. Both guards then verified against a rebuilt image | `docker/entrypoint.sh`, `romarr/store.py`, [`tests/test_docker_install.py`](../tests/test_docker_install.py), `test_a_state_file_that_cannot_be_read_stops_the_service` in [`tests/test_store.py`](../tests/test_store.py). Four cases run: clean start, root-owned state (now preserved), non-root container with unreadable state (exit 1, file untouched), corrupt JSON (still starts) | | `${VAR:-?ERROR}` was not the validation it was documented as | `docker compose config` resolves it to the literal string `?ERROR`. The file whose stated purpose was env-var validation was setting the Prowlarr API key to that string and connecting with it | Measured with `docker compose config` on the same host; the correct `${VAR:?msg}` form was then verified to stop the run with the message the docs quote | | A placeholder bind-mount path is not inert | The shipped compose file, unedited, started and reported **healthy** while creating `/path/to/roms` and `/path/to/downloads` on the host and filing into them | Same run. That is why the two paths now have no default | | The container healthcheck catches a wedged process | `SIGSTOP` on the container's main process moved it to `unhealthy` after three consecutive failures, ~100s. It does **not** cause a restart — see below | `docker inspect .State.Health`: exit codes `-1 1 -1` then `unhealthy fails=3` | | The Proxmox installer works and says what it did | Run twice on a throwaway CT, creation to `/api/health` answering, reporting `ROMarr 0.8.0 is up` with the paths and log command | 2026-08-12, CT destroyed afterwards. The auth-check fallback fired as designed (`no published release; using main`) | | The Proxmox updater can go backwards | A build that **passes** the `auth.py` guard and then cannot start was installed deliberately (a tarball with a syntax error in `__main__.py`, served through a shimmed `curl`). The updater reported `0.8.0 -> 9.9.9-broken`, saw no answer on `/api/health`, restored the previous package, restarted it, and exited 1 | Same session: version back to `0.8.0`, service active, health 200, API key / `min_seeders=42` / claimed state all intact. Also verified the stale-module case: a file dropped into the package before an update is gone afterwards | ## The browser lane, and the sites it is for Added 2026-08-12. Until this date the headless-browser backend had been tested against fixtures and **never deployed** — `pip install playwright` had never been run and no browser container existed. It is now deployed, and the rows below separate what a live run proved from what it could not reach. The browser container is **CT 134 `browser-proof`** on node Thiccc, `192.168.0.134`, 2 GB / 8 GB, running `playwright run-server` on `ws://…:3000` under a systemd unit. | Claim | How it is proven | Evidence | |---|---|---| | ROMarr drives a browser in another container over `ws://` | A real `playwright run-server` on CT 134 and the pip package (no browser) on the ROMarr host. `POST /api/v1/downloadclient/test` for the saved row answered `{"ok": true, "message": "Connected"}`, and the client list reports `"detail": "connected to chromium 151.0.7922.34"` | Run 2026-08-12 against ROMarr at commit `d0931bc` on LXC 182. Both sides pinned to playwright **1.62.0** — a version skew between client and run-server is the first thing to check if this ever stops connecting | | A real file arrives, through ROMarr's own importer | The browser client was saved through `POST /api/v1/downloadclient`, handed a job through the same `build_client()` the grab path uses, and the bytes were fetched by **`POST /api/import`** — ROMarr's own sweep, over HTTP, not a script calling Playwright | `360799main_base_E.zip` from `archive.org/download/basestation-198101/`, **3,053 bytes**, `md5 87f0d1988849b67aad826fd81dbbd3f1`, `sha1 d2605c1e1a848a5eb86b06addfe11870cb08e761`, `sha256 2ab873a8e5704d156e64b2ffd216904484b705a43955321cbfef6cdae49cf800`. The md5 and sha1 **match Archive.org's own published checksums** for that file, so the bytes are the file and not an error page. A NASA item, deliberately: no copyrighted game was downloaded to prove this | | Chromium is genuinely absent from the ROMarr host | With no endpoint configured, `GET /api/v1/downloadclient/browser` reports `Executable doesn't exist … chrome-headless-shell` and `available: false` | Same run. The ROMarr container has ~400 MB of RAM free; `apt-get install -s chromium` there is 236 packages / 867 MB, which is why the browser lives in its own container | | The robots gate refuses before it fetches | `https://archive.org/control/…` — a path that site's own `robots.txt` disallows — was queued and came back `failed / robots.txt disallows this path` with no request made | Same run, in the ledger beside the successful row | | The browser survives a reboot | `pct reboot 134`; the unit came back `active` with a new PID and port 3000 listening | Same session | | The extension captures from a live site into a real index | The **unpacked extension in real Chromium 151, headed**, opened real `vimm.net` vault pages; its service worker's own log reports `{ok: true, indexed: 1, skipped: 0}` for each | Two rows in `idx-nes.jsonl` on the proof install, carrying `"platform": "nes"`, real sizes (144384, 205824) and `media_id` 253 and 850 — the two fields the previous 23,980-row Vimm capture got wrong on every row | ### Per site: what was checked, and what could not be The extension is the lane for sites a plugin cannot read. Each verdict below says how it was reached, because "it works" would be worthless here. | Site | Verdict | Why, and what remains | |---|---|---| | **Vimm's Lair** | **Verified live** | Every field read off `vault/265` and `vault/866` in a real browser, then captured into a real ROMarr. `robots.txt` is `Disallow:` (allow-all). Its file hosts `dl`/`dl2`/`dl3.vimm.net` each serve `Disallow: /` and **were never fetched** | | **Retrostic** | **Verified live, and still indexes nothing** | `robots.txt` allows; selectors confirmed on three pages reached through the site's own advertised sitemap. But `File Size:` reads **`unknown`** on every page observed, and ROMarr refuses a row whose size it does not know. The selectors are right; the site does not publish the fact | | **The ROM Depot** | **Needs your browser — unverified** | `GET /api/getContents` returns the whole tree as one JSON to a signed-in browser and **401** to anyone else. No login was attempted and the owner's session was not used, so the tree's **shape is unconfirmed**: the reader walks the JSON and matches key names from a candidate list rather than a fixed path. First run will show either "not the JSON document the adapter expects" (signed out) or "found N entries but none carried an id, a title and a usable link" (key names differ) | | **CDRomance** | **Needs your browser — unverified** | Every path including `robots.txt` answers a Cloudflare managed challenge to a plain client. **No attempt was made to defeat it.** Selectors are WordPress shapes written from structure; the platform falls back to the URL's first segment, which cannot be wrong about markup. A wrong selector shows in the popup as "platform not found" / "size not found" before anything is sent | | **CoolROM** | **Deliberately untouched** | Its `robots.txt` names `anthropic-ai` and `Claude-Web` with `Disallow: /`. Nothing was fetched from it, no adapter ships, and none should | ## What is NOT proven, in the same breath This list was five entries long on the morning of 2026-08-10 and is now this. Each survivor names exactly what would close it. - **No ROM has ever been downloaded through the browser lane.** What is proven is the mechanism — a page opened, a real control clicked, the file streamed back from a browser in another container and landing on ROMarr's disk with the right hash. The file was a 3 KB NASA zip from Archive.org, chosen because that is a site whose robots.txt permits automated fetching and because proving a download client does not require downloading a game. Vimm's actual download is a POST to `dl3.vimm.net`, which serves `Disallow: /`, so ROMarr's own policy refuses it and the lane reports that refusal rather than performing it. Whether that POST would succeed for a person clicking it is not something this codebase has tested or will. - **Two of the four adapters have never seen the page they describe.** The ROM Depot needs a session and CDRomance answers a bot challenge to anything that is not a person, so their selectors are written from structure. Both are marked unverified in the extension's own popup, and ROMarr refuses what it cannot validate — but "sound" is not "seen", and the first person to open those pages with the extension installed is the one who closes this. The ROM Depot's `/api/getContents` shape in particular is inferred, not observed. - **The browser lane's proof ran against a second ROMarr, not the production one.** The production install on LXC 182 runs 0.8.0, which predates the browser lane entirely; the proof instance is the same host running commit `d0931bc` on its own port with its own state file. That proves the code, not an upgrade of the production service, which was deliberately left alone. - **Chrome-branded builds remain untested.** Chrome 151.0.7922.137 was installed in the test container and launches, but cannot reach the network there at all — `ERR_INTERNET_DISCONNECTED` on every navigation, as root and as an ordinary user, while Playwright's Chromium on the same host is fine. The extension is proven in Chromium 151 and in Edge; Brave, which is what the setup guide is written for, is Chromium and is expected to behave identically, but nobody has run it. - **Nothing has ever streamed a frame, and ROMarr never asks it to.** Wolf, Sunshine and a Moonlight client are now proven — probe, app list, both Wolf transports, Sunshine's TLS, and pairing completing on both — but `/launch` sits behind a paired *client* certificate and is deliberately not built. A client pairing is not a session starting. What remains is a human confirming they can actually play something on a host ROMarr routed them to. - **Steam Headless has still never been run.** It is modelled as a Sunshine with a web desktop attached, read from its compose files. Sunshine's half is now proven; the reduction is not. Running the container and pointing `MOONLIGHT_KIND=steam-headless` at it closes the row. - **The live proof covers one Wolf profile, no GPU, and one session.** `_wolf_apps` unions app titles across profiles; the host had one. Wolf ran on software encoders throughout. Nothing here says how either host behaves after a month of uptime or with many paired clients. Re-run [`scripts/moonlight_proof.py`](../scripts/moonlight_proof.py) on your own hardware and post the output — full detail in [design/streaming-hosts.md](design/streaming-hosts.md) §6. - **No file has ever been handed to js-dos or to a self-hosted Emularity by this code.** Both are modelled from their own documentation and neither has been stood up: js-dos 8.4.1 takes a `.jsdos` or `.zip` bundle and Emularity names MAME, EM-DOSBOX and SAE, and ROMarr has never watched either open a file out of a library. It reports them as capabilities and prints a link only when told a URL, which is the honest shape for something unproven — one person setting either up and posting what happened closes this. The DOS shelf on the maintainer's install cannot close it: 148 of its 163 rows carry the extension `.dos`, which **no** browser player declares, so ROMarr reports them as unplayable and that has not been checked against what those rows actually are on disk. - **Ruffle has never been asked to play anything here**, because all 94,415 `.swf` rows on the maintainer's library are catalogue entries with no file. The Ruffle side of the model is asserted against fixtures and against RomM's own source; the first person with SWFs actually on disk proves it. - **Synology Download Station and Real-Debrid** have only met protocol fakes: one needs Synology hardware, the other a paid account. [`scripts/live_proof.py`](../scripts/live_proof.py) is ready for both — running it against either and posting the output closes the row. - **The five account connectors** (Steam, GOG, Xbox, PSN, itch.io) are credential-gated, so only an account holder can prove them: `python scripts/account_proof.py ` does it in one command through ROMarr's own fetcher and prints your library. Open an issue titled "account proof: " with the (redacted) output. - **LaunchBox's real assembly signatures** — the plugin compiles and its logic runs, but against a reconstruction of the API, because the DLL is not redistributable. One person dropping the built plugin into a real LaunchBox closes it. - **Gameyfin and Retrom** are asserted against their declared behavior, not lived-in installs. - **armv7 Docker** ships without ROM Hub plugins (no pydantic musl wheel) — by design, stated at build time, and not going to change until pydantic ships the wheel. - **The armv7 image has never been booted.** It is built on every push and the published artefact was unpacked and inspected — `python3.13` and `bsdtar` are genuine 32-bit ARM EABI5 binaries and `rom_hub` is absent as documented — but nothing has ever run it, and unpacking an image is not starting one. The maintainer has no 32-bit ARM hardware and installing binfmt handlers to emulate one would have modified a production Proxmox node. `docker run` on a Raspberry Pi 2/3 in 32-bit mode, followed by `curl localhost:6868/api/health`, closes this row in one command. - **The Home Assistant add-on has not been installed from the add-on store.** The image is the same one Docker users run and the options-to-environment translation is tested directly ([`tests/test_ha_options.py`](../tests/test_ha_options.py)), but nobody has added the repository to a real Home Assistant and clicked Install. - **The community-scripts installer** (`proxmox/install/romarr-install.sh`) has never been executed. It only runs inside their framework, which resolves helpers from their repository, so it cannot be run until ROMarr is accepted there. `proxmox/ct/romarr.sh` is the one that is tested and the one the README documents. Turning any of these into a row of the first table is the most valuable contribution this project can receive today.