# πŸ—ΊοΈ Roadmap Where Auditarr is heading. Each line starts with its status: 🟒 **done** and in the app today Β· 🟑 **in progress** right now Β· πŸ”΄ **not started** Β· πŸ™‹ **waiting on the owner** (a step only the project owner can do, like a submission). The first πŸ”΄ in each list is what's next. (The short codes like `T1`/`U1` are just internal labels.) The current focus: make the app **trustworthy to run** (a real test harness) and **easy to read** for someone who isn't a developer. ## 🚧 Now: solid testing + a friendlier interface πŸ§ͺ **Testing β€” proving it does the right thing** - 🟒 **T1** A read-only mode (`READ_ONLY=1`): every change it would send to an arr, Bazarr or Plex is blocked and counted instead, so test runs can't touch anything real. - 🟒 **T2** Update-testing compares findings check-by-check, not just by the total count. - 🟒 **T3** Code linting in CI. - 🟒 **T4** Fake media with known defects (built with ffmpeg) that the detectors must catch in CI. - 🟒 **T5** "Tonight's plan": a panel and API showing exactly what the night's automatic actions would do; the self-test confirms the planners run. - 🟒 **T6** Backup proof: a nightly integrity check of the newest backup, shown in History. - 🟒 **T7** Failure drill: a test run with Bazarr / Plex / an arr switched off must still finish and report "degraded" rather than crash. - 🟒 **T8** A real-browser test (Playwright) at phone and desktop width that fails if raw codes or setting names leak into the normal view, with screenshots kept as CI artifacts. πŸ™‚ **Usability β€” making it readable** - 🟒 **U1** Plain names for every check and severity; codes moved to tooltips. - 🟒 **U2** A human sentence first on every finding, technical detail folded away. - 🟒 **U3** Verb buttons ("Download a better copy", "Make subtitles"…), old names kept as tooltips; an Undo toast after any file move. - 🟒 **U4** A simple view by default (just "Needs you" + "Done overnight"); low-priority groups, Live and History tucked behind an Advanced switch. - 🟒 **U5** In-page help: a "?" on each section opens the matching part of the user guide. - 🟒 **U6** Settings shown as grouped switches and sliders with plain sentences; setting names only in Advanced. - 🟒 **U7** A visual identity (first pass, 2026-11-11: teal accent, magnifier-check logo and favicon, light theme via system preference or the ◐ toggle, themed chips/buttons/confirms; icon.png for the Unraid template). Polish continues with the screenshot baselines. Original brief: a visual identity: give Auditarr its own look, the way Sonarr/Radarr/Bazarr/Seerr each have one β€” an accent colour, a logo/icon, light and dark themes, and consistent chips and buttons; the plain-language rules and the test bots' selectors stay intact. πŸš€ **Going public (added 2026-11-11)** - 🟒 **U8a** Social preview + README banner in the Homarr style (`docs/social-preview.png`, `docs/banner.png`, generated by `docs/make_banner.py` from the app's own screenshots). - 🟒 **U8b** Repository public (2026-11-11): GPL-3.0, description and topics set, container package public and linked, social preview uploaded, Pages demo live. - 🟒 **U9** (built 2026-11-11; Pages deploy switches on automatically when the repo is public) Static demo on GitHub Pages: the real page with the sanitized fixtures as its API, a "demo data, actions simulated" banner, republished by CI on every release; linked from the README as "Try it in your browser". - πŸ”΄ **U10** Live sandbox (later, once public): the app + the fake arr stack on a small host behind a tunnel, reset hourly, real buttons on synthetic files. - 🟒 **U8c** (2026-11-11) README screenshots section (Findings, Today, Setup) refreshed from CI's screenshots on each release. ## πŸ€– Now: the self-testing bots (same rule β€” checked means done) - 🟒 **Q1** A fake arr stack in CI: mock Sonarr/Radarr/Bazarr + fake media + the real app, with the browser bot clicking real buttons that hit the real API (trash β†’ Undo β†’ restored, re-download β†’ arr receives the command, AI subtitles β†’ job queued). - 🟒 **Q2** Plain-language guardrails: every check's sentence and every button label must be free of codes and arr jargon, 30 words or fewer, and end in a full stop β€” enforced in CI. - 🟒 **Q3** Living test data: a nightly sanitised snapshot from a real library, plus hand-made edge cases (empty library, zero findings, a 2000-finding show, every service down, API errors). - 🟒 **Q4** A contract test tying the UI and API together: every button the UI offers is accepted by the backend, and every finding the API returns renders as a row. - 🟒 **Q5** Accessibility scan (contrast, labels, focus order) in the browser run; fails on serious issues. - 🟒 **Q6** Visual-regression check: baseline screenshots (in `uibot/baseline/`, taken from CI) that fail on a >1 % pixel change at phone and desktop width; refresh them on purpose after a visual change. - 🟒 **Q7** "Monkey" run: 60 seconds of random clicking with no JavaScript errors, and every save preceded by a confirm box. - 🟒 **Q8** Rotating failure drill: a different service down each week (all / Bazarr / Plex / Whisparr by week number). The slow-service variant is split out as Q9. - 🟒 **Q9** (2026-11-11) Slow-service drill: `auditarr-deploy.sh drill 4` puts Sonarr behind a 200 s latency proxy (a throwaway python container); the staged run must finish within 25 min, report Sonarr degraded and write nothing. Part of the weekly rotation. πŸ€– **Bots, next layer (added 2026-11-11, ranked)** - 🟒 **Q10** (2026-11-11) Mutation check: deliberately break one detector in CI and assert the synthetic suite fails; rotate the detector nightly. - 🟒 **Q11** (2026-11-11) Regression corpus: one replayable case per real incident (`tests/regressions/`): Bazarr prose timestamp, self-test matching MISSING_FILE, batch killed by a deploy, orphan race. - 🟒 **Q12** (2026-11-11) Fuzzed detectors: thousands of random valid ffprobe results; invariants: no exception, known codes, a sentence for every finding, severity from the fixed set. - 🟒 **Q13** (2026-11-11) Upgrade test: sample database per release; the new daemon must migrate, run and render on each. - 🟒 **Q14** (2026-11-11) Restore drill: `auditarr-deploy.sh restore` integrity-checks the newest nightly state backup, boots a throwaway read-only container on it and checks the page, open findings and run history come back; fails if the backup is older than 48 h. Weekly cron on the host. - 🟒 **Q15** (2026-11-11) Chaos in the fake stack: `tests/e2e_chaos.py` + `mock_stack.chaos()` (500s, slow answers, Radarr vanishing); degraded finish, findings kept, no writes, clean recovery. - 🟒 **Q16** (2026-11-11) Performance budget: `tests/perf_budget.py` seeds 10k files / 5k findings and fails CI when the report, page state, Today, plan or memory exceed their budgets (today: all under 0.1 s, 55 MB). - 🟒 **Q17** (2026-11-11) Persona scripts in the live browser bot: "why is this episode missing" (search β†’ row β†’ sentence β†’ Search now), "make AI subtitles for the whole show" (Tools box β†’ confirm β†’ queued), "stop showing this one" (Ignore β†’ row gone), plus the existing "undo the last move". - 🟒 **Q18** (2026-11-11) Golden digest: `tests/golden_digest.py` pins the push text and checks delivery (title, priority, Click link) against a mock ntfy. - 🟒 **Q19** (2026-11-11) Authorization test: a POST from a non-private address without the Authelia header is refused. - 🟒 **Q20** (2026-11-11) Security scans in CI: dependency audit + container image scan, failing on critical issues. - 🟒 **Q21** (2026-11-11) Flakiness ledger: reads get one retry; a retry that works, or a service that drops and comes back, lands in a 30-day table on the History tab. Proven by the chaos test. - 🟒 **Q22** (2026-11-11) Published as [arr-mock](https://github.com/chrizantoz/arr-mock): the fake stack, chaos mode and `synthlib`, with a README and self-check. `tests/mock_stack.py` stays the source; sync by hand when it changes. - 🟒 **Q23** `docs/TESTING.md`: how the harness works (read-only staging + write ledger, per-check promotion gate, synthetic media, plain-language lint, fault drill) so other projects can copy the method. ## πŸ“¦ Next: let someone else install it - 🟒 **L1** Setup wizard β€” the **Setup** tab asks for your Sonarr / Radarr / Bazarr / Plex addresses and keys, tests each one, keeps them in Auditarr's own database (masked afterwards) and checks that the media folders are visible; config-file mounts remain the hands-off alternative. - 🟒 **L2** A guided first run: the Setup tab follows the first audit phase by phase in plain words with a progress bar and a time expectation; the Findings page shows a banner until the first run is done. - 🟒 **L3** Install docs for a first-timer: `docs/INSTALL.md` (Unraid + compose, where the keys are, the media-path rule), `docs/docker-compose.yml`, a generic `docs/unraid-template.xml`. Submitting to Community Apps is the owner's step (needs a public repo and an icon). - 🟒 **L4** (2026-11-11) Community Apps: [chrizantoz/unraid-templates](https://github.com/chrizantoz/unraid-templates) submitted and **auto-approved** the same day. Auditarr installs from the Apps tab on Unraid. - πŸ™‹ **L5** Listings: Unraid forum support thread first (then its URL goes into the template's Support tag), r/unRAID and r/selfhosted posts with the banner and the demo link (texts prepared 2026-11-11). awesome-selfhosted only accepts projects whose first release is 4+ months old β†’ earliest **2027-03-12**; pull request against awesome-selfhosted-data, category Media Management. ## πŸš€ Releases - 🟒 **R1** (2026-11-11) First public release `v0.9.0` with release notes; images tagged `latest`, `0.9`, `0.9.0` and the commit; CI badges in the README. - 🟒 **R2** (2026-11-11) `scripts/release.sh vX.Y.Z`: checks, notes from the CHANGELOG, baselines from the green CI run, commit + tag + push, GitHub release, boot check of the tagged image. See `docs/RELEASING.md`. (The demo refreshes itself on every push to main.) ## ✨ Later: features (added 2026-11-11, ranked) - 🟒 **F1** (2026-11-11) Import hook: Sonarr/Radarr webhook audits a file within a minute of import and pushes "re-downloading" before anyone presses play; the night run becomes the safety net. - 🟒 **F13** (2026-11-11) Built-in sign-in: user + hashed password set on the Setup page or via env, session cookie, login page, Sign out; `/health` and the token-protected webhook stay open; `TRUST_PROXY_AUTH=1` for Authelia-style proxies. - 🟒 **F2** (2026-11-11) Watch-ahead priority: Plex On Deck + Tautulli history decide which files get checked first and hardest (content check, subtitles, defaults). - 🟒 **F3** (2026-11-11) Library score and trend per library, from open findings weighted by severity, graphed over weeks. - 🟒 **F4** (2026-11-11) Device profiles: declare your players (PS5, Shield, TV, Plex Web); compatibility notes become real "will not direct-play on X" verdicts, and noise for players you do not own disappears. - πŸ”΄ **F5** Image subtitles to text: OCR PGS/VobSub tracks to SRT once so files direct-play with proper subtitles. - πŸ”΄ **F6** Per-person preferences: language defaults per Plex user, applied to each account's track selection. - πŸ”΄ **F7** Upgrade candidates: files below the profile cutoff, oldest encodes, same movie at two resolutions, with the disk space each decision frees or costs. - πŸ”΄ **F8** Loudness check: flag files whose dialogue is far quieter than the library norm; optional lossless gain fix. - 🟒 **F9** Viewer role: a second, read-only login (Setup β†’ Protect the page β†’ read-only account). Buttons hidden, every change refused by the server. _(2026-11-11)_ - πŸ”΄ **F10** Web push + installable app (PWA): the digest as a phone notification without ntfy. - πŸ”΄ **F11** More than one instance per arr kind (two Sonarrs). - πŸ”΄ **F12** Translations: one JSON per language for names, sentences, buttons and help. - 🟒 **F14** (2026-11-11) Same-day AI subtitles: after a new download with no subtitles, if Bazarr still has nothing `AUTO_WHISPER_IMPORT_MINUTES` later, Whisper runs right away (Bazarr's API does not expose per-item search attempts, so the import hook plus a head start is the signal). Nightly `AUTO_WHISPER_DAYS` stays as the catch-all. Owner request. ### 🌍 Reach, usefulness and upkeep (added 2026-11-11, ranked β€” owner-approved) - πŸ”΄ **F15** Jellyfin and Emby support: the same checks, Up-next, track selection and library health for a Jellyfin or Emby server, chosen on the Setup page next to Plex. - πŸ”΄ **F16** Stuck downloads: a queue item sitting for days, a failed import or a dead indexer becomes a finding with one-click "remove and search again". - πŸ”΄ **F17** Discord, Telegram and email for the morning push and alerts (Apprise-style adapters; ntfy stays). - πŸ”΄ **F18** Close the loop with the arr: after two re-grabs come back with the same defect, mark the release failed and blocklist it so the arr stops fetching that group. - πŸ”΄ **F19** Dashboard endpoint: a token-protected summary (needs-you count, health, last run, library scores) for Homepage, Homarr and Home Assistant widgets. - πŸ”΄ **F20** Subtitle sync check: spot a subtitle that drifts from the dialogue and drive Bazarr's sync for that file. - πŸ”΄ **F21** Quiet hours for heavy jobs: no Whisper or remux while Plex has active streams or inside a time window. - πŸ”΄ **F22** Space hogs: files whose size is absurd for their resolution, two copies of a movie at different qualities, encodes below the profile cutoff, with the gigabytes each decision frees (extends F7). - 🟒 **F23** (2026-11-11) About panel + update check: version/build/runtime/uptime, newest release checked daily, "update available" chip, Check for updates, Update now via `UPDATE_HOOK_URL`. Owner request. - πŸ”΄ **F24** Export and import: settings, device profiles and the ignore list as one file, for moving to a new box or recovering (pairs with the restore drill). - πŸ”΄ **F25** Per-library rules: different language, subtitle and automation policies for anime, kids, movies. - πŸ”΄ **F26** Act from the phone: ntfy action buttons on the morning push ("Ignore", "Download a better copy", "Make AI subtitles") that call the API with a one-time token, so the common fixes never need the page. - πŸ”΄ **F27** Ask in plain words: a small box that answers questions about your library from the findings ("why does Jersey Shore keep transcoding", "what is wrong with the anime library this week") using a local model (Ollama) or none at all for the simple ones. - πŸ”΄ **F28** Health badge: a public, token-free `/badge.svg` with the library score for a README, Homepage or a forum signature. - πŸ”΄ **F29** Snooze instead of ignore: "remind me in 30 days" on a finding; it disappears and comes back once, with a note that it was snoozed. - πŸ”΄ **F30** Weekly report card: one push or email on Sunday with the score trend, the week's fixes and the three things most worth doing, in the same plain words as the morning push. ## πŸ™‚ Later: even friendlier (added 2026-11-11, ranked) - πŸ”΄ **U11** Guided first visit: a three-step overlay on the first open (Today β†’ a row β†’ its buttons), dismissible, never shown again. - πŸ”΄ **U12** "What if" on every row: one tap expands what happens if you ignore it, fix it or wait, in two sentences each. - πŸ”΄ **U13** Undo for more than file moves: a ten-second undo on Ignore, on settings changes and on accept-language. - 🟒 **U14** (2026-11-11) Setup auto-discovery: **Find my apps** probes the Docker host, the gateway, compose-style container names and `DISCOVER_EXTRA` addresses on the usual ports, identifies each app by its public page, says whether a key is already mounted, and fills the address on **Use this**. - 🟒 **U15** (2026-11-11) Errors that say what to do: connection tests answer with what happened and a next step (wrong key β†’ where to copy it; nothing listening β†’ the Docker 'localhost' trap; name not found; timed out; URL base; certificate). Shown under the red chip on the Setup page. - πŸ”΄ **U16** One-click "fix everything safe" per show: all reversible, low-risk fixes for a title behind one confirm, with a summary of what it did. - πŸ”΄ **U17** Search that understands intent: "anime no subtitles", "4k movies wrong language" filter by library, kind and problem at once. - πŸ”΄ **U18** Phone layout, second pass: big Today buttons, swipe to ignore or fix on a row, sticky filter bar. - πŸ”΄ **U19** Keyboard and screen-reader pass: shortcuts for the common actions, a larger-text mode, focus order checked by the browser bot. - πŸ”΄ **U20** Help that answers questions: a search box over the user guide inside the "?" panel ("how do I make subtitles for a whole show"). - πŸ”΄ **U21** Try it on one show first: the first audit offers to check a single show before the whole library, so a new user sees a result in two minutes instead of twenty-five. - πŸ”΄ **U22** "What is in this file" in plain words: a per-row panel that lists the tracks like a person would ("1 video 1080p Β· English audio 5.1 Β· Japanese audio stereo Β· no subtitles"), instead of ffprobe output. - πŸ”΄ **U23** Schedule from the page: pick the nightly time and the days in Settings instead of an environment variable; show the next run time on Today. - πŸ”΄ **U24** Progress you can read: the first audit shows "checked 1,200 of 14,000 Β· about 18 minutes left" on Today, not only on the Live tab. - πŸ”΄ **U25** Done-for-you defaults by library: when a library is named Anime or Kids, the matching language and subtitle switches are suggested on the Setup page with one click to accept. ## πŸ”§ Later: smaller improvements - 🟒 **S1** Content verification for movies (Radarr): dialogue against the movie's synopsis plus two dozen decoys from the same library. _(2026-11-11)_ - 🟒 **S2** A findings-over-time chart in History (active findings per run plus the per-library health trend). - 🟒 **S3** Phone pushes open the page on the right tab (`#findings`, `#trash`); set `PUBLIC_URL` when the page has a public address. _(2026-11-11)_ - πŸ”΄ **S4** Per-show learning: problem types you always ignore quietly drop to low on their own. - 🟒 **S6** Re-check orphans against the arr at the end of a run, so files imported mid-run aren't flagged as orphans (added 2026-11-11). - 🟒 **S7** (2026-11-11) MP4 lossless rewrite safety: after the swap, re-probe the final file and compare its checksum with the verified temporary file; refuse the bulk button for MP4 until the root cause of one corrupted result is understood (details in the private incident notes). - 🟒 **S8** (2026-11-11) Run visibility: progress heartbeat during the quiet Plex-selection phase, "last activity" and a "possibly stuck" chip on the Live tab; settings never echo or log secrets. - πŸ”΄ **S5** A Sonarr custom format that penalises AV1 if certain re-downloads keep coming back as AV1. ## βœ… Recently done (2026-11-10/11) Plex sync, missing-item evidence, the subtitles pipeline + Whisper (GPU router), DUB_ONLY detection, original-language track defaults (anime native + English subs), the Space tab, the Today/digest view, Settings/History, staged updates, the full nightly automation (fix header, verify, auto-whisper, accept-language, re-download learning), REMEDIATE on, the user guide, and the update-flow explainer.