# Contributing to Nudgarr Thanks for your interest in contributing. This document covers the project structure, how the pieces connect, how to run locally, and where to make common types of changes. Nudgarr is a lightweight project -- the goal is to keep it that way. If you're considering a larger change, opening an issue to discuss it first is always appreciated. --- ## Project structure As of v5.0.0, Nudgarr uses a SQLite database for all persistence via the `nudgarr/db/` package and an Alpine.js single-file frontend. Patch releases (v5.0.3, etc.) are listed at the top of `CHANGELOG.md`; `constants.py` `VERSION` must match the latest changelog heading (enforced by `validate.py`). ``` nudgarr/ <- Python package __init__.py <- package metadata, exposes __version__ constants.py <- VERSION, file paths, DEFAULT_CONFIG utils.py <- shared helpers: time, file I/O, HTTP, URL validation, jitter config.py <- load, validate, and deep-copy config cf_effective.py <- CF Score effective enablement: effective_cf_score_enabled(), allowed_cf_score_instance_ids(), prune_cf_entries_on_effective_disable_transition(). Also defines CF_LAST_INSTANCE_SYNC_PREFIX and CF_SCAN_SNAPSHOT_PREFIX -- import from here, not the syncer. db/ <- SQLite persistence layer (package) __init__.py <- public API -- re-exports everything below connection.py <- thread-local connection, _SCHEMA_SQL, init_db, close_connection history.py <- search_history table; get_search_history() accepts optional since (ISO UTC, last_searched_ts), type_filter (Cutoff Unmet / Backlog / CF Score), and title_search (substring on title). Each row includes eligible_again, import_iteration, and imported (bool). Confirmed imports set eligible_again to "Imported". Sonarr rows join stat_entries on series_id (imports are per-series). Used by Library History and any caller of /api/state/items entries.py <- stat_entries table; get_imports_since(since_utc) counts confirmed imports by app since a UTC timestamp exclusions.py <- exclusions table lifetime.py <- sweep_lifetime and lifetime_totals tables appstate.py <- nudgarr_state key/value table; get_state, set_state, delete_state, delete_states_with_prefix backup.py <- JSON export helper intel.py <- intel_aggregate and exclusion_events tables; get_intel_aggregate, update_intel_aggregate, reset_intel, get_pipeline_search_counts, get_cf_score_health. Also reads CF_SCAN_SNAPSHOT_PREFIX from cf_effective for per-instance scan count display in Intel. cf_scores.py <- cf_score_entries table; upsert, query, reset; count_cf_score_entries() / get_cf_score_entries() support search (title LIKE), sort_col/sort_dir, and arr_instance_id filter (composite key must match cf_score_syncer: radarr|url or sonarr|url, trailing slash stripped); delete_cf_scores_for_instance(arr_instance_id); get_cf_max_last_synced_at_for_instance(arr_instance_id) -- MAX last_synced_at before prune, used to preserve last sync time state.py <- exclusions and history helpers built on top of nudgarr.db; also exposes state_key(name, url) -- the canonical composite key used across search_history, stat_entries, and cooldown lookups auth.py <- password hashing, lockout, session checks notifications.py <- Apprise wrappers (sweep complete, import, error) log_setup.py <- logging initialisation and runtime level control arr_clients.py <- Radarr and Sonarr API calls; pagination handled internally; CF Score API functions (cf_get_quality_profiles, cf_radarr_*, cf_sonarr_*) in the CF Score Scan section at the bottom cf_score_syncer.py <- CustomFormatScoreSyncer; full library sync logic for Radarr and Sonarr; applies tag/profile sweep filters at write time; writes live sync progress and CF_SCAN_SNAPSHOT_PREFIX state to nudgarr_state per instance stats.py <- import tracking, cooldown logic, stat recording globals.py <- Flask app instance, STATUS dict, RUN_LOCK, security headers, persistent secret key sweep.py <- run_sweep orchestrator + per-instance helpers; CF Score third pipeline pass after Cutoff Unmet and Backlog scheduler.py <- cron scheduler, import check loop, cf_score_sync_loop; writes STATUS["last_sweep_start_utc"] before run_sweep() and STATUS["imports_confirmed_sweep"] after; persists per-pipeline last run timestamps to nudgarr_state routes/ <- Flask blueprints (one file per domain) __init__.py <- register_blueprints() -- called once from main.py auth.py <- /, /login, /setup, /api/auth/*, /api/setup; the GET / handler (index()) is the only place that calls render_template('ui.html', VERSION=VERSION) config.py <- /api/config, /api/instance/toggle, onboarding, /api/whats-new/dismiss arr.py <- /api/arr/tags, /api/arr/profiles sweep.py <- /api/status, /api/run-now, /api/test, /api/test-instance state.py <- /api/state/*, /api/state/clear, /api/file/*, /api/exclusions*, /api/arr-link; GET /api/state/items passes type & search query params through to get_search_history() stats.py <- /api/stats (instance, type, search, period, pagination), /api/stats/clear, check-imports intel.py <- /api/intel, /api/intel/reset cf_scores.py <- /api/cf-scores/status, /api/cf-scores/entries (instance_id, search, sort, dir), /api/cf-scores/scan, /api/cf-scores/reset notifications.py <- /api/notifications/test diagnostics.py <- /api/diagnostic, /api/log/clear static/ <- JS and CSS served as static assets app.js <- entire Alpine.js frontend; single nudgarr() function with all state, computed props, and methods alpine.min.js <- Alpine.js v3.15.11, self-hosted (no CDN dependency) fonts/ <- Outfit and JetBrains Mono, served locally templates/ <- HTML served by Flask render_template() login.html <- login page setup.html <- first-run setup page ui.html <- full Alpine.js UI;
main.py <- entry point: signals, startup ping, thread launch and join nudgarr.py <- compatibility shim for source runners (deprecated) validate.py <- pre-package static analysis tool ``` --- ## Database All persistence goes through the `nudgarr/db/` package. Import from it as `from nudgarr.db import ...` or `from nudgarr import db`. The database lives at `/config/nudgarr.db` by default (controlled by the `DB_FILE` env var). Schema is defined in `_SCHEMA_SQL` in `nudgarr/db/connection.py` and applied by `init_db()`. Migrations are versioned in the `schema_migrations` table. If adding a new migration, write a new `_run_migration_vN` function in `nudgarr/db/connection.py` and call it from `init_db()`. Do not modify or remove existing migration functions -- they may have already run on installed databases. **Intel aggregate write points** `intel_aggregate` is a protected accumulator -- it must never be cleared by any normal operation (Clear History, Clear Imports, pruning). It is only reset by the explicit Reset Intel action at the bottom of the Intel panel. The aggregate is updated at two write points: - `confirm_stat_entry()` in `db/entries.py` -- snapshots turnaround, searches per import, pipeline import split (Cutoff Unmet via `entry_type="Upgraded"`, CF Score via `entry_type="CF Score"`, Backlog via all other types), quality upgrades, iteration counts, and per-instance imports and turnaround at the moment each import is confirmed. - `reset_intel()` in `db/intel.py` -- the only operation that clears both `intel_aggregate` and `exclusion_events`. Note: `success_total_worked` and `library_age_buckets` remain as columns in `intel_aggregate` but are no longer written to as of v4.3.0. They are unused orphan columns retained to avoid a migration. All aggregate writes happen inside the same transaction as the operation that triggers them. A rollback undoes both the primary write and the aggregate update atomically. Live Intel queries (pipeline search counts, CF Score health) read directly from `search_history` and `cf_score_entries` at request time via `get_pipeline_search_counts()` and `get_cf_score_health()` in `db/intel.py`. These are not stored in the aggregate and are not affected by Reset Intel. **Exclusion event write points** `exclusion_events` is append-only. A row is written at every exclude and unexclude action in `db/exclusions.py`: `add_exclusion()`, `add_auto_exclusion()`, `remove_exclusion()`, and `clear_auto_exclusions()`. The table is never modified outside of `reset_intel()`. | Table | Purpose | |---|---| | `search_history` | Every item Nudgarr has searched, with cooldown timestamps | | `stat_entries` | Items pending import confirmation and confirmed imports | | `quality_history` | Per-import quality upgrade records for the Imports tooltip | | `exclusions` | Titles excluded from sweeps -- source, search count, acknowledged flag | | `exclusion_events` | Append-only audit log of every exclude/unexclude action | | `intel_aggregate` | Single protected row accumulating lifetime Intel metrics | | `cf_score_entries` | CF Score index -- monitored files below their cutoff score | | `sweep_lifetime` | Per-instance lifetime sweep stats | | `lifetime_totals` | Lifetime confirmed import counts (movies/shows) | | `nudgarr_state` | General key/value persistent state (e.g. last run time) | | `schema_migrations` | Records which migrations have been applied | --- ## Import graph Understanding what imports what avoids circular dependency issues. The rule is simple: modules lower in the list may import from modules higher up, never the reverse. ``` constants |-- log_setup (imports constants only -- sits alongside utils) |-- cf_effective (imports constants + stdlib only; db imported lazily | inside prune_cf_entries_on_effective_disable_transition | to avoid a circular import) `-- utils `-- db `-- config `-- state |-- auth |-- notifications `-- stats `-- arr_clients `-- sweep `-- scheduler globals <-- imports only constants + stdlib (Flask, threading, os) routes/* <-- import from globals + any module above them main.py <-- imports from routes, scheduler, globals, log_setup ``` `globals.py` is the one module with a special rule: it must only import from `constants` and the Python standard library. Everything else imports `globals` to get the `app`, `STATUS`, and `RUN_LOCK` objects. Breaking this rule will create a circular import. `cf_effective.py` follows a similar rule: it imports only from `constants` and the standard library at module level. The `db` import inside `prune_cf_entries_on_effective_disable_transition` is deferred (local import) to avoid a circular dependency since `db/cf_scores.py` imports from `cf_effective`. **Known exception -- `routes/stats.py` imports from `scheduler`:** The manual import-check endpoint calls `_run_auto_exclusion_check` directly from `scheduler.py`. This is the only place a route file reaches up into `scheduler`. Do not add further route-to-scheduler imports without a clear reason. --- ## How a sweep works 1. `scheduler_loop` in `scheduler.py` runs on a timer (or responds to `run_requested`) 2. It writes `STATUS["last_sweep_start_utc"]` immediately before calling `run_sweep(cfg, session)` 3. `run_sweep` in `sweep.py` runs `_check_queue_depth` -- if queue depth is enabled, one `GET /api/v3/queue/status` call is made per enabled instance, totals are summed, and the sweep is skipped entirely if the sum meets or exceeds the threshold (fail-open on instance errors). If skipped, `STATUS["last_skipped_queue_depth_utc"]` is set and persisted. 4. `run_sweep` then runs `_run_auto_unexclude` and iterates over configured Radarr and Sonarr instances in a unified loop, calling `_sweep_instance(app=...)` for each 5. Each instance helper calls `arr_clients.py` to fetch eligible items -- pagination is handled internally with no item cap. The helper then applies exclusions, tag/profile filters, and queue filtering, applies cooldown logic from `stats.py`, calls the search API, then records results in a single batched transaction via `nudgarr/db/` 6. `run_sweep` returns a summary dict 7. `scheduler_loop` stores the summary in `STATUS["last_summary"]`, persists `last_run_utc` and per-pipeline timestamps (`last_run_cutoff_utc`, `last_run_backlog_utc`, `last_run_cfscore_utc`) to `nudgarr_state`, populates `STATUS["imports_confirmed_sweep"]` via `get_imports_since()`, triggers notifications, and runs import checks 8. A separate `import_check_loop` thread runs independently, polling for confirmed imports without waiting for a sweep. After each cycle it also runs the auto-exclusion evaluation --- ## How a request works 1. `main.py` calls `register_blueprints()` which registers all route blueprints with the Flask app 2. A request arrives at one of the endpoints 3. The `@requires_auth` decorator in `auth.py` checks session validity and runs a CSRF origin check on POST requests before the handler runs 4. The handler reads/writes state via `nudgarr/db/` and `state.py`, config via `config.py`, and updates `STATUS` in `globals.py` 5. Flask serialises the response as JSON (most endpoints) or renders a template --- ## Running locally (without Docker) **Requirements:** Python 3.12+, pip **Shutdown:** Nudgarr handles SIGTERM and SIGINT cleanly via a `threading.Event`. On `docker stop`, any in-progress sweep is allowed to finish before the process exits. ```bash # Install dependencies (includes Waitress) pip install -r requirements.txt # Run python main.py ``` The app will start on port 8085 by default. The database and config are written to `/config/` -- you can override with environment variables: ```bash export CONFIG_FILE=./config/nudgarr-config.json export DB_FILE=./config/nudgarr.db export PORT=8085 python main.py ``` --- ## Running with Docker ```bash docker build -t nudgarr . docker run -p 8085:8085 -v ./config:/config nudgarr ``` Or use the provided `docker-compose.yml`. --- ## Making changes ### Adding a new config key 1. Add the key with its default value to `DEFAULT_CONFIG` in `constants.py` 2. Add a validation rule in `validate_config()` in `config.py` if needed 3. Read the value via `cfg.get("your_key", default)` wherever it is used If the key accepts a fixed set of string values, define the allowed values as a tuple constant in `constants.py` (see `VALID_SAMPLE_MODES` for the pattern) and import it in both `config.py` and wherever the value is consumed. **Frontend wiring (if the key is exposed in Settings, Pipelines, or Advanced):** The config arrives in the browser via `/api/config` and is applied to the Alpine.js state object by `applyConfig()` in `app.js`. To wire up a new key: 4. **HTML control** -- add the input or toggle element to the relevant panel section in `ui.html`. Use `x-model` to bind to the matching state property (e.g. `x-model="myCooldownHours"`), or `@change="unsaved.settings = true"` for fields that require a Save button. 5. **State declaration** -- add the property to the matching state section in `nudgarr()` in `app.js` with a sensible default (e.g. `myCooldownHours: 0`). 6. **applyConfig()** -- in the `applyConfig()` method, read from `this.cfg` into your state property: `this.myCooldownHours = this.cfg.my_cooldown_hours ?? 0;` 7. **_syncFullCfgFromUi()** -- in the `_syncFullCfgFromUi()` method, write back to `this.cfg`: `this.cfg.my_cooldown_hours = Number(this.myCooldownHours);` This method is called by every panel's `savePanel()` before the POST to `/api/config`. ### Adding a new API endpoint 1. Decide which blueprint it belongs to (or create a new one under `routes/`) 2. Add the route handler to that blueprint file 3. Apply the `@requires_auth` decorator from `nudgarr.auth` to every handler that requires an authenticated session -- which is every endpoint except `/login`, `/setup`, and the `POST /api/auth/*` and `POST /api/setup` endpoints. The decorator also runs a CSRF origin check on every POST. 4. If it is a new blueprint, register it in `routes/__init__.py` If your endpoint returns any part of the config, use `_mask_config()` from `routes/config.py` to strip API keys before serialising the response. ### Changing sweep behaviour The sweep logic lives entirely in `sweep.py`. `_sweep_instance` is the shared per-instance worker -- most sweep changes happen there. It accepts an `app` parameter (`"radarr"` or `"sonarr"`) and handles all per-app differences internally via conditional blocks. **Backlog (missing) pipeline filters** The missing search pipeline applies filters in this order before handing items to `pick_items_with_cooldown`: 1. Excluded titles filter 2. Queue filter -- items already actively downloading are skipped 3. `minimumAvailability` filter -- Radarr items not yet past their availability status are dropped 4. Age filter -- Radarr only; items added within `missing_added_days` days are dropped 5. Grace period filter -- items whose release date falls within `missing_grace_hours` are skipped `_release_date(rec)` checks `releaseDate`, `physicalRelease`, `digitalRelease`, `inCinemas`, `airDateUtc`, `airDate` in that order. If you add a new date field to the API response, add it to this helper's field list. **`pick_items_with_cooldown` and max_per_run** `pick_items_with_cooldown` in `stats.py` applies the cooldown filter, sorts by sample mode, and caps the result. `max_per_run=0` means all eligible items are returned. The guard in `_sweep_instance` is `if backlog_enabled:` (not `if backlog_enabled and missing_max > 0:`). Supported sort branches: `random`, `alphabetical`, `oldest_added`, `newest_added`, `round_robin`, `largest_gap_first`. Unrecognised mode strings fall through without sorting, preserving input order. ### Changing database schema 1. Add new columns or tables to `_SCHEMA_SQL` in `nudgarr/db/connection.py` for fresh installs 2. Write a new `_run_migration_vN` function in `nudgarr/db/connection.py` that applies the change to existing databases 3. Call it from `init_db()` in the migration chain 4. Never modify existing migration functions -- they may have already run on user databases ### Changing the UI The frontend is a two-file Alpine.js app -- no build step required. `nudgarr/templates/ui.html` contains the full HTML with Alpine directives. `nudgarr/static/app.js` contains the single `nudgarr()` function that defines all reactive state, computed properties, and methods. `alpine.min.js` is self-hosted -- no CDN dependency. **Alpine.js basics for this codebase** - The root element is ``. Every Alpine directive in `ui.html` resolves against the object returned by `nudgarr()`. - Use `x-show="someCondition"` to show/hide elements. Use `x-model="someProperty"` for two-way input binding. Use `x-for`, `x-bind`, and `@click` / `@input` for loops, attribute binding, and event handlers. - All state lives on the `nudgarr()` object. Do not use global JS variables for reactive state -- Alpine will not track them. - `this.cfg` holds the raw config object returned by `/api/config` (with API keys masked). `applyConfig()` reads from `this.cfg` into flattened state properties for the UI. `_syncFullCfgFromUi()` writes those properties back to `this.cfg` before a save. - The `el()` helper from v4 is gone. Reference elements via Alpine directives, or use `document.getElementById()` only for non-reactive imperatives (e.g. focusing an input after a modal opens). **Panels and navigation** `this.panel` holds the name of the currently visible panel (e.g. `'sweep'`, `'library'`, `'instances'`). Call `navigateTo(name)` to switch panels -- this sets `this.panel` and triggers any data fetch needed for that panel. Each panel section in `ui.html` is wrapped in `