# Backend architecture This file contains an overview of the most important parts of the backend architecture. ## Request, library and downloads modules These modules drive the requests, files management and downloads. Requests don't know of downloads or the library, and are only responsible for reacting to users requests and watch status changes, managing which requests are active, and notifying the library of changes. Library matches requests to files on disk, or tells downloads module if a file is missing, as far as the quota of the user and the disk limit allow. It also takes care of disk cleanup and housekeeping. ```mermaid flowchart LR subgraph requests["Requests: what a user wants on disk"] RS["RequestsService"] RQ[("request")] RR[("request_rule")] SR[("selected_release")] end subgraph library["Library: the hub"] FS["FulfilmentService"] IS["ImportService"] DEL["DeletionService"] RB[("request_binding")] MF[("media_file")] end subgraph downloads["Downloads: torrents and what is wanted of them"] DS["DownloadsService"] T[("torrent")] TI[("torrent_item")] end requests -.->|"library.reconcileBinding
{requestId}"| library library -->|"deleteRequests"| requests library -->|"ensureTorrent
requestTorrentItem
releaseTorrentItem"| downloads downloads -.->|"library.importTorrentItem
{infoHash, torrentItemId}
library.sweepRequests"| library ``` ## Request creation to fulfillment overview The sequence of a request that goes well, from a user picking a release to the file playing: solid arrows are calls, dashed arrows are the jobs that carry the events between the modules, named by their type. ```mermaid sequenceDiagram actor User participant Requests participant Library participant Downloads participant Engine as Torrent engine User->>Requests: requests an episode, picks a release for its season Requests->>Requests: resolves the release into selected_release, reconciles the requests, the request is needed Requests-->>Library: library.reconcileBinding {requestId} Library->>Requests: readRequest, findSelectedReleases Library->>Library: the expected size of the download fits the quota of the user, and no disk is at the limit Library->>Downloads: ensureTorrent, requestTorrentItem Library->>Library: binds the request to the torrent item Downloads-->>Downloads: downloads.reconcileTorrent {infoHash} Downloads->>Engine: start Engine-->>Downloads: downloads.reconcileTorrent, the files are known Downloads->>Downloads: matches the torrent item to its file Downloads->>Engine: setWantedFiles Engine-->>Downloads: downloads.reconcileTorrent, the file is complete Downloads->>Downloads: stamps the torrent item complete Downloads-->>Library: library.importTorrentItem {infoHash, torrentItemId} Library->>Downloads: getTorrentItemFile Library->>Library: imports the file, hard link else copy, as a managed version, and binds the request to it Library->>Downloads: releaseTorrentItem Downloads-->>Downloads: downloads.reconcileTorrent {infoHash} Downloads->>Engine: seeds up to the ratio, then stops and deletes the data User->>Library: plays the file ``` ## The request lifecycle The life of a request row, from a user's wish to its deletion: a request is inactive until it passes gates, such as user has picked a source and the episode is needed. An active request gets what it wants when it fits the quota of its user, and a download only while no disk is at the limit; else it is held back, which is to say bound to nothing, until the sweep has it looked at again once room may have been made. ```mermaid stateDiagram-v2 state "No source" as nosource state "Deleted, the row is gone" as deleted [*] --> Approved: a user requests an episode state Approved { [*] --> nosource nosource --> Ahead: a release or a file is picked, outside the download-ahead window (not needed yet) nosource --> Active: a release or a file is picked, within the window Ahead --> Active: the window reaches it (needed, one-way) Active --> nosource: the source is cleared, unless the request has its file state Active { state "No file in torrent" as nofile state "Held back (over quota, disk full)" as heldback [*] --> Resolving Resolving --> Downloading: the torrent has a file for the episode Resolving --> nofile: the torrent has no file for it Resolving --> Failed: the torrent failed, or the selected file is not in the library Resolving --> Available: its file is in the library already Resolving --> heldback: what it is to get does not fit the quota of the user, or it is a download and a disk is at the limit heldback --> Resolving: room may have been made, so the sweep has it looked at again Downloading --> Available: imported into the library nofile --> Resolving: another release is picked Failed --> Resolving: the release is picked again, or another one Available --> Resolving: its file is forgotten by a scan (deleted externally), or another file or torrent is picked for the episode itself } } Approved --> deleted: watched, and no keep setting retains it Approved --> deleted: unrequested, or its file is deleted by an admin deleted --> [*] ``` ## The torrent lifecycle The lifecycle of a torrent in downloads, from the library ensuring it for a selected release until its data goes: the reconcile job decides every transition from what is wanted of the torrent, what the engine reports and the seed ratio, and the row, keyed by info hash, stays until no torrent item refers to it. ```mermaid stateDiagram-v2 state "Idle, no data" as idle state "Stopped, with data" as stopped [*] --> idle: the library ensures the torrent of a selected release idle --> Starting: a torrent item is wanted Starting --> Downloading: the engine knows the files, and a wanted torrent item has one Starting --> stopped: the files are known, and no wanted torrent item has one Downloading --> stopped: every wanted file is complete, the torrent items wait for their import Downloading --> Seeding: every torrent item is released (imported, or let go), and the ratio is not reached stopped --> Seeding: every torrent item is released, but the ratio is not reached stopped --> Starting: a torrent item is wanted again stopped --> idle: every torrent item is released, and seeding is done, so its data is deleted Seeding --> Downloading: a torrent item is wanted again Seeding --> idle: the seed ratio is reached, so its data is deleted Starting --> Failed: the engine refuses the source, or reports an error Downloading --> Failed: the engine reports an error Seeding --> Failed: the engine reports an error Failed --> Starting: the library ensures it again, when the release is picked again or a request of it becomes active again Failed --> idle: nothing wants it any more, so its data is deleted idle --> [*]: purged at night, when no torrent item refers to it ``` ## The library file lifecycle The lifecycle of a file in the library's index, from a scan finding it or an import placing it until its row is deleted: a file a scan found is external and never the server's to delete, an imported file is managed and stays as long as a request holds it, after which the cleanup deletes it. ```mermaid stateDiagram-v2 state "External" as external state "Gone, the row is deleted" as gone state "Managed" as managed { state "Held by a request" as held state "Unretained" as unretained unretained --> held: a request is bound to it, as the version of its torrent, as a selected file, or from the server page held --> unretained: the last request that had it is deleted, or lets go of it for another source } [*] --> external: a scan finds a video file in a library folder [*] --> held: imported from a torrent for the requests that wanted it, as a version named after the torrent external --> held: an admin marks it managed, and a request has it external --> unretained: an admin marks it managed, and no request has it managed --> external: an admin marks it external unretained --> external: the cleanup finds it changed since its import, or outside the library folders, and keeps it unretained --> gone: the cleanup deletes it, on a cleanup day or when an admin runs it managed --> gone: an admin deletes it with the requests that have it, or a scan finds it gone and forgets it, so that its requests download it again external --> gone: an admin deletes it, or a scan finds it gone and forgets it gone --> [*] ``` ## Quotas Quotas and storage limits are enforced server side only when a request is binding (FulfilmentService.reconcileBinding), not in any route or request creation time. The frontend has a separate client side check to enforce quotas in the UI.