# Security

niche-finder is built to run on your own machine for yourself. Everything
listens on `127.0.0.1`, and nobody else can reach it. Multi-user mode
(`NF_MULTI_USER=1`, plan 15) lets one server work for several people. This file
covers what protects each mode and what you have to do yourself before you let
anyone else in.

## Reporting a problem

Open a private security advisory on GitHub
(Security → Advisories → Report a vulnerability). Do not open a public issue.
Please include the steps to reproduce and what an attacker gains.

Fixes land on `main` and ship in the next release; only the
[latest release](https://github.com/pandich93/youtube-niche-finder/releases/latest)
and `main` are supported.

## Single-user mode (default)

- **No accounts.** The dashboard, API and MCP work for whoever can reach them,
  so they are bound to `127.0.0.1`.
- **Hostile web pages are blocked.** Every request must carry an allow-listed
  `Host` header, which stops DNS rebinding. Every POST, PUT, PATCH or DELETE
  must be JSON or carry `X-NF-Client`, so a cross-site form or a simple
  cross-origin POST is refused. CORS answers only `chrome-extension://`
  origins: the IDs in `NF_EXTENSION_IDS`, or any extension when it is unset.
- **Secrets stay local.** Your YouTube key lives in `.env`. The OAuth refresh
  tokens of your own channels are encrypted with `OWN_TOKENS_KEY` and never
  returned or logged. The key can be rotated: earlier keys go into
  `OWN_TOKENS_KEY_OLD` (read-only), and `cli.py rotate-secrets --apply`
  re-encrypts the stored values. A value no key reads is logged without its
  content and counted on the admin screen
  ([docs/hosting.md](docs/hosting.md)).
- **The YouTube key stays out of URLs and logs.**
  - The key goes in the `X-Goog-Api-Key` header, never in the request URL
    (`_get` in `backend/infrastructure/youtube/client.py`).
  - A network error is re-raised as `YouTubeUnreachable` without the URL.
  - The worker log and HTTP 500 answers pass through
    `backend/infrastructure/redact.py`, which masks `key=`, `access_token=`,
    `refresh_token=` and `client_secret=` parameters, `AIza...` keys and the
    configured secrets.
  - A key that ever appeared in a log (older versions put it in the URL) should
    be rotated in Google Cloud Console.

## Multi-user mode (`NF_MULTI_USER=1`)

What the server does for you:

- **Accounts.**
  - Without mail (`SMTP_HOST`, `SMTP_FROM`) accounts exist only by invitation:
    `cli.py create-user` or the admin screen. With mail and
    `NF_REGISTRATION=open` (the default) anyone can sign up, but only with an
    address they can read: the password is set from a one-time link mailed to
    it, so nobody can take someone else's email.
  - Sign-up and "forgot password" answer the same whether the address is known
    or not. A known address gets a "you already have an account" email with a
    reset link instead of a second account; a blocked one gets nothing.
  - Mailed links (sign-up and email change: 24 hours, reset: 1 hour) are
    random, single use, and the database keeps only their SHA-256
    (`email_tokens`). The token travels in the URL fragment, so it is not in
    server or proxy logs. A new link of the same kind replaces the old one.
  - At most `NF_MAIL_PER_IP_HOUR` (10) emails per client address and
    `NF_MAIL_PER_EMAIL_HOUR` (3) per recipient each hour, so the server cannot
    be used to flood an inbox. The links point at `NF_PUBLIC_URL` or
    `https://NF_DOMAIN`, never at a host a request names, when either is set.
  - Changing the email needs the password, and the change happens only when
    the link mailed to the new address is opened.
  - Passwords are scrypt hashes and must be at least 10 characters. Common
    long passwords (`password123`, `1234567890`, keyboard and alphabet runs,
    one repeated character) and passwords containing the email's name are
    refused. Nothing is sent to a breach service.
  - A wrong password and an unknown email get the same answer.
  - **Guessing is slowed down.** After 5 failed sign-ins for one email in
    15 minutes, the next attempt waits 1 minute, then 2, 4, up to 15. This
    applies to the right password too, so guessing cannot simply continue.
    After 30 failures from one address (any emails), sign-in from it is closed
    for 15 minutes. The answer is HTTP 429 with `Retry-After`, the same for a
    known and an unknown email. There is no permanent lock, which would let
    anyone lock out someone else's account. The counters live in Postgres
    (`login_attempts`), so restarts do not reset them. The limits can be tuned
    with `NF_LOGIN_MAX_FAILS`, `NF_LOGIN_MAX_IP_FAILS` and `NF_LOGIN_WINDOW_MIN`.
- **Two-factor sign-in** (multi-user plan P2-04, optional per account).
  - Turned on in the profile with an authenticator app: TOTP (RFC 6238),
    30-second steps, 6 digits, one step of clock drift either way. The same
    code never works twice: the step of the last accepted code is kept.
  - The secret is encrypted with `OWN_TOKENS_KEY`, like channel tokens and
    notification secrets, and `cli.py rotate-secrets` re-encrypts it too.
    Without the key it cannot be turned on; if the key is lost, the app codes
    stop working and only backup codes or `cli.py disable-2fa` get the person
    back in.
  - The dashboard shows the secret as text and as an `otpauth://` link, not as
    a QR code: drawing one would need an external service or a QR encoder in
    the bundle, and neither is worth it for one screen.
  - Ten backup codes (about 49 bits each) are shown once and kept only as
    scrypt hashes; each works once. New ones need a current code and replace
    the old set.
  - With 2FA on, a right password gives no session, only a one-time
    challenge (SHA-256 in `mfa_challenges`, 5 minutes, dies after 5 wrong
    codes). Wrong codes count in the same `login_attempts` counters as wrong
    passwords, and a right password does not reset the email's counter, so a
    known password does not buy free guesses.
  - Turning it on ends your other sessions; turning it off needs the password
    and a code. A reset by emailed link keeps 2FA: after it comes the code
    step too. `cli.py set-password` creates no session and says that 2FA is
    still on; the operator removes it with `cli.py disable-2fa EMAIL`
    (written to the action log).
  - API tokens do not ask for a code (they are a factor of their own), but
    with 2FA on a token can be created only from a session that passed it.
  - `NF_ADMIN_2FA=required`: an admin without 2FA can sign in but sees only
    the 2FA setup until they turn it on (the API answers 403
    `TwoFactorRequired`); they cannot turn it off. Off by default, so an
    upgrade never locks admins out; it needs `OWN_TOKENS_KEY`.
  - The action log records turning it on and off, failed codes and every
    used backup code.
- **Sessions.**
  - A session is a random token in an `HttpOnly`, `SameSite=Strict` cookie, and
    the database keeps only its SHA-256.
  - Changing a password ends every session of that user (the profile screen
    keeps the browser it was changed in signed in); a reset by link ends all
    of them.
  - A session ends 14 days after the last activity (`NF_SESSION_IDLE_DAYS`)
    and 90 days after the sign-in whatever happens (`NF_SESSION_MAX_DAYS`).
    The idle expiry moves forward at most once per 15 minutes
    (`NF_SESSION_TOUCH_MINUTES`), so a request only reads the database. The
    absolute limit is checked on every request, so lowering it also ends
    older sessions.
  - Every sign-in, sign-up, password reset and password change starts a new
    token; an old one is never taken over. The cookie is not rotated on its
    own while the session lives: it never reaches JavaScript, a URL or a log,
    the idle and absolute limits bound a stolen one, and a mid-session swap
    would race the dashboard's parallel requests (one that left with the old
    token would get 401) or need a grace window during which both tokens
    work.
  - The profile lists your sessions: when they started and were last active,
    the client address (as `NF_TRUSTED_PROXIES` resolves it) and the browser
    (User-Agent, cut to 200 characters). You can end one or all but the
    current. Tokens and hashes are never shown. Only a dashboard sign-in can
    do this, never an API token. These rows go when the session ends and with
    the account.
  - Without a session, `/api` answers 401. Only sign-in, sign-up and password
    reset by link, email confirmation, `/api/health` and the Google OAuth
    return are open.
- **No CSRF token, on purpose.** Two layers stand in for it:
  - The browser never sends the `SameSite=Strict` session cookie on a request
    that starts on another site.
  - Every POST, PUT, PATCH or DELETE must be JSON or carry `X-NF-Client`. Both
    force a CORS preflight, and CORS allows only the extension IDs listed in
    `NF_EXTENSION_IDS` (none when it is unset), so another site or another
    extension cannot send such a request even if a browser mishandled the
    cookie. The project's own extension does not need CORS: it has host
    access to the service the user set.

  A token would guard against the same thing a third time. The API token
  (`Authorization: Bearer`) is never sent by a browser on its own, so it
  carries no CSRF risk. Keep this in mind if you add a route that accepts a
  plain form or a GET that changes data: it would bypass both layers.
- **Personal API tokens.** They are for the extension and for MCP over HTTP
  (`Authorization: Bearer nf_...`).
  - The token is shown once and stored as SHA-256.
  - It can be revoked, and it cannot create or list other tokens.
  - It expires: 90 days by default, chosen when it is made (1 to 365 days, or
    none). An expired token is refused like a wrong one.
  - It does only what it was made for. `read` covers GET routes, `write`
    covers routes that change something, and `mcp` covers MCP over HTTP. A
    script that only reads needs no `write`; the Chrome extension needs
    `read` + `write` (its list badges are POST requests) and no `mcp`.
  - The extension sends its token to a remote service only over `https`.
- **Separated data.** Each user sees and changes only their own:
  - watchlist, swipe file, drafts and transcript queue;
  - alert read marks, connected channels, notification settings and API tokens.

  Public YouTube data (channels, videos, niches, transcripts, alert events) is
  shared. A user sees only the alert events of their own watchlist.

  Two layers keep this apart (multi-user plan P1-02):
  - every query filters by its user (`AND user_id = ?`);
  - **Postgres Row-Level Security** catches a query that forgot to. Every
    personal table (`PERSONAL_TABLES` in
    `backend/infrastructure/postgres/schema.py`) has the policy
    `nf_owner_rows`: a row is visible and writable only when its `user_id`
    is the user the session names. A dashboard or API request (from its
    session or token) and an MCP tool call (from its token, user 1 over
    stdio) run as the role `NF_DB_APP_ROLE` (`<POSTGRES_USER>_app` by
    default), which has no `BYPASSRLS` and owns no tables, and set
    `app.user_id` with a signature `app.user_sig`. The function
    `nf_rls_uid()` accepts the number only with the right signature: SHA-256
    over a random secret in the table `nf_rls_key` (only the owner reads it),
    the number and the Postgres process id of that connection. So a session
    cannot name another user with `set_config`, and a signature seen in
    `pg_stat_activity` is no good in another connection. Without a valid one
    it sees no personal rows at all. Shared alert events (`user_id` empty)
    stay visible to everyone.

  How far this goes depends on `POSTGRES_APP_PASSWORD`
  ([docs/hosting.md](docs/hosting.md#row-level-security)):
  - **Set (recommended):** the dashboard and MCP open the connections for
    people's requests with the login `NF_DB_APP_ROLE` itself. It is not the
    owner, not a member of the owner's role and has no `BYPASSRLS`, so no SQL
    on such a connection gets out from under the policies: `RESET ROLE`
    leaves it the same login, `SET ROLE` to the owner is refused, the secret
    is unreadable, and it is granted no `CREATE` on the schema (on `public`
    that is the default since Postgres 15). Temporary
    tables are dropped each time a connection is handed out, so one request
    cannot plant a look-alike table for the next one. The services set the
    login's password themselves at start.
  - **Not set:** the role is entered with `SET ROLE` from the owner's login,
    as before. That guards against a missing filter, not against someone who
    can run arbitrary SQL (`RESET ROLE`). The server logs a warning.

  RLS does not apply to the table owner (`POSTGRES_USER`). The worker, the
  CLI, migrations, and in the dashboard and MCP sign-in, sign-up, password
  reset, the session and token lookup, the admin routes and admin MCP tools,
  and the shared LLM budget and transcript delivery (`system_scope`) use the
  owner's connections, because they act for every user or before anyone is
  known; there the `user_id` filter in each query is the only guard. These
  are a small, fixed set of queries that take no user-controlled SQL; every
  other request runs on the restricted login.

  Not covered by RLS: account tables without a `user_id` column (`users`,
  `login_attempts`, `audit_log`). A user request can still read them,
  because it reads and changes its own account there. If the role cannot be
  created, requests run as the owner, as before, and the server logs a
  warning.
- **Limits.**
  - YouTube quota: each user has a daily share of the installation's one key
    (`NF_USER_DAILY_UNITS`, `NF_USER_DAILY_SEARCH_CALLS`). Running out stops
    only that user.
  - The LLM budget (`NF_USER_DAILY_LLM_USD`) and the request rate limit also
    count per user.
- **Notifications.**
  - A user's webhook must be https to a public address. It is checked when
    saved and again before every send, and redirects are off, so it cannot
    reach into the server's own network.
  - Bot tokens and webhook addresses are encrypted with `OWN_TOKENS_KEY`.
- **Connecting your own channel.** It can be finished only in the browser that
  started it, thanks to a short-lived state cookie. So nobody can attach your
  channel to their account by sending you a link.

What you have to do before you give anyone an account:

1. **Put it behind HTTPS.** `docker-compose.prod.yml` (`make prod-up`, see
   [docs/hosting.md](docs/hosting.md)) does everything in this point for you.
   By hand: use a reverse proxy (Caddy, nginx) with a real
   certificate. Set `NF_COOKIE_SECURE=1` and add your domain to
   `NF_ALLOWED_HOSTS`. Do not publish port 8080 or Postgres (5433) to the
   internet. Set `NF_TRUSTED_PROXIES` to the proxy's address or subnet (for
   example the compose network, `172.16.0.0/12`). Then the request limit and
   the sign-in limits count each real client from `X-Forwarded-For` instead
   of lumping everyone together as the proxy. Never use `*` (it is ignored):
   anyone could then claim any address.
2. **Set `OWN_TOKENS_KEY`** (a Fernet key) and keep it only in `.env`. Losing
   it makes stored tokens unreadable. Leaking it exposes them. Set
   `POSTGRES_APP_PASSWORD` too, so people's requests use a database login
   that cannot read other people's rows
   ([docs/hosting.md](docs/hosting.md#row-level-security)).
3. **Back up Postgres.** It now holds accounts, sessions and encrypted
   secrets. Protect the backups like the server itself. The production setup
   dumps it on a schedule and checks each dump by a trial restore; copy the
   dumps off the server ([docs/hosting.md](docs/hosting.md#backups)).
4. **Give each person their own account.** Admins are marked `--admin`; over
   HTTP their extra powers are tariffs and accounts: creating, disabling and
   deleting them (`/api/admin/*`, signed-in dashboard sessions only).
   Otherwise accounts are created from the command line on the server.
5. **Read YouTube's API policies for a service others use.**
   - III.D.1.c: one API project per application. That is why the installation
     uses one key, and users do not bring their own.
   - III.E.4.d: non-authorized YouTube data may be kept at most 30 days, then
     refreshed or deleted. The worker re-reads every stored video and channel
     not refreshed for 25 days (`REFRESH_STALE_DAYS`, under a daily cap), so
     current rows stay fresh. But niche-finder never deletes anything: the
     stats history, title and thumbnail changes and archived thumbnails are
     kept without limit.
   - Keeping statistics longer, and computing scores from them, is allowed
     only to API clients approved under YouTube's derived-metrics policy
     (statistics and derived metrics up to 36 months; titles, descriptions
     and other text still 30 days):
     <https://developers.google.com/youtube/terms/derived-metrics-policy>.
     The approval is per Google Cloud project, so each installation's owner
     applies for their own.
   - For yourself this is your own risk. For a service other people use, you
     must solve it before you open the service.
6. **Letting customers connect their own channels (plan 25).** Use
   `OWN_OAUTH_MODE=web` with one OAuth client of type Web application and an
   https `OWN_OAUTH_REDIRECT_URI`; fill `NF_SERVICE_NAME`, `NF_OPERATOR_NAME`
   and `NF_CONTACT_EMAIL` so `/privacy` and `/terms` name you and a contact.
   Pass Google's OAuth app verification before inviting more than 100 people,
   and answer deletion requests sent to that contact (the privacy page promises
   7 days): `cli.py delete-user EMAIL --yes`, or Delete on the Admin page.
   People can also delete their own account on the Plan page. Back up first
   if you delete many at once: deleting cannot be undone.
7. **Keep MCP over HTTP behind the TLS front door (`mcp-https`)** and set
   `MCP_PUBLIC_URL` to the address clients use. MCP over stdio is local only.

Known limits (by design, worth knowing before you invite anyone):

- **Shared data has an owner.** A video's transcript (one shared copy) and its
  curated tags are shared by everyone. Any signed-in user may add them, but
  only the person who added a transcript or a tag, or an admin, may replace,
  re-index or remove it; pasting over someone else's transcript returns the
  existing one (`alreadyExists`). Rows from before this rule, the worker and
  the CLI have no owner and are the admins'. Accepting or rejecting the
  background LLM's proposed tags is for admins. Naming niches by collecting
  them is still open to everyone.
- **The server's own work is for admins.** Refreshing stats, AI enrichment,
  the alert scan, thumbnail vectors and recomputing the niche map run on the
  worker's schedule; starting them by hand (`/api/refresh`, `/api/enrich/*`,
  `/api/events/scan`, `/api/thumbnails/embed`, `/api/niche-clusters/recompute`
  and the MCP tools `refresh_stats`, `enrich_channels`, `tag_new_videos`,
  `scan_for_alerts`, `embed_thumbnails`) needs an admin, by session or API
  token. Multi-user plan P2-11 lets an admin open two of them to regular users
  through the tariff limit `manual_runs` (off by default,
  `NF_USER_MANUAL_RUNS_PER_HOUR=0`): `/api/refresh` and `/api/events/scan` then
  run over that user's own watchlist only, on their own quota, at most that
  many times an hour (`429` with `Retry-After` past it, counted in Postgres
  under a lock); the rest, and the MCP tools, stay admin-only
  ([docs/manual-runs.md](docs/manual-runs.md)). The first author of a transcript can still paste junk or a prompt
  injection into it, and the LLM hook review of that text is cached for
  everyone; an admin can replace it.
- **Admins.** `users.is_admin` guards `/api/admin/*` (tariffs, users'
  tariffs, creating, blocking and deleting accounts): only an admin signed in
  to the dashboard, never an API token. Admins are not limited by tariffs.
  Blocking an account (`users.disabled_at`) ends its sessions and API tokens
  (MCP included) at once, and it cannot sign in until unblocked. An admin
  cannot block or delete themself there, the `local` user (the data from
  before accounts) can be blocked but not deleted, and the server always
  keeps one active admin. Each action goes into the action log.
- **Action log** (multi-user plan P1-07). `audit_log` records sign-ins, failed
  sign-ins (also the throttled ones), sign-ups, password changes and resets,
  email changes, API tokens created and revoked, tariff changes, and admins'
  actions on accounts and tariffs -- with the time, who acted, on whom and
  the client address (`NF_TRUSTED_PROXIES` decides it, as for the sign-in
  throttle). Details pass a filter that drops any key that looks like a
  credential (password, token, secret, hash, key, cookie, auth), so neither a
  password nor a token nor a reset link can end up there; an unknown email
  typed at sign-in is not stored either. Only an admin signed in to the
  dashboard reads it (`GET /api/admin/audit`, the "Admin" page with filters by
  event, user and period); an API token cannot, even an admin's. Entries
  survive a deleted account and leave after `NF_AUDIT_RETENTION_DAYS`
  (default 180). The rows of P0-04's `admin_actions` were moved into it on
  the first start, and that table is gone.
- **Support view** (multi-user plan P2-05). An admin signed in to the
  dashboard can look at an account through its owner's eyes ("Посмотреть
  как" on the "Admin" page, `POST /api/admin/users/{id}/impersonate`) to
  answer a support question. It is a mark on the admin's own session
  (`sessions.impersonating_user_id`), never on an API token or MCP, and:
  a reason is required; not on another admin, a blocked account or oneself;
  not nested; it lasts `NF_IMPERSONATION_MINUTES` (30 by default) and then
  the next request clears it. While it lasts, `_uid` and Row-Level Security
  (`app.user_id`) are the account's, not the admin's, admin routes are closed
  (the account is no admin), and so are the account's session list, API
  tokens and every profile change. **Every request but GET/HEAD/OPTIONS is
  refused with 403** (`ReadOnlyView`) except leaving the view
  (`POST /api/auth/impersonation/end`) and signing out. Spending is frozen:
  the YouTube client refuses before reserving a unit and the LLM gateway
  answers from its cache only, so a view uses nobody's quota and starts
  nothing. The start (who, whom, why, until when), the end and the expiry
  are in the action log; blocking the account, deleting it, making it an
  admin or removing the admin's role ends the view too. The dashboard shows
  a banner "Вы смотрите как ... — только чтение" on every screen.
- **Tokens made before expiry existed do not expire.** New tokens live 90 days
  by default (1 to 365 days, or no expiry if you choose it). Older tokens keep
  every scope and no expiry until revoked; the MCP screen marks them
  "no expiry". Changing a password revokes all of that user's tokens and
  sessions.
- **MCP over HTTP needs `NF_MULTI_USER=1` to ask for a token.** Without that
  flag anyone who reaches it acts as user 1, so `server.py` refuses to listen
  on anything but loopback (`MCP_HOST` defaults to `127.0.0.1`) unless
  `MCP_ALLOW_UNAUTHENTICATED=1` says that is intended. The compose `mcp-http`
  service sets it: its port is not published, and only `mcp-https`, which
  listens on 127.0.0.1, reaches it. The production file turns it off.
- **Background work is shared; what you see is yours.** Thumbnail
  fingerprints, title and thumbnail swaps and stats refreshes run for every
  channel someone tracks. With accounts on, the feeds built from them show
  only your own watchlist: title changes, the repackaging feed (dashboard and
  MCP), the watchlist and change totals, and the digest. In the digest,
  rising channels are limited to your channels and the niches they are in.
  `/api/health` tells a visitor who has not signed in only that the server is
  up. What stays shared is the public corpus itself: channels, videos and
  niches anyone collected, which is the point of one shared collection.

Security reviews: the plan-14 OAuth flow and each plan-15 sub-stage were
reviewed. The review of sign-in found a way to attach someone else's channel
through the OAuth callback, and it was fixed (see CHANGELOG, "Fixed").