# Architecture kei is a Rust CLI that transfers iCloud Photos media and metadata to local storage. This guide identifies the owner of each major decision and the boundaries that protect user data. Use it as a starting map. Read the owning module and its direct callers before changing behavior. When statements overlap, the Owners table defines ownership, the Safety contract catalog defines required conditions, and source code wins over both. ## Navigation This index lists the highest-traffic sections, not every heading. Link labels match their headings, so searching a label finds the section. - [Owners](#owners): find the decision owner and its boundary. - [Design rules](#design-rules) and [Wrong-layer traps](#wrong-layer-traps): check layer placement before editing. - [Command dispatch](#command-dispatch) and [Container startup](#container-startup): find entry points. - [Sync and provider checkpoints](#sync-and-provider-checkpoints) and [Full and incremental enumeration](#full-and-incremental-enumeration): trace source progress. - [Download and publication](#download-and-publication), [Durable pending retry](#durable-pending-retry), and [Import-existing](#import-existing): trace local work. - [Account ownership and legacy adoption](#account-ownership-and-legacy-adoption), [Transactional source catalog and replay](#transactional-source-catalog-and-replay), and [Current provider work admission](#current-provider-work-admission): trace durable source ownership. - [Selection generation shadow](#selection-generation-shadow), [Private selection and destination replay](#private-selection-and-destination-replay), and [Observed incremental shadow capture](#observed-incremental-shadow-capture): trace selection and destination projections. - [Data-safety invariants](#data-safety-invariants), [Safety contract catalog](#safety-contract-catalog), and [Secrets and diagnostics](#secrets-and-diagnostics): find the required safety conditions. - [Change-impact traversal](#change-impact-traversal) and [Change-impact checklist](#change-impact-checklist): find related owners and consumers before editing. - [Tests](#tests): find proof entry points and the test guide. - [Managed edited-primary layouts](#managed-edited-primary-layouts) and [Support evidence boundary](#support-evidence-boundary): trace specialized flows. - [Maintaining this guide](#maintaining-this-guide): update this file with the change. ## Design rules - User media and metadata must not be lost, corrupted, truncated, overwritten, or silently discarded. - Local file and metadata rewrites are opt-in. - Provider-specific parsing and identity rules stay in the iCloud adapter. - Sync policy stays in the sync and download orchestration layers. - Path rendering does not decide whether an asset should sync. - SQLite transitions and provider checkpoints must survive interruption. - Prefer the smallest complete implementation in the owning module. ### Wrong-layer traps - `src/commands/service.rs` plans provider libraries and collection passes. Platform service installation and execution belong in `src/service/`. - `src/selection.rs` parses selectors. Live collection lookup and pass planning belong in `src/commands/service.rs`; path rendering cannot resolve them. - `src/sync_loop/precheck.rs` handles the database change pre-check. Zone checkpoint decisions belong in `src/sync_cycle.rs`, not in the Photos adapter or file-transfer code. - Facades such as `src/download/mod.rs`, `src/download/file.rs`, and `src/state/db.rs` expose entry points. Read the child owner in the Owners table before adding behavior or looking for its tests. `src/state/db.rs` also owns connection lifecycle and blocking-pool dispatch; keep that work there. - Reports, health, metrics, and support diagnostics consume observations. They do not supply provider completeness or checkpoint authority. ## Owners | Area | Owner | Boundary | |------|-------|----------| | Process startup and dispatch | `src/lib.rs` | Starts the runtime, resolves bootstrap paths, configures logging, dispatches commands, and maps exit codes. | | CLI shape | `src/cli.rs` | Defines clap arguments and parsing. It does not execute command behavior. | | Runtime configuration facade | `src/config.rs` | Preserves configuration types, entry points, and visibility. | | Configuration input | `src/config/input.rs` | Owns the TOML schema and file loading. | | Runtime policy types | `src/config/runtime.rs` | Owns resolved configuration types, media selection, and date-bound semantics. | | Configuration resolution | `src/config/resolve.rs` | Resolves TOML, environment, and command inputs into runtime policy, including shared sync/import path fields. | | Configuration paths | `src/config/paths.rs` | Resolves bootstrap, data, and credential paths and validates download directories. | | Folder-template validation | `src/config/templates.rs` | Owns default category templates and validates token placement. | | Configuration persistence | `src/config/persistence.rs` | Projects runtime policy to TOML and writes first-run configuration. | | Selection grammar | `src/selection.rs` | Parses album, smart-folder, library, exclusion, and unfiled selectors. | | Sync/watch facade and runner | `src/sync_loop.rs`, `src/sync_loop/runner.rs` | Preserves command-facing entry points; the runner composes startup, sync cycles, and watch control. | | Sync session recovery | `src/sync_loop/session.rs` | Coordinates authentication, 2FA recovery, library initialization, and idle session-lock handoffs through the shared auth and command owners. | | Database change pre-check | `src/sync_loop/precheck.rs` | Owns scoped database tokens, selected-zone checks, and inclusion of pending local work. It does not replace the per-zone checkpoint gate. | | Sync plan refresh | `src/sync_loop/planning.rs` | Resolves and refreshes library pass plans through the command owner; reports shared-library notices and path warnings. | | Local drift checks | `src/sync_loop/reconcile.rs` | Runs bounded local drift recovery and periodic read-only catalog diagnostics. | | Watch policy | `src/sync_loop/watch.rs` | Owns watch cadence, metadata follow-up timing, and one-shot recovery rules. | | Cycle ledger reporting | `src/sync_loop/reporting.rs` | Records cycle-run summaries and combines refresh-tail outcomes for reporting. | | One sync cycle | `src/sync_cycle.rs` | Chooses source enumeration, reconciles config drift, dispatches each library, and advances or preserves provider checkpoints. | | Library and pass planning | `src/commands/service.rs` | Resolves libraries, collection scope, album plans, smart folders, unfiled passes, and cross-zone hydration. | | iCloud Photos adapter | `src/icloud/photos/` | Owns CloudKit records, queries, change streams, provider identity, albums, smart folders, and metadata decoding. | | Album facade and counts | `src/icloud/photos/album.rs`, `src/icloud/photos/album/counts.rs` | Preserves album configuration and entry points; counts owns count queries and same-library batching. | | Targeted provider lookup | `src/icloud/photos/album/lookup.rs` | Resolves requested records and merges identity evidence conservatively. | | Provider enumeration | `src/icloud/photos/album/planning.rs`, `src/icloud/photos/album/enumeration.rs` | Planning selects rank ranges and page sizes; enumeration starts fetchers and delivers completion tokens. | | Provider page fetching | `src/icloud/photos/album/fetch.rs` | Owns query construction, empty-page probes, record pairing, deduplication, and asset emission. | | Enumeration completion | `src/icloud/photos/album/completion.rs` | Collects fetcher completion evidence and requires unanimous tokens before returning a checkpoint candidate. | | Album hydration | `src/icloud/photos/album/hydration.rs` | Matches named-album members across source zones and hydrates durable asset or master identities. | | Provider change streams | `src/icloud/photos/album/changes.rs` | Owns sequential change scans, delta buffering, and last-good-token delivery. | | Download facade and dispatch | `src/download/mod.rs`, `src/download/orchestration/dispatch.rs` | Preserves download entry points and composes full, incremental, targeted-backfill, and durable-retry work. | | Download models and configuration | `src/download/orchestration/models.rs`, `src/download/orchestration/config.rs` | Owns controls, results, per-zone checkpoint evidence, reporting projections, checkpoint reasons, coverage fingerprints, and configuration hashes. | | Download context and selection | `src/download/orchestration/context.rs`, `src/download/orchestration/selection.rs` | Loads library-scoped state and identity evidence, derives pass configurations, and checks incremental routing eligibility. | | Full enumeration | `src/download/orchestration/full.rs` | Owns bounded enumeration, album snapshots, pass counts, and token evidence. | | Incremental enumeration | `src/download/orchestration/incremental.rs`, `src/download/orchestration/delta.rs` | Runs separate streaming and collecting strategies with one shared delta-state owner for provider-event bookkeeping and routing facts. | | Download recovery | `src/download/orchestration/recovery.rs`, `src/download/orchestration/url_refresh.rs` | Runs durable pending recovery and refreshes exact download tasks with current provider URLs. | | Local catalog maintenance | `src/download/orchestration/reconciliation.rs`, `src/download/orchestration/cleanup.rs`, `src/download/orchestration/maintenance.rs` | Reconciles catalog paths, removes durably owned stale temporary files, and repairs metadata-capture revisions. | | Asset planning | `src/download/planner.rs` | Applies filters, derives tasks, records dispatched pending work, and persists membership and identity mappings. | | Filter facade and configuration | `src/download/filter.rs`, `src/download/filter/config.rs` | Preserves filter entry points and shares path and filter settings between sync and import. | | Asset eligibility | `src/download/filter/eligibility.rs` | Owns content, date, and filename filters and media classification, without path or filesystem decisions. | | Rendition selection | `src/download/filter/versions.rs` | Owns RAW alignment, primary and companion selection, and selected-rendition metadata keys. | | Task metadata | `src/download/filter/metadata.rs` | Builds metadata payloads with preloaded album and people groupings. | | Expected asset paths | `src/download/filter/expected_paths.rs` | Derives paths shared by sync and import before collision resolution; uses the rendition-selection owner. | | Collision planning | `src/download/filter/collisions.rs` | Owns normalized path claims and existing-file collision resolution. | | Download task derivation | `src/download/filter/tasks.rs` | Composes eligibility, expected paths, collision results, and metadata into download tasks. Path rendering stays in `src/download/paths.rs`. | | Download pipeline facade | `src/download/pipeline.rs`, `src/download/pipeline/streaming.rs` | Preserves entry points and composes run modes, producer execution, consumer execution, and pass finalization. | | Local-file adoption | `src/download/pipeline/adoption.rs` | Validates current and pending local-file evidence before adoption or an on-disk skip. | | Streaming producer | `src/download/pipeline/producer.rs` | Plans assets, forecasts disk space, records skip counts, and dispatches pending tasks. | | Streaming consumer | `src/download/pipeline/consumer.rs` | Runs bounded workers and accumulates transfer results and deferred state writes. | | Single-task execution | `src/download/pipeline/task.rs` | Coordinates one transfer, metadata completion, temporary-file ownership, and worker error classification. | | Download pass and outcome | `src/download/pipeline/pass.rs`, `src/download/pipeline/outcome.rs` | Executes explicit task passes, finalizes streaming state, retries failed tasks, and aggregates sync outcomes. | | Download progress and summary | `src/download/pipeline/progress.rs` | Formats durations and sync summaries and reports rate-limit pressure. | | File facade and transfer | `src/download/file.rs`, `src/download/file/transfer.rs` | Preserves entry points and owns HTTP retry, resume, stream writes, and temporary-file cleanup. | | File validation and fingerprints | `src/download/file/validation.rs`, `src/download/file/fingerprint.rs` | Validates response lengths and media, checks local size evidence, and captures same-read hashes and snapshots. | | File publication and replacement | `src/download/file/publication.rs`, `src/download/file/replacement.rs` | Handles no-overwrite collisions and conditional replacement with displaced-file verification and restoration. | | Confined local copies | `src/download/file/reconciliation.rs` | Retains verified files and parent-directory capabilities through local copy and state finalization. | | File platform primitives | `src/download/file/platform.rs` | Owns platform rename, exchange, hard-link, and directory durability operations. | | Linux replacement recovery | `src/download/file/replacement_recovery.rs` | Journals conditional replacement when atomic exchange is unsupported, and recovers interrupted publication without overwriting concurrent edits. | | Staging identity | `src/download/file/staging.rs` | Coordinates destination guards, versioned staging names, and cleanup leases so retries and collisions keep independent staging files. | | Managed layout file handover | `src/download/file/primary_layout.rs` | Snapshots layout fingerprints, copies verified members, publishes, recovers prepared links, and retires aliases for a confined managed layout. Transition policy stays in `src/download/orchestration/primary_layout.rs`. | | State finalization | `src/download/finalize.rs` | Persists downloaded or failed outcomes and retries deferred state writes. | | Sparse identity retries | `src/download/orchestration/delta/sparse_identity.rs`, `src/state/db/sparse_identity.rs` | Selects bounded source retries and persists generation-fenced evidence; the cycle owner retains checkpoint authority. | | Durable retry resolution | `src/download/retry.rs` | Revalidates pending provider identity and builds exact retry tasks. | | Path rendering | `src/download/paths.rs` | Expands folder templates, normalizes names, and handles collision suffixes. | | Metadata facade and values | `src/download/metadata.rs`, `src/download/metadata/values.rs`, `src/download/metadata/xmp_fields.rs` | Preserves writer entry points and shared values, with one owner for managed XMP properties, namespace initialization, and field encoding. | | Metadata evidence | `src/download/metadata/probe.rs`, `src/download/metadata/source_gps.rs` | Probes existing EXIF/XMP fields and performs bounded source EXIF GPS decoding for write planning. | | Embedded writer dispatch | `src/download/metadata/embedded.rs`, `src/download/metadata/formats.rs` | Validates approved input and selects a format writer using content and extension detection. | | Format-specific metadata preparation | `src/download/metadata/heif_writer.rs`, `src/download/metadata/xmp_writer.rs`, `src/download/metadata/native_writer.rs` | Prepares HEIF-family, XMP Toolkit, or native-only EXIF output. Publication stays in the prepared-file owner. | | Embedded replacement publication | `src/download/metadata/prepared.rs` | Owns exclusive temporary files, fingerprints, and stable-input publication through the file replacement primitives. | | Sidecar ownership and publication | `src/download/metadata/sidecar.rs` | Prepares sidecars, validates ownership, and conditionally publishes writes, including reconciled sidecars. | | HEIF facade and file-backed reads | `src/download/heif.rs`, `src/download/heif/file.rs` | Preserves HEIF entry points and typed errors; locates native Exif with bounded file reads without loading media payloads. | | HEIF layout | `src/download/heif/boxes.rs`, `src/download/heif/items.rs` | Owns box boundaries, item tables, extent resolution, and item-location encoding. | | HEIF metadata relationships | `src/download/heif/relationships.rs` | Resolves primary-image metadata ownership and encodes item references. Ambiguous ownership fails closed. | | HEIF native Exif | `src/download/heif/exif.rs` | Extracts native Exif and repairs capture timestamps, offsets, and paired subseconds within existing TIFF storage. Unsafe or insufficient storage leaves repair pending; native values are verified before publication. | | HEIF XMP | `src/download/heif/stream.rs`, `src/download/heif/xmp_write.rs` | Reads the selected packet and plans bounded item-map replacements without re-encoding image payloads. The in-memory `xmp_read` remains a test and fuzz oracle. | | HEIF streaming | `src/download/heif/stream.rs` | Reads bounded control data and metadata packets, streams unchanged file ranges, and compares protected payloads with fixed buffers. | | HEIF preservation checks | `src/download/heif/stream.rs`, `src/download/heif/preservation.rs` | Streams production checks of non-XMP payloads and opaque metadata; the in-memory validator remains a test and fuzz oracle. | | Metadata rewrite facade and immediate execution | `src/download/metadata_rewrite.rs`, `src/download/metadata_rewrite/immediate.rs` | Preserves rewrite entry points and coordinates download-time metadata writes. The pipeline retains media publication and state finalization. | | Metadata write planning | `src/download/metadata_rewrite/planning.rs` | Owns opt-in flags and embedded field decisions shared by immediate and queued execution. | | Metadata write execution | `src/download/metadata_rewrite/embedded.rs`, `src/download/metadata_rewrite/sidecar.rs` | Coordinates fingerprint-guarded embedded writes and source-aware sidecar writes through the metadata owners. | | Queued metadata retries | `src/download/metadata_rewrite/queued.rs` | Tags durable rewrite work, executes bounded retry pages, and retires completed markers. | | Rewrite evidence and groupings | `src/download/metadata_rewrite/capture.rs`, `src/download/metadata_rewrite/grouping.rs` | Verifies capture-repair fingerprints and timestamps; loads current library-scoped groupings for queued work. | | Account namespace | `src/account.rs` | Hashes exact configured login and provider realm without alias inference. | | Account database boundary | `src/state/db/account.rs`, `src/commands/migrate_state.rs` | Validates an independent owner before state consumption, creates authenticated state, and explicitly adopts a preserved legacy snapshot. | | SQLite facade and connection lifecycle | `src/state/mod.rs`, `src/state/db.rs` | Preserves state entry points, opens connections, and dispatches blocking SQLite work. Child owners retain their transaction boundaries. | | Observed provider pages | `src/icloud/photos/inbox.rs`, `src/state/db/provider_inbox.rs` | Binds validated incremental observations to the account, retains source identities and original bytes atomically with a shadow receipt, and applies bounded backpressure. It does not authorize provider checkpoints. | | Provider source catalog | `src/icloud/photos/projection.rs`, `src/state/db/provider_catalog.rs` | Validates replay provenance, derives versioned source facts, and atomically indexes records, references and unresolved evidence. Its receipt does not authorize queues or checkpoints. | | Selection shadow manifests | `src/state/db/provider_selection.rs`, `src/download/orchestration/queue_projection.rs` | Retains account-bound current planning evidence and per-pass destinations, then validates lossless replay. This shadow receipt has no selection, publication or checkpoint authority. | | Provider work admission | `src/icloud/photos/album/work.rs`, `src/download/orchestration/queue_projection.rs`, `src/state/db/provider_work.rs` | Confirms retained identities against current provider records, applies current task planning, and atomically admits guarded existing queue obligations and retained conflict debt. Checkpoint authority stays with the existing cycle. | | SQLite schema | `src/state/schema.rs` | Owns schema versions and migrations. | | State-store contracts | `src/state/db/contracts.rs` | Defines store roles and records exchanged with callers. | | Asset state transitions | `src/state/db/assets.rs` | Owns asset lifecycle, download finalization, retry eligibility, and provider source-state transitions. | | Shared SQLite writes and row decoding | `src/state/db/asset_writes.rs`, `src/state/db/rows.rs` | Shares asset writes within caller-owned transactions, column projections, date codecs, and row decoding. | | Metadata capture retries | `src/state/db/metadata_capture_retry.rs` | Stores generation-fenced ambiguity evidence and bounded retry deadlines. The cycle owner retains checkpoint authority. | | Durable provider identity | `src/state/db/identity.rs` | Stores library-scoped asset/master mappings and legacy state ownership. Provider record parsing stays in the iCloud adapter. | | Durable membership | `src/state/db/membership.rs` | Stores album snapshots, provider relations, and grouping projections. | | Durable metadata work | `src/state/db/metadata.rs` | Stores capture revisions, path-specific rewrite work, and repair receipts. It does not write media files. | | Durable checkpoints | `src/state/db/checkpoints.rs` | Stores provider checkpoints, scoped database tokens, and enumeration progress. Sync policy decides when checkpoint proof permits a commit. | | Temporary-file ownership | `src/state/db/temp_files.rs` | Stores losslessly encoded paths and durable temporary-file ownership. Filesystem cleanup stays in download orchestration. | | Durable reconciliation | `src/state/db/reconciliation.rs` | Stores destination reservations and reads catalog evidence for reconciliation. | | Durable import adoption | `src/state/db/import.rs` | Atomically adopts imported files into state and reads imported records. Matching policy stays in the import command. | | Durable layout journals | `src/state/db/primary_layout.rs` | Stores preservation-backed layout bindings, operations, members, path claims, and immutable historical receipts. | | Prior publication recovery | `src/state/db/pending_publication.rs` | Recovers a rendition's unchanged prior current publication and retains conditional recovery evidence. | | Legacy preservation evidence | `src/state/db/legacy_preservation.rs` | Stores immutable legacy evidence and generation-fenced current-coverage receipts. It never assigns identity, writes files, or infers provider completion. | | State reports | `src/state/db/reports.rs` | Reads status, verification and manifest data, and records sync-run history. | | Import-existing | `src/commands/import.rs` | Matches existing files to expected iCloud paths and adopts verified files into state. | | Service integration | `src/service/` | Owns install, uninstall, status, service execution, and platform renderers. | | Support evidence | `src/support/`, `src/state/db/support.rs` | Allowlisted bounded runtime history and strictly offline immutable collection; see [support export](support-export.md). | | Operator surfaces | `src/commands/status.rs`, `src/commands/doctor.rs`, `src/commands/manifest.rs` | Read local state for status, redacted diagnostics, and catalog export. | | Reports and monitoring | `src/cycle_reporter.rs`, `src/report.rs`, `src/health.rs`, `src/metrics.rs`, `src/notifications.rs` | Converts cycle facts into reports, health, metrics, and notifications. | Album tests use `icloud::photos::album::::tests::` instead of `icloud::photos::album::tests::`. The owners are `lookup`, `counts`, `planning`, `enumeration`, `fetch`, `completion`, `hydration`, and `changes`. All 88 original tests retain their names, attributes, and assertions, including the ignored live lookup test. Shared fixtures stay in `src/icloud/photos/album/test_support.rs`. Album tracing keeps the `kei::icloud::photos::album` target. Album-owner dependencies are one-way: hydration uses enumeration, lookup and change scans; enumeration uses planning, fetching, and completion; fetching uses planning types and completion evidence. Lookup, counts, planning, completion, and changes do not call sibling owners. The facade retains stream routing and public entry points. No album child owns durable checkpoint policy. Targeted lookup rejections retain fixed `master_status` and `child_status` response categories in bounded support history. These distinguish omitted, not-requested, returned, tombstoned and provider-error members without storing raw codes, messages or identifiers. `returned` describes response presence, not owner, identity, resource or checkpoint proof. Lookup resolution and checkpoint policy do not consume these diagnostic categories. `child_reference_status` separately describes a missing, malformed, sparse, matching or different reference, or an omitted, unavailable, invalid or unrequested child. These fixed labels expose no reference values and do not establish a usable pair. Best-effort album identity hydration has task-scoped anonymous progress in `icloud/photos/hydration_progress.rs`. Every 30 seconds and on return or drop, it reports fixed phase labels, source ordinal/role (`owner` then `fallback`), aggregate received-page and record counts, request attempts/retries, and current-wait elapsed time. Counts are observations, not completeness or checkpoint evidence. The scope does not change request timeouts, retry policy, source priority, or identity acceptance; provider identifiers, cursors, request bodies and raw errors never enter its state. Other provider operations have no active diagnostic scope. File tests use `download::file::::tests::` instead of `download::file::tests::`. The owners are `transfer`, `validation`, `fingerprint`, `publication`, `replacement`, and `reconciliation`. HTTP tests retain their nested `wiremock_tests` module under `transfer::tests`. Test names and assertions are unchanged; the Windows partial-replacement test remains Windows-only. File tracing keeps the `kei::download::file` target. File-owner dependencies are one-way: transfer calls validation and publication; publication calls replacement, platform primitives, and fingerprinting; replacement calls platform primitives and fingerprinting; reconciliation calls platform primitives and fingerprinting; validation uses fingerprinting for local-size evidence. Platform primitives do not decide replacement or retry policy. In particular, Windows partial-exchange recovery stays in replacement. Windows native move and backup-replacement calls share an absolute, lossless UTF-16 path adapter. It resolves ordinary Win32 separators and components without filesystem traversal, then supplies extended drive or UNC paths. Existing verbatim namespaces and native units remain intact; embedded NUL is rejected. Stored paths, confinement, no-overwrite flags, ownership checks and replacement recovery do not change. This avoids MAX_PATH failures in nested managed-primary staging without requiring a registry or manifest change. Pipeline tests use `download::pipeline::::tests::` instead of `download::pipeline::tests::`. The owners are `adoption`, `consumer`, `outcome`, `pass`, `producer`, `progress`, `streaming`, and `task`. Test names and assertions are unchanged. Shared test fixtures stay in `src/download/pipeline/test_support.rs`. XMP Toolkit preparation checks movie box boundaries with bounded file reads before updating the temporary copy. Nested QuickTime `meta` boxes without a supported FullBox prefix are rejected because the toolkit can corrupt their existing child boxes. The optional four-byte zero terminator is accepted only at the end of a `udta` child list. Rejection preserves the original media and leaves embedded rewrite work pending through the existing retry owner. ## Main flows ### Container startup `docker/entrypoint.sh` owns command collision routing and PUID/PGID privilege drop. The explicit command list mirrors visible clap commands before checking external executables. Without PUID/PGID, it executes as root without ownership repair. With both numeric IDs, it recursively repairs UID and GID on /config and /photos, including symlinks themselves without following their targets. Traversal or repair failures warn before privilege drop. Every restart scans all inodes; KEI_CHOWN_PHOTOS=0 skips /photos repair when host permissions are already managed, while /config remains repaired. For `support-export`, the wrapper queries the owning Rust CLI parser before volume initialization. The private query ignores captured CLI environment bindings and returns an exit status without runtime, config-file reads or credential resolution. Export launches and their help routes skip both volume traversal and ownership repair, including incomplete export arguments, then drop to the requested numeric UID:GID. Option values cannot select this exception. Status 0 selects export, and only status 1 permits ordinary volume initialization. An unexpected query exit, including a crash or signal, aborts before traversal. Normal launches retain their existing ownership initialization. The Docker HTTP healthcheck defaults to port 9090. A deployment using a custom `[server].port` must set `KEI_HEALTHCHECK_PORT` to that same port, including when overriding the command with a custom config path. Docker does not parse TOML separately. Disabled HTTP servers require a deployment healthcheck override. See `docker-compose.yml` for runtime examples. ### Command dispatch ```text src/main.rs -> kei::main_inner -> src/lib.rs::run -> src/cli.rs -> command owner, service owner, or src/sync_loop.rs ``` `run` composes private phases in `src/lib.rs`: `load_startup_config`, `resolve_startup_output`, `initialize_logging`, and `dispatch_command`. Config loading retains the Docker fallback and passes parse errors to `doctor` without blocking its diagnostic report. Output resolution keeps the friendly-mode request separate from the mode allowed by the environment. The log-writer guard stays in `run` until dispatch returns. Startup injects the captured environment password before dispatch. Command dispatch retains the setup wizard's one-shot sync path and the service owner's entry point. The CLI requires a subcommand. `kei sync` enters the sync path. Commands such as `status`, `doctor`, and `manifest` read local state without entering the normal iCloud sync loop. `main_inner` records whether stdin is interactive before it starts the async runtime. Command owners use that single input mode for password and 2FA prompts. A foreground command returns `TwoFactorRequired` to the exit classifier, which reports auth exit code 3 and the `login get-code` and `login submit-code` recovery flow. Only the sync owner may convert that error into a durable wait, and only when the resolved configuration is in watch or service mode. Interactive login and `login get-code` may retry once from clean local auth state when Apple rejects verification-code delivery with HTTP 403 and the attempt loaded a persisted cookie jar or session. The failed session removes only its cookie jar, session, and validation cache while it still holds the per-account lock, then the shared auth owner creates one clean replacement session. Password material and SQLite state remain unchanged. `reset session` exposes the same locked cleanup explicitly. Cleanup advances a generation in the existing lock file. A watch process that released its lock for idle sleep detects a changed generation when it wakes and stops before its old in-memory session can recreate reset credentials. Reauthentication handoffs carry their pre-release generation into replacement-session creation, so a reset in that gap also wins. The session guard explicitly unlocks on drop, including failed generation validation and rejected principal replacements. Closing only its own descriptor would leave a forked child's copy holding the shared lock until exec or exit. An explicitly released guard does not unlock again on drop; generation checks and independent-process exclusion remain in force. Blocking generation writes retain the shared guard until completion even if their caller is canceled, so cleanup cannot unlock while a generation write is pending. ### Account ownership and legacy adoption Configured login and realm determine collision-resistant names for state, auth artifacts and encrypted credentials. Account-owner format version 1 is independent of the application schema version and must be checked before its migrations. Production connections require an owner expectation; unbound connection helpers are restricted to tests. All state commands, including read-only export and reset, enforce this boundary. State reset checks ownership before confirmation and requires `--yes` when the process input mode cannot prompt. An absent database remains a successful no-op. Sync and import require a usable authenticated provider identity and pin it independently of filenames. Watch validation, replacement sessions and initialization retries retain that principal pin. Changed or missing authenticated identity stops before further provider or state work; it cannot become a recoverable quiet-watch warning. Legacy discovery fails closed rather than creating fresh state over retained debt. Explicit `migrate-state` requires verified operator ownership confirmation and a fresh login under the account lock. SQLite backup retains rows, unknown tables and committed WAL content, then binding, schema migration and integrity validation occur in a private stage. Same-directory publication refuses replacement and fsyncs the file and parent. The legacy source and companions remain. Conflicting scoped provenance is rejected, not rewritten. Legacy session and encrypted credential files are preserved and ignored; keyring identity remains unchanged. See [account migration](account-state-migration.md) for operator steps and compatibility limits. ### Observed incremental shadow capture After opening an authenticated, account-owned database, sync attaches capture to its Photos service and libraries. Normal incremental `changes/zone` responses are bounded and validated before `DeltaRecordBuffer` pairing or selection. The adapter rejects ambiguous JSON keys and unusable source identity or scope for the whole page. Original response bytes retain unknown fields, tombstones and unresolved relationships, including numeric lexemes that a typed JSON re-encode could alter. Cookies and authentication headers are not captured; resource URLs remain transient payload, never durable identity. Schema 29 adds `provider_shadow_pages`, ordinal source identities in `provider_shadow_records`, and a per-scope last-observed pointer in `provider_shadow_receipts`. An immediate SQLite transaction rechecks the independent account owner and commits all three together. Provenance includes the account namespace and authenticated provider fingerprint, versioned realm, CloudKit container and environment, private/shared database, validated zone and returned owner when present, request cursor and successor. Unknown zone metadata is retained in the payload without changing the scope key. Exact page replay is idempotent; a changed payload remains a separate observation. Original decoded response bytes are limited to 16 MiB per page. The inbox allows 512 MiB of charged payload and provenance bytes, including source identities. This is a logical budget, not a physical SQLite/WAL file quota. An oversized page, full inbox, failed write or unusable identity stops that page before emission or successor acceptance. Such a refusal propagates through orchestration; an uncaptured rank inventory cannot replace it and advance the checkpoint. Existing typed invalid-token fallback and authentication handling remain intact. Exact already-captured replay remains possible at capacity. No observations are automatically pruned. SQLite WAL `NORMAL` remains unchanged; a successful commit does not promise that every acknowledged observation survives power loss. Network filesystems are not newly qualified. Schema 36 adds a private-only canonical transport index. The provider adapter recomputes evidence from the retained page after account validation. It recognizes exactly two direct descriptor paths: a complete `CPLMaster` `fields.resOriginalRes` `ASSETID`, and a complete `CPLAsset` or `CPLMaster` `fields.resComputeStateData` `ASSETID`. Each must have a nonblank `recordName` and `recordChangeTag`, a numeric `size`, nonblank `fileChecksum`, `referenceChecksum` and `wrappingKey`, and a validated download URL. Only the whole validated `downloadURL` token is masked. Record type, tag, descriptor bytes and unknown bytes stay exact, and nested or arbitrary `downloadURL` fields are not eligible. Each candidate is bound to the envelope that produced its byte path: a changes candidate is resolved only against `zones[0].records` and a lookup candidate only against the paired root `records`, so a lookalike record at the same ordinal in another envelope can never lend eligibility or mask an unknown-envelope URL. This is a provider-versioned asset identity, not a verified content hash. The private compute path is conditional coverage: it does not claim that observed reporter records satisfy this descriptor shape. A scope that lacks the realm, database, zone name, or exact nonblank owner is a raw observation with no coalescing proof, not a rejected capture. An invalid private page, malformed scope, or mismatched index entry still fails validation. The canonical key is an unambiguous hash of account namespace, provider fingerprint, exact scope, request cursor, successor, `moreComing`, classifier version and masked body digest, so different cursors or scopes remain independent classes. A hit reloads the representative through the checked page loader, recomputes its key and masked bytes, and requires exact metadata and source-identity equality. Unsupported scopes stay ordinary raw observations; an invalid private page or a mismatched index entry fails validation. A replay always reprojects the durable source, so an interruption after capture but before projection recovers on the next equivalent observation. The same schema step compacts finite legacy duplicates before ordinary admission. It streams one body at a time inside the migration savepoint and requires a validated raw hash, provider-owned evidence, exact account/provider/scope/cursor/ successor/`moreComing` binding, complete masked-byte equality, and a fresh pure projection that equals every stored record, reference, debt and charge. Only then may an old body become empty. The old row keeps its original ID, ordinal source identities, historical observation hash and catalogue/work/selection references; a typed alias stores the canonical page, classifier version, masked digest, canonical certificate and an immutable provenance binding over the original ID and hash, canonical ID and hash, class key, version and ordinal identities. Known work receipts, work retries/scans and selection sources must still match that old identity, source hash and independent confirmation bytes before the body may be dropped; an unsupported or mismatched dependency pins the page as raw. Loads expose `Raw` versus `LegacyAlias` provenance and verify the representative hash, certificate, provenance binding and index on every access. Alias chains and cycles are refused. The historical hash is never presented as a hash of representative bytes, and no original hash is fabricated. Unknown formats, unsupported projector versions or failed projection proof stay raw; inability to prove the optional projection (including its byte budget) makes a page ineligible for compaction rather than failing migration, while SQLite and storage errors still abort. Migration charges retained provenance plus alias and index overhead, does not raise budgets or prune catalogue rows, and rolls every alias and body drop back with the schema version on any failure. Replaying a discarded URL variant validates the alias and returns the canonical ID; it creates no new alias. A shadow receipt is only the last observed page, and replay can move it back. It is not a coverage frontier, materialization receipt or verified filesystem progress. Existing `CheckpointEvidence`, sparse and legacy generation proofs, recovery debt and filesystem guards retain checkpoint authority. Capturing a tombstone cannot authorize local media deletion. Existing download and metadata queues still consume the current stream. Capture alone cannot admit queue work; the bounded retained-source admission in [Current provider work admission](#current-provider-work-admission) is a separately validated owner. Capture covers only incremental pages observed by existing sync in selected scopes. Rank inventories, bootstrap, lookup and deletion-validation scans are not represented as complete zone capture or historical completeness. Rank EOF plus a delta bridge is not an atomic snapshot or proof of absence. New future observations cannot retire old expired-epoch debt. Complete queue projection, capture-owned cursors, durable catalog selection and separate progress remain later stages; the bounded current-generation slice in [Current provider work admission](#current-provider-work-admission) does not complete them. Historical-version retention, unrecoverable-debt acknowledgment, epoch recovery and compaction still require explicit policy and support proof. The additive migration preserves existing rows and uses the migration owner's savepoint. Re-entry validates the table columns, primary keys, replay key and SQLite-assigned page identity required by source and receipt links. A conflicting unknown table fails without partial schema or version changes. Older binaries supporting only schema 30 must refuse the current database. ### Transactional source catalog and replay Schema 30 adds `provider_catalog_records`, literal source references in `provider_catalog_references`, unresolved indexing evidence in `provider_catalog_debt`, and versioned per-page receipts in `provider_catalog_pages`. Records refer to the original captured page and source ordinal. All payload versions remain separate observations; replay order and opaque provider cursors do not establish a current version. Original inbox bytes retain fields that the catalog does not interpret. Startup replays pending captured pages before planning libraries, one page at a time with cancellation between transactions. Live capture indexes its committed page before lossy pairing. The provider adapter revalidates account-bound scope, original hash, page metadata and source identities, then derives facts on the blocking pool. Read transactions provide a coherent source snapshot; the immediate write transaction rechecks ownership and reloads the source before inserting all records, reference facts, unresolved evidence and the final receipt together. A failed projection retains the captured page and existing checkpoint for restart. It cannot trigger an uncaptured rank fallback. Unknown kinds, typeless tombstones, malformed references and incomplete asset or container links retain unresolved evidence. A literal target name is not proof of that target's identity or materialization. No tombstone authorizes local deletion. Receipt version 1 means source facts were indexed, including unresolved evidence; it does not mean selected, downloaded, rewritten, verified, historically complete or acknowledged recovery debt. Existing queues also accept the bounded current work admission in [Current provider work admission](#current-provider-work-admission); source indexing alone cannot authorize it. Derived page facts have a 16 MiB logical charge limit and a separate 512 MiB catalog index budget. These are operational backpressure limits, not physical SQLite/WAL quotas or a retention promise. A full index holds the checkpoint and retains pending source observations; exact completed-page replay validates facts and remains possible at capacity. Startup has no fixed latency guarantee. Nothing is automatically pruned or discarded. WAL `NORMAL`, filesystem qualifications, existing checkpoint evidence, publication receipts and metadata obligations are unchanged. Historical intermediate retention and unrecoverable-debt acknowledgment still require explicit policy decisions before dependent stages. ### Current provider work admission Schema 31 adds `provider_work_receipts`, generation-bound `provider_work_obligations`, and the `provider_work_scan` scheduling position. Before normal dispatch, up to 64 retained, indexed CPLAsset observations per cycle can trigger scoped child lookup followed by a complete current child/master lookup. Original bounded confirmation bytes and source page/ordinal provenance are retained. Duplicate keys, duplicate or omitted records, provider errors, wrong owners, wrong zones and changed child/master relationships cannot confirm work. Provider confirmation is an observation, not an atomic provider snapshot. This slice supports one private library-wide pass with an explicit default owner, without recent caps, album exclusions, retry-only selection or metadata backfill-only execution. Default `PrimarySync` selection lazily discovers the private `/zones/list` response using the pinned authenticated session. Discovery requires bounded original JSON with unique keys, an explicit zones array, unique zone names, no zone errors and no unsupported continuation. Only a live exact `PrimarySync` entry with a nonblank string owner replaces the constructor's ownerless primary library after that exact selected scope returns a recognized indexing state. `RUNNING` retains provisional readiness and requires durable reconciliation; owner qualification alone never certifies a checkpoint. The complete discovered scope is preserved, including explicit owners other than `_defaultOwner`; lookup and inventory references must still match it. The complete validated private zone descriptors are cached separately from initialized libraries. Primary-only lookup initializes only the selected primary; unrelated private indexing failures do not block it. Private and shared library maps publish only after all requested library initialization succeeds. A failed full-map initialization preserves an already qualified primary and retries initialization using the validated private descriptors. Shared endpoint evidence cannot replace private primary ownership. Missing, deleted, blank or malformed primary owners preserve the ownerless constructor; no owner is inferred from a name. The current-work admission slice still requires `_defaultOwner` and is unchanged. Default selection needs only the cached private listing and selected primary's indexing success. Normal sync planning and import still request all libraries; their existing full-map initialization contract is unchanged. The discovered primary flows through default library selection, `get_library`, `all_libraries`, `PhotoLibrary::all`, and the existing sole Unfiled admission checks. Private-discovered non-primary photo zones (`SharedSync-*`) retain their existing explicit-owner route. Attaching capture before or after discovery preserves the owned DB context. Retained primary source pages with explicit returned ownership already use the qualified scope and can confirm current work; ownerless source pages remain separate retained observations. No source scope is rewritten or history acknowledged. Shared-endpoint zones, ownerless zones and different owners remain outside this slice. Other selections retain observations and use their existing pipeline without an admission receipt. Current media/date/filename filters and RAW, companion and path planning use the existing owners. The work configuration fingerprint binds their frozen evidence separately from enumeration and checkpoint hashes. Hidden or deleted current records retain deferred evidence without new media work. An immediate transaction independently validates authenticated account ownership, source bytes, indexed identity, complete scope and current confirmation. Frozen resource checksums, sizes and metadata must match the retained confirmation. New rows, asset/master mappings, grouping/retry markers, work obligations, deferred conflict evidence and the final receipt commit together. Any conflicting existing generation, metadata, mapping or legacy identity defers unchanged. Identical existing queue rows retain their retry and publication metadata. Admitted-receipt replay verifies frozen obligations and never reconstructs an old generation over a queue that has since advanced. Receipt state describes work admission, not download completion, metadata publication or verified media. Future confirmations that differ from a retained receipt only in complete eligible Asset `downloadURL` tokens reuse that receipt's original raw bytes, hash, scope, source and master as the charged representative. The current authenticated confirmation and planned records are still validated first, and the retained URL is never used for current download work. This requires exact masked-byte equality, the same work source/config/scope/master and historical source hash, and an existing receipt whose own hash still matches its bytes. Non-URL bytes, unknown or incomplete descriptor shapes, and genuine checksum, tag, metadata or size changes remain distinct raw confirmations. Reuse adds no receipt row and does not clear conflict/debt, resurrect admitted work or merge sources or configurations. Retained receipt confirmations stay historical, finite evidence tied to their admitted obligation; reuse reads the existing bytes and never rewrites or duplicates that stored confirmation set. Equivalent replay therefore grows no further, while genuinely novel confirmations are still admitted subject to the existing budget. Matching unfinished projected generations fence shared asset admission, identity mapping and metadata refresh, including independent path publication receipts. Content and metadata fences permit safe collision-leaf replanning by the existing path owners; the original receipt still retains its admission-time filename. Completed generations do not indefinitely pin the mutable canonical queue. Schema 32 adds source/config retry deadlines in `provider_work_retries`. Deferred admission commits its attempt and deadline with the receipt and scan bookkeeping; admission clears that retry schedule in the same transaction. The first delay is one hour, doubles on repeated deferral and stops at 24 hours. A new source or current configuration has its own immediately eligible work. Each bounded admission selects the oldest due unadmitted source independently of the historical scan position. A due early source can be retried while new observations continue to arrive; a future retry does not consume the 64-attempt budget or block fresh tail work. The SQL query can scan a mostly admitted source prefix; the attempt limit bounds provider/planner work, not SQLite VM steps or retained-catalog scan latency. Retry schedules never acknowledge debt or promote a cursor. The watch precheck includes due retained work for the same eligible selected passes and current work fingerprint as admission. Matching unfinished pending/failed download obligations also wake normal dispatch after restart. The existing targeted recovery pass owns their publication. Metadata-only markers keep the existing option-aware metadata precheck owner. Local wake-up adds selected zones without inventing a provider database successor; provider and zone checkpoints retain their existing gates. No due local work permits the normal no-change shortcut. Deferred sources remain retained and become due at their deadlines; an idle cycle does not imply materialization or completeness. Work plans have a 16 MiB logical charge limit and work evidence has a separate 512 MiB logical budget. Nothing is automatically pruned; backpressure retains source observations, unfinished work and the current checkpoint. These are not physical database/WAL quotas or fixed startup latency promises. Current versions under current configuration are the approved materialization scope. Historical observations and all unfinished debt remain retained. Admission of a current version does not prove historical completeness, acknowledge old unrecoverable debt, authorize local tombstone deletion or select a historical retention policy. Existing CheckpointEvidence, sparse/legacy proofs, opt-in metadata writes, publication guards and WAL NORMAL qualification remain in force. ### Selection generation shadow Schema 33 adds account-bound `provider_selection_generations`, source links in `provider_selection_sources`, per-pass `provider_selection_decisions`, and rendition/path obligations in `provider_selection_destinations`. The existing private library-wide retained-work path captures its planning result before queue admission and verifies strict provider-owned equivalence on replay. Every non-confirmation manifest field stays exact; an eligible paired current confirmation may differ only in validated private `ASSETID` URL tokens. The retained representative keeps its original bytes and hash, while the current response and URLs remain the downstream planning input. A changed decision, identity, version, metadata or resource fact stays a distinct generation. Original current child/master confirmation bytes retain unknown fields and numeric lexemes. Current confirmation is separate from the historical trigger observation; source order, page IDs and opaque cursors cannot establish a current version. Manifest format 1 explicitly records `confirmed_sources_only` coverage. It does not represent a complete library inventory, a global recent frontier, album membership coverage or absence. Selected, excluded and deferred decisions remain separate from admitted queue work and verified media. Destinations include the pass, child, logical rendition and path so one rendition can have distinct copies without changing the canonical asset identity. Native path encoding is lossless, including non-UTF-8 roots; relative task destinations are anchored without changing the task path. This stage captures only the existing admission profile. Multi-pass selection and activation require the next qualified stage. Capture and replay independently check authenticated account ownership, complete private source scope, indexed original source bytes and identity, current paired confirmation, resource and metadata fingerprints, and all normalized manifest rows. One immediate transaction commits a manifest and its source, decision and destination rows. Failure retains source observations and the existing cursor; no queue admission can substitute for missing shadow evidence. Exact replay is idempotent at capacity. A manifest is bounded to 16 MiB of serialized evidence and this store has a separate 512 MiB logical charge budget including normalized row payload. These are backpressure limits, not physical SQLite/WAL quotas. Nothing is pruned, retired or acknowledged. Existing work receipts, unfinished generation fences, metadata queues, filesystem publication guards and checkpoint proofs remain authoritative. WAL NORMAL and filesystem support are unchanged. This shadow storage is the first PR of the selection phase, not completion of replay-driven multi-pass materialization. ### Private selection and destination replay Schema 34 adds active generations in `provider_active_generations`, original validated rank pages and record indexes in `provider_selection_rank_pages` and `provider_selection_rank_records`, bounded decisions in `provider_active_decisions`, source links in `provider_active_sources`, and exact physical obligations in `provider_active_destinations`. These tables supplement the schema-33 shadow store and the existing source inbox and catalog. `icloud/photos/album/selection_capture.rs` validates complete bounded query responses before the existing pairing and filtering stream consumes them. `download/orchestration/generation.rs` composes existing selection policy, current confirmation, transactional queue projection and destination replay. `state/db/provider_generations.rs` owns stored declaration, provenance, admission, frozen metadata intent and completion validation. Activation requires an authenticated account-owned database and one exact private provider scope with an explicit default owner. Every selected pass must be a supported private named album or Unfiled query using the same concrete store and scope. Mixed, shared, smart-folder and unsupported profiles keep the whole existing dispatcher. Retry-only, metadata backfill, read-only execution and capture-local timestamp replacement keep their existing owners. Ordinary explicit metadata refresh with preservation can use active destinations. A generation binds account and provider ownership, full zone identity, selected passes and queries, source dependencies and selection, rendition, path and metadata configuration. Lossless native destination paths are independently anchored. A rank page records its exact request and original response bytes, including unknown fields and tombstones. Rank pagination remains an observation window. EOF plus a delta bridge does not establish a snapshot or prove absence. Current paired confirmation determines resource identity; observed order and opaque rank or source tokens do not select versions. Ambiguous or incompatible facts remain deferred. Existing global/per-filter recent, ties, exclusions, RAW, companion and naming owners decide eligibility. An immediate projection transaction commits the validated decision, source links, per-pass/rendition/path obligations, existing queue records, mappings and grouping changes together. Unfinished old versions and frozen grouping/writer intent fence incompatible newer work. Independent healthy work can proceed. Current query coverage is sealed only from that generation's own evidence; replaying a newer inventory cannot seal an interrupted old window. Configuration or source changes create a distinct generation or require existing validated reseed. They cannot acknowledge omitted eligibility or old unresolved debt. Destination media completion requires an actual confined file fingerprint and independently verified selected source checksum. An exact active physical receipt can be reused across compatible passes after current selection and file verification; another path retains its own obligation. Opaque provider signatures remain independent from source and local SHA-256. Fresh or legacy fallback publication requires a separately verified per-path source receipt; validated native reuse relies on its own lossless generation/destination receipt, including operation-owned prepared metadata output. Imported generic local checksums cannot establish source provenance. Metadata completion is separate and uses the existing option-aware rewrite owner with the exact native path, frozen grouping and writer flags. Disabled flags cannot finish older writer debt or enable writes implicitly. Before an embedded replacement, the active receipt stores the measured prepared output tied to the exact intent, input and source. Reopen can recognize only that operation-owned output; arbitrary damage cannot become publication proof. Filesystem publication and receipt finalization retain their existing compare-and-swap guards. Legacy TEXT paths remain compatibility projections. A non-UTF-8 receipt cannot update a checksum or clear markers through a lossy text path. Both legacy metadata queues retain ambiguous native-path markers without dispatching that text path to a sibling file; exact native work uses its active receipt. These markers are retained evidence, not an acknowledgment of another physical path. XMPFiles retains its existing UTF-8 filename requirement. Native-capable HEIF and sidecar owners use native paths without granting additional rewrite options. Startup and watch can wake unfinished selected work even when the canonical asset row already has one downloaded destination. An unchanged completed coverage window reuses exact selection and publication evidence. Historical scheduling visits one due old root per cycle, at most 64 decisions and 16 MiB of serialized manifests, with a persisted continuation and retry deadlines. Within that transaction, each unique original rank page is strictly decoded and its complete ordered record index audited once. Compact page proofs are discarded when the operation ends; new reads use a stable read transaction. The existing 512 MiB evidence quota bounds unique serialized input, without retaining all parsed pages. Original-fact reconstruction separately limits retained serialized page inputs to 64 MiB; unavailable bounded evidence remains deferred debt. The current generation receives a bounded priority metadata lane; remaining writers share the existing physical writer budget and persisted general continuation. HTTP 401, 403 and 421 current-confirmation failures stop additional hydration in the selected run and preserve the existing bounded reauthentication route. Non-authentication proof failures retain their own deferred debt. Active cleanup reconstructs original failed tasks' child/master and provider-rendition binding from their validated declarations, then uses the existing exact URL-refresh owner. A refreshed URL cannot replace frozen selection, metadata, path or publication authorization. A transient per-invocation registry composes failed historical and current runs; it is not durable identity or a separate memory quota. The active store has a separate 512 MiB logical evidence charge budget and 16 MiB page/manifest limits. Capacity refusal rolls back the whole observation or projection; no partial page can authorize progress. No records or debt are pruned. Logical charges do not bound physical SQLite files or WAL usage. Seals and retained source audits can still read substantial history; no fixed startup latency or constant total decode-work guarantee is advertised. Captured observations, admitted decisions, verified media and verified metadata are separate durable facts. CLI download counts retain the existing task semantics and do not claim unique-file counts or full-zone completion. Active progress cannot replace `CheckpointEvidence`, legacy/sparse generation proofs, filesystem guards or source checkpoint policy. Unfinished old selection debt vetoes the source successor, including when later configuration uses the legacy dispatcher. Local wake-up cannot invent a provider cursor. Tombstones do not authorize local media deletion. WAL `NORMAL`, filesystem support and power-loss limits remain unchanged. Capture-owned cursor promotion, historical intermediate retention, unrecoverable-debt acknowledgment, expired-epoch replacement, full-scan absence and compaction require separate decisions and qualification. ### Sync and provider checkpoints ```text sync_loop::run_sync -> resolve configuration, credentials, libraries, and pass plans -> optional scoped changes/database pre-check -> sync_cycle::run_cycle -> download::download_photos_with_sync for each active library -> sync_cycle source checkpoint decision -> cycle reporting and watch control ``` Photo readiness is explicit provider evidence. `PhotoLibrary` accepts `FINISHED` or `RUNNING`; missing and unknown states fail closed. Sync checks readiness before watch shortcuts and again at the checkpoint boundary. Before downloading from `RUNNING`, the cycle persists `indexing_reconciliation_required:` in the existing metadata table. Presence is the obligation, including unknown future payloads; no schema migration is needed. The previous trusted zone and database checkpoints remain unchanged. Cross-zone album sources also contribute readiness. While indexing, the download dispatcher uses full queries without incremental replay, retained-work admission, metadata repair, retry retirement, or album snapshot completion. Ordinary media still uses the existing verified publication and durable finalization path. Managed primary layouts wait for `FINISHED`. Watch cycles with a hold bypass the database no-change shortcut and rebuild collection and album plans so omitted albums can reappear after indexing. A retained hold forces full reconciliation after `FINISHED`. A scope refreshed while readiness is `RUNNING` remains stale even if the cycle's later probe says `FINISHED`; the next wake rebuilds it after readiness. Retry-only inventory can download known failed work but cannot discharge recovery of previously omitted assets. The held full-query path continues to suppress deletion-capable maintenance, album snapshot completion, and the retained-token bridge even when its initial probe reports `FINISHED`. Complete natural EOF, successful local work, a fresh FINISHED scope, and a final FINISHED probe qualify that inventory. One fenced transaction removes `indexing_reconciliation_required:` and writes `indexing_replay_required:` without moving the trusted source cursor or database cursor. The later trusted cycle replays the old cursor and applies source deletions. Replay presence, including an unknown future payload, bypasses the watch database shortcut across reopen. Only normal checkpoint promotion removes replay debt, including aggregate config-drift promotion. Rejection of the retained cursor preserves the existing checkpoint gates. Bounded coverage, interruption, retry-only selection, failed transfers or metadata, stale scope, missing proof, state-write failure, and readiness regression cannot qualify an inventory or advance its cursor. Provisional inventory cannot replace a comparable inventory anchor. Managed primary configuration, recorded bindings and pending operations retain the RUNNING guard, including a later suffix configuration. Once FINISHED, the held phase uses the same full-enumeration owner and token/EOF evidence, but only consumes validated provider identities. It does not invoke local task planning, journal recovery, publication, metadata drains, retry retirement or snapshot pruning. A fresh FINISHED plan, unbounded selected scope, valid token and final FINISHED observations from every source qualify that inventory. The fenced transition additionally writes `indexing_materialization_required:`. Presence, including unknown payloads, forces a fresh full managed materialization through the existing journal and publication owners before retained-cursor replay. It bypasses quiet watch shortcuts and refreshes plans across restarts. An empty change feed therefore cannot omit assets that inventory-only qualification never queued. Managed materialization debt survives retry-only/bounded runs, failed transfers, metadata failure, interruption, regression and checkpoint-write failure. When an old cursor exists, both full publication and its incremental bridge must qualify before one checkpoint transaction removes materialization and replay debt. Existing local-byte, collision, sidecar, ownership and interrupted-journal guards still control every managed write. Readiness probes are observations, not an atomic snapshot guarantee from Apple. Inventory-drop diagnostics use a complete count measurement from the initial full-query invocation, captured before activity and delta-bridge composition. The download owner records natural completion, usable counts, and canonical resolved pass/query/filter coverage. The cycle compares only equivalent complete scopes after the existing success, interruption, cancellation and stale-plan vetoes. Configured bounds, missing counts, targeted subsets and retries cannot replace a complete anchor. A complete selected smart-folder-only query can qualify; source replay preserves that query's measurement without adding delta activity. Mixed refresh subsets cannot supply a complete anchor. Public `api_total_at_start` remains additive activity telemetry. Diagnostic anchors use `inventory_anchor_v1:` metadata keys scoped by enumeration hash, zone and coverage digest, with version and exact canonical coverage in the payload. Legacy `inventory_anchor:` rows remain untouched and are never certified retroactively. Malformed or unsupported evidence is ignored; the next eligible inventory seeds a fresh baseline without an upgrade-time drop warning. Inventory evidence never authorizes source checkpoints or proves retained deletion history. The same accepted drop facts feed JSON reports and the durable cycle ledger. CloudKit HTTP 401, 403, and 421 failures during enumeration or metadata hydration return a session-expired outcome. The cycle preserves its provider checkpoint and pending capture work. It does not treat these failures as a reason to fall back to full enumeration or retry incomplete pass tokens. Incremental producers pass session failures to the download pipeline before stream completion. The pipeline records the interrupted run before status commands read its outcome. Before bounded mid-cycle reauthentication, the watch loop removes the cached validation result and resets the HTTP connection pool. A recovered session replays the retained checkpoint. Ambiguous provider identities remain pending; authentication recovery does not authorize choosing a child or discarding work. Incremental asset-identity lookups, paired-asset lookups, and selected-album relation hydration in `src/download/orchestration/delta.rs` cancel provider reads on shutdown. Started state writes finish outside these cancellable waits. `support_hydration_wait_v1` reports a fixed phase and started, waiting, returned, or cancelled outcome at INFO, with a waiting observation every 30 seconds. Returned means the read returned, including errors; it does not certify replay, identity, or a checkpoint. Interrupted hydration retains the existing checkpoint and pending configuration. These waits have no new overall time limit and do not establish a universal shutdown deadline. Selected relation targets use bounded child lookups followed by complete-pair validation before a full-zone fallback scan in each configured source. Omitted, deleted, changed-pair or foreign-scope results remain unresolved; lookup absence does not authorize deletion. Authentication and request failures propagate to the checkpoint hold. Sources retain their existing priority. Within one library sync, relation targets from album passes share a read only when the request context, endpoint, full zone/owner and ordered fallback sources match. Targets are combined before reading; no absence cache survives an error, interruption or cycle boundary. A remaining unresolved target still blocks the checkpoint through the existing relation gate. A fallback scan can still take multiple pages and has no new overall deadline. Legacy-master hydration scans all current visible and hidden children before metadata-capture repair or pending-download recovery selects an owner. Hidden children remain identity candidates; soft- and hard-deleted children do not. Without a saved owner, multiple matching children remain unresolved, even across visibility states. This does not change download selection or permit checkpoint advancement without the existing completion proof. Hidden children can supply identity evidence without a valid `assetDate`. Recovery counts all matching children before validating a selected child's capture date. Missing, null, or out-of-range dates cannot authorize an owner claim, metadata refresh, pending-file adoption, or download-policy exclusion. Invalid selected dates retain repair or retry work and hold the checkpoint. The display/path epoch fallback is not recovery evidence. A new legacy-owner claim requires consistent, present per-rendition `added_at` evidence matching the provider's `addedDate`, in addition to the existing rendition checks. Missing provider dates do not use the epoch fallback as identity evidence. Matching dates do not rank otherwise ambiguous children. Whole-second legacy dates match within that second because older writers discarded milliseconds. Fractional stored dates require an exact match. Different stored rendition dates, including mixed whole-second and fractional rows, remain unresolved at the atomic claim gate. Direct pending lookups also require matching rendition evidence before they can persist a new owner. Another historically mapped child blocks a new claim even when it is now missing or has its own catalogue row. The state owner checks family history and mixed rendition dates inside the claim transaction. Normal enumeration and pending recovery use the same restrictions; read-only retry planning does not persist a claim. Unresolved pending ownership holds the checkpoint. Metadata-capture retry evidence includes each rendition's added date, at millisecond precision. A changed date or an older fingerprint without dates makes retained work eligible again. Conflicting or missing date evidence uses the existing bounded retry queue. This does not delete or retire legacy rows, replace saved owners, or restore dates overwritten by an earlier repair. Recovery of those rows requires separately preserved historical evidence. Ambiguous metadata-capture repairs retain a schema-27 retry row keyed by library, catalogue asset ID, and target capture revision. An unchanged failure retries after one hour, doubling up to 24 hours. Changed catalogue identity or rendition evidence is eligible immediately. A durable generation rejects stale attempts; its counter survives retry retirement. The state reader excludes deferred candidates before applying the existing 500-identity batch limit. Unresolved and deferred counts are separate from failed repair attempts. Deferred work keeps the capture revision pending and its checkpoint blocked, even when a cycle makes no repair attempt. The cycle checks each library before committing inventory anchors or provider checkpoints, including `--refresh-metadata` runs that bypass automatic capture repair. Idle health and other-library success cannot turn retained ambiguity into a successful backup. Completion retires retry rows only after the existing repair or source-deletion rules remove the stale work. Retry scheduling does not select a child or authorize metadata rewrites, deletion, or checkpoint advancement. The download owners store these holds in `CheckpointEvidence`. Report counter changes and result composition cannot clear them. Unresolved asset-only delta hydration is incomplete work, even when no media transfer fails. The producer records `unresolved_asset_identity: = 1` in the existing metadata table before stream completion. The cycle owner clears that zone's marker atomically with a proven incremental checkpoint, including an inventory followed by a successful delta bridge. An inventory alone cannot clear it. Another zone's success and process restart retain the marker. Status aggregates all markers; cycle reporting does not advance health's last success while any remain. Selected zones with markers bypass watch-mode no-change shortcuts. Idle health also checks markers in unselected zones. Intentional bounded checkpoint holds alone are not failures. The marker keys remain in metadata; JSON report version 3 is unchanged. The Photos adapter emits aggregate, fixed-label identity lookup diagnostics. They distinguish omitted records, unexpected types, decode failures, invalid master references, and record-level provider errors. Reference-zone context is classified without logging provider identifiers or response bodies. These observations do not authorize identity guesses or cross-zone retries. `exact_lookup_rejection_v1` describes a rejected lookup request with fixed scope/pairing/presence labels and an expected-owner category. It applies the existing scope matcher; the older reference-zone JSON-equality classification alone does not establish rejection. Missing or malformed master references and record errors are distinguished from an explicit different-master reference. No owner or record value is emitted. A paired lookup can retain an inconclusive changed-master hint from a live, scoped child, including when the old master explicitly returns `NOT_FOUND` or `UNKNOWN_ITEM`. An old-master tombstone or any other record error does not authorize a hint. Incremental asset hydration makes at most one additional paired lookup in that same scope during a normal download run. Dry-run and filename-only modes retain their prior held behavior. Only a complete current pair may update the mapping through the existing state owner. A missing or deleted redirected master, foreign scope, another relationship change, cancellation or failed write holds the checkpoint. Other lookup consumers keep the hint unresolved. Sparse share links do not authorize this same-scope retry. Selected sparse-target recovery uses the separate strict qualification described below. Unpaired asset deltas retain typed sparse-share evidence from `isSparsePrivateRecord` and the `linkedShare*` fields. The marker accepts exact `INT64` and `NUMBER_INT64` type spellings, or an absent type, with integer value `1`; this does not broaden deletion-field validation. Targeted record lookups request these fields too; ordinary media enumeration keeps its existing field projection. A structurally valid link remains unverified. Malformed links and valid links without a usable master remain unresolved. Link identifiers are opaque and their debug output is redacted. Lookup changes do not overwrite the original delta evidence. A link alone does not authorize a target lookup. Selected sparse-target recovery is owned by `src/download/sparse_target.rs`. The cycle supplies only current selected, unrestricted full-library plans; it does not search an unselected library or infer membership for album, rank, bounded, managed-primary or provisional selections. A fresh source lookup must agree with its retained link. The Photos adapter qualifies target owner/zone and the exact child/master pair before the normal target download pipeline can materialize missing selected renditions. Ordinary downloadable validation remains strict; no deletion exception is added. A target receipt is useful only with exact current identity, metadata, source provenance and independent protected-path-safe files. Trusted Legacy File acceptance does not satisfy this source proof. The source checkpoint transaction revalidates the source generation, target catalogue/configuration snapshot and held target files. Partial downloads retain the source obligation. Existing bounded source retry selection remains authoritative; no general retry budget, checkpoint reset or debt-backed advancement is introduced. Recovery decisions use fixed allowlisted labels in retained support history; a target-ready or prepared historical file is partial work, not a complete backup. Schema 26 adds `unresolved_sparse_identities`, keyed by library and source record in the account-scoped database. `src/state/db/sparse_identity.rs` stores the first observed link, latest delta link, separate lookup evidence, attempt time, and retry deadline. The Photos adapter owns the versioned link encoding. Observation and the existing unresolved marker commit in one transaction. The `sparse_identity_generation` metadata counter assigns monotonically increasing generations, including when a cleared source reappears. `src/download/orchestration/delta/sparse_identity.rs` owns retry selection for both incremental paths. A matching valid sparse lookup starts a one-hour delay. Repeated matching results double the delay to a 24-hour maximum. Each library execution selects at most 100 due, identified sparse source-only lookups, ordered by retry deadline or last attempt, then generation and source key. Exact source/master mappings bypass suppression and do not consume that budget. Malformed or changed incoming evidence cannot inherit a cached negative result. Unclassified sources still need an initial lookup before sparse retry policy can apply. A deferred source remains unresolved and blocks the checkpoint. Retained sources absent from replay re-enter normal hydration and media planning. Authoritative source-only deletion lookups retain the completed delta token in `last_outcome` as `["source_deleted_v1", token]`. This is provider evidence, not a persisted claim that local processing completed. On restart or the next batch, the same complete delta snapshot and unchanged source evidence can reuse that result. The ordinary source-state transition runs again before a current receipt is issued. A missing delta token, changed link, or authoritative master mapping requires normal recovery. For a different token, the Photos adapter scans the complete raw zone delta since each saved deletion checkpoint. One scan validates a whole batch. Any source change, including restoration with the same link, invalidates its cached deletion. Sources absent from a complete scan retain their deletion evidence at the current boundary with fresh generation fences. Missing tokens, invalid pages, incomplete scans, and cancellation supply no reuse evidence. Normal source lookups can still establish fresh results after a failed scan. Validation does not advance the sync checkpoint or complete local processing. This lets more than 100 deleted sources complete over bounded lookup batches while unrelated zone records change, without relaxing checkpoint or state-write guards. Existing schema-26 outcome labels remain readable. Sparse observation and retry writes acquire an immediate SQLite transaction before reading generation or stale-attempt evidence. Independent WAL writers therefore serialize before those reads, preserving generation fences without a deferred snapshot upgrade. Hydration, explicitly soft-deleted source `CPLAsset` deltas, and exact-source hard-deletion tombstones supply generation-fenced receipts, not permission to advance a checkpoint. Incremental results carry these receipts in `CheckpointEvidence`, not `SyncStats`. Inventory and delta-bridge composition retain the receipts in zone-local evidence. The cycle owner still requires normal processing and checkpoint proof. The checkpoint transaction validates every retained source receipt before clearing its row and zone marker. Missing, changed, or stale receipts roll back the transition. Inventory alone, interruption, failed state writes, and another library's success cannot clear the obligation. Configuration reconciliation uses the same fence when publishing its staged checkpoints. Status reports unresolved and deferred counts without provider identifiers. A linked record name, missing/deleted shared zone, or history of removing a Shared Photo Library is not a source deletion or master identity. No automatic cross-zone recovery or checkpoint relaxation is added. A complete inventory followed by an explicitly rejected retained cursor creates `retained_checkpoint_hold:`. The versioned receipt contains a cursor fingerprint, requested enumeration/preservation policy hashes, an attempt count, and the next attempt deadline; it contains no provider identity or cursor value. Unchanged held zones defer provider, inventory and legacy preparation work for one hour. The initial rejection and two automatic retries form a finite budget of three attempts per boundary and policy. Exhausted holds perform no further provider work. Due attempts reserve their count and next deadline durably before requests, so interrupted or incomplete attempts consume a reservation. A failed reservation performs no provider work. Version-one receipts without a count retain one spent attempt. Malformed or unsupported receipts remain an actionable hold with no provider work. Genuine policy or active cursor changes permit bounded re-evaluation. Original assets, owner/date facts, legacy generations, transfer retries and sparse/deletion evidence remain intact. A fresh inventory and its counts cannot establish lost historical deletion facts or resolve this hold. Permanently rejected boundaries remain actionable holds; this protocol does not rebootstrap them. A successful validated replay of the retained boundary removes the hold in the same transaction that stages or activates its successor. Transaction-time metadata comparisons fence both hold staging and checkpoint promotion against changes to the planned cursor and enumeration configuration, including the exact staged candidate read during aggregate promotion. Existing legacy/sparse generation validation still applies. Global configuration activation remains atomic across selected zones. Zone-local hold aggregation keeps status and backup safety truthful even when another zone succeeds. Ordinary incremental producers carry validated changes/zone completion separately from query enumeration. Smart-folder-only passes also replay the source zone; that delta has no media destinations, and the selected folder query still owns media admission. Its query token never substitutes for the source successor, and a missing query token retains the selection-proof hold and same-cycle query recovery policy. Mixed unfiled and smart-folder refreshes retain the actual delta successor and every identity, partial-scan, cancellation and durable-write veto. A successful smart query does not certify retained-boundary replay. Typed provider cursor rejection is distinct from malformed or missing successor data. The per-zone provider checkpoint and the scoped database pre-check token have different gates: - A zone checkpoint may advance after a transfer failure when the exact retry work is durable and enumeration/token proof is complete. - An exhausted album grouping write preserves the zone checkpoint. Replay retries the relationship before it skips media that is already downloaded, and the relationship insert is idempotent. - The broader database pre-check token advances only after a clean aggregate cycle for the exact account, selection, filter, config, and selected-zone scope. Eligibility-config drift preserves the active checkpoint while a complete inventory and delta bridge build a replacement. Path-config drift preserves provider checkpoints while local catalog paths are reconciled. Schema 25 stores local reconciliation destination choices in `reconciliation_paths`. The planner reserves current and historical catalog paths and previous choices across all libraries, with library-qualified asset and rendition ownership. Unselected renditions keep their paths reserved when live resolution changes. Before it copies any media, it commits the new choices in one SQLite transaction. A failed reservation write prevents publication. Retries reuse the reserved leaf even when provider lookup order or availability changes. The recorded source path stays unchanged until file and metadata validation and state finalization pass. Each choice is keyed by the provider checksum and byte size. A changed rendition gets a separate destination; retries of the same content reuse its choice. Migration retains schema-24 choices with unknown content as occupied paths. It does not infer their content from a catalog row that may already describe a newer version. Reconciliation can create a new sibling for these legacy choices. Choices remain reserved after completion and config drift because old copies remain on disk. This ledger does not authorize overwrite or deletion. Full downloads, incremental downloads, and pending retries load this ledger and the catalog paths before planning. They reuse their own reserved destinations and cannot download into or adopt a path owned by another asset or rendition. When the ledger is active, downloads commit new destinations before publication. Pending retries commit the final choice after recorded-path overrides. A failed reservation write prevents download dispatch. After a failed finalization, retries can adopt a verified reserved sibling even when the catalog still records the previous source. Dry runs and filename listings do not commit choices. Recorded retry paths follow the same ownership checks. Explicit truncated-file repair can still use its own path when the existing fingerprint and repair authorization pass. Durable pending retries can record a destination in another directory after a recorded-path override. Only pending retry planning replays that exact choice when its library, asset, rendition, checksum, size and path ownership match and the destination is beneath the current download root without parent traversal. It preserves the root spelling and validates the saved destination key. Ordinary downloads and local reconciliation retain the requested-directory replay check. Retries stop planning another pass once every pending rendition for that asset is matched or safely adopted; outstanding companions still require planning. This does not migrate or discard reservations, alter selection completeness, or authorize promotion of a pending path configuration with smart-folder selection. Cross-parent pending adoption requires an independently durable local SHA-256 for the exact destination and current provider generation. It checks every saved alias before allowing a pass to use ordinary adoption. MMCS is a generation signature, not a local SHA-256; size and media headers cannot supply this proof. The file owner retains no-follow directory capabilities and the regular-file handle, revalidates identity and bytes before the immediate state write, and retains those capabilities through that write. Failed adoption writes retain debt and later attempts must reopen and reverify; no hash-only deferred adoption write is queued. Both ordinary and selection enumeration inspect all matching aliases before metadata actions, skip adoption, task adoption or touched-state promotion. A refusal holds that asset's debt while other enumerated assets may continue; it cannot enter the size-only fallback. Fresh downloads to an affected reserved pending destination retain the original publication inode and current-root no-follow namespace through immediate and bounded deferred finalization. Each attempt revalidates both identity and the publication fingerprint. Changes observed at revalidation retain the media and pending debt instead of committing stale verified provenance. Retained ownership and revalidation do not atomically exclude arbitrary Unix changes after the last validation while a database write awaits execution. Cleanup re-enumeration and collecting incremental dispatch preserve the affected task's proof requirement; cleanup also requires the original failed generation's checksum and size. Equivalent root spelling is rebased component-wise using platform ownership keys; this does not relax filesystem no-follow traversal or mutate the saved ledger. **Recovery limitation:** an existing cross-parent reserved file without a trusted matching local hash remains untouched and pending. This includes a crash after publication but before any durable receipt commits. These bytes cannot be automatically adopted, replaced or discarded; this slice does not fetch fresh provider bytes to establish an independent comparison. Download planning also loads current-content publication receipts from `asset_metadata_paths`. An additional album copy can satisfy a download only in that pass's destination and filename family. The planner rejects foreign catalog ownership, opens the file without following links, and checks its local SHA-256. Old provider generations cannot satisfy current downloads. These checks do not move the catalog's current path or change retry receipts. Without reconciliation reservations, an unsafe historical path disables optional receipt reuse for that pass instead of blocking ordinary downloads. Reserved paths retain strict validation. Live Photo planning applies filters before file checks and does not hash the still when same-size current companions satisfy the existing skip rules. New companions use a verified still filename. Existing companions with older numbered still stems remain usable after hash verification; Kei does not rename or delete them. Historical smart-folder membership cannot complete local path reconciliation. It reports separate conditional evidence when the fresh smart query is its only remaining dependency: album snapshots and catalog identity must be complete, with no deferred missing file or failed local work. The cycle owner can discharge that dependency only after successful current smart enumeration and durable work. During pending path drift, recent-count and lower-date-bounded selections drain ordered provider queries to EOF while forwarding only their configured selection to the existing download planner. The global recent frontier also drains to EOF and must report the same usable token as every selected pass. This costs a complete query on path drift, with bounded retained frontier IDs and streaming tail filtering. Cancellation stops the raw drain. Fresh query evidence certifies selected local work separately from source checkpoint coverage. Any excluded tail, including equal-date global-frontier IDs, still holds zone and database checkpoints. Stale plans, interruption, unresolved identity, failed enumeration, local work or state transactions retain the pending path hash. Every selected library must finish. Atomic promotion compares the active path hash and exact pending target before publishing the new hash and removing its marker. Eligibility inventory-and-bridge promotion remains separate. Unchanged cycles skip catalog reconciliation and the exhaustive path proof; ordinary bounded smart-folder refreshes may still query the provider. Local path reconciliation retains an unowned legacy master with multiple historical children as unresolved work, even when targeted lookup returns one usable sibling. It does not reserve that master's path or relax preservation eligibility. Independently owned current children can still reconcile, and later complete inventory can prepare preservation and release the checkpoint hold. A historical child media receipt at a protected original is not a current claim when the exact mapped child and rendition have a separate ready, source-backed current file. A missing current file remains unsupported by this historical exception; historical rows do not authorize initial materialization. The state owner snapshots the family dependencies and rederives that exception inside preservation preparation. The file owner retains confined handles, verifies their bytes and namespace immediately before commit, and refuses cancellation or drift. Historical receipts, originals and owner facts remain unchanged. Explicit local-byte acceptance does not satisfy this exception. Ordinary adoption and skip decisions cannot reuse a protected original merely because a child has a historical receipt there. Local path reconciliation also loads these publication receipts. A verified copy in the selected pass satisfies that rendition even when the catalogue's current path belongs to another album. Reconciliation preserves a verified numbered still filename when planning its companion, so changing Live Photo selection or album order does not create another copy of an owned pair. Local path reconciliation opens source and destination leaf entries without following symlinks. It hashes the opened file and rechecks its identity before accepting a destination, including entries that appear during publication. Temporary copies use new unique names, verified bytes, and no-overwrite publication. On Unix, they start with owner-only permissions, so a failed copy remains private. A completed copy receives the source permissions. An unsafe or replaced entry preserves the previous catalogue path and leaves reconciliation incomplete. A blocking reconciliation failure skips that library's normal source/download pass, so adoption cannot bypass the rejection. Its provider checkpoint and the aggregate database pre-check token remain unchanged. Ambiguous temporary entries remain for inspection. Reconciliation retains directory capabilities through state finalization. Configured roots, recorded sources, destinations, and temporary siblings must not contain `..` components. Reconciliation rejects these paths before lexical normalization or filesystem writes, because a linked preceding component can change which directory `..` selects. Ordinary relative roots remain supported. Descendant directories are opened without following symlinks. Source reads, temporary creation, validation, and publication use these retained handles; namespace replacement leaves reconciliation incomplete. Windows directory handles deny rename and delete sharing while the receipt is live. Unix checks that the retained parent still occupies the planned namespace before accepting the result. Publication and temporary-file checks remain descriptor-relative. When the configured root changes, the recorded old source is opened beneath the common lexical ancestor of the old path and new root, with no-follow traversal below that anchor. This allows root moves without trusting linked source directories. Destination writes stay beneath the new configured root. Relative paths are normalized inside the filesystem owner, not in durable catalogue keys. A reconciled media copy receives the same capture mtime as a normal download. If the retained source and destination paths resolve to the same pathname, reconciliation only updates the catalogue path spelling. It preserves media mtime and existing or absent sidecars, then validates media before finalization. This permits changes between relative and absolute download roots. Before changing timestamps, reconciliation rejects a distinct destination that shares the source file's identity, including a hard link. The source mtime and catalogue path stay unchanged until the conflicting entry is resolved. When XMP sidecars are enabled, an existing source packet is validated and copied byte-for-byte, including custom properties and ownership markers. Path migration does not upgrade or regenerate an existing packet. If no source sidecar exists, the normal metadata planner generates one from the current payload and source GPS facts. A source read failure stops before publication; it cannot leave an incomplete packet that blocks the next attempt. Generated packets do not infer native accuracy provenance from a local checksum. Reconciliation reserves recorded and planned paths by asset and rendition. Keys use the filesystem owner's checked absolute paths, so equivalent relative and absolute roots share ownership. These keys do not change task or catalogue path spellings. Invalid reservation paths leave reconciliation incomplete and block file finalization. A collision with another asset or rendition selects a stable identity-suffixed path. Each rendition can reuse its own reservation across passes and retries, including after a partial state-write failure. Existing files alone do not select another name; the confined copy owner checks their bytes and rejects conflicts. Sidecar publication uses the same retained directory capabilities and refuses conflicting destination bytes. Reconciliation records the new catalogue path only after capture mtime, sidecar work, and final input checks succeed. It retains the old files and keeps failed work pending. Reconciliation planning includes existing media destinations so a retry after a metadata or state-write failure can finish without downloading or creating a second media copy. Reconciliation retries the selected destination and leaves conflicting media or sidecars untouched; it does not allocate another collision filename on each attempt. Ordinary download collision naming and on-disk skip rules are unchanged. Ordinary download planning also treats a state-protected legacy original or exact sidecar as occupied when the directory cache cannot see that spelling, and selects an independent identity suffix instead of targeting the protected identity. Legacy preservation inventory failures retain the existing reason and budgets. `legacy_inventory_failure_v2` adds fixed phase/subreason labels, whether a valid terminal marker was observed, and the failing child reference's known family context and soft-deletion flag. A missing master reference leaves family membership unknown. Context describes the failed validation, not counts for all records. EOF observation is not successful hydration or checkpoint proof. These fields exclude identifiers, tokens, URLs, paths and provider error text. ### Full and incremental enumeration Debug builds can set `KEI_REQUEST_DUMP_DIR` to capture Photos session bodies as `000001.req..json` and `000001.res..json`. The counter pairs requests and responses; timestamps are captured separately. Bodies are unredacted and private; request URLs and headers are not captured. Use private local storage and redact bodies manually before sharing. This dump is not a redacted doctor bundle. HTTP error bodies retain the existing size bound and may contain non-JSON text. Successful responses are serialized from parsed JSON without pretty-printing. Writes are best-effort, with no run subfolder or cleanup. Release builds without debug assertions do not include this diagnostic code. A signed URL expiry stops the current full-download batch. The pipeline retains failed tasks, including queued tasks cancelled by that expiry, and makes one cleanup attempt through targeted asset and master lookups. User shutdown stops cleanup lookup and download dispatch. It does not enumerate albums again to refresh those URLs or rebuild unfiled exclusions. Cleanup updates only the URL on an already-selected task, preserving its destination and publication authorization. The library, child identity, rendition, checksum, and size must still match. Changed resources, missing lookups, a second expiry, and cancelled tasks retain durable retry work. Successful cleanup does not convert incomplete enumeration into checkpoint proof. `exact_refresh_outcomes_v1` separates requested/refreshed/remaining task keys from planned unique child identities, observed child results, planned paired identities and observed paired results. These are identity counts, not HTTP request counts. Planned counts may include work cancelled before a result. Fixed child/task rejection categories distinguish selected-master, rendition, checksum and size mismatches. Task rejection counts deduplicate durable task keys; categories are observations, not permission to change selection or retry. Authentication failures during refresh stop cleanup and return a session-expired outcome. Refresh rate-limit observations contribute to the cycle count even when provider retries are exhausted. The `expired_url_refresh_failed` diagnostic reports failure, authentication, and rate-limit counts without provider details. Collecting incremental preflight and expiry recovery use the same bounded child/master lookup owner. The authenticated session and exact requested zone scope each lookup. Explicit zone/owner evidence and child/master references must agree; incremental recovery also pins the originally selected master and provider rendition. The selection owner translates virtual RAW/JPEG task keys back to provider keys once, before refresh; later provider policy changes cannot retarget the selected resource. Refresh changes only the URL when child, master, rendition, provider checksum and size still agree. Unmatched resources retain durable failure work; expiry and lookup absence never authorize deletion. No zone enumeration, path replanning, or cross-instance mapping import is used for URL refresh. Explicit-task passes isolate CDN expiry to the failed resource so healthy peers finish, including the single cleanup retry. Authentication thresholds, user shutdown and fatal state-write failures still stop dispatch. Interrupted and queued unfinished tasks remain in the returned failure debt and are persisted before checkpoint decisions. Only successful exact tasks retire retry debt. A collecting pass makes at most one expiry refresh and retry after its optional aged-URL preflight. Failed resources stay durable for the next cycle. Refresh logs record phase start/completion, counts and elapsed time. Refreshed resources carry their lookup-observation time into worker dispatch for a DEBUG age measurement. The first enumeration observation is labelled separately; it does not measure a refreshed URL's age. Neither observation establishes provider issuance time or expiry, which remain unknown. Full enumeration streams records/query results and gathers a provider token from every active pass. Natural stream completion and usable, unanimous pass tokens are the authoritative proof. Count probes and pagination differences are diagnostics. Recoverable pass-token gaps can be retried in the same cycle. Download orchestration normalizes each child to its `CPLAsset.recordName` before streaming or collecting paths plan it, so page boundaries and sibling order cannot change its state ID. It retains a legacy master-keyed state ID only when the provider checksum matches and durable identity history permits that child to adopt it. Before a download pass adopts a legacy master-keyed row, it atomically records that child as the durable owner. Later sibling mappings cannot change the owner. Print-only and dry-run paths select from existing state without claiming an owner. Cleanup retries carry the child record name and reapply the state ID selected by the first pass before they rebuild download tasks. The iCloud adapter validates a complete changes/zone page before pairing records, invoking hydration callbacks, or accepting its successor. A page must contain one zone matching the requested name and any requested owner, an explicit records array without record errors, and a nonblank token. Continuing pages must use a new cursor. An empty terminal page may return the request cursor. Rejection emits no records from that page; earlier accepted pages remain replayable from the durable checkpoint. These checks also protect raw hydration and source change validation. Source change validation retains its stricter record identity and cancellation requirements. Diagnostics omit provider cursors and identifiers. Incremental enumeration consumes changes/zone events. It persists provider identity mappings before applying created, soft-deleted, hard-deleted, or hidden transitions. An asset-only `CPLAsset` creation hydrates its paired master through `masterRef`, the durable mapping, or a targeted identity lookup before routing; an inconclusive lookup preserves the prior zone checkpoint. Album snapshots and smart folders may require targeted refresh work before or alongside the incremental stream. Both strategies use `IncrementalDeltaState` in `orchestration/delta.rs` for identity mappings, source-state transitions, album and relation bookkeeping, event counts, and completion tokens. The streaming strategy emits created assets through a bounded channel and defers album, relation, and unpaired-asset work. The collecting strategy observes all events before state writes, hydrates missing identities, and applies relations before routing created assets. These orders preserve each strategy's token-safety and selected-album behavior. Shutdown races provider-event and successor waits in both strategies; the owned receiver is dropped before the successor wait to release provider backpressure. Receiver closure cancels pending changes/zone requests and retry delays, but does not interrupt an already-started durable shadow capture. Cancellation during either consumer wait supplies no replay-completion proof. Downloadable photo records require non-blank `CPLMaster` and `CPLAsset` record names and a usable `assetDate` before they enter filtering or path planning. Full enumeration reports a malformed record as incomplete. Incremental enumeration marks its zone token unsafe. Both routes preserve the prior checkpoint so an unchanged provider record remains retryable, and neither route counts the record as a policy, filename, or date skip. Recent and date-bounded runs may advance only when the producer proves the bound did not truncate the stream. Incremental count-form recent selection uses the full selection owner after complete strict delta replay. Global uses the library-wide ranked window before filters; per-filter uses each pass's window. Delta arrival order and asset/pass-pair counts never define that window. `orchestration/recent.rs` stores a fixed-size `recent_selection_recovery:` receipt in the account-owned database. It binds the resolved provider/pass scope, exclusions, selection and download configuration, and the source cursor hash. Missing, malformed, mismatched or pending receipts require current inventory even when the next delta has no changes. The pending receipt is written before selection. A completed receipt is usable only after its actual changes/zone successor becomes the requested cursor, so a failed checkpoint commit cannot skip recovery. Cap increases and removal consume this obligation without a manual token reset. Watch prechecks include this local recovery even when the provider reports no changed zones. Enumeration safety hash version 3 includes count and recent scope, forcing one conservative inventory/bridge for legacy cursors, including prior cap removal, before trusting quiet streams. Selection rank tokens prove only the existing selection EOF contract. The combined result returns the delta successor and carries separate completed replay evidence for configuration bridges; a pure full fallback has no such evidence. Existing identity, state-write, interruption, sparse/legacy and bounded-inventory vetoes remain authoritative. Unresolved delta identity does not starve independently valid recent media selected by current inventory; its source veto, failed-write counters and pending recovery receipt remain in force. Current queue/publication guards remain authoritative for each job. A truncated count window keeps its recovery obligation and prior cursor; repeated bounded inventory remains a known cost until a larger catalog selection generation stage qualifies it. Typed provider session errors retain the existing bounded reauthentication route across current selection; other selection failures do not trigger fallback. This receipt neither claims an atomic rank snapshot nor proves absence or permits local deletion. ### Download and publication ```text PhotoAsset -> planner::TaskPlanner -> pipeline::run_download_pass -> file::download_file -> optional metadata_rewrite -> verified .part publication -> finalize downloaded or failed state ``` Only producer-dispatched work becomes pending through `upsert_seen`. Filtered or skipped assets must not be left as retryable work unless a dedicated state transition owns that result. Full enumeration divides the configured download-worker limit across concurrent album passes. Each pass gets the same allocation, rounded down. The remainder stays unused so replacement passes cannot exceed the total limit. Passes keep their existing cancellation, publication, and state-finalization paths. Asset dates used for filtering, path rendering, file metadata, and sidecars are resolved from Apple's UTC instant plus the asset's `timeZoneOffset` when that offset is usable. Missing or invalid offsets retain host-local rendering, preserving the previous path and metadata behaviour. Date-only lower bounds use a conservative UTC enumeration bound and then apply the exact resolved calendar-date filter, so enumeration safety does not depend on the backup host timezone. Capture-local path rendering does not change the download config hash, so a sync that sees no other drift stays incremental and never re-enumerates an already-downloaded asset. Date-only created bounds change the eligibility hash but not the path hash, so expanding a date window runs the required inventory without treating existing media as path drift. Matching legacy hashes that mixed date eligibility into path state migrate without reconciliation. When a legacy hash is ambiguous, reconciliation uses the same durable path-family and file-integrity proof as normal sync to discard the planned collision task before copying or changing state, so an asset cannot collide with its own recorded file. For an asset carrying a usable offset, a path derived under host-local rendering is not a current derived path, so a full sweep forwards that asset and downloads it into its capture-local folder. The earlier copy stays where it is: kei never deletes local media, and no command removes a file that no longer matches a derived path. `import-existing` adopts through the same derivation. Assets without a usable offset retain host-local paths and remain compatible with icloudpd's date-folder layout. Existing embedded and sidecar timestamps are repaired only through the explicit `sync --refresh-metadata` flow, which is bounded by the same no-overwrite probe gate as any other embedded write. An offset tag names the zone of one specific timestamp, so the embed path attaches it only to a timestamp it writes in the same pass, or to an existing one the probe proves already renders the capture-local instant. When writing a timestamp into a file that has orphaned datetime offsets, the writer clears those offsets before installing the timestamp and its resolved offset. Attaching an offset to a wall clock left by host-local rendering would assert an instant the asset never had. `sync --refresh-metadata --repair-capture-timestamps` explicitly relaxes the no-overwrite gate for state-recorded downloaded files. It requires embedded datetime output, a usable Apple offset, and bytes that still match the recorded checksum. Rows without that provenance, and files whose embedded format is not supported by the active build, remain pending. The writer replaces the timestamp and its offsets together through the stable-input publication path. For HEIF media, an existing native Exif timestamp is patched in place with its paired offset before XMP is written; ambiguous, shared, or unsupported native layouts remain pending. Normal downloads and ordinary metadata refreshes still preserve an existing timestamp. Capture repair is separate durable debt, so an ordinary metadata drain cannot retire it. If ordinary embedded metadata would change the same media while capture debt is pending, that embed waits for the explicit repair; verified no-write and sidecar-only work may complete independently. ### Durable pending retry Failed and pending rows are not recovered by replaying the entire provider inventory. The retry owner: 1. Removes only work already proven source-deleted. 2. Resolves current provider records in targeted batches. 3. Uses durable asset/master mappings and checksum/size evidence for legacy identities. A persisted legacy owner resolves matching siblings without changing the selected child. 4. Adopts an existing matching local file when safe. 5. Marks current filter exclusions as policy-excluded. 6. Persists unknown or transient verification state. 7. Queues exact unresolved asset/version/path tasks. Unknown identity is not permission to delete or forget work. A pending rendition can retain a valid prior publication after a later download attempt failed. Ordinary sync and targeted retry recovery may reuse only its current catalog path when prior publication time, unchanged provider checksum and size, a matching path receipt, equal recorded local/download hashes, the current filename family and positive catalog/reservation ownership all agree. Recovery runs before fresh plan choices are saved; recovered renditions and failed recovery writes stay out of transfer and reservation dispatch. A healthy rendition about to receive a fresh choice uses the same current-file proof so that a later failure can still recover it. Existing durable choices keep their replay policy; saved different destinations still retain debt. The file is hashed beneath the current root without following links. A retained file and namespace, plus the process's destination guard, remain live through immediate conditional state finalization. The state owner rechecks the row, receipt, identity, protected legacy evidence, reservations and provider generations in one transaction. Recovery preserves the original path, publication timestamp, all receipts, source provenance and metadata debt; it does not mint provider progress from local bytes. Unknown identity or a saved different destination remains pending. Cancellation before commit or a failed state write cannot dispatch a fallback copy from that recovery attempt. The destination guard coordinates participating writers in this process; it is not an interprocess or network-filesystem lease. Managed selection keeps its separate destination and source-provenance finalization. Policy-excluded rows stay outside the actionable pending reader. After a successful source pass, targeted revalidation checks their durable provider identities only for explicit deletion. Present, omitted, malformed, and transient responses retain the policy-excluded rows. A provider checksum change makes the previous local path historical. Retry adoption requires durable proof that the path holds the current provider version and that its filename matches the recorded task filename. ### Import-existing `src/commands/import.rs` shares configuration, selection, pass planning, and path derivation with normal sync. Before matching or consulting prior imports, it uses the download context's identity selector to install the same child or guarded legacy master state ID as sync. It prepares selected rendition rows and paths for every selected pass before adoption, then checks the complete candidate census. Incomplete enumeration, fetcher panic, or cancellation before completion leaves all adoption rows, mappings, and legacy owners unwritten. The preflight uses memory proportional to selected renditions, all durable catalogue paths and current receipts, and cached directory entries. Metadata and receipt slices are shared; raw provider JSON is not retained. There is no fixed memory cap or disk spool. Catalogue paths include historical and omitted assets, so an unselected owner still prevents adoption of its file. Equivalent relative and absolute paths use the same confined ownership key. Import uses sync's collision-family owner for full-ID and ordinal filenames. Different-size candidates remain distinguishable. An ambiguous short name, sanitized full-ID collision, or multiple ordinal siblings is refused unless an exact current-generation durable receipt and verified local SHA-256 identify the file. Provider MMCS checksums are compared to provider checksums, never to local SHA-256. It optionally compares remote prefix bytes, hashes the local candidate, and calls `ImportStateStore::import_adopt`. Adoption commits the provider child/master mapping and any compatible legacy owner in the same transaction as the imported row. Failed adoption rolls back both. Dry-run selects identity without persisting mappings or ownership. An unclaimed legacy row completes adoption before it can skip later rehashing. Adoption checks durable destination reservations in the same transaction as catalog updates. A different asset, rendition, or provider content generation cannot adopt a reserved path. Unknown legacy content remains occupied. A refused adoption logs a warning and does not count as a match. Size/mtime snapshots may skip later rehashing only while the current provider checksum, path, size, mtime, and stored local checksum still match. Ambiguous paths always require a local hash verification before that shortcut. ## Data-safety invariants ### File landing The media publication sequence is: ```text write or resume .part -> validate response and content length -> validate expected size -> validate SHA-256 checksum -> validate content type and sniffed bytes -> apply configured pre-publish metadata -> publish without replacing an existing final path -> fsync the parent directory -> finalize SQLite state ``` The transfer owner opens an existing resume file through a retained parent handle with no-follow access and requires a regular file. It reads the Range offset from that file handle and uses the same handle for append writes. Resume probes reject symlinks. Identity checks reject a changed leaf before body writes, content validation, and publication. A final identity or byte check must pass before downloaded state can finalize. The task retains the file through metadata work and publication; an opt-in metadata replacement must match the writer's output fingerprint before it is accepted. The durable temporary-path claim alone does not authorize an append. Staging names use fixed-length SHA-256 digests of the lossless absolute destination and provider checksum generation, encoded as `kei-v1--`. MMCS checksums select a provider generation; they do not verify downloaded bytes. Different destinations have independent staging even when their bytes match. `file::staging` acquires an in-memory destination guard before claiming or inspecting a part. The task retains it through retries, resume, metadata, publication and claim retirement. Different generations and suffixes for one destination share that guard. Cancelled waiters cannot retire a peer's claim. The weak registry upgrades and prunes entries under one short mutex; active owned guards and waiting futures keep the shared destination mutex alive. Completed and cancelled guards retire dead entries. No permanent lock files are created. This coordination covers workers in one kei process, including separate databases used inside that process. It does not coordinate independent processes or hosts; existing cross-process limits remain, including network filesystems whose advisory locking is disabled or unsupported. Cleanup of versioned staging names takes the same destination guard through claim retirement. A busy guard preserves both bytes and the claim. Legacy checksum-only files are never adopted into a new destination's staging identity. Existing exact-path ownership claims can still authorize their stale orphan cleanup under the same age and confinement rules. Unclaimed legacy files remain untouched; this change performs no migration or discard. The streaming consumer cancels and drains after its authentication threshold, so detached filesystem or metadata operations complete before guard release. A resumed request rejected with HTTP 416 gets at most one fresh request per attempt. The retained part stays intact until an acceptable response can restart the transfer; a second 416 terminates that attempt. Transfer retry pauses and bandwidth waits observe shutdown before another request or chunk write. A cancelled bandwidth reservation is refunded, and interruption reports the retained byte count without publishing or finalizing the file. Before returning a body-stream error or interruption, the transfer owner flushes pending filesystem writes. Retry offset reads and temporary-claim retirement therefore follow write completion. A flush failure is a disk error rather than a resumable transport error. Successful transfers also sync the data before validation and publication. Schema v19 records the exact temporary path before a state-backed download can write or resume it. Normal completion and graceful interruption retire that claim. A process crash leaves the claim as durable cleanup authority. Later cleanup inspects only claimed paths, rejects every symlink in the path, and holds verified directory or file handles through removal so an ancestor swap cannot redirect deletion. It retires the claim after deletion or when the path is missing or no longer the claimed stale file. A suffix and file age alone never authorize deletion. `kei sync --repair-truncated` is the only media-replacement path. Pending retry planning requires the durable reconcile truncation marker, confirms the recorded file is still truncated, and fingerprints its bytes. After the new download and configured metadata writes pass, the file owner replaces only that exact fingerprint. It uses atomic exchange where supported and the journaled fallback owned by `src/download/file/replacement_recovery.rs` otherwise. A changed target is restored or retained and the state row stays failed. All other downloads keep no-overwrite publication. When no-overwrite publication finds an existing destination, the file owner compares it with the verified `.part` file. Identical bytes are deduplicated. Different or unverifiable bytes return a typed collision and retain the `.part` file. The pipeline records the task as failed before it can write a sidecar or finalize downloaded state. A file may be safe on disk while its state write is still a cycle failure. Do not weaken deferred state-write handling or infer that a visible final file means the database transition succeeded. ### Checkpoint advancement `SyncResult.checkpoint` carries the per-zone completion, interruption, state durability, identity, and token-block evidence. The source-checkpoint classifier reads this evidence and the result's source token and outcome. It does not read the reporting snapshot in `SyncResult.stats`. Full enumeration obtains evidence directly from pipeline finalization. Attached pending recovery and metadata-capture repair also supply execution evidence. Same-zone composition combines this evidence once, then projects its counters into the report. Recovery pass and record lists stay in the per-zone evidence; cycle reports do not accumulate them. Report fields and reason values keep their existing JSON representation. Incremental result assemblers still use `SyncResult::from_execution` to capture their execution statistics at the result boundary. This is the boundary of the pilot, not a second authority after result assembly. Later composition must use the captured evidence, never reconstruct it from a reporting snapshot. Query token holds, inventory/delta bridges, and unresolved identities update the evidence in their existing policy owners. Preserve the zone checkpoint on: - Dry-run - Stale pass planning - Session expiry or interruption - Incomplete enumeration - Non-durable state - Missing, blank, mismatched, or otherwise blocked token proof Transfer and metadata failures may use a newer checkpoint only when the recovery work needed after that checkpoint is durable. ### Metadata Provider metadata may be captured in SQLite without changing local media. Embedding EXIF/XMP or writing sidecars requires explicit configuration. The iCloud GPS decoder selects the first valid coordinate pair from `locationV2Enc`, `locationEnc`, then plain latitude/longitude fields. Coordinates must be finite and within inclusive latitude [-90, 90] and longitude [-180, 180] bounds. Nonfinite optional altitude is omitted independently of the pair. The iCloud adapter retains shared metadata once and compact resource facts for each rendition, independently of download URL availability. Catalogue dimensions describe the provider-declared resource dimensions, not probed track or displayed dimensions. Live Photo motion duration comes from the companion duration ratio; missing or invalid companion facts remain unknown rather than inheriting still-image values. Planning, adoption, and import project metadata for the selected resource. RAW preference can swap logical original and alternative keys; download policy supplies both resource candidates for those historical keys. Refresh resolves every rendition against its current provider checksum inside the state transaction, not against preloaded row identities or current RAW preference. Missing or conflicting matches update shared library metadata but leave dimensions and duration unknown. Full metadata snapshots and hashes are computed only when needed, rather than retained for every possible rendition. Refresh updates the downloaded rendition family atomically, checking each prepared capture-repair receipt against its own rendition's incoming hash. An individual upsert guards the affected rendition and provider checksum without invalidating another rendition's prepared receipt. Failed receipt validation or state writes preserve checkpoint failure evidence. HEIF probes and embedded preparation share one process-wide blocking admission lane, independent of download concurrency. This conservative admission bounds simultaneous retained parser work; it is not needed for shared parser state and also serializes file I/O. Concurrency may be widened only with measured resource evidence. Production keeps a compact view of the `pitm`, `iinf`, `iloc`, and `iref` controls. Other `meta` children, including image-bearing `idat`, stay as file ranges. The separate file-backed source-GPS locator also budgets its required control tables rather than the complete `meta` body. Headers are scanned without a growing top-level or opaque-box inventory. Preparation patches changed controls and streams all unchanged ranges with 64 KiB buffers; preservation checks compare original file ranges with fixed buffers. A large `meta` or `idat` payload alone is not a resource refusal. The retained-control view, selected XMP packet, complete native Exif item, and replacement XMP packet each have an 8 MiB materialization allowance. This reuses the prior writer's 8 MiB whole-`meta` allowance for the data still materialized; the selected Exif/XMP allowances are new operation restrictions, not format limits or a claim about all valid assets. Complete native TIFF parsing/alias validation and the XMP toolkit graph still require materialization. Larger operations return an explicit resource-unsupported error naming the required bytes and allowance, preserve the original, and retain truthful metadata retry work. The allowance is not a total-process RSS bound: C++ graphs, parsed structures, source/output control views, and fixed stack buffers also consume memory. Existing fixture sizes do not establish a representative camera/iCloud metadata distribution. Source fingerprints are checked before and after preparation, and guarded publication rechecks both source and output. Errors, panics, and dropped prepared work release admission and the existing temporary-file guard removes owned output. Native-only builds identify unsupported HEIF before the generic JPEG/TIFF probe or preparation can read the whole file; approved-input checks still use streaming fingerprints before preparation reports unsupported. HEIF-family embedded XMP updates use the byte-preserving item-map writer and reject layouts that cannot be changed without re-encoding unknown item-graph data. A file may hold one XMP item per image, so probe and write both resolve the packet through its `cdsc` association with the primary image and refuse item maps where several are equally plausible. An unassociated packet answers for the primary only when it is the sole candidate. The HEIF probe reads both XMP and the standalone Exif item associated with the primary image before planning datetime or GPS writes. A new XMP item receives a `cdsc` reference naming the primary image. When the primary is the first input of one `tmap` and its sole Exif descriptor already names exactly the primary and that tone map, and no existing XMP item already describes that tone map, the XMP receives the same two targets so Apple keeps the HDR rendition. Missing, conflicting, additional, or ambiguous relationship or ownership evidence fails closed. External item data references and top-level boxes whose absolute offsets are not adjusted are also rejected. Before publication, validation confirms that every construction-method-0 item other than the resolved XMP packet, and every opaque `meta` sub-box, remains byte-identical, and that re-reading the rewritten file resolves the packet just written, allowing for the space padding an in-place replacement leaves in the reused extent. Every embedded writer exclusively creates a unique sibling and replaces the source only while the displaced bytes and prepared replacement still match their approved fingerprints. Atomic exchange is preferred. Metadata-only retries also pass the checksum-gate fingerprint into the writer, closing the interval between catalogue validation and the writer's read. Existing regular files and links at candidate temporary paths remain untouched. Safe pre-exchange failures remove only the uniquely owned prepared file. Concurrent edits preserve the source and its catalogue checksum evidence, keep the durable rewrite marker, and leave any ambiguously displaced entry at its reported sibling path. The replacement file retains the source permissions and mtime. On Linux, filesystems that reject `RENAME_EXCHANGE` use a journaled fallback. An exclusively created, owner-only sibling directory holds a version-1 JSON manifest with lossless filename bytes and both SHA-256 fingerprints. Its name is `.kei-replace-` plus the SHA-256 hash of the destination filename. The prepared file is hard-linked into that directory and synced before the original is displaced. Installation and restoration use hard links that refuse an existing destination. The final path can be absent between displacement and installation. Directory syncs precede the durable commit marker. Normal completion removes only byte-verified recovery entries. Before download discovery or catalog path reconciliation, Linux walks the download tree through retained directory handles and recovers these journals. The configured root may be a symlink alias; recovery resolves that trusted anchor while refusing links beneath it. Reconciliation also recovers each recorded source media and sidecar before planning, including old roots after a directory change. Missing parent namespaces fail verification rather than being treated as missing journal entries. Directory reads require `/proc/self/fd`. Journal links are rejected. Dry-run and filename-only download modes do not recover journals. Queued metadata work also checks its media and sidecar journals before testing existence or checksums. Recovery restores uncommitted originals or completes cleanup of committed replacements. Unknown versions, malformed manifests, changed bytes, and conflicting destinations retain the journal and fail the operation. Journal file locks prevent local writers from recovering an active transaction. Terminal cleanup holds the lock through manifest unlink, then closes all manifest handles before directory removal. Only `ENOTEMPTY` from removal of a reverified empty, manifest-less journal is deferred with a warning; publication still verifies the target and removes only its matching prepared inode and bytes. Unknown entries, changed bytes or namespaces, other removal errors, and directory fsync failures remain errors. Empty journal recovery applies the same bounded cleanup. Historical prepared files are not swept by this path. NFS mounts with `nolock` do not provide cross-host locking; do not run kei on the same destination from multiple NFS clients. Recovery does not authorize provider checkpoint progress or downloaded-state finalization by itself. `METADATA_CAPTURE_REVISION` identifies the catalogue semantics produced by the current binary. Schema v18 stores per-asset revisions and per-library active and pending repair state. A normal sync hydrates stale downloaded rows in bounded provider-lookup batches, independent of album, media, and date filters. Only an identity that cannot be resolved from durable asset/master evidence uses the bounded legacy hydration path. Unselected libraries keep separate pending state and do not force work in selected libraries. The legacy missing-hash fallback checks only the selected library. While revision repair has pending work, it owns that backlog instead of forcing a second full enumeration. Ambiguous provider children remain pending and block the affected checkpoint; a matching rendition alone does not prove which child owns the metadata. The `metadata_capture_ambiguity_counts_v1` diagnostic reports stored rendition, matching child, and full-evidence matching child counts. It contains no IDs, paths, checksums, or provider metadata. These counts do not authorize identity selection, even when only one child matches every stored rendition. Schema v28 can preserve an unclaimed legacy master receipt with at least two historical child mappings without assigning it to a child. A complete current inventory must contain at least one live child. A sole survivor must satisfy the same independent current-child receipt and file checks as a multi-child family; historical siblings missing from that inventory do not establish ownership or authorize source deletion. Zero-current-child families and saved legacy owners remain excluded. `download/legacy_preservation.rs` owns eligibility and file validation; `state/db/legacy_preservation.rs` owns immutable evidence and generation-fenced receipts. Migration creates empty tables and protection triggers only. It does not classify or rewrite existing records. Schema-27 binaries reject a migrated database; read-only reports require the current schema and never migrate it. Preparation protects the original catalogue rows, capture revisions, retry receipts, relationship history, media paths and sidecar presence. It preserves the old checkpoint. A valid original path already claimed by another catalogue asset defers that family without creating preservation receipts or changing its ownership, retries or checkpoint. Independent families in the same preparation scan can still progress; malformed evidence remains an error. Before preparation, unclaimed masters with multiple historical children already protect every catalogued original path and its exact `.xmp` sidecar across libraries. Pending metadata debt does not remove that protection. Ordinary writers, import adoption, publication proofs, layout handovers and replacement recovery share this path census; deferred originals retain their rewrite markers. A complete unfiltered zone inventory must cover hidden children, all pages, an explicit completion marker and a nonblank final cursor. Each scan is limited to 10,000 pages and 1,000,000 observed records. It retains compact latest identity/reference/deletion evidence for unrelated records and full latest records for candidate masters and children, then hydrates those families after the final page. Historical child mappings cannot replace complete family discovery. Later updates and tombstones replace earlier evidence, including references that enter or leave a candidate family. The byte guards allow at most 64 MiB for one serialized response page and 256 MiB for retained record representations, identity storage and pagination cursors. Cumulative transferred bytes are counted separately and do not consume the retained-evidence budget. These counts are representation bounds, not process RSS; one response is decoded before its page-size guard. A cycle considers at most 64 new candidates. Exhausting a limit retains the hold. Sparse, malformed, cross-library or incomplete evidence cannot activate preservation. No diagnostic count is ownership proof. Failed preparation discovery records a versioned, library-scoped `legacy_preservation_inventory_retry:` receipt in the metadata table. An unchanged configuration and durable original-family evidence defer another preparation scan for at most one hour across restart. Retry scheduling, capture cycle counters and unchanged relation refresh timestamps do not reset that delay. Changed configuration or original-family evidence permits a new attempt; malformed, unsupported or out-of-range retry receipts never defer work. Deferral supplies no provider evidence or checkpoint permission. Prepared records still require fresh certification, and metadata-capture retry scheduling remains independent. Successful discovery clears its failure delay. Preparation and certification use separate complete scans; a preparation result cannot certify work performed later. Failure warnings expose only a fixed stage and category plus aggregate page, record, transferred-byte and retained-byte counts. Provider error text, record identities, paths and URLs remain private. Activation requires a normal completed current inventory and, when a retained cursor exists, a successful delta bridge. Independently identified current children need separate, finalized receipts and verified files for every selected rendition. Qualification requires independently verified per-path source provenance for the exact child, rendition, provider generation and catalogue path, with a freshly verified current local SHA-256. Provider MMCS signatures are generation identities and are never compared to local content hashes. Historical download hashes and publication without durable source provenance remain insufficient. A new unverified publication that changes local bytes clears prior source provenance; verified metadata rewrites retain their source receipt. Exact-byte relocation can reuse an unambiguous source receipt bound to the same library, child, rendition and provider generation. Pending rewrite, capture-repair or path-reconciliation debt keeps this conservative qualification blocked. Retained reconciliation destination choices are naming evidence, not pending work, and remain part of the dependency comparison. No metadata writer is enabled automatically. A replay that creates debt holds the cursor; a later cycle may qualify after configured current-child work completes. Original media and sidecars are fingerprinted through confined paths. Symlinks, pending temporary writes, replacement journals, shared paths and changed bytes reject qualification. Multiple hard links also reject qualification by default. Preparation warnings retain the aggregate `invalid_original_files` candidate count and expose `shared_file_links` as a subset, counting each candidate once at its first rejected media or sidecar. Typed shared-link errors also expose only the fixed reason `shared_file_links` at certification and checkpoint stages. The TOML-only `[download].legacy_preservation_allow_hardlinks` flag defaults to false. True waives only the hard-link-count check for original and current-child media and sidecars during legacy qualification and revalidation. Actual use emits one redacted aggregate warning per library cycle, deduplicated by evidence path; enabling the flag without linked evidence emits no warning. Other confined-path, byte, receipt, debt and checkpoint validation stays intact. Preparation, certification and checkpoint validation consume the same flag. Its value is bound into the separate legacy policy hash; ordinary download and enumeration hashes remain unchanged. False retains the previous legacy hash. The opt-in assumes external tools are coordinated so they do not change linked contents, metadata, or kei-managed names during the operation. It does not prove inode independence: distinct original and current paths can share an inode. It adds no account-wide alias census or read-only enforcement. Existing identical-file reuse can set timestamps through aliases even when metadata outputs are disabled. Metadata repair, timestamp repair, pending-file handling and replacement recovery remain unchanged. General hardlink support and stronger preservation guarantees remain open under #884. Shared-root replacement recovery checks protected paths from all libraries before opening a journal, including unselected libraries. Missing protected parents retain their path reservation through the nearest existing ancestor; they do not abort unrelated recovery or clear preservation/checkpoint holds. Other filesystem errors and existing protected journals still block recovery. Current child files and sidecar presence are also bound into the receipt. The preservation owner never moves, rewrites, adopts or deletes original paths. With the hardlink opt-in, that path protection cannot prevent changes made through another alias by an unchanged write path. Normal downloads, metadata repair, local path reconciliation, imports and source tombstones respect the protection before byte writes; SQL guards provide defense in depth. `reconcile` reports drift in protected originals without converting them into download retries. Verification and manifest export retain the original identities, dates, paths and history; download status does not establish ownership. Unprepared ambiguous originals use the same protection census, including supported relative paths and their exact sidecars. A connection-local SQLite TEMP cache indexes confined path keys; relevant identity and path changes invalidate it transactionally, and external commits, schema changes and working-directory changes require a fresh census. Ordinary writes and mapping timestamp refreshes do not rebuild it. This cache does not change durable preservation evidence. Source metadata recovery distinguishes protected legacy originals from healthy managed primary-layout sources. Managed sources continue through their layout owner; pending media or sidecar replacement journals still hold reconciliation. The final SQLite transaction compares the original snapshot, current child and family evidence, receipt generation, configuration metadata and prior cursor. It appends the proof and commits the cursor together. Staged config-reconciliation cursors use the same transaction. Invalid or stale proof, incomplete current work, interruption and write failure retain the prior cursor. Existing current-provider identity and checkpoint guards remain independent and are never cleared to permit preservation. An unchanged activated receipt can support ordinary incremental cycles without another full inventory. Changed family, configuration, file or receipt evidence reactivates the hold while retaining every original snapshot and prior proof. Reactivation does not authorize historical attribution or rewriting. `status`, cycle reports and health JSON expose `unattributed_legacy_assets` and `unattributed_legacy_pending` separately from download failures. Pending current proof makes one-shot exit status, cycle reports and health incomplete, including idle health for unselected libraries. Certified but unattributed history alone does not fail current sync. Certification refusals include the fixed-label `legacy_certification_refusal_v1` diagnostic and a bounded `subreason`; unknown errors use `unclassified`, without logging their private source text. The first counts legacy records still needing attribution; the second counts records awaiting valid current-work preservation proof. Original capture revision and retry history remain stored. Advancing current sync does not claim a complete historical backup; backup safety remains false while attribution is unresolved. A missing selected receipt also emits `legacy_receipt_prerequisite_v1` with one fixed `prerequisite`: child readiness, absent rendition or scoped path receipt, binding mismatch, or missing/invalid verified source. This is a read-only snapshot after refusal, not acceptance evidence. A newly eligible snapshot reports `receipt_state_changed`; failed or unsupported inspection reports `diagnostic_unavailable`. Both retain the original `child_receipt_missing` hold. The lookup cannot grant qualification, retry work, write state or inspect media. It emits no child/library identifiers, paths, hashes or private database errors; support history retains only the enumerated prerequisite. One refusal does not establish how many families lack proof. Primary sparse-identity recovery remains independent. Automatic repair processes at most 500 stale assets per library in one sync cycle. When a clean batch makes progress and work remains, watch and service mode wait at most 60 seconds before the next cycle. A stalled or failed batch uses the configured watch interval so persistent provider failures do not cause rapid retries. A one-shot sync processes one batch. `kei status` reads durable remaining counts, and `sync_report.json` reports the current cycle's refreshed, failed, and remaining counts. Revision storage uses one row per distinct asset. A synthetic one-million-asset database grew by 87.363 MiB, or 17.51 percent. One library with one million stale assets needs 2,000 successful batches, which adds about 33 hours of follow-up waits plus provider and cycle processing time. The watch pre-check includes only libraries with pending capture work or serviceable rewrite work, even when iCloud reports no provider changes. A rewrite marker is serviceable only when a metadata writer is enabled. A capture refresh preserves download status, paths, checksums, and media bytes. It stores corrected catalogue metadata, the current revision, and any configured rewrite marker before the library revision can become active. Lookup or state failures leave the row stale, report the remaining work, and preserve the affected provider checkpoint. A library promotes its active revision only when every live downloaded row is current. Persisted rewrite markers may drain later because they are durable recovery evidence. Metadata failure markers must survive so a later run can retry metadata without downloading the media again. Full enumeration, single-pass incremental, and collecting incremental sync all commit changed catalogue metadata and its configured rewrite marker together, before filtering, album routing, or deciding whether unchanged media needs downloading. A failed commit preserves the provider checkpoint. Queued rewrites drain once per cycle, after every producer has finished, so a single writer owns each file. Read-only runs never drain. Only that drain retires a marker on success, because it writes the file from the same row it then clears. Each bounded rewrite batch loads current album and people rows for its library-scoped asset IDs. A grouping-read failure leaves the affected markers pending. A completed download writes the snapshot it was planned from, so it records a marker on failure but never retires one. The first task payload includes its concrete pass album, even when the cycle's grouping preload is empty. Provider membership changes update the `asset_albums` read model and metadata retry markers in one transaction. The projection uses child identity or the recorded legacy state owner, never a sibling's master reference alone. Individual membership changes use indexed identity lookups; container refreshes enumerate only that container. Schema v22 adds a reverse index for legacy owners without changing their recorded identities. Completed snapshots can remove missing memberships; interrupted snapshots preserve the prior live memberships. External grouping sources remain unchanged. The end-of-cycle writer uses the accumulated memberships for each tracked media path. Retry markers do not authorize writes when metadata outputs are disabled. `asset_metadata_paths` records each successfully finalized media path, its provider rendition checksum, local fingerprints, and metadata retry receipts. Registration and downloaded-state finalization share one transaction. The catalogue still owns sync selection and keeps one path per rendition. Metadata rewrites also visit the additional recorded paths that match the current provider rendition. Each completion checks the selected path and fingerprints, then clears only that path's debt. A failed copy does not block successful copies or lose its retry evidence. Capture-repair receipts remain path-specific. Missing paths retain retry evidence; old provider renditions remain recorded but do not receive current-rendition metadata. Schema migration seeds only the path already recorded in each catalogue row. It does not scan for, infer ownership of, or authorize writes to older copies that were never recorded. Additional paths enter the registry through verified download finalization or explicit import adoption. XMP sidecars record the exact properties that kei writes in the kei namespace. A later rewrite deletes a cleared property only when that marker proves kei owned the prior value. Unmarked standard properties and unrelated third-party namespaces remain unchanged. An existing sidecar must be readable and parseable, and its bytes must still match the writer's initial read at publication. A failed check preserves the sidecar and its durable rewrite marker. When the complete merged packet already matches the existing bytes, a retained regular file snapshot rechecks identity, parent confinement and exact bytes before reporting success. This avoids republishing an unchanged packet. Changed packets use a new temporary path so retained ambiguous bytes cannot block a later retry. Source GPS facts for sidecars are read through file-backed parsers. JPEG APP1, TIFF-based RAW, PNG `eXIf`, and HEIF Exif items use checked seeks and fixed-size TIFF fields. When a HEIF carries several Exif items, `pitm` and `cdsc` select the one describing the primary image. HEIF atom and item-location counts are streamed without allocating from provider-controlled lengths. CloudKit is authoritative for the currently decoded coordinates, altitude, and capture timestamps. The location decoder currently maps only `lat`, `lon`, and `alt` from `locationEnc`, so source EXIF supplies GPS receiver time, speed, speed units, and horizontal positioning error. Native horizontal accuracy is emitted only when both validated native coordinates equal the provider latitude and longitude after decoding to decimal degrees. There is no geographic proximity tolerance: rounding differences can omit accuracy, but nearby or edited locations do not establish that accuracy describes the same fix. Matching coordinates alone are insufficient after GPS embedding: kei may have inserted them beside an older native accuracy value. Sidecar planning also requires the current file to match SHA-256 evidence from a verified download, captured before any kei metadata write. Normal downloads pass that baseline to the sidecar phase, even when embedding is disabled. Download finalization atomically stores it in the path's `asset_metadata_paths.source_checksum`. Queued rewrites use only that source checksum, never `local_checksum` or `download_checksum`. Those older columns also support size validation and can be populated from already-rewritten bytes during checksum recovery. Schema v23 adds the nullable source checksum without backfilling historical rows. Missing evidence stays unknown across embedded writes, failed state updates, restart, and later retries. Adoption and local reconciliation do not establish new source evidence. Existing evidence remains scoped to its path and provider rendition; a different rendition invalidates it. Metadata-only finalization preserves source evidence without changing it. This conservative rule can omit valid accuracy from older downloads or after unrelated embedded edits. It does not delete native metadata or add an output-revision sweep. Missing, invalid, or different coordinates omit accuracy. A supported rewrite removes the obsolete value only if the existing property marker proves kei ownership. Unowned values remain unchanged. This rule applies to new sidecars and already-supported rewrites; automatic catch-up for untouched sidecars is separate work in #799. GPS receiver time and speed keep their existing policy. Source I/O failures still publish current CloudKit metadata, preserve prior kei-owned source GPS fields as unknown, and retain the metadata retry marker. Readable unsupported or malformed metadata permits a CloudKit-only sidecar. Source media is never opened for writing on this path. A drain only rewrites a file whose bytes still match the checksum on its row, because the alternative is embedding into damage and then vouching for it. A file that no longer matches keeps its marker and its recorded checksum, so `kei verify --checksums` and `kei reconcile` continue to report it. When a rewrite does change the media, the new hash is stored before the marker retires, and the pre-rewrite hash is kept as the provider download checksum so reconcile can still tell an intentional rewrite from a truncated file. A retired marker therefore implies the row describes the bytes on disk. Capture timestamp repair records the current provider metadata hash as durable intent. Before publishing changed media, it records the exact prepared output checksum and size. If publication succeeds but state finalization is interrupted, a later explicit repair can recognize that exact output and finish the checksum transition without trusting unrelated bytes. Input bytes matching neither the recorded local checksum nor the prepared receipt remain drifted. Provider metadata updates that would invalidate a prepared receipt remain retryable until the published bytes are finalised in state. Source-deletion transitions follow the same rule so a tombstone cannot hide the only proof of published repair bytes. Capture-instant changes are guarded across every version of the asset, even when the metadata hash is unchanged. Addition-date-only updates preserve the prepared receipt. An upsert replacing the receipt's own rendition with a new provider checksum may invalidate that receipt; a sibling replacement may not. Tasks rejected by the state upsert are not dispatched or marked failed, so the existing receipt and catalogue evidence remain intact and the state-write failure keeps the checkpoint ineligible. Provider-version or non-metadata file replacement invalidates the repair state; byte-identical re-adoption preserves it. When replacement or adoption occurs during the explicit repair run, downloaded-state finalization creates fresh pending repair debt for the installed checksum. A rewrite whose result could not be measured records the checksum as unknown rather than leaving the superseded value in place, because a known-stale hash would make the next pass read kei's own rewrite as damage and refuse it. A row that never recorded a checksum is rewritten by ordinary metadata refresh but claims no provider download hash, so reconcile keeps reporting a file that is short of its provider size. Capture timestamp repair instead leaves that row pending because it cannot prove file provenance. ### State and serialization Asset capture and addition dates use fractional Unix seconds in the existing non-STRICT SQLite INTEGER columns. Readers accept legacy whole seconds and round fractional seconds to milliseconds without scaling the whole timestamp. Invalid stored dates fail decoding. Metadata refresh commits these dates with the metadata and rewrite markers; operational timestamps remain whole seconds. This representation requires no schema migration or historical backfill. Schema, primary-key, sentinel, durable-key, and serialization changes are cross-cutting. Search every reader and writer, migrations, fixtures, reports, status output, and round-trip tests before changing them. ### Secrets and diagnostics Passwords stay behind `SecretString` and password-source boundaries. Logging uses a redacting writer as a backstop, but code must not send Apple IDs, passwords, session cookies, bearer tokens, or unredacted provider identifiers to logs or machine output. Preserve process hardening that limits credential exposure through core dumps. `password set` prompts only in interactive input mode. Headless callers must use a password file or password command. The command resolves the secret and passes it directly to the credential store without printing it. ## Safety contract catalog Stable IDs connect safety rules to production owners and focused tests. `scripts/check-contracts` rejects missing links in that chain. | Contract | Owner | Required behavior | |----------|-------|-------------------| | `FILE_PUBLISH_NO_OVERWRITE` | `src/download/file.rs`, `src/download/pipeline.rs` | Publishing a completed `.part` file never replaces an existing final file unless `--repair-truncated` carries exact durable path and fingerprint authorization, or opt-in `photos.edited_naming="primary"` carries a journaled owned-slot handover with an independent exact preservation receipt. A no-replace collision succeeds only when the verified `.part` and destination bytes are identical. Different or unverifiable bytes retain retry evidence and cannot reach metadata writes or downloaded finalization. | | `TEMP_FILE_DELETE_REQUIRES_DURABLE_OWNERSHIP` | `src/download/orchestration/cleanup.rs`, `src/download/pipeline.rs`, `src/fs_util.rs`, `src/state/db.rs` | Orphan cleanup deletes only an exact stale path claimed in durable state. It retains verified filesystem handles through removal and never follows a directory or file symlink. Normal completion and graceful interruption retire the claim. | | `SYNC_TOKEN_ADVANCE_REQUIRES_CLEAN_CYCLE` | `src/sync_cycle.rs` | The database pre-check token advances only after a successful non-dry-run cycle with a current pass plan. | | `SOURCE_CHECKPOINT_REQUIRES_DURABLE_RECOVERY` | `src/sync_cycle.rs`, `src/download/orchestration/` | A zone checkpoint advances only with complete token evidence and durable recovery for unfinished work. | | `MALFORMED_REQUIRED_ASSET_FIELDS_BLOCK_CHECKPOINT` | `src/icloud/photos/asset.rs`, `src/icloud/photos/album/fetch.rs`, `src/icloud/photos/album/lookup.rs`, `src/download/orchestration/incremental.rs`, `src/sync_cycle.rs` | A live asset with a missing or invalid required identity or capture date blocks its zone checkpoint before filtering or path planning. | | `UNKNOWN_PROVIDER_IDENTITY_REMAINS_PENDING` | `src/download/retry.rs` | Inconclusive provider identity retains the pending row and records verification evidence. | | `POLICY_EXCLUDED_REQUIRES_EXPLICIT_SOURCE_DELETION` | `src/download/retry.rs`, `src/state/db.rs` | Policy-excluded rows become source-deleted only after targeted provider deletion evidence. Present or inconclusive responses retain them outside actionable pending work. | | `METADATA_WRITES_REQUIRE_OPT_IN` | `src/download/metadata_rewrite.rs` | Media and sidecar metadata writes run only for explicitly enabled metadata flags. | | `METADATA_EMBED_REWRITE_REQUIRES_STABLE_INPUT` | `src/download/metadata.rs`, `src/download/file.rs`, `src/download/metadata_rewrite.rs` | Every embedded metadata rewrite prepares a uniquely owned sibling and replaces the media only while both the destination and prepared bytes match their approved fingerprints. Failure preserves concurrent edits and durable retry evidence. | | `HEIF_EMBED_REWRITE_REQUIRES_STABLE_INPUT` | `src/download/heif.rs`, `src/download/metadata.rs`, `src/download/file.rs`, `src/download/metadata_rewrite.rs` | A HEIF-family embedded rewrite accepts tone-map insertion only when `dimg` and primary Exif `cdsc` relationships prove the exact target and no existing XMP owns that tone map, prepares a uniquely owned sibling, and replaces the media only while both the destination and prepared bytes match their approved fingerprints. Failure preserves concurrent edits and durable retry evidence. | | `XMP_SIDECAR_REWRITE_REQUIRES_STABLE_INPUT` | `src/download/metadata.rs`, `src/download/metadata_rewrite.rs` | An existing XMP sidecar is replaced only when it parses and its bytes still match the writer's initial read. Failure preserves the sidecar and durable rewrite marker. | | `XMP_GPS_ACCURACY_REQUIRES_MATCHING_LOCATION` | `src/download/metadata.rs`, `src/download/metadata_rewrite.rs`, `src/state/db.rs` | Native horizontal accuracy requires valid native latitude and longitude equal to the exported provider coordinates, plus a matching verified-download source checksum. Recovered local or download checksums cannot establish native provenance. Readable missing, invalid, or mismatched evidence omits accuracy and clears only obsolete kei-owned values. Source I/O failure preserves unknown fields and durable retry evidence. | | `METADATA_CAPTURE_REVISION_REPAIR_IS_DURABLE` | `src/download/orchestration/maintenance.rs`, `src/state/db.rs` | Revision repair updates catalogue metadata and configured rewrite evidence before promotion, stays library-scoped, and preserves the provider checkpoint on unresolved work. | ## Change-impact traversal Use these search anchors after finding the owner. The related owners are an impact set, not a claim that every run calls each one in sequence. Read direct callers and follow shared values through their consumers. Each row names the flow section that owns its behavior and safety details; this table is not a second call graph. This table covers the highest-traffic changes only. Use the Owners table to find the owning module, then this table to find related consumers. Use the Change-impact checklist to choose verification. | Change | Search anchors and related owners | Flow or invariant | |--------|-----------------------------------|-------------------| | Collection selection | `Selection` in `src/selection.rs`; `collection_libraries`, `pass_scope_for_zone`, and `resolve_passes_for_scope` in `src/commands/service.rs`; sync callers in `src/sync_loop/planning.rs` and import callers in `src/commands/import.rs` | [Sync and provider checkpoints](#sync-and-provider-checkpoints), [Import-existing](#import-existing) | | Provider progress | `check_changes_database` and `store_scoped_db_sync_token` in `src/sync_loop/precheck.rs`; cycle calls in `src/sync_loop/runner.rs`; `source_checkpoint_decision` in `src/sync_cycle.rs`; `commit_checkpoint_transition` in `src/state/db/checkpoints.rs` | [Checkpoint advancement](#checkpoint-advancement), [Sync and provider checkpoints](#sync-and-provider-checkpoints) | | Enumeration proof | `download_photos_with_sync` in `src/download/orchestration/dispatch.rs`; full owner in `src/download/orchestration/full.rs`; incremental owner in `src/download/orchestration/incremental.rs`; `src/icloud/photos/album/enumeration.rs`, `src/icloud/photos/album/completion.rs`, and `src/icloud/photos/album/changes.rs` | [Full and incremental enumeration](#full-and-incremental-enumeration) | | File landing and state | `TaskPlanner` in `src/download/planner.rs`; `src/download/pipeline/producer.rs`, `consumer.rs`, `task.rs`, and `pass.rs`; `download_file_with_mode` in `src/download/file/transfer.rs`; `publish_part_to_final` in `src/download/file/publication.rs`; `finalize_downloaded_with_proof` in `src/download/finalize.rs`; `mark_downloaded` in `src/state/db/assets.rs` | [File landing](#file-landing), [Managed edited-primary layouts](#managed-edited-primary-layouts) | | Durable retry | `src/download/orchestration/recovery.rs`; `build_pending_retry_download_tasks` in `src/download/retry.rs`; targeted lookup in `src/icloud/photos/album/lookup.rs`; `DownloadStateStore` pending-state methods in `src/state/db/contracts.rs` | [Durable pending retry](#durable-pending-retry) | | Paths and import parity | `src/config/templates.rs`; `DownloadConfig::with_pass` in `src/download/orchestration/config.rs`; `src/download/paths.rs`; `expected_paths_for` in `src/download/filter/expected_paths.rs`; callers in `src/commands/import.rs`; local reconciliation in `src/download/orchestration/reconciliation.rs` | [Import-existing](#import-existing), [File landing](#file-landing) | | Metadata refresh and writes | `Config::runtime.refresh_metadata` reads in `src/sync_loop/runner.rs` and `src/sync_cycle.rs`; capture repair in `src/download/orchestration/maintenance.rs`; immediate and queued owners in `src/download/metadata_rewrite/`; format and publication owners in `src/download/metadata/` | [Metadata](#metadata) | | Import adoption | `run_import_existing`, `import_assets`, and `StrictImportVerifier` in `src/commands/import.rs`; `ImportStateStore::import_adopt` and its SQLite implementation in `src/state/db/import.rs` | [Import-existing](#import-existing), [State and serialization](#state-and-serialization) | | Service execution | dispatch in `src/lib.rs`; install, status, uninstall, and platform owners in `src/service/`; `src/service/run.rs` enters `src/sync_loop.rs::run_sync` | [Command dispatch](#command-dispatch), [Container startup](#container-startup) | For schema, durable-key, sentinel, serialization, and shared-type changes, follow [State and serialization](#state-and-serialization). Use [the test guide](../tests/README.md) to choose focused proof; a diagnostic count or a source-only assertion does not replace production-path state-transition proof. ## Change-impact checklist This table chooses verification for a change. It assumes the owner is already known; use [Change-impact traversal](#change-impact-traversal) to find the code to read first. | Change | Check | |--------|-------| | CLI command or flag | `src/cli.rs`, dispatch in `src/lib.rs`, help output, Docker, services, Homebrew, docs, CLI tests | | TOML or runtime config | defaults, setup output, CLI precedence, hashes, persisted examples, docs | | Selection or pass scope | list/sync/import parity, shared libraries, unknown names, unfiled scope, membership snapshots | | Provider checkpoint | full and incremental proof, interruption, config drift, retry durability, scoped DB pre-check | | SQLite schema/query | migration from every supported version, all readers/writers, status/report/manifest output, real SQLite tests | | Provider record parsing | missing/malformed fields, shared zones, identity mapping, metadata capture, fixtures | | File or path behavior | `.part`, checksum, no-overwrite publish, fsync, import compatibility, collision handling | | Metadata writes | opt-in gate, pre-publish mutation, sidecars, retry markers, feature combinations | | Service behavior | Linux, macOS, Windows, container defaults, status, install/uninstall renderers | | Credentials or logging | secret wrappers, source lifetime, redaction, core-dump hardening, diagnostics, error paths | | Machine output | JSON/CSV shape, redaction, reports, health, metrics, downstream compatibility | ## Tests - Unit tests live near their owner module. - Cross-module and binary behavior lives under `tests/`. - Strict changes-page recovery is also exercised through production `run_cycle` with real earlier downloads, malformed later work, reopened SQLite and valid replay in both incremental strategies. The proof preserves historical bytes and retry/identity debt, then checks exactly-once transfers, unchanged download receipts and two quiet cycles. See the strict changes/zone section of the [test guide](../tests/README.md#strict-changeszone-pages). - Live iCloud tests are ignored by default and run single-threaded. - `tests/data/media-manifest.json` owns the bundled media inventory and size budget. `download::orchestration::fixture_tests` exercises these bytes through enumeration, planning, download, publication, and reopened SQLite. Its selection, naming, metadata, and recovery modules replace content-dependent live assertions. `tests/data/live-migration.md` records the mapping. General live entry points share `tests/data/live-selection.toml` and reject an empty eligible selection before the suite. `scripts/fixtures/` owns maintainer-only generation and sanitization and the optimized extracted-source-package check. Tests do not fetch media. - Shell suites cover crash, concurrency, state-machine, and container behavior. - Fuzz targets cover parser and metadata trust boundaries. - `ci.yml` defers draft PR work and keeps the required `CI` marker failing until ready-for-review validation succeeds. Ignored PR metadata has a separate concurrency key and check name; base retargets and full-label transitions revalidate. `scripts/ci/policy.py` owns path and native-service routing. `ci.yml` requires selected native smoke once, or delegates it to qualification. `checks.yml` owns shared quality and always-on safety-contract validation; `qualification.yml` composes heavy checks and tested artifacts. `scripts/ci/artifact.py` binds archive/container evidence to one commit. `local_release.py` validates local live/host-service attestations. Release publication depends on both local evidence and hosted qualification; nightly runs cannot publish. See [CI qualification](ci.md). - `scripts/ci/opencode_review.py` owns the data-only advisory review client. PR review requires a new standalone `/opencode review` comment from a human with API-verified repository write access, or explicit default-branch dispatch. Requests pin the current PR head; per-PR publication serialization and head markers suppress duplicates. `opencode-review.yml` executes trusted default-branch code and separates read-only inference from deterministic COMMENT publication. Issue-open intake pins main and issue content, runs independent product/code reviews, and publishes separate advisory comments with stale-state and dedup checks. Code validation is static inspection only. PR reviews submit the complete diff without local usage caps; model capacity and Go allowance govern inference. Transport/job safeguards and publication validation remain. GitHub 403/429 responses stop the operation without automatic retry. See [review security and setup](opencode-review.md). - `justfile` owns local script and workflow lint commands. Protected CI runs the same shellcheck, shfmt, ruff, and actionlint checks with pinned versions. See [the test guide](../tests/README.md) for the current suites and commands. ## Maintaining this guide Update this file in the same pull request when a change: - Moves an owning decision to another module - Adds a command or cross-cutting state transition - Changes file publication or metadata mutation - Changes provider checkpoint or retry evidence - Changes schema, durable keys, or serialization - Changes the best test for an invariant - Adds, renames, or moves a major section, or changes its link label - Changes a traversal anchor, owner module, or flow link Keep the guide focused on stable ownership and safety. Source code remains the final authority. Do not cite build output, local session files, retained investigation notes, or release history as an ownership claim. Describe the current owner instead. ## Managed edited-primary layouts `download/orchestration/primary_layout.rs` owns opt-in `photos.edited_naming` transitions. The shared selected-rendition renderer assigns current/original-archive roles without changing rendition keys, RAW selection or URL deduplication. The layout owner confirms current provider resources through the Photos adapter, prepares all selected members, preserves displaced media and sidecars as independent exact copies, then conditionally publishes and retires verified obsolete aliases. Default suffix families retain ordinary append-only behavior. Schema 35 adds independently addressable layout bindings, operations, members, path claims and immutable preservation receipts under `state/db/primary_layout.rs`. Scoped native paths, stable child and pass identity, source/config decisions and a checked binding generation pin each operation. Durable phases are planned, prepared, preserved, publishing and committed; cancellation retains its journal and private prepared files. Recovery runs after filesystem replacement recovery and before ordinary source dispatch. A started handover finishes its recorded generation before new source/config work may plan another operation. Config/path reconciliation performs this recovery before reading catalogue paths, including operations at a previous root or template. First-binding migration inventories each scoped per-path rendition receipt and its own recorded placement independently of current selection or the latest catalogue filename; exact local bytes and the complete path-owner census remain required before preservation or retirement. Managed recovery stops at the first unresolved operation; later independent operations in that recovery pass remain pending. Reconciliation retains its checkpoint hold. A separately completed independent family does not establish continuation past a blocked operation in the coordinator. The claims fence ordinary transfer, finalization, adoption and queued metadata. Current-publication retry recovery leaves managed plans with the layout owner and rechecks slot generation and exact local-byte claims in its state transaction, including claims created after the reader snapshot. Metadata for selected renditions is prepared off the visible path; it commits with the new binding and matching provider selection receipts. Ordinary whole-asset metadata refresh resolves ownership across every binding and pending claim for the scoped provider child, including when the caller has no concrete pass key; it does not queue competing writes for managed paths. Explicit metadata refresh uses the preserving owner and records its intent in the operation; pending operations recover before unchanged-binding shortcuts. Retained-source reuse carries the SHA-verified fingerprint through candidate receipt capture and independent copying; a second snapshot must equal the trusted first snapshot. Local path reconciliation uses this same layout owner for managed families. Current/archived media and sidecars must match their pinned fingerprints. Unknown ownership or local changes hold completion and protect existing contents. Real album membership is persisted before managed reconciliation commits matching metadata; an unfiled path label is never recorded as album membership. Original archives retain their prior metadata and retry evidence outside the ordinary writer queue; prepared capture-repair evidence stays visible to source-deletion guards. Archive receipt and whole-asset ownership lookups use scoped library/child indexes. Historical catalogue receipts remain evidence, not permission to write a superseded slot. Files in a Live Photo family are reserved and prepared together. Publication across separate files is recoverable rather than simultaneously atomic to external readers. Filesystems without the required atomic conditional publication/retirement operations leave durable work pending. See [edited primary usage](edited-primary.md) for storage, exclusion and downgrade guidance. ## Support evidence boundary `support-export` dispatches before startup configuration construction, logging, credentials or provider initialization. `src/support/collect.rs` reads bounded local files; `src/state/db/support.rs` owns immutable SQLite inspection without normal `open_owned` or schema migration. Unsupported schemas and nonempty WAL or journal snapshots degrade explicitly. Export output uses exclusive creation. Normal sync, import and service operations publish typed evidence from their existing owners through a bounded optional worker. The worker groups only allowlisted fields, persists account-scoped history with private atomic writes and a single-writer lock, and never feeds decisions back to provider, checkpoint, state or publication policy. Its tracing layer is independent of log verbosity. Stored history is revalidated against the same contract before export. Unknown fields, labels, arbitrary errors and provider identifiers cannot broaden it. The Photos owner projects only fixed owner morphology and JSON types for rejected inventory scopes. Private discovery records identify primary versus other zones and validated primary eligibility. A primary-publication observation follows the actual qualified-primary assignment; selected indexing alone does not imply it. Owner categories are revalidated as strings, so a malformed numeric owner cannot escape through an aggregate-count allowance. Missing older fields remain missing. Full enumeration also emits `support_token_equivalence_v1` from `full/token_diagnostics.rs`. Each self-contained snapshot relates initial and same-cycle recovery token groups for at most 64 selected passes. Tokens are compared only in a bounded invocation-local table (128 values, at most 4096 bytes per value); only assigned ordinals reach the support boundary. The provider owner supplies fixed query and scope-comparison categories. A passive stream wrapper records drainage order without extra polling or tasks. Missing drainage is null; this order is not token-arrival, HTTP-completion or media-publication order. `support/token_equivalence.rs` revalidates nested rows at capture and export, rejects malformed or duplicate rows, and preserves explicit omission markers. Group numbers have meaning only within one snapshot, never across invocations, cycles or accounts. The recovery snapshot repeats the initial evidence so history retention cannot silently sever cross-round relationships. This evidence has no consumer in enumeration, recovery or checkpoint policy. `photos/album/preservation_proof/missing_reference.rs` owns observation-only lookup at the first missing-parent inventory refusal. Four byte-sized typed pre-compaction categories live alongside every latest inventory record and count toward its retained-byte budget, without a separate identity/count sampling cap. Lookup payload shape is separate from scope availability. Scope `matched` means the child scope matched; reference scope is also required and checked for a `live_reference` payload before family classification. `PhotosSession::post_diagnostic` uses a dedicated 64 KiB transport with fixed errors, no request dumps, error-body logging or retries. The caller owns cancellation and a five-second deadline. Its response never enters inventory or proof construction; only fixed categories enter support history and export. Random process-local aliases are bounded and renumbered at collection time. Counts are scoped observations and must not be added as unique asset counts. SQL observations before transaction commit are explicitly identified as attempts. `src/support/mod.rs::classify_support_error` owns diagnostic typed-error projection. It exports fixed classes and numeric status codes from the cause chain, never messages or provider payloads, and does not decide retry policy. The inbox, catalogue, selection and work owners publish fixed storage counters after successful transaction commit and at existing capacity refusal branches. The history worker retains a fixed summary per cycle, not response rows. Budgets reuse existing admission accounting and are labeled last observed, pre-admission logical charges. Migration returns its compaction summary to the schema owner, which publishes it only after the migration savepoint commits; failed migration reports no committed reclaim. Optional charge observations cannot fail migration. Retained reasons and counters contain no source identity, URL or arbitrary error text, and remain subject to queue loss and bounded history retention. Unknown observations stay explicitly unavailable. History and export schema version 1 are separate from report version 3 and state schema 36. Raw reports and databases are never packaged. Interrupted writes preserve the previous history; read errors and limits appear as partial evidence. See [the operator guide](support-export.md) for limits and unavailable evidence. ### Explicit missing-reference inventory diagnostic The opt-in `doctor --live --json --missing-reference-inventory` command branches before normal startup, logging, service hooks and ordinary doctor filesystem checks. `inventory_diagnostic.rs` reads bounded configuration and copied cached auth only. It does not open a catalogue or construct a download/state owner. `photos/diagnostic.rs` adapts the existing PhotosSession interface to privacy-safe streamed requests with shared limits and no redirect/retry/auth fallback. The existing PhotosService qualifies the private PrimarySync scope before the preservation scanner's raw latest-record helper supplies fixed missing-reference observations. The diagnostic performs no family hydration and returns no cursor or CompleteLegacyInventory proof. At most one later lookup is observational. Ordinary preservation retains its candidate, scope, date and completeness gates. See `docs/missing-reference-inventory.md` for bounds and supervision requirements. ### Explicit trusted local bytes for legacy migration `commands/trusted_legacy.rs` owns the offline `reconcile trust-legacy-files plan/apply` interface and account lock. `download/trusted_legacy.rs` owns bounded plans, confined retained file handles and final byte validation. `state/db/trusted_legacy.rs` owns typed `TrustedLegacyFile` acceptances in versioned, content-addressed `metadata` keys. Apply installs prefix-scoped immutability guards in the acceptance transaction; no-acceptance catalogues are unchanged. Their account/provider identity, current-child generation, metadata/path and protected-family snapshots are rebuilt from current state at apply and receipt lookup. Apply commits all entries atomically after final file and cancellation checks. No source checksum is inferred or written. Only legacy current-child coverage consumes this alternate provenance. Its fresh inventory, readiness, ownership, file and checkpoint guards still run. Nonempty library-scoped acceptance rows extend the legacy dependency fence; provider evidence records `verified_source` or `trusted_legacy_file` explicitly. No-acceptance dependency serialization is unchanged. Native source verification, GPS and ordinary adoption retain their verified-source requirements. See [Trusted Legacy Files](trusted-legacy-files.md) for the command boundary.