# cloudsite Current release: **1.0.2**. [Release notes](docs/releases/v1.0.2.md) and [upgrade/rollback contract](docs/releases/v1.0.0.md). The current development phase is complete; the project enters maintenance. New feature development requires a new explicit request. Product: [cloud.netioi.com](https://cloud.netioi.com/). CloudSite provides account login, a file library, previews, browser-supported media playback, direct downloads, revocable single-file shares, cloud-download admission and a role-protected admin console. Admin capabilities include user management, task queries, audited resource presentation edits and operation history. The release includes the completed admin UI cleanup and magnet `&` preparation fix. Schema remains **20**. IndexCore owns physical truth; provider completion alone is not READY. Running/uncertain work has no automatic retry, and storage activation does not prove provider authentication. AI/private UserSpace work is deferred. See the release notes for precise supported behavior and remaining limits. Product and storage accounts remain separate. Existing Go imports, `cmd/panta`, `PANTA_*` settings, cookies, deployment filenames and durable identities retain compatibility names; do not rename them or recreate data volumes for branding. The prior [alpha deployment contract](docs/CLOUDSITE-V1-ALPHA.md) is historical. cloudsite is a provider-neutral resource acquisition and catalog platform. The MVP focuses on proving the resource lifecycle end to end before investing in UI polish: ```text Storage Provider ↓ OpenList ↓ IndexCore ↓ cloudsite Catalog ↑ Acquisition / Agent / Product API ``` ## MVP goals The lifecycle MVP must prove five things, separately from the alpha version label: 1. Existing resources can be observed through **OpenList → IndexCore → cloudsite Catalog**. 2. A user who already knows a path can browse/resolve it directly; **search is optional, not mandatory**. 3. Logical resource identity is separated from physical files through **Asset → Release → Variant → Copy**. 4. A logged-in user can submit a supported source to the **115 cloud-download provider**, after which cloudsite sends a trusted scoped Mutation Hint to IndexCore; IndexCore performs the OpenList-backed scoped verification and cloudsite waits for canonical Query/Journal evidence before the new Copy becomes READY. 5. Provider-specific logic is replaceable. cloudsite domain modules must not depend directly on 115 semantics. ## Architecture rules - OpenList is the upstream storage observation/access layer used by IndexCore, not the product database. - IndexCore is the canonical physical-resource truth, not the product catalog or search engine. - cloudsite acquisition does not directly query OpenList to decide visibility; it sends trusted Mutation Hints to IndexCore and consumes IndexCore Query/Journal truth. - cloudsite Catalog owns Asset / Release / Variant / Copy semantics. - Provider actions go through provider contracts and the Job Engine. - Agent output is advisory/structured input; agents do not own product state. - Cloud download and share require product login. - cloudsite product account is independent from storage-provider accounts. - Git is the project memory. Architecture, decisions, current state and acceptance evidence must live in this repository. ## Project documents - [AI Developer Entry Point](AGENTS.md) — mandatory repository entry for coding/review agents - [AI Architecture Memory](docs/AI-ARCHITECTURE-MEMORY.md) — read first for current invariants / anti-drift rules - [MVP Blueprint](docs/MVP-BLUEPRINT.md) - [Project Context](PROJECT-CONTEXT.md) - [Project State](PROJECT-STATE.md) - [Development & Acceptance Rules](docs/DEVELOPMENT-RULES.md) - [Catalog logical read model](docs/CATALOG-READ-MODEL.md) - [Physical resource view](docs/PHYSICAL-RESOURCE-VIEW.md) - [Copy classification](docs/COPY-CLASSIFICATION.md) - [Acquisition Manifest](docs/ACQUISITION-MANIFEST.md) ## Collaboration model cloudsite MVP uses a deliberately small team model: - **Architect:** owns boundaries, contracts, gate acceptance and scope. - **Developer:** implements one accepted stage at a time, with tests and evidence. No stage starts because “the code seems ready.” It starts only after the previous gate is accepted. ## Gate 0.1 backend skeleton The backend requires Go 1.27.1. The Gate 0.1 skeleton can be verified with: ```text make test make build ``` The binary reads `PANTA_ENV` (default: `development`) and stays alive until it receives an interrupt or termination signal. No provider or external service is required to build or start it. Package dependencies follow the modular-monolith boundary: ```text cmd/panta → internal/app → modules and cloudsite-owned ports ↑ future adapters/integrations ``` - `catalog`, `resourceview`, `acquisition`, and `jobs` own product behavior. - `providers/contracts` owns provider-neutral ports; `providers/registry` binds implementations only at the composition boundary. - `integrations/indexcore` is the production physical-truth integration boundary. Any `integrations/openlist` port is access-layer/future capability only unless a separate Gate explicitly authorizes it; acquisition observation remains owned by IndexCore and must not grow a second cloudsite → OpenList verification lane. - In-memory doubles live only in tests. There is no real provider, persistence, network integration, search, agent, authentication, HTTP API, or UI in Gate 0.1. ## Gate 0.2 product database cloudsite product state uses a PostgreSQL database that is configured independently through `PANTA_DATABASE_URL`. It is never shared with or inferred from an IndexCore database setting. Schema changes are ordered SQL files in `migrations/`. Applied versions, names, and checksums are recorded in `schema_migrations`; missing, changed, or unknown future migrations make schema status incompatible. Apply or inspect them with: ```text make db-migrate make db-status ``` `internal/catalog` owns the Asset → Release → Variant → Copy types and the persistence port. The pgx/v5 implementation is isolated in `internal/store/postgres`. A Copy may have a null `variant_id` while awaiting logical classification. Gate 1.3 later hardens `storage_binding_id` with its now-valid foreign key to `storage_bindings`. PostgreSQL integration tests require a dedicated disposable database named `panta_test` or ending in `_test`; the tests recreate its `public` schema: ```text PANTA_TEST_DATABASE_URL='postgres://user:password@127.0.0.1:5432/panta_test?sslmode=disable' make test-integration ``` ## Gate 0.3 durable jobs `internal/jobs` owns the provider-neutral Job model, frozen states, commands, errors, and Repository port. `internal/store/postgres` implements that port with atomic PostgreSQL claims and lease-checked transitions. The dependency remains: ```text future application/worker → internal/jobs ← internal/store/postgres ``` The generic engine stores opaque JSON payloads and never imports provider, IndexCore, OpenList, auth, or agent types. Future job types extend through product-level `job_type` and payload contracts outside the state kernel; they do not add provider-specific columns to `jobs`. Claimable work is limited to `QUEUED` and due `RETRY_WAIT` jobs. A running job with an expired lease moves to `RECOVERY_REQUIRED`; it is not automatically retried because an external side effect may already have happened. Only the active, unexpired lease owner can renew, succeed, fail, or schedule a retry. Gate 0.3 deliberately contains no worker loop, provider execution, backoff policy, job DAG, or event/audit table. ## Gate 0.4 closeout verification Gate 0 closes through test-only composition, not by introducing a real provider. `internal/providers/testprovider` is a deterministic in-memory implementation of the frozen Storage, Downloader, and Share ports. It is used by the reusable `internal/providers/contracttest` suite and is not registered by the production application. The Gate 0 dependency and test boundaries are: ```text core domains → cloudsite-owned ports ← production adapters (future gates) ↖ testprovider + contracttest (tests only) catalog/jobs ← internal/store/postgres → cloudsite PostgreSQL ``` - Provider conformance covers descriptor consistency, storage stat/access, download start/status/cancel, share create/inspect/access/revoke, opaque references, and the provider-neutral unsupported-operation error. - The Gate 0 acceptance test composes the registry, a known-path IndexCore port double, the current append-only migration chain, Catalog persistence, and a generic durable Job without coupling those modules or adding an acquisition workflow. - `internal/architecture` automatically guards Catalog, Jobs, and the IndexCore port from forbidden database, provider, 115, and cross-module imports. - `.github/workflows/gate0.yml` runs unit/contract tests, race detection, vet, both command builds, PostgreSQL 16 integration tests, and migration apply/status checks on pull requests and pushes to `main`. No 115 adapter, real OpenList/IndexCore client, worker loop, Search, Agent, authentication, API, or UI is part of Gate 0.4. ## Gate 1.1 storage root mapping `internal/storage` owns the provider-neutral StorageConnection and StorageBinding model and its persistence port. A binding records the product mapping from one connection and normalized OpenList mount path to one canonical external IndexCore root ID: ```text StorageConnection ── StorageBinding ── OpenList mount path ├──────────── IndexCore root_id (external identifier) └──────────── provider_scope (optional opaque target) ``` `internal/store/postgres` implements the port. Migration `0003_storage_bindings.sql` enforces the connection foreign key, globally unique IndexCore root mapping, unique connection/mount mapping, the small ACTIVE/DISABLED status sets, and canonical mount-path storage. Mount paths use absolute slash form. `/` is the root; repeated and trailing slashes normalize away for non-root mounts. Empty, relative, backslash, `.`, and `..` component forms are rejected. `credential_ref` is nullable opaque metadata only; credentials and secrets are not stored in these rows. Gate 3.3 migration `0007_storage_binding_provider_scope.sql` adds the third, independent coordinate. A nullable scope keeps observation-only bindings valid; acquisition requires an explicit scope and never derives it from the OpenList mount or IndexCore root. Provider scope is bounded opaque text and is preserved exactly without path normalization. Gate 3.4 migrations `0008_acquisition_provider_tasks.sql` and `0009_provider_task_side_effect_fence.sql` add the durable provider-task linkage: one Manifest and one Job each map to at most one opaque provider task reference, stored exactly as the provider returned it. The durable row is written *before* the external call, in two phases (`START_RESERVED`, then `REFERENCE_KNOWN`) under a per-Manifest advisory lock, so at most one execution can ever call `StartDownload` and a lost reference fails closed instead of starting a second provider task. An existing reference is never overwritten or re-created. The provider stays authoritative for its own task status, the fence carries no provider lifecycle state, and provider success is not Manifest `READY`. Gate 3.5 scopes provider execution to a StorageConnection instead of a provider identity alone: `internal/providers/session` binds one downloader port to an exact `ProviderID` + `ConnectionID` + opaque `CredentialRef`, and the execution step resolves that session before any provider call. One `ProviderID` may back several connections with different credentials, and a missing or mismatched binding fails closed. `contracts.SecretResolver` freezes the opaque secret lookup port with no real backend yet; secret contents never enter the Manifest, Job payload, or provider-task tables. Gate 3.6 adds the first concrete provider: `internal/providers/115` implements `contracts.DownloaderProvider` on the pinned `github.com/SheltonZhu/115driver` `v1.3.5`, mapping cloudsite's `Source.Value` to the exact 115 offline URI and `Target.Scope` to the 115 destination directory ID. `Target.Path` is cloudsite's observation path only and is never translated into provider scope. The adapter is provider-isolated: it imports only `internal/providers/contracts`, the pinned upstream driver package, and the standard library, so no 115 type reaches the acquisition domain. Gate 3.7 commits a fenced provider-stage outcome as one transaction over the Manifest and its linked Job, so the two can never diverge. Provider success moves the Manifest to `AWAITING_VISIBILITY` and returns the **same** Job to `RETRY_WAIT` for the later visibility stage — it is deliberately not Job `SUCCEEDED` and not Manifest `READY`. A committed outcome replays idempotently with `Changed=false` and no timestamp rewrite, and a conflicting proposal fails closed. Because one Job now runs the whole acquisition workflow, migration `0010_job_claim_generation.sql` splits the stale-worker fence from the failure budget: `claim_attempts` is an unbounded claim generation that every claim increments and that fences lease mutations, while `attempt_count` (`jobs.Job.FailureCount`) is consumed only by `RetryAt` and bounded by `max_attempts`. Claiming therefore never refuses a due `RETRY_WAIT` row because the budget is spent, so provider polling, the visibility stage, and later stages can each be scheduled repeatedly without starving the failure retry budget. The module stores no physical inventory and has no IndexCore/OpenList database or network dependency. Real clients, Journal cursors/projectors, Copy updates, visibility checks, Mutation Hints, and provider-specific behavior remain outside Gate 1.1. ## Gate 1.2 IndexCore read adapter `internal/integrations/indexcore` owns a typed, server-side HTTP adapter for the accepted IndexCore Q4/Q5/Q8/Q9 read surface. Configure its base URL with `PANTA_INDEXCORE_BASE_URL`; this is an HTTP endpoint only and never an IndexCore database setting. The consumer-owned ports preserve the upstream semantics instead of flattening them: Q4 browses one parent hierarchy level and round-trips opaque cursors, Q5 returns every canonical-path match plus its ambiguity flag, Q8 sends `after_seq` exactly as supplied and keeps per-root event order, and Q9 exposes a narrow read-only root status. Stable remote errors, transport failures, malformed responses, and unexpected statuses remain separately classifiable with `errors.Is`. The adapter uses only the standard HTTP client with a bounded timeout. Contract tests run against `httptest.Server`; no IndexCore process or database is needed. Journal cursor persistence, Catalog projection, OpenList integration, retries, Mutation Hints, scoped refresh, and root mutations remain outside Gate 1.2. ## Gate 1.3 one-shot Journal projection `internal/projector` owns one bounded `ProjectOnce` application operation. It loads an ACTIVE StorageBinding, reads that binding's durable cursor, calls Q8 with the exact cursor, validates the ordered page, and maps resource events to unresolved Copy availability. Root lifecycle events advance only the cursor; Journal payload remains opaque and does not drive identity or state. `internal/store/postgres` owns the atomic persistence boundary: ```text lock and compare expected cursor -> upsert ordered Copy mutations without changing CopyID/VariantID/binding -> advance cursor to the last event_seq actually seen -> commit one PostgreSQL transaction ``` Migration `0004_indexcore_projection.sql` adds the per-binding cursor and the `copies.storage_binding_id` foreign key. Missing cursor state reads as zero. Concurrent stale cursors and attempted binding rebinding fail closed. The Q8 network call happens before the transaction begins. Candidate Copy UUIDs are created by the application-side projector and supplied to persistence; an upsert uses a candidate only for a genuine insert and preserves the existing CopyID on conflict. Gate 1.3 deliberately adds no polling loop, scheduler, OpenList client, Mutation Hint, classification, provider execution, API, or UI. ## AI / Architect preflight Before planning or reviewing any Gate, reconstruct project truth from Git in this order: ```text docs/AI-ARCHITECTURE-MEMORY.md ↓ PROJECT-STATE.md ↓ docs/DECISIONS.md ↓ PROJECT-CONTEXT.md / docs/MVP-BLUEPRINT.md ↓ current Issue / PR ↓ exact code + tests + CI ``` Chat context is not authoritative when it conflicts with current Git. Newer explicit accepted decisions supersede older design prose, Issues, PR descriptions, and historical assumptions.