# Architecture

> Full-system architecture for the Orla Numis PWA. Covers all three services, their interactions, and the design rules that keep the system maintainable.

---

## Table of Contents

- [System Overview](#system-overview)
- [Container Topology](#container-topology)
- [Go API — Layered Architecture](#go-api--layered-architecture)
  - [Layer Diagram](#layer-diagram)
  - [Architecture Rules](#architecture-rules)
  - [Package Map](#package-map)
  - [Dependency Injection Wiring](#dependency-injection-wiring)
  - [Route Groups and Authorization](#route-groups-and-authorization)
  - [Shared GORM Scopes](#shared-gorm-scopes)
  - [Architecture Tests](#architecture-tests)
- [Vue 3 Frontend](#vue-3-frontend)
  - [Application Structure](#application-structure)
  - [Routing](#routing)
  - [State Management (Pinia)](#state-management-pinia)
  - [API Client](#api-client)
  - [Composables](#composables)
  - [PWA Configuration](#pwa-configuration)
- [Python Agent Service](#python-agent-service)
  - [Service Overview](#service-overview)
  - [Supervisor and Team Routing](#supervisor-and-team-routing)
  - [Team Pipelines](#team-pipelines)
  - [LLM Provider Abstraction](#llm-provider-abstraction)
  - [SSE Streaming](#sse-streaming)
- [Coin Copilot Durable Harness](#coin-copilot-durable-harness)
- [Data Flow Diagrams](#data-flow-diagrams)
  - [Standard API Request](#standard-api-request)
  - [Agent Chat (SSE Streaming)](#agent-chat-sse-streaming)
  - [Authentication Flow](#authentication-flow)
  - [Wishlist Availability Check](#wishlist-availability-check)
- [Database Schema](#database-schema)
  - [Core Domain Models](#core-domain-models)
  - [Authentication Models](#authentication-models)
  - [AI and Agent Models](#ai-and-agent-models)
  - [System Models](#system-models)
- [Authentication and Authorization](#authentication-and-authorization)
- [AI/Agent Integration Pattern](#aiagent-integration-pattern)
- [Background Schedulers](#background-schedulers)
- [Build and Deployment](#build-and-deployment)
- [Configuration Reference](#configuration-reference)
- [Key Design Decisions](#key-design-decisions)

---

## System Overview

Orla Numis is a full-stack PWA for managing a personal coin collection, with deep support for ancient and historical coins. The system is composed of three services:

| Service | Tech Stack | Port | Path |
|---------|-----------|------|------|
| **Go API** | Go 1.26.6, Gin, GORM, SQLite | 8080 | `src/api/` |
| **Vue Frontend** | Vue 3, TypeScript, Pinia, Vite, PWA | (bundled) | `src/web/` |
| **Python Agent** | Python 3.12, FastAPI, LangGraph, LangChain | 8081 | `src/agent/` |

The Vue SPA is bundled into the Go binary's `wwwroot/` directory at build time. The Go API serves both the SPA and the REST API. The Python agent service runs as a separate container and is accessed only by the Go API — never directly by the frontend.

## Container Topology

```
┌──────────────────────────────────────────────────────────────────────┐
│                          Docker Host                                 │
│                                                                      │
│  ┌─────────────────────────────┐    ┌──────────────────────────────┐ │
│  │     Go API Container        │    │   Python Agent Container     │ │
│  │                             │    │                              │ │
│  │  ┌───────────────────────┐  │    │  ┌────────────────────────┐ │ │
│  │  │  Vue SPA (wwwroot/)   │  │    │  │  FastAPI + LangGraph   │ │ │
│  │  └───────────────────────┘  │    │  │                        │ │ │
│  │  ┌───────────────────────┐  │    │  │  12 Team Pipelines     │ │ │
│  │  │  Gin HTTP Server      │  │    │  │  Supervisor Router     │ │ │
│  │  │  REST API + SSE Proxy │──────────│  SSE Streaming         │ │ │
│  │  └───────────────────────┘  │    │  └────────────────────────┘ │ │
│  │  ┌───────────────────────┐  │    │           │                  │ │
│  │  │  SQLite Database      │  │    │     ┌─────┴──────┐          │ │
│  │  └───────────────────────┘  │    │     v            v          │ │
│  │         :8080               │    │  Claude API   Ollama+SearXNG│ │
│  └─────────────────────────────┘    └──────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────┘
         ▲
         │ HTTPS
    ┌────┴────┐
    │ Browser │
    │  (PWA)  │
    └─────────┘
```

---

## Go API — Layered Architecture

### Layer Diagram

```
Request
  │
  ▼
[Handler]     ── Parses HTTP, calls service/repo, returns response
  │
  ▼
[Service]     ── Business logic, orchestration, domain rules (HTTP-agnostic)
  │
  ▼
[Repository]  ── GORM queries, transactions (one repo per domain)
  │
  ▼
[Database]    ── *gorm.DB (initialized once in main.go, injected everywhere via deps.go)
```

### Architecture Rules

**1. Only the composition root imports the `database` package.**
That means `main.go` (which calls `database.Connect`) and `deps.go` (which
constructs every repository from `database.DB`). Every other package receives
`*gorm.DB` or a repository/service through its constructor. Enforced by
`architecture_test.go`, which forbids the import in `handlers/`, `services/`,
`middleware/`, and `repository/`.

```go
// GOOD (deps.go, the composition root)
coinRepo := repository.NewCoinRepository(database.DB)

// BAD (anywhere else)
database.DB.Find(&coins)  // violates DI rule
```

**2. Handlers are thin.**
Parse request, call service/repo, return HTTP response. No business logic, no raw SQL.

**3. Services contain business logic.**
Orchestrate repos, enforce domain rules, perform data transformations. HTTP-agnostic — no `gin.Context`, no `net/http`.

```go
func (s *CoinService) UpdateCoin(existing *models.Coin, updates *models.Coin, ...) error {
    s.repo.Update(existing, updates)
    if valueChanged { s.repo.RecordValueHistory(...) }
}
```

**4. Repositories own all database access.**
Every GORM query lives in a repository. Repositories take `*gorm.DB` via constructor.

```go
func (r *CoinRepository) FindByID(id, userID uint) (*models.Coin, error) {
    var coin models.Coin
    err := r.db.Scopes(OwnedByID(id, userID)).Preload("Images").First(&coin).Error
    return &coin, err
}
```

**5. Multi-step writes use transactions.**
`r.db.Transaction()` wraps 2+ writes that must atomically succeed or fail:
- `CoinRepository.Delete` (coin + images + journals + value history)
- `AdminRepository.DeleteUserCascade` (all user data)
- `AuthRepository.RotateRefreshToken` (revoke old + create new)
- `ImageRepository.SetPrimaryAndCreate` (clear primary + insert)

**6. Errors stay internal.**
Never expose raw `err.Error()` from GORM or Ollama to clients. Log server-side, return generic messages. Validation errors from `ShouldBindJSON` are acceptable.

**7. Go API contains zero LLM logic.**
All AI inference is proxied to the Python agent via `services/agent_proxy.go`.

### Package Map

| Package | Responsibility | Imports Allowed |
|---------|---------------|-----------------|
| `main.go` | Entry point: config, DB connect, router assembly | Everything |
| `deps.go` | Composition root: builds `appDeps` (repos, services, schedulers) | Everything |
| `routes_*.go` | Route registration per auth tier, reading from `appDeps` | Everything |
| `handlers/` | HTTP layer (thin) | `services/`, `repository/`, `models/` |
| `services/` | Business logic, orchestration | `repository/`, `models/` |
| `repository/` | Database access, GORM queries | `models/`, `gorm.io/gorm` |
| `models/` | Data structures (GORM models) | Standard library only |
| `middleware/` | Auth, rate limiting | `models/`, `gorm.io/gorm` |
| `config/` | Environment configuration | Standard library only |
| `database/` | DB initialization, AutoMigrate | `gorm.io/gorm`, SQLite driver |

### Dependency Injection Wiring

Wiring is split between `main.go` (the sequence) and `deps.go` (the
construction). `appDeps` is a plain hand-wired struct rather than a DI
framework, because the construction order encodes real initialisation
dependencies that are better left explicit.

```
main.go:
config.Load()
    → database.Connect(cfg.DBPath)
    → buildDeps(cfg) ─────────────┐   deps.go
    → newHTTPRouter(cfg)          │     → construct repositories (each takes *gorm.DB)
    → registerPublicRoutes        │     → construct services (take repos + config)
    → registerProtectedRoutes     │     → construct schedulers + worker pools
    → registerAdminRoutes         │     → returns *appDeps + a CancelFunc for shutdown
    → registerExternalToolRoutes  │
    → registerInternalToolRoutes ─┘
    → runServer(...)  → schedulers start, http.Server serves,
                        SIGTERM triggers graceful shutdown
```

Each `register*Routes` function lives in its own `routes_*.go` file and takes
`(group, d *appDeps)`. Adding an endpoint means adding a line there and, if it
needs a new service, a field to `appDeps`.

> Route files are gated: `route_openapi_drift_test.go` parses every file listed
> in `routeSourceFiles` and fails if a registered route is missing from the
> OpenAPI spec. A new `routes_*.go` must be added to that list —
> `TestRouteSourceFilesCoverAllRouteRegistrations` fails if it is not.

### Route Groups and Authorization

Three route groups with distinct auth levels:

| Group | Prefix | Auth | Example Routes |
|-------|--------|------|----------------|
| `api` (public) | `/api` | None (rate-limited) | `/auth/login`, `/auth/register`, `/auth/refresh` |
| `protected` | `/api` | JWT | `/coins`, `/coins/bulk`, `/sets`, `/coin-catalogue`, `/agent/chat`, `/deep-identification/*`, `/stats`, `/wishlist/availability-runs`, `/wishlist/availability-runs/:id` |
| `admin` | `/api/admin` | JWT + admin role | `/users`, `/settings`, `/logs`, `/availability-runs` (legacy), `/availability-runs/:id` (legacy), `/availability/run`, `/availability-cycles`, `/availability-cycles/:id`, `/valuation-runs`, `/deep-identification/observability`, `/backups`, `/test-anthropic`, `/test-searxng` |

### Shared GORM Scopes

Reusable query scopes in `repository/scopes.go`:

| Scope | Purpose |
|-------|---------|
| `OwnedBy(userID)` | Filter by `user_id` |
| `OwnedByID(id, userID)` | Filter by both |
| `ActiveCollection(userID)` | Non-wishlist coins |
| `OfKind(kind)` | Coins or banknotes |

### Architecture Tests

Run with:
```bash
cd src/api
go test -v -run "TestNoDirectDatabase|TestHandlersDoNotUseRawSQL" .
```

Enforced rules:
- **No direct database imports** — `handlers/`, `services/`, `middleware/`, `repository/` must not import the `database` package
- **No raw SQL in handlers** — flags `SELECT`, `INSERT INTO`, `UPDATE`, `DELETE FROM`, `.Raw(`, `.Exec(` in handler files

---

## Vue 3 Frontend

### Application Structure

```
src/web/src/
├── main.ts                  # App bootstrap (Vue + Pinia + Router)
├── App.vue                  # Top-level layout (nav, sidebar, agent drawer)
├── api/
│   └── client.ts            # Axios instance, JWT interceptor, 401 refresh queue
├── router/
│   └── index.ts             # Route definitions with lazy loading
├── stores/
│   ├── auth.ts              # Auth state, login/register/logout
│   └── coins.ts             # Collection state, CRUD, stats
├── composables/
│   ├── useCoinSearchChat.ts # Agent chat streaming state
│   ├── useCollectionFilters.ts
│   ├── useBulkSelect.ts
│   ├── useDialog.ts
│   ├── useImageProcessor.ts
│   ├── usePullToRefresh.ts
│   ├── usePwa.ts
│   ├── useSettingsProfile.ts
│   └── useAdminConfig.ts
├── pages/                   # Full-page views (21 pages)
├── components/              # Reusable UI components
│   ├── chat/                # Agent chat sub-components
│   ├── admin/               # Admin panel components
│   ├── coin/                # Coin detail sub-components
│   ├── settings/            # Settings sub-components
│   └── ...                  # Shared components (forms, cards, modals)
├── types/
│   └── index.ts             # TypeScript interfaces
└── assets/                  # Static assets
```

### Routing

Routes are lazy-loaded. Auth-protected routes use `meta: { requiresAuth: true }` enforced by a `beforeEach` guard.

**Key pages:** Collection (gallery), CollectionOverview, Banknotes, CoinDetail, AddCoin, EditCoin, Wishlist, Sets, SetDetail, CommemorativeCatalogue, Stats and its subviews, CoinLookup, DeepAnalysis, Settings, Admin.

**Public pages:** Login, Register.

### State Management (Pinia)

| Store | Key State | Key Actions |
|-------|-----------|-------------|
| `auth` | `token`, `user`, `refreshToken` | `doLogin`, `doRegister`, `logout` |
| `coins` | `coins[]`, `currentCoin`, `stats`, `valueHistory`, `searchQuery`, `selectedCategory` | `fetchCoins`, `fetchCoin`, `createCoin`, `updateCoin`, `deleteCoin`, `fetchStats` |

Auth state is persisted in `localStorage` and rehydrated on app load.

### API Client

`src/web/src/api/client.ts` provides:

- **Axios instance** with `baseURL = ${VITE_API_BASE_URL}/api`
- **Request interceptor** — attaches `Authorization: Bearer <token>` header
- **Response interceptor** — handles 401 with single-flight refresh:
  1. First 401 triggers refresh token exchange
  2. Concurrent requests queue behind the refresh promise
  3. On success, retries all queued requests with new token
  4. On failure, redirects to login
- **Agent chat** uses native `fetch` + manual SSE parsing (not Axios) for streaming

### Composables

| Composable | Purpose |
|-----------|---------|
| `useCoinSearchChat` | Agent chat streaming, message history, SSE parsing |
| `useCollectionFilters` | Search, sort, face value, country, set and other filters, grouping |
| `useBulkSelect` | Multi-select with bulk actions (add to set, delete) |
| `useDialog` | Confirmation dialog state |
| `useImageProcessor` | Client-side image resize/compress before upload |
| `usePullToRefresh` | Touch-based pull-to-refresh for mobile |
| `usePwa` | PWA install prompt and update detection |
| `useSettingsProfile` | Profile edit form state |
| `useAdminConfig` | Admin settings read/write |

### PWA Configuration

Configured via `vite-plugin-pwa` in `vite.config.ts`:

- **Register type:** `autoUpdate` (no user prompt for updates)
- **Manifest:** Standalone display, dark theme, coin branding icons (192/512)
- **Workbox caching strategies:**
  - API mutations (`POST`/`PUT`/`DELETE`): `NetworkOnly`
  - API reads (`GET`): `NetworkFirst`
  - Uploaded images (`/uploads/`): `CacheFirst`
- **Dev proxy:** `/api` and `/uploads` proxy to Go backend during development

---

## Python Agent Service

### Service Overview

The agent is a **stateless** FastAPI service. It owns no database. All configuration (API keys, model names, system prompts, user context, portfolio data) is passed per-request from the Go API.

**Endpoints:**

| Method | Path | Response | Purpose |
|--------|------|----------|---------|
| `GET` | `/health` | JSON | Health check |
| `POST` | `/api/search/coins` | SSE stream | Coin search / shows / general chat |
| `POST` | `/api/analyze` | JSON | Vision-based coin analysis |
| `POST` | `/api/portfolio/review` | SSE stream | Portfolio review and valuation |
| `POST` | `/api/check-availability` | JSON | Wishlist URL availability check |
| `POST` | `/api/deep-identify/stream` | SSE stream | Deep-identification image analysis, routing, provider fan-out, and synthesis |
| `POST` | `/api/copilot/capability` | JSON | Internal provider/model tool-calling preflight |
| `POST` | `/api/copilot/execute` | SSE stream | Stateless Coin Copilot execution |
| `GET` | `/logs` | JSON | Log ring buffer |
| `PUT` | `/log-level` | JSON | Dynamic log level |

### Supervisor and Team Routing

The supervisor (`app/supervisor.py`) uses an LLM router to classify user intent into one of 11 teams (plus a general fallback):

```
User message
    │
    ▼
[LLM Router] ── classifies intent using last 4 messages
    │
    ├── coin_search    → Coin Search Team
    ├── coin_shows     → Coin Shows Team (may ask for ZIP/city)
    ├── analysis       → Coin Analysis Team
    ├── grading        → Coin Grading Team
    ├── portfolio      → Portfolio Review Team
    ├── gap_analysis   → Gap Analysis Team
    ├── photo_guide    → Photo Guide Team
    ├── price_trends   → Price Trends Team
    ├── similar_lots   → Similar Lots Team
    ├── auction_search → Auction Search Team
    └── general        → General assistant (describes capabilities)
```

The supervisor enforces a max iteration count to prevent infinite loops.

### Team Pipelines

Each team is a LangGraph `StateGraph` with verification stages:

| # | Team | Pipeline | Purpose |
|---|------|----------|---------|
| 1 | Coin Search | Search → Fetch dealer pages → Format | Find available coins for sale |
| 2 | Coin Shows | Search → Verify dates are future → Format | Upcoming numismatic events |
| 3 | Coin Analysis | Vision model analysis → Format | AI image analysis |
| 4 | Portfolio Review | Read holdings → Valuate → Analyze | Collection recommendations |
| 5 | Auction Search | Search auction sources → Fetch → Format | Auction lot discovery |
| 6 | Availability Check | Check URLs → Analyze results → Verdict | Verify listings still for sale |
| 7 | Coin Grading | Analyze photos → Grade → Format | Grade estimation with confidence |
| 8 | Gap Analysis | Read portfolio → Analyze gaps → Suggest | Collection completeness |
| 9 | Photo Guide | Analyze photos → Evaluate → Tips | Photography improvement |
| 10 | Price Trends | Search auctions → Analyze trends → Format | Market price analysis |
| 11 | Similar Lots | Search → Rank → Format | Find similar auction lots |
| 12 | Deep Identification | Vision hypothesis → deterministic router → provider fact-checking → synthesize | Resumable, cited coin identification |

**Pipeline design rules:**
- Search agents pass only tool-returned data downstream — never invented details
- Verification agents confirm every URL is live and every date is in the future
- All worker outputs conform to defined Pydantic schemas — no free-form text
- Deep Identification is **vision-first**: the single
  vision LLM call already made on every job produces a typed `CoinHypothesis`
  (per-field confidence, no citation) instead of a discarded prose string.
  `route()` is then a **deterministic pure function** of the catalog, bounds,
  quick evidence, and hypothesis — not an LLM call — so provider selection is
  reproducible for identical inputs and the pipeline makes one fewer LLM call
  per job than its Feature 344 predecessor. Providers **fact-check** the
  hypothesis rather than starting from nothing: query terms are composed
  deterministically from the highest-confidence hypothesis fields (never
  LLM-authored), and the evaluator treats `image` as a first-class claim
  source alongside provider claims when detecting disagreements
- Deep Identification receives provider evidence only through authenticated Go
  internal tools; Python remains stateless and does not persist jobs or artifacts
- Full provider claims stay on the internal stream; replayable owner-facing
  events persist bounded status, confidence, count, and link-out data

### LLM Provider Abstraction

`app/llm/provider.py` provides a factory for LLM instances:

| Function | Returns | Use Case |
|----------|---------|----------|
| `get_chat_model()` | `ChatAnthropic` or `ChatOllama` | Nodes that don't need web search |
| `get_search_model()` | Chat model with `web_search` tool bound | Nodes that need web search |
| `create_search_agent()` | ReAct agent with SearXNG tool | Ollama web search (mirrors Anthropic pattern) |

**Provider selection:**
- **Anthropic:** Claude models. Web search uses Claude's built-in `web_search_20250305` tool via `bind_tools`.
- **Ollama:** Self-hosted models. Web search uses `langchain.agents.create_agent` with a SearXNG HTTP tool.

### SSE Streaming

`app/streaming.py` converts LangGraph execution events into SSE:

```
LangGraph events → stream_graph_events() → SSE text/event-stream
```

Event types emitted: `status`, `text`, `done`, `error`.

Deep Identification uses a separate typed event contract. Go persists
monotonically sequenced, replayable job events and can reconnect clients from
the last observed sequence. Terminal job states are completed, partial, failed,
or cancelled. Proposal writes remain explicitly confirm-gated.

Final message extraction priority:
1. Last node's message content
2. Last AI content from any node
3. Full accumulated text

JSON suggestion blocks (coin suggestions, show listings) are extracted from final text when present.

---

## Coin Copilot Durable Harness

Feature 359 adds a bounded, read-only harness without changing the legacy chat
protocol. The existing drawer first calls
`GET /api/agent/copilot/capability`. Coin Copilot is selected only when the
default-off `CoinCopilotEnabled` flag is on and the configured model passes a
fail-closed tool-calling preflight. Anthropic must successfully bind the fixed
schemas; Ollama `/api/show` must explicitly advertise `tools`. Missing,
ambiguous, malformed, or timed-out checks select the unchanged
`POST /api/agent/chat` path.

```text
Vue drawer
  ├─ legacy mode ──> POST /api/agent/chat
  └─ copilot mode ─> Go durable run API/SSE
                       │
                       ├─ SQLite threads, runs, checkpoints, events
                       ├─ worker cancellation, retention, idempotency
                       └─ POST Python /api/copilot/execute
                              └─ execution-scoped callbacks to Go reads
```

Go is the sole owner of durable state, authorization, collection data,
retention, event sequence allocation, and cancellation races. Python remains
stateless and DB-free. Each execution or resume receives a new read-only
credential bound to owner, run, execution, tool allowlist, and expiry. Its TTL
is the smaller of the remaining execution budget plus 30 seconds or 180
seconds. The configurable execution timeout is therefore capped at 150 seconds
(default 120).

Runs enforce 8 reasoning iterations, 12 tool calls, one concurrent tool, the
wall-clock timeout, one active run per owner, and 32 KiB persisted tool results
by default. Dollar-cost enforcement is deferred until a trustworthy,
provider/model-specific pricing source exists. Reliable provider-reported
input/output token counts remain observable; iteration, tool-call, wall-clock,
sequential-concurrency, and payload limits remain enforced.

Only sanitized, typed public events are persisted and replayed. Checkpoints
contain public messages, concise plan state, bounded validated tool facts,
counters, and pending clarification—not chain-of-thought, scratchpads, hidden
messages, raw prompts, credentials, provider-native traces, or unrestricted raw
tool payloads. A browser disconnect does not cancel the run: reconnect uses
`since` or `Last-Event-ID`, replays persisted events in order, and then follows
live events. Clarification pauses durably; resume uses the same run, a new
execution id, a new credential, and an idempotency key. Cancellation is
cooperative in Python and authoritative in Go.

Operational retention is 7 days after terminal state for public events and 30
days for checkpoints/bounded tool results. Final answers, usage totals, and
thread messages remain until owner deletion. To roll back, set
`CoinCopilotEnabled=false`; reject new starts/resumes, retain read/cancel access
for existing state, and continue using legacy chat. The janitor and stale-run
recovery remain active until outstanding state settles.

---

## Data Flow Diagrams

### Standard API Request

```
Browser                Go API                    SQLite
  │                      │                         │
  │── GET /api/coins ───▶│                         │
  │                      │── AuthRequired() ──────▶│ validate JWT
  │                      │◀── userId, role ────────│
  │                      │                         │
  │                      │── coinRepo.List() ─────▶│
  │                      │◀── []Coin ──────────────│
  │                      │                         │
  │◀── 200 JSON ────────│                         │
```

### Agent Chat (SSE Streaming)

```
Browser              Go API               Python Agent          LLM Provider
  │                    │                       │                      │
  │── POST /agent/chat▶│                       │                      │
  │                    │── read settings ─────▶│ (DB)                 │
  │                    │── POST /api/search ──▶│                      │
  │                    │   (enriched request)  │                      │
  │                    │                       │── classify intent ──▶│
  │                    │                       │◀── team_id ─────────│
  │                    │                       │                      │
  │                    │                       │── run team graph ───▶│
  │                    │                       │◀── stream events ───│
  │                    │                       │                      │
  │◀── SSE: status ───│◀── SSE: status ──────│                      │
  │◀── SSE: text ─────│◀── SSE: text ────────│                      │
  │◀── SSE: text ─────│◀── SSE: text ────────│                      │
  │◀── SSE: done ─────│◀── SSE: done ────────│                      │
```

Go's `agent_proxy.go` sets `text/event-stream`, `no-cache`, `X-Accel-Buffering: no` and proxies each line with flush-on-boundary.

### Authentication Flow

```
Browser                Go API                  SQLite
  │                      │                       │
  │── POST /auth/login ─▶│                       │
  │   {user, pass}       │── find user ─────────▶│
  │                      │◀── user record ───────│
  │                      │── bcrypt.Compare() ───│
  │                      │── sign JWT (15min) ───│
  │                      │── generate RT (30d) ──│
  │                      │── store RT hash ─────▶│
  │◀── {token, refresh} ─│                       │
  │                      │                       │
  │   ... 15 min later ...                       │
  │                      │                       │
  │── POST /auth/refresh▶│                       │
  │   {refreshToken}     │── hash + lookup ─────▶│
  │                      │── rotate (revoke+new)▶│
  │◀── {token, refresh} ─│  (transaction)        │
```

### Wishlist Availability Check

```
Scheduler / User        Go API               Python Agent
  │                       │                       │
  │── trigger check ─────▶│                       │
  │                       │── load wishlist coins │
  │                       │── for each URL:       │
  │                       │   ├── HTTP GET URL    │
  │                       │   ├── keyword scan    │
  │                       │   └── if ambiguous: ──▶│ POST /check-availability
  │                       │       agent verdict ◀──│
  │                       │── update listing_status│
  │                       │── save run + results   │
```

---

## Database Schema

SQLite via GORM. All tables are auto-migrated from Go model structs in `database/database.go`.

### Core Domain Models

| Model | Table | Key Fields | Relations |
|-------|-------|-----------|-----------|
| `Coin` | `coins` | Kind (coin/banknote), Name, Category, Denomination, Ruler, Era, Year, Country, Mint, Material, Weight, Grade, Quantity, PurchasePrice, CurrentValue, IsWishlist, ListingStatus, UserID | → User, → []CoinImage, → []CoinReference, → []CoinSet |
| `CoinImage` | `coin_images` | CoinID, FilePath, ImageType (obverse/reverse/detail), IsPrimary | → Coin |
| `CoinSet` | `coin_sets` | UserID, Name, Color, SetType (`standard`/`goal`/`smart`/`agentic`/`checklist`), SmartCriteria, TargetCompletionDate, AgenticStatus | → User, → []CoinSetMembership, → []CoinSetTarget |
| `CoinSetMembership` | `coin_set_memberships` | SetID, CoinID, Notes | Join table for manual set membership |
| `CoinSetTarget` | `coin_set_targets` | SetID, Label, Year, MintMark, Denomination, Country, Material, MatchRules, SortOrder | Target slots for compatible completion workflows |
| `CoinSetValuationSnapshot` | `coin_set_valuation_snapshots` | SetID, UserID, SnapshotDate, TotalValue, TotalInvested, CoinCount, CompletionPercentage | Time-series trend data |
| `CatalogueOverride` | `catalogue_overrides` | UserID, CoinKey, SameAs | A catalogue coin hidden, or joined to another |
| `CoinJournal` | `coin_journals` | CoinID, UserID, Entry, CreatedAt | |
| `CoinValueHistory` | `coin_value_histories` | CoinID, UserID, Value, Confidence, RecordedAt | |

### Authentication Models

| Model | Table | Key Fields |
|-------|-------|-----------|
| `User` | `users` | Username, Email, PasswordHash, Role (user/admin), AvatarPath, Bio, LockedUntil |
| `RefreshToken` | `refresh_tokens` | UserID, TokenHash (SHA-256), ExpiresAt, RevokedAt |
| `SecurityEvent` | `security_events` | Type, UserID, Username, ClientIP, UserAgent, Message |

### AI and Agent Models

| Model | Table | Key Fields |
|-------|-------|-----------|
| `AgentConversation` | `agent_conversations` | UserID, Title, Messages (JSON), CreatedAt |
| `AvailabilityCycle` | `availability_cycles` | TriggerType, TriggerUserID (nullable, admin-only), Status (queued/running/completed/failed/partial_failure), aggregated child counts, StartedAt/CompletedAt |
| `AvailabilityRun` | `availability_runs` | UserID (>0 for per-owner child runs), CycleID (nullable — set on child runs, null on legacy rows), TriggerType (owner/scheduled/admin), CoinsChecked, Available, Unavailable, DurationMs |
| `AvailabilityResult` | `availability_results` | RunID, CoinID, URL, Status, Reason, AgentUsed |
| `ValuationRun` | `valuation_runs` | UserID, TriggerType, Status, TotalCoins, CoinsUpdated, DurationMs |
| `ValuationResult` | `valuation_results` | RunID, CoinID, PreviousValue, EstimatedValue, Confidence, Reasoning |
| `DeepIdentificationJob` | `deep_identification_jobs` | UserID, Source, CoinID, Status, timeout/retention metadata, report and proposal |
| `DeepIdentificationEvent` | `deep_identification_events` | JobID, sequence, typed public payload |
| `DeepIdentificationProviderRun` | `deep_identification_provider_runs` | JobID, provider, status, timing and bounded outcome metadata |
| `DeepIdentificationArtifact` | `deep_identification_artifacts` | JobID, role, ephemeral path and cleanup metadata |
| `CoinCopilotThread` | `coin_copilot_threads` | UserID, title, last run, timestamps |
| `CoinCopilotRun` | `coin_copilot_runs` | UserID, thread, state, execution identity, snapshotted limits, token usage, terminal result |
| `CoinCopilotCheckpoint` | `coin_copilot_checkpoints` | Immutable public continuation state and bounded tool facts |
| `CoinCopilotEvent` | `coin_copilot_events` | Owner-scoped monotonic replay event |
| `CoinCopilotResumeRequest` | `coin_copilot_resume_requests` | Hashed idempotency key, request fingerprint, accepted checkpoint/execution |

### System Models

| Model | Table | Key Fields |
|-------|-------|-----------|
| `AppSetting` | `app_settings` | Key, Value — key-value store for runtime configuration |
| `NumistaUsage` | `numista_usages` | Month, Requests — Numista API requests counted by month |

---

## Authentication and Authorization

Username and password, with JWT tokens:

- Access token: HS256, 15-minute TTL, claims: `userId`, `username`, `role`
- Refresh token: random `rt_...` string, SHA-256 hashed in DB, 30-day TTL
- Rotation on refresh: old token revoked + new token issued in a single transaction

**Auth middleware** (`middleware/auth.go`): checks the `Authorization: Bearer <token>` header and sets `userId` and `userRole` in the Gin context. There is no API-key or query-parameter alternative.

**Lockout:** 5 failed passwords in 15 minutes lock the account for 15 minutes; 20 failures from one IP in 15 minutes block the IP until they age out.

**Rate limiting:** Public auth endpoints are rate-limited to 10 requests/minute per IP.

**Admin guard:** `handlers.AdminRequired()` middleware checks `userRole == "admin"`.

---

## AI/Agent Integration Pattern

The Go API acts as a **thin proxy** between the frontend and the Python agent service. This separation ensures:

- The Go API owns all data access and authentication
- The Python agent is stateless and horizontally scalable
- LLM provider credentials live in the Go DB, passed per-request

**Proxy flow (`services/agent_proxy.go`):**

1. Handler reads settings from DB (API keys, model names, system prompts)
2. Handler enriches the request with user context and portfolio data
3. `AgentProxy` POSTs to the Python service with full context
4. For streaming: Go sets SSE headers and proxies each line with flush-on-boundary
5. For structured responses: Go reads the full JSON response and returns it

**Two HTTP clients in the proxy:**
- `streamClient` — no timeout (SSE can run indefinitely)
- `requestClient` — 5-minute timeout (analysis, availability checks)

**Key proxy methods:**
- `StreamChat()` → `POST /api/search/coins` (SSE)
- `CollectPortfolioReview()` → `POST /api/portfolio/review` (collects full SSE, returns final message)
- `AnalyzeCoin()` → `POST /api/analyze` (JSON)
- `CheckAvailability()` → `POST /api/check-availability` (JSON)

---

## Background Schedulers

Goroutine-based schedulers run in the Go API: wishlist availability, wishlist search alerts, collection valuation, set value snapshots (`SetSnapshotEnabled`, off by default) and the automatic full backup (`BackupEnabled`, `BackupStartTime` 03:00, `BackupKeep` 7; `services/backup_scheduler.go`). The first two and valuation are described below.

### Wishlist Availability Scheduler

- **Initial delay:** 30 seconds after startup
- **Config:** `WishlistCheckEnabled`, `WishlistCheckStartTime` (HH:MM, default `02:00`), `WishlistCheckInterval` (minutes, default `120`)
- **Behavior:** Enqueues an `AvailabilityCycle` (parent) and fans out one child `AvailabilityRun` per wishlist owner (including owners with zero URLs) via `RunAdminCycle()`; a scheduled run processes synchronously so the next-run anchor still derives from `GetLastScheduledRun().CompletedAt`
- **Observability:** Duplicate cycles (queued/running within 5 minutes) return the existing cycle and `ErrAvailabilityRunInProgress`; stale queued/running cycles and orphaned child runs are recovered as `failed` on worker startup; terminal child/cycle rows are pruned to the most recent 20 per owner/overall after each cycle finalizes
- **Source:** `services/availability_scheduler.go`, `services/availability_service.go`, `repository/availability_cycle_repository.go`

### Collection Valuation Scheduler

- **Initial delay:** 60 seconds after startup
- **Config:** `ValuationCheckEnabled`, `ValuationCheckStartTime` (HH:MM, default `03:00`), `ValuationCheckInterval` (days, default `7`)
- **Behavior:** Gets users with owned coins, runs `ValuateCollectionForUser()` for each
- **Source:** `services/valuation_scheduler.go`

They compute the next run from a daily anchor time plus an interval.

### Shared schedule timing

All schedulers use `services/schedule_timing.go`:

- **Time zone:** start times are read in `ScheduleTimezone` (IANA name, blank = server time, which is UTC in the default Docker image). Unknown zones are rejected on save.
- **Slot grid:** a job runs at its start time, then every interval, restarting at the start time each day. Intervals of a day or longer run at the start time every whole number of days. Wall-clock time is kept across daylight-saving changes.
- **Restarts:** jobs with run history resume at the first slot after the last completed scheduled run. A slot missed while the server was down runs once at startup; a slot skipped while running (for example after moving the start time earlier) is not run late.
- **Setting changes:** each scheduler re-reads its settings at least once a minute, so saved changes apply without a restart.

---

## Build and Deployment

### Docker Multi-Stage Build (Go + Vue)

`Dockerfile` (root):

```
Stage 1: node:24-alpine@sha256:...
  → npm install + npm run build (Vue SPA)
  → Output: dist/

Stage 2: golang:1.26.6-alpine@sha256:...
  → go build -o ancient-coins-api
  → Output: binary

Stage 3: alpine:3.21@sha256:...
  → Copy binary + Vue dist → /app/wwwroot
  → Create /app/uploads, /app/data
  → EXPOSE 8080
```

### Docker Build (Python Agent)

`src/agent/Dockerfile`:

```
Stage 1: python:3.12-slim@sha256:... (builder)
  → pip install from pyproject.toml

Stage 2: python:3.12-slim@sha256:...
  → Copy site-packages + app/
  → CMD: uvicorn app.main:app --host 0.0.0.0 --port 8081
  → EXPOSE 8081
```

### Build Commands

```bash
# Local development
task build          # Build API + web
task up             # API + web dev servers
task up-all         # API + web + agent dev servers

# Individual builds
cd src/api && go build ./...
cd src/web && npm run build
cd src/agent && pip install -e ".[dev]"

# Tests
cd src/api && go test -v ./...
cd src/agent && pytest tests/ -v

# Linting
cd src/api && go vet ./...
cd src/agent && ruff check app/ tests/
cd src/web && npx vue-tsc --noEmit
```

---

## Configuration Reference

### Go API (`src/api/config/`)

| Env Var | Default | Purpose |
|---------|---------|---------|
| `DB_PATH` | `./ancientcoins.db` | SQLite database file |
| `JWT_SECRET` | (required in prod, min 32 chars) | JWT signing key |
| `PORT` | `8080` | HTTP listen port |
| `UPLOAD_DIR` | `./uploads` | Coin image storage |
| `BACKUP_DIR` | — | Folder of the full backups |
| `CORS_ORIGINS` | (derived) | Allowed CORS origins |
| `AGENT_SERVICE_URL` | `http://localhost:8081` | Python agent base URL |
| `AGENT_INTERNAL_SERVICE_TOKEN` | — | Shared API → agent credential for hardened agent endpoints |

### Python Agent (`src/agent/app/config.py`)

| Env Var | Default | Purpose |
|---------|---------|---------|
| `AGENT_DEBUG` | `false` | Debug mode |
| `AGENT_LOG_LEVEL` | `INFO` | Log level |
| `AGENT_INTERNAL_SERVICE_TOKEN` | — | Shared API → agent credential; required for every endpoint except `/health` and `/ready` |
| `AGENT_SEARXNG_URL` | — | SearXNG instance URL (Ollama search) |
| `AGENT_MAX_SEARCH_RESULTS` | — | Limit search results |
| `AGENT_VERIFICATION_TIMEOUT` | — | URL verification timeout |
| `AGENT_MAX_SUPERVISOR_ITERATIONS` | — | Loop prevention |

### Runtime Settings (DB `app_settings` table)

AI provider, model names, system prompts, scheduler configs, and feature toggles are stored as key-value pairs and managed via the Admin UI. Constants and defaults live in `services/settings_service.go`.

Deep Analysis defaults to disabled. Its settings cover enablement, worker count,
per-user concurrency, queue depth, hard timeout, event/result retention,
provider count and Numista call budget. Numista is the only automated provider;
RPC remains disabled and non-automatable.

The admin-only Deep Analysis operations view combines durable aggregate job and
provider-run status/latency with process-local SSE, cleanup, and janitor
counters. It never returns notes, query terms, claims, reports, tokens, or
per-job owner content.

Coin Copilot defaults to disabled. Its settings cover worker count, per-owner
active-run limit, queue depth, reasoning iterations, tool calls, execution
timeout (15–150 seconds; default 120), persisted tool-result bytes, event
retention, checkpoint retention, and resume window. There is no dollar-cost
setting. Provider-reported input/output token totals remain observable.

---

## Key Design Decisions

| Decision | Rationale |
|----------|-----------|
| **SQLite over Postgres** | Single-user/small-team app. No connection pooling overhead. Portable file-based DB. |
| **Separate Python agent service** | Go has limited LLM/ML ecosystem. Python owns LangGraph, LangChain, and vision model integrations. Keeps Go API focused on CRUD and auth. |
| **Stateless agent service** | Horizontal scaling without shared state. All context passed per-request. Go owns the source of truth (DB). |
| **SSE over WebSockets** | Unidirectional streaming (server → client) is sufficient. SSE is simpler — no connection upgrade, works through most proxies, auto-reconnects. |
| **Constructor injection (no globals)** | Testable. Explicit dependencies. Enforced by architecture tests. |
| **Refresh token rotation** | Single-use refresh tokens. Revoke-on-rotate prevents replay attacks. |
| **Keyword heuristics before agent escalation** | Availability checks try cheap HTTP + keyword scan first. Only ambiguous cases go to the LLM agent. Saves cost and latency. |
| **PWA over native app** | One codebase for desktop and mobile. Offline-capable with Workbox. Installable from browser. |
| **No emojis in UI** | Numismatic audience. Professional tone. Enforced by convention. |
| **Settings as key-value pairs** | Flexible runtime config without schema migrations. Admin UI for non-technical users. |
| **Provider-specific authority boundaries** | Numista provides catalog evidence; NGC is link-out only; RPC automation is paused. |
| **Confirm-gated deep writes** | Provider evidence can propose fields but cannot mutate a coin or draft without explicit owner review and apply. |