---
title: "Architecture"
description: "Full technical architecture documentation."
order: 16
section: "Infrastructure"
---

Signet Architecture
===================

Technical reference for the Signet [[daemon]] and supporting packages.
This document covers the full system — from package boundaries through
database schema — with enough detail to reason about correctness,
performance, and failure modes.

This is a substrate document. It explains how Signet stores, structures,
and routes memory today. It should not be read as a claim that the graph
or retrieval stack is the product by itself. Those layers exist to
support bounded, high-quality context selection.

---

Package Overview
----------------

Signet is organized as a bun workspace monorepo under `signetai/`.
The repository is grouped by developer intent:

- `platform/` contains the core runtime substrate.
- `surfaces/` contains human-facing applications.
- `integrations/` contains external harness integrations, grouped by tool.
- `libs/` contains reusable developer libraries.
- `plugins/` contains Signet-native plugins.
- `dist/` contains assembled shipping artifacts.
- `runtimes/` contains separate runtime ecosystems.
- `web/` contains the marketing site and Cloudflare workers.
- `memorybench/` contains the benchmark harness, providers, reports, and UI.

See [Repository Map](./REPO_MAP.md) for the full path map. The important
ownership boundary is unchanged: `@signet/core` owns types and data; the
daemon owns runtime behavior; connectors own install-time harness integration;
runtime plugins and adapters live beside the tool they extend.

`@signet/core` lives in `platform/core/`. It is the shared foundation and
defines TypeScript
interfaces, the SQLite wrapper, hybrid search, manifest parsing, and
constants. Every other package imports from core; core imports from
nothing internal.

`@signet/daemon` lives in `platform/daemon/`. It is the background service
and runs the Hono HTTP
server on port 3850, the pipeline workers, the file watcher, and the
retention and maintenance workers. It targets bun directly, which
gives it access to `bun:sqlite` and JSX for the dashboard.

`@signet/cli` lives in `surfaces/cli/`. It is the user-facing tool and
handles setup, config
editing, daemon lifecycle, secrets, and skills. It targets Node for
broad compatibility, but runs fine under bun.

`signet-dashboard` lives in `surfaces/dashboard/`. It is built to static
assets and served by the daemon.

`@signet/connector-base` lives in `libs/connector-base/` and provides the abstract `BaseConnector` class
that all platform connectors extend. It re-exports shared utilities
(block injection, skill symlinking) so connector implementations stay
thin.

`@signet/connector-claude-code`, `@signet/connector-opencode`,
`@signet/connector-openclaw`, and the other `@signet/connector-*`
packages live under `integrations/<tool>/connector/`. They are concrete
install-time platform adapters. Each implements `install`, `uninstall`,
`isInstalled`, and `getConfigPath`.

`@signet/sdk` lives in `libs/sdk/`. It is the embedding library for third-party apps that want
to call the daemon [[api|HTTP API]] without shelling out to the [[cli|CLI]].

`@signet/opencode-plugin` lives in `integrations/opencode/plugin/`. It
is the runtime plugin for OpenCode and
provides memory tools and session lifecycle hooks that call the daemon
API during OpenCode sessions.

`@signetai/signet-memory-openclaw` lives in
`integrations/openclaw/memory-adapter/`. It is the runtime adapter for OpenClaw.
It bridges OpenClaw's plugin interface to the daemon API for memory
operations during conversations.

`@signet/desktop` lives in `surfaces/desktop/`. It is the Electron desktop
application and provides the native desktop UI, menu bar tray, bundled daemon
runtime, quick actions, and notifications. `@signet/tray` lives in
`surfaces/tray/` and is a shared tray/menu state utility package only.

Predictor-related tables and diagnostics remain in the schema for future
scoring work, but there is no shipped `platform/predictor/` sidecar package in
the current tree.

`@signet/native` lives in `platform/native/` and provides Rust/NAPI bindings for SIMD vector operations
(cosine similarity, normalization) used by the daemon for fast
embedding math. Targets bun/node.

`signetai` lives in `dist/signetai/`. It is the npm/Bun wrapper that downloads,
verifies, and exposes the same compiled Signet binary used by the curl
installer.

`@signet/web` lives in `web/marketing/`. It is the Astro marketing site
deployed to Cloudflare Pages. Web workers live under `web/workers/<worker>/`.

---

End-to-End Data Flow
--------------------

The path from a conversation event to a searchable memory is:

```
Harness hook fires (session-start / user-prompt / session-end)
    → connector calls daemon HTTP API
    → /api/hooks/remember enqueues memory_jobs row (type: extract)
    → inline entity linker runs synchronously at write time
      (no LLM — links candidate proper nouns to existing same-agent entities)
    → extraction worker leases job, calls LLM for facts + entities
    → decision worker evaluates each fact against existing memories
    → controlled writes: new memories inserted via txIngestEnvelope
    → hints worker generates hypothetical future queries per memory,
      indexes them in FTS5 for prospective matching
    → graph persistence: entities and relations written in a
      separate transaction
    → embeddings prefetched outside write lock, stored atomically
    → memory_history records every proposal (shadow or applied)
    → /api/memory/recall runs traversal-primary search:
      graph traversal produces the base candidate pool,
      flat FTS5/vector search fills remaining slots,
      structured evidence shaping balances lexical, semantic,
      prospective hint, and traversal evidence,
      currentness shaping dampens grouped claim-key superseded structured facts,
      predictor score slots remain nullable until a scorer is wired in
```

The database is the source of truth. The daemon's file watcher is
responsible for syncing agent config changes to harness-specific
files (CLAUDE.md, AGENTS.md). That flow is independent from the
memory pipeline:

```
User edits $SIGNET_WORKSPACE/AGENTS.md
    → chokidar detects change
    → 2s debounced sync: regenerate ~/.claude/CLAUDE.md etc.
    → 5s debounced git commit: auto-commit with timestamp
```

---

Pipeline V2
-----------

The memory pipeline lives at `platform/daemon/src/pipeline/`. It
processes memories asynchronously through a job queue, using an LLM
for extraction and a second LLM pass for decision-making. The key
architectural constraint is the transaction boundary rule: no LLM
calls inside write locks. Embeddings and LLM completions are always
fetched before `withWriteTx` is entered.

**Extraction stage** (`extraction.ts`): given raw memory content,
prompts the LLM to return a JSON object with `facts` and `entities`
arrays. Facts carry a type (`fact`, `preference`, `decision`,
`procedural`, `semantic`) and a confidence score. Entities carry
source, relationship, target, and confidence. Output is strictly
validated — malformed fields produce warnings but do not fail the
job. Input is capped at 12,000 characters; facts are capped at 20
per call, entities at 50. The extractor strips `<think>` blocks from
chain-of-thought models (qwen3, etc.) before parsing.

**Decision stage** (`decision.ts`): for each extracted fact, a
focused hybrid search retrieves up to 5 candidate memories. If no
candidates exist, the system proposes an `add` action immediately.
Otherwise it sends a second LLM prompt with the fact and candidates
and parses an action (`add`, `update`, `delete`, `none`) with a
target memory ID and confidence. `update` and `delete` decisions must
reference a valid candidate ID or they are rejected. Decision results
are called "shadow decisions" because they are always proposals first.

**Controlled writes** (`worker.ts`, `applyPhaseCWrites`): when
`enabled && !shadowMode && !mutationsFrozen`, the worker enters
controlled-write mode. For each `add` proposal, the worker checks
confidence against `minFactConfidenceForWrite`, normalizes and hashes
the content, checks for an existing memory with the same hash, and
inserts via `txIngestEnvelope`. `update` and `delete` proposals are
blocked unless `autonomous.allowUpdateDelete` is true. When enabled,
updates go through `txModifyMemory`, deletes go through `txForgetMemory`,
and the previous target state is archived to the cold tier first. Pinned
memories are not deleted without force. Contradiction detection can still
block high-risk update proposals and record them for review.

**Inline entity linking** (`inline-entity-linker.ts`): runs
synchronously at write time inside `withWriteTx`, before any async
pipeline work. It extracts candidate proper nouns from memory
content and links the memory to entities that already exist for the same
`agent_id` by writing `memory_entity_mentions` rows. It does not create
entities, aspects, attributes, or dependency edges from raw text. Structured
remember payloads, explicit user/agent actions, and reviewed repair passes
own semantic graph authorship. The async pipeline still runs later for
extraction, decisions, and optional graph persistence.

**Hints worker** (`prospective-index.ts`): generates hypothetical
future queries ("hints") for each memory at write time. For each new
memory, it prompts the LLM for diverse questions a user might ask
when the fact would be helpful. Hints are indexed in `memories_fts`
so search can match memories by anticipated cue — bridging the
semantic gap between stored facts and natural-language queries.
Gated on `hints.enabled` in pipeline config.

**Graph persistence** happens in a separate transaction after fact
writes complete. A failure here is non-fatal — it logs a warning and
does not revert the extracted memories.

**Lossless transcripts**: Signet stores the cleaned conversation transcript as
JSONL under `$SIGNET_WORKSPACE/memory/{harness}/transcripts/transcript.jsonl`
and keeps `session_transcripts` (migration 040) as a compatibility/indexing
surface alongside extracted memories. Tool calls, tool outputs, and thinking
traces are kept out of this memory surface so retrieval and summarization
operate on the human/agent exchange. Raw auditable traces may still be written
to daemon logs outside the memory lineage. The recall endpoint's `expand: true`
flag joins transcript content back into search results via `source_id`.

**Shadow mode**: when `shadowMode = true`, all proposals are logged
to `memory_history` under the `pipeline-shadow` actor but no
memories are written. This lets operators observe what the pipeline
would do before enabling writes.

**Configuration flags**:

| Flag | Effect |
|------|--------|
| `enabled` | Master pipeline switch |
| `shadowMode` | Extract and propose, never write |
| `mutationsFrozen` | Reads only; pipeline stays quiet |
| `graph.enabled` | Enable graph reads, traversal, and recall boosting |
| `graph.extractionWritesEnabled` | Let background extraction persist graph entity triples |
| `autonomous.enabled` | Allow scheduled maintenance and repair |
| `autonomous.frozen` | Hard stop on autonomous maintenance actions |
| `hints.enabled` | Run prospective hint generation at write time |
| `autonomous.maintenanceMode` | `observe` or `execute` for maintenance worker |

---

Job Queue
---------

The job queue is backed by the `memory_jobs` table. This makes it
durable — jobs survive daemon restarts. The queue supports two job
types: `extract` (memory pipeline) and `document_ingest` (document
worker). Both types use the same lease/complete/fail mechanics.

A job's lifecycle is: `pending` → `leased` → `completed` or
`failed` → (on max retries) `dead`.

**Enqueue**: callers insert a row with `status = 'pending'`, `attempts
= 0`, and a `max_attempts` (default 3). Duplicate jobs for the same
target (same memory_id + job_type with pending/leased status) are
silently dropped.

**Lease**: the worker calls `leaseJob` inside `withWriteTx`. It
selects the oldest pending job with `attempts < max_attempts`, then
updates `status = 'leased'`, increments `attempts`, and records
`leased_at`. This is atomic — no two workers can lease the same job.

**Failure and retry**: on error, the worker calls `failJob`. If
`attempts < max_attempts`, the job goes back to `pending`. On the
final attempt it transitions to `dead` (dead-letter state).

**Backoff**: the worker uses exponential backoff on consecutive
failures. The delay is `min(BASE_DELAY * 2^n, MAX_DELAY)` plus up
to 500ms of jitter. The base delay is 1 second; the cap is 30 seconds.

**Stale lease reaper**: a separate `setInterval` (every 60 seconds)
calls `reapStaleLeases`, which resets `leased` jobs whose `leased_at`
is older than `leaseTimeoutMs` back to `pending`. This handles the
case where a worker crashes mid-job without completing or failing it.

**Dead-letter**: jobs with `status = 'dead'` stay in the table until
the retention worker purges them (default: 30 days after `failed_at`).
The repair action `requeueDeadJobs` can reset them to `pending` with
`attempts = 0` to force a retry.

---

Knowledge Graph
---------------

The knowledge graph stores entities and relations extracted from
memories. It is an augmentation layer — search still works without it,
and graph persistence errors never revert fact extraction.

**Tables**: `entities` stores named entities with a `canonical_name`
(lowercased, for lookups), a `mentions` count, and an optional
embedding. `relations` stores typed edges between entity pairs with
a `strength`, a `mentions` count (incremented on each re-extraction),
and a `confidence`. `memory_entity_mentions` is a junction table
linking memories to the entities they mention, with optional
`mention_text` and `confidence` provenance fields.

**Graph extraction**: when the extractor returns entity triples
(source, relationship, target), `txPersistEntities` is called inside
its own `withWriteTx`. Entities are upserted by canonical name;
relations are upserted by the (source, target, type) triplet with
mention counts incremented. Mention links are inserted into
`memory_entity_mentions`.

**Traversal-primary search** (`memory-search.ts`,
`graph-traversal.ts`): when `traversal.primary` is enabled (the
default when both `graph.enabled` and `traversal.enabled` are true),
graph traversal is the primary candidate-building path. It resolves
focal entities from query tokens, traverses the knowledge graph through
aspects, attributes, and dependency hops, and produces a scored
candidate pool blended with cosine similarity (70% cosine, 30%
structural importance). Flat FTS5/vector search fills remaining
slots — at least 40% of the result budget is reserved for flat
candidates so hub entities cannot exclude keyword/vector matches
entirely. After merging, structured evidence shaping keeps lexical,
semantic, prospective hint, and traversal evidence as separate channels.
Traversal-only candidates are capped below directly anchored evidence,
while exact prospective hints can rescue memories whose stored text uses
a specific instance rather than the user's query class. When traversal
is disabled or the graph has no matching entities, the system falls back
to the legacy path: flat BM25 + vector search with optional graph boost
(`getGraphBoostIds`). This improves the quality of the pool the rest of
the system ranks; it is not, by itself, the whole Signet thesis.

**Post-fusion dampening** (`dampening.ts`): three corrections run
after fusion scoring but before the final sort/return. (1) *Gravity*
penalizes high-cosine results that share zero query-term overlap
with the actual content (0.5x). (2) *Hub* penalizes results whose
linked entities are all in the top-10% by degree (P90 threshold,
0.7x). (3) *Resolution* boosts constraints, decisions, and
date-anchored memories (1.2x). All three stages are independently
toggleable via `DampeningConfig`.

**Recall surface parity**: explicit recall entry points should route through
the same daemon recall implementation whenever possible. Current daemon HTTP
recall, search aliases, hook recall, and MCP memory search call `hybridRecall`,
so they receive the same structured evidence shaping behavior. Prompt-submit is
intentionally not an explicit recall surface: it listens for known ontology
entities or active aliases, uses that entity match as the search scope, and
injects compact current-view attributes only when scoped attribute relevance
clears the configured confidence gate. Any future recall surface, including CLI
shortcuts, SDK helpers, desktop UI search,
connector-specific recall, or daemon-rs parity work, must either call the
daemon recall API or implement the same evidence-channel contract. Do not add a
separate recall path that bypasses lexical, semantic, prospective hint, and
traversal evidence shaping.

**Graph boost fallback** (`graph-search.ts`): `getGraphBoostIds`
is the legacy graph-augmented search path, used when traversal is
disabled. It tokenizes the query, resolves matching entities by
`canonical_name LIKE ?` (ordered by `mentions DESC`, limit 20),
then expands one hop through `relations` in both directions (limit
50 neighbors). Finally it collects all `memory_id` values from
`memory_entity_mentions` for the expanded entity set (limit 200).
The result is a set of IDs whose scores are boosted. Any error
returns an empty set — the graph never degrades core search.

**Entity communities** (`community-detection.ts`): the Louvain
algorithm clusters entities into functional neighborhoods based on
`entity_dependencies` edge weights. Results are persisted to the
`entity_communities` table and `entities.community_id` is updated.
Community structure provides quality signals (fragmented, moderate,
strong) and enables community-scoped retrieval.

**Retention and orphaning**: when memories are tombstoned past their
retention window, the retention worker purges `memory_entity_mentions`
rows for those memories, decrements `entities.mentions`, and removes
entities whose mention count reaches zero (orphan collection).

---

Auth Middleware
--------------

The daemon supports three deployment modes, controlled by `authMode`
in the config.

**local** (default): no authentication required. All requests are
accepted and `auth` is set to `{ authenticated: false, claims: null }`.
Rate limiting is also skipped in local mode.

**team**: a Bearer token is required on every request. Tokens are
HMAC-SHA256 signed using a 32-byte secret loaded from disk. The token
format is `base64url(payload).base64url(hmac)`. Payload is a JSON
object with `sub`, `scope`, `role`, `iat`, and `exp` fields. Expired
or malformed tokens return 401.

**hybrid**: localhost requests (identified by the `Host` header)
bypass the token requirement and get implicit full access. Remote
requests require a valid token. In hybrid mode, if a localhost caller
sends a token anyway, it is validated and its claims are used.

**Roles and permissions**: four roles exist — `admin`, `operator`,
`agent`, and `readonly`. Each role maps to a static permission set.

| Role | Permissions |
|------|-------------|
| admin | all |
| operator | all except `admin` |
| agent | remember, recall, modify, forget, recover, documents |
| readonly | recall only |

The `requirePermission` middleware enforces permission checks per
route. The `requireScope` middleware checks whether a token's scope
(project, agent, user fields) matches the request target. Unscoped
tokens and admin-role tokens bypass scope checks.

**Rate limiting** (`rate-limiter.ts`): a sliding-window rate limiter
keyed by `actor:operation`. In team and hybrid modes, the actor is
the token's `sub` claim or the `x-signet-actor` header. When the
limit is exceeded, the response is 429 with a `Retry-After` header.

---

Analytics
---------

`platform/daemon/src/analytics.ts` implements an in-memory [[analytics]]
accumulator. All state is ephemeral — it resets on daemon restart.
Durable history lives in `memory_history` and structured logs.

**Usage counters**: four Maps track endpoints, actors, providers, and
connectors. Endpoint stats record call count, error count, and total
latency. Actor stats classify requests as remember/recall/mutate/other
by path pattern. Provider stats track LLM call count, failures, and
latency. Connector stats track syncs, errors, and documents processed.

**Error ring buffer**: a fixed-capacity array (default 500 entries)
of `ErrorEntry` records. When full, the oldest entry is evicted. Each
entry carries timestamp, stage, error code, message, and optional
memory ID and actor. Error codes form a taxonomy by stage:
`EXTRACTION_TIMEOUT`, `EXTRACTION_PARSE_FAIL`, `DECISION_TIMEOUT`,
`DECISION_INVALID`, `EMBEDDING_PROVIDER_DOWN`, `EMBEDDING_TIMEOUT`,
`MUTATION_CONFLICT`, `MUTATION_SCOPE_DENIED`, `CONNECTOR_SYNC_FAIL`,
`CONNECTOR_AUTH_FAIL`.

**Latency histograms**: four operations are tracked (`remember`,
`recall`, `mutate`, `jobs`) using a ring-buffer of 1,000 samples each.
Snapshots expose p50, p95, p99, count, and mean. The sort is deferred
until a snapshot is requested.

---

Connector Framework
-------------------

The connector framework manages external data source integrations that
push documents and memories into the [[pipeline]]. It is distinct from the
[[harnesses|harness connector packages]] (claude-code, opencode, openclaw, oh-my-pi, pi) — those
handle platform hook installation; this framework handles ongoing sync.

**Registry** (`connectors/registry.ts`): CRUD operations on the
`connectors` table. `registerConnector` inserts a new row with
`status = 'idle'` and returns its UUID. `updateConnectorStatus`
transitions a connector between `idle`, `syncing`, and `error` states.
`updateCursor` persists the sync cursor after a successful run. The
cursor is a JSON object stored in `cursor_json`; it tracks the
high-water mark for incremental sync (typically a timestamp or offset).

**Filesystem connector** (`connectors/filesystem.ts`): watches a
directory path, ingests files as documents, and tracks the cursor
based on file modification times.

**Document count** is tracked by querying `documents.source_url` with
a prefix match against the connector's configured root path.

**Health**: connector health is one of the six diagnostic domains.
It measures the count of connectors with `last_error IS NOT NULL` and
the age of the oldest unresolved error.

---

Document Ingest
---------------

The document worker handles URL fetches and raw content ingestion.
It follows the same `memory_jobs` queue as the extraction worker,
using job type `document_ingest`.

**Lifecycle**: a document row starts at `status = 'queued'` when
registered. The worker transitions it through `extracting` → `chunking`
→ `embedding` → `indexing` → `done`. Each transition is a separate
`withWriteTx` call so the current status is always visible without
holding a write lock during I/O.

**URL fetch**: if `source_type = 'url'`, the worker calls `fetchUrlContent`
with a configurable byte limit. The fetched title is written back to
the document row if not already set.

**Chunking**: `chunkText` splits content into overlapping fixed-size
chunks. The chunk size and overlap are configurable via
`documentChunkSize` and `documentChunkOverlap`. Each chunk becomes
a memory row of type `document_chunk` with `importance = 0.3`.

**Embedding and deduplication**: the embedding call happens outside
the write lock. Each chunk is normalized and hashed; if an identical
hash already exists as a memory linked to the same document, the
chunk is skipped. Embeddings are stored in the `embeddings` table
keyed by content hash.

**Linking**: each chunk memory is linked to its source document via
`document_memories(document_id, memory_id, chunk_index)`.

**Failure**: on error the document status is set to `failed` with an
error message. The job follows standard retry logic — up to
`max_attempts` tries before going `dead`.

---

Diagnostics and Repair
-----------------------

`platform/daemon/src/diagnostics.ts` provides read-only health signals
across six domains. All functions accept a `ReadDb` or `ProviderTracker`
and return plain data — no mutations, no side effects.

**Composite score**: each domain score is multiplied by a fixed weight
and summed. Scores range from 0 to 1. Status thresholds are: `>= 0.8`
healthy, `>= 0.5` degraded, `< 0.5` unhealthy.

| Domain | Weight | Key signals |
|--------|--------|-------------|
| queue | 0.28 | depth > 50, dead rate > 1%, age > 5min, stale leases |
| storage | 0.14 | tombstone ratio > 30% |
| index | 0.19 | FTS/memory count mismatch > 10%, embedding coverage < 80% |
| provider | 0.24 | LLM availability rate from ring buffer |
| mutation | 0.10 | recovery events > 5 in last 7 days |
| connector | 0.05 | connectors with errors, age of oldest error |

**Provider tracker**: a ring buffer (default 100 entries) of
`success`/`failure`/`timeout` outcomes. Evicted entries have their
count decremented so the running totals stay accurate without a full
scan.

**Repair actions** (`repair-actions.ts`): four actions are defined.

- `requeueDeadJobs`: resets dead jobs to `pending` with `attempts = 0`
  (batch limit 50 per call).
- `releaseStaleLeases`: resets `leased` jobs whose `leased_at` predates
  the lease timeout back to `pending`.
- `checkFtsConsistency`: compares FTS row count to active memory count.
  If mismatch > 10% and `repair = true`, runs
  `INSERT INTO memories_fts(memories_fts) VALUES('rebuild')`.
- `triggerRetentionSweep`: calls the retention worker's `sweep()` method
  immediately outside the normal schedule.

All repair actions pass through a policy gate (`checkRepairGate`). The
gate checks `autonomous.frozen` first (hard stop), then
`autonomous.enabled` for agent-role callers (operators and daemon bypass
this), then a rate limiter with per-action cooldown and hourly budget.
Each successful repair writes an audit event to `memory_history` with
`memory_id = 'system'`.

**Maintenance worker** (`pipeline/maintenance-worker.ts`): runs on a
configurable interval. Each cycle calls `getDiagnostics`, builds repair
recommendations from the report, and either logs them (`observe` mode)
or executes them (`execute` mode). A halt tracker prevents the same
ineffective repair from running more than 3 consecutive cycles without
improving the composite score. The worker only starts its interval
timer when `autonomous.enabled && !autonomous.frozen`.

---

Database Schema
---------------

SQLite with WAL mode. Migrations are numbered sequentially under
`platform/core/src/migrations/`. Each migration is idempotent — safe
to re-run against an existing database. Schema version is tracked in
`schema_migrations`. The latest migration is `078-api-keys.ts`.

**schema_migrations**

Tracks applied migration versions with checksum and timestamp. A
separate `schema_migrations_audit` table records duration per run.

**conversations**

Session-scoped records from harness hooks. Fields: `session_id`,
`harness`, `started_at`, `ended_at`, `summary`, `topics`, `decisions`,
`vector_clock`, `version`, `manual_override`. Indexed on `session_id`
and `harness`.

**memories**

The central table. Core fields: `id` (UUID), `type`, `category`,
`content`, `confidence`, `importance`, `source_id`, `source_type`,
`tags` (JSON array), `who`, `why`, `project`.

Pipeline v2 additions: `content_hash` (SHA-256 of normalized content),
`normalized_content`, `is_deleted` (soft delete flag), `deleted_at`,
`extraction_status` (`none`, `pending`, `completed`, `failed`),
`embedding_model`, `extraction_model`, `update_count`.

Access tracking: `last_accessed`, `access_count`, `pinned`.

A unique partial index enforces `content_hash` uniqueness among
non-deleted memories:

```sql
CREATE UNIQUE INDEX idx_memories_content_hash_unique
    ON memories(content_hash)
    WHERE content_hash IS NOT NULL AND is_deleted = 0
```

**embeddings**

Stores raw embedding vectors as BLOBs. Keyed by `content_hash`
(unique). Fields: `vector` (BLOB), `dimensions`, `source_type`,
`source_id`, `chunk_text`. The `vec_embeddings` virtual table
(sqlite-vec `vec0`) provides ANN search when the extension is loaded.

**memories_fts**

FTS5 external content table backed by `memories`, created with the
`unicode61` tokenizer to avoid overly aggressive stemming on recall
queries. Three triggers (`memories_ai`, `memories_ad`, `memories_au`)
keep the index in sync with inserts, deletes, and updates. Queried with
BM25 scoring via `bm25(memories_fts)`.

**memory_jobs**

Durable job queue. Fields: `job_type`, `status` (`pending`, `leased`,
`completed`, `failed`, `dead`), `payload`, `result`, `attempts`,
`max_attempts`, `leased_at`, `completed_at`, `failed_at`, `error`,
`document_id` (for document_ingest jobs). Indexed on `status`,
`memory_id`, `completed_at` (partial, status=completed), and
`failed_at` (partial, status=dead).

**memory_history**

Immutable audit trail. Fields: `memory_id`, `event` (`created`,
`updated`, `deleted`, `recovered`, `none`), `old_content`,
`new_content`, `changed_by`, `reason`, `metadata` (JSON), `actor_type`
(`operator`, `agent`, `daemon`), `session_id`, `request_id`. The
pipeline writes shadow proposals here as `event = 'none'` with a JSON
`metadata` blob containing the full proposal.

**entities**

Knowledge graph nodes. Fields: `name`, `entity_type`, `description`,
`canonical_name` (lowercased for lookup), `mentions` (denormalized
count), `embedding` (BLOB, optional). Indexed on `canonical_name`.

**relations**

Knowledge graph edges. Fields: `source_entity_id`, `target_entity_id`,
`relation_type`, `strength`, `mentions`, `confidence`, `metadata`,
`updated_at`. Unique on (source, target, type). Indexed on source,
target, and a composite (source, type) for outgoing edge traversal.

**memory_entity_mentions**

Junction table linking memories to entities. Composite primary key
`(memory_id, entity_id)`. Additional fields: `mention_text`,
`confidence`, `created_at`. Indexed on `entity_id` for inbound
traversal during graph boost.

**documents**

Documents queued for ingest. Fields: `source_url`, `source_type`,
`content_type`, `content_hash`, `title`, `raw_content`, `status`
(`queued`, `extracting`, `chunking`, `embedding`, `indexing`, `done`,
`failed`), `error`, `connector_id`, `chunk_count`, `memory_count`,
`metadata_json`, `completed_at`. Indexed on `status`, `source_url`,
`connector_id`, and `content_hash`.

**document_memories**

Links documents to the memory chunks generated from them. Composite
primary key `(document_id, memory_id)`. Includes `chunk_index` for
ordering.

**connectors**

External data source registrations. Fields: `provider`, `display_name`,
`config_json` (full config as JSON), `cursor_json` (incremental sync
state), `status` (`idle`, `syncing`, `error`), `last_sync_at`,
`last_error`. Indexed on `provider`.

**summary_jobs**

Session summary queue. Fields include `session_key`, `session_id`,
`trigger`, `captured_at`, `started_at`, `ended_at`, `harness`,
`status`, `error`, and `created_at`. The summary worker polls this
table, writes canonical immutable `--summary.md` artifacts for normal
session-end jobs, and keeps checkpoint extracts DB-native.

**memory_artifacts**

Derived DB index over canonical markdown history. Fields include
`agent_id`, `source_path`, `source_sha256`, `source_kind`,
`session_id`, `session_key`, `session_token`, `project`, `harness`,
timing fields, `manifest_path`, `memory_sentence`,
`memory_sentence_quality`, `content`, and `updated_at`. This table is
rebuildable from markdown artifacts and powers rolling ledger reads.

**memory_artifact_tombstones**

Privacy-removal guardrail for canonical artifact sessions. Fields:
`agent_id`, `session_token`, `removed_at`, `reason`, `removed_paths`.
Re-index honors tombstones so deleted canonical history does not
reappear.

**session_transcripts** (migration 040)

Lossless session transcript storage. Fields: `session_key` (PK),
`content` (cleaned conversation transcript), `harness`, `project`,
`agent_id`, `created_at`. The transcript keeps only user/assistant
conversation turns for memory use. Raw tool traces may be retained in
daemon logs for audit. The recall endpoint supports `expand: true` to
join transcript content back into results via `source_id`, preserving
facts that extraction may drop. Indexed on `project` and `created_at`.

**memory_search_telemetry** (migration 066)

Local-only recall QA ledger created only when
`memory.pipelineV2.telemetry.memorySearchQaEnabled` is enabled. Fields:
`id`, `created_at`, `route`, `agent_id`, `session_key`, `project`,
`query`, `keyword_query`, `filters_json`, `method`, `result_count`,
`top_score`, `no_hits`, `duration_ms`, `timings_json`, `results_json`,
and `sources_json`. This table intentionally stores recall query text
and recalled result snapshots, so it is treated as sensitive memory
content: list/export routes require `analytics` permission and enforce
the authenticated token's agent/project scope before serialization.
Rows stay local and are retained until explicitly pruned or the local
SQLite database is removed; they are never sent through anonymous
telemetry event sinks.

**umap_cache**

UMAP projection cache. Fields: `id`, `dimensions`, `embedding_count`,
`result_json` (full projection as JSON), `cached_at`. One row per
dimension value. Invalidated and replaced whenever the embedding count
changes.

**tokens**

(Planned) Persistent token store for team mode token management.
Currently tokens are issued and verified against the in-memory secret;
revocation requires a daemon restart to rotate the secret.

**skill_meta** (migration 018)

Procedural memory metadata for installed skills. Fields: `skill_name`,
`decay_rate`, `use_count`, `role_classification`, `filesystem_path`.
Supports retention decay and role-based skill prioritization.

**entity_aspects** (migration 019)

Knowledge architecture: conceptual domains per entity. Fields:
`entity_id`, `aspect_name`, `description`, `confidence`. Organizes
entity knowledge into thematic clusters for structured retrieval.

**predictor_comparisons** (migration 020)

Predictive scorer: session comparison pairs used for preference
learning. Fields: `session_id`, `memory_a_id`, `memory_b_id`,
`preferred`, `confidence`, `created_at`.

**entity_attributes** (migration 021)

Knowledge architecture: facts and constraints under aspects. Fields:
`aspect_id`, `entity_id`, `attribute_key`, `attribute_value`,
`confidence`, `source_memory_id`. Stores structured facts about entity
aspects.

**entity_dependencies** (migration 022)

Knowledge architecture: structural edges between entities distinct from
semantic `relations`. Fields: `source_entity_id`, `target_entity_id`,
`dependency_type`, `strength`, `metadata`. Models build-time or
logical dependency graphs.

**predictor_training_pairs** (migration 023)

Predictive scorer: labeled training data for the preference model.
Fields: `session_id`, `memory_id`, `feature_vector` (BLOB), `label`,
`created_at`. Used for incremental model updates.

**agent_feedback** (migration 024)

Storage for the `memory_feedback` MCP tool. Fields: `memory_id`,
`session_id`, `feedback_type` (`positive`, `negative`, `correction`),
`correction_text`, `actor`, `created_at`. Records agent-provided
feedback for memory quality improvement.

**task_meta** (migration 025)

Knowledge architecture: task-specific entity metadata. Fields:
`entity_id`, `task_type`, `priority`, `status`, `due_at`,
`context_json`. Extends entities with actionable task properties.

**entity_pinning** (migration 026)

KA-6: user-driven entity weight overrides. Fields: `entity_id`,
`pin_type` (`pin` or `suppress`), `weight_override`, `reason`,
`created_at`. Allows users to amplify or suppress specific entities
in graph-augmented search results.

---

Content Normalization
---------------------

`platform/daemon/src/content-normalization.ts` provides deterministic
normalization and hashing for deduplication.

The pipeline is:

1. `normalizeContentForStorage`: trim whitespace, collapse internal
   runs of whitespace to a single space. This is what gets stored in
   the `content` column.
2. `deriveNormalizedContent`: lowercase the storage content, strip
   trailing punctuation. This is the canonical form used for hashing.
3. Hash: SHA-256 of the normalized content. If normalization produces
   an empty string, the hash falls back to the lowercased storage
   content.

The returned `contentHash` is stored in `memories.content_hash`. The
unique partial index on that column ensures that two memories with
semantically identical content (differing only in case or trailing
punctuation) cannot both exist as non-deleted rows. Collision at insert
time (UNIQUE constraint violation) is handled gracefully — the worker
treats it as a dedup hit and records a `dedupedExistingId` in history.

Contradiction detection in the worker (`detectContradictionRisk`) runs
a lightweight token-level analysis: it checks for negation token
asymmetry (one side has a negation word, the other doesn't) and
antonym pair conflicts across a predefined set of boolean pairs
(`enabled`/`disabled`, `allow`/`deny`, etc.). At least two tokens must
overlap before either check is applied.

---

UMAP Projection
---------------

`platform/daemon/src/umap-projection.ts` computes server-side 2D or 3D
projections from stored embeddings using the UMAP algorithm.

Key implementation details:

- `nNeighbors = min(15, max(2, n-1))` — adapts to dataset size to prevent
  UMAP from requesting more neighbors than data points.
- **Exact KNN** for ≤ 450 embeddings (`O(n²)` distance matrix).
  **Approximate KNN** for larger sets — uses sliding windows over the
  X- and Y-sorted projected points, trading a small accuracy loss for
  much faster edge construction.
- Output coordinates are min-max normalized to the range `[-210, 210]`
  on each axis.
- Results are cached in `umap_cache`. Cache is invalidated when the
  embedding count changes between requests. `GET /api/embeddings/projection`
  returns `202 Accepted` while computing, then the full result once cached.

---

Retention
---------

`platform/daemon/src/pipeline/retention-worker.ts` purges expired data
on a configurable interval (default 6 hours). Each purge step runs in
its own short `withWriteTx` to avoid holding write locks across the
full sweep.

**Purge order** (from spec section 32.5 D2.3):

1. **Graph links**: delete `memory_entity_mentions` rows for tombstoned
   memories past `tombstoneRetentionMs` (default 30 days). Decrement
   `entities.mentions` for affected entities; remove entities whose
   count reaches zero.
2. **Embeddings**: delete `embeddings` rows for those same expired
   tombstone IDs.
3. **Tombstones**: hard-delete the `memories` rows. The `memories_ad`
   trigger fires synchronously and cleans the FTS index. Row count is
   taken from the pre-delete ID list to avoid FTS trigger inflation in
   the change count.
4. **History**: delete `memory_history` rows older than
   `historyRetentionMs` (default 180 days).
5. **Completed jobs**: delete `memory_jobs` rows with
   `status = 'completed'` older than `completedJobRetentionMs`
   (default 14 days).
6. **Dead jobs**: delete `memory_jobs` rows with `status = 'dead'`
   older than `deadJobRetentionMs` (default 30 days).

Each step is capped at `batchLimit` rows (default 500) per sweep to
bound latency. Backpressure accumulates until the next interval fires.

Default retention windows:

| Data | Default |
|------|---------|
| Soft-deleted memories (tombstones) | 30 days |
| History events | 180 days |
| Completed jobs | 14 days |
| Dead-letter jobs | 30 days |

---

User Data Layout
----------------

All agent data lives at `$SIGNET_WORKSPACE/`:

```
$SIGNET_WORKSPACE/
├── agent.yaml           # Config manifest
├── AGENTS.md            # Agent identity and instructions
├── SOUL.md              # Personality and tone
├── IDENTITY.md          # Structured identity metadata
├── USER.md              # User profile
├── MEMORY.md            # Generated working memory summary
├── memory/
│   ├── memories.db      # SQLite database (source of truth)
│   └── scripts/         # Optional batch tools (Python)
├── signetai/            # Managed local Signet source checkout
├── skills/              # Installed skills (subdirs)
├── .secrets/            # Encrypted secret store
└── .daemon/
    ├── pid
    └── logs/
        └── daemon-YYYY-MM-DD.log
```

By default the daemon binds to loopback. It can also bind for a configured
network mode such as Tailscale, with auth and CORS controls governing remote
access. All data stays local by design. The daemon collects local operational
telemetry (latency histograms, usage counters, error ring buffer) accessible
at `/api/telemetry/*`. Anonymous telemetry events never include prompts or
memory content. Optional recall QA capture writes a separate local-only search
ledger with query text and result snapshots for manual review.

---

HTTP API Reference
------------------

All endpoints are served by the Hono server on port 3850.

| Endpoint | Method | Auth | Description |
|----------|--------|------|-------------|
| `/health` | GET | none | Uptime, pid, version |
| `/api/status` | GET | none | Full daemon status |
| `/api/features` | GET | none | Feature flags |
| `/api/config` | GET | local | List config files |
| `/api/config` | POST | local | Save a config file |
| `/api/identity` | GET | local | Agent identity |
| `/api/auth/whoami` | GET | none | Current auth identity |
| `/api/auth/token` | POST | admin | Issue auth token |
| `/api/memories` | GET | recall | List with pagination |
| `/api/memory/remember` | POST | remember | Save a memory, enqueue extraction |
| `/api/memory/recall` | POST | recall | Hybrid search |
| `/api/memory/forget` | POST | forget | Batch forget memories |
| `/api/memory/modify` | POST | modify | Modify a memory |
| `/api/memory/search` | GET | recall | Search memories |
| `/api/memory/:id` | GET | recall | Get a memory |
| `/api/memory/:id` | PATCH | modify | Update a memory |
| `/api/memory/:id` | DELETE | forget | Delete a memory |
| `/api/memory/:id/history` | GET | recall | Memory version history |
| `/api/memory/:id/recover` | POST | recover | Recover a deleted memory |
| `/memory/search` | GET | recall | Legacy keyword search |
| `/memory/similar` | GET | recall | Vector similarity search |
| `/api/embeddings` | GET | recall | Export embeddings |
| `/api/embeddings/status` | GET | recall | Embedding provider status |
| `/api/embeddings/health` | GET | recall | Embedding health metrics |
| `/api/embeddings/projection` | GET | recall | UMAP 2D/3D projection |
| `/api/hooks/session-start` | POST | remember | Inject context into session |
| `/api/hooks/user-prompt-submit` | POST | recall | Per-prompt entity context load |
| `/api/hooks/session-end` | POST | remember | Extract session memories |
| `/api/hooks/remember` | POST | remember | Save a memory via hook |
| `/api/hooks/recall` | POST | recall | Search via hook |
| `/api/hooks/pre-compaction` | POST | remember | Pre-compaction instructions |
| `/api/hooks/compaction-complete` | POST | remember | Save compaction summary |
| `/api/hooks/synthesis/*` | GET/POST | local | MEMORY.md synthesis |
| `/api/harnesses` | GET | local | List configured harnesses |
| `/api/harnesses/regenerate` | POST | local | Regenerate harness configs |
| `/api/skills` | GET | local | List installed skills |
| `/api/secrets` | GET | admin | List secret names |
| `/api/secrets/exec` | POST | admin | Queue with multiple secrets |
| `/api/secrets/exec/:jobId` | GET | admin | Inspect queued secret exec job |
| `/api/secrets/:name/exec` | POST | admin | Queue with single secret (legacy) |
| `/api/documents` | GET/POST | documents | List or enqueue documents |
| `/api/documents/:id` | GET/DELETE | documents | Get or delete a document |
| `/api/documents/:id/chunks` | GET | documents | Get document chunks |
| `/api/connectors` | GET/POST | connectors | List or register connectors |
| `/api/connectors/:id` | GET/DELETE | connectors | Get or delete a connector |
| `/api/connectors/:id/sync` | POST | connectors | Trigger incremental sync |
| `/api/connectors/:id/sync/full` | POST | connectors | Trigger full re-sync |
| `/api/connectors/:id/health` | GET | connectors | Connector health |
| `/api/diagnostics` | GET | diagnostics | Full health report |
| `/api/diagnostics/:domain` | GET | diagnostics | Per-domain health score |
| `/api/pipeline/status` | GET | diagnostics | Pipeline status snapshot |
| `/api/repair/requeue-dead` | POST | operator | Requeue dead-letter jobs |
| `/api/repair/release-leases` | POST | operator | Release stale job leases |
| `/api/repair/check-fts` | POST | operator | Check/repair FTS consistency and tokenizer drift |
| `/api/repair/retention-sweep` | POST | operator | Trigger retention sweep |
| `/api/repair/embedding-gaps` | GET | operator | Count unembedded memories |
| `/api/repair/re-embed` | POST | operator | Batch re-embed missing vectors |
| `/api/repair/clean-orphans` | POST | operator | Remove orphaned embeddings |
| `/api/repair/dedup-stats` | GET | operator | Deduplication statistics |
| `/api/repair/deduplicate` | POST | operator | Deduplicate memories |
| `/api/checkpoints` | GET | recall | Session checkpoints by project |
| `/api/checkpoints/:sessionKey` | GET | recall | Session checkpoints by session |
| `/api/analytics/usage` | GET | analytics | Usage counters |
| `/api/analytics/errors` | GET | analytics | Recent error events |
| `/api/analytics/latency` | GET | analytics | Latency histograms |
| `/api/analytics/logs` | GET | analytics | Structured log entries |
| `/api/analytics/memory-safety` | GET | analytics | Mutation diagnostics |
| `/api/analytics/continuity` | GET | analytics | Continuity scores over time |
| `/api/analytics/continuity/latest` | GET | analytics | Latest score per project |
| `/api/telemetry/events` | GET | analytics | Query telemetry events |
| `/api/telemetry/stats` | GET | analytics | Aggregated telemetry stats |
| `/api/telemetry/export` | GET | analytics | Export telemetry as NDJSON |
| `/api/telemetry/memory-search` | GET | analytics | Query local recall QA telemetry |
| `/api/telemetry/memory-search/export` | GET | analytics | Export recall QA telemetry as NDJSON |
| `/api/git/status` | GET | local | Git sync status |
| `/api/git/pull` | POST | local | Pull from remote |
| `/api/git/push` | POST | local | Push to remote |
| `/api/git/sync` | POST | local | Pull then push |
| `/api/git/config` | GET/POST | local | Git sync configuration |
| `/api/update/check` | GET | local | Check for updates |
| `/api/update/config` | GET/POST | local | Update configuration |
| `/api/update/run` | POST | local | Apply pending update |
| `/api/tasks` | GET/POST | local | List/create scheduled tasks |
| `/api/tasks/:id` | GET/PATCH/DELETE | local | Get/update/delete task |
| `/api/tasks/:id/run` | POST | local | Trigger immediate run |
| `/api/tasks/:id/runs` | GET | local | Paginated run history |
| `/api/tasks/:id/stream` | GET | local | SSE stream of task output |
| `/api/logs` | GET | local | Daemon log access |
| `/api/logs/stream` | GET | local | SSE log streaming |
| `/mcp` | ALL | none | MCP server (Streamable HTTP) |
| `/*` | GET | none | Dashboard static files |

---

Key Files
---------

```
platform/core/src/
    types.ts                  TypeScript interfaces
    database.ts               SQLite wrapper (runtime-detecting)
    search.ts                 Hybrid search
    migrations/               Numbered migration scripts

platform/daemon/src/
    daemon.ts                 HTTP server + file watcher
    db-accessor.ts            withReadDb / withWriteTx wrappers
    transactions.ts           txIngestEnvelope and history helpers
    content-normalization.ts  SHA-256 dedup normalization
    analytics.ts              In-memory counters and histograms
    diagnostics.ts            Six-domain health scoring
    repair-actions.ts         Policy-gated repair functions
    session-tracker.ts        Plugin vs legacy runtime mutex
    memory-config.ts          PipelineV2Config type and defaults
    embedding-tracker.ts      Incremental embedding refresh tracker
    embedding-health.ts       Embedding health metrics
    inline-entity-linker.ts   Synchronous write-time entity linking
    memory-search.ts          Hybrid recall search orchestration
    session-checkpoints.ts    Session checkpoint persistence
    continuity-state.ts       Continuity state for compaction boundaries
    telemetry.ts              Local telemetry event collection
    feature-flags.ts          Runtime feature flags

    auth/
        types.ts              AuthMode, TokenRole, Permission
        tokens.ts             HMAC-SHA256 token sign/verify
        middleware.ts         Hono middleware: auth, scope, rate limit
        policy.ts             Permission matrix, scope enforcement

    connectors/
        registry.ts           CRUD for connectors table
        filesystem.ts         Filesystem connector

    pipeline/
        worker.ts             Extraction job worker
        extraction.ts         LLM fact + entity extraction
        decision.ts           LLM shadow decision engine
        graph-transactions.ts txPersistEntities, entity decrement
        graph-search.ts       Query-time graph boost (entity resolution)
        document-worker.ts    Document ingest job worker
        retention-worker.ts   Purge worker (6-step ordered purge)
        maintenance-worker.ts Autonomous diagnostics + repair loop
        provider.ts           LlmProvider interface + Ollama impl
        reranker.ts           Optional result reranking
        prospective-index.ts  Hints worker (hypothetical query generation)
        graph-traversal.ts    Traversal-primary retrieval path
        community-detection.ts Entity community clustering (Louvain)
```

---

Multi-Agent Support
-------------------

Multiple agents can share a single Signet daemon and database. The database
uses `agent_id` columns on all key tables to keep agent data separate.

**Agent roster** is declared in `agent.yaml` under `agents.roster`. Each
entry defines a named agent and its read policy. On daemon startup the
roster is synced to the `agents` table in SQLite.

**Memory ownership** — every memory row carries:
- `agent_id TEXT DEFAULT 'default'` — which agent wrote this memory
- `visibility TEXT DEFAULT 'global'` — who can read it:
  - `global`: any agent whose read policy permits it
  - `private`: only the owning agent
  - `archived`: soft-deleted when the owning agent is removed

**Read policies** control what a given agent sees on recall:

| policy    | SQL filter |
|-----------|------------|
| `isolated` | `agent_id = self` |
| `shared`  | `visibility = 'global' OR agent_id = self` |
| `group`   | `(visibility = 'global' AND agent_id IN group) OR agent_id = self` |

The default agent uses `shared` policy for backward compatibility — existing
installs see all their memories unchanged.

**Identity inheritance** — each agent can have its own identity files under
`$SIGNET_WORKSPACE/agents/{name}/`. On session start, the daemon checks the
agent directory first for the standard identity files (`AGENTS.md`, `SOUL.md`,
`IDENTITY.md`, `USER.md`, `TOOLS.md`, `HEARTBEAT.md`, `MEMORY.md`,
`BOOTSTRAP.md`) and falls back to the workspace root when an agent-local file
does not exist. This lets named agents override prompt identity and working
memory without copying the whole workspace. If no agent-local `MEMORY.md`
exists, the shared root `MEMORY.md` remains the working-memory projection for
that agent. The daemon's file watcher monitors `$SIGNET_WORKSPACE/agents/`
and triggers a harness sync on change.

**OpenClaw session keys** — OpenClaw encodes the agent ID in session keys as
`agent:{id}:{rest}`. The daemon's `resolveAgentId()` helper auto-parses this
format, so memories are routed to the correct agent without any extra config.

**Per-agent workspace** — when syncing to OpenClaw, the daemon writes an
assembled `AGENTS.md` to `$SIGNET_WORKSPACE/agents/{name}/workspace/` for
each agent. OpenClaw is configured to use this directory as the agent's
workspace, giving each agent its own context on session start.

**Single-agent installs** — fully backward compatible. Omitting
`agents.roster` from `agent.yaml` keeps the single-agent behavior. All new
API parameters (`agentId`, `visibility`) are optional with sensible defaults.