# dsh-memory Architecture

> Version: 0.2.0 · License: MIT · [Changelog](../CHANGELOG.md)

## Overview

Cross-session memory plugin for DeepSeek Harness (dsh):

- **Persist** preferences, lessons, project facts across sessions
- **Retrieve** by keyword and semantic (embedding) search
- **Inject** automatically so the agent "naturally carries" memory
- **Stay safe**: no data loss on concurrent writes or corrupt files

## Module Layout

```
src/
├── index.ts    # plugin entry: tools + session-first injection + pre-step listener
├── storage.ts  # Storage interface + YamlStorage (write queue, mtime cache, quarantine)
├── search.ts   # KeywordSearch / EmbeddingSearch / HybridSearch (RRF) / OllamaEmbedder
├── inject.ts   # selectForInjection / renderSection (title-first) / deriveTitle / entryTitle
├── dynamic.ts  # DynamicInjector (per-session dedup + retrieval gate + fixed-set exclusion)
└── types.ts    # data model (+ reserved fields: vector/scope/weight)
```

Layering: `Storage` and `SearchEngine` are interfaces — YAML is the v1 backend,
keyword + embedding the v1/v2 search; both can be swapped without touching the rest.

## Data Model

```yaml
- id: MEM-YYYYMMDD-NNN
  title: "Summary title (auto-derived if absent)"   # injected as title
  content: "..."
  category: preference | project | lesson | fact
  tags: [tag1, tag2]
  importance: high | normal | low
  created: YYYY-MM-DD
  updated: YYYY-MM-DD
  source: user | agent | conversation
  retired: false
  forceInject: false           # full behavioural rules, always resident
  injectLevel: session | auto  # resident vs on-demand
  vector: [...]   # embedding, auto-computed on write
```

Reserved fields: `vector` (active), `scope` (multi-project), `weight` (feedback scoring).

## Injection

Two channels:

1. **Session-level** (`injectLevel: session` + `forceInject`): injected ONCE on the
   session's first `agent/pre-step` as a `Session memory` block (into history) —
   later rounds ride the history + prefix cache instead of re-rendering every round.
   `forceInject` entries render full text (behavioural rules); `session` entries
   render as summary titles with id.
2. **Dynamic** (`agent/pre-step` listener): retrieves memories matching the current
   message via HybridSearch, appended at the message tail with a provenance preamble
   so the model never mistakes them for dialogue. A retrieval gate skips trivial
   messages; dynamic results exclude the fixed injection set (one memory appears at
   most once per prompt).

Injected memories render as `- [category] title (id)` — details expand via the
`view` tool by id.

## Search

- **KeywordSearch**: ASCII words (min 2 chars) + CJK 2-grams, stopword-filtered.
  Generic words (文件/测试/文档…) count as half a hit; queries with ≥3 tokens need
  ≥2 weighted hits. No-match → score 0.
- **EmbeddingSearch**: cosine similarity against stored vectors, `minSimilarity`
  relevance gate (default 0.65). Dimension mismatch is detected and logged.
- **HybridSearch**: Reciprocal Rank Fusion (k=60) over both ranked lists;
  score-0 entries are filtered before fusion.

Embedding provider: local [ollama](https://ollama.com) (`qwen3-embedding:0.6b`,
1024-dim), overridable via `DSH_MEMORY_EMBED_MODEL`. Degrades to keyword-only when
ollama is unavailable.

## Cache & Data Safety

- Process-local cache with **mtime invalidation** — external edits to memory.yaml are picked up
- **Serialized write queue** — concurrent appends never lose entries
- **Atomic writes** (tmp + rename) — a crash never leaves a truncated file
- **Corrupt-file quarantine** — a parse failure backs up the file and refuses writes instead of silently overwriting

## Tools

| Tool | Purpose |
|---|---|
| `remember` | Save (content / title / category / tags / importance / forceInject / injectLevel / source) |
| `recall` | Hybrid search (keyword + embedding) top-N |
| `view` | Expand ONE memory's full content by id |
| `list` | List memories as titles (category/tag/importance filters, `full` for content) |
| `forget` | Soft-delete (marks `retired`, keeps history) |
| `pin` | Toggle `forceInject` — pinned memories always appear at session start |

## Roadmap

| v | Capability |
|---|---|
| v1 | keyword + injection + YAML |
| v2 | embedding search (HybridSearch) |
| v3 | dynamic topic injection + provenance labeling |
| v4 | title-first injection + two-level (session/auto) injection |
| v5 | feedback scoring (planned) |
| v6 | multi-scope isolation (planned) |