# HyperMnesia [](https://github.com/Recluse/HyperMnesia/actions/workflows/ci.yml) [](LICENSE) [](.github/workflows/ci.yml) [](https://modelcontextprotocol.io) [](https://github.com/pgvector/pgvector) [](https://hub.xerj.org/contributors/Recluse) *The opposite of amnesia* (Greek *hypermnesia* — abnormally complete recall). Self-hosted long-term memory for AI coding agents — a Postgres-backed store that gives an agent (Claude Code, or any MCP client) two things: 1. **Architectural memory (doc-RAG + Tier 0/1)** — your repos' docs made searchable, plus a *component -> constraint* map that resolves a file path to the rules covering it, so the applicable invariants reach the agent **before** it edits, without a search. 2. **Personal memory** — durable facts, preferences and decisions distilled from work sessions and available in later ones, instead of being re-explained. One store (Postgres + [pgvector](https://github.com/pgvector/pgvector)), local-model friendly (embeddings via [Ollama](https://ollama.com), [TEI](https://github.com/huggingface/text-embeddings-inference), or anything that speaks the OpenAI `/v1/embeddings` API — LM Studio, llama.cpp's `llama-server`, vLLM), no cloud dependency. Runs on a laptop, one server, or Kubernetes. **Two interfaces, and they are not the same thing.** Search, the map and memory are exposed as MCP tools, so any MCP client can *ask* for them. The automatic half — constraints injected before an edit, memory recalled per prompt, sessions captured — is a set of **Claude Code hooks**. Another client gets the data through the same tools, but only when the agent decides to call one, which is the weakness the hooks exist to remove. ## PolyDaemon (optional integration) [PolyDaemon](https://github.com/Recluse/PolyDaemon) is an open-source agent/Telegram bridge published under the MIT license. HyperMnesia and PolyDaemon are independent projects: the bridge works without memory, and HyperMnesia works with MCP clients without the bridge. To use them together, register HyperMnesia's MCP server in your agent client alongside the bridge — see the [MCP setup](docs/INSTALL.md#mcp-client) and [OpenCode V2 configuration](docs/INSTALL.md#opencode-v2). Bridge connectivity does not register memory tools or install automatic profile, recall or transcript capture. The OpenCode sidebar can read project document/chunk counts, source hash freshness and map presence through the read-only `project_status({})` tool, after checking that the returned physical root matches the session directory. A connection or cached map alone does not establish freshness, and `checked_at` is the time of the check, not the last indexing time. PolyDaemon's [companion documentation](https://github.com/Recluse/PolyDaemon/blob/main/docs/companions.md) describes the optional integration and links back to HyperMnesia. ## Why A coding agent repeats mistakes when the rule, the earlier decision or the stated preference is not in its context at the moment it acts. Having it there is not a guarantee — an agent can be handed a rule and break it anyway, and it can get the same case right unaided. Missing context just makes the mistake much more likely. Three common gaps: - **The rule was written down and not read.** Your repo documents that only the data layer talks to Postgres. The agent opens a handler, writes a query, and the rule was two directories away in a file it had no reason to open. In that case it did not disobey; it never saw it. - **You explain yourself again every session.** The preference you stated last week, the decision you took last month, the reason the old approach was abandoned — all of it left with the context window. - **Search does not fire when it matters.** Retrieval only helps if something calls it, and an agent mid-edit does not stop to wonder whether it should. Storage is solved. Delivery is not. The usual answer is one big instructions file, and it loses for a specific reason: every rule in it costs tokens on every request whether or not the file being edited has anything to do with it, so the file gets trimmed to the rules that apply everywhere — and those are the vaguest ones. It also goes stale without saying so. Nothing tells you a path in it moved. **The map here is written by hand too. What is automatic is the selection and the delivery.** You maintain the components and their invariants; from there a file path is matched against the component globs, and that component's `must` rules — plus those reached through one hop of the dependency graph — are injected before the edit by a hook, rather than waited for. Documentation and past decisions stay reachable behind that, by search. ### Where it came from, and why it would rather say nothing This began somewhere else: retrieval over a large reference catalogue, where the task is to pick the right entry out of tens of thousands. That problem sets the standard the rest of this inherits. A confidently wrong pick is worse than no pick at all — it travels downstream into a document and is found much later, while a refusal is noticed in the same minute. Applying the same machinery to a pile of internal documentation came second, as a transfer, once it was clear the documents had become the bigger problem. So when this system is unsure, it is built to say so rather than to return the nearest thing it has: memory search abstains past a distance threshold instead of offering a loose match, the ingester refuses to write an empty corpus rather than emptying the scope it was asked to refresh, and the console reports a reading it could not take instead of drawing zeroes. That is the same rule each time, and it is older than the code. A hand-authored map rots, so the rot is made visible rather than assumed away: globs that match no file are reported, a store that will not answer says so instead of resembling a project with no rules, and `./hm doctor` names the faults that leave an install answering normally with a missing index, half its embeddings, or a scope that matches nothing. ### What it changed for one user [Boris Khodok](https://github.com/boris-virto) put the document side in on 3 September 2026 and pointed it at a folder of markdown notes that until then lived in his `CLAUDE.md` — so the whole folder reached the model on every request, and was re-read from cache at every step of every session. His usage panel, below, shows cache reads falling roughly fiftyfold within days and staying down; his output tokens barely moved. He had not been hitting the five-hour limits before the change, and has not since, so a smaller quota does not account for it.

Read that number as narrowly as it was measured: it is retrieval replacing a folder pinned into the context. He had no Tier 0/1 map — his `get_project_map` came back empty — and no personal-memory pipeline running, so it is not evidence about either of those. Those deserve their own measurement, and this repository does not have one yet. He is also this repository's first outside contributor, and both contributions are of a piece with the rest of it: [issue #1](https://github.com/Recluse/HyperMnesia/issues/1), a git-ignored directory that ingested zero documents and said nothing about it, and [pull request #2](https://github.com/Recluse/HyperMnesia/pull/2), `--known-hashes`, which stopped a re-ingest from burning every embedding it already had. If you want to see it rather than read about it: **[docs/DEMO.md](docs/DEMO.md)** — two minutes, real output, no install beyond a Postgres. ## How it works ```mermaid flowchart TB subgraph store["🗄️ One Postgres + pgvector"] direction LR DOC[("doc chunks
embedding + tsvector")] MAP[("component / constraint
map + graph")] MEM[("mem.* personal memory
perspectives + evidence")] end subgraph ingest["📥 Ingest · offline"] MD["repo *.md"] --> CH["chunk by heading"] CH --> EMB["embed · bge-m3
Ollama / TEI"] EMB --> DOC CH --> TS["composite tsvector
(stem || simple)"] --> DOC SEED["hand-authored
Tier 0/1 seed"] --> MAP end subgraph ask["🔎 Agent asks · per request"] FP["file path"] -->|"deterministic"| T01["Tier 0/1: resolve
path → component"] T01 --> RULES["must / should constraints
+ 1-hop graph"] Q["query"] --> QE["embed query"] QE --> RRF["Tier 2: RRF fuse
vector cosine + FTS"] RRF --> RRK["cross-encoder rerank
bge-reranker-v2-m3"] RRK --> TOPK["top-k docs"] end subgraph pm["🧠 Personal memory · background"] SESS["session transcript"] --> EX["extract · LLM
durable facts only"] EX --> MEM MEM --> RC["recall → inject
into the prompt"] MEM --> CO["consolidate / Dreamer
every derived fact review-gated"] MEM --> CTX["context depth
fixed or evidence-selected"] CTX --> DIA["Dialectic
grounded claims or abstention"] end MAP -.-> T01 MAP -.-> RULES DOC -.-> RRF MEM -.-> RC ``` - **Tier 2 search** fuses dense (bge-m3 embeddings, HNSW) and lexical (composite `tsvector`, works for code identifiers and non-English) via **Reciprocal Rank Fusion**, then an optional **cross-encoder reranker** ([bge-reranker-v2-m3](https://huggingface.co/BAAI/bge-reranker-v2-m3)) reorders the top candidates. (It measurably helped on a private evaluation set; the figure is in [eval/README.md](eval/README.md) with what it is and is not — one corpus, not a benchmark.) - **Personal memory** is bi-temporal (event time vs ingestion time), **supersede-not-overwrite**, and perspective-aware: an observer and an observed entity make the point of view explicit. Evidence edges form an acyclic provenance graph. `memory_context` can select a bounded recall depth from retrieval evidence; `memory_ask` accepts only claims cited to active rows in that exact context and abstains when the grounding check fails. - **Derived memory stays human-gated.** The optional Dreamer proposes deductive or inductive conclusions with exact premise IDs. It cannot write them directly; every proposal waits in the same review queue used by consolidation. - **Capture/recall** run as Claude Code hooks: session profile injected at start, relevant memories injected per prompt, transcripts distilled to memories by a small LLM on a schedule. A malformed transcript line or a model answer with a `null` where a number belongs costs that one item, not the whole queue behind it, and a correction is never filed away as a duplicate of the fact it corrects — `100` is not `1000` with a digit missing ([#4](https://github.com/Recluse/HyperMnesia/issues/4), [#5](https://github.com/Recluse/HyperMnesia/issues/5)). - **Reading memory without paying for all of it.** `memory_search` with `brief` returns one line per hit — id, type, date, project, and roughly how many tokens reading it would cost — and `memory_get` then takes up to twenty ids in one call; `memory_timeline` shows what the same author learned just before and after one memory, for the hit that makes no sense on its own. The index-first reading path is borrowed from [claude-mem](https://github.com/thedotmack/claude-mem). - **Keeping things out of memory.** `...` in a prompt or a message keeps that span out whatever it looks like: it is stripped first, before anything is cut, sent to a model, searched for or stored, and spans nest. Under that, capture redacts secrets — key shapes, `name=value` credentials, authorization headers, PEM blocks, credentials in URLs — before a model or the store sees them. That part is pattern matching, so it catches the shapes it knows and says so in the docs instead of calling itself a guarantee; anything known to be sensitive belongs inside the tags ([docs/MEMORY.md](docs/MEMORY.md)). - **Constraint injection** is a hook too: a `PreToolUse` hook (`hooks/arch_invariants.py`) resolves the file you're about to edit to its component and injects the applicable `must` invariants before the edit — so Tier 1 is delivered deterministically, not left to the agent to ask for. - **Diagnostics, and what happens when something breaks.** A hand-authored map and a self-hosted store both fail in ways that still answer, so the failures are reported rather than inferred: `ci/freshness.py` flags globs matching no file and documents indexed at an older commit, and refuses a scope nothing is mapped under; `./hm doctor` (and the `status` MCP tool, which runs the same script) checks the ANN index, migrations, embedding coverage, model consistency and drift, the connection's identity mode and the scope name; search says when it fell back to lexical-only or to plain RRF order; a cached map carries its staleness and expires; an empty enumeration refuses to write rather than emptying the store. Each of those, and why it exists, is in **[docs/DIAGNOSTICS.md](docs/DIAGNOSTICS.md)**. - **Who can read a memory.** Every row records its author and its audience (private, or shared with a project), and row-level security applies that rule to every query, including the ones that forget it. On the shared `hm_app` login the caller names itself, so this is a filter against buggy queries, not a boundary between people; with one login per author (`./hm author add`) the database itself decides who is reading. The document corpus is shared and writable by every application login. See **[Trust](docs/DESIGN.md#trust)**. - **Timeouts, output limits and cache expiry.** Every child process the MCP server spawns is on a clock, a document comes back capped with the cut announced, and the structural map is re-read on a TTL rather than held for the life of the process. ## Where code fits HyperMnesia indexes **docs, the architecture map, and memory** — not code symbols. Live code structure ("where is `foo` defined, who calls it") is best answered by a **language server**, which already keeps a precise index and updates it as you type. Pair HyperMnesia with an LSP-backed symbol MCP such as [Serena](https://github.com/oraios/serena): both run as MCP servers in the same client, with no overlap — | Agent's question | Answered by | |---|---| | where is a symbol defined / who calls it / its type | **Serena / LSP** (live, no re-embed) | | what rules apply to this file, before I edit it | **HyperMnesia** Tier 0/1 | | where's the doc, and what do I know about this project/owner | **HyperMnesia** Tier 2 + memory | Live code → the LSP layer; anything you want to remember or that lives in prose → HyperMnesia. See **[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)** for the full system picture, an example `.mcp.json` pairing both, and how it relates to managed memory offerings. ## Components | Path | What | |------|------| | `hm` | one wrapper over the documented steps: `init` (compose + migrations), `install` (a commit as the machine's release under `~/.HyperMnesia`, with a rollback), `migrate` (the one apply order, recorded with checksums; a second run is a no-op, an edited applied file is refused), `author` (per-author logins), `ingest` (ingest -> embed -> ANN index, incremental when the scope already exists), `refresh`, `doctor`, `latency` | | `sql/` | doc-RAG and personal-memory schemas, the migrations `./hm migrate` applies and records in order, and the roles + row-level security behind memory visibility: a filter on the shared `hm_app` login, an enforced boundary with one login per author ([trust](docs/DESIGN.md#trust)) | | `ingest/` | markdown chunker, embedder (Ollama, TEI or OpenAI-compatible), secret redaction, hybrid RRF search, `mem_ops`; incremental re-ingest via `--known-hashes` (unchanged docs keep their embeddings) | | `rerank/` | optional cross-encoder reranker service + search orchestrator | | `hooks/` | Claude Code hooks plus model-backed maintenance: constraint/profile/recall injection, capture, extract, consolidate, reflect, review-gated Dreamer, and grounded Dialectic Q&A | | `ci/` | `doctor.py` — health check for faults that leave an install answering normally (missing index or migration, partial embeddings, mixed or drifted models, wrong scope, a connection that bypasses row-level security, a stalled extractor); `latency.py` — where the time goes (hook, embedder, database, reranker); `freshness.py` — map-staleness / orphan-glob checker (run against a target repo); `check_graph_sql_parity.py` — keeps the Python and Rust copies of the graph query identical | | `tests/` | contract tests, all wired into CI: hook I/O, ingest behavior, memory perspectives/evidence/context/Dreamer/Dialectic, glob parity, query hygiene, diagnostics, settings and the live SQL boundary | | `mcp-server/` | Rust MCP server exposing project map, constraints, doc search, scoped memory, grounded context/Q&A, `project_status`, and install `status` | | `console/` | the operator's side: a menu-bar tray and four command-line tools (`stats`, `jobs`, `settings`, `setup`) over any deployment — volumes, scheduled jobs, tunables, and a first-run wizard | | `deploy/` | Docker Compose, Kubernetes runbooks, and reusable Ansible roles | | `examples/` | an example structural-tier seed for a project | | `skills/` | `onboard-project` — the six steps to connect a new repo; `just` — answer-only / audit mode (agent-readable skills) | ## Install Start with **[docs/INSTALL.md](docs/INSTALL.md)** for prerequisites and the first working installation. The focused runbooks cover the next choice without making one page carry every variant: - **[deployment architectures](docs/DEPLOYMENT.md)** — local, single-server, split-compute, Kubernetes and CPU-only layouts; - **[components](docs/COMPONENTS.md)** — start and verify Postgres, embedding, ingest, reranking, model jobs, MCP, hooks and the console independently; - **[LLM backends](docs/LLM_BACKENDS.md)** — Ollama, vLLM, hosted OpenAI-compatible services, CLI adapters and model-sizing guidance; - **[Kubernetes](deploy/k8s/README.md)** — Postgres/PVC, TEI, optional reranker and vLLM, Jobs/CronJobs, client connectivity and network boundaries. - **[Ansible](deploy/ansible/README.md)** — immutable releases, fresh Claude Code/Codex registration, one-box servers, and a digest-pinned Kubernetes data plane. TL;DR (single box). This ends at the first thing you can *see*: an invariant arriving before an edit. ```bash ./hm init # writes .env with a generated password, compose up, # waits for Postgres, applies the migrations (hm migrate) export DATABASE_URL=... # init prints the exact line ./hm ingest /path/to/your/repo myrepo # ingest -> embed -> ANN index -> doctor ./hm install --wire # this commit as ~/.HyperMnesia/current: the MCP server built, # agent configs pointed at it rather than at this checkout ``` `./hm install` is what keeps a working machine from running whatever the checkout happens to be on this afternoon. It copies a commit into `~/.HyperMnesia/releases/` and points `current` at it; hooks, the MCP server, scheduled jobs and the console all run from there. Settings (`hypermnesia.env`) and state live in the same folder, `previous` is kept for a rollback (`./hm install --ref `), and the layout is the same on macOS, Linux and Windows (`HYPERMNESIA_HOME` moves it). The details are in [docs/INSTALL.md](docs/INSTALL.md). **Have a folder of notes wired into `CLAUDE.md`?** Ingest it and then take it out of `CLAUDE.md`. That is the fastest thing here and it needs neither the map nor the hooks: the notes stop riding along with every request and start arriving a few chunks at a time, when they are relevant. ```bash ./hm ingest /path/to/your/notes mynotes # --walk if the folder is git-ignored ``` Re-run that same line whenever the notes change: it is incremental by content hash, so unchanged files keep their chunks and their embeddings. `./hm refresh` does every scope at once and is the thing to put on a timer; `ci/freshness.py` reports when the index has fallen behind. If something in that folder must stay OUT of the corpus, list it in a `.hmignore` beside it — one pattern per line, a trailing `/` for a whole subtree (`scratch-*/` works too, and indexing and status checks never enter such a folder). Two kinds of thing belong there: a file of credentials, and a superseded set of documents that near-duplicates the current one. The second matters more than it looks. An obviously old document is harmless; one that says almost the same thing with a different answer ranks right beside the correct one and gives no sign in the text of which is which. `hm` is the recommended path because the *order* of those steps is load-bearing and getting it wrong is silent: the ANN index must be built after the first bulk embed, and a re-ingest without a known-hashes snapshot deletes the scope's documents and every embedding with them. [docs/INSTALL.md](docs/INSTALL.md) has the same steps by hand, the non-Docker and Kubernetes paths, and the flags (`--walk`, `--known-hashes`) that matter on later runs. The Tier 0/1 map is what defines which files belong to each component and which constraints apply to them. Nothing can generate it honestly from a directory listing, so writing it is the work — start from [examples/seed_example.sql](examples/seed_example.sql), load it under your own scope, then point your MCP client at `mcp-server` and register the hooks ([docs/INSTALL.md](docs/INSTALL.md#mcp-client)). To see a rule reach an edit, run the hook by hand against a file in your repo. It lives in the HyperMnesia checkout, and `cwd` must be the project being edited, so give both explicitly: ```bash HM=/path/to/hypermnesia # this checkout PROJ=/path/to/your/repo # what you ingested as `myrepo` printf '{"hook_event_name":"PreToolUse","tool_name":"Edit","cwd":"%s", "tool_input":{"file_path":"%s/src/api/users.py"}}' "$PROJ" "$PROJ" \ | HM_REPO=myrepo python3 "$HM/hooks/arch_invariants.py" ``` A `hookSpecificOutput` block naming your invariant confirms that the hook resolves that path against the map and returns the rule. It does **not** confirm that Claude Code is running the hook, or that your MCP client reached the server — for the first, make an edit from Claude Code and look for the same block; for the second, call the `status` tool, which answers from the store. **[docs/DEMO.md](docs/DEMO.md)** walks the same path in two minutes with real output. ## The console Everything above answers questions asked of it. The console is the other direction: what the store holds, whether the scheduled passes are running, and what the tunables are set to — without writing a query. ```bash cd console && cargo build --release # the CLI tools; zero dependencies ./target/release/hypermnesia-setup # the walkthrough: from nothing to a menu-bar icon ``` The tray itself needs a GUI toolkit, so it is not part of that plain build: unconditional on macOS, and on Linux behind `cargo build --release --features tray` (GTK3 + libayatana-appindicator, see [docs/INSTALL.md](docs/INSTALL.md)). It reaches the database exactly one way: a command that receives SQL on stdin. Direct psql, `docker exec`, `kubectl exec`, ssh to a machine that has kubectl — all of them are one string with different contents, which is why there is one setting and not five. The wizard tries the command before writing it, because an untried setting is a guess, and a console showing an empty screen cannot be told from an empty store. | Command | What | |---------|------| | `hypermnesia` | the menu-bar tray (macOS) / system tray (Linux, `--features tray`) | | `hypermnesia-stats` | the same numbers on stdout | | `hypermnesia-jobs` | scheduled passes: what is configured, when each last worked, run one now, change a schedule | | `hypermnesia-settings` | the tunables, each with the value in force and where that value came from | | `hypermnesia-setup` | the first-run walkthrough, and `--connect` / `--show` / `--test` on their own | ### The tray

A menu, not a window. Everything the console has to show is a dozen lines and a dozen buttons; a window would mean a GUI framework for the same result. The tray's GUI dependencies are declared per platform and behind a feature on Linux, and the data layer under them has none at all — a console that takes a minute to build is a console nobody rebuilds. What is in the menu: - **the first line is the age of what you are reading.** Not a status light: "updated 4m ago, in 1.2s", and after a failure "! not updated: , showing state from 12m ago". It is computed when the menu is drawn, from the wall clock, so a Mac that slept for two hours says two hours. The store's own numbers refresh every minute; the jobs — a local, cheap read on either platform — refresh every few seconds, faster still for a little while after a button is pressed, so pressing one does not mean waiting out the idle cadence to see whether it worked. - **the volumes** — memories active of total, knowledge pages, documents and chunks, how many chunks have no embedding, how many embedding models are in the store, the review queue, the stale count, the database size. - **Run now** — every scheduled pass, with its schedule, when it last wrote to its log, and its last exit code. Pressing one runs it — through launchd on macOS, through `systemctl --user start --no-block` on Linux — and reports what the service manager then did, not that the request was accepted: a program that is not there fails immediately, and that is read back rather than taken on faith. - **Schedule** — the common intervals and times, per job. It writes the unit, validates it, and reloads the job, because both launchd and systemd keep their own copy from the moment they loaded it: writing the file without the reload would show a new schedule while the old one is still in force. A backup goes down next to the file first, and on any slip along the way it is restored — what actually landed is read back and compared to what was asked before the tray calls it done. - **Timers** (Linux only) — arm or disarm a job's schedule without removing it. systemd tracks "loaded but not armed" as a state of its own, which the read side of this tray could already show as a fault; this is what fixes it. launchd has no such state — a job it has loaded runs on its schedule, full stop — so there is nothing here to switch on macOS. - **Refresh now** and **Quit**. The tray carries one mark for "something here needs a look", derived from the lines below rather than computed beside them: anything the menu would show with a `!` raises it. That is the whole design in one detail — the mark is where a problem is noticed, so it must not be able to disagree with the menu. The embedded icon is calm green or warned red; its tooltip names the status. On macOS, background updates wait until the open menu closes, so refreshing data does not dismiss it while you read or navigate a submenu. `hypermnesia --install` puts it in autostart to start at login, restarted if it crashes and not if you quit it — launchd on macOS, a systemd user unit on Linux. `--uninstall` takes it back out. Autostart is optional: an absent macOS plist does not raise a warning. **Try it without a database.** The console's only connection setting is a command that prints the query's answer, so a file works: ```bash HM_PSQL_CMD="cat demo/store.json" ./target/release/hypermnesia-stats # or hypermnesia, for the tray ``` That is the store in `console/demo/store.json` — a small fictional one, not anybody's real numbers. ### The rule it is written to **Stale must not look fresh, and missing must not look empty.** A reading that failed keeps the old numbers and labels them with their age. A job that never ran says so rather than showing launchd's zero as success — launchd prints exit 0 both for "finished well" and for "never finished". A settings file the hooks refuse is reported as refused, not displayed knob by knob as if it applied. An answer that is not this query's JSON is an error, not a store full of zeroes. Every complaint the console can print, and the two readings that are deliberately not complaints, are listed in [docs/DIAGNOSTICS.md](docs/DIAGNOSTICS.md). ## Design docs - [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — the whole system: the LSP/code layer + HyperMnesia, how to pair them, and related work. - [docs/DESIGN.md](docs/DESIGN.md) — architecture and the reasoning behind the tiers. - [docs/MEMORY.md](docs/MEMORY.md) — the personal-memory model (bi-temporal, supersede, consolidation). - [docs/COMPARISON.md](docs/COMPARISON.md) — where HyperMnesia fits vs. neighbours, and honest non-goals/limitations. - [skills/onboard-project/SKILL.md](skills/onboard-project/SKILL.md) — connecting a repository: ingest, the Tier 0/1 map, pairing with Serena, verification, and what changes per deployment. - [skills/just/SKILL.md](skills/just/SKILL.md) — `/just`: answer the question literally with read-only tools and stop; the contract is checkable from the tool log. Session-wide as "audit mode". ## License MIT — see [LICENSE](LICENSE).