# remembrane

**Local-first persistent memory for AI agents.** One SQLite file, zero required dependencies. Exact hybrid recall (vector + BM25 — never approximate), explainable ranking, time-travel over memory history, conflict-aware recall that admits uncertainty, salience learned from task outcomes, budget-capped context packing (exactly optimal when numpy is present), and deterministic behavior you can unit-test in CI. Adapters for LangChain and CrewAI, plus a built-in MCP server.

```bash
pip install remembrane
```

## Why

Agents forget everything between sessions. Existing memory solutions are cloud APIs, require a vector database, or drag in a heavyweight framework. `remembrane` is the opposite:

- **One file.** Your agent's entire memory is a SQLite database you can copy, back up, diff, or delete.
- **Zero required dependencies.** The default embedder is pure stdlib. `pip install remembrane` pulls in nothing else.
- **Human-like recall.** Results are ranked by a weighted sum of similarity, recency decay (halves every week by default), importance, and outcome-earned usefulness. Recalled memories are *reinforced* — spaced repetition for agents.
- **Exact, not approximate.** Large systems use approximate nearest-neighbor search and accept missed results. At agent-memory scale, remembrane scores *every* memory — hybrid vector + BM25 keyword in one pass, guaranteed complete.
- **A memory you can debug.** Every store/forget/reinforce is journaled. Snapshot, diff, and reconstruct what your agent knew at any point in time. Every recall result explains exactly why it ranked where it did.
- **Testable in CI.** Deterministic embedder + frozen-time recall = reproducible memory behavior. `remembrane.testing` ships pytest-friendly assertions.
- **Framework-agnostic.** Use it bare, through the LangChain or CrewAI adapters, or expose it to any MCP-capable agent (like Claude) as an MCP server.

## Quick start

```python
from remembrane import MemoryStore

mem = MemoryStore("agent.db")            # or ":memory:" for ephemeral

mem.store("User prefers dark mode", importance=0.8)
mem.store("Deploy target is AWS us-east-1", namespace="ops")

results = mem.recall("what theme does the user like?")
print(results[0].memory.content)         # → "User prefers dark mode"
print(results[0].score)                  # weighted: similarity + recency + importance + usefulness
```

### Memory lifecycle

```python
mem.reinforce(memory_id)                  # strengthen: slower decay, higher rank
mem.forget(memory_id)                     # delete one
mem.forget(namespace="ops")               # delete a namespace
mem.forget(older_than_seconds=30*86400)   # prune stale memories
mem.consolidate()                         # merge near-duplicates
mem.export()                              # plain dicts, ready for json.dump
```

### Tuning recall

```python
from remembrane import MemoryStore, ScoringConfig

mem = MemoryStore(
    "agent.db",
    scoring=ScoringConfig(
        weight_similarity=0.65,
        weight_recency=0.15,
        weight_importance=0.10,
        weight_usefulness=0.10,            # earned from mark_useful()/mark_useless()
        half_life_seconds=7 * 24 * 3600,   # recency halves every week
    ),
)
```

## Embedders

The default `HashEmbedder` is deterministic, offline, and dependency-free — it hashes word and character n-grams. That makes similarity *lexical*, not semantic. It works well for typical agent memories (facts, preferences, short statements). For true semantic recall, plug in a real model:

```python
from remembrane import MemoryStore, SentenceTransformerEmbedder, OpenAIEmbedder

mem = MemoryStore("agent.db", embedder=SentenceTransformerEmbedder())   # local, pip install remembrane[sentence-transformers]
mem = MemoryStore("agent.db", embedder=OpenAIEmbedder())                # API,   pip install remembrane[openai]
```

Any object with `embed(texts) -> List[List[float]]` and a `dimension` attribute works.

Note: don't mix embedders in one database. Vectors from different embedders aren't comparable.

## LangChain

For current LangChain (verified against langchain-core 1.4):

```python
from langchain_core.runnables.history import RunnableWithMessageHistory
from remembrane import MemoryStore
from remembrane.adapters import RemembraneChatMessageHistory

store = MemoryStore("agent.db")
chain = RunnableWithMessageHistory(
    runnable,
    lambda session_id: RemembraneChatMessageHistory(store, session_id),
)
```

Needs `pip install langchain-core` (lazily imported — the rest of remembrane stays dependency-free). For legacy pre-1.x code, `RemembraneChatMemory` still provides the old `save_context` / `load_memory_variables` interface with semantic retrieval — no langchain install required at all.

## CrewAI

```python
from remembrane import MemoryStore
from remembrane.adapters import RemembraneStorage

storage = RemembraneStorage(MemoryStore("crew.db"))
storage.save("the deadline is next friday", metadata={"task": "planning"})
storage.search("when is the deadline?")          # also: delete / update / list_records / reset
```

A storage helper, duck-typed (save/search/delete/update/list_records/get_record/count/reset, kwargs-tolerant). **Known limitation:** it is not a registered `crewai.StorageBackend` subclass and returns dicts rather than `(MemoryRecord, score)` tuples, so plugging it directly into `crewai.Memory(...)` does not work as of crewai 1.14 — use it directly or behind a thin shim. Native StorageBackend integration is on the roadmap. Note CrewAI itself phones home (`telemetry.crewai.com`); set `CREWAI_DISABLE_TELEMETRY=true` if that matters to you — bare remembrane opens no sockets (verified by audit under Python audit hooks).

## MCP server

Give any MCP-capable agent (e.g. Claude Desktop, Claude Code) persistent memory:

```bash
pip install remembrane[mcp]
remembrane-mcp --db ~/agent-memory.db
```

```json
{
  "mcpServers": {
    "remembrane": {
      "command": "remembrane-mcp",
      "args": ["--db", "/path/to/agent-memory.db"]
    }
  }
}
```

Tools exposed: `memory_store`, `memory_recall`, `memory_forget`, `memory_reinforce`, `memory_conflicts`, `memory_resolve`, `memory_feedback`, `memory_pack`, `memory_stats`. Stored content is capped at 100k chars per memory (`REMEMBRANE_MAX_CONTENT` to change).

## CLI

```bash
remembrane --db agent.db store "the user prefers dark mode" --importance 0.8
remembrane --db agent.db recall "what theme?"
remembrane --db agent.db list
remembrane --db agent.db stats
remembrane --db agent.db export > backup.json
```



## Conflict-aware recall

Every other memory system silently resolves contradictions and returns one confident answer — which is how agents end up confidently wrong. remembrane surfaces the tension and lets the agent adjudicate (or ask the user):

```python
mem.store("the user lives in London")
mem.store("the user moved to Tokyo, no longer in London")

for c in mem.conflicts("where does the user live?"):
    print(c.describe())
# Conflicting memories (likely, change_markers=['longer', 'moved', 'no']):
#   older: 'the user lives in London' (recalled 4x)
#   newer: 'the user moved to Tokyo, no longer in London' (recalled 0x)

mem.resolve(keep_id=newer.id, drop_ids=[older.id], reason="user confirmed Tokyo")
```

Detection is deterministic and free (anchor-word overlap, negation markers, numeric mismatches, value substitution — honest heuristics, not hidden LLM judgments). Two confidence tiers: `likely` (strong negation, a numeric/weekday/month mismatch with corroboration, or a same-template value swap like "written in Python" → "written in Rust") and `possible` (topical tension worth a look). Independent audit on a 30-pair adversarial set measured the `likely` tier at 0.875 precision / 0.70 recall on v0.4 and 0.889 / 0.80 on v0.5.0 — better, not perfect. v0.5.1 adds the value-substitution signal, which catches the two false negatives that audit named (technology and region swaps); the full set has not been re-measured. Known limitations: rephrased (non-template) contradictions can stay in the `possible` tier, and "old X / new X" pairs describing two coexisting things can flag as `likely` (suppressing those would also suppress real old→new contradictions, so we don't). It remains a heuristic: treat conflicts as *candidates for the agent to adjudicate*, which is the design intent. Filter with `conflicts(min_confidence='likely')`. Resolutions are journaled, so every settled conflict stays auditable via `log()` and `as_of()`. Also exposed as the `memory_conflicts` / `memory_resolve` MCP tools and `remembrane conflicts` CLI.

## Salience earned from outcomes

Cloud systems decide what matters *at write time*, with an LLM call you pay for on every memory. remembrane inverts it: writes are free, and importance is **earned by helping**:

```python
results = mem.recall("how do I deploy this?")
# ... agent completes its task using results[0] ...
mem.mark_useful(results[0].memory.id)     # this memory rises
mem.mark_useless(results[2].memory.id)    # this one fades
```

Feedback accumulates into a usefulness signal (sigmoid-squashed into ranking, neutral at zero). Memories that keep helping outrank memories that merely match — learned per-deployment, from real outcomes, with zero LLM calls.

## Token-budget packing

Agents don't want "top 5 results"; they want the best use of the context window space they have left:

```python
context = mem.pack("user preferences", budget_tokens=800)
sum(r.tokens for r in context)   # <= 800, guaranteed
```

`pack()` scores every candidate exactly, suppresses near-duplicates so the budget is never spent saying the same thing twice, then solves the selection with a 0/1 knapsack. The budget is a hard guarantee in every configuration (verified over thousands of randomized trials). Optimality depends on the path: with numpy installed the solution is *exact* at 1-token granularity; the pure-python fallback uses coarsened weights plus a greedy refill and is documented as near-optimal, not optimal (worst observed loss 16% on adversarial random instances — real memory stores sit nowhere near that). Deterministic, no LLM. Pass `token_estimator=your_tokenizer` for exact counts.

## Time travel

Every mutation is journaled, so the past is queryable:

```python
mem.snapshot("before-research")
# ... agent runs, learns things, forgets things ...

mem.diff("before-research")
# {'added': [{'content': 'competitor launched a new pricing tier', ...}],
#  'removed': [...], 'changed': [...]}

mem.as_of("before-research")          # full memory state at that point
mem.log()                             # newest-first history of every operation
```

Or from the CLI: `remembrane snapshot v1`, `remembrane diff v1`, `remembrane log`.
"What did my agent believe last Tuesday, and what changed its mind?" is now an answerable question.

## Explainable recall

No black boxes — every result carries its full ranking breakdown:

```python
r = mem.recall("what theme does the user like?")[0]
r.explain()
# {'score': 0.6087, 'components': {'vector_similarity': 0.71, 'keyword_bm25': 1.0,
#   'combined_similarity': 0.81, 'recency': 0.98, 'importance': 0.8}, ...}
r.explain_text()
# 'score 0.609 = similarity 0.812 (vector 0.713, keyword 1.000) + recency 0.984 + importance 0.80 | recalled 3x'
```

## Testing your agent's memory

Deterministic recall means memory behavior is unit-testable — something no cloud memory API can offer:

```python
from remembrane.testing import assert_recalls, assert_recalls_first, assert_not_recalls

def test_agent_remembers_allergies():
    mem = build_agent_memory()
    assert_recalls_first(mem, "any food allergies?", "peanuts")
    assert_not_recalls(mem, "any food allergies?", "dark mode", k=1)
```

Pass `now=...` to `recall()` to freeze time and make recency scoring reproducible.

## Merging memories

Memory files are portable — merge two agents' brains, with near-duplicate absorption:

```python
mem.merge_from("other-agent.db")            # {'added': 12, 'merged': 3}
mem.merge_from("backup.db", namespaces=["prefs"], dedupe_threshold=0.95)
```

CLI: `remembrane --db a.db merge b.db`


## Performance

Performance numbers don't travel between machines, so measure your own first:

```bash
python -m remembrane.bench
```

Two reference points (hybrid recall, 512-dim default embedder, warm cache):

| memories | recall / pack (Linux sandbox, py3.10, numpy) | recall / pack (independent audit: Windows, py3.12, numpy) | recall (pure python, audit machine) |
|---|---|---|---|
| 1,000 | ~2 ms / ~17 ms | ~5 ms / ~32 ms | ~113 ms |
| 10,000 | ~30 ms / ~44 ms | ~51 ms / ~76 ms | ~1.2 s |
| 50,000 | ~205 ms / ~222 ms | ~1.0 s / ~430 ms | ~6.8 s |

The core stays dependency-free; if numpy is importable it is used automatically (`pip install remembrane[fast]`), and a broken numpy install is ignored rather than fatal. For sub-10ms recall beyond ~10k memories, or anything beyond ~50k, you've outgrown the design — that's vector-database territory, and remembrane won't pretend otherwise.

### Concurrency

Multiple connections, threads, and processes can share one memory file: file-backed stores default to SQLite WAL mode with a busy timeout and immediate write transactions, caches detect external writes via SQLite's `data_version`, and residual lock races are retried. Our test suite hammers 3 connections × 6 threads and 8 processes against a single file with zero errors. Two caveats: WAL keeps transient `-wal`/`-shm` sidecar files next to the db (pass `journal_mode="DELETE"` for strict single-file behavior), and SQLite on network filesystems (NFS/SMB) is unsafe regardless of mode — keep memory files on local disk.

## How ranking works

```
score      = 0.65·similarity + 0.15·recency + 0.10·importance + 0.10·usefulness
recency    = exp(−ln2 · age / half_life)
usefulness = sigmoid(outcome feedback)
```

Scoring is a weighted sum (weights normalize to 1), with one hard rule on top: similarity must be positive for a memory to be returned at all — recency and importance rank relevant memories, they never substitute for relevance.

`age` is measured from the memory's **last access**, not creation — every recall resets the decay clock. Frequently-used memories stay vivid; untouched ones fade. In the default hybrid mode, similarity is `0.65·cosine + 0.35·bm25`. All weights, the mode, and the half-life are configurable.

## Design choices

- **SQLite over a vector DB** — agent memory stores are small (thousands, not billions, of rows). Exact brute-force scoring at that scale is fast enough (see Performance for measured numbers), and you gain transactions, a single portable file, and zero infra.
- **No background daemon** — decay is computed at read time, so nothing runs when your agent doesn't.
- **Duck-typed adapters** — `remembrane` never imports langchain or crewai; the adapters match their interfaces structurally, so there are no version-pinning fights.

## Scope notes

- The CLI writes wherever `--db` points, with the invoking user's permissions — it is a local tool, not a sandbox. Wrap it if you expose it to untrusted input.
- OS argv limits apply to `remembrane store "<content>"`; use `--file path` or `--file -` (stdin) for large content.
- MCP argument validation follows pydantic's lax coercion (e.g. `useful="yes"` coerces to `True`).
- Recall `touch` updates (access stats) are statistics, not events — they are intentionally not journaled, and `as_of()` reconstructs content/importance state only.
- `export()`/`merge_from()` carry memories (content, importance, metadata, access stats, usefulness) but not the source's journal history; embeddings are regenerated by the destination's embedder.
- Journal entries with corrupt payloads are surfaced in `log()` (with a `_corrupt` key) and skipped by `as_of()` reconstruction.
- A missing or corrupt embedding blob is self-healed on first read (re-embedded from content) and a `RuntimeWarning` is emitted naming the affected ids — availability is preserved, and the repair is loud, not silent.
- WAL `-wal`/`-shm` sidecar files can grow during sustained concurrent writes; they shrink back on checkpoint and disappear when all handles close. Reported sizes in our tests are settled-state, not peak.
- A process killed during initial db creation can leave an empty file; reopening it repairs the schema automatically.

## Development

```bash
git clone https://github.com/satyasairay/remembrane
cd remembrane
pip install -e .[dev]
pytest
```

## License

MIT