# Octobrain

> Persistent memory for AI assistants — store insights, decisions, and knowledge that survives across conversations.

[![Crates.io](https://img.shields.io/crates/v/octobrain.svg)](https://crates.io/crates/octobrain)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
[![Rust](https://img.shields.io/badge/rust-1.95%2B-orange.svg)](https://www.rust-lang.org/)
[![CI](https://github.com/Muvon/octobrain/actions/workflows/ci.yml/badge.svg)](https://github.com/Muvon/octobrain/actions/workflows/ci.yml)
[![Coverage](https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmuvon%2Foctobrain%2Fbadges%2Fcoverage.json&style=flat-square)](https://github.com/Muvon/octobrain/actions/workflows/ci.yml)

_MCP Registry: `mcp-name: io.github.Muvon/octobrain`_

## Table of Contents
- [Why Octobrain?](#why-octobrain)
- [Quick Start](#quick-start)
- [Installation](#installation)
- [Usage](#usage)
- [Features](#features)
- [Benchmarks](#benchmarks)
- [Configuration](#configuration)
- [Memory Types](#memory-types)
- [MCP Integration](#mcp-integration)
- [Storage Locations](#storage-locations)
- [Contributing](#contributing)
- [License](#license)

**Octobrain** gives your AI assistant a long-term memory. Store code insights, architecture decisions, bug fixes, and knowledge — then retrieve them with semantic search in future sessions. Works as a CLI tool or as an MCP server for integration with Claude Desktop and other AI tools.

## Why Octobrain?

AI assistants start every conversation with zero context. You explain your project, your preferences, your decisions — every single time. Octobrain breaks that cycle:

- **Persistent memory** — Insights survive across sessions, not just within them
- **Semantic search** — Find memories by meaning, not exact keywords
- **Auto-linking** — Related memories connect automatically (Zettelkasten-style)
- **Knowledge indexing** — Ingest docs, articles, and files for retrieval
- **MCP integration** — Works with Claude Desktop and other MCP-compatible tools

## Quick Start

```bash
# Install from crates.io
cargo install octobrain

# Store your first memory
octobrain memory memorize --title "API Design Pattern" \
  --content "Use REST for CRUD, GraphQL for complex queries" \
  --memory-type architecture --tags "api,design"

# Search memories
octobrain memory remember "how should I design APIs"

# Start MCP server for Claude Desktop integration
octobrain mcp
```

## Installation

### From crates.io (Recommended)

```bash
cargo install octobrain
```

### From Source

```bash
# Clone and build
git clone https://github.com/muvon/octobrain.git
cd octobrain
cargo build --release

# Binary location
./target/release/octobrain --help
```

### Docker

```bash
docker pull ghcr.io/muvon/octobrain:latest
```

### Homebrew (macOS/Linux)

```bash
brew install muvon/tap/octobrain
```

### Feature Flags

Octobrain supports multiple embedding providers:

| Flag | Description | API Key Required |
|------|-------------|------------------|
| `fastembed` | Local embeddings via FastEmbed | No |
| `huggingface` | Local embeddings via HuggingFace | No |
| (default) | Both `fastembed` + `huggingface` | No |
| (no features) | API-based: Voyage, OpenAI, Google, Jina | Yes |

```bash
# Build with local embeddings (default, no API keys needed)
cargo build --release

# Build with API-based embeddings only
cargo build --no-default-features --release
```

For API-based embeddings, set the appropriate environment variable:
- `VOYAGE_API_KEY` for Voyage AI
- `OPENAI_API_KEY` for OpenAI
- `GOOGLE_API_KEY` for Google
- `JINA_API_KEY` for Jina

## Usage

### Memory Management

Store and retrieve insights, decisions, and context:

All `memory` subcommands accept global flags:
- `--scope <string>` — Override project scope (default: auto-detected from Git remote)
- `--role <string>` — Filter by role (e.g. "developer", "reviewer")

```bash
# Store a memory
octobrain memory memorize --title "API Design" \
  --content "Use REST for CRUD, GraphQL for complex queries" \
  --memory-type architecture --tags "api,design"

# Search memories (semantic search)
octobrain memory remember "api design patterns"

# Multi-query search for broader coverage
octobrain memory remember "authentication" "security" "jwt"

# Get a memory by ID
octobrain memory get <id>

# Get recent memories
octobrain memory recent --limit 20

# Filter by type
octobrain memory by-type architecture --limit 10

# Filter by tags
octobrain memory by-tags "api,security"

# Find memories related to files
octobrain memory for-files "src/main.rs,src/lib.rs"

# Update a memory
octobrain memory update <id> --title "New Title" --add-tags "new-tag"

# Delete a memory
octobrain memory forget --memory-id <id>

# Get memories relevant to the current Git commit
octobrain memory current-commit

# Show memory statistics
octobrain memory stats

# Clean up old/low-importance memories
octobrain memory cleanup

# Compact tables and refresh indices (run when searches slow down)
octobrain memory maintenance

# ⚠️ Delete ALL memory data
octobrain memory clear-all --yes
```

### Memory Consolidation

Close a goal and fold all its contributing memories into a consolidated summary:

```bash
# Consolidate a goal (Achieves-link sources → Consolidated state, importance dampened)
octobrain memory consolidate <goal-id> --summary "Final summary"

# Sleep consolidation: auto-cluster recent similar memories
octobrain memory sleep-consolidate --threshold 0.85 --min-size 3
```

### Memory Relationships

Connect related memories for context-rich retrieval:

```bash
# Create a relationship between memories
octobrain memory relate <source-id> <target-id> \
  --relationship-type "depends_on" \
  --description "Source requires target to function"

# View relationships for a memory
octobrain memory relationships <memory-id>

# Find related memories through relationships
octobrain memory related <memory-id>

# Auto-link similar memories (Zettelkasten-style)
octobrain memory auto-link <memory-id>

# Explore memory graph
octobrain memory graph <memory-id> --depth 2
```

### Knowledge Base

Index and search web content, docs, and files:

```bash
# Index a URL
octobrain knowledge index https://docs.rs/tokio/latest/tokio/

# Index a local file (.txt, .md, .pdf, .docx, .html)
octobrain knowledge index ./docs/architecture.md

# Search knowledge base
octobrain knowledge search "how to handle async tasks"

# Search within a specific source (auto-indexes if outdated)
octobrain knowledge search "spawn blocking" --source https://docs.rs/tokio/

# Read full content of a URL or local file
octobrain knowledge read https://docs.rs/tokio/latest/tokio/

# Search indexed content by regex pattern
octobrain knowledge match "spawn_blocking|block_in_place"

# Store raw text content
octobrain knowledge store "meeting-notes" --content "Discussion points..."

# List indexed sources
octobrain knowledge list --limit 20

# Show statistics
octobrain knowledge stats

# Delete a source
octobrain knowledge delete https://example.com/docs

# Delete stored content by key
octobrain knowledge delete-stored "meeting-notes"
```

### Knowledge Boxes

Import and sync git-backed knowledge bundles scoped to your projects:

```bash
# Import a remote git repo as a knowledge box
octobrain box import https://github.com/org/docs-repo.git

# Import at global scope (visible in every project)
octobrain box import https://github.com/org/docs-repo.git --global

# Pull and re-index all subscribed boxes + local .box/ directories
octobrain box sync

# List subscribed boxes
octobrain box list

# Remove a box
octobrain box remove github.com/org/docs-repo
```

### MCP Server

Run as an MCP server for integration with Claude Desktop and other AI tools:

```bash
# Start with stdio transport (for Claude Desktop)
octobrain mcp

# Start with HTTP transport (for web-based tools)
octobrain mcp --bind 0.0.0.0:12345
```

**Available MCP Tools:**

| Tool | Description |
|------|-------------|
| `memorize` | Store memories with metadata; optional `related_to` for inline relationships |
| `remember` | Semantic search with filters; returns 1-hop graph neighbors |
| `forget` | Delete memories (requires confirmation) |
| `knowledge` | Unified tool: `search`, `store`, `delete`, `read`, `match` via `command` field |
See [MCP Integration](#mcp-integration) for Claude Desktop setup.

## Features

- **Semantic Search** — Find memories by meaning using vector embeddings, not exact keyword matches
- **Hybrid Search** — Combines BM25 full-text search with vector similarity for better results
- **Reranking Support** — Pluggable cross-encoder reranking stage (provider-dependent)
- **Auto-Linking** — Automatically connects semantically similar memories (Zettelkasten-style)
- **Temporal Decay** — Ebbinghaus forgetting curve for importance management
- **Knowledge Indexing** — Ingest URLs, PDFs, docs for retrieval
- **Project Scoping** — Isolate memories per Git project or share across projects
- **Role Filtering** — Tag memories by role (developer, reviewer, etc.)
- **Query Expansion (HyDE-lite)** — Pseudo-relevance feedback for +10-30% recall on long-tail queries
- **MCP Protocol** — Full MCP 2026-07-28 compliance for AI tool integration

## Benchmarks

Retrieval quality of octobrain's knowledge system on standard [BEIR](https://github.com/beir-cellar/beir) datasets — **nDCG@10**, fully local, no LLM judge, using the harness's default embedder `bge-small-en-v1.5` (384-dim, 33M params). Each corpus passage is indexed through octobrain's real retrieval path and scored against the official qrels (metrics reproduce `pytrec_eval`).

| Dataset | octobrain vector | octobrain hybrid | BM25¹ | bge-small-en-v1.5² |
|---|---|---|---|---|
| **SciFact** (5.2K docs, 300 q) | 0.722 | **0.742** | 0.665 | 0.713 |
| **NFCorpus** (3.6K docs, 323 q) | 0.341 | **0.363** | 0.325 | 0.343 |

- **vector** = dense-only retrieval; reproduces the embedder's published BEIR numbers (validates the harness).
- **hybrid** = BM25 + vector fused with Reciprocal Rank Fusion (k=60) — octobrain's default. Adds **+2 nDCG@10** over the bare embedding and beats classic BM25 on both datasets.

¹ Canonical BM25 from the [BEIR paper](https://arxiv.org/abs/2104.08663) (Anserini/Lucene, k1=0.9 b=0.4).
² From the [bge-small-en-v1.5 model card](https://huggingface.co/BAAI/bge-small-en-v1.5) (MTEB).

> Scope: this measures the **ranking** layer (embedding + BM25 fusion + reranking). BEIR passages are pre-chunked, so octobrain's chunking strategy is not exercised here.

> ⚠️ **Reranker status:** The cross-encoder reranker is currently a no-op with fastembed models (byte-identical results to hybrid-only). Rerank is excluded from the numbers above pending investigation. See [`benches/README.md`](benches/README.md#findings) for details.

Reproduce (downloads the datasets, builds a release binary, runs fully offline):

```bash
cd benches && bash scripts/run_retrieval.sh
```

## Configuration

Configuration is stored in `~/.local/share/octobrain/config.toml`. The template written on first run defines every option with sensible defaults, but loading is strict — a field you delete by hand causes startup to fail rather than falling back to a default.

Override the config location with `OCTOBRAIN_CONFIG_PATH=/path/to/config.toml`.

### Key Settings

| Section | Option | Default | Description |
|---------|--------|---------|-------------|
| `[embedding]` | `model` | `fastembed:Qdrant/all-MiniLM-L6-v2-onnx` | Embedding model (provider:model format). Default is a local fastembed model — no API key, runs on CPU. |
| `[search]` | `similarity_threshold` | `0.3` | Minimum relevance (0.0-1.0) |
| `[search.hybrid]` | `enabled` | `true` | Enable BM25 + vector fusion |
| `[search.reranker]` | `enabled` | `true` | Enable cross-encoder reranking |
| `[search.hyde]` | `enabled` | `true` | Pseudo-relevance feedback query expansion |
| `[memory]` | `max_memories` | `10000` | Maximum stored memories |
| `[memory]` | `auto_linking_enabled` | `true` | Auto-connect similar memories |
| `[knowledge]` | `chunk_size` | `1200` | Characters per chunk |
| `[embedding]` | `batch_size` | `32` | Texts embedded per batch |
| `[embedding]` | `timeout_secs` | `30` | Embedding call timeout (0 = no timeout) |
| `[search]` | `max_results` | `50` | Hard ceiling on results from any search |
| `[search.reranker]` | `model` | `fastembed:jina-reranker-v2-base-multilingual` | Cross-encoder model |
| `[search.reranker]` | `top_k_candidates` | `50` | Candidates retrieved before reranking |
| `[search.hyde]` | `top_k` | `3` | Neighbors averaged for the centroid |
| `[search.hyde]` | `alpha` | `0.5` | Blend weight on the original query embedding |
| `[memory]` | `max_search_results` | `50` | Default page size when no limit is given |
| `[memory]` | `sleep_consolidation_enabled` | `true` | Lazy sleep consolidation on manager init |
| `[memory]` | `sleep_consolidation_interval_hours` | `24` | Hours between sleep-consolidation passes |
| `[knowledge]` | `outdating_days` | `15` | Days before indexed content is reindexed on search |
| `[knowledge]` | `max_results` | `5` | Results returned from knowledge search |

### Embedding Providers

```toml
[embedding]
# Local models (no API key, runs on CPU, model auto-downloaded on first use)
model = "fastembed:Qdrant/all-MiniLM-L6-v2-onnx"                       # Default: 384-dim quantized, ~125MB resident
model = "fastembed:nomic-ai/nomic-embed-text-v1.5"                      # 768-dim, 8192-ctx, ~550MB weights, highest quality
model = "fastembed:BAAI/bge-small-en-v1.5"                              # 384-dim, ~62 MTEB, fast + good quality
model = "fastembed:sentence-transformers/all-MiniLM-L6-v2-quantized"  # Smallest (~22MB), fastest
model = "fastembed:BAAI/bge-base-en-v1.5"                              # Larger (~440MB), higher quality
model = "fastembed:intfloat/multilingual-e5-small"                     # Multilingual

# Cloud providers (require API keys, generally higher quality)
model = "voyage:voyage-3.5-lite"          # VOYAGE_API_KEY
model = "openai:text-embedding-3-small"   # OPENAI_API_KEY
model = "google:text-embedding-004"       # GOOGLE_API_KEY
model = "jina:jina-embeddings-v3"         # JINA_API_KEY
```

### Full Configuration

See [`config-templates/default.toml`](config-templates/default.toml) for all available options with documentation.

## Memory Types

Organize memories by category for better filtering:

| Type | Use For |
|------|---------|
| `code` | Code patterns, solutions, implementations |
| `architecture` | System design, decisions, patterns |
| `bug_fix` | Bug fixes, troubleshooting, solutions |
| `feature` | Feature specs, implementations |
| `documentation` | Docs, explanations, knowledge |
| `user_preference` | Settings, preferences, workflows |
| `decision` | Project decisions, trade-offs |
| `learning` | Tutorials, notes, education |
| `configuration` | Setup, config, deployment |
| `testing` | Test strategies, QA insights |
| `performance` | Optimizations, benchmarks |
| `security` | Vulnerabilities, fixes, considerations |
| `validation` | Idea/product validation, hypothesis testing |
| `research` | Technical/market research, analysis |
| `workflow` | SOPs, playbooks, process descriptions |
| `requirement` | Business requirements, specs, constraints |
| `design` | UI/UX decisions, wireframes, system design |
| `integration` | API integrations, third-party services |
| `communication` | Stakeholder updates, team decisions |
| `process` | Deployment procedures, runbooks, operations |
| `insight` | General insights, tips |
| `goal` | Task/intent anchors for consolidation workflow |

## MCP Integration

### Claude Desktop Setup

Add to your Claude Desktop config (`~/Library/Application Support/Claude/claude_desktop_config.json` on macOS):

```json
{
  "mcpServers": {
    "octobrain": {
      "command": "/path/to/octobrain",
      "args": ["mcp"]
    }
  }
}
```

Restart Claude Desktop. Octobrain tools will be available in your conversations.

### HTTP Transport

For web-based integrations:

```bash
octobrain mcp --bind 0.0.0.0:12345
```

The server exposes endpoints at `/mcp` for MCP protocol communication.

## Storage Locations

Data is stored in platform-specific directories:

| Platform | Location |
|----------|----------|
| macOS | `~/.local/share/octobrain/` |
| Linux | `~/.local/share/octobrain/` or `$XDG_DATA_HOME/octobrain/` |
| Windows | `%APPDATA%\octobrain\` |

Project-specific memories are isolated by normalized Git remote URL (e.g., `github.com/org/repo`).

## Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Run `cargo fmt --all` to format code
4. Run `cargo clippy` and fix all warnings
5. Run `cargo test --no-default-features`
6. Submit a pull request

### Development Setup

```bash
# Clone and build
git clone https://github.com/muvon/octobrain.git
cd octobrain
cargo build --no-default-features

# Run tests
cargo test --no-default-features

# Run clippy
cargo clippy --no-default-features
```

## License

Apache-2.0 — see [LICENSE](LICENSE) for details.

## Credits

Developed by [Muvon Un Limited](https://muvon.io).