Gnosis MCP

Stop pasting files into context. Your AI agent searches your local docs instead.
5–10× fewer tokens per lookup. 92 % Hit@5 on real dev docs. Zero cloud dependencies.

PyPI Downloads Python MIT License CI

Quick Start · Documentation · Tools · Configuration · Full Reference

Gnosis MCP — ingest docs, search, view stats, serve
Ingest docs → Search with highlights → Stats overview → Serve to AI agents
--- ### Without a docs server - LLMs hallucinate API signatures that don't exist - Entire files dumped into context — 3,000–15,000 tokens per doc - Architecture decisions buried across dozens of files - Every repeated lookup pays full context cost ### With Gnosis MCP - `search_docs` returns ranked, highlighted excerpts — typically 300–800 tokens - Real answers grounded in your actual docs, not guesses from training data - One local index across hundreds of files — instant multi-doc search - **5–10× token savings** per lookup when your corpus covers the question --- ## What makes gnosis-mcp different - **Your data stays on your machine.** SQLite by default, PostgreSQL at scale — nothing leaves the host. - **Index anything that's docs-shaped.** Markdown, git commit history, crawled websites — one index, one search API. - **Measured, not marketed.** Ships BEIR SciFact numbers (0.671 nDCG@10 — within 1 % of the Lucene BM25 baseline), a reproducible eval harness (`gnosis-mcp eval`), and a chunk-size sweep showing where the quality plateau actually sits. Full side-by-side vs Context7 / docs-mcp-server / mcp-local-rag: [gnosismcp.com#compare](https://gnosismcp.com/#compare). --- ## Features - **Zero config** — SQLite by default, `pip install` and go - **Hybrid search** — keyword (BM25) + semantic (local ONNX embeddings, no API key). Tune RRF fusion with `GNOSIS_MCP_RRF_K`. - **Cross-encoder reranking** — optional `[reranking]` extra with a 22M-param ONNX model. Off by default. **[Test on your own corpus before enabling](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/bench-experiments-2026-04-18.md)** — the bundled MS-MARCO reranker hurts dev-doc retrieval in our measurements. - **Git history** — ingest commit messages as searchable context (`ingest-git`) - **Web crawl** — ingest documentation from any website via sitemap or link crawl - **Multi-format** — `.md` `.txt` `.ipynb` `.toml` `.csv` `.json` + optional `.rst` `.pdf` - **Auto-linking** — `relates_to` frontmatter creates a navigable document graph - **Watch mode** — auto-re-ingest on file changes - **Prune stale docs** — `gnosis-mcp ingest --prune` removes chunks whose source file was deleted. `--wipe` for a full reset before re-ingest. - **Built-in eval harness** — `gnosis-mcp eval` prints Hit@K / MRR / Precision@K in one command, against a bundled fixed fixture set - **PostgreSQL ready** — pgvector + tsvector when you need scale ## Performance **Fast.** 8.7 ms mean MCP round-trip. Hybrid search p50 < 30 ms on a 700-doc corpus. Keyword QPS scales from 9,463 @ 100 docs to 471 @ 10,000 docs ([full numbers](https://gnosismcp.com/#numbers)). **Finds the right answer.** On 558 real dev docs with 25 hand-written golden queries: Hit@5 = **0.92**, nDCG@10 = **0.87**, MRR = **0.79**. On BEIR SciFact (5,183 docs, public retrieval benchmark): nDCG@10 = **0.671** — within 1 % of the Lucene BM25 baseline. **Tokens saved.** Each `search_docs` call returns 200–500 tokens of on-point snippets instead of the 3,000–15,000 tokens a full-file Read would have cost. Track your own with `gnosis-mcp savings` (v0.12.0+) — the ledger writes to `search_access_log` on every call and aggregates per tool per `--days N`: ``` $ gnosis-mcp savings --days 7 Tool calls: 142 Tokens returned: 7,104 Tokens baseline: 231,580 Tokens saved: 224,476 Ratio: 32.6× ``` Typical compression runs 10–60× depending on corpus coverage and query specificity — verify on yours. `access_log` is on by default; `GNOSIS_MCP_ACCESS_LOG=false` opts out. The same ledger answers the other question — **what is this corpus being asked, and where does it come up empty** — with `gnosis-mcp usage` (v0.17.4+): calls, the queries that matched nothing, which documents are served most, which have never been served, and which clients are doing the asking. **Reproducible.** `gnosis-mcp eval` runs a bundled retrieval-quality harness locally in one second — but it ingests nine hardcoded sample documents into a temporary database and answers ten bundled queries, so it returns the same numbers for every corpus and is a smoke test, not a measurement of your docs. To score your own corpus, run `python tests/bench/bench_real_corpus.py --corpus --golden ` (the numbers above come from `tests/bench/golden-knowledge.jsonl`). `tests/bench/*.py` reproduce every number. Methodology: [`docs/benchmarks.md`](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/benchmarks.md). **Rerankers stay off by default.** The bundled MS-MARCO cross-encoder drops nDCG@10 by 27 points on dev-docs and adds 400× latency; BGE-reranker-v2-m3 drops it 31 points at 2400×. Test on your corpus before enabling — full write-up: [bench-experiments-2026-04-18](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/bench-experiments-2026-04-18.md). ## Quick Start ```bash pip install gnosis-mcp # or: uv tool install gnosis-mcp gnosis-mcp ingest ./docs/ # loads docs into SQLite (auto-created) gnosis-mcp serve # starts MCP server ``` That's it. Your AI agent can now search your docs. **Connect your client** — see [`llms-install.md`](https://github.com/nicholasglazer/gnosis-mcp/blob/main/llms-install.md) for copy-paste JSON snippets for Claude Code, Claude Desktop, Cursor, Zed, opencode, Windsurf, VS Code, JetBrains, Cline, and any other MCP client. **Re-organized your docs?** `gnosis-mcp ingest ./docs --prune` re-ingests and removes any DB chunk whose source file no longer exists. `--wipe` resets the entire index first. Or run `gnosis-mcp prune ./docs --dry-run` to preview what would be deleted. Pruning only touches documents this root is responsible for: crawled URLs and generated documents (git history) are left alone unless `--include-crawled` / `--include-generated` says otherwise. **Want semantic search?** Add local embeddings — no API key needed: ```bash pip install gnosis-mcp[embeddings] gnosis-mcp ingest ./docs/ --embed # ingest + embed in one step gnosis-mcp serve # hybrid search auto-activated ``` Test it before connecting to an editor: ```bash gnosis-mcp check # FTS5 + schema + row counts; non-zero exit = stop gnosis-mcp search "getting started" # keyword search gnosis-mcp search "how does auth work" --embed # hybrid semantic+keyword gnosis-mcp stats # see what was indexed ``` Then wire it into the clients you actually use — one command, no paths to edit: ```bash gnosis-mcp setup # preview what each client's config would become gnosis-mcp setup --write # Claude Code, DeepSeek Harness, Codex, Cursor, VS Code, … gnosis-mcp doctor # is it wired, and has anything called it yet? ``` `setup` resolves the command path for the machine it runs on instead of the one the README assumed, and each block it installs is marker-delimited so re-running rewrites it in place. Where a client does not read the server's own MCP `instructions`, it also installs the short rule that gives the agent a reason to prefer gnosis over reading files — a mounted server nobody calls is the silent failure mode. `doctor` is the check for that: it reads the access log and tells you whether a client has really called the server. Details: [`docs/cli.md`](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/cli.md#setup). `gnosis-mcp check` is the gate: it exits `0` only when the backend started and the schema the server needs is present, and it names whatever is missing before exiting `1`. Keyword search needs SQLite **FTS5**, which is compiled into your Python's SQLite rather than guaranteed by it — `check` reports it as `FTS5: ready`. Anything else: [`docs/troubleshooting.md`](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/troubleshooting.md).
Run with Docker (zero install) Multi-arch image, ~140 MB, ships with local ONNX embeddings + REST: ```bash # Serve your ./docs on http://localhost:8000 — MCP at /mcp, REST at /api/* docker run -p 8000:8000 \ -v "$PWD/docs:/docs:ro" -v gnosis-data:/data \ ghcr.io/nicholasglazer/gnosis-mcp:latest # First-run: ingest into the persistent volume docker run --rm \ -v "$PWD/docs:/docs:ro" -v gnosis-data:/data \ ghcr.io/nicholasglazer/gnosis-mcp:latest \ ingest /docs --embed ``` Or use the committed [`docker-compose.yaml`](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docker-compose.yaml): ```bash docker compose up -d docker compose exec gnosis gnosis-mcp ingest /docs --embed ``` Images tagged `:latest`, `:`, `:`, `:main`, `:sha-`. Production recipes — systemd, reverse proxy, security checklist, upgrades — are in [`docs/deployment.md`](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/deployment.md).
Try without installing (uvx) ```bash uvx gnosis-mcp ingest ./docs/ uvx gnosis-mcp serve ```
## Documentation Canonical index — every topic is one hop away: **[`docs/overview.md`](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/overview.md)**. The handful most readers need: - [**Install & client setup**](https://github.com/nicholasglazer/gnosis-mcp/blob/main/llms-install.md) — every editor's copy-paste config, Windows paths, write mode, embeddings - [**CLI**](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/cli.md) — every subcommand and flag - [**Configuration**](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/config.md) — all `GNOSIS_MCP_*` env vars - [**MCP tools & resources**](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/tools.md) — what your agent can call, and what each returns - [**Deployment**](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/deployment.md) — Docker, systemd, reverse proxy, security checklist - [**Troubleshooting**](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/troubleshooting.md) — first-run failures and how to recover - [**Benchmarks**](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/benchmarks.md) — reproducible retrieval and latency numbers Also in `docs/`: [REST API](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/rest-api.md) · [embeddings service](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/embeddings-service.md) · [how we measure search](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/how-we-measure-search.md) · [reranker experiments](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/bench-experiments-2026-04-18.md) · [releasing](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/releasing.md). ## Tools Gnosis MCP exposes nine tools and three resources over [MCP](https://modelcontextprotocol.io/). Your AI agent calls these automatically when it needs information from your docs: | Tool | What it does | Mode | | -------------------- | ------------------------------------------------------------------ | ----- | | `search_docs` | Search by keyword or hybrid semantic+keyword | Read | | `get_doc` | Retrieve a full document by path | Read | | `get_related` | Find linked/related documents (multi-hop, relation type filtering) | Read | | `search_git_history` | Search indexed git commit history | Read | | `get_context` | Usage-weighted context summary | Read | | `get_graph_stats` | Knowledge graph topology: orphans, hubs, relation distribution | Read | | `upsert_doc` | Create or replace a document | Write | | `delete_doc` | Remove a document and its chunks | Write | | `update_metadata` | Change title, category, tags | Write | The six read tools are always advertised. The three write tools require `GNOSIS_MCP_WRITABLE=true` — without it they are withdrawn from `tools/list` entirely, so a read-only client is never handed a tool it cannot call. | Resource URI | Returns | | ---------------------- | -------------------------------------------------- | | `gnosis://docs` | All documents — path, title, category, chunk count | | `gnosis://docs/{path}` | Full document content | | `gnosis://categories` | Categories with document counts | With embeddings configured, `search_docs` fuses keyword and semantic results with Reciprocal Rank Fusion and returns a `highlight` field carrying the matched terms in `` tags — that is what keeps a lookup at a few hundred tokens instead of a full file. `get_context` is the session-start tool: it ranks documents by how often they are actually retrieved, unless `GNOSIS_MCP_ACCESS_LOG=false` turns tracking off. Full reference — every parameter, return shape, error, and the graph relation types (`related`, `content_link`, `git_co_change`, `git_ref`, plus the typed frontmatter edges): **[`docs/tools.md`](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/tools.md)**. ## Configuration Nothing required for SQLite — zero config works. Override via `GNOSIS_MCP_*` env vars. The four most-asked: | Variable | Default | Description | | --------------------------- | ----------- | ---------------------------------------------------------------------------------------------- | | `GNOSIS_MCP_DATABASE_URL` | SQLite auto | PostgreSQL URL (backend auto-detects) or SQLite file path | | `GNOSIS_MCP_WRITABLE` | `false` | Enable `upsert_doc` / `delete_doc` / `update_metadata` | | `GNOSIS_MCP_EMBED_PROVIDER` | unset | `local` turns on hybrid search (needs `[embeddings]` extra); or `openai` / `ollama` / `custom` | | `GNOSIS_MCP_API_KEY` | unset | Optional Bearer auth for every REST endpoint except `/health` | The remaining 45 — chunking, search limits, RRF, reranking, crawl, webhooks, column overrides, logging — are documented in **[`docs/config.md`](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/config.md)**. Backend selection is automatic: set `GNOSIS_MCP_DATABASE_URL` to a `postgresql://` URL and it uses PostgreSQL; leave it unset and it uses SQLite at `~/.local/share/gnosis-mcp/docs.db`. Override with `GNOSIS_MCP_BACKEND=sqlite|postgres`. PostgreSQL setup — `gnosis-mcp[postgres]`, `gnosis-mcp init-db`, `CREATE EXTENSION IF NOT EXISTS vector`, then `gnosis-mcp embed` — is in [`docs/overview.md`](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/overview.md). ### Embeddings Semantic search is the one optional add-on most setups want. Local ONNX is the recommended path: zero config, no API key, ~23 MB quantized model ([MongoDB/mdbr-leaf-ir](https://huggingface.co/MongoDB/mdbr-leaf-ir), Apache 2.0) auto-downloaded on first run. ```bash pip install gnosis-mcp[embeddings] gnosis-mcp ingest ./docs/ --embed # ingest + embed in one step gnosis-mcp embed # or backfill existing chunks separately ``` Remote providers work the same way via `gnosis-mcp embed --provider openai` (needs `GNOSIS_MCP_EMBED_API_KEY`) or `--provider ollama`. Model choices, dimensions, the OpenAI-compatible `POST /v1/embed` service, and pre-computed vectors for your own pipeline: [`docs/embeddings-service.md`](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/embeddings-service.md) · [`docs/config.md`](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/config.md#embeddings). ## Available On [MCP Registry](https://registry.modelcontextprotocol.io) (feeds VS Code MCP gallery and GitHub Copilot) · [PyPI](https://pypi.org/project/gnosis-mcp/) · [mcp.so](https://mcp.so) · [Glama](https://glama.ai) · [cursor.directory](https://cursor.directory) ## AI-Friendly Docs | File | Purpose | | ------------------------------------------------------------------------------------------- | -------------------------------------------- | | [`llms.txt`](https://github.com/nicholasglazer/gnosis-mcp/blob/main/llms.txt) | Quick overview — what it does, tools, config | | [`llms-full.txt`](https://github.com/nicholasglazer/gnosis-mcp/blob/main/llms-full.txt) | Complete reference in one file | | [`llms-install.md`](https://github.com/nicholasglazer/gnosis-mcp/blob/main/llms-install.md) | Step-by-step installation guide | ## Development ```bash git clone https://github.com/nicholasglazer/gnosis-mcp.git cd gnosis-mcp python -m venv .venv && source .venv/bin/activate pip install -e ".[dev]" pytest # 862 tests, no database needed ruff check src/ tests/ ``` All tests run without a database. Keep it that way. Good first contributions: new embedding providers, export formats, ingestion for new file types (via optional extras). Open an issue first for larger changes. ## Sponsors If Gnosis MCP saves you time, consider [sponsoring the project](https://github.com/sponsors/nicholasglazer). ## License [MIT](https://github.com/nicholasglazer/gnosis-mcp/blob/main/LICENSE) ## Learn more - [Documentation index](https://github.com/nicholasglazer/gnosis-mcp/blob/main/docs/overview.md) — the canonical map of every docs page - [Installation guide](https://github.com/nicholasglazer/gnosis-mcp/blob/main/llms-install.md) — copy-paste setup for every MCP client - [Published docs](https://gnosismcp.com/doc/) — the same docs, rendered for browsing - [PyPI](https://pypi.org/project/gnosis-mcp/) — releases, version history, install - [Why this exists](https://teru.sh/blog/gnosis-mcp-docs-in-60ms/) — the write-up behind gnosis-mcp