# Kindex — Knowledge Graph for AI-Assisted Workflows

Kindex is a persistent knowledge graph that indexes conversations, projects, and intellectual work so that Codex never starts a session blind. The CLI is `kin`.

## Running Tests

Install Python test dependencies (`pip install -e ".[dev,mcp,vectors]"`) and Node.js 24 or newer.
Node executes the modern Claude adapter contract tests; it is not a Kindex package dependency.

```bash
# All tests
pytest tests/ -v

# Specific module
pytest tests/test_cli.py -v
pytest tests/test_hooks.py -v
pytest tests/test_store.py -v

# With coverage
pytest tests/ --cov=kindex --cov-report=term-missing
```

The `CI` workflow runs full `pytest` with Python 3.12, Node.js 24, and `.[dev,mcp,vectors]` on pull
requests and pushes to `main`. It also supports manual dispatch and reusable
workflow calls. `Publish to PyPI` runs only on `v*` tag pushes and invokes `CI`
before building and publishing. The `test` check must be required in branch
protection to prevent merging a failing suite.

## Key File Locations

| Path | Purpose |
|------|---------|
| `src/kindex/cli.py` | CLI entry point and all `kin` subcommands |
| `src/kindex/dream.py` | Dream cycle: fuzzy dedup, suggestion auto-apply, domain edges |
| `src/kindex/referent.py` | R0 referent binding: digest validation, staleness sweep, rebind |
| `src/kindex/snapshots.py` | Pre-merge SQLite snapshots (rotating, fail-closed) to XDG state dir |
| `src/kindex/dream_deep.py` | LLM-powered cluster consolidation (--deep mode only) |
| `src/kindex/store.py` | SQLite + FTS5 storage layer |
| `src/kindex/config.py` | Configuration loading and `Config` model |
| `src/kindex/graph.py` | Graph analytics (centrality, communities, bridges) |
| `src/kindex/retrieve.py` | Hybrid search (FTS5 + graph BFS + RRF merge) |
| `src/kindex/extract.py` | Knowledge extraction from text and sessions |
| `src/kindex/ingest.py` | Project and session ingestion pipeline |
| `src/kindex/sessions.py` | Session tag lifecycle (start, update, segment, resume) |
| `src/kindex/coordination.py` | Collab conversations (members, cursors, inject messages, resources) |
| `src/kindex/locks.py` | Advisory node locks (acquire, release, expiry sweep) |
| `src/kindex/hooks.py` | Claude Code hook handlers (prime, compact) |
| `src/kindex/setup.py` | System setup (Codex hooks, launchd, crontab, `.kin` git merge driver) |
| `src/kindex/kin_merge.py` | Structured 3-way union merge for `.kin/index.json` + `.kin/code-map.json` (`kin merge-kin` git driver) |
| `src/kindex/daemon.py` | Background daemon and cron cycle |
| `src/kindex/budget.py` | LLM spend tracking and budget enforcement |
| `src/kindex/vectors.py` | Vector embedding, contextual chunk indexing, reindex maintenance, and similarity search |
| `src/kindex/vault.py` | Markdown vault import/export |
| `src/kindex/models.py` | Core data models |
| `src/kindex/schema.py` | Database schema and migrations |
| `src/kindex/llm.py` | LLM provider abstraction |
| `tests/` | All tests (pytest) |
| `~/.kindex/` | Default data directory (SQLite DB, logs) |
| `~/.config/kindex/kin.yaml` | User config |
| `.kin/config` | Git-tracked project context and optional work policy |
| `.kin/index.json` | Git-tracked project graph snapshot generated by `kin index` |
| `.kin/code-map.json` | Git-tracked repo code map generated by `kin export code-map` |
| `.gitattributes` | Points `.kin/index.json` + `.kin/code-map.json` at the `merge=kindex` driver (auto-written by `kin index`) |

## Session Directives

When working in this codebase, follow these practices:

- **Capture new knowledge**: When you learn something significant, use `kin add "<insight>" --tags <domain1>,<domain2>` to persist it with tags.
- **Create links**: When you discover a relationship between concepts, use `kin link <a> <b> <relationship> --why "<reason>"` to connect them.
- **Track decisions**: Use `kin add "<decision>" --type decision --tags <project>` for architectural or design decisions.
- **Record constraints**: Use `kin add "<rule>" --type constraint --trigger <event> --action <verify|warn|block>` for invariants.
- **Flag attention items**: Use `kin add "<item>" --type watch --owner <person> --expires <date>` for things that need monitoring.
- **Search before adding**: Use `kin search <term>` or `kin search <term> --tags <domain>` to check if knowledge already exists before duplicating.
- **Bind claims about code**: `kin add "<claim>" --referent <path>` hashes the referent so `kin stale` can later demote claims whose referent moved (re-verify with `kin stale --rebind <id>`).
- **Edit, don't re-add**: Use `kin edit <id-or-title> --content "..."` to correct an existing node instead of creating a near duplicate. Additive types (decision, constraint, directive, checkpoint, watch) only accept `--append` and `--expires`; use `kin supersede <id> "<new text>" --reason "..."` to replace them with history. `kin changelog` shows per-field diffs.
- **Coordinate multi-agent work**: Use `kin coord join <name>` to become a member (unread tracking), `kin coord attach <name> <node>` for shared resources, `kin coord inject <name> set "<msg>" [--to <agent>]` for standing messages, and `kin lock <id> --ttl <min> --note "<why>"` / `kin unlock <id>` for advisory locks before editing contested nodes.
- **Profiles**: When `profiles:` are configured in `~/.config/kindex/kin.yaml`, the active graph resolves by `--profile` flag > `KIN_PROFILE` env > `.kin` chain `profile:` key > cwd roots match > `default_profile`. `kin profile which` shows the resolution; `kin whoami` shows the agent identity used for locks/claims/collabs.
- **Filter by tags**: Use `kin list --tags <tag1>,<tag2>` to find nodes by tag (AND logic). Tags supplement auto-categorization.
- **Check status**: Use `kin status` to see graph health and active operational nodes.
- **Session tags**: Use `kin tag start <name> --focus "what you're working on"` to create a named session handle. Update with `kin tag update <name> --focus "..."` or `kin tag segment <name> --focus "new topic" --summary "what happened"`. Resume in a new session with `kin tag resume <name>`. End with `kin tag end <name> --summary "..."`. See `kin tag list` for all tags.
- **Project policy**: `.kin/config` ships with the repo. Run `kin policy check --event agent-start` before meaningful code work when shell access is available. Linear is opt-in only; enforce it only when `work_policy.linear.enabled` is true in the project config.

## Project knowledge preservation

Kindex databases hold durable state and may be its only complete copy. Capture
and update knowledge through the supported Kindex tools in the selected graph.
Do not delete a Kindex database, including an ignored worktree database, as a
cache or assume tracked JSON/JSONL files can reconstruct it.

- Stage and commit matching `.kin/config` and `.kin/index.json` changes with code.
- Use `kin repo-memory publish` for explicitly selected active public/team
  concepts, decisions, and questions. It is an evidence transport, not a full
  database backup. Keep private knowledge, credentials, and local state out of Git.
- Preserve both knowledge artifacts if JSON and JSONL coexist. Publish/import
  refuse that layout until their records are explicitly reconciled into one file.
- Preserve databases and relevant source records during migration or worktree
  cleanup. Report unsupported fields, missing evidence, and unresolved references;
  neither a summary index nor selected exports prove complete recovery.

The planned canonical JSONL architecture and its unimplemented recovery
requirements are documented in
[canonical-knowledge-migration.md](docs/canonical-knowledge-migration.md).

## Auto-Context Loading

At the start of each session, load relevant context:

```bash
kin prime --for hook
```

This auto-detects the current project from `$PWD` and outputs a context block with relevant knowledge, active constraints, and recent changes.

## Release Checklist

When asked to release, follow these steps exactly. Do NOT install twine or attempt manual PyPI upload — it's fully automated.

1. Run full test suite: `pytest tests/ -v`
2. Bump `version` in `pyproject.toml` and add a `## [X.Y.Z] - YYYY-MM-DD` entry to `CHANGELOG.md`. The pre-commit hook (`scripts/sync-version.sh`) syncs `src/kindex/__init__.py`, the README badge, `docs/index.html`, `server.json`, `docs/.well-known/mcp/server-card.json` and both plugin manifests from it; `tests/test_release_metadata.py` fails CI if any surface disagrees. If `git config core.hooksPath` is set globally (it is on Jeremy's machine), the repo hook never runs: run `bash scripts/sync-version.sh` by hand before committing, and run `pytest tests/test_release_metadata.py` before tagging
3. Commit version bump and all changes
4. Push to main: `git push origin main`
5. Tag: `git tag vX.Y.Z && git push origin vX.Y.Z`
6. Create GitHub release: `gh release create vX.Y.Z --title "..." --notes "..."`
7. Watch the workflow: `gh run watch <id> -R wandercom/kindex` -- every job (test, build, publish, mcp-registry) must pass
8. Verify on PyPI: `pip index versions kindex 2>/dev/null | head -1` or check https://pypi.org/project/kindex/
9. The release publishes the server entry to the MCP Registry after PyPI succeeds (`.github/workflows/publish-mcp.yml`, OIDC, no token; it refuses to publish a version PyPI does not have, and `gh workflow run publish-mcp.yml -f tag=vX.Y.Z -R wandercom/kindex` backfills a release tagged before this existed). The workflow confirms the registry itself, but to check by hand -- note that `search` matches substrings across every server and every historical version, so `io.github.jmcentire/kindex` and old releases come back too; match the exact name and the registry's `isLatest` flag rather than a position in the list:

   ```bash
   curl -s "https://registry.modelcontextprotocol.io/v0/servers?search=io.github.wandercom/kindex&version=latest" \
     | python3 -c "import json,sys; n='io.github.wandercom/kindex'; e=[x['server'] for x in json.load(sys.stdin)['servers'] if x['server']['name']==n and x['_meta']['io.modelcontextprotocol.registry/official']['isLatest']]; print(e[0]['version'] if e else 'MISSING')"
   ```

   The registry is how MCP clients discover the server.
10. Verify the MCP listing metadata is current: `server.json` version/package fields match the release, and https://mcpmarket.com/server/kindex reflects the published package after indexing.

**Definition of done:** The release is complete when (a) all workflow jobs are green, (b) the new version appears on PyPI, (c) `pip install kindex==X.Y.Z` succeeds, and (d) MCP/server metadata has been updated or a marketplace refresh has been requested. If any job fails, fix the issue, bump to a new patch version, and repeat from step 1 -- do not re-tag or force-push an existing tag.

## Architecture Notes

- Storage: SQLite with FTS5 full-text search, triggers for index sync
- Search: Hybrid retrieval combining FTS5 BM25 scores, graph BFS traversal, and optional vector similarity, merged via Reciprocal Rank Fusion (RRF)
- Embeddings: Voyage defaults to `voyage-context-4`; contextual chunking activates only for provider/model support. Use `kin embed plan/enqueue/drain/reindex/status` for scoped or gradual reindexing.
- Context tiers: full (~4000 tokens), abridged (~1500), summarized (~750), executive (~200), index (~100) -- auto-selected by budget
- Node types: concept, document, session, person, project, decision, question, artifact, skill, constraint, directive, checkpoint, watch
- Edge types: relates_to, depends_on, derived_from, contradicts, etc. with weights and provenance
- Node mutability classes (enforced by `kin edit` / MCP `edit`, overridable via config `edit_policy`):

| Class | Node types | Allowed edits |
|-------|-----------|---------------|
| editable | concept, document, artifact, skill, person, project, question | Full in-place: title, content, append, tags, intent, expires |
| additive | decision, constraint, directive, checkpoint, watch | append + expires only; replace via `kin supersede` |
| managed | task, session, coordination | Refused — use `kin task` / `kin tag` / `kin coord` |
- Audience: private / team / public scoping with export boundary enforcement
- Weight decay: Nodes and edges naturally fade unless accessed, keeping the graph fresh
- Generated `.kin` snapshots are not complete canonical knowledge or lossless conflict archives. `kin merge-kin` selects same-ID index conflicts by timestamp (ties keep ours); code-map collisions can keep ours. Preserve both input snapshots and relevant knowledge before regenerating with `kin index` / `kin export code-map` from the selected graph and merged code; an incomplete local graph must not overwrite another branch's knowledge. Do not hand-edit generated snapshots. `kin index` auto-registers the driver; `kin setup-merge` (re)installs it per clone.