# kb-jev

A Markdown knowledge base with Jev classifications you can inspect and correct.

Import a document, classify its subject, review uncertain results, and search the original text. Sources keep their exact bytes. Each decision records the source hash, taxonomy, question, model, probabilities, token usage when reported, and elapsed time.

**Status: early working prototype.** TypeSafe direct and Vercel AI Gateway use the same TypeSafe-compatible HTTP client. A configurable endpoint supports other services that implement that contract. The included tests and demo run offline. Live provider quality and a Vercel deployment have not been verified for this release.

The toolkit is MIT licensed. Jev itself is a hosted model. Vercel AI Gateway is another way to access TypeSafe's model, not an independent replacement for it.

## Try it without a key

Requires Node.js 22 or newer.

```bash
git clone https://github.com/robebots/kb-jev.git
cd kb-jev
npm ci --include=dev --ignore-scripts
npm run demo
```

The demo creates a temporary KB, imports a sample runbook, classifies it with a clearly marked keyword fixture, demonstrates a cache hit, records a sample human correction, and checks source integrity and links. Its output is not evidence of Jev accuracy or speed.

## Use your own sources

```bash
npm run build
npm run kb -- init --root ./my-kb
npm run kb -- ingest ./examples/runbook.md --root ./my-kb

# Set TYPESAFE_API_KEY in your environment, or in a gitignored .env.
npm run kb -- classify --root ./my-kb --provider typesafe
npm run kb -- review --root ./my-kb
npm run kb -- search 'database migration' --root ./my-kb
npm run kb -- doctor --root ./my-kb
```

To use Vercel AI Gateway, set `AI_GATEWAY_API_KEY` or `VERCEL_OIDC_TOKEN` and pass `--provider vercel`. See [provider configuration](docs/providers.md) for model IDs, compatible services, and the optional Vercel API.

Edit `.kb/taxonomy.json` to define your categories. One Choice selects one primary subject. `other`, low probability, and a small gap between the top two choices require review. The starter thresholds are policy defaults, not calibrated results for your documents.

To record a correction, use the full source ID from `review`:

```bash
npm run kb -- review SOURCE_ID engineering --note 'Read the source and confirmed the subject.' --root ./my-kb
```

## What lives on disk

```text
.kb/
  taxonomy.json       Categories, rubric descriptions, and review thresholds
  manifest.json       Source records and pointers to current decisions
  raw/<sha256>.md     Original Markdown or UTF-8 text, never rewritten
  decisions/         Immutable classification records
  reviews/           Explicit corrections tied to individual decisions
  wiki/sources/      Generated source records with provenance links
  wiki/articles/     Articles written by you or an agent
  wiki/TOC.md        Compact source counts
  wiki/INDEX.md      Complete source index
  output/            Current ingestion plan for an agent
  log.jsonl          Append-only operation history
```

Open `.kb` as an Obsidian vault. Generated pages use links relative to that vault root. `compile` rebuilds the source pages and indexes while preserving `wiki/articles/`. Search currently checks raw text with lexical matching, including unclassified and review items.

## Research and build notes

- [Who built similar projects](docs/related-work.md), including pi-jev-wiki, DocJev, and a Jev/Sanity KB router.
- [Audit of the old KB skill](docs/legacy-skill-audit.md).
- [Architecture and current limits](docs/architecture.md).
- [Verifiable benchmarks and comparison protocol](docs/benchmarks.md).
- [Build log and validation evidence](docs/build-log.md).
- [LinkedIn draft and recording outline](content/linkedin.md).
- [Portable agent skill](skills/kb-jev/SKILL.md).

## Measure what Jev changes

```bash
npm test
npm run benchmark -- run --dataset benchmarks/datasets/synthetic-v1.json --output benchmarks/runs/offline-001
npm run benchmark -- verify benchmarks/runs/offline-001 --dataset benchmarks/datasets/synthetic-v1.json
```

The benchmark pairs each candidate prediction with development-fitted majority and keyword baselines. It reports accuracy, macro F1, accepted accuracy, coverage, wrong automatic accepts, review rate, failures, latency, usage, and per-document wins and regressions. It can import recorded results from your previous workflow. Full inputs and outputs are saved for replay; unknown costs stay unknown.

CI verifies a synthetic run and retains its raw evidence as a downloadable artifact. These offline results test the benchmark tools, not Jev quality. Use a frozen, independently labeled corpus and live calls before publishing model claims. The [protocol](docs/benchmarks.md) covers TypeSafe, Vercel, compatible endpoints, costs, old-workflow imports, and evidence limits. The earlier `npm run evaluate` command remains a 12-case smoke test.

Search currently ignores classification when ranking results. A test verifies that correct labels, wrong labels, and provider failures preserve the same rankings and source excerpts. There is no measured retrieval improvement from Jev in this version.