# Shared recall and connection checks XKB's intended behavior is proactive reuse during conversation. A user can reveal a need by stating a plan, describing a difficulty, adding a constraint, or changing direction. Question answering alone does not validate that behavior. ## Proactive delivery The suggestions are read by the agent answering the turn, not by the user. That agent already sees the whole conversation and is the strongest judge of whether a piece of knowledge applies. XKB therefore does only what the agent cannot: search the library and keep the injected context small. Per turn: up to three retrieval queries (the utterance, its bounded dialogue context and the last substantive sentence) run concurrently; Jev judges the merged candidates once against the utterance with its dialogue; `xkb_delivery` then applies local rules and calls no generation model: - verdict at least 0.5 (`DELIVERY_FLOOR`), at most three items; - reusable knowledge (cards, wiki) first. History, meaning daily notes and other sessions' conversation traces, records what happened and is often stale: it needs a verdict of at least 0.75 (`HISTORY_FLOOR`) and takes at most one slot left over by knowledge (`MAX_HISTORY`). On 2026-10-06, 27 of 33 suggestions while working on XKB itself were such history, months-old summaries that still recommended deleted scripts. Replaying the stored 10/5 and 10/6 turns under this rule cut XKB-development suggestions from 43 to 17 (13 knowledge) and gave video work 6 more wiki items, because traces no longer held slots; - nothing already suggested in this session since Claude Code last compacted it. The hook reads the last `compact_boundary` timestamp from the transcript (backward, matching the serialized field so a message that merely mentions compaction does not count) and sends it as `context_since`; without one the whole session is still in view. A turn-count window was tried in both directions and rejected: 10 turns re-sent suggestions still on screen, the whole session forever hid decisions lost to compaction. Delivered keys are read with SQLite's JSON functions, so there is no turn limit and no per-turn parsing of whole packets. Wiki sections, notes and traces also store a fingerprint of their normalized body, so an edited source can be suggested again. Cards do not: each retrieval returns whichever chunk matched, so their text is not a version. An unknown version on either side (cards, older turns) blocks; - none of the session's own conversation traces within the same visible range, excluded in the conversation query itself so that they cannot fill its 500-row window or result limit; decisions from before the last compaction remain recallable; - no exact duplicate text and nothing already quoted in a recent assistant reply; - display-only truncated snippets are never injected, and neither are card form fields: of a card's ten sections, "2. Claim 等級", "6. 與現有知識的關係" (a list of other card IDs) and "9. 原始來源" are metadata. Cards are retrieved section by section, and Jev scored such sections 0.57 and 0.82 on 2026-10-07. Explicit recall still returns them; - a title with no letters (bookmark cards titled by tweet ID), or a short prefix plus a hash such as `Xkb Case 6769...`, is replaced by the first sentence of the body; long bodies keep whole paragraphs and are labelled as excerpts. Delivery chooses from every candidate that passed ACL and per-layer quotas, not only from the response's `limit`: if the top ten were already suggested, the eleventh relevant candidate can still be delivered. `records` keeps its limit. A partial judgement still delivers its positive verdicts and marks the packet degraded; no judgement delivers nothing and marks it degraded. The hook header tells the agent to ignore unrelated items without mentioning them. All candidates remain inspectable in `records`. Presentation does not imply acceptance, and silence does not imply rejection. The hook skips only sessions explicitly marked unattended: interactive Claude Code sets `CLAUDE_CODE_SESSION_ATTENDED=1`, while `claude -p` scheduled agents set `0`. The entry point is not used as a guess: front ends that drive the CLI with stream-json also report `sdk-cli`. An older `claude -p` without the flag therefore recalls (one extra Jev call) rather than an interactive session being silently dropped. Skipped turns neither recall nor write a trace. Suggestions picked from beyond the response limit count as returned in usage accounting and `source_memory_ids`. When a turn completes, `delivery_outcomes` records, for each suggestion, the share of its excerpt terms that reappear in the answer. `python scripts/xkb_delivery_report.py --days 7` summarizes it. This is a lexical proxy: answers on the same topic share words, so read the distribution and the relatively unused repeated items, not a single threshold. ### Why the generation-model selector was removed (2026-10-06) From 2026-10-05 a separate model call (default `claude-sonnet-4-6` on the paid LLM gateway) first resolved the turn's information needs, and a second selected at most two paragraphs. Its fixed-evidence and live trials remain in the Git history of this file before 2026-10-06. In the first day of real use (26 turns) the whole recall took a median 12.8 s per turn, both calls were billed every turn, the same advice was injected three times in one session and the agent's own earlier replies came back as knowledge. Recomputing those 26 stored turns with Jev verdicts and the local rules above, at no provider cost: on substantive turns the Sonnet selection was almost always within Jev's top three; the repeats and self-echoes disappeared; what Sonnet added was staying quiet on turns such as "好 直接送", where the local rules instead deliver one to three items for the agent to ignore. That trade (a few hundred characters of context against two paid calls and several seconds on every turn) is why the selector was replaced rather than kept behind a switch. ## Retrieval breadth In shared recall with Jev enabled, Wiki retrieval now supplies count-bounded nearest candidates before relevance judging. Its legacy 0.65 cosine cutoff no longer removes natural-language candidates before Jev can see them. Legacy callers and `XKB_JEV_DECIDE=0` retain their existing cutoff. Shared judging can inspect two distinct sections per document, with up to 2,000 characters per excerpt. For leading Wiki documents the companion slot prefers the strongest explicit-term heading match after the first semantic section, retaining its real cosine score; legacy callers keep one section and 200 characters. Section identities prevent an introduction from erasing an actionable paragraph from the same document. Keyword fallback operates per missing layer, so a Wiki hit cannot suppress an exact card while card semantic retrieval is unavailable. Namespace checks and per-layer budgets still apply. Retrieval keeps the original utterance, its bounded dialogue context, and the last substantive sentence as at most three distinct queries. Each receives its own candidate budget, so closing an old topic cannot consume the new topic's entire retrieval budget. These read branches and their card/Wiki reads run concurrently; vector cache initialization is synchronized. All branch diagnostics are retained, and a failed branch degrades overall quality. Relevance is still judged against the original utterance with its dialogue; sentence splitting never decides intent. This costs up to three retrievals and remains a bounded heuristic: needs buried in a middle sentence or written without sentence boundaries can still be missed. ## Conversation input `xkb_recall` accepts optional `conversation` messages (`role`: `user`/`assistant`, `content`: text), in chronological order, excluding the current `message`. HTTP recall/context accepts the same field. Inputs accept at most eight messages; the runtime retains the last four eligible messages, up to 600 characters each. The current utterance stays separate in captured turns and takes precedence over prior goals. Responses acknowledge the normalized context with `conversation_fingerprint` and `conversation_messages`; a client refuses an older server that ignores it. Current-message retrieval and contextual retrieval each keep their candidate budget, so a long previous topic cannot crowd out current-message candidates before judging. Ranking fuses their evidence identities after Jev; contextual backend diagnostics are reported separately in `context_retrieval`. The Claude hook reads a bounded transcript tail, omits tool output and harness messages, and passes recent dialogue on prompt submission. Without supplied dialogue, turn-start uses completed turns from that same session only. MCP alone does not create automatic invocation; the calling agent must notice conversational needs and invoke the tool. Refresh tool discovery after a schema update. The prompt hook allows 40 seconds for turn-start recall, matching the HTTP recall client, while ordinary session/capture requests retain their 6-second timeout. `XKB_HOOK_RECALL_TIMEOUT` overrides recall; `XKB_HOOK_TIMEOUT` overrides both when no recall-specific value is supplied. The installer gives the hook 50 seconds overall. Existing installations need their XKB hook entries refreshed as well as the script update; a shorter outer timeout still terminates the hook early. Run `python scripts/xkb_eval.py --cases evals/conversation-cases.json` for the offline conversational checkpoints. These preserve statements and follow-ups instead of converting them to questions. `need` describes the intended assistance; `expected_delivery` distinguishes useful evidence from a turn that should stay quiet. `expected_any_ids` accepts equivalent sources for one evidence requirement. It measures coverage of labelled requirements, not exhaustive corpus recall. Unexpected records still fail precision checks. The offline corpus exposes weak keyword overlaps; it does not certify semantic usefulness or the answering agent's actual use of evidence. Live suites and full responses for a private library should stay outside the public repository, with labels frozen before requests and any later review recorded separately. For live acceptance, include non-question plans and obstacles, context-dependent follow-ups, new constraints, explicit topic changes, and ordinary acknowledgements. Inspect timing and evidence content, including contradictions and irrelevant or repeated advice. A complete judge response is a readiness check, not proof that the agent recognized the user's need or used the evidence well. MCP (`xkb_recall`), the default `recall_router.py` and `xkb_ask.py` CLIs, and HTTP (`POST /v1/recall`) use `Store.knowledge_recall`: cards, wiki and conversation evidence go through the same namespace checks, relevance judge and ranking. `xkb_evidence.py` defines the title/body/source fallbacks used by the service, judge, hook and ask client. Original evidence fields remain intact in the packet; MCP preserves it. The old continuity/action/contrarian presentation is available through `recall_router.py --legacy-router`; its Python `route()` API and classification-only `--dry-run` retain their previous behavior. Its special hints have not been removed. The ask client's standalone search and answer branches have been retired; `--legacy-search` prints a migration notice and uses the common core. Both default CLIs accept `--env-file` and `--limit` (1–50). `xkb_ask.py --no-wiki`, `--no-cards`, `--no-conversations`, `--no-gbrain`, `--max-wiki` and `--max-cards` now send validated options to the service. Quotas are upper bounds (0–50), applied after identity fusion and before the global limit. Disabled layers are not searched. `--no-gbrain` disables vector retrieval and uses the core's keyword search; the shared relevance judge still applies. `wiki` selects the wiki/daily-note index; `cards` selects the card backend. HTTP clients can send the same contract in `POST /v1/recall` or `/v1/context`: ```json {"query":"deployment","limit":10,"options":{"wiki":false,"semantic":false,"max_cards":3}} ``` The response echoes the complete effective `options`. The client rejects an old server that silently ignores options instead of falling back to standalone search. Omitting options preserves the common defaults: all layers, semantic retrieval, and no additional per-layer cap beyond the request's global limit. The ask client generates an answer from the shared records, including conversation traces, and returns `evidence_refs` and the complete `recall` packet in JSON. `wiki_refs` and `card_refs` remain compatible subsets. These are available evidence, not proof that the generated answer used every item. A remote recall does not require local embedding credentials; answer generation still needs its configured LLM credentials. Recall failure stops before answer generation. ## Select the library Without `XKB_MEMORY_SERVICE_URL`, MCP runs the service core locally in a child process. `XKB_DATA_DIR` (or `.xkb.json`) selects the cards/wiki/indexes; `XKB_SERVICE_DB` selects the conversation database, defaulting to `~/.xkb-runtime/knowledge.sqlite`, the HTTP service's default. Recall can create that database and records retrieval usage; it does not ingest or promote knowledge. For agents to share the **same deployed library**, point them at the same HTTP service, not different local copies. Configure these in the MCP environment or in the file named by `XKB_ENV_FILE`: ```dotenv XKB_MEMORY_SERVICE_URL=http://127.0.0.1:18972 XKB_SERVICE_TOKEN=your-service-token XKB_NAMESPACE=private ``` The namespace must match the token's namespace. A missing token, forbidden namespace, timeout or invalid response is an error. An explicit remote target never falls back to local data. Redirects are rejected to keep the token on the configured endpoint. Use HTTPS or the existing SSH tunnel for remote access. MCP does not implicitly read the Claude hook's separate configuration file. Keep credentials in runtime configuration. Process variables override `XKB_ENV_FILE`; its settings are loaded before the recall worker imports path or provider modules. Configure the HTTP service with the same data paths and database when using both local entry points. MCP still accepts `message`, with an optional integer `limit` (1–50). Its metadata contains the full HTTP recall packet (`records`, `context`, `retrieval_mode`, `judge`, `warnings`, ACL and backend diagnostics), plus `connection`. The older `results` and `formatted_text` names remain aliases for compatibility. The old router-specific trigger classification is no longer returned: non-skipped calls use `trigger_class: knowledge`, `state: recall`. ## Evidence identity and source diagnostics Each merged recall record includes `evidence_key`, produced by the same identity function used for RRF fusion, usage counting and demotion. Explicit IDs take precedence over citation URLs. Card identity ignores display titles; wiki and daily-note identity includes the source document and section. Conversations use their trace ID. The namespace is part of the identity. Distinct IDs sharing a URL remain separate, and anonymous records are neither merged by title nor assigned persistent usage counters. Original IDs and provenance remain available. The versioned `ev1:` keys are stored in `recall_usage.record_id`. Earlier unversioned rows remain untouched but no longer drive current demotion or its report. A former document-level counter cannot be split reliably across sections, so no inferred backfill is performed. New counters accumulate only observed evidence identities. `backends.cards`, `backends.wiki` and `backends.conversation` report their own `attempted`, `used` and `status`. States distinguish `disabled`, `unavailable`, `empty`, `used`, `filtered`, `error`, `timeout` and `invalid_response`; `unknown` means an adapter omitted diagnostics. Keyword fallback has its own nested status. `filtered` means retrieval produced hits but the eligibility/relevance gates removed them; it is not a backend outage. ACL counts record filtering events, not unique IDs. Wiki-only semantic results use `retrieval_mode: wiki_semantic`; they no longer claim that XBrain succeeded. Explicit keyword mode uses `keyword`; disabling both knowledge indexes uses `conversation_only`. The compatibility `semantic_backend` field aggregates the semantic legs, while `backends` retains partial failures. Doctor accepts Wiki semantic retrieval for `--require-semantic`, but reports degradation if another enabled source failed. A disabled layer is not a failure. ## Relevance and usage accounting Jev decides relevance at the merged recall boundary by default (`XKB_JEV_DECIDE=1`). Legacy cosine gates remain on legacy callers; shared Wiki candidates reach the merged judge under the count budget. `XKB_JEV_DECIDE=0` disables the final judge; optional shadow comparisons then provide observations for `xkb_jev_shadow_report.py`. A missing/failed judge is reported and retains the candidates; it is never counted as a negative verdict. `recall_usage` records one observation per namespace and evidence ID per call: | Counter | Meaning | | --- | --- | | `considered_count` | Reached the merged recall boundary, after upstream retrieval gates. | | `judged_count` | Received a decisive verdict: at least one positive leg, or every leg explicitly negative. | | `relevant_count` | At least one leg passed the final judge's relevance floor. | | `returned_count` | Survived filtering, ranking and the final limit into the recall packet. | Repeated legs with the same ID do not inflate counts. A missing verdict on one leg prevents treating a partly judged record as rejected. Returned evidence is not proof of network receipt, hook injection, answer citation or task success. Demotion requires at least five explicit judged observations and no positive verdict in that namespace. One positive verdict lifts it immediately, even if the result limit prevents returning the item. Unknown verdicts neither add rejections nor erase prior verdicts. Demotion affects rank, never deletes data. The unused gain/EMA experiment and its tests were retired; Git retains history. Existing `knowledge_usage` is preserved and still measures legacy cosine passes. Its historical `injected_count` and `ever_useful` names do **not** mean final delivery or answer use. The `/v1/knowledge/cold` report labels that basis explicitly. No historical count is backfilled into the new counters: current demotion starts fresh. Inspect it with `python scripts/xkb_evict_report.py --namespace private --json`. Rolling back the code leaves this additive SQLite table harmlessly in place. ## Check a real connection ```bash python scripts/xkb_doctor.py --query "a topic in your library" --json python scripts/xkb_doctor.py --query "a known source" --expect-id source-card-id python scripts/xkb_doctor.py --query "a known source" --require-semantic --require-judge ``` Doctor starts the actual MCP process and verifies initialization, tool discovery and a real tool call. It reports the selected local paths or remote endpoint, namespace, actual search mode, judge status, result IDs, warnings and elapsed time. - `ready`: the requested checks passed without the reported degradation. - `degraded`: connection works, but retrieval uses a fallback or the judge is unavailable/disabled. This is not proof of semantic retrieval quality. - `failed`: connection, expected evidence, or a requested capability failed. Exit code is 0 for a working connection (including explicit degradation), 1 for failure. Use `--expect-id`, `--require-semantic` and `--require-judge` for stricter acceptance. An empty successful query alone does not verify a populated library. Doctor verifies a **fresh subprocess**, not the tool list of an already-open agent; restart/reconnect that agent after changing its MCP configuration. ## Repeatable evaluation ```bash python scripts/xkb_eval.py --output recall-report.json python -m unittest discover -s tests -p 'test_recall_transports.py' python -m unittest discover -s tests -p 'test_recall_eval.py' ``` The bundled 24 cases use an isolated synthetic library with no inherited credentials, remote endpoint or production data. They cover multiple subjects, Chinese queries, both sides of conflicting evidence, cards plus wiki, namespaces, no-answer queries and greetings. Each case calls the real MCP process. Reports include expected-evidence recall, allowed-evidence precision, no-answer success, duplicate evidence, latency, actual retrieval modes and judge availability. Exit code is 1 if any case fails; transport failures count as failures, not empty successful retrievals. HTTP/MCP parity and judge/auth behavior are covered by the transport tests, with a deterministic judge double instead of a paid model. This suite measures offline keyword/ACL/transport behavior, **not semantic model quality**. The initial measured baseline is 20/24: all required evidence is found, but three cases return extra keyword matches, and `好，收到` is not classified as an acknowledgement. Those are visible failures, not relabelled successes. Keep the labels intact when improving these behaviors. After fixing punctuation in compound acknowledgements, the same unchanged cases score 21/24: `好，收到` is now skipped, while the three extra-keyword-match cases remain failures. `evals/recall-baseline.json` preserves the initial measurement. The 2026-09-27 keyword correction passes **24/24 with those labels unchanged**; `evals/recall-keyword-report.json` records the run. Keyword fallback searches titles, summaries, tags and keywords, plus Wiki body text. IDs, filenames, source URLs and ACL metadata are not content matches. Among ACL-allowed candidates, complete query-term matches take precedence over partial matches; if none are complete, partial matches remain available. Shared stopwords are ignored. This preserves natural-language CJK fallback, but can omit useful partial matches when a complete lexical match exists. It is a lexical policy, not a replacement for semantic relevance evaluation. The agent hook renders conversation evidence from `query`/`answer`, preserving `trace_id` as provenance, as well as cards and wiki records from `title`/`summary`. MCP availability alone does not establish automatic recall: verify that the agent's actual `UserPromptSubmit` event invokes the XKB hook. The current installer targets Claude Code settings; it does not install a Codex/Orca hook. For a real-library quality evaluation, create labels from reviewed evidence: ```json { "cases": [ {"id": "known-answer", "query": "Your real question", "expected_ids": ["reviewed-id"]}, {"id": "no-answer", "query": "An unrelated question", "expected_ids": []} ] } ``` `allowed_ids` can list additional genuinely relevant evidence; it defaults to `expected_ids`. Optional `namespace`, `limit` and `retrieval_mode` allow targeted cases. The default `label_unit: "document"` measures recall and precision over unique display IDs, so multiple sections of one allowed document do not reduce precision. For section/namespace-specific evaluation, set `label_unit: "evidence"` and use the shared `ev1:` evidence keys in `expected_ids` and `allowed_ids`. Duplicate detection always uses evidence identity, regardless of label unit; unidentified records fail explicitly. Reports include both IDs and evidence keys. Include paraphrases, stale/current evidence, contradictions, multilingual questions and genuinely unanswerable questions. Review labels independently of the retrieval output; do not label results relevant merely because the system returned them. ```bash python scripts/xkb_eval.py --live --cases reviewed-cases.json --require-semantic --require-judge --output live-report.json ``` Live evaluation uses the configured library and providers and can incur provider costs. Synthetic success cannot establish a production judge threshold; reserve separate questions for validation rather than tuning and reporting on the same set. ## Jev input-limit audit (2026-09-27) [TypeSafe's model reference](https://docs.typesafe.ai/models) documents 64k tokens for state plus all questions, and 32k for state plus the longest question. Question count and JSON byte count are not token counts. The XKB adapter trims queries to 600 characters at the time of that audit and each candidate to 900 characters before adding instructions; these are character caps, not a token-budget guarantee. A read-only copy of the VPS conversation database was used to reproduce the default `Seedance workflow` retrieval with the production providers. The actual request contained 25 questions, 31,647 serialized bytes, a 24-character state, 9,695 total instruction characters and a longest question of 922 characters. The provider returned HTTP 200, all 25 answers, and usage of **7,344 input tokens** (444 output tokens). Requested model: `jev-1.13`; returned model: `typesafe/jev-1.13-20260917`. This usage is provider-reported, not an independent count with TypeSafe's tokenizer. No production conversation/usage rows were changed by this measurement. This successful sample is below the documented budgets. It does not establish the cause of earlier HTTP 400 responses or guarantee that other payloads fit. No fixed 8/25/30-question limit or batching change was introduced. Larger limits can produce more input; any future batching must account for both total input and state-plus-longest-question budgets, including instructions. A previous small JSON payload producing a provider body-size error must not be described as proof that the official context limit was reached. Current implementation: conversational judge input is capped at 4,096 characters to retain the bounded current utterance and all four context messages. Each serialized request remains capped at 32 KiB; batches share a deadline. These character and byte limits are not claims about the provider's token limits.