# First-class semantic language Exomem keeps Markdown readable while making small observations, richer typed units, retrieval filters, ranking evidence, and authored graph relations addressable through the public product commands. Markdown remains the source of truth; lexical, vector, and graph stores are rebuildable sidecars. ## Semantic authoring contract Choose homes by future use and accumulated content, not mentions or broad scope. Keep coherent long notes; reuse or create a home for independent material. Live compiled creates, replacements and transitions to live need a valid nonempty semantic unit; pending pages may remain unit-free. Either compact or rich form satisfies the minimum; compact is preferred, and a valid rich unit does not need a duplicate compact restatement. Semantic roles: - Category: One primary open-vocabulary label describes what a unit is about; rich category defaults to its governed kind unless explicitly overridden. - Tag: Zero or more optional secondary retrieval labels refine lookup and never replace category or determine kind. - Kind: The governed semantic form: compact units always use `observation`; rich units use their recognized heading kind. Compact grammar: `- [category] content #tags (context) ^anchor`. Parse valid compact observations anywhere outside fenced code blocks. Exomem writers use `-` under the canonical `## Observations` section. Parser bullet markers are `-`, `*`, `+`; the canonical marker is `-`. Parse from the end by taking anchor, then context, then trailing tags; the authored display order remains tags, context, anchor. Category uses open vocabulary. - Compact category: the unit's one primary open-vocabulary subject or domain label. After trimming, use 1-64 Unicode code points; begin with a Unicode letter; then use only Unicode letters or digits, spaces, `_`, or `-`. Apply Unicode NFKC and casefold, then collapse runs of spaces, `_`, and `-` to one `_`. Registry alias resolution is separate from authored canonicalization; an unseen valid category needs no registry write. - Compact content: the unit's substantive observation. Use non-empty content that remains on one Markdown line. Escaped parentheses, embedded hashes, and non-trailing tag-like text remain content. - Compact tags: zero or more optional secondary retrieval labels; tags do not replace the primary category or governed kind. Write `#slug`. Use 1-64 Unicode letters or digits, `_`, `-`, or `/`; begin with a letter or digit; do not use empty path segments or a trailing `/`. Use one contiguous trailing run after content and before optional context and anchor. - Compact context: one optional authored qualifier for the observation. Write `()`. Use one balanced, unescaped parenthesized suffix preceded by whitespace. - Compact anchor: one optional stable authored unit identifier. Write `^anchor`. Use 1-64 ASCII letters, digits, or hyphens and begin and end alphanumeric. Place it at the end of the line. - Compact exclusions: observation-shaped rows inside fenced code blocks; task labels `[ ]`, `[x]`, `[X]`, and `[-]`; reserved or punctuation-bearing bracket labels outside category grammar. Compact units do not carry typed unit relations; use a canonical note-level relation or the rich form. - Rich: write `## ` with optional leading metadata `- category: `, `- id: `, `- tags: `, `- context: `, `- relations: : [[Target]]`, `- verdict: `, `- check_by: `, `- as_of: `, `- until: `. Metadata rows are optional and leading; the canonical writer emits category, id, tags, context, relations, verdict, check_by, as_of and until; category defaults to the governed kind when omitted. Accepted metadata order is flexible while rows remain leading. After optional leading metadata, add a blank line and a substantive Markdown body. Typed unit relations require the rich form. A `#anchor` target addresses a unit. - Epistemic corrections: Inspect affected claims and evidence. Distinguish attributed accounts from established facts. Correct suitable existing units with guarded writers, justified verdicts and exact links; verify current state. Refutation alone changes neither lifecycle nor rank. Writers retain prior interpretations; request read_instructions with profile=full for history guidance. Keep an anchored unit's identity when its wording changes. Preserve a false historical proposition and author its verdict; add a distinct corrective unit when useful. Refutation and contradiction do not imply supersession or reduce rank. The semantic writer retains prior interpretations atomically. Read revision_history with read_page(part=history), optionally for unit_ref, using history_limit and history_continuation in options. When it carries lookup=stale, repeat the same call shortly instead of reading absence or paging on. Existing history remains the page's operation log. Entities and legacy pages use their existing edit owner; Records use guarded row corrections and history. Corrections need no dated child, special user nudge, or confirmation beyond the active engagement policy. - Rich boundary: A heading at level N owns content until the next non-fenced heading at level N or shallower; deeper headings remain in its body. `empty_rich_unit` means a recognized rich heading has no substantive body; Add substantive body content or remove the empty recognized heading. - Exact applicability: obtain `read_instructions(profile='compact', section='authoring')` and read `semantic_authoring.minimum_semantic_unit` for current structural applicability and the status-class rule. Apply `compiled_intent(after_state) = canonical_compiled_destination(path) OR the normalized type's note-type role is compiled`. Shipped compiled types and their default folders: `research-note` in `Notes/Research`, `insight` in `Notes/Insights`, `failure` in `Notes/Failures`, `pattern` in `Notes/Patterns`, `experiment` in `Notes/Experiments`, `production-log` in `Notes/Productions`, `procedure` in `Notes/Procedures`. More may be registered; `inspect_memory(operation='schema.inspect', arguments={'subject': 'note-types'})` lists the live set. Reject missing, invalid, or mismatched compiled frontmatter before evaluating the minimum-unit predicate. A page in a type folder needs a compiled type; a compiled type may sit in any Notes folder, and a compiled page outside Notes is refused. Folders are defaults. `create_note` files research notes in their `project` home. Other compiled notes use the first `topics` entry with a home. `inspect_memory(operation='schema.inspect', arguments={'subject': 'homes'})` lists homes. The draft's `placement` gives the folder, basis and call to make a topic a home. The minimum predicate applies when the path and normalized compiled type structurally match; the result is writable managed Markdown in the governed subtree; the result is outside Sources, Evidence, and trash; no activation exclusion applies; the admitted effective status class is live. Check live creates, replacements and transitions to live; pending compiled notes may remain unit-free until activation. - Existing active pages: A post-activation compliant page cannot lose its final valid semantic unit. - Exempt content: arbitrary non-compiled Markdown, dataset cards, Evidence artifacts, hubs, indexes, logs, non-Markdown files, schema and admin artifacts, snapshots, Sources, templates, trash. - Routes: use `create_note` for a new compiled note, `supersede_note` for a replacement, `add_observation/edit_note` for one unit, and `edit_note` for a small edit or activation. Tier 2 manage_memory `files.create` and `files.append` receive the same semantic precommit contract on the complete resulting compiled Markdown; prefer create_note or supersede_note when their typed route fits. - Findings: `missing_semantic_unit` means an applicable live-class compiled result has no valid non-empty unit; `empty_rich_unit` means a recognized rich heading has no substantive body. Add substantive body content or remove the empty recognized heading. - Compact remediation: Add `## Observations` and `- [operating constraint] Keep retries bounded #reliability`. - Rich remediation: Alternatively add `## Decision`, a blank line, and a substantive body. - Semantic-unit coverage and relation-review disposition are independent obligations. - Validation review: Validation evaluates the proposed result without saving content or review state; the request includes relation assessment. Inspect the current page and candidate evidence. If no truthful link applies, preview again with a bounded evidence-based unlinked_reason. Missing edges prove nothing; preserve typed relations and metadata. Report insufficient evidence or unresolved material choices. Commit needs separate write authority. - Portable categories: Choose exactly one primary category: prefer a meaningful epistemic or operational role and put the domain in tags, but if the role would only be a generic fact, finding, or observation and the domain is the durable lens, use a domain category instead. The category vocabulary is open: these core keys are a shared starting point, not a closed list. When no core key is a good primary fit, author a new meaningful category rather than forcing an ill-fitting one. Use exactly one primary category; kind is the governed form, tags are secondary facets, and relations are typed edges. The rich form's category defaults to its kind, so `category: decision` is redundant when the kind is Decision. Create multiple distinct semantic observations and typed relations when the source genuinely supports them, but never multiply units or relations to satisfy a quota and never duplicate the same fact. Core keys are `action`, `assumption`, `code`, `config`, `constraint`, `decision`, `design`, `fact`, `finding`, `insight`, `preference`, `problem`, `question`, `requirement`, `risk`, `technique`. Core aliases are `actions` → `action`, `assumptions` → `assumption`, `configs` → `config`, `configuration` → `config`, `configurations` → `config`, `constraints` → `constraint`, `decisions` → `decision`, `designs` → `design`, `facts` → `fact`, `findings` → `finding`, `insights` → `insight`, `open_question` → `question`, `open_questions` → `question`, `preferences` → `preference`, `problems` → `problem`, `questions` → `question`, `requirements` → `requirement`, `risks` → `risk`, `techniques` → `technique`. Role example: `- [decision] Relocate to a coastal city next spring #life ^relocation`. Domain example: `- [nutrition] Evening protein improves adherence #experiment ^evening-protein`. Breadth example (life, finance, legal/travel, and career alongside one retained code line): ```markdown - [constraint] Keep retry windows bounded #code ^retry-windows - [risk] Variable-rate mortgage payments could spike #finance ^mortgage-rate-risk - [question] Does the destination require a visa before travel #legal #travel ^visa-requirement - [career] Weigh a sabbatical before the next promotion cycle #growth ^sabbatical-timing ``` Rich example: ```markdown ## Decision - id: commit-to-morning-training - tags: health - relations: supports: [[Knowledge Base/Notes/Health/Morning training]] Commit to a fixed 6am training block on weekdays so consistency compounds and health stays the durable lens for this decision. ``` For rich units without explicit `- category:`, the heading supplies the raw and normalized category identities before reviewed category-alias resolution; the resolved public category falls back to the governed kind when no category alias applies. Rich `tags` and `context` are first-class retrieval fields, not merely generic metadata. Category, kind, tags, context, and authored relations remain independent axes. Rich units also carry two governed epistemic-loop metadata keys. `check_by` is one exact ISO calendar date (`YYYY-MM-DD`) naming the day a claim should be revisited; it is filterable as the typed date field `unit.check_by`, so "what is due" is an ordered query rather than a scan. `verdict` records how the claim turned out — exactly one of `confirmed`, `refuted`, `qualified`, `inconclusive`, or `abandoned` — and is filterable as `unit.verdict`. Both appear on unit hits when present and are omitted when absent, so an unjudged unit is distinguishable from a judged one. Both are answered from the maintained catalogue rather than by reading pages, so a due-date query costs an index read on a served cell. One filter field has no index answer: `page.frontmatter:/`, which is open-ended by construction. A served request naming it is refused with `UNSUPPORTED_FILTER_FIELD` rather than silently falling back to a corpus scan, while an offline CLI caller keeps the exact scan. `unit.dimension` (below) has no column either, and a served request answers it without a scan. A date bound is answered exactly when it names a whole day, so `updated_after`, `updated_before`, `recency_days` and a bare ISO date can all be negated with `$not`; a bound carrying a timestamp is refused under the same code when nothing else in the filter narrows it, because a day column cannot decide the hour and a wrong negation would silently drop pages. `verdict` is state, not supersession, and not a score. A refuted unit keeps its page's active standing and its ordinary rank: nothing replaced it, the question simply got an answer, and the negative result stays retrievable with the evidence that refuted it. A claim carries no confidence score and a verdict is a judgment in words, so a probability or percentage is never a valid `verdict`; the parser rejects one outright. A degree of belief that its holder stated is a `## Belief` unit; "Recording a degree of belief" in the scaffold's `references/page-types.md` gives its form. The `prediction` governed kind names a unit that is a claim about the future, and pairs naturally with both keys. None of this adds a relation kind: attach the refuting or confirming material with the governed vocabulary you already have and let `verdict:` carry the judgment itself. Use `observe_memory(operation="add"|"update"|"remove"|"validate")` for one unit. Update and remove operations must carry both the current parent `content_hash` and the unit fingerprint. Compact observations cannot author typed unit relations: use a rich unit or a canonical note-level relation under `## Relations`. ## Retrieve pages, units, or both `search_memory` and `find` accept `result_level="page"`, `"unit"`, or `"mixed"`. `auto` preserves the normal page-first product behavior unless unit predicates require a unit-capable result. ```json { "query": "battery policy", "categories": ["config"], "kinds": ["observation"], "result_level": "mixed" } ``` The `categories` and `kinds` lists are shortcuts. The general `filters` object uses a bounded typed expression over reserved `page.*` fields, RFC-6901 frontmatter paths, and the closed `unit.*` field set. Lists are OR within an axis; category/kind/text/filter axes combine with AND. Invalid or over-limit plans fail before candidate retrieval and never silently widen. ```json { "query": "", "filters": { "$and": [ {"page.type": {"$eq": "insight"}}, {"unit.category": {"$in": ["config", "rule"]}}, {"page.frontmatter./status": {"$ne": "superseded"}} ] }, "result_level": "unit" } ``` An empty query plus filters is a filter-only lookup. It uses the documented filtered-most-recent sort tuple; it does not fabricate a text score. `unit.dimension` filters units by epistemic dimension, with `$eq`, `$in` and `$ne`. It takes a dimension key or a label in any language, and a name that no registry entry holds is refused with the known keys. A unit belongs to a dimension when its kind is a member, when it carries a member key, or when it is an end of a typed edge whose family is a member. A unit's own `relations` row starts an edge at that unit, and a link with an anchor, such as `[[#before]]`, ends one at the unit the anchor names. An edge written in frontmatter, in a page's relations section or as a link with no anchor starts or ends at the whole page, and that end reaches every unit of the page. A vault adds a dimension in `_Schema/semantic-language-registry.yaml`, and the filter takes it with no code change. A vault may also remove a dimension it added: no page stores a dimension key, and a filter that still names it is refused with the keys that exist. ```json {"filters": {"unit.dimension": {"$eq": "currency"}}, "result_level": "unit"} ``` A dimension with a family member reads the edge graph in one query for the units it tests. While the graph sidecar is missing or stale the request raises the typed warming outcome and never answers as if no edge existed. A caller sees an edge only when it may see both of its pages. A served request answers `unit.dimension` without reading pages. A dimension with no family member, whose member keys each have an index column, becomes the equivalent kind and key predicates, anywhere in the filter. Any other dimension is checked on the best units the search found, with one graph read for them, so the predicate must sit at the top level of the filter or under a top-level `$and`; under `$or` or `$not` it is refused with `UNSUPPORTED_FILTER_FIELD`. The limit applies before that check, so when it drops units the reply's `user_summary` says that the filter ran only on the best matches, how many of them it kept, and that more may exist. The offline CLI evaluates every dimension on every unit. ### Dimension groups A unit search asks for dimension groups through the same filter, with no other argument. When `unit.dimension` sits at the top level, alone or under a top-level `$and`, `$eq` returns one group and `$in` returns one group for each named dimension, in the order given. `$ne` asks for no group. The limit counts distinct units, and it cuts the hits by rank before the groups order them. Each unit hit in a group carries `dimension`, the key of its group, and `display` when the unit has a mark. The reply's `dimension_groups` lists each group with its `dimension`, `label`, `count`, mark counts and `units` in display order. A unit that belongs to two requested dimensions is in full in the first group, and a later group lists only its ref, with `shown_in` naming the first group and `display` holding the later group's marks. A deep search's pack carries the same `display` on its units and a `groups` list with each group's unit refs. The reply's `user_summary` names each group by its `en` label, else by its first label in language-tag order, and gives its counts, for example "4 units in 'Uncertainty'; 1 is open." The display behaviours that a dimension names set its group's order and marks. Each one reads only relation behaviours, kind traits and governed keys, so a vault dimension that names one needs no code change: - `flag_open` marks a unit `open` when its kind has the `closable` kind trait, it has no `verdict`, and no edge whose relation has the `settles` behaviour ends at the unit or at its page. - `lead_with_current` puts current units first and marks each replaced unit `replaced`, with `by` naming its successor when one is known. A unit is replaced when its page's lifecycle class is history, or when an edge whose relation has the `replaces` behaviour ends at the unit or at its page. - `show_origin` gives each unit an `origin`. `by` is its `asserted_by` or `credence_holder`. `from` lists up to three targets of the edges that start at the unit or at its page and whose relation has the `sources` behaviour, and `more` counts the rest. - `order_in_time` sorts the group by `occurred_at`, earliest first, with undated units after the dated ones. Each edge between two members whose relation has the `earlier_than_target` or `later_than_target` behaviour then orders its two ends. A unit that no such edge orders keeps its place in the date order wherever the edges allow. The edges inside a cycle decide nothing: its units keep the date order, and the group lists them in `order_conflict`. A display behaviour changes only order and marks. It never removes a unit or changes the unit's fields. Marks read only the edges that the caller may see, so `open` means that nothing the caller can see settles the unit. When the edges are unknown, no unit is marked `open`, no edge orders the group, the group carries `edges: "unknown"`, and the summary says that its links could not be read. The automatic working set (`recall_context`) does not group or mark units yet. The category report (`schema_memory` with `operation="infer"` and `subject="categories"`, which counts units by category) lists every dimension with its members and display behaviours, and with `unit_count` and `page_count` for the units it scanned. Its `edges` value says how the count was made: `not_needed` when no edge decides the dimension, `counted` when the graph answered, and `unavailable` when it could not. In an `unavailable` row, a count that edges decide is `null` instead of a number that leaves out the edges. A row also counts each mark that its dimension's display behaviours produce: `open` where it names `flag_open`, and `replaced` where it names `lead_with_current`. While the graph is missing or stale, the report schedules its rebuild, and the row's `repair` names the `maintain_memory` call that runs the rebuild now. A dimension a vault adds appears there with no code change. ## Explain ranking without inventing confidence Set `explain=true` only when ranking interpretation is useful. The response adds a versioned, bounded `retrieval_profile` and per-hit `ranking_explanation`; the default response remains unchanged when explanation is off. The profile reports requested/effective modes and result levels, normalized filters, participating and unavailable lanes, backend/model/metric metadata, fusion constants, rerank decisions, and final ordering policy. Per-hit evidence contains only lanes that actually participated. The values are deliberately separate: - BM25 is a backend relevance measurement with the direction/range declared by the profile. - cosine is vector similarity, not probability. - RRF contribution is exact rank-fusion math and exists only when fusion ran. - reranker raw and adjusted values are separate when reranking ran. - final rank follows the recorded boost, rerank, and deterministic tie-break chain. None of these is a confidence score. A disabled, unavailable, warming, or nonparticipating lane is reported at profile level and is never represented as a fabricated per-hit zero. ## Read and traverse exact units Unit-level recall returns an exact `unit_ref`. Pass it to `read_page` (in `options`) to read the current unit with bounded parent context. Missing, stale, ambiguous, and superseded references are explicit statuses; Exomem never substitutes a nearby unit. `connect_memory(operation="graph-context")` accepts an exact `unit_ref` or registry-resolved `categories`/`kinds` filters. Compact and rich units appear as derived graph nodes, but traversal follows authored edges only. Exomem does not infer a typed edge from a compact observation's category or prose. ## Reviewed creation and adoption For a validate/review/commit creation flow, call the intended writer with `validate_only=true`, retain the returned `draft_id`, `draft_hash`, `draft_token`, candidate, and semantic feedback, then commit the unchanged draft through the same writer. If validation returns `reviewed_none_required=true`, commit with `relation_disposition="reviewed_none"`, the returned `relation_review_hash`, and an explicit `relation_review_reason`. Never invent a reviewed-none decision from an empty Relations section. The legacy advertised spelling `reviewed-none` is accepted at the boundary but stored canonically as `reviewed_none`. `adopt_vault(mode="scan-only")` includes a bounded read-only semantic census: compact/rich counts, raw/canonical/resolved categories, collisions, malformed candidates, contract/schema debt, relation dispositions, coverage, and safe next actions. It never rewrites originals or requires categories. `compile-selected` may copy selected legacy text into governed Sources and then returns a proposal only. Review it and hand it to `create_note` (the `note()` writer); normal semantic precommit remains mandatory. Default initialization, scanning, reading, registry upgrades, and index rebuilds do not fabricate observations, categories, relations, page IDs, or review decisions. Existing pages are activation-snapshotted by stable ID or portable path/hash fallback without rewriting their Markdown. ## Relation vocabulary workflow Before authoring a typed relation, use `connect_memory(operation="resolve-relation")` to inspect registered meanings. Choose the most specific registered relation that is truthful. `relates_to` is an honest generic connection; no edge is the right result when there is no durable relationship. Resolution is evidence for the caller, never automatic semantic selection or edge authoring. For a durable missing distinction, use `schema_memory(subject="relations", operation="propose-relation")`, review its delta and `content_hash`, then submit the reviewed delta through `schema_memory(subject="relations", operation="save-relations")` with `expected_hash` and `why`. Save is hash-guarded; proposal and resolve are read-only. For example, a reviewed synthetic `applies_to` relation can remain a proposal until its explicit guarded save. Registry reuse, generic fallback, and abstention do not require a registry change.