# SC-Compose Architecture > Status: Active Release Baseline > Product: `sc-composer` (library), `sc-compose` (CLI), `sc-sha` (hash library), and the `bindings/python`, `bindings/sc-sha-python`, and `bindings/sc-sha-go` adapters > Document role: Normative release architecture for all in-repo packages This document supersedes the prior high-level placeholder. It is the normative release architecture baseline for `sc-compose` v1.4.1. ## 1. Architectural Intent This document defines the required architecture of `sc-composer`, `sc-compose`, `sc-sha`, and the Python and Go adapters for release work. It is not a description of the current implementation. The goals are: - one implementation of prompt and template composition semantics, - deterministic outputs and diagnostics, - runtime-agnostic library behavior, - a thin CLI over library APIs, - clear separation between reusable core logic and integration-specific edges. ## 2. Boundary Diagram ```text outside this repo +-----------------------------------------------+ | ATM adapter / other host integration | | - builds ComposeRequest | | - calls render_template() or Renderer | | - may inject an observer implementation | +-------------------------+---------------------+ | v +-------------------------------+ | sc-compose | | CLI / UX / logger wiring | +-----------+-------------------+ | uses concrete logger v +-------------------------------+ | sc-observability | | Logger + file/console sinks | +-------------------------------+ ^ | injects CLI-owned observer adapter | +-----------+-------------------+ | sc-composer | | core composition API + | | local observer hook layer | +-----------+-------------------+ ``` ATM-specific integration attaches above the two-crate boundary. `sc-composer` never imports ATM types, defines its observer hooks locally, and receives concrete logging behavior through trait injection rather than a direct dependency on `sc-observability`. ## 3. Crate Layout ### 3.1 `sc-composer` `sc-composer` is the core library crate. It owns: - template parsing, - frontmatter parsing and normalization, - include expansion, - variable discovery and validation, - resolver policy evaluation, - rendering, - composition pipeline assembly, - diagnostics production, - reusable workspace helpers for initialization tasks. ### 3.2 `sc-compose` `sc-compose` is the CLI binary crate. It owns: - argument parsing, - command dispatch, - output formatting, - exit codes, - file-writing UX, - CLI-facing observability wiring, - bundled example-pack discovery, - user template-pack discovery and storage, - pack metadata parsing, - templates add workflows, - bundled CLI feature manuals (`help_topics` module: one ordered `(topic_name, content)` registry, each entry's content embedded from a single `crates/sc-compose/docs/manual/.md` file via `include_str!`; no per-topic Rust modules or hand-written string constants; manual sources live inside the crate directory, not under top-level `docs/`, because `cargo package`'s isolated verify build cannot see files outside the package root). - exclusive ownership of the `help_topics` module and ordered manual-topic registry; `sc-composer` and `bindings/python` do not define, import, or mutate manual-topic metadata. ### 3.3 `bindings/python` `bindings/python` is the Python-facing adapter package. It owns: - PyO3 wrapper classes and functions, - maturin packaging metadata, - Python type stubs and `py.typed` markers, - Python wheel smoke tests, - Python-facing request/result shims over `sc-composer`. It must remain an adapter layer only. It does not own: - CLI argument parsing, - observability or logger wiring, - report runtime helpers, - ATM-specific integration, - any semantic reimplementation of composition or validation behavior. ### 3.4 `sc-sha` `sc-sha` is the standalone pure-computation crate for content and composition identity. It owns the two published hash operations and their input validation/encoding contracts; it has no filesystem, template, CLI, or ATM behavior. ### 3.5 `bindings/sc-sha-go` `bindings/sc-sha-go` is the generated UniFFI Go adapter for the two public `sc-sha` operations. It owns Go-facing generated types, CGo/native artifact selection, and Go consumer packaging. It depends only on `sc-sha`; it does not depend on `sc-composer`, `sc-compose`, the Python adapter, or ATM/runtime code. The adapter follows [ADR-0020: Generated Go Binding Strategy](adrs/0020-generated-go-binding-strategy.md), which is **Accepted**. Consumers download and extract the target-specific release bundle before building; `go get` alone does not provide the native library. ### 3.6 Dependency Direction Required dependency direction: - `sc-compose` -> `sc-composer` - `sc-compose` -> `sc-observability` - `sc-composer` -> `sc-sha` - `bindings/python` -> `sc-composer` - `bindings/sc-sha-python` -> `sc-sha` - `bindings/sc-sha-go` -> `sc-sha` - `sc-observability` -> `sc-observability-types` Required observability split: - `sc-observability` is the concrete logging integration target for the CLI. - `sc-composer` keeps its observer interfaces local. Forbidden dependency direction: - `sc-composer` -> `sc-compose` - `sc-composer` -> `bindings/python` - `sc-composer` -> `sc-observability` - `bindings/python` -> `sc-compose` - `bindings/python` -> `sc-observability` - `bindings/python` -> orchestration-specific runtime crates - `sc-composer` -> orchestration-specific runtime crates - `sc-composer` -> mailbox helpers, daemon helpers, team-state helpers, or runtime-specific home-resolution helpers ### 3.6 ATM Integration Model ATM integration is an adapter concern outside this repository. - An ATM adapter depends on `sc-composer` or `sc-compose`; this repository does not depend on ATM crates. - The adapter constructs `ComposeRequest` values and calls either `render_template()` for one-shot usage or the `Renderer` API for repeated rendering. - If ATM needs telemetry, the adapter or CLI injects a sink implementation through the library's trait-based observability hooks. - `sc-composer` never imports ATM types, mailbox abstractions, spool paths, or runtime-management helpers. ## 4. Module Architecture `sc-composer` should be organized around these modules: - `frontmatter` - parses YAML frontmatter, - normalizes omitted fields to schema defaults, - exposes typed frontmatter structures. - `types` - defines shared composition data structures such as pass configuration and verify result types, - centralizes multi-pass request/result shapes that are reused across parser, validation, render, and verify code paths. - `resolver` - resolves explicit file paths and profile-mode prompt lookup, - records search traces, - applies resolver policy. - `include` - expands `@` directives, - enforces path confinement, - tracks include stack, - detects cycles and depth overflow. - `validation` (context merge and token discovery implemented here, not as separate `context.rs`/`tokens.rs` files) - merges explicit variables, environment variables, and defaults in precedence order (explicit > env > frontmatter defaults), - tracks variable origin, - applies unknown-variable policy, - exposes `discover_tokens(text) -> BTreeSet` for standalone token discovery workflows, - distinguishes declared, undeclared, missing, and extra variables. - `render` - configures the template engine, - exposes the long-lived `Renderer` session type as the primary API for repeated rendering, - keeps `render_template()` as a one-shot convenience wrapper, - renders template content under normal or strict undeclared-token policy. - `template_ext` - exposes shared, case-insensitive template-path classification that removes stacked `.j2`/`.jinja2`/`.jinja` suffixes before determining the content extension; it keeps renderer auto-escape, checked-render validation, and CLI JSON detection in agreement. - `template_scanner` - exposes the shared lexical Jinja variable-expression scanner used by library JSON diagnostics and the `sc-compose` template-lint command. - `directive_inspection` - validates raw UTF-8 template bytes with MiniJinja, - exposes `inspect_template_directives` and the purpose-built `TemplateDirective`/`SourceSpan`/`TemplateDirectiveKind` types, - classifies include, import, and from-import statements without exposing parser or AST types and without resolving their targets. - `validate` - produces validation reports and diagnostics without writing output. - `verify` - renders templates through all configured passes and compares the result against deployed content, - returns structured drift-check results for both library callers and the CLI wrapper. - `error` - defines crate-owned error types and shared recovery-hint structures, - maps lower-level failures into stable public categories. - `diagnostics` - defines diagnostic types, - defines the JSON diagnostic schema contract. - `workspace` - implements `frontmatter-init` and `init` logic for reuse by the CLI and any future embedded callers. - `observer` - defines the local observer and sink traits used by embedded hosts and the CLI, - owns the no-op observer used when no caller injects a concrete implementation, - emits structured composition-stage events, - never binds directly to `sc-observability`. ## 5. Resolver Path Policy (FR-5) Resolver policy must be data-driven and not embedded in CLI-only conditionals. The policy model must express: - runtime name, - profile kind, - ordered candidate directories, - ordered filename probes, - ambiguity rules when runtime is omitted. ### 5.1 Runtime-Specific Directories - `.claude/agents/` - `.claude/commands/` - `.claude/skills/` - `.hermes/agents/` - `.hermes/commands/` - `.hermes/skills/` - `.codex/agents/` - `.codex/commands/` - `.codex/skills/` - `.gemini/agents/` - `.gemini/commands/` - `.gemini/skills/` - `.opencode/agents/` - `.opencode/commands/` - `.opencode/skills/` ### 5.2 Shared Directories - `.agents/agents/` - `.agents/commands/` - `.agents/skills/` There is no flat shared fallback such as `.agents/`. ### 5.3 Probe Rules For agent and command prompts, candidate probe order within a directory is: 1. `.md.j2` 2. `.md` 3. `.j2` For skills, candidate probe order within a directory is: 1. `/SKILL.md.j2` 2. `/SKILL.md` 3. `/SKILL.j2` ### 5.4 Ambiguity Rules - If a runtime is explicitly provided, only that runtime chain is evaluated. - If a runtime is omitted, all runtime and shared roots are evaluated. - If multiple candidates match, resolution fails with an ambiguity diagnostic. - If exactly one candidate matches, it may be selected without an explicit runtime. ## 6. Frontmatter Model (FR-1, FR-2) Frontmatter is a first-class typed structure. Target frontmatter shape: ```text Frontmatter { required_variables: Vec, defaults: Map, metadata: Map, } ``` Normalization rules: - If frontmatter exists but omits `required_variables`, normalize to `[]`. - If frontmatter exists but omits `defaults`, normalize to `{}`. - If frontmatter uses `input_defaults`, normalize it into `defaults`. - If frontmatter exists but omits `metadata`, normalize to `{}`. - If no frontmatter exists, the document has no declarations and no defaults. - If both `defaults` and `input_defaults` appear, merge both maps, let `input_defaults` override overlapping keys, and emit `WARN_VAL_CONFLICTING_DEFAULT_SECTIONS`. Semantic rules: - `required_variables` declares variables that must exist after context merge. - `defaults` supplies optional values that may satisfy a required variable. - `metadata` is descriptive only and does not affect render semantics in the initial design. - An empty sequence is a valid `InputValue` and may satisfy a required variable. - When a referenced or required variable is satisfied by a default instead of explicit caller input, validation emits `INFO_VAL_DEFAULT_USED`. ### 6.1. ADR-E1: Recursive Structured Input Contract (2026-07-29) The historical H2 nested-array restriction is superseded by Sprint E.1. The shipped contract accepts finite JSON/YAML-compatible arrays and objects at any depth because `InputValue` already uses `serde_json::Value` and Minijinja can traverse those values. The top-level var-file object boundary, YAML string-key rule, and string-only `--var` interface remain unchanged. Sections 15 and 18 link here for the template-pack and diagnostic implications; they intentionally do not repeat this decision record. `InputValue` in H1/H2/E.1 means one of: - string - number - boolean - null - object/map with string keys - finite recursive sequences and objects containing any supported value Rust type contract: - `InputValue` is represented as `serde_json::Value`, - object values with string keys may cross the CLI-to-library boundary, - nested sequences are accepted at any depth, - arrays of objects and jagged arrays are supported in E.1, - object trees may contain scalar leaves, nested objects, and recursive arrays. Sequence values are recursively validated in E.1: - sequence members may contain scalars, objects, arrays, or null, - nested sequences may be jagged and may occur inside object fields, - object-valued sequence members are supported at every variable path. Historical H2 boundary and E.1 decision: - The dated E.1 architecture decision record supersedes the former H2 shape restriction. The implementation follows the existing `serde_json::Value` representation and Minijinja traversal capability rather than introducing a second recursive value model. - The var-file document remains a top-level JSON/YAML object, YAML map keys remain string-only, and `--var key=value` remains string-only. `MetadataValue` may be any YAML value: - scalar - sequence - mapping Supporting public newtypes: - `VariableName` - validated variable identifier used by `required_variables`, `defaults`, diagnostics, and variable-source maps, - prevents accidental use of arbitrary strings in the public API. - `IncludeDepth` - non-negative bounded include-depth value used by include policy and errors. - `ConfiningRoot` - canonicalized root path newtype used by path-confinement checks and configuration validation. ## 7. Variable and Token Semantics (FR-2) The architecture must distinguish these cases: - declared required variable, - declared optional variable with a default, - undeclared referenced token, - extra provided input variable. ### 7.1 Default Mode In default mode: - undeclared referenced tokens are preserved in rendered output, - undeclared referenced tokens produce diagnostics, - undeclared referenced tokens are not implicitly promoted to required variables. ### 7.2 Strict Mode In strict mode: - undeclared referenced tokens are fatal during validation, - undeclared referenced tokens are fatal during rendering. ### 7.3 Missing Required Variables Missing required variables remain a separate diagnostic class: - they fail validation and rendering, - they are reported with file, line and column when available, and include chain. ### 7.4 Built-In Render-Context Tier (FR-2c) Built-in render-context variables are injected after caller-provided inputs (explicit `--var` and `--env-prefix`) are merged but before template-owned defaults take effect. Caller-provided values always win over built-ins. The built-in set is: - `TEMPLATE_NAME` - `HOSTNAME` - `USERNAME` - `RENDER_DATE` - `RENDER_TIMESTAMP` Merge order is therefore: 1. explicit input variables, 2. environment-derived variables, 3. built-in render-context variables, 4. user-template `input_defaults`, 5. frontmatter defaults. ## 8. Public API Shape (FR-6, FR-7) The library API should expose explicit request and result types. Required library surface: - `resolve_profile(request) -> Result` - `compose(request) -> Result` - `validate(request) -> Result` - `init_workspace(root, options) -> InitResult` - `frontmatter_init(path, options) -> FrontmatterInitResult` - `Renderer::render(compiled, context) -> Result` as the primary repeated-render API - `Renderer::with_delimiters(open, close) -> Result` as the only public renderer-customization seam - `render_loaded_template(request) -> Result` as the runtime-agnostic entry point for callers that already loaded template text outside `sc-composer` Primary render-entrypoint decision: - `Renderer` is the primary long-lived rendering API because it can retain a pre-built `minijinja::Environment` across multiple render operations. - `render_template()` remains a stable convenience API for one-shot rendering and simple callers. - Callers rendering the same template or environment repeatedly should use `Renderer` once implemented rather than paying per-call environment setup and AST re-parse cost. ### 8.1 API Ownership Matrix The rendering and composition surfaces have distinct responsibilities. | Surface | Owns | Does not own | | --- | --- | --- | | `Renderer` | reusable template-engine environment setup plus inline/named rendering over caller-supplied template text and context, including delimiter customization through `with_delimiters(open, close) -> Result` | profile resolution, include expansion, variable validation, block assembly, repository bootstrap, arbitrary third-party engine configuration | | `compose()` | top-level composition orchestration: resolve, include expansion, validation, built-in context injection, render, and block assembly | direct CLI UX decisions | | `render_template()` | one-shot rendering entry point for callers that already have template text and context | profile resolution, repository scanning, include expansion, validation, workspace bootstrap | | `validate()` | validation phase only; returns structured diagnostics without writing output | output generation or file writing | | `frontmatter_init()` | frontmatter discovery and rewrite helper | template composition pipeline execution | | `init_workspace()` | repository bootstrap helper | template composition pipeline execution | ### 8.2 Core Request Types `ComposeRequest` - `runtime: Option` - `mode: ComposeMode` - `root: ConfiningRoot` - `vars_input: Map` - `vars_env: Map` - `vars_defaults: Map` - `guidance_block: Option` - `user_prompt: Option` - `policy: ComposePolicy` `ComposeMode` - `Profile { kind: ProfileKind, name: String }` - `File { template_path: PathBuf }` Semantics: - `runtime = None` is valid and enables the omit-runtime search behavior defined in the requirements. - `ComposeMode` is variant-specific and must not be represented as a bag of unrelated optional fields. - In `File` mode, `runtime` may be `None` and is ignored unless a caller wants to attach runtime context for logging or policy selection. `ComposePolicy` - `strict_undeclared_variables: bool` - `unknown_variable_policy: UnknownVariablePolicy` - `unbound_variable_policy: Option`; when omitted, referenced-but-unbound diagnostics inherit `unknown_variable_policy` for compatibility, while an explicit value keeps the two policy axes independent - `max_include_depth: IncludeDepth` - `allowed_roots: Vec` - `resolver_policy: ResolverPolicy` - `passes: Vec` `PassConfig` - `pass_number: u8` - `required_variables: Vec` - `defaults: Map` - `metadata: Map` `LoadedTemplateRequest` - `template_name: String` - `template_text: String` - `context: BTreeMap` `RenderedArtifact` - `rendered: String` - `template_name: String` `ParsedTemplate` - `passes: Vec` - `body: String` Compatibility rule: - the existing `frontmatter() -> Option<&Frontmatter>` accessor remains the compatibility seam for current callers, - single-header templates preserve existing semantics, - stacked templates may define `frontmatter()` as the first (outermost) pass while `passes` exposes the full multi-pass structure. ### 8.3 Core Result Types `ResolveResult` - `resolved_path: PathBuf` - `attempted_paths: Vec` - `ambiguity_candidates: Vec` `ComposeResult` - `rendered_text: String` - `resolved_files: Vec` - `resolve_result: ResolveResult` - `variable_sources: Map` - `warnings: Vec` `ValidationReport` - `ok: bool` - `warnings: Vec` - `errors: Vec` - `resolve_result: ResolveResult` `ComposeError` - `Resolve(ResolveError)` - `Include(IncludeError)` - `Validation(ValidationError)` - `Render(RenderError)` - `Config(ConfigError)` `FrontmatterInitResult` - `target_path: PathBuf` - `frontmatter_text: String` - `discovered_variables: Vec` - `changed: bool` - `would_change: bool` Template-init contract: - `template-init` consumes an input file plus one or more pass-scoped variable maps from the CLI wrapper. - Replacement planning is CLI-owned in `sc-compose` and sorts all pass-scoped literal values globally longest-first, with higher pass numbers breaking ties, so specific strings are reserved before substrings anywhere in the file. - Generated headers are emitted in outer-to-inner order and include `pass: N` only when the output must remain genuinely multi-pass. - If the resulting template is effectively single-pass, the emitted header is normalized back to the shipped `1.2.x` single-header shape: `required_variables`, `defaults: {}`, `metadata: {}`, and no `pass: 1`. - `template-init` remains CLI-owned in `sc-compose`; `sc-composer` owns only the reusable workspace/helper types needed to support the conversion. `InitResult` - `prompts_dir: PathBuf` - `gitignore_updated: bool` - `scanned_templates: Vec` - `recommendations: Vec` - `validation_passed: bool` Entrypoint contract: - `compose(request) -> Result` - `validate(request) -> Result` - `resolve_profile(request) -> Result` - `Diagnostic` is not a failure type. Diagnostics describe warnings and user-actionable validation findings; `ComposeError` describes operation failure. ### 8.4 Known-Template Reverse Extraction (FR-16, ADR-0011) The known-template reverse-extraction API is a pure `sc-composer` capability defined by [ADR-0011](adrs/0011-reverse-extract-known-template-contract.md). It accepts template and rendered text in memory and returns a generic `ExtractionReport` containing string values, structural occurrence evidence, report-level confidence, and typed diagnostics. The initial format adapter is XML; the generic occurrence/path/source types are part of the public contract so later XML matching can specialize them without creating a second report model. The API distinguishes invalid requests, malformed XML, unsupported syntax, and ambiguous structure with canonical diagnostic codes. A repeated variable at distinct structural occurrences is ambiguous and must not silently replace an entry in the recovered value map. File I/O, CLI parsing, unknown-template identification, loop or branch reconstruction, JSON/Markdown extraction, and typed-value inference remain outside this contract. A dotted expression is object-field access, not a literal variable identifier, and Phase G only supports the scalar (flat) variable subset. Extraction parses each `{{ expression }}` into a `VariableName` for occurrence tracking, but the shared `VariableName` grammar (also used by composition/rendering token discovery) permissively accepts `.` as an ordinary name character. The extraction call site must reject any parsed variable containing `.` as unsupported syntax (`ERR_EXTRACT_UNSUPPORTED`) before it reaches the report; Phase G.7 will add this check locally to the extraction XML adapter without changing `VariableName`'s shared grammar or its use in composition/rendering. The Python adapter exposes this in-memory report through `extract_variables(template, rendered, *, format="xml", include=None, exclude=None)`. XML remains the backward-compatible default, while the approved JSON adapter selects the same shared extraction entry point with `format="json"`. Its report and provenance objects are wrappers over the `sc-composer` values, and fatal extraction conditions use the adapter's existing `ScConfigError` family with the canonical extraction code; Python does not implement a second extraction algorithm. Here, “reuse the existing exception hierarchy” means that all fatal extraction inputs use that established `ScConfigError` class and expose the Rust diagnostic code, recovery hints, and diagnostic detail; it does not introduce a Python-only extraction exception subclass. This capability is implemented from scratch in the production Rust library, Python adapter, and CLI. Prior reverse-extraction research informs the contract and its intentional boundaries, while the committed cross-surface corpus provides regression evidence. The research harness is not part of the product interface or a runtime dependency. Phase-H planning records the three in-scope real-customer format candidates from issue #193: JSON, YAML, and TOML adapters. XML mixed-content extraction and a narrow non-XML preamble policy remain outside Phase H and are owned by Phase I. [ADR-0012](adrs/0012-phase-h-reverse-extraction-extension-gates.md) now records the accepted format-specific path/source contract and malformed- input policy. Cross-surface evidence remains required before any adapter is delivered. The generic report model may be extended, but the library/CLI/Python ownership boundary and Phase-G fail-closed XML behavior remain unchanged while H.2 through H.8 implement, harden, and validate the accepted extensions. H.8 is the phase-ending remediation gate and does not reopen H.7's settled QA findings. The stable cross-format diagnostic inventory is maintained in [`docs/error-code-registry.md`](error-code-registry.md) and is part of the H.1 contract rather than an implementation-time choice. The format adapters do not own independent placeholder matchers. The accepted H.1 design defines the internal migration seam from the current XML value-matching path to a shared raw-text matching core. That core owns delimiter scanning, template-segment parsing, static-prefix/suffix matching, capture boundaries, and adjacent-variable ambiguity handling. Format adapters own structural parsing, occurrence paths, provenance, and format-specific diagnostics, then delegate candidate-value matching to the shared core. XML remains the first consumer of the extracted seam, followed by JSON, YAML, and TOML. The shared core is also the architectural foundation for a future customer-facing best-effort/degraded-parse mode and a cross-format raw-text mode for arbitrary text such as Markdown. Those modes are not exposed or implemented in Phase H; the future mode must reuse this seam rather than require another matcher rewrite. H.6 closure evidence is recorded in [`docs/phase-H/evidence/h-6-cross-format-campaign.json`](phase-H/evidence/h-6-cross-format-campaign.json) and its generated multi-worker report package under `site/reports/`. The campaign proves equivalent JSON/YAML/TOML report semantics across library, CLI, and Python surfaces, while preserving the explicit Phase-H boundary that XML mixed-content and dirty-prefix handling belong to Phase I. The H.6 execution record is bounded local evidence rather than a distributed agent campaign; its report and summary must retain that caveat. ### 8.5 Phase-I Raw-Text and Boundary Extensions (FR-17–FR-21, ADR-0013) Phase I extends the generic extraction bridge without introducing a second report model or matcher. The accepted contract is defined by [ADR-0013](adrs/0013-phase-i-raw-text-and-input-safety.md). The Rust format selector adds `ExtractFormat::Raw`. The dispatching path and source sums add `Raw(RawPathSegment)` and `Raw(RawExtractionSource)` variants, where `RawPathSegment` stores zero-based half-open rendered byte offsets and one-based line/column coordinates, and `RawExtractionSource::TextSpan` marks the provenance. `sc-compose` maps `--format raw`; Python maps `format="raw"`; both call `sc_composer::extract` and do not implement matching. Raw mode is known-template, in-memory text matching for Markdown and other unstructured text. It uses the H shared matcher, applies include/exclude filters before report construction while retaining filtered variables for neighboring capture matching, and uses only the stable raw diagnostic set `ERR_EXTRACT_INVALID_REQUEST`, `ERR_EXTRACT_TEMPLATE_UNSUPPORTED`, `ERR_EXTRACT_AMBIGUOUS`, and `WARN_EXTRACT_LOW_CONFIDENCE`. XML's Phase-I structural extension allows one full element-content placeholder to capture text plus approved child markup using deterministic canonical child serialization. A separate rendered-only normalizer accepts a bounded leading text/whitespace preamble before one XML document, preserves allowed prolog constructs, and emits `WARN_EXTRACT_DIRTY_PREFIX_STRIPPED` when it removes bytes. It rejects unmatched/truncated markup, malformed suffixes, multiple roots, second documents, post-root content, and DTDs. I.3 emits `ERR_EXTRACT_XML_CHILD_STRUCTURE_MISMATCH` when rendered child markup falls outside the approved template structure, `ERR_EXTRACT_XML_CONTROL_FLOW_UNSUPPORTED` when extraction would require unsupported control-flow reconstruction, and `ERR_EXTRACT_XML_DYNAMIC_ELEMENT_NAME` for dynamic element names. These stable codes keep XML structural rejection distinct from generic malformed or unsupported extraction failures. Validation token discovery recognizes the listed Jinja loop-context names only inside active `for` scopes; `loop` outside a loop and arbitrary dotted names remain ordinary validation inputs. Var-file decoding rejects YAML merge keys with `ERR_CONFIG_VARFILE` and a source line/column before tagged-value unwrapping, so inherited fields cannot disappear silently; callers recover by writing the mapping explicitly. These changes are Phase-I runtime work and are not retroactive claims about the completed Phase-H implementation. ## 9. Include and Frontmatter Merge Rules (FR-3) The include graph is evaluated deterministically. Merge behavior: - required-variable declarations from included files participate in validation of the overall composition result, - defaults from included files participate in context construction, - parent-file defaults override defaults from included files, - environment-derived variables override all defaults, - explicit input variables override environment-derived values and defaults. Metadata behavior: - metadata from included files does not affect rendering, - metadata may be retained in trace structures in a future API, but metadata is not part of current render semantics. ## 10. Diagnostics Model (FR-8) Diagnostics are structured records used by both the library and CLI. Required fields: - `code` - `message` - `path` - `line` - `column` - `include_chain` - `severity` The JSON representation must be versioned. The version belongs to the schema contract, not to any single CLI command. Top-level diagnostics envelope (payload fields are command-specific; `"valid"` shown here matches the `validate` command — see §13.1 for per-command schemas): ```json { "schema_version": "1", "payload": { "valid": false }, "diagnostics": [ { "severity": "error", "code": "ERR_VAL_MISSING_REQUIRED", "message": "missing required variable: name", "path": "templates/example.md.j2", "line": 12, "column": 4, "include_chain": [] } ] } ``` Minimal diagnostic record: ```json { "severity": "info", "code": "INFO_VAL_DEFAULT_USED", "message": "variable name not provided, using default: \"world\"", "location": "templates/example.md.j2" } ``` ## 11. Error Model (FR-7, FR-8) `sc-composer` must expose crate-owned canonical public error types. Required error structs: - `ResolveError` - `IncludeError` - `ValidationError` - `RenderError` - `ConfigError` Error requirements: - every canonical error carries an underlying `source()` cause chain when one exists, - include-related errors carry the include chain when applicable, - configuration and validation failures may carry structured recovery hints, - recovery hints must remain structured data rather than prose-only strings. CLI boundary rule: - `sc-compose` may wrap library errors with `anyhow` or `eyre` at the command boundary, - `sc-composer` public APIs must return the canonical error types defined in this document, not `anyhow::Error` or third-party engine error types. ## 12. Request Lifecycle (FR-2, FR-3, FR-6) For `compose` and `validate`, the target lifecycle is: 1. Resolve explicit path or profile path. 2. Read the root template file. 3. Parse frontmatter and body. - For multi-pass templates, continue parsing only while the next bytes at the current cursor begin another leading header. Later `---` lines in the body remain literal content. 4. Expand includes while enforcing path and depth policy. 5. Merge frontmatter declarations and include-derived declarations. 6. Discover referenced variables from the expanded template graph. 7. Merge context in precedence order: - explicit input, - environment, - defaults. 8. Apply validation policy: - missing required variables, - undeclared referenced tokens, - extra provided variables. 9. Render in normal or strict mode according to policy. - When `policy.passes` or parsed stacked headers indicate nested-template rendering, render outer-to-inner, using pass-specific delimiters and `protect_higher_braces`-style higher-brace protection between passes. 10. Assemble final output blocks. 11. Return composed output or validation report with diagnostics and trace data. Internal lifecycle encoding: - the composition pipeline must preserve the documented ordering of resolve, parse, include expansion, validation, render, and output assembly, - internal helpers may use staged data structures to make ordering violations difficult to represent, - the initial release does not expose a public typestate API or a public `pipeline` module. ## 13. CLI Command Architecture (FR-6, FR-7) `sc-compose` should be a command router over library operations. Command mapping: - `render` -> `compose` - `resolve` -> `resolve_profile` - `validate` -> `validate` - `frontmatter-init` -> `frontmatter_init` - `template-init` -> CLI-owned `template_init_file` rewrite path - `init` -> `init_workspace` - `verify` -> `verify` - `extract` -> `extract` - `observability-health` -> CLI logger initialization, then `Logger::health()` - `examples list` -> list bundled example packs - `examples ` -> resolve the bundled example-pack file, merge pack `input_defaults`, then `compose` - `templates list` -> list user template packs - `templates add` -> copy a source file or directory into the user template root as one pack - `templates ` -> resolve the user pack entry template, merge pack `input_defaults`, then `compose` - `reports init` -> initialize the shared report scaffold and starter catalog - `reports smoke` -> run the shared smoke fixture render path and emit the smoke latest-artifact set - `reports finalize` -> materialize one producer-owned report artifact set into the shared sidecar and archive shape - `reports render-spec` -> parse one TOML semantic spec and emit one Mermaid latest-artifact set - `reports index` -> aggregate and summarize latest report entrypoints from the report catalog - `reports verify` -> verify required report artifacts exist for the catalog - `reports publish-manifest` -> emit one machine-readable publish handoff from current latest report outputs - `help [topic]` / `help --list` -> CLI-owned `help_topics` registry lookup (no library call; see FR-22) The CLI must not reimplement core composition semantics. If a command requires logic useful to non-CLI callers, that logic belongs in the library. Command-specific rules: - `render` - accepts `file` mode and `profile` mode, - requires `--file ` in file mode, - accepts optional guidance and user prompt blocks, - writes to stdout by default unless an output path is chosen. - `resolve` - is defined for `profile` mode only, - fails for `file` mode. - `validate` - uses the same resolver and include graph as `render`, - never writes rendered output. - `frontmatter-init` - rewrites or inserts frontmatter for a single target file, - uses token discovery but does not render the file. - `template-init` - rewrites a single target file into a single-pass or multi-pass template, - accepts one or more `--pass N` groups with pass-scoped `--var` and `--var-file` inputs, - honors `--force` for existing frontmatter/template rewrites, - honors `--dry-run` without writing the rewritten file, - returns exit code `3` when requested literal values are not found because that outcome is a usage/configuration failure rather than a successful drift result. - `init` - performs repository bootstrap and validation-oriented scanning. - `verify` - compares one deployed file against the rendered output of `--against