# VT Conformance Tooling

This PR adds a focused validator/report generator for `docs/vt_coverage_matrix.md`.

## Command

Generate a deterministic JSON report and fail on validation errors:

```powershell
dotnet run --project src/Ntilde.Conformance/Ntilde.Conformance.csproj -- --validate --report artifacts/vt-conformance/vt-conformance-report.json
```

Validate that the shipped embedded app artifact is still current:

```powershell
dotnet run --project src/Ntilde.Conformance/Ntilde.Conformance.csproj -- --validate --check-report src/Ntilde.App/Resources/vt-conformance-report.json
```

Defaults:

- repo root: current working directory
- matrix path: `docs/vt_coverage_matrix.md`
- output: stdout unless `--report` is provided

## Report Format

The tool emits a stable JSON document with:

- `schemaVersion`
- `matrixPath`
- `matrixSha256`
- `summary`
- `sections`
- `rows`
- `errors`
- `warnings`

Each row includes:

- section title
- feature name
- status text
- raw evidence text
- normalized evidence kinds
- extracted repo-linked evidence paths
- ownership text
- known deviations text
- source line number

No timestamps are included, so identical matrix content produces identical report bytes.

## Validation Rules

Hard failures:

- feature tables must have consistent markdown columns
- feature rows must use a known status symbol
- `✅ Supported` rows must declare automated evidence
- linked evidence paths must exist in the repo
- `🚫 Won’t support` rows must include a rationale

Warnings:

- `✅ Supported` rows that mention evidence generically but do not yet link a concrete repo path

Warnings are included in the report but do not fail CI.

## Capability Contract

`src/Ntilde.VtContract/vt-capabilities.json` is the machine-readable source
of truth for sequences promoted into the capability contract. Each supported
entry names:

- the CSI key and mnemonic exposed to consumers
- the exact coverage-matrix feature row
- a repository evidence path
- an executable contract-case identifier

`tests/Ntilde.VT.Tests/VtCapabilityContractTests.cs` maps those case
identifiers to parser assertions. A supported entry without an executable case
fails the VT test suite. The conformance tool independently rejects missing or
duplicated matrix rows, status mismatches, missing evidence files, and matrix
rows that do not link the declared evidence. MCP VT explanations read the same
catalog, so documentation, tooling, and tested parser behavior share one claim.

Keep the catalog deliberately narrow: add a sequence only when its behavior is
covered across relevant defaults, bounds, invalid forms, and split-input parser
boundaries. The broader matrix remains the inventory for capabilities that have
not yet been promoted into this executable contract.

## CI

`.github/workflows/vt-conformance.yml` runs the validator on:

- `docs/vt_coverage_matrix.md`
- `src/Ntilde.VtContract/vt-capabilities.json`
- `tests/Ntilde.VT.Tests/VtCapabilityContractTests.cs`
- conformance tooling source changes
- workflow/doc updates for this tooling

It uploads the JSON report as an artifact for PR review and push runs.

## Shipped App Artifact

Ntilde ships an embedded copy of the generated report at:

- `src/Ntilde.App/Resources/vt-conformance-report.json`

Regenerate it from the canonical matrix with:

```powershell
dotnet run --project src/Ntilde.Conformance/Ntilde.Conformance.csproj -c Release -- --report src/Ntilde.App/Resources/vt-conformance-report.json
```

Verify that it has not drifted with:

```powershell
dotnet run --project src/Ntilde.Conformance/Ntilde.Conformance.csproj -c Release -- --validate --check-report src/Ntilde.App/Resources/vt-conformance-report.json
```

This keeps the main app lightweight:

- the app does not parse `docs/vt_coverage_matrix.md` at runtime
- the app does not probe terminal behavior at runtime
- `--vt-report --json` emits the embedded artifact bytes
- `--vt-report` prints a concise summary derived from the embedded artifact

When `docs/vt_coverage_matrix.md` changes, regenerate and ship the embedded JSON in the same change.

The CI workflow also runs the `--check-report` command, so a stale embedded artifact fails the VT conformance job.

## Public CLI

User-facing commands:

```powershell
Ntilde.Cli --vt-report
Ntilde.Cli --vt-report --json
```

Default output is a short summary:

- matrix path
- support-status counts
- linked-evidence counts
- validation counts

`--json` prints the full machine-readable report.

Implementation note:

- `Ntilde.exe` is the normal GUI entry point
- `Ntilde.Cli.exe` is the console-side executable for VT-report and other headless tooling
- on Windows, interactive shell use should prefer `Ntilde.Cli.exe` rather than the GUI binary

## Intentional Limitations

- The parser only reads markdown tables that include both `Status` and `Evidence` columns.
- The tool does not infer support from source code; catalog-owned claims are validated against executable evidence, while the matrix remains the broader inventory.
- Ownership paths are reported verbatim and are not path-validated.
- Generic evidence text such as `Replay` or `Unit/Replay` is accepted for `✅ Supported` rows, but reported as a warning until concrete links are added.