# ucp-schema

CLI and library for [Universal Commerce Protocol](https://ucp.dev) (UCP) schemas.

UCP defines an open extensibility protocol on top of JSON Schema. Agents negotiate capabilities at runtime via self-describing payloads, extensions compose dynamically via `allOf`, and `ucp_request`/`ucp_response` annotations control field visibility per direction and operation. This tool implements UCP's composition and resolution pipeline: compose capability schemas, resolve annotations into standard JSON Schema, and validate payloads.

## How It Works

The CLI exposes a progressive pipeline. Each command runs it up to its named step:

```
  ┌───────────────────┐     ┌───────────────────┐     ┌───────────────────┐
  │      compose      │ ──▶ │      resolve      │ ──▶ │     validate      │
  └───────────────────┘     └───────────────────┘     └───────────────────┘
   merge capability          apply annotations           check payload
   schemas into one          for direction + op           against schema
```

Like `gcc -E` (preprocess only) vs `gcc` (full build), each command runs the pipeline to a different depth: `compose` stops after merging capability schemas, `resolve` applies annotations, and `validate` runs through to payload checking. When the input is a self-describing payload, earlier stages run automatically. `lint` is independent static analysis.

For example, a field annotated with `"ucp_request": {"create": "omit", "update": "required"}` disappears from create schemas but becomes required on update — one source schema, different views per operation. See [Visibility Rules](#visibility-rules) for the full worked example.

**"I want to..."**

| Goal                                                | Command                                                         |
| --------------------------------------------------- | --------------------------------------------------------------- |
| Inspect the composed schema (annotations preserved) | `compose payload.json --schema-local-base ./schemas --pretty`   |
| Get JSON Schema for an operation                    | `resolve payload.json --op read --schema-local-base ./schemas`  |
| Resolve a single schema file (no composition)       | `resolve schema.json --request --op create`                     |
| Validate a payload end-to-end                       | `validate payload.json --op read --schema-local-base ./schemas` |
| Check schemas for errors before runtime             | `lint schemas/`                                                 |
| Debug what the pipeline is doing                    | Add `--verbose` to any command                                  |

## Installation

```bash
# Install from crates.io
cargo install ucp-schema

# Or build from source
git clone https://github.com/Universal-Commerce-Protocol/ucp-schema
cd ucp-schema
cargo install --path .
```

## CLI Reference

### `compose` — Compose schemas from capabilities

Pure composition: merges capability schemas from a self-describing payload into one schema. Output preserves UCP annotations (no resolve step).

```bash
ucp-schema compose <payload> [options]

Options:
  --schema-local-base <dir>   Local directory for schema resolution
  --schema-remote-base <url>  URL prefix to strip when mapping to local (see Concepts > Local Resolution)
  --pretty                    Pretty-print JSON output
  --output <path>             Write to file instead of stdout
  --verbose, -v               Print pipeline stages to stderr
```

`compose` does not accept `--request`/`--response`/`--op` — those belong to `resolve` and `validate`.

```bash
# Inspect the merged schema before resolution
ucp-schema compose response.json --schema-local-base ./schemas --pretty

# Save for debugging
ucp-schema compose response.json --schema-local-base ./schemas --output composed.json
```

### `resolve` — Generate operation-specific schema

Accepts a schema file or a self-describing payload. When given a payload, automatically composes schemas from capabilities before resolving.

```bash
# Schema input (direction required)
ucp-schema resolve <schema> --request|--response --op <operation> [options]

# Payload input (direction auto-inferred)
ucp-schema resolve <payload> --op <operation> --schema-local-base <dir> [options]

Options:
  --request / --response      Direction (required for schema input, auto-inferred for payloads)
  --op <operation>            Operation: create, read, update, complete
  --pretty                    Pretty-print JSON output
  --output <path>             Write to file instead of stdout
  --bundle                    Inline external $ref pointers (schema input only; payloads bundle automatically)
  --schema-local-base <dir>   Local directory for schema resolution
  --schema-remote-base <url>  URL prefix to strip when mapping to local
  --strict                    Inject additionalProperties: false (see Concepts > Strict Mode)
  --verbose, -v               Print pipeline stages to stderr
```

```bash
# Schema file → resolved schema
ucp-schema resolve checkout.json --request --op create --pretty

# Self-describing payload → auto-compose, auto-detect direction, resolve
ucp-schema resolve response.json --op read --schema-local-base ./schemas

# Bundle external $refs into a self-contained schema
ucp-schema resolve checkout.json --request --op create --bundle --pretty

# Bundle a 3P extension with absolute URL refs (maps URLs to local copies)
ucp-schema resolve my_extension.json --response --op read --bundle \
  --schema-remote-base "https://ucp.dev/schemas" \
  --schema-local-base ./local-ucp-schemas

# Bundle a 3P extension with absolute URL refs (fetches from network)
ucp-schema resolve my_extension.json --response --op read --bundle

# Resolve from URL
ucp-schema resolve https://ucp.dev/schemas/checkout.json --request --op create
```

### `validate` — Validate payload against resolved schema

```bash
ucp-schema validate <payload> --op <operation> [options]

Options:
  --schema <path|url>          Explicit schema (skips self-describing detection)
  --profile <path|url>         Agent profile (REST request pattern)
  --request / --response       Direction (required with --schema, auto-detected otherwise)
  --op <operation>             Operation: create, read, update, complete
  --schema-local-base <dir>    Local directory to resolve schema URLs
  --schema-remote-base <url>   URL prefix to strip when mapping to local
  --strict                     Reject unknown fields (see Concepts > Strict Mode)
  --json                       Machine-readable JSON output
  --verbose, -v                Print pipeline stages to stderr
```

The validator auto-detects how to find the schema based on what flags you provide and what metadata the payload contains (see [Validation Modes](#validation-modes) in Concepts):

| Pattern                        | Command                                                       | Schema Source           | Direction |
| ------------------------------ | ------------------------------------------------------------- | ----------------------- | --------- |
| **Response** (self-describing) | `validate response.json --op read`                            | `ucp.capabilities` URLs | Auto      |
| **JSONRPC request**            | `validate envelope.json --op create`                          | `meta.profile` URL      | Auto      |
| **REST request**               | `validate payload.json --profile profile.json --op create`    | `--profile` URL         | Request   |
| **Explicit schema**            | `validate payload.json --schema s.json --request --op create` | `--schema`              | Specified |

```bash
# Self-describing response
ucp-schema validate response.json --op read --schema-local-base ./schemas

# Explicit schema
ucp-schema validate order.json --schema checkout.json --request --op create

# Machine-readable output for CI
ucp-schema validate order.json --schema checkout.json --request --op create --json
# → {"valid":true}
# → {"valid":false,"errors":[{"path":"","message":"..."}]}
```

Exit codes: `0` valid, `1` validation failed, `2` schema error, `3` file/network error.

### `lint` — Static analysis of schema files

Catch schema errors before runtime.

```bash
ucp-schema lint <path> [options]

Options:
  --format <text|json>  Output format (default: text)
  --strict              Treat warnings as errors
  --quiet, -q           Only show errors, suppress progress
```

| Code | Category    | Issue                                                          | Severity |
| ---- | ----------- | -------------------------------------------------------------- | -------- |
| E001 | Syntax      | Invalid JSON                                                   | Error    |
| E002 | References  | `$ref` to missing file                                         | Error    |
| E003 | References  | `$ref` to missing anchor (`#/$defs/foo`)                       | Error    |
| E004 | Annotations | Invalid visibility value or schema transition                  | Error    |
| E005 | Annotations | Invalid `ucp_*` type (must be string or object)                | Error    |
| E006 | Requires    | Invalid `requires` structure (wrong types, bad version format) | Error    |
| E007 | Requires    | `requires.capabilities` key not found in `$defs`               | Error    |
| W002 | Hygiene     | Missing `$id` field                                            | Warning  |
| W003 | Hygiene     | Unknown operation in annotation (e.g., `{"delete": "omit"}`)   | Warning  |
| W004 | Requires    | Version constraint has `min` > `max`                           | Warning  |
| W005 | Requires    | Unknown key in `requires` or version constraint                | Warning  |

```bash
# Lint a directory of schemas
ucp-schema lint schemas/

# CI-friendly: fail on warnings, JSON output
ucp-schema lint schemas/ --strict --format json
```

Exit codes: `0` passed, `1` errors found, `2` path not found.

<details>
<summary>JSON output format</summary>

```json
{
  "path": "schemas/",
  "files_checked": 5,
  "passed": 4,
  "failed": 1,
  "errors": 1,
  "warnings": 2,
  "results": [
    {
      "file": "checkout.json",
      "status": "error",
      "diagnostics": [
        {
          "severity": "error",
          "code": "E002",
          "path": "/properties/buyer/$ref",
          "message": "file not found: types/buyer.json"
        }
      ]
    }
  ]
}
```

</details>

## Concepts

### Visibility Rules

`ucp_request` and `ucp_response` annotations control which fields appear in the resolved schema. Given a schema where `id` is server-generated:

```json
{
  "type": "object",
  "properties": {
    "id": {
      "type": "string",
      "ucp_request": { "create": "omit", "update": "required" }
    },
    "name": { "type": "string" }
  }
}
```

Resolving for `--request --op create` removes `id` — clients don't send server-generated fields:

```json
{
  "type": "object",
  "properties": {
    "name": { "type": "string" }
  }
}
```

Resolving for `--request --op update` makes `id` required — you must specify which resource to update:

```json
{
  "type": "object",
  "properties": {
    "id": { "type": "string" },
    "name": { "type": "string" }
  },
  "required": ["id"]
}
```

Annotations are stripped; output is standard JSON Schema.

**Resolution rules:**

| Value                                                                   | Effect on Properties | Effect on Required Array |
| ----------------------------------------------------------------------- | -------------------- | ------------------------ |
| `"omit"`                                                                | Field removed        | Field removed            |
| `"required"`                                                            | Field kept           | Field added              |
| `"optional"`                                                            | Field kept           | Field removed            |
| (no annotation)                                                         | Field kept           | Unchanged                |
| `{ "transition": { "from", "to", "description" } }` (schema transition) | Matches `from` value | Matches `from` value     |

Annotations can be **shorthand** (all operations) or **per-operation**, and request/response are independent:

```json
{
  "id": {
    "type": "string",
    "ucp_request": { "create": "omit", "update": "required" },
    "ucp_response": "required"
  },
  "status": {
    "type": "string",
    "ucp_request": "omit"
  }
}
```

Valid operations: `create`, `read`, `update`, `complete`.

#### Schema transitions

Use a **schema-transition object** to signal a field contract will change, with a human-readable reason:

```json
{
  "transition": {
    "from": "required",
    "to": "omit",
    "description": "Legacy id will be removed in v2; use resource_id instead."
  }
}
```

- **`from`** and **`to`** must be one of: `"omit"`, `"optional"`, `"required"`, and must be **distinct** (same value for both is invalid).
- **`description`** is required and should explain the change and what to do instead.

During the transition period the resolved schema **uses the `from` value** as the field's visibility, so previous implementers are not immediately affected. The resolver emits the schema-transition context into the output schema:

- **`x-ucp-schema-transition`**: `{ "from", "to", "description" }` on the property for tooling and docs.
- **`deprecated`: true** on the property only when the field is being **removed** (`to` is `"omit"`).

**Example: Removing a required field**

```json
// Phase 1: Field is required
{ "ucp_request": { "update": "required" } }

// Phase 2: Schema transition (field stays required in resolved schema; tooling offers warnings)
{ "ucp_request": { "update": { "transition": { "from": "required", "to": "omit", "description": "Will be removed in v2." } } } }

// Phase 3: Remove
{ "ucp_request": { "update": "omit" } }
```

**Shorthand schema transition** (same transition for all operations):

```json
{
  "ucp_request": {
    "transition": {
      "from": "required",
      "to": "omit",
      "description": "Removed in v2."
    }
  }
}
```

### Schema Composition

UCP payloads are self-describing — they embed `ucp.capabilities` metadata declaring which schemas apply. This lets multiple capability schemas compose into one:

```json
{
  "ucp": {
    "capabilities": {
      "dev.ucp.shopping.checkout": [{
        "version": "2026-01-11",
        "schema": "https://ucp.dev/schemas/shopping/checkout.json"
      }],
      "dev.ucp.shopping.discount": [{
        "version": "2026-01-11",
        "schema": "https://ucp.dev/schemas/shopping/discount.json",
        "extends": "dev.ucp.shopping.checkout"
      }]
    }
  },
  "id": "chk_123",
  "discounts": [...]
}
```

**How composition works:**

1. **Root capability** — one capability has no `extends`, providing the base schema
2. **Extensions** — capabilities with `extends` add fields to the root
3. **Merge** — extensions define their additions in `$defs[root_capability_name]`; the tool composes them via `allOf`

**Graph rules:** exactly one root capability (no `extends`), all `extends` targets must exist in capabilities, all extensions must transitively reach the root.

**Schema authoring for extensions:**

Extension schemas define their additions in `$defs` keyed by the root capability name:

```json
{
  "$id": "https://ucp.dev/schemas/shopping/discount.json",
  "name": "dev.ucp.shopping.discount",
  "$defs": {
    "dev.ucp.shopping.checkout": {
      "allOf": [
        { "$ref": "checkout.json" },
        {
          "type": "object",
          "properties": {
            "discounts": { "type": "array" }
          }
        }
      ]
    }
  }
}
```

**Version constraints (`requires`):**

Extension schemas can declare which protocol and capability versions they depend on. Composition fails if the declared constraints are not satisfied by the profile's versions:

```json
{
  "$id": "https://acme.com/ucp/schemas/loyalty.json",
  "name": "com.acme.shopping.loyalty",
  "requires": {
    "protocol": { "min": "2026-01-23" },
    "capabilities": {
      "dev.ucp.shopping.checkout": { "min": "2026-06-01" }
    }
  },
  "$defs": {
    "dev.ucp.shopping.checkout": { ... }
  }
}
```

Each constraint is an object with a required `min` and optional `max` (both inclusive). Keys in `requires.capabilities` must be a subset of `$defs` keys. Schemas without `requires` compose as before — the composer asserts compatibility.

### Validation Modes

The validator supports four patterns for discovering which schema to validate against.

**Response (self-describing)** — The payload's `ucp.capabilities` declares schema URLs. Direction is auto-detected as response:

```bash
ucp-schema validate response.json --op read --schema-local-base ./schemas
```

**JSONRPC request** — The envelope has `meta.profile` at root, with the payload nested under the capability short name (e.g., `checkout`). The validator fetches the profile, extracts capabilities, extracts the nested payload, then composes and validates:

```json
{
  "meta": { "profile": "https://agent.example.com/.well-known/ucp" },
  "checkout": { "line_items": [...] }
}
```

```bash
ucp-schema validate envelope.json --op create
```

**REST request (`--profile`)** — The profile URL comes via flag (equivalent to an HTTP header in production). The payload is the raw object, not wrapped in an envelope:

```bash
ucp-schema validate raw-checkout.json --profile agent-profile.json --op create
```

The `--profile` flag implies `--request` direction.

**Explicit schema** — Bypass self-describing metadata entirely. Requires explicit `--request` or `--response`:

```bash
ucp-schema validate order.json --schema checkout.json --request --op create
```

#### Local Resolution

When working offline or testing schema changes, `--schema-local-base` maps schema URL paths to local files. This applies to self-describing payloads (capability schema URLs), explicit `--schema` input, and `--bundle` mode (absolute URL `$ref` values in schema files):

```bash
# Schema URL: https://ucp.dev/schemas/shopping/checkout.json
# Path extracted: /schemas/shopping/checkout.json
# Local file: ./local/schemas/shopping/checkout.json
ucp-schema validate response.json --schema-local-base ./local --op read
```

When schema URLs have a prefix that doesn't match your local directory layout, `--schema-remote-base` strips it:

```bash
# URL:   https://ucp.dev/draft/schemas/shopping/checkout.json
# Strip: https://ucp.dev/draft
# Local: ./site/schemas/shopping/checkout.json
ucp-schema validate response.json \
  --schema-local-base ./site \
  --schema-remote-base "https://ucp.dev/draft" \
  --op read
```

### Bundling

Schemas often use `$ref` to reference external files. The `--bundle` flag inlines all external references into a self-contained schema:

```bash
ucp-schema resolve checkout.json --request --op create --bundle --pretty
```

Bundling applies to **schema file input only**. When resolving payloads, composition already handles fetching and merging external schemas.

How it works:

- File refs (`"$ref": "types/buyer.json"`) are loaded and inlined
- Fragment refs (`"$ref": "types/common.json#/$defs/address"`) navigate to the target definition
- Internal refs in external files (`"$ref": "#/$defs/foo"`) resolve against their source file
- Self-referential types (`"$ref": "#"`) are preserved (can't be inlined)
- Circular references are detected and reported as errors

### Strict Mode

By default, validation allows unknown fields — payloads may contain fields from capabilities the validator hasn't seen, and forward compatibility requires tolerating them. For closed systems or catching typos, `--strict` injects `additionalProperties: false` into all object schemas:

```bash
ucp-schema validate order.json --schema schema.json --request --op create --strict
ucp-schema resolve schema.json --request --op create --strict --pretty
```

**Warning:** Strict mode conflicts with `allOf` composition. Each `allOf` branch validates independently and rejects properties from other branches. Use default (non-strict) mode for composed schemas.

## Debugging with `--verbose`

All commands accept `--verbose` (or `-v`) to print pipeline stages to stderr:

```bash
$ ucp-schema resolve response.json --op read --schema-local-base ./schemas --verbose
[load] reading response.json
[detect] payload with 3 capabilities (1 root, 2 extensions)
[detect]   root dev.ucp.shopping.checkout → https://ucp.dev/schemas/shopping/checkout.json
[detect]   ext dev.ucp.shopping.discount → https://ucp.dev/schemas/shopping/discount.json
[detect]   ext dev.ucp.shopping.fulfillment → https://ucp.dev/schemas/shopping/fulfillment.json
[compose] composing schemas from payload capabilities
[resolve] resolving for response/read
```

Verbose output goes to stderr; JSON output on stdout is unaffected.

## More Information

See [FAQ.md](./FAQ.md) for common questions about validator behavior, design decisions, and edge cases.

## License

Apache-2.0