# 03 — Architecture

fabric-emulator is a **local Microsoft Fabric runtime**: the control-plane
contract (workspaces, items, RBAC, git, jobs, LROs, Fabric Core MCP), a real
OneLake ADLS/Blob data plane, and attached engines for Spark, T-SQL, pipelines,
Airflow, KQL, and Eventstream. It is the sibling of
[entra-emulator](https://github.com/calvinchengx/entra-emulator): where that
project is an Entra ID STS, this one is the Fabric surface that *consumes*
Entra tokens.

## Version grounding

Fabric is evergreen SaaS: no product version exists, and
[`MicrosoftDocs/fabric-docs`](https://github.com/MicrosoftDocs/fabric-docs) has
**no tags or releases** (a continuously published `main`, ~100+ commits/week).
The only contract version Microsoft exposes is the **`/v1` path segment** of the
REST API — that is what this emulator targets.

For clean-room reproducibility we therefore pin the docs **commit**, not a
version. All claims in this doc set were verified against:

> `MicrosoftDocs/fabric-docs @ 0d63906ac29d8e8befa42b13f3d1d31c0f92081a` (2026-07-10)

When re-auditing, diff the grounding files against this SHA
(`git diff 0d63906a.. -- docs/onelake/onelake-api-parity.md
docs/cicd/git-integration/git-automation.md docs/security/workspace-identity.md
docs/security/permission-model.md`) and bump the pin.

## The four-system model

A Fabric environment is four independent Azure products with four protocols.
Each stays a separate emulator, reached only over HTTP, because that is what the
real boundary is — and it is what lets any one of them be swapped for the live
service unchanged (see [21-real-fabric-toggle.md](21-real-fabric-toggle.md)):

1. **Entra ID** — issues tokens (service-principal client credentials for the
   Fabric audience; **workspace identities** = auto-managed app registrations +
   service principals). Emulated by **entra-emulator**.
2. **Azure Key Vault** — the secrets data plane. Fabric never holds the secret
   behind an **Azure Key Vault reference** connection: it stores a pointer and
   resolves it at use with a vault-audience token. Same for
   `notebookutils.credentials.getSecret()` inside a notebook. Emulated by
   **azure-keyvault-emulator**; fabric-emulator is a *client* of it
   ([internal/akv](../internal/akv/)).
3. **Azure Resource Manager** — the resource lifecycle. A Fabric **capacity** is
   an ARM resource (`Microsoft.Fabric/capacities`), created and deleted through
   `management.azure.com`, not through the Fabric REST API. Emulated by
   **arm-emulator**; fabric-emulator *reads* from it (below).
4. **The Fabric control plane** — `https://api.fabric.microsoft.com/v1/…`:
   workspace RBAC, item CRUD, item **definitions** (the CI/CD source format),
   git integration, deployment pipelines, long-running operations, and the
   workspace-identity *lifecycle orchestration*. Plus **OneLake**
   (`https://onelake.dfs.fabric.microsoft.com`), an ADLS-Gen2-shaped data plane.
   Emulated by **fabric-emulator**.

Keeping these as four composable emulators preserves single responsibility, and
the token flow is one-way: **entra ISSUES; fabric, keyvault and arm only
VALIDATE**, each against entra-emulator's JWKS, exactly as the real products
validate against Entra. No validator can mint a token.

**ARM capacities, on by default.** `FABRIC_ARM_URL` defaults to the
arm-emulator service in [`docker-compose.yml`](../docker-compose.yml), so a
capacity created with `PUT …/Microsoft.Fabric/capacities/{name}` appears on
`GET /v1/capacities` under the Fabric REST GUID ARM assigned at create. This
process polls `GET /_family/capacities` (the same localhost family channel the
Key Vault sibling uses for authorization); the seeded capacity stays alongside
whatever ARM contributes. Setting `FABRIC_ARM_URL=` empty opts out and leaves
the seed only, which is how the hermetic test paths run.

```mermaid
flowchart LR
    Client["Client / SDK (SP or user)"]
    subgraph fab["fabric-emulator"]
        direction TB
        API["/v1/workspaces/… — RBAC, CRUD, git, LRO, jobs; Livy/Spark data plane"]
        OL["onelake.dfs / .blob — ADLS-Gen2 + Blob (Delta commits)"]
        WH["TDS endpoint — T-SQL over FedAuth (internal/tds)"]
        Ident["workspace-identity lifecycle"]
    end
    subgraph entra["entra-emulator"]
        direction TB
        JWKS["/{tenant}/discovery/v2.0/keys"]
        Forge["token forge / MSI endpoint"]
    end
    subgraph kv["azure-keyvault-emulator"]
        direction TB
        Secrets["/secrets/{name} — data plane 7.4"]
    end
    subgraph arm["arm-emulator"]
        direction TB
        Caps["Microsoft.Fabric/capacities — ARM resource lifecycle"]
        Feed["/_family/capacities — localhost feed"]
    end
    subgraph engines["engine sidecars"]
        direction TB
        Spark["Spark agent (Livy) — default"]
        SQL["SQL Server (warehouse) — default"]
        KQL["kustainer (Eventhouse) — profile rti"]
        Kafka["Apache Kafka (Eventstream) — profile eventstream"]
    end
    Client -->|"Bearer (aud = fabric / storage)"| API
    Client -->|"Bearer (aud = storage)"| OL
    Client -->|"FedAuth (aud = database.windows.net)"| WH
    API -->|"verify (iss + aud + sig)"| JWKS
    Ident -->|"mint SP / identity tokens"| Forge
    API -->|"resolve AKV reference (aud = vault)"| Secrets
    Secrets -->|"verify (iss + aud + sig)"| JWKS
    Client -->|"Bearer (aud = management)"| Caps
    Caps -->|"verify (iss + aud + sig)"| JWKS
    Caps --> Feed
    API -.->|"poll (no token, localhost only)"| Feed
    API -.->|"native execution"| Spark
    WH -.->|"session splice"| SQL
    API -.->|"KQL relay"| KQL
    API -.->|"produce / consume"| Kafka
```

Every core service in [`docker-compose.yml`](../docker-compose.yml) appears in
that diagram, and `scripts/check_arch_services.py` fails the build if one stops
appearing. The check exists because this document described a *two*-system model
for as long as Key Vault had been a default service — the shape was a decision
and stayed true, but the **cast of services is a list**, and lists drift.

## Design principle: mirror entra-emulator

fabric-emulator deliberately reuses entra-emulator's stack and idioms so the two
form a coherent pair and the testing primitives carry over:

| Concern | Choice (same as entra-emulator) |
|---|---|
| Language / HTTP | Go, stdlib `net/http`, host-routed muxes |
| Storage | `modernc.org/sqlite` (pure-Go, no CGO) |
| Surface routing | `Host`-header router (`onelake.*` data plane; everything else = control plane + portal) |
| Determinism | Controllable **clock** (drives LRO completion) + **fault injection** |
| Portal | Svelte 5, `go:embed all:dist`, committed `dist` + CI drift guard |
| Docs site | Astro Starlight on GitHub Pages, pinned, `/docs` = source of truth |
| Distribution | GoReleaser: binaries, distroless Docker (GHCR), Homebrew, winget |
| Tests | Go unit/integration + real-SDK e2e matrix + Playwright mount smoke |
| License | Apache-2.0, clean-room from `fabric-docs` |

The **payoff of reuse**: entra-emulator's deterministic clock becomes Fabric's
LRO controller — a test can force any async operation to `Succeeded` instantly or
hold it `Running` to exercise polling. Real Fabric cannot do this.

## Token acceptance — the seam

Every `/v1/…` and OneLake request carries `Authorization: Bearer <jwt>`.
fabric-emulator validates:

- **Signature** against entra-emulator's JWKS (`{issuer}/discovery/v2.0/keys`),
  fetched once and cached, keyed by `kid`.
- **Issuer** equals the configured entra-emulator issuer.
- **Audience** ∈ {`https://api.fabric.microsoft.com`,
  `https://analysis.windows.net/powerbi/api`} for the control plane; the
  `Storage` audience (`https://storage.azure.com`) for OneLake.
- **Expiry / nbf** via the standard claim checks.

It then maps the token's `oid`/`appid`/`sub` to a **workspace role** for RBAC.
This mirrors entra-emulator's own `ValidateAccessToken` model; the only new work
is the audience set and the role lookup. Config: `--entra-issuer` +
`--entra-jwks-url` (or a discovery URL). It can point at a real tenant unchanged.

## Surfaces (host-routed)

| Host mux | Serves |
|---|---|
| `api.fabric.microsoft.com` | the `/v1` control plane (workspaces, items, RBAC, git, LRO, jobs, admin) + the Livy/Spark data plane (`…/livyapi/…`, high-concurrency sessions) |
| `onelake.dfs.fabric.microsoft.com` | ADLS-Gen2 (DFS) subset (filesystem = workspace, path = item/…) |
| `onelake.blob.fabric.microsoft.com` | the OneLake **Blob** dialect (Put Blob/Block, `If-None-Match:*` Delta put-if-absent) — what delta-rs / `object_store` use |
| a raw **TCP/TDS** listener (`--sql-tds-addr`) | the warehouse **T-SQL over TDS** endpoint — Entra FedAuth terminated, session spliced to a SQL Server sidecar (`internal/tds`) |
| any other host, at `/` | the Svelte operator portal (embedded; reads state via `/_emulator/portal/*`) |

See [07-control-plane-api.md](07-control-plane-api.md) for the endpoint catalog and wire
shapes, and [13-roadmap.md](13-roadmap.md) for what lands in each phase.

## Data model (SQLite)

One pure-Go SQLite database holds the entire state — workspaces, items and
their verbatim definition parts, RBAC, capacities, operations, jobs, git
remotes, OneLake blobs, and the workspace-identity link — with cascading
deletes matching the control plane's semantics. The full schema, seed, and
state enums live in [06-data-model-and-seed.md](06-data-model-and-seed.md).

## Long-running operations

Nearly every mutation returns `202 Accepted` with an `x-ms-operation-id` header
(what the documented automation scripts read), a `Location:
/v1/operations/{id}`, and `Retry-After`. Clients poll `GET /v1/operations/{id}`
until `Status` leaves {`NotStarted`, `Running`}. The emulator models this as a first-
class `operation` row whose `completeAt` is a function of the **controllable
clock**:

- default: completes on the next poll (fast, deterministic for CI);
- `--lro-delay` or per-request fault: stays `Running` for N virtual seconds;
- fault injection: forces `Failed` with a Fabric-shaped error body.

## Non-goals

Capacity/SKU **billing**, Purview audit, and real network/firewall enforcement.
Emulating engine *internals* is also out of scope.

Two entries left this list, because the paragraph below overtook them and a
non-goal that shipped is worse than one never written down: **Power BI
semantic-model evaluation** is a bounded DAX engine, measured against Power BI
Desktop itself; **KQL execution** runs on Microsoft's own Kusto engine behind an
opt-in profile. What stays out of scope is narrower than either heading was —
full DAX, where the evaluator errors rather than mis-evaluates outside its
pinned subset, and semantic-model *permissions*, since TMSL `roles` /
`tablePermissions` are not modelled ([07](07-control-plane-api.md)).

**Not a non-goal (any more): real compute.** The core Go binary stays a contract
emulator, but real engines attach as **opt-in sidecars** — so notebooks and
pipelines actually *run* (Spark-compatible Sail compute via a Livy agent), T-SQL runs for real over
TDS against a SQL Server sidecar, and Delta lands in OneLake — and where no real
engine can be attached, the surface returns an honest **501** rather than faking
a result. The principle is *never fake compute*, not *no compute*. See
[14-real-compute.md](14-real-compute.md).

## Decoupling from the sibling emulators

- Depends on entra-emulator **only over HTTP** (JWKS + issuer; plus a token-mint
  call for workspace identities — the shipped identity handshake). No shared process.
- Depends on azure-keyvault-emulator **only over HTTP** too, and only as an
  outbound client: `internal/akv` GETs `{vaultURI}/secrets/{name}` with a
  vault-audience bearer when a credential's `keyReference` (or sibling) is
  resolved. The
  secret value is returned to the caller and **never persisted** in the
  emulator's database — only the `{vaultUri, secretName}` pointer is — which is
  the property that makes credential-by-reference worth having.
- There is no global vault setting to point elsewhere: **each connection carries
  its own `vaultUri`**, as in real Fabric, so a connection naming a real vault
  works alongside one naming the emulator with no reconfiguration.
- For Go integration tests, it *may* import entra-emulator's public `emulator`
  package to run both in one process with no network — an ergonomics option, not
  a coupling requirement.
- Primary local composition: a sibling [`docker-compose.yml`](../docker-compose.yml)
  brings up both, fabric pre-wired to entra's issuer/JWKS. An auto-loaded
  [`docker-compose.override.yml`](../docker-compose.override.yml) additionally
  attaches real engines (Spark agent, SQL Server) by default — see
  [14-real-compute.md](14-real-compute.md).