# 21 — Design: one toggle between fabric-emulator and real Fabric

**Status: T0 + T1 + T2 shipped** (`python/fabric-target/`, CI `fabric-target`; conformance emulator leg on every push, real leg via the secret-gated `real-fabric` workflow). Goal: a user's Python functionality — SDK calls,
`fabric-cicd` pipelines, notebooks on the `notebookutils` shim, dbt projects,
plain `requests` — runs against **either** the local emulator family **or**
the real Fabric service, switched by **one setting**, with zero code edits.

## Why this is nearly free

Everything this project did to run *real clients unmodified against the
emulator* is exactly the machinery a toggle needs, pointed the other way:

- Every client is already parameterized by an **API root**
  (`FABRIC_API_ROOT_URL` / `DEFAULT_API_ROOT_URL`), a **token authority +
  credential** (azure-identity), a **storage endpoint** (OneLake), and a
  **vault URL** — because that's how the e2es aim them at localhost.
- The emulator deliberately speaks v2.0/JWKS/challenge auth exactly like
  production, so azure-identity, MSAL, and the SDKs cannot tell the
  difference — only the endpoints and the credential values change.

So the toggle is not an emulation feature; it is **one resolver** that turns a
target name into a coherent set of endpoints + credentials, plus guardrails
for the places the two worlds genuinely differ.

Where your artifacts persist — definition parts, the `{name}.{type}/` +
`.platform` source format, and the hard line that OneLake data is never in Git —
is [docs/46](46-artifact-persistence.md). This document resolves *where to talk*;
that one says *what you are storing*, and both have to hold for a run to be
portable.

## The contract

One switch: `FABRIC_TARGET=emulator | real` (default `emulator`).

| Resolved value | `emulator` (zero-config defaults) | `real` (from standard env) |
|---|---|---|
| API root | `https://localhost:9443/v1` | `https://api.fabric.microsoft.com/v1` |
| Token authority | entra-emulator (`https://localhost:8443/{tenant}`) | `https://login.microsoftonline.com/{AZURE_TENANT_ID}` |
| Credential | seeded daemon SP (`cccccccc-…0002` / `daemon-app-secret`) | `AZURE_CLIENT_ID`/`AZURE_CLIENT_SECRET`, or `DefaultAzureCredential` (az CLI, managed identity, browser) |
| OneLake | `https://localhost:9443` + Host/`--resolve` (or `az://` via Sail) | `https://onelake.dfs.fabric.microsoft.com` |
| Key Vault | azure-keyvault-emulator (`https://localhost:8444`) | the user's real vault URI |
| TLS verify | off (self-signed family certs) | on |
| Workspace | by id **or name** against the emulator | **by name** (`FABRIC_WORKSPACE`), resolved to the real GUID at startup |

Ids are the one thing that can never match across targets — so the contract
is **name-based**: user code holds workspace/item display names; the resolver
translates to GUIDs per target.

### Variable names, and why there are two sets

Emulator-mode knobs are named `FABRIC_*`/`*_EMULATOR_URL`; real mode reads the
standard `AZURE_*` names. A consumer driving **both** targets from one compose
file writes the Azure names, because real mode leaves it no choice — so
emulator mode accepts them as aliases rather than making the same URL be
declared twice:

| Resolved value | Preferred | Also accepted |
|---|---|---|
| Fabric API | `FABRIC_EMULATOR_URL` | `FABRIC_URL` |
| Entra | `ENTRA_EMULATOR_URL` | `ENTRA_URL` |
| Key Vault | `VAULT_EMULATOR_URL` (emulator), `FABRIC_VAULT_URL` (real) | `AZURE_KEY_VAULT_URL` |
| Tenant | `FABRIC_TENANT` | `AZURE_TENANT_ID` |
| Client id / secret | `FABRIC_CLIENT_ID` / `FABRIC_CLIENT_SECRET` | `AZURE_CLIENT_ID` / `AZURE_CLIENT_SECRET` |

The `FABRIC_*` name wins when both are set, so the aliases change nothing for
anyone already using them. **They do widen one hole, which is closed
explicitly:** because emulator mode now reads `AZURE_CLIENT_SECRET`, a shell
left over from a local run could carry the seeded daemon into real mode. Real
mode therefore refuses the seeds **by value**, not by variable name — see
"Seeded values never leak into real mode" below.

## The family — trust direction constrains the toggle

Tokens flow one way: entra (or real AAD) **issues**; fabric and keyvault only
**validate**. That means the toggle is not three independent switches — only
chains rooted in a single issuer are coherent:

| Combination | Works? | Why |
|---|---|---|
| All-emulator | ✅ | the default family — one trust chain rooted in entra-emulator |
| All-real | ✅ | real AAD issues; real Fabric + real Key Vault validate |
| Real AAD + emulated fabric/keyvault (**hybrid**) | ✅ | both emulators already accept any configured issuer — `FABRIC_ENTRA_ISSUER` / `KV_ENTRA_ISSUER` can point at `login.microsoftonline.com/{tenant}` today |
| Emulated entra + real fabric/keyvault | ❌ | real Microsoft services will never trust the emulator's JWKS |

So `FABRIC_TARGET` flips the whole chain; the one supported refinement is a
**hybrid profile** (`FABRIC_TARGET=emulator` + `AUTH_TARGET=real`) for teams
with a real AAD app registration but no Fabric capacity.

Per member, "real" resolves as:

- **entra-emulator → real Entra ID.** azure-identity *is* the toggle: the
  resolver builds `ClientSecretCredential(..., authority=<entra-emulator>)`
  with the seeded SP in emulator mode, and **exactly
  `DefaultAzureCredential()`** in real mode. Its chain order does the rest
  with no branching of ours: explicit `AZURE_TENANT_ID`/`AZURE_CLIENT_ID`/
  `AZURE_CLIENT_SECRET` win when set (CI, service contexts); otherwise
  **`az login` wins** — tokens are minted from the developer's own CLI
  session (delegated identity; workspace RBAC and Conditional Access apply
  to *them*), refresh handled by azure-identity re-invoking `az`. All three
  family scopes (Fabric, Storage, Vault) mint through the CLI. Non-Python
  tools follow the same split via the env emitter: `fabric-cicd` already
  uses `DefaultAzureCredential` internally, `dbt-fabric` takes
  `authentication: CLI`, `azcopy` takes `AZCOPY_AUTO_LOGIN_TYPE=AZCLI`.
- **azure-keyvault-emulator → the user's actual vault.** Key Vault's
  **challenge-based auth does the discovery**: the SDK hits the vault, reads
  the 401 `WWW-Authenticate` challenge naming the authority, and follows it —
  the AKV emulator implements that same challenge advertising entra-emulator's
  authority, so identical `SecretClient(vault_url, credential)` code walks
  either chain. The resolver supplies only the **vault URL** per target
  (`https://localhost:8444` and its default vault vs
  `https://{name}.vault.azure.net`). Vault **names** are the cross-target
  contract, exactly like workspace names. A connection references the vault
  by **account name**, not URL, so that shape is identical on both sides
  while the URL each target composes differs.
- **Seeded values never leak into real mode.** Tenant `11111111-…`, the
  daemon SP, and `daemon-app-secret` are emulator-mode defaults only; in real
  mode the resolver requires a **real credential source** — env SP vars *or*
  a live `az login` (the `DefaultAzureCredential` chain probe) — and refuses
  to fall back to seeds. No source found → fail at startup with "run
  `az login` or set AZURE_* credentials", never a silent seed. Since emulator
  mode accepts the `AZURE_*` aliases, "not a seed" is checked **by value**:
  if `AZURE_TENANT_ID`, `AZURE_CLIENT_ID` or `AZURE_CLIENT_SECRET` *equals* a
  seeded constant, real mode refuses to construct at all. A leftover shell
  fails loudly at startup instead of authenticating against nothing.

## Deliverable A — `fabric_target` (Python helper, `python/fabric-target/`)

**Published.** The release workflow builds it beside the fixture wheels and
attaches it to the GitHub Release, stamped at the release version:

```
uv pip install "fabric-target[real,sessions] @ \
  https://github.com/calvinchengx/fabric-emulator/releases/download/vX.Y.Z/fabric_target-X.Y.Z-py3-none-any.whl"
```

The extras matter. The core is stdlib-only and imports `azure-identity` (real
credentials) and `requests` (`session()`) lazily, so a bare install succeeds
and then fails at the first authenticated call.

*Why it is published at all:* while it was not, a consumer could only restate
the contract, and `contoso-data-platform` did — losing the
`DefaultAzureCredential` branch in the copy, so its real target demanded a
client secret. That made `az login` unusable, a managed identity unusable, and
a Fabric notebook — which has no client secret to give — unable to run the
platform at all. The emulator never noticed, because it does not care which
identity shows up. A contract that must be copied is a contract that gets its
untested branch wrong.

Small sibling of the `notebookutils` shim, same env-driven style:

```python
from fabric_target import target

t = target()                     # reads FABRIC_TARGET, resolves the profile
t.credential                     # azure.identity credential for this target
t.session()                      # requests.Session: base URL, bearer auth,
                                 # verify flag, retry-on-429 — same object
                                 # whichever target is active
ws = t.workspace("analytics")    # name → id, either target
t.session().post(f"/workspaces/{ws.id}/items", json={...})

t.onelake                        # adlfs/azure-storage-blob-ready endpoint + credential
t.vault_url                      # keyvault base for this target
t.emulator_only("clock freeze")  # raises TargetError under FABRIC_TARGET=real
```

Implementation notes: authority override is plain azure-identity
(`ClientSecretCredential(..., authority=...)` — the e2es already prove entra
works as an authority); `verify=False` only in emulator mode; the profile is
resolved once and printable (`python -m fabric_target show`).

## Deliverable B — env emitter (non-Python tools, one command)

The same resolver, exported for tools that only read env — `fabric-cicd`,
dbt profiles, azcopy, the `notebookutils` shim, `fab` CLI:

```bash
eval "$(python -m fabric_target env real)"      # or: emulator
```

Emits the full coherent set: `FABRIC_API_ROOT_URL`, `DEFAULT_API_ROOT_URL`,
`AZURE_TENANT_ID`/`AZURE_CLIENT_ID`/…, `NOTEBOOKUTILS_*` (mapped onto real
endpoints in real mode), `REQUESTS_CA_BUNDLE`/`SSL_CERT_FILE` handling, and —
emulator mode only — the DNS-pin guidance for hostname-strict tools
([05-tls-and-hosts.md](05-tls-and-hosts.md)).

## Guardrails — where the worlds differ on purpose

The toggle must make these differences *loud*, not paper over them:

1. **Emulator-only surfaces hard-fail in real mode.** `/_emulator/*` (clock,
   faults, portal data), forged tokens, seeded principals: the helper's
   `emulator_only()` raises `TargetError("clock control does not exist on
   real Fabric")` rather than letting a test silently no-op.
2. **Time is real.** No frozen clock: LROs poll for real minutes; the helper's
   `session()` bakes in `Retry-After`-honoring polling either way, so code
   written against the emulator's instant LROs still behaves.
3. **Real mode costs money and touches real state.** Destructive verbs
   (workspace/item DELETE) require `FABRIC_TARGET_ALLOW_DESTRUCTIVE=1` in
   real mode; the resolver refuses to start in real mode without an explicit
   `FABRIC_WORKSPACE` scope, so nothing ever enumerates a whole tenant.
4. **Throttling exists.** 429/`Retry-After` handling is on by default in the
   session (the emulator can rehearse it via fault injection).
5. **RBAC is real.** The SP needs actual workspace roles; the resolver's
   startup probe (`GET /workspaces` + the scoped workspace) fails fast with
   a "grant your SP access" message instead of 403s mid-run.

## Running an example against a real tenant, step by step

What a person actually types, and what happens at each boundary. The example is
not edited: everything below is environment.

```bash
az login                                   # as YOURSELF for a trial (see below)
```

```bash
curl -s -H "Authorization: Bearer $(az account get-access-token \
  --resource https://api.fabric.microsoft.com -o tsv --query accessToken)" \
  https://api.fabric.microsoft.com/v1/capacities
```

```bash
cd examples/medallion-pyspark && FABRIC_TARGET=real FABRIC_WORKSPACE=contoso-analytics FABRIC_CAPACITY_ID=<capacity-guid> uv run --frozen python provision.py
```

Then, with a Key Vault, the next three steps in order: `secret.py`,
`extract_load.py`, `bronze.py` — adding
`FABRIC_VAULT_URL=https://<vault>.vault.azure.net`.

| Step | Real Fabric | Note |
|---|---|---|
| `provision.py` | workspace + lakehouse + warehouse + workspace identity | needs `FABRIC_CAPACITY_ID`; the resolver refuses without it rather than creating a capacity-less workspace whose every item then fails |
| `secret.py` | secret in your vault + an AKV-reference connection | the workspace identity needs *get* on the vault |
| `extract_load.py` | `notebookutils.credentials.getSecret`, then ~170 MB into `Files/landing` | the brokered path, unchanged from local |
| `bronze.py` | deploys the Notebook and DataPipeline definitions, runs the pipeline | the notebook activity executes on the workspace's **starter pool**; its body is already portable (`spark`, `abfs://<ws>@onelake.dfs.fabric.microsoft.com/...`) |
| `engine.py` | **skips** | Fabric ran the notebook itself |
| `silver.py` | deploys `silver.Notebook`, submits `RunNotebook`, verifies the Delta tables with delta-rs | the transform runs on the **starter pool**; this file never touches Spark |
| `reflect.py` | queries the lakehouse SQL analytics endpoint | address discovered from `sqlEndpointProperties` |
| `gold.py`, `dq_gate.py` | dbt-fabric builds the star in the warehouse | address discovered from `properties.connectionString` |
| `semantic_model.py` | publishes and queries the model over `executeQueries` | — |
| `lineage.py` | **skips** | the flow graph is the emulator's own record; Purview is Fabric's answer and a different integration |

**Verified against a real trial on 2026-08-11**, and every one of these was a
code-reading until then:

```
==> provisioned workspace=f2c82a4e-… lakehouse=450d5027-… warehouse=c6c4ba99-…
lakehouse  server=<opaque>-<opaque>.datawarehouse.fabric.microsoft.com
           database='lake' encrypt=True endpoint_id=803c8e33-…
warehouse  same server, database='dw' encrypt=True endpoint_id=None
```

Four things that only a real run could establish:

1. **`FABRIC_CAPACITY_ID` works**: the workspace came up with the trial capacity
   assigned, from an unmodified example.
2. **A Warehouse create is ASYNCHRONOUS on real Fabric** — 202 with an operation
   and no body, where the emulator answers 201 with the item. `provision.py` read
   `None["id"]` and failed three calls after the assumption that caused it. Every
   create now resolves an operation when one is offered (`post_and_wait`).
3. **`AZURE_TENANT_ID` did not reach the `az` CLI credential.**
   `DefaultAzureCredential` takes tenant hints for the browser, VS Code,
   shared-cache and workload-identity links and has none for the CLI, so a
   developer whose `az` default tenant differs from the configured one got tokens
   for the wrong tenant — and Fabric answered `UserNotLicensed`, which reads as a
   licensing problem rather than a tenant one. Fixed in `fabric_target`.
4. **The lakehouse's `sqlEndpointProperties.id` is a DIFFERENT GUID from the
   lakehouse** (`803c8e33…` vs `450d5027…`), and a Warehouse has none. That
   confirms the emulator is right to omit the field rather than report the
   lakehouse id: code using it as an endpoint id would work locally and address
   the wrong thing on a tenant.

Three more facts that decide whether this works, all Microsoft's rather than ours:

- **A trial capacity can only be assigned by the account that started the
  trial.** A service principal cannot, so use `az login` as that user. CI uses an
  F-SKU capacity for the same step.
- **A workspace with no capacity** is created successfully and then rejects every
  Fabric item in it. That is why the resolver refuses instead.
- **Workspace identity** can be created in any workspace except *My workspace*,
  so a trial is fine; trusted access to firewalled storage additionally needs an
  F SKU.

## Verification — the same tests, both targets

A pytest marker ties it together:

```python
@pytest.mark.target          # runs under either FABRIC_TARGET
def test_publish_roundtrip(t): ...
```

- **CI (every push):** the marked suite runs with `FABRIC_TARGET=emulator` —
  free, deterministic, offline.
- **Nightly / manual (`workflow_dispatch`):** the same suite with
  `FABRIC_TARGET=real`, gated on repo secrets (`AZURE_TENANT_ID`, SP creds,
  a dedicated throwaway workspace). Every divergence found feeds the
  [parity map](parity.md) — the toggle doubles as a **fidelity oracle**:
  the emulator's behavior is continuously diffed against the real service.

## Phasing

| Phase | Lands | Proves |
|---|---|---|
| **T0** ✅ | `python/fabric-target/` (resolver, TokenCredential-shaped emulator credential, guarded session + LRO poll, `env` emitter) + unit tests + `e2e/fabric-target` (CI 3-OS) + quickstart section | toggle commands are real; emulator profile CI-verified end to end |
| **T1** ✅ | `conformance/` suite (7 target-agnostic tests: per-scope minting, name resolution, item+LRO lifecycle, throttle shape, both guards) — emulator leg in the `fabric-target` CI job; real leg in `.github/workflows/real-fabric.yml` (workflow_dispatch + weekly, self-skips until `AZURE_*`/`FABRIC_TEST_WORKSPACE` secrets exist) | same suite, one env var apart; divergences feed the parity map |
| **T2** ✅ | `notebookutils` reads `FABRIC_TARGET`: real mode resolves the real control plane, the real OneLake endpoint (a different host, not Host-routing), real Entra, TLS verification on, and mints via `DefaultAzureCredential` — the seeded dev identity is `None` in real mode by construction | notebook code runs unchanged locally *and* as a genuine Fabric notebook |

## Non-goals

Proxying or recording real Fabric traffic through the emulator, translating
ids between targets persistently (names are the contract), emulating tenant
onboarding/capacity purchase, and hiding real-mode latency or cost.