### [Docker Model Runner](https://docs.docker.com/ai/model-runner/)

> Handle: `dmr`<br/>
> URL: [http://localhost:34920](http://localhost:34920)

Docker Model Runner is Docker's local model runtime (Docker Desktop on macOS/Windows, or the `docker-model-plugin` package on Linux). Harbor exposes it as a backend service by starting a small proxy container while Docker manages the host-native runner. This keeps Harbor's Compose workflow intact and lets containers call an OpenAI-compatible `/v1` API.

DMR is especially useful on Apple Silicon because Metal inference runs on the macOS host while Harbor services consume it over HTTP.

#### Starting

```bash
harbor up dmr
```

When `HARBOR_DMR_MANAGE_HOST=true`, Harbor automatically installs or enables missing Docker Model Runner host components before starting the proxy. On Linux this uses the official `docker-model-plugin` package through `apt` or `dnf` when the `docker model` CLI is missing; on Docker Desktop systems Harbor uses Docker Desktop's Model Runner enablement command when it is available. Harbor then installs the runner runtime when supported, attempts to enable the TCP endpoint on Docker Desktop when `HARBOR_DMR_ENABLE_TCP=true`, and pulls `HARBOR_DMR_MODEL` when `HARBOR_DMR_AUTO_PULL=true`.

Start a frontend against DMR:

```bash
harbor up webui dmr
```

Use it with host tools:

```bash
harbor launch --backend dmr --model ai/smollm2 codex
```

#### Stopping

```bash
harbor down dmr
```

This stops the Harbor proxy container. `harbor dmr stop` only logs that Docker Model Runner remains managed by Docker; it does not shut down the host runner.

#### Configuration

##### Environment Variables

Following options can be set via [`harbor config`](./3.-Harbor-CLI-Reference.md#harbor-config):

```bash
# Harbor proxy port
HARBOR_DMR_HOST_PORT              34920

# Proxy image
HARBOR_DMR_IMAGE                  caddy
HARBOR_DMR_VERSION                2-alpine

# Host Docker Model Runner endpoint used by Harbor helpers and proxy
HARBOR_DMR_UPSTREAM_URL           http://model-runner.docker.internal:12434
HARBOR_DMR_RUNNER_PORT            12434

# Default model and host management
HARBOR_DMR_MODEL                  ai/smollm2
HARBOR_DMR_MANAGE_HOST            true
HARBOR_DMR_ENABLE_TCP             true
HARBOR_DMR_AUTO_PULL              true
HARBOR_DMR_API_KEY                sk-dmr
```

##### Volumes

The Harbor `dmr` service mounts only `services/dmr/Caddyfile`. Model files are managed by Docker Model Runner outside the Harbor service directory.

#### Model Management

```bash
harbor models ls --source dmr
harbor models pull --source dmr ai/smollm2
harbor models rm --source dmr ai/smollm2
```

The equivalent source-subcommand form is also supported:

```bash
harbor models dmr pull ai/smollm2
```

#### API

Harbor exposes DMR at:

```bash
http://localhost:34920/v1
```

Containers use:

```bash
http://dmr:8080/v1
```

The proxy rewrites Harbor's `/v1/*` requests to Docker Model Runner's `/engines/v1/*` API.

#### Integrations

`dmr` is exposed to Harbor containers at `http://dmr:8080/v1` (integration API key `sk-dmr`). Harbor wires the same consumer set as [oMLX](./2.2.24-Backend-oMLX.md#integrations) through `compose.x.*.dmr.yml` overlays, including `webui`, `chatui`, `aider`, `boost`, `litellm`, `bifrost`, `optillm`, `opint`, `astrbot`, `cognee`, `mindsdb`, `mi`, `ml-intern`, `npcsh`, `open-design`, `opennotebook`, `openterminal`, `anythingllm`, `cmdh`, `hermes`, `plandex`, `sillytavern`, and `traefik` when enabled.

```bash
harbor up webui dmr
```

#### Troubleshooting

```bash
harbor dmr status
harbor logs dmr
```

##### Docker Model Runner CLI is missing

`harbor up dmr` attempts automatic setup when host management is enabled. If setup cannot proceed on your platform, install or update Docker Desktop, or install Docker's `docker-model-plugin` package on Linux, then verify:

```bash
docker model --help
```

##### Proxy is unhealthy

Check that Docker Model Runner is enabled and reachable at `HARBOR_DMR_UPSTREAM_URL`. If automatic TCP enablement fails, enable Model Runner from Docker Desktop or adjust `HARBOR_DMR_UPSTREAM_URL` to the endpoint Docker exposes on your machine.

#### Links

- [Docker Model Runner](https://docs.docker.com/ai/model-runner/)
- [Docker Model Runner API](https://docs.docker.com/ai/model-runner/api-reference/)
- [Docker Model Runner with Compose](https://docs.docker.com/ai/compose/models-and-compose/)