# Container Backends

A container backend runs its server from a container image instead of a downloaded binary. `lemonade backends` lists these backends with the format `container`. Installing one pulls its image, pinned by digest, and loading a model starts a container from that image.

Container backends run on Linux with Podman or Docker. Lemonade picks one when it runs a container:

| Engine | Used when |
|--------|-----------|
| Rootless Podman (default) | Podman is installed. Containers run as the account `lemond` runs under. |
| Rootful Podman | Podman is installed and `CONTAINER_HOST` points at a root Podman socket, or `lemond` runs as root. |
| Docker | Podman is not installed. |

Rootful Podman and Docker give that account root-equivalent control of the host's containers.

| Backend | Engine | Image |
|---------|--------|-------|
| `ds4:rocm` | DS4 | [kyuz0/strix-halo-ds4-toolbox](https://hub.docker.com/r/kyuz0/strix-halo-ds4-toolbox) |
| `llamacpp:nathanw` | llama.cpp Vulkan performance fork | [kyuz0/amd-strix-halo-toolboxes](https://hub.docker.com/r/kyuz0/amd-strix-halo-toolboxes) |
| `llamacpp:halobox` | llama.cpp Halo Box ROCm fork | [kyuz0/amd-strix-halo-toolboxes](https://hub.docker.com/r/kyuz0/amd-strix-halo-toolboxes) |
| `rocmfpx:rocm` | llama.cpp ROCm FPX fork | [kyuz0/amd-strix-halo-toolboxes](https://hub.docker.com/r/kyuz0/amd-strix-halo-toolboxes) |
| `halogen:rocm` | Halogen Flash; needs Linux 6.18.4 or newer, built with `CONFIG_HSA_AMD_SVM` | [peonist-ai/halogen-flash-server](https://github.com/orgs/peonist-ai/packages/container/package/halogen-flash-server) |
| `gufo:rocm` | Gufo | [kyuz0/amd-strix-halo-toolboxes](https://hub.docker.com/r/kyuz0/amd-strix-halo-toolboxes) |

## Setup

When `lemond` runs under your own account, from a shell, a `systemctl --user` unit, or an app that embeds it, its containers sit beside your own in `podman ps`, and it uses whatever `CONTAINER_HOST` your environment sets. The `systemctl --user` unit sets it to your `podman.socket`, so Podman's API service starts the containers outside the unit's sandbox. If the unit is ever killed outright, as with `systemctl --user kill -s KILL lemond`, a container can keep running; loading the same model replaces it, or `podman rm -f` removes it.

**With Podman:**

1. Install Podman and crun, for example with `sudo apt install podman crun`, `sudo dnf install podman crun` or `sudo pacman -S podman crun`.
2. Add your account to `render`: `sudo usermod -aG render $USER`.
3. Log out and back in.

Rootless Podman must run containers with crun, not runc. Containers reach the GPU through your account's `render` group, and only crun can keep that group inside a rootless container; under runc the backend can't open the GPU and silently runs on the CPU. Podman normally installs crun, but on Debian and Ubuntu a machine that had Docker first already has runc, and Podman uses it. `lemonade backends` reports this with the command that installs crun, and Podman switches to crun as soon as it's installed.

**With Docker:**

1. Add your account to the `docker` group: `sudo usermod -aG docker $USER`.
2. Log out and back in.

Docker Desktop and rootless Docker can't run container backends. With either one, install Podman instead.

When `lemond` runs as `lemond.service`, installed by the `.deb`, PPA or `.rpm` package, it runs under the `lemonade` account. The package pulls in Podman unless Docker is already installed, and crun, adds `lemonade` to `render`, and allocates its subordinate UID and GID range, so rootless Podman needs no further setup.

**`lemond.service` with rootful Podman:** give `lemonade` its own root Podman socket, leaving root's `podman.socket` as it is. Rootful Podman needs the netavark network backend, Podman's default; on Podman 4's older CNI backend, `lemond` cannot reach its containers. Switching root's Podman to netavark (`network_backend = "netavark"` in `/etc/containers/containers.conf`) needs `podman system reset`, which deletes all of root's containers and images.

1. Create `/etc/systemd/system/podman-lemonade.socket`:
   ```ini
   [Socket]
   ListenStream=/run/podman-lemonade.sock
   SocketGroup=lemonade
   SocketMode=0660

   [Install]
   WantedBy=sockets.target
   ```
2. Create `/etc/systemd/system/podman-lemonade.service`:
   ```ini
   [Service]
   Delegate=true
   Type=exec
   KillMode=process
   ExecStart=/usr/bin/podman system service
   ```
3. `sudo systemctl daemon-reload && sudo systemctl enable --now podman-lemonade.socket`
4. Add `CONTAINER_HOST=unix:///run/podman-lemonade.sock` to `/etc/default/lemond`, then `sudo systemctl restart lemond`.

**`lemond.service` with Docker:**

1. Remove Podman, which takes precedence: `sudo apt remove podman` or `sudo dnf remove podman`.
2. `sudo usermod -aG docker lemonade`, then `sudo systemctl restart lemond`.

**With SELinux enforcing**, as on Fedora, ROCm backends also need containers allowed to map GPU devices: `sudo setsebool -P container_use_devices 1`.

Until setup is complete, the backend's state is `action_required`. `lemonade backends` lists every missing step and the commands that complete them, and the Backend Manager shows the same. The state updates as soon as a step takes effect.

## Install and Run

```bash
lemonade backends install llamacpp:nathanw
lemonade run Qwen3-4B-GGUF --llamacpp nathanw
```

Install pulls the image, so loading a model starts from the image already on disk. `lemonade backends uninstall llamacpp:nathanw` removes it.

## How Containers Run

Each loaded model gets its own container, named `lemonade-<port>-<recipe>-<backend>-<model>`, where `<port>` is the port Lemonade listens on, and labeled `ai.lemonade`, so `podman ps --filter label=ai.lemonade` lists them. Every load logs its full `podman run` or `docker run` command. Each container:

- mounts only the model's files, read-only, under `/mnt/models`, with the shared SELinux label so other containers and programs can still read them
- opens only the GPU device nodes its backend needs
- drops every Linux capability its backend does not need, and runs with `no-new-privileges`
- joins its own `--internal` network, so only this machine reaches its API and the server inside stays off the internet

Unloading a model stops its container.