Basalt
ghcr.io/williamkunz/basalt:latest
https://github.com/williamkunz/unraid-templates/pkgs/container/basalt
bridge
bash
false
https://github.com/williamkunz/unraid-templates/issues
https://github.com/jesdga95/basalt
Basalt runs large language models on an NVIDIA Blackwell GPU plus system RAM: Qwen3.8-Flash-Next, DeepSeek-V4.1-Flash and the dense Qwen3.8-27B. The experts of a mixture-of-experts model that do not fit in graphics memory are kept in RAM. It serves an OpenAI- and Anthropic-compatible API and a chat page. This template runs an image built from Basalt's own Dockerfile.
Requirements: the Nvidia Driver plugin on driver 580 or newer and an RTX 50 series (Blackwell) card; older cards are not supported. You also need room for the model on an SSD pool and enough RAM for the part of the model that does not fit on the card.
Choosing a model: a model is one .basalt file. Download it into the Models folder from the Unraid terminal, for example:
wget -c -P /mnt/cache/appdata/basalt/models https://huggingface.co/jesdga/Qwen3.8-27B-Basalt/resolve/main/Qwen3.8-27B-QUASAR-NVFP4.basalt
(-c resumes a stopped download). The sizes below are the download.
Qwen3.8-Flash-Next, the main model, for chat, code and tools (huggingface.co/jesdga/Qwen3.8-Flash-Next-Basalt):
- Qwen3.8-FN-NVFP4-DQ6_K: the default. 132 GB.
- Qwen3.8-FN-UD-IQ1_S, UD-IQ1_M, UD-IQ2_XS, UD-IQ3_XXS: smaller, for less RAM, at some cost in quality. 70-88 GB.
- Qwen3.8-FN-UD-IQ4_XS, UD-Q4_K_XL, UD-Q5_K_XL, UD-Q6_K_XL, Q8_0: more bits, for servers with a lot of RAM. 98-192 GB.
Qwen3.8-27B, a dense model that runs on the card alone (huggingface.co/jesdga/Qwen3.8-27B-Basalt):
- Qwen3.8-27B-QUASAR-NVFP4: 21 GB.
DeepSeek-V4.1-Flash, the largest model (huggingface.co/jesdga/DeepSeek-V4.1-Flash-1017-Basalt):
- DeepSeek-V4.1-Flash-1017-UD-IQ1_M: the default. 260 GB.
- UD-IQ2_XXS: 286 GB. NVFP4: 428 GB.
Config file: Basalt reads its settings from basalt_config.json in the AppData folder and does not start without it. Name the model by its path inside the container (/models/...), and keep server.port at 8080. A minimal file:
{"version": 2, "models": [{"model": "/models/Qwen3.8-FN-NVFP4-DQ6_K.basalt", "kv": "int8", "max-context": 32768}], "engine": {"resident-experts": true}, "server": {"port": 8080}}
The context defaults to 8,192 tokens. To open the chat page by a host name (tower.local) rather than the server's IP, add "allowed-host": ["tower.local"] to the server block. Every setting: docs/configuration in the Basalt repository.
First start: the page is up at once and the model loads behind it, with the progress shown in the page header; watch the log. A failed load stops the container and the log says why.
API key: Basalt takes the key on the command line only, not in the config file. Without one the API is open to your network. To set one, switch to Advanced View and enter --api-key KEY in Post Arguments (openssl rand -hex 24 makes one). Clients send it as Authorization: Bearer KEY or x-api-key.
Changing settings: edit basalt_config.json and restart the container. The model files stay in the Models folder.
Template and Unraid problems: the Support link. Basalt itself: the Project link.
AI:
llm qwen deepseek local ai openai anthropic api inference chat nvidia cuda gpu blackwell strata
http://[IP]:[PORT:8080]/
https://raw.githubusercontent.com/williamkunz/unraid-templates/main/templates/basalt.xml
https://raw.githubusercontent.com/williamkunz/unraid-templates/main/images/basalt.png
MIT
Support Basalt's developer.
https://github.com/sponsors/jesdga95
--runtime=nvidia --ulimit memlock=-1:-1 --shm-size=2g --stop-timeout 120
**Nvidia Driver plugin** on driver 580 or newer and an **RTX 50 series (Blackwell)** card. A **basalt_config.json** in the AppData folder (see the Overview). Do not set a memory limit on the container: Basalt plans from the server's free RAM.
8080
/mnt/cache/appdata/basalt
/mnt/cache/appdata/basalt/models
all
compute,utility