Strata ghcr.io/niko1221/strata:latest-cuda13 https://github.com/Niko1221/Strata/pkgs/container/strata bridge bash false https://github.com/williamkunz/unraid-templates/issues https://github.com/Niko1221/Strata https://raw.githubusercontent.com/williamkunz/unraid-templates/main/docs/strata.md Strata runs large mixture-of-experts models on an NVIDIA GPU plus system RAM: Qwen3.8-Flash-Next and its variants, the smaller Qwen3.6-35B-A3B, and DeepSeek-V4.1-Flash. It serves an OpenAI- and Anthropic-compatible API, a chat page and a live monitor. This template runs Strata's official image. Requirements: the Nvidia Driver plugin on driver 580 or newer and an RTX 20/30/40/50 card. For a GTX 10, P100 or V100, or a driver from 575 to 579, change the Repository tag to latest-cuda12. You also need 32 GB of RAM or more (more RAM allows a larger, better model) and room for the model on an SSD pool. Choosing a model: leave Model and Model family empty and Strata picks one for this server, and the log says which and why. To choose yourself, set both. The model names are compression levels: the number is roughly the bits kept per weight, so a smaller number means a smaller download and less RAM, at some cost in quality. RAM below is the server's total RAM. Qwen3.8-Flash-Next, the main model, for chat, code and tools (Model family qwen): - Q2_0: the fastest, good quality. 66 GB download, 48 GB RAM. - IQ2_XS: a little better, nearly as fast. 68 GB, 48 GB RAM. - IQ3_XXS: great quality, slower. 76 GB, 60 GB RAM. - IQ3_S: the best quality, matches the full model; the slowest. 84 GB, 64 GB RAM. Other versions of Qwen3.8 (set the family shown): - swift, with the same four sizes as qwen: a fine-tune by UkisAI that thinks for less time before it answers. - coder, Model IQ1_M: for code only, with half the model removed. 58 GB, 32 GB RAM. - gyro, Model Gyro-S: the least memory (about 28 GB of RAM and graphics memory together), lower quality. 59 GB. Gyro-M: about IQ3_XXS quality in 35 GB, 16-21% slower than Q2_0. 92 GB. - unsloth, more bits for servers with a lot of RAM (the RAM figure keeps the whole model in memory): UD-IQ4_XS 94 GB, 72 GB RAM; UD-Q4_K_XL 111 GB, 88 GB RAM; UD-Q5_K_XL 158 GB, 108 GB RAM; UD-Q6_K_XL 169 GB, 118 GB RAM; Q8_0 188 GB, 136 GB RAM (the reference quality). - nvfp4: NVIDIA's 4-bit format, fastest on RTX 50 cards. NVFP4 (the best of the three), NVFP4-NVIDIA or NVFP4-QAD. 102-128 GB, 80 GB RAM. Qwen3.6-35B-A3B, a smaller model for servers with less RAM (Model family qwen36): - UD-Q4_K_XL: 23 GB download, 32 GB RAM. - UD-Q6_K_XL: closer to the full model. 33 GB, 44 GB RAM. - Q8_0: the closest to the full model. 38 GB, 52 GB RAM. DeepSeek-V4.1-Flash, the largest model (Model family deepseek41). Each download includes a 111 GB lookup table and an 8 GB helper model: - ist-das-2.25bpw: the best quality, the best for code and agents. 274 GB. - ist-das-2.00bpw: the middle choice. 257 GB. - ist-das-1.75bpw: the smallest, noticeably weaker at writing prose. 239 GB. - ista-das-coder: for code only, with half the model removed. 191 GB. DeepSeek needs the latest-cuda13 tag, an RTX 30/40/50 card with 12 GB or more, 46 GB of RAM or more (64 GB is the tested size) and an NVMe pool. Expect about 5-10 tokens per second on a 12 GB card. Its own engine is downloaded into the Data folder on the first start. It is checked at 8k context and offers up to 1m. First start: downloads the model into the Data folder and prepares it, which takes a while; watch the log. A stopped download resumes where it left off. Later starts go straight to serving. API key: always on. Leave it empty and the first start makes one and keeps it in the Data folder; show it with: docker exec Strata strata launch --show-key --preset docker. Clients send it as Authorization: Bearer KEY. Changing settings: edit them and Apply. The new values replace the saved ones on the next start; the model files and tuning stay in the Data folder. Template and Unraid problems: the Support link. Strata itself: the Project link. AI: llm qwen deepseek local ai openai anthropic api inference chat nvidia cuda gpu http://[IP]:[PORT:8080]/ https://raw.githubusercontent.com/williamkunz/unraid-templates/main/templates/strata.xml https://raw.githubusercontent.com/williamkunz/unraid-templates/main/images/strata.png MIT --runtime=nvidia --ulimit memlock=-1:-1 --shm-size=2g --stop-timeout 120 **Nvidia Driver plugin** on driver 580 or newer (575+ with the cuda12 tag). DeepSeek needs the cuda13 tag and an RTX 30 card or newer. Do not set a memory limit on the container: Strata plans from the server's whole RAM. Do not pin CPUs: the engine sets its own thread affinity. 8080 /mnt/cache/appdata/strata all 32k off skip 0 0 int8 0