llama.cpp-CUDA-Router ghcr.io/ggml-org/llama.cpp:server-cuda https://ghcr.io bridge sh false https://github.com/ggml-org/llama.cpp/discussions https://github.com/thunderheadserver/llama-cpp-Cuda-router-unraid https://raw.githubusercontent.com/thunderheadserver/llama-cpp-Cuda-router-unraid/main/icons/llamacpp-icon.png llama.cpp Docker container for NVIDIA GPUs. Configured in Router Mode to support multiple models with automatic model loading and unloading. Includes configurable inference settings such as context size, GPU offloading, KV-cache precision, reasoning controls, Jinja templates, and parallel processing. Not actively and/or very poorly maintained because I don't know what the hell I'm doing. http://[IP]:[PORT:8000]/ --runtime=nvidia --health-cmd="curl -fsS http://localhost:8000/health" --health-interval=30s --health-timeout=10s --health-retries=3 --health-start-period=120s 8000 /mnt/user/appdata/llama_cpp/model 8000 /models 1 true /mnt/user/appdata/llama_cpp/cache all compute,utility 99 q8_0 q8_0 16384 1 true on deepseek 1024 Tools:Utilities https://raw.githubusercontent.com/thunderheadserver/llama-cpp-Cuda-router-unraid/main/templates/llama-cpp-cuda-router.xml https://raw.githubusercontent.com/thunderheadserver/llama-cpp-Cuda-router-unraid/main/README.md