{ "stable": true, "versions": { "0.1.0": { "manifest": { "id": "io.github.orcvole.vllm", "title": "vLLM", "author": "OrcVole", "tagline": "High-throughput OpenAI-compatible LLM inference server", "description": "vLLM is a high-throughput inference server for large language models with the most widely\nsupported OpenAI-compatible API of the self-hosted options. This package runs it as a\nCloudron app: one install serves one model at `https:///v1`, protected by a\ngenerated API key, with weights cached locally and every OpenAI-style client able to connect\nunchanged.\n\nThis is CPU inference (amd64). It will not match GPU speed and it is not meant to: it gives\nintegrations, automations and privacy-sensitive workloads a fully self-hosted OpenAI endpoint\non the server you already run. Pair it with a chat frontend, an AI gateway, embeddings and a\nvector store to complete a private AI stack on Cloudron.\n\nUnofficial community package. Not affiliated with the vLLM project or Cloudron.\n", "changelog": "* Initial release: vLLM 0.26.0 OpenAI-compatible inference server, CPU backend (amd64, AVX2 minimum)\n* One model per install, default Qwen/Qwen3-0.6B, configurable via LLM_MODEL\n* Generated API key protects /v1; health, readiness and landing page open\n* Model cache on a persistent path outside backups; restores stay small and fast\n* Full gate ladder passed: install, auth, streamed flows through the platform proxy, backup/restore, memory sizing (10 GiB limit from measured 7.4 GiB steady footprint)\n", "icon": "file://logo.png", "version": "0.1.0", "upstreamVersion": "0.26.0", "healthCheckPath": "/health", "httpPort": 8000, "addons": { "localstorage": {} }, "persistentDirs": [ "/var/lib/vllm" ], "memoryLimit": 10737418240, "configurePath": "/", "optionalSso": true, "postInstallMessage": "This app is an **API server**. The domain serves a small landing page and the OpenAI-compatible\nAPI at `/v1`; there is no web interface to log in to.\n\n**Your API key** was generated on first run. Open a Terminal for this app (the `>_` button) and\nrun:\n\n```\ncat /app/data/.secrets/keys.env\n```\n\nSend it as `Authorization: Bearer ` with every `/v1` request.\n\n**First boot is slow on purpose:** the default model downloads and loads before the API\nanswers. Watch the app's Logs for progress. The landing page shows how to make your first\nrequest once the model is ready.\n\n**Change the model** by setting `LLM_MODEL` in the app's Environment section and restarting.\nLarger models need a larger memory limit (Resources section) first.\n", "checklist": { "apikey": { "message": "Get your API key: open a Terminal for this app (the `>_` button) and run `cat /app/data/.secrets/keys.env`. Send it as `Authorization: Bearer ` to the /v1 endpoints." }, "model": { "message": "First boot downloads the default model before the API answers. Watch progress in the app logs. To serve a different model, set LLM_MODEL in the app's Environment and restart." }, "streaming": { "message": "CPU inference is deliberate, not fast. Use streaming responses (stream: true); long non-streamed completions can hit the platform's 60 second proxy timeout." } }, "tags": [ "ai", "llm", "inference", "openai" ], "mediaLinks": [ "https://raw.githubusercontent.com/OrcVole/vllm-cloudron/main/screenshots/overview.png" ], "website": "https://github.com/OrcVole/vllm-cloudron", "documentationUrl": "https://github.com/OrcVole/vllm-cloudron", "contactEmail": "Most+github@OrcadianVole.com", "packagerName": "OrcVole", "packagerUrl": "https://github.com/OrcVole/vllm-cloudron", "manifestVersion": 2, "minBoxVersion": "9.1.0", "iconUrl": "https://raw.githubusercontent.com/OrcVole/vllm-cloudron/main/logo.png", "dockerImage": "ghcr.io/orcvole/vllm-cloudron@sha256:254b0295f6c3842d1c6415fe77b26e86578acc7bbd7d82c19cc572b5e8e970d9" }, "creationDate": "Thu, 30 Jul 2026 17:14:48 GMT", "ts": 1785431688000, "publishState": "published" }, "0.2.0": { "manifest": { "id": "io.github.orcvole.vllm", "title": "vLLM", "author": "OrcVole", "tagline": "High-throughput OpenAI-compatible LLM inference server", "description": "0.27.1\n\nvLLM is a high-throughput inference server for large language models with the most widely\nsupported OpenAI-compatible API of the self-hosted options. This package runs it as a\nCloudron app: one install serves one model at `https:///v1`, protected by a\ngenerated API key, with weights cached locally and every OpenAI-style client able to connect\nunchanged.\n\nThis is CPU inference (amd64). It will not match GPU speed and it is not meant to: it gives\nintegrations, automations and privacy-sensitive workloads a fully self-hosted OpenAI endpoint\non the server you already run. Pair it with a chat frontend, an AI gateway, embeddings and a\nvector store to complete a private AI stack on Cloudron.\n\nUnofficial community package. Not affiliated with the vLLM project or Cloudron.\n", "changelog": "* vLLM 0.27.1 (from 0.26.0), which upstream calls a breaking environment change: PyTorch moves to 2.13.0 with torchvision 0.28.0 and Triton 3.7.1\n* Upstream removed the `max_num_partial_prefills` and `max_long_partial_prefills` arguments and dropped the Plamo2 and Ouro models; this package used none of them, so no packaging change was needed\n* New models available upstream include Qwen3.5 dense and MoE, K-EXAONE-2.0, VaultGemma and jina-embeddings-v5-text-nano\n* Packaging fix required by 0.27.x: its engine now runs in a separate process and brokers work over a shared-memory ring buffer needing 160 MiB of /dev/shm, while a Cloudron app container has a fixed 64 MiB. The app would not have started at all. start.sh now measures /dev/shm at boot and runs the engine in-process when it is under 200 MiB, which costs nothing on a single-model CPU server, and logs which mode it chose. Written as a measurement rather than a constant, so the package returns to upstream's default automatically if the platform limit ever rises\n", "icon": "file://logo.png", "version": "0.2.0", "upstreamVersion": "0.27.1", "healthCheckPath": "/health", "httpPort": 8000, "addons": { "localstorage": {} }, "persistentDirs": [ "/var/lib/vllm" ], "memoryLimit": 10737418240, "configurePath": "/", "optionalSso": true, "postInstallMessage": "This app is an **API server**. The domain serves a small landing page and the OpenAI-compatible\nAPI at `/v1`; there is no web interface to log in to.\n\n**Your API key** was generated on first run. Open a Terminal for this app (the `>_` button) and\nrun:\n\n```\ncat /app/data/.secrets/keys.env\n```\n\nSend it as `Authorization: Bearer ` with every `/v1` request.\n\n**First boot is slow on purpose:** the default model downloads and loads before the API\nanswers. Watch the app's Logs for progress. The landing page shows how to make your first\nrequest once the model is ready.\n\n**Change the model** by setting `LLM_MODEL` in the app's Environment section and restarting.\nLarger models need a larger memory limit (Resources section) first.\n", "checklist": { "apikey": { "message": "Get your API key: open a Terminal for this app (the `>_` button) and run `cat /app/data/.secrets/keys.env`. Send it as `Authorization: Bearer ` to the /v1 endpoints." }, "model": { "message": "First boot downloads the default model before the API answers. Watch progress in the app logs. To serve a different model, set LLM_MODEL in the app's Environment and restart." }, "streaming": { "message": "CPU inference is deliberate, not fast. Use streaming responses (stream: true); long non-streamed completions can hit the platform's 60 second proxy timeout." } }, "tags": [ "ai", "llm", "inference", "openai" ], "mediaLinks": [ "https://raw.githubusercontent.com/OrcVole/vllm-cloudron/main/screenshots/overview.png" ], "website": "https://github.com/OrcVole/vllm-cloudron", "documentationUrl": "https://github.com/OrcVole/vllm-cloudron", "contactEmail": "Most+github@OrcadianVole.com", "packagerName": "OrcVole", "packagerUrl": "https://github.com/OrcVole/vllm-cloudron", "manifestVersion": 2, "minBoxVersion": "9.1.0", "iconUrl": "https://raw.githubusercontent.com/OrcVole/vllm-cloudron/main/logo.png", "dockerImage": "ghcr.io/orcvole/vllm-cloudron@sha256:0601613416b50b5cd832a7ff7153228745e9d2c0076c68ae293fe90c1284192e" }, "creationDate": "Wed, 19 Aug 2026 18:17:05 GMT", "ts": 1787163425000, "publishState": "published" }, "0.2.1": { "manifest": { "id": "io.github.orcvole.vllm", "title": "vLLM", "author": "OrcVole", "tagline": "High-throughput OpenAI-compatible LLM inference server", "description": "`0.28.0\n\nvLLM is a high-throughput inference server for large language models with the most widely\nsupported OpenAI-compatible API of the self-hosted options. This package runs it as a\nCloudron app: one install serves one model at `https:///v1`, protected by a\ngenerated API key, with weights cached locally and every OpenAI-style client able to connect\nunchanged.\n\nThis is CPU inference (amd64). It will not match GPU speed and it is not meant to: it gives\nintegrations, automations and privacy-sensitive workloads a fully self-hosted OpenAI endpoint\non the server you already run. Pair it with a chat frontend, an AI gateway, embeddings and a\nvector store to complete a private AI stack on Cloudron.\n\nUnofficial community package. Not affiliated with the vLLM project or Cloudron.\n", "changelog": "- Update vLLM 0.27.1 -> 0.28.0\n- Security: fixes prevent denial of service via forged audio sample rates and oversized image inputs, unauthenticated access to API endpoints, arbitrary code execution from untrusted model repositories, and resource exhaustion from unbounded generation requests\n- Breaking: bitsandbytes quantization moved to an out-of-tree plugin; Transformers dependency bumped to 5.15.0; removed flags `calculate_kv_scales` and `override_attention_dtype`; `reasoning_content` no longer included in output; MoE legacy code paths removed; `cache_salt` now required to be non-empty\n- Behaviour changes: `max_num_batched_tokens` default raised to 16384; prefix caching now enabled by default for Mamba models; Blackwell CUDA graph capture default raised to 1024; KV offload tiering metrics renamed from block to chunk\n- Packaging: pin moved; /dev/shm detection in start.sh unchanged and still required\n", "icon": "file://logo.png", "version": "0.2.1", "upstreamVersion": "0.28.0", "healthCheckPath": "/health", "httpPort": 8000, "addons": { "localstorage": {} }, "persistentDirs": [ "/var/lib/vllm" ], "memoryLimit": 10737418240, "configurePath": "/", "optionalSso": true, "postInstallMessage": "This app is an **API server**. The domain serves a small landing page and the OpenAI-compatible\nAPI at `/v1`; there is no web interface to log in to.\n\n**Your API key** was generated on first run. Open a Terminal for this app (the `>_` button) and\nrun:\n\n```bash\ncat /app/data/.secrets/keys.env\n```\n\nSend it as `Authorization: Bearer ` with every `/v1` request.\n\n**First boot is slow on purpose:** the default model downloads and loads before the API\nanswers. Watch the app's Logs for progress. The landing page shows how to make your first\nrequest once the model is ready.\n\n**Change the model** by setting `LLM_MODEL` in the app's Environment section and restarting.\nLarger models need a larger memory limit (Resources section) first.\n", "checklist": { "apikey": { "message": "Get your API key: open a Terminal for this app (the `>_` button) and run `cat /app/data/.secrets/keys.env`. Send it as `Authorization: Bearer ` to the /v1 endpoints." }, "model": { "message": "First boot downloads the default model before the API answers. Watch progress in the app logs. To serve a different model, set LLM_MODEL in the app's Environment and restart." }, "streaming": { "message": "CPU inference is deliberate, not fast. Use streaming responses (stream: true); long non-streamed completions can hit the platform's 60 second proxy timeout." } }, "tags": [ "ai", "llm", "inference", "openai" ], "mediaLinks": [ "https://raw.githubusercontent.com/OrcVole/vllm-cloudron/main/screenshots/overview.png" ], "website": "https://github.com/OrcVole/vllm-cloudron", "documentationUrl": "https://github.com/OrcVole/vllm-cloudron", "contactEmail": "Most+github@OrcadianVole.com", "packagerName": "OrcVole", "packagerUrl": "https://github.com/OrcVole/vllm-cloudron", "manifestVersion": 2, "minBoxVersion": "9.1.0", "iconUrl": "https://raw.githubusercontent.com/OrcVole/vllm-cloudron/main/logo.png", "dockerImage": "ghcr.io/orcvole/vllm-cloudron@sha256:746f7aa0495ea3abac0a114291a9917bef572da29ae52b8250cb1f2a4ea428ba" }, "creationDate": "Wed, 02 Sep 2026 21:11:02 GMT", "ts": 1788383462000, "publishState": "published" }, "0.3.0": { "manifest": { "id": "io.github.orcvole.vllm", "title": "vLLM", "author": "OrcVole", "tagline": "High-throughput OpenAI-compatible LLM inference server", "description": "`0.29.0\n\nvLLM is a high-throughput inference server for large language models with the most widely\nsupported OpenAI-compatible API of the self-hosted options. This package runs it as a\nCloudron app: one install serves one model at `https:///v1`, protected by a\ngenerated API key, with weights cached locally and every OpenAI-style client able to connect\nunchanged.\n\nThis is CPU inference (amd64). It will not match GPU speed and it is not meant to: it gives\nintegrations, automations and privacy-sensitive workloads a fully self-hosted OpenAI endpoint\non the server you already run. Pair it with a chat frontend, an AI gateway, embeddings and a\nvector store to complete a private AI stack on Cloudron.\n\nUnofficial community package. Not affiliated with the vLLM project or Cloudron.\n", "changelog": "- Update vLLM 0.28.0 to 0.29.0\n- Model Runner V2 is now the default engine; new model support includes Hy4-preview, Qwen3.8-Flash-Next, GraniteSWA/MoE and Kimi K3 NVFP4\n- Performance improvements across CUDA ROCm and CPU backends\n- New opt-in ASGI authentication middleware (available via `--api-key`)\n- Ten deprecated model architectures removed: Arctic, Chameleon, Cheers, Fairseq2Llama, FireRedLID, GritLM, HCXVision, MPT, RWForCausalLM/StableLMEpochForCausalLM aliases and PrithviGeoSpatialMAE\n- PyAV video decoder backend removed; use OpenCV or Torchcodec instead\n- Environment variables `VLLM_TEST_FORCE_FP8_MARLIN` and `VLLM_ROCM_USE_AITER_FP4_ASM_GEMM` removed\n- FlashInfer all-reduce now enabled by default for TP CUDA groups (opt out with `VLLM_ALLREDUCE_USE_FLASHINFER=0`)\n- Check before updating: vLLM 0.29 removes the ten model architectures listed above and makes Model Runner V2 the\n default engine. If your install serves one of those models, or passes a removed option or environment variable\n through `LLM_EXTRA_ARGS`, it will fail to start on this version; change the model or options first.\n", "icon": "file://logo.png", "version": "0.3.0", "upstreamVersion": "0.29.0", "healthCheckPath": "/health", "httpPort": 8000, "addons": { "localstorage": {} }, "persistentDirs": [ "/var/lib/vllm" ], "memoryLimit": 10737418240, "configurePath": "/", "optionalSso": true, "postInstallMessage": "This app is an **API server**. The domain serves a small landing page and the OpenAI-compatible\nAPI at `/v1`; there is no web interface to log in to.\n\n**Your API key** was generated on first run. Open a Terminal for this app (the `>_` button) and\nrun:\n\n```bash\ncat /app/data/.secrets/keys.env\n```\n\nSend it as `Authorization: Bearer ` with every `/v1` request.\n\n**First boot is slow on purpose:** the default model downloads and loads before the API\nanswers. Watch the app's Logs for progress. The landing page shows how to make your first\nrequest once the model is ready.\n\n**Change the model** by setting `LLM_MODEL` in the app's Environment section and restarting.\nLarger models need a larger memory limit (Resources section) first.\n", "checklist": { "apikey": { "message": "Get your API key: open a Terminal for this app (the `>_` button) and run `cat /app/data/.secrets/keys.env`. Send it as `Authorization: Bearer ` to the /v1 endpoints." }, "model": { "message": "First boot downloads the default model before the API answers. Watch progress in the app logs. To serve a different model, set LLM_MODEL in the app's Environment and restart." }, "streaming": { "message": "CPU inference is deliberate, not fast. Use streaming responses (stream: true); long non-streamed completions can hit the platform's 60 second proxy timeout." } }, "tags": [ "ai", "llm", "inference", "openai" ], "mediaLinks": [ "https://raw.githubusercontent.com/OrcVole/vllm-cloudron/main/screenshots/overview.png" ], "website": "https://github.com/OrcVole/vllm-cloudron", "documentationUrl": "https://github.com/OrcVole/vllm-cloudron", "contactEmail": "Most+github@OrcadianVole.com", "packagerName": "OrcVole", "packagerUrl": "https://github.com/OrcVole/vllm-cloudron", "manifestVersion": 2, "minBoxVersion": "9.1.0", "iconUrl": "https://raw.githubusercontent.com/OrcVole/vllm-cloudron/main/logo.png", "dockerImage": "ghcr.io/orcvole/vllm-cloudron@sha256:d73953eb51c16f39c199eea795767a6433409c1c002021d8ddd12371f0a4caec" }, "creationDate": "Tue, 15 Sep 2026 09:55:25 GMT", "ts": 1789466125000, "publishState": "published" } } }