# Sardeenz runner catalog — the list of available, ready-to-import runner SIFs. # # Sardeenz's control plane loads this file (from SARDEENZ_RUNNER_CATALOG_URL, default the official # school-of-sardeenz catalog; this in-repo copy is the dev/reference source) and presents it in the # dashboard. Each entry is a signed SIF published to an OCI registry via ORAS; "Import" streams # the digest-checked SIF layer onto the shared module store and reports byte-level progress. # # Schema: # apiVersion: sardeenz.io/v1 # kind: RunnerCatalog # runners: # - id: stable unique id, [A-Za-z0-9_.-]+ (usually == sifName) # title: human-readable name shown in the catalog # description: one-paragraph summary # engine: optional engine family (e.g. "vLLM", "Triton") # runnerType: runner type identifier (e.g. "vllm") — matches the runner contract # version: engine/build version string # image: ORAS reference to pull (oras:///:@sha256: — the # digest is required so a repointed tag can't silently change what gets pulled) # sifName: module filename stem, [A-Za-z0-9_.-]+ → /modules/.sif == runtimeModule # protocol: REQUIRED — proxy protocol family: "openai" (OpenAI-compatible, /openai/v1) # or "oip" (KServe V2 Open Inference Protocol, /oip/v2). Must be one the # running proxy advertises, else import is rejected (proxy upgrade required). # tags: optional list of free-form labels for filtering # minVRAMGiB: optional minimum device memory hint (GiB) # license: optional SPDX license id # icon: optional URL to an icon # supportedModelTypes: optional list of ModelType values (e.g. [LLM]) # supportedDeviceTypes: optional list of DeviceType values (e.g. [CUDA]) # supportedSleepLevels: optional list of SleepLevel values (e.g. [L1_HOST_RAM]) # maxTensorParallelism: optional max devices this runner can span for tensor parallelism # kvCacheElasticSharing: optional bool — supports elastic, reclaimable KV-cache sharing # features: optional map of engine-specific feature flags # entrypoint: optional verbatim argv the worker execs in the SIF to launch the runner's # management shim (e.g. [python3, -m, sardeenz_mlserver_runner]); falls back # to the worker's SARDEENZ_RUNNER_ENTRYPOINT when omitted apiVersion: sardeenz.io/v1 kind: RunnerCatalog runners: - id: vllm-0.21 title: vLLM 0.21 (+ kvcached) description: >- vLLM 0.21 inference engine with kvcached enabled for elastic, reclaimable KV-cache sharing across co-located runners on one GPU. OpenAI-compatible; supports sleep/wake. engine: vLLM runnerType: vllm version: '0.21' image: oras://quay.io/rh-aiservices-bu/sardeenz-runners/vllm:0.21-rc1@sha256:0e1e72a7b39b063a35f30b2fea995a1d6133b5e447e045ab8d0ac2ae01570de7 sifName: vllm-0.21 protocol: openai tags: [llm, cuda, kvcached, openai] minVRAMGiB: 16 license: Apache-2.0 supportedModelTypes: [LLM] supportedDeviceTypes: [CUDA] supportedSleepLevels: [L1_HOST_RAM] maxTensorParallelism: 8 kvCacheElasticSharing: true features: streamingInference: true chatTemplate: true toolUse: true kvCacheOffload: false prefixCaching: true - id: vllm-0.24 title: vLLM 0.24 (+ kvcached) description: >- vLLM (0.24) with kvcached, kept for models pinned to that engine version. Forces Model Runner V1 for kvcached compatibility. Runs side by side with newer versions — hot-add without recycling workers. engine: vLLM runnerType: vllm version: '0.24' image: oras://quay.io/rh-aiservices-bu/sardeenz-runners/vllm:0.24-rc1@sha256:540551a591958cc23b1c2663ede6f74e3172f1cf0272a651b8b08e5385d7f58b sifName: vllm-0.24 protocol: openai tags: [llm, cuda, kvcached, openai] minVRAMGiB: 16 license: Apache-2.0 supportedModelTypes: [LLM] supportedDeviceTypes: [CUDA] supportedSleepLevels: [L1_HOST_RAM] maxTensorParallelism: 8 kvCacheElasticSharing: true features: streamingInference: true chatTemplate: true toolUse: true kvCacheOffload: false prefixCaching: true - id: mlserver-1.7.1 title: MLServer 1.7.1 (KServe V2) description: >- Seldon MLServer 1.7.1 inference server speaking the KServe V2 Open Inference Protocol (/oip/v2). Runtimes: scikit-learn (CPU-classical), HuggingFace/torch, and plain ONNX graphs via onnxruntime-gpu (CUDA with CPU fallback). Supports sleep/wake via model unload/load. Not OpenAI-compatible. engine: MLServer runnerType: mlserver version: '1.7.1' image: oras://quay.io/rh-aiservices-bu/sardeenz-runners/mlserver:1.7.1-rc4@sha256:c9c6f0496086a73f4405ca469169034458c198df34d57300ef1d984207835cd3 sifName: mlserver-1.7.1 protocol: oip entrypoint: [python3, -m, sardeenz_mlserver_runner] tags: [oip, cuda, sklearn, huggingface, onnx] minVRAMGiB: 1 license: Apache-2.0 supportedModelTypes: [PREDICTIVE, LLM, EMBEDDING] supportedDeviceTypes: [CUDA] supportedSleepLevels: [L1_HOST_RAM] maxTensorParallelism: 1 kvCacheElasticSharing: false