# vllm-ops-mcp

[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
[![tests](https://img.shields.io/badge/tests-102%20%2895%20passing%2C%207%20live--skipped%29-brightgreen)](tests)
[![CI](https://github.com/jaimenbell/vllm-ops-mcp/actions/workflows/ci.yml/badge.svg)](https://github.com/jaimenbell/vllm-ops-mcp/actions/workflows/ci.yml)

Read-only ops/health MCP server for a local, WSL2/systemd-managed vLLM
server -- liveness vs. real-completion health tiers, GPU/VRAM status,
systemd service status, and live serve-flag introspection. Built to the same
standard as [mcp-factory](https://github.com/jaimenbell/mcp-factory) and
[desktop-mcp](https://github.com/jaimenbell/desktop-mcp), and other local
tooling in this operator's portfolio: own pyproject, own fastmcp server,
honest README, real test suite, no product code before a spec was confirmed.

This is **not** a container-lifecycle manager. It assumes vLLM already runs
as a bare-metal `systemd` unit inside WSL2 (this operator's actual setup --
see [vllm-autostart](../vllm-autostart)), not inside Docker/Podman.

## Quickstart

```jsonc
// ~/.claude.json (or any MCP-host stdio client config)
{
  "mcpServers": {
    "vllm-ops-mcp": {
      "type": "stdio",
      "command": "C:\\Users\\<you>\\projects\\vllm-ops-mcp\\.venv\\Scripts\\python.exe",
      "args": ["C:\\Users\\<you>\\projects\\vllm-ops-mcp\\run_server.py"]
    }
  }
}
```

## Tools (Phase 1 -- all read-only, no gating required beyond a throughput cap)

| Tool | What it does | What it can't do |
|---|---|---|
| `check_health(deep: bool = False)` | `deep=False`: GET `/v1/models` liveness check (same signal as `llm_router.LocalLLM.is_healthy()`). `deep=True`: additionally fires one minimal real `/v1/chat/completions` call (mirrors `check-vllm.cmd`'s two-stage probe) to confirm the server actually *generates*, not just *reports a model loaded*. | Cannot fix a degraded server -- reports ground truth only. `deep=True` is rate-limited (default 20/min, see below). |
| `list_models` | `/v1/models` passthrough. | No detail beyond what vLLM's API itself exposes. |
| `test_completion(prompt, max_tokens=16)` | On-demand real completion with a caller-supplied prompt, for manual sanity checks. | Rate-limited (same bucket as `check_health(deep=True)`); refuses if the liveness check fails first (no point firing a completion at a server that isn't even listening). `max_tokens` is clamped server-side to 1024 (`config.MAX_TEST_COMPLETION_TOKENS`) and the prompt is capped at 8000 chars (`config.MAX_TEST_COMPLETION_PROMPT_CHARS`, oversized prompts are rejected, not truncated) -- the rate limiter only bounds call *frequency*, not the cost of a single call. |
| `get_gpu_status` | `nvidia-smi` wrapper: per-GPU VRAM used/total + utilization%, plus per-process VRAM via `--query-compute-apps`. | **Per-process VRAM attribution does not work on this operator's actual WSL2 setup** -- live-verified empty even while the vLLM process holds 13.7GB (see Limitations). Treat `processes` as best-effort/often-empty, not a reliable per-PID breakdown. |
| `get_service_status` | `systemctl show <unit> -p ...` (structured property output, not free-text parsing) via `wsl -d <distro>` from the Windows host, or native `systemctl` if this server itself runs inside WSL2/Linux. Returns load/active/sub state, restart count, main PID, last-active timestamp. | Read-only -- cannot start/stop/restart the unit. No computed uptime duration (deliberately -- see Limitations, clock-discipline note). |
| `get_serve_config` | Read-only launch-flag introspection. **Prefers the LIVE process's actual argv** (`/proc/<pid>/cmdline` of the systemd unit's `MainPID` -- ground truth of what's running right now), falling back to the static exec script (local mirror via `VLLM_OPS_MCP_SERVE_CONFIG_PATH`, else a live `cat` of the WSL-side exec script) only when the process isn't up. | vLLM doesn't expose its own launch flags over the API, so this never queries the server itself -- it's always shelling out. Exec-script fallback parsing is best-effort shell-text parsing (less precise than the live-argv path). |

### Future extension: `restart_service` (Phase 2, not built)

Deliberately **not implemented in this release**. Per the spec: vLLM is
shared infra other bots/tools depend on (options-bot, `qwen_cli.py`
callers, DeerFlow fallback), so a restart tool needs its own gating design
(off by default, `VLLM_OPS_MCP_ENABLE_RESTART=1`, refuse to touch a healthy
instance unless `force=True`, rate-capped) before it ships -- not
retrofitted onto a read-only MVP. Tracked as a v2 idea, not a current
capability. If built, it should wrap `run-vllm.cmd`'s exact recipe (`systemctl
start vllm` only, never blind `stop`/`restart`) rather than reinvent it.

## The differentiator: probe HTTP, not the listener

Every health signal in this server comes from an actual HTTP request to the
OpenAI-compatible API (`urllib.request`, not a socket/connect check). This
is deliberate, not incidental: `Get-NetTCPConnection` and other
listener-based checks **miss WSL2-mirrored-networking-relayed ports
entirely** -- a genuinely healthy `127.0.0.1:8000` can be invisible to a
listener probe run from the Windows host. `check-vllm.cmd` already embodies
this lesson; this server generalizes it into two tiers (liveness vs. real
generation) that nothing else in this operator's fleet exposes as MCP tools.

## Security

Read this before you register vllm-ops-mcp. Honesty about blast radius is
the point.

- **This server shells out to `nvidia-smi`, `systemctl` (via `wsl -d`), and
  reads `/proc/<pid>/cmdline`.** All six tools are read-only -- nothing here
  starts, stops, restarts, or kills a process, and no tool accepts an
  argument that gets passed to a shell unescaped (unit/distro names come
  from server-side config, not caller input; `shlex.quote` wraps the one
  caller-adjacent path that does reach a shell string, the unit name in
  `systemctl show`).
- **`check_health(deep=True)` and `test_completion` fire a real inference
  call against a shared GPU server.** Rate-limited (default 20 calls/min,
  `VLLM_OPS_MCP_DEEP_RATE_LIMIT_PER_MIN`) specifically because this vLLM
  instance is shared infra other bots/tools depend on -- a caller hammering
  this MCP could starve or slow other consumers even though every call
  individually looks "read-only."
- **`get_serve_config`'s live-process path reads `/proc/<pid>/cmdline`,**
  which can include secrets if any were ever passed as CLI flags (none are,
  in this operator's actual setup -- the model path and flags are all
  non-secret; auth, if any, is environment-injected inside the exec script,
  not passed as an argv flag). Defense-in-depth: any argv entry whose
  preceding flag name matches a known-sensitive pattern (`token`, `key`,
  `secret`, `password`, `passwd`, `credential` -- `config.
  SENSITIVE_ARGV_FLAG_PATTERNS`) has its **value** redacted to
  `[REDACTED]` before being returned, in both the structured `argv`/`flags`
  fields and `raw_text`; the flag name itself is left visible so the shape
  is still diagnosable. This applies across all three sources
  (`live_process`, `exec_script`, `local_config_mirror`).
- **No tool has any write/mutate capability in this release.** There is no
  env var that turns one on -- `restart_service` doesn't exist in this
  codebase yet (see Future extension above), so there's nothing to
  accidentally enable.

## Honest-capabilities table

| Claim | Implementation | Verified by |
|---|---|---|
| Liveness health tier (`/v1/models`) | `vllm_ops_mcp/probes.py::_probe_models`, `check_health` | `tests/test_probes.py::TestCheckHealthLiveness`; live: `tests/test_live_smoke.py::test_live_check_health_liveness` |
| Deep/real-completion health tier | `vllm_ops_mcp/probes.py::_probe_completion`, `check_health(deep=True)` | `tests/test_probes.py::TestCheckHealthDeep`; live: `test_live_check_health_deep` |
| Model list passthrough | `probes.py::list_models` | `tests/test_probes.py::TestListModels`; live: `test_live_list_models` |
| On-demand completion sanity check | `probes.py::test_completion` | `tests/test_probes.py::TestTestCompletion`; live: `test_live_test_completion` |
| GPU/VRAM status via nvidia-smi | `probes.py::get_gpu_status` | `tests/test_probes.py::TestGetGpuStatus`; live: `test_live_get_gpu_status` |
| Per-process VRAM attribution | `probes.py::get_gpu_status` (`--query-compute-apps`) | Unit-tested with a mocked non-empty response; **live-verified to return EMPTY on this operator's actual WSL2 setup** -- see Limitations |
| systemd unit status via structured `systemctl show` | `probes.py::get_service_status` | `tests/test_probes.py::TestGetServiceStatus`; live: `test_live_get_service_status` |
| Live-process launch-flag introspection via `/proc/<pid>/cmdline` | `probes.py::get_serve_config`, `parse_launch_flags` | `tests/test_probes.py::TestGetServeConfig`, `TestParseLaunchFlags`; live: `test_live_get_serve_config` -- live-confirmed it recovers flags (`--enable-auto-tool-choice --tool-call-parser qwen3_xml`) that were **not** in this repo's own grounding spec, proving the live-argv approach beats trusting a static doc |
| Exec-script fallback when the unit is down | `probes.py::get_serve_config`, `_parse_exec_script_text` | `tests/test_probes.py::TestGetServeConfig::test_falls_back_to_exec_script_when_service_down` |
| Local config-mirror override path | `probes.py::get_serve_config` + `config.serve_config_override_path` | `tests/test_probes.py::TestGetServeConfig::test_uses_local_override_file`, `test_override_file_missing_reports_error` |
| Sensitive-flag-value redaction in `get_serve_config` | `probes.py::redact_argv`, `redact_raw_text`, `config.SENSITIVE_ARGV_FLAG_PATTERNS` | `tests/test_probes.py::TestGetServeConfig::test_live_process_redacts_sensitive_flag_values`, `test_exec_script_raw_text_redacts_sensitive_flag_values`, `test_local_override_file_redacts_sensitive_flag_values` |
| Deep-inference rate limiting (shared-infra protection) | `config.py::TokenBucket`, `DeepRateLimiter`, `check_deep_rate_limit` | `tests/test_config.py::TestTokenBucket`, `TestDeepRateLimiter` |
| WSL-vs-native shell-out abstraction | `probes.py::_shell_args`, `config.runs_in_wsl` | `tests/test_config.py::TestRunsInWsl`; `tests/test_probes.py::TestGetServiceStatus::test_uses_wsl_prefix_on_windows`, `test_no_wsl_prefix_when_native` |
| All 6 tools registered, no Phase-2 tool present | `vllm_ops_mcp/server.py` | `tests/test_server.py` |

## Limitations (read before relying on this)

- **Per-process VRAM attribution is unreliable under this operator's actual
  WSL2 GPU-passthrough config.** `get_gpu_status`'s live smoke test (real
  `nvidia-smi.exe` against the real running vLLM process) returned an
  **empty** `processes` list even though the same call's `gpus[0].
  memory_used_mib` correctly showed 13.7GB in use by that process. This is a
  documented `nvidia-smi --query-compute-apps` limitation under some
  WSL2/passthrough configurations, not a bug in this server's parsing --
  the tool reports the empty result honestly (`ok: true, processes: []`)
  rather than fabricating attribution. Don't rely on `processes` for
  VRAM-overshoot root-causing on this setup; `gpus[].memory_used_mib` is the
  reliable signal.
- **No computed service uptime.** `get_service_status` returns the raw
  `ActiveEnterTimestamp` string from `systemctl` (already timezone-labeled,
  e.g. `MDT`) rather than computing an elapsed duration server-side --
  deliberately, per this operator's own clock-discipline rule (never state
  an elapsed/ETA time from estimation; artifact timestamps are often UTC
  while local is UTC-6/-7). Compute the delta at the call site if you need
  it, with an explicit timezone conversion.
- **`get_serve_config`'s exec-script fallback is best-effort shell-text
  parsing**, not an exact argv array -- it handles `\` line continuations
  and basic quoting via `shlex.split`, but a sufficiently unusual exec
  script could parse incorrectly. The live-process path (`/proc/<pid>/
  cmdline`) does not have this limitation (it's the kernel's exact argv,
  null-separated) and is always preferred when the unit is up.
- **WSL2-specific by default, but the seam is explicit.** `get_service_status`
  and `get_serve_config`'s live-process path assume this server runs on the
  Windows host and shells into WSL2 via `wsl -d <distro>`. Set
  `VLLM_OPS_MCP_RUNS_IN_WSL=1` if this server process itself runs inside
  WSL2/Linux natively (native `systemctl`/`cat`, no `wsl -d` hop) -- both
  paths are unit-tested (`TestGetServiceStatus::test_uses_wsl_prefix_on_windows`
  / `test_no_wsl_prefix_when_native`), but only the Windows-host path has
  been live-verified against this operator's actual setup.
- **Single vLLM instance, single GPU, this operator's setup.** No
  multi-instance/multi-GPU aggregation; `get_gpu_status` returns whatever
  `nvidia-smi --query-gpu` enumerates, which is every GPU visible to the
  process, not scoped to "the GPU vLLM is using" if more than one exists.
- **No lifecycle control whatsoever in this release.** See "Future
  extension" above -- this is intentional, not an oversight.

## Env vars

| Var | Effect | Default |
|---|---|---|
| `VLLM_OPS_MCP_BASE_URL` | OpenAI-compatible base URL | `http://127.0.0.1:8000/v1` (explicit `127.0.0.1`, not `localhost` -- see `qwen_cli.py`'s IPv6-hang footgun note) |
| `VLLM_OPS_MCP_MODEL` | served-model-name, used in completion payloads | `qwen3-14b` |
| `VLLM_OPS_MCP_WSL_DISTRO` | WSL distro hosting vLLM | `Ubuntu-22.04` |
| `VLLM_OPS_MCP_SERVICE_UNIT` | systemd unit name | `vllm` |
| `VLLM_OPS_MCP_EXEC_SCRIPT_PATH` | WSL-side path to the serve exec script (fallback source) | `~/vllm-systemd-exec.sh` |
| `VLLM_OPS_MCP_SERVE_CONFIG_PATH` | local file mirror of the exec script; if set, `get_serve_config` reads it directly instead of shelling into WSL as a fallback source | unset |
| `VLLM_OPS_MCP_NVIDIA_SMI_PATH` | `nvidia-smi` binary override | `nvidia-smi` (Windows host) / `nvidia-smi.exe` (native WSL2) |
| `VLLM_OPS_MCP_RUNS_IN_WSL` | `1` if this server process itself runs inside WSL2/Linux (skips the `wsl -d` hop) | auto-detected via `sys.platform` |
| `VLLM_OPS_MCP_DEEP_RATE_LIMIT_PER_MIN` | cap on `check_health(deep=True)` + `test_completion` calls/min | `20` |
| `VLLM_OPS_MCP_LIVE` | `1` to run real-infra smoke tests (see Testing) | unset (skip) |

## Usage examples

```jsonc
// A tool call from the MCP host, illustrative -- not a shell command.
{"tool": "check_health", "arguments": {"deep": true}}
// -> {"ok": true, "status": "up", "tier": "deep", "model_id": "qwen3-14b",
//     "latency_ms": 32.0, "completion_s": 0.218, "error": ""}

{"tool": "get_serve_config", "arguments": {}}
// -> {"ok": true, "source": "live_process", "pid": 5645,
//     "flags": {"model": "Qwen/Qwen3-14B-AWQ", "served_model_name": "qwen3-14b",
//               "port": "8000", "max_model_len": "8192",
//               "gpu_memory_utilization": "0.72", "max_num_seqs": "8",
//               "enable_auto_tool_choice": true, "tool_call_parser": "qwen3_xml", ...}}

// rate limit exceeded (21st deep call within a minute):
{"tool": "test_completion", "arguments": {"prompt": "hi"}}
// -> {"ok": false, "error": {"type": "rate_limited", "group": "deep_inference",
//     "tool": "test_completion", "limit_per_min": 20, "retry_after_s": 4.87}}
```

## Testing

CI (`.github/workflows/ci.yml`) runs this suite on every push/PR (ubuntu-latest --
the unit suite mocks HTTP/subprocess, no live server/GPU/WSL required) and
fails the build if the Tests badge above drifts from what the suite
actually reports -- see `scripts/check_readme_counts.py`.

```
# unit suite (mocked HTTP/subprocess, no live server/GPU/WSL required)
.venv\Scripts\python.exe -m pytest -q
# -> 95 passed, 7 skipped

# handshake check -- prints every registered tool name
.venv\Scripts\python.exe scripts\list_tools.py

# real-infra smokes (real vLLM completion, real nvidia-smi, real systemctl
# via WSL, real /proc/<pid>/cmdline read) -- read-only, safe to run anytime
# vLLM is up
VLLM_OPS_MCP_LIVE=1 .venv\Scripts\python.exe -m pytest -v -m live
# -> 7 passed (all live, against the actual running qwen3-14b instance)
```

## Install

```
python -m venv .venv
.venv\Scripts\python.exe -m pip install -r requirements.txt
```

Registered in `~/.claude.json` as `vllm-ops-mcp` (stdio, own `.venv`, no env
overrides needed for this operator's default setup).

## Commercial support

Maintained by [Jaimen Bell](https://jaimenbell.dev). For production MCP
integrations, custom servers, or agent-reliability work, see
[jaimenbell.dev](https://jaimenbell.dev).

Building your own MCP server? The [MCP Starter Kit](https://jaimenbell.gumroad.com/l/adnojp)
has templates, a build playbook, and packaging war-stories from shipping this one.

<!-- MCP registry ownership marker -->
mcp-name: io.github.jaimenbell/vllm-ops-mcp