# kvm-pilot
**Smart hands for your AI agents.** A write-capable, multi-plane
(KVM + BMC + SSH) MCP server for controlling physical machines —
**gated, verified, audited.**
`kvm-pilot` lets an agent drive a headless box through POST, firmware, the
bootloader, and an OS install **with no agent on the target**: it works at the
pixel level through an IP-KVM (PiKVM, the GL.iNet GLKVM fork GL-RM1 /
GL-RM1PE, BliKVM), at the structured-state level through a BMC (Redfish on
iDRAC/iLO/OpenBMC, IPMI on BMCs that predate Redfish), at the firmware level
through **Intel AMT/vPro** (a BIOS/POST/GRUB screenshot + power + SOL on a
business laptop an HDMI-capture KVM can't see boot on), and over SSH once an OS
is up. A pluggable vision subsystem reads a KVM screenshot and tells you what
boot phase the machine is in — `bios_menu`, `grub_menu`, `installer_progress`,
`login_prompt`, `crash_screen`, and so on — and a safety layer gates every
destructive operation behind operator opt-ins and per-call approvals.
Vision runs on Claude **or** any local OpenAI-compatible VLM (LM Studio,
Ollama, vLLM, llama.cpp). Point it at a model on your own GPU and the
screenshots never leave your network and cost nothing per frame.
## How it works
`kvm-pilot` runs a **see → decide → act** loop, and the screen is its only sensor:
it pulls a screenshot from the KVM, a vision model classifies the boot phase, and
`kvm-pilot` acts back through the KVM's keyboard and power. Because it works at the
pixel level, there is **no agent on the target** — the same loop drives POST,
firmware, the bootloader, and an OS install.
A real, unedited run against a GLKVM on the home fleet — an honest preflight,
a headless snapshot (watch it wake the on-demand encoder), a **gated** dry-run
power-cycle, and the boot console the agent actually saw:
## Quickstart
One install gives you the whole product — the **`kvm-pilot` CLI**, the
**`kvm-pilot-mcp` MCP server**, and the bundled **Claude skill** — nothing to
clone.
```bash
pip install kvm-pilot # CLI + skill + MCP server + WebSocket events
pip install "kvm-pilot[totp]" # + 2FA / TOTP support (pyotp)
kvm-pilot install-skill # optional: put the bundled skill where Claude Code loads it
```
### Driving a KVM from an AI agent (MCP)
```bash
claude mcp add kvm-pilot -s user \
-e KVM_PILOT_PROFILE=mykvm -e KVM_PILOT_MCP_READ_ONLY=1 -- \
kvm-pilot-mcp
```
`KVM_PILOT_MCP_READ_ONLY=1` is the recommended first rung of the trust ladder
— the agent can see everything and touch nothing until your hardware is
verified. The [Getting started guide](https://github.com/DustinTrap/kvm-pilot/blob/main/docs/getting-started.md)
covers credentials, Claude Desktop JSON config, sample prompts, and climbing
the ladder. The server is published to the official
[MCP registry](https://registry.modelcontextprotocol.io/v0/servers?search=kvm-pilot)
as **`io.github.DustinTrap/kvm-pilot`**, so registry-aware hosts can discover
and install it by name. Agents: the repo root carries an
[`llms.txt`](https://github.com/DustinTrap/kvm-pilot/blob/main/llms.txt) doc map.
### Scripting from Python
```python
from kvm_pilot import KVMClient
from kvm_pilot.vision import ScreenAnalyzer, make_backend
kvm = KVMClient("192.168.8.1", "admin", "secret")
# Classify the current screen with Claude (model auto-resolved at runtime)
analyzer = ScreenAnalyzer(kvm, make_backend("anthropic"))
print(analyzer.classify().phase)
# Or run entirely on a local VLM — nothing leaves your network
local = make_backend("local", base_url="http://127.0.0.1:1234/v1", model="qwen2.5-vl-7b")
analyzer = ScreenAnalyzer(kvm, local)
# Block until the box reaches the GRUB menu, then pick the first entry
analyzer.wait_for_state("grub_menu", timeout=120)
kvm.press_key("Enter")
```
For the latest unreleased tree:
```bash
pip install "kvm-pilot[totp] @ git+https://github.com/DustinTrap/kvm-pilot"
```
### CLI
```bash
kvm-pilot info --host 192.168.8.1 --user admin --ask-passwd # prompt (no echo)
kvm-pilot capabilities --profile homelab # what this driver supports
kvm-pilot snapshot screen.jpg --profile homelab
kvm-pilot --timeout 60 power-cycle --profile homelab --dry-run # log, don't send
kvm-pilot eject --profile homelab # detach virtual media
kvm-pilot events --profile homelab --count 5 # stream device events
kvm-pilot watch grub_menu --profile homelab \
--backend local --vision-url http://127.0.0.1:1234/v1 --vision-model qwen2.5-vl-7b
```
The CLI prompts for confirmation before any destructive action (power, virtual
media — including uploads — keyboard/mouse injection, GPIO). Use `--yes` to
skip prompts in automation, or `--dry-run` to log intended actions without
sending them — dry-run short-circuits *before* the prompt, so it never blocks
waiting for input. `--timeout` (HTTP per-request timeout) is a global flag and
goes *before* the subcommand; `watch` keeps its own `--timeout` for the vision
wait deadline.
Profiles like `homelab` live in `~/.config/kvm-pilot/config.toml`. See
[docs/cli.md](https://github.com/DustinTrap/kvm-pilot/blob/main/docs/cli.md) for the full command table (every subcommand, the
capability it needs, and its gating), and
[docs/configuration.md](https://github.com/DustinTrap/kvm-pilot/blob/main/docs/configuration.md) for the config-file format,
every `KVM_PILOT_*` environment variable, and the precedence between flags,
env, and profiles.
> **GLKVM setup note:** on GL.iNet firmware the PiKVM REST API is **disabled by
> default** and a firmware upgrade can re-disable it — enable it first, per
> [Every `/api/*` call returns 404](https://github.com/DustinTrap/kvm-pilot/blob/main/docs/troubleshooting.md#every-api-call-returns-404-glkvm).
## The tool surface, by plane
The same capability protocols span three actuation planes, so one agent
workflow can mix pixels, structured BMC state, and shell access — with every
destructive effect gated per class:
| Plane | Read | Act (operator-gated) |
|---|---|---|
| **KVM — pixels & HID** (PiKVM · GLKVM · BliKVM) | `snapshot` · `classify_screen` · `wait_for_state` · `power_state` · `logs` · `list_virtual_media` | `power` · `type_text` / `press_key` / `send_shortcut` / `mouse` · `calibrate_mouse` · `mount_iso` / `eject` |
| **BMC — structured state** (Redfish · IPMI) | `info` · `boot_options` · `logs` (SEL) · sensors (CLI) | `power` · `set_boot_device` · SOL console (CLI `console`) |
| **Firmware — Intel AMT/vPro** (spans both planes) | `info` · `snapshot` (firmware BIOS/POST/GRUB) · `boot_options` · `power_state` | `power` · `set_boot_device` · `type`/`mouse` · SOL (`console`) · `amt_enable` (open SOL/KVM listeners) |
| **SSH — in-band & appliance** | `ssh_reachable` · `appliance_status` · `access_paths` | `ssh_exec` · `host_exec` (ssh/winrm auto-picked) · `wake` (WoL) · `appliance_reboot` |
| **Meta — evidence & intake** | `capabilities` · `support_matrix` · `healthcheck` | `file_firmware_report` |
A machine with **no KVM and no BMC at all** is itself a first-class target:
`driver = "ssh"` **plus `ssh_host`** (the machine's own address, #248) stands
kvm-pilot on the recovery ladder's in-band rung — every device capability
honestly absent, and the healthcheck saying plainly that there is no
out-of-band recovery path there.
The canonical per-tool reference — annotations, effect gates, approval
lifecycle — is the [MCP server README](https://github.com/DustinTrap/kvm-pilot/blob/main/src/kvm_pilot/mcp/README.md);
the CLI covers the full surface in [docs/cli.md](https://github.com/DustinTrap/kvm-pilot/blob/main/docs/cli.md).
## Status & maturity
> **Status: 1.0, general availability** — on the strength of the run ledger,
> not beyond it. Three device types have run live (GL-RM1PE, Dell R710 over
> iDRAC6/IPMI, Dell Latitude 5411 over Intel AMT); three of the seven device
> drivers (`redfish`, `pikvm`, `blikvm`) have only ever run against emulators.
> See the
> [Hardware-Compatibility list](https://github.com/DustinTrap/kvm-pilot/wiki/Hardware-Compatibility)
> for what has actually been exercised — and please add to it. (The exact version lives in the
> [CHANGELOG](https://github.com/DustinTrap/kvm-pilot/blob/main/CHANGELOG.md).)
> The core paths have graduated
> from mocked-only to live-verified: a fleet of GL-RM1PE units has exercised
> `snapshot`/`healthcheck`/`logs`/`power_state`/`virtual_media`/`info` across
> two firmware lines — on V1.9.1 the run ledger that ships in the wheel puts
> `info`/`logs`/`healthcheck`/`power_state` at **ga**, `snapshot` at **rc**, and
> `virtual_media` at **beta** — derived from real runs, never hand-edited — and a Dell iDRAC6 has exercised the IPMI driver live end-to-end
> (power, boot-device, sensors, event log, SOL serial console). The paths that
> can hurt are hardened: transports never re-fire a destructive request, MCP
> approvals are signed single-use receipts with an audit trail, and every
> destructive effect — power, HID, media, boot-config, appliance, SSH,
> external writes — has its own operator opt-in gate. Recent betas added
> remote boot-device control (Redfish, IPMI, and in-band `efibootmgr`),
> Wake-on-LAN, an IPMI driver for BMCs that predate Redfish, a serial (SOL)
> console, mouse auto-calibration, and headless native-resolution GLKVM
> snapshots; `kvm-pilot test-report` turns contributing evidence into one
> command, and the firmware registry feeds itself (`firmware-check` auto-files
> registry updates).
> **Now we need your hardware.** PiKVM, BliKVM, other GLKVM models, and
> Redfish BMCs (iDRAC/iLO/OpenBMC) are the combos the matrix needs most —
> success *or* failure, a
> [hardware report](https://github.com/DustinTrap/kvm-pilot/issues/new?template=hardware-report.yml)
> takes two minutes and the hourly ingest does the rest. Anything the
> [Hardware-Compatibility list](https://github.com/DustinTrap/kvm-pilot/wiki/Hardware-Compatibility)
> doesn't show as exercised is still unverified: the 1.x line keeps the public
> API stable, but a rating only moves when a live run moves it; note the remote
> firmware-flash no-op on GL-RM1PE
> ([#94](https://github.com/DustinTrap/kvm-pilot/issues/94)/[#95](https://github.com/DustinTrap/kvm-pilot/issues/95)),
> and don't point destructive ops at a machine you can't afford to have
> power-cycled unexpectedly. See [Compatibility](#compatibility).
## Boot-phase detection
The vision classifier maps each screenshot to a **phase** — `bios_menu`,
`grub_menu`, `installer_progress`, `login_prompt`, `crash_screen`, and so on.
`wait_for_state()` polls the screen and blocks until the phase you asked for
appears (or a timeout fires), so an unattended install becomes a few waits with
actions wired between them:
## Sensing model
Vision is the most expensive way to read a screen — a model call per frame — and
most of what it infers (power state, boot phase, liveness, a crash) is also
available as a **field, an event, or a line of text**. The direction of
`kvm-pilot` is to treat classification as a hierarchy: answer from the cheapest
signal the device exposes, and fall through to OCR and finally a vision model
only when nothing cheaper can.
The PiKVM/GLKVM client already exposes the cheap end — ATX and HID LEDs,
video-signal and resolution, on-device OCR (`?ocr=true`), logs, Prometheus
metrics, and a WebSocket event stream. The [capability protocols](https://github.com/DustinTrap/kvm-pilot/blob/main/docs/architecture.md)
add `Logs`, `BootProgress`, `Sensors`, `SerialConsole`, `Watchdog`, and
`BootConfig` as the seam for BMC drivers (Redfish/IPMI), where the boot phase
is a structured enum (`BootProgress.LastState`) and the console is a serial
text stream rather than pixels. Different device classes are nearly
complementary: capture devices are strong on pixels, BMCs on structured state
and serial text.
## Safety model
Power-offs, hard resets, virtual-media connect/disconnect and image uploads,
keyboard/mouse injection (`type_text`, `press_key`, shortcuts, clicks), GPIO,
boot-config changes, and Redfish/IPMI resets are classified as **destructive**
and pass through a safety layer:
- **dry-run** short-circuits *first*: it logs the intended call and skips it
entirely — the confirm callback is never invoked, so dry runs never prompt
or block.
- **confirmation** — a callback that can veto any destructive call that would
really be sent. The library default allows everything (so plain scripts
work); the CLI installs an interactive `y/N` prompt unless you pass `--yes`.
The destructive set is defined explicitly in `kvm_pilot.safety.DESTRUCTIVE_OPS`
so it is auditable rather than guessed. A vision classification can never
trigger a destructive action on its own — you wire that yourself, and the
safety layer still applies. On the MCP side each destructive *effect class*
additionally needs an operator opt-in env gate plus a per-call approval backed
by a signed single-use receipt — the **trust ladder**
(`READ_ONLY` → `DRY_RUN` → per-effect `ALLOW_*`) is drawn in the
[MCP server README](https://github.com/DustinTrap/kvm-pilot/blob/main/src/kvm_pilot/mcp/README.md).
This software controls real hardware and can power-cycle or interrupt a running
machine. Read [SECURITY.md](https://github.com/DustinTrap/kvm-pilot/blob/main/docs/SECURITY.md) before exposing a KVM to the internet.
## No hard-coded model version
There is no model version string anywhere in the code. The Anthropic backend
resolves the newest vision-capable model at runtime via the Models API and
caches it; set `KVM_PILOT_VISION_MODEL` or pass `model=` to pin one. The local
backend uses whatever model you loaded on your server. Bring your own backend,
endpoint, and model.
## How this differs from other clients
[`pikvm-lib`](https://github.com/guanana/pikvm-lib) is a fine general-purpose
PiKVM client. `kvm-pilot` is aimed at a different job:
- **Vision-based boot-phase detection** — classify BIOS/GRUB/installer/crash
states from screenshots, with blocking `wait_for_state` loops. This is the
core feature and `pikvm-lib` has no equivalent.
- **Pluggable local or cloud VLM** — run inference on your own GPU at zero
per-frame cost, or on Claude.
- **A safety layer** around destructive operations (dry-run + confirmation).
- **GLKVM-fork awareness** — documents the API-enable prerequisite and GL
hardware quirks that bite GL-RM1PE users.
- **Stdlib-only client core** — the driver/vision code imports only the standard
library (the bundled MCP server pulls the `mcp` SDK; feature extras are opt-in).
If you just want to script power and HID against a stock PiKVM and don't need
the vision layer, `pikvm-lib` may be the simpler choice.
On the BMC side, [sushy](https://opendev.org/openstack/sushy), DMTF's
[python-redfish-library](https://github.com/DMTF/python-redfish-library), and
[pyghmi](https://opendev.org/x/pyghmi) (IPMI) are mature, far more complete BMC
management SDKs — if you need account/firmware/network configuration,
EventService subscriptions, or hardware-proven maturity, use them. `kvm-pilot`
trades that completeness for one uniform capability surface across device
classes (IP-KVMs and BMCs behind the same protocols), the same safety layer
gating every destructive call, and the vision loop on devices that have pixels.
## Compatibility
| Device | Status |
|--------|--------|
| GL-RM1PE (Comet PoE) | Primary target — **exercised live**: read/`healthcheck`/`logs` verified on firmware V1.5.1 release2 & V1.9.1 release1; `snapshot` verified on V1.9.1 (on V1.5.1 it fails with a clear error — undecodable H.264 frame, [#107](https://github.com/DustinTrap/kvm-pilot/issues/107)/[#151](https://github.com/DustinTrap/kvm-pilot/issues/151)); remote flash a no-op ([#94](https://github.com/DustinTrap/kvm-pilot/issues/94)/[#95](https://github.com/DustinTrap/kvm-pilot/issues/95)); encoder wedges >1080p ([#107](https://github.com/DustinTrap/kvm-pilot/issues/107)) |
| Dell iDRAC6 — IPMI (PowerEdge R710) | **Exercised live**: power / boot-device / sensors / event log (SEL) / SOL serial console all verified over `ipmitool` lanplus (fw 1.95) |
| Dell Latitude 5411 — Intel AMT/vPro | **Exercised live** (AMT 14.1.67): WS-Man power / info / single-use boot, remote SOL + KVM enablement, and a **1920×1080 BIOS/POST screenshot over KVM redirection** — the firmware screen an HDMI-capture KVM can't see on a laptop. Captures graphical screens only (not legacy VGA text mode) |
| GL-RM1 (Comet) | Expected to work (same firmware family); untested |
| PiKVM v3 / v4 | Expected to work (upstream API); untested |
| BliKVM | Expected to work (PiKVM-compatible API); untested |
| Redfish BMCs (iDRAC7+, iLO, OpenBMC) | Emulator-verified (in-repo emulator + DMTF sushy-tools in CI); live-BMC validation pending ([#29](https://github.com/DustinTrap/kvm-pilot/issues/29)) |
The GL-RM1PE (read/snapshot paths), a Dell iDRAC6 over IPMI, and a Dell Latitude
5411 over Intel AMT are the combos run live so far — everything else is "expected
to work" pending validation. The
[Hardware-Compatibility list](https://github.com/DustinTrap/kvm-pilot/wiki/Hardware-Compatibility)
is the authoritative, per-capability record. ATX power control needs the
ATX adapter wired to the target's front-panel header: on the GL Comet family
(GL-RM1 / GL-RM1PE) that is GL.iNet's separately sold ATX board (GL-ATXPC),
while PiKVM v3/v4 kits include the ATX adapter in the box and BliKVM bundles
vary by model — check yours. Without ATX wiring, ATX calls return errors from
the device. Reports of success or failure on *any* hardware are exactly what
this project needs — please open a
[hardware report](https://github.com/DustinTrap/kvm-pilot/issues/new?template=hardware-report.yml).
## Architecture
`kvm-pilot` is built on a modular, **driver-plugin** architecture so support can
expand to many KVM/BMC devices (PiKVM family, Redfish BMCs, IPMI BMCs, JetKVM, …).
Each device implements only the capability protocols its hardware supports; the
CLI, safety layer, and vision subsystem stay device-agnostic. A `make_driver(kind)`
registry (mirroring `make_backend`) builds drivers by name, and a hardware-free
`FakeDriver` lets you exercise the whole loop — capabilities, safety gating, the
analyzer — with no device (`kvm-pilot capabilities --driver fake`). See
[docs/architecture.md](https://github.com/DustinTrap/kvm-pilot/blob/main/docs/architecture.md) for the design and diagram.
A **`RedfishDriver`** (`make_driver("redfish")`) speaks the DMTF Redfish API to
server BMCs — Dell iDRAC, HPE iLO, Supermicro, Lenovo XCC, OpenBMC — in one
stdlib-only client. It shows why capabilities are segmented: a BMC's set is
*complementary* to a PiKVM's (strong on structured state — power, boot phase,
sensors, logs, virtual media — with no keyboard/mouse/screenshot), and the driver
stays portable by following Redfish hypermedia rather than hard-coding vendor ids:
```python
from kvm_pilot.drivers import make_driver
bmc = make_driver("redfish", host="idrac.lan", user="root", passwd="…")
bmc.get_boot_progress() # 'os_running' — structured, no screenshot
bmc.read_sensors()["temperatures"]
bmc.power_off(wait=True) # mapped to the target's actual ResetType, gated
```
An **`IpmiDriver`** (`make_driver("ipmi")`) covers BMCs that predate Redfish
(e.g. Dell iDRAC6) over the system `ipmitool`: power, boot-device control,
sensors, the SEL event log, and an SOL serial console (`kvm-pilot console`).
Both are on the CLI too — `kvm-pilot info --driver redfish --host idrac.lan …`.
Capability-specific subcommands a BMC can't serve (`type`, `snapshot`, `events`)
fail cleanly rather than crashing. Add `--redfish-auth basic` for an endpoint
without a SessionService (emulators, or a BMC with session auth disabled).
An **`AmtDriver`** (`make_driver("amt")`) manages Intel AMT/vPro laptops and
desktops out-of-band over AMT's three native protocols — WS-Man (power / info /
single-use boot), SOL (`kvm-pilot console`), and **KVM redirection** — all
pure-stdlib. It is the **first non-PiKVM driver with `Video` + `HID`**: `snapshot`
returns a real firmware-level **BIOS/POST/GRUB screenshot** on a machine whose
HDMI a capture-KVM can't see boot on a laptop. SOL and KVM listeners are opened
remotely over WS-Man (`kvm-pilot amt enable-sol` / `enable-kvm`), no MEBx trip.
## Help shape it — this is where you come in
`kvm-pilot` is built in the open. 1.0 shipped on the evidence of three device
types; what it still needs is **not code — it's breadth of real hardware**.
Three of the seven drivers
(`redfish`, `pikvm`, `blikvm`) have never run against a physical device, only
emulators. Every rating you see is derived from a run ledger that ships inside
the wheel, never hand-set, so the matrix only grows when someone runs it on real
metal. That someone could be you.
**The single most valuable thing you can send:** a
[hardware report](https://github.com/DustinTrap/kvm-pilot/issues/new?template=hardware-report.yml).
`kvm-pilot test-report --profile ` turns it into one command, it is
read-only by default, and an hourly job folds the result into the published
compatibility matrix. **Failures are as welcome as successes** — a driver that
returns a confident wrong answer on your BMC is a more useful report than one
that works, and it will be treated as a first-class finding, not swept up.
Also genuinely wanted:
- ⭐ **[Star the repo](https://github.com/DustinTrap/kvm-pilot)** if the idea is
useful to you. It is the cheapest signal that this problem is worth solving,
and it is what brings in the operators whose hardware the matrix still needs.
- 🐛 **[Open an issue](https://github.com/DustinTrap/kvm-pilot/issues/new/choose)**
for anything: a bug, a device that misbehaves, a confusing error, a doc that
lied to you. This repo is *issue-per-finding* — an issue is the unit of record,
and "the error message didn't tell me what to do" is a legitimate report.
- 💬 **Tell us what it got wrong.** Review the safety model, the approval flow,
the tool surface. Push back on the defaults. If an agent did something with
your machine that surprised you, that is exactly the feedback that makes this
safe for everyone else.
- 🔌 **Ask for your device.** Missing driver, unsupported BMC, a KVM we've never
heard of? Open a driver request — the plugin architecture exists precisely so
that adding one doesn't mean forking the project, and
[`docs/plugin-development.md`](https://github.com/DustinTrap/kvm-pilot/blob/main/docs/plugin-development.md)
is the contract.
- 🛠️ **Send a PR.** [`docs/CONTRIBUTING.md`](https://github.com/DustinTrap/kvm-pilot/blob/main/docs/CONTRIBUTING.md)
has the full pre-PR checklist; good first issues are
[labelled](https://github.com/DustinTrap/kvm-pilot/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22).
Nothing here phones home. The project has no telemetry that reports itself —
every row in the compatibility matrix is there because a human chose to send it.
That's the deal, and it's why your report actually matters.
## Documentation
Full user and developer docs live in [`docs/`](https://github.com/DustinTrap/kvm-pilot/tree/main/docs/) (architecture, design
decisions, the Redfish reference, the
[troubleshooting & FAQ](https://github.com/DustinTrap/kvm-pilot/blob/main/docs/troubleshooting.md),
contributing, and the security policy). The
[project wiki](https://github.com/DustinTrap/kvm-pilot/wiki) is an
auto-generated, nicely formatted mirror of that folder, and the repo root
carries an [`llms.txt`](https://github.com/DustinTrap/kvm-pilot/blob/main/llms.txt)
doc map for AI agents.
## License
Apache License 2.0 — see [LICENSE](https://github.com/DustinTrap/kvm-pilot/blob/main/LICENSE) and [NOTICE](https://github.com/DustinTrap/kvm-pilot/blob/main/NOTICE). `kvm-pilot` is
independent and not affiliated with or endorsed by the PiKVM project, GL.iNet,
or Anthropic; those names are used only for compatibility description.