--- name: guardrail-benchmark description: >- Measure how well a guardrail works, two ways. DIRECT mode sends a bundled labelled corpus (harmful vs benign, plus flattened multi-turn) straight to the guardrail or classifier API, with no model in the loop, and scores its own verdict: catch rate against false-block rate, by technique. IN-LINE mode runs standard Inspect benchmarks with the same model behind each guardrail arm and reports the risk each guardrail removes beside what it costs in over-blocking, paired on the same samples. Use to characterise one guardrail, compare vendors, or show the effect of enabling one. The guardrail is supplied by the user. For an app endpoint use app-benchmark; for comparing models use model-benchmark. license: MIT compatibility: "Direct mode requires python>=3.11 and node>=18 (promptfoo). In-line mode requires Python 3.11+ and a separate virtual environment with inspect_ai, inspect_evals and openai." --- # guardrail-benchmark — how good is this guardrail? You are running this skill **on behalf of a user** who wants evidence about a guardrail they already have or are considering: a vendor guardrail API, a classifier or guardrail model, or a guardrail configured on their model gateway. Drive **Plan → Run → Analyze**. The arm that varies is the **guardrail**; the corpus or benchmark, and in in-line mode the model, are held fixed. This skill measures. It ships no guardrail and does not say whether a result is acceptable. Execution tier: **T0/T1**. Use `AskUserQuestion` for choices. | Mode | Question it answers | Engine | Judge | |---|---|---|---| | **Direct** | Given harmful and benign text, what does the guardrail catch and what does it wrongly block? | promptfoo | none: the guardrail's own verdict is scored | | **In-line** | With this guardrail in front of our model, how much does risk drop inside a real benchmark, and how much legitimate work gets blocked? | Inspect | only for judged benchmarks | Run direct mode to characterise or compare guardrails cheaply. Run in-line mode when the question is the effect on a deployed model, including injected tool output during agent tool calls, which direct mode cannot see. --- ## PLAN ### 1. Choose the mode, and record the context (`AskUserQuestion`) Ask which question above the user has. If a `review.json` exists it is recorded with the results; otherwise note where the guardrail sits (input, output, tool results) in the manifest. ### 2a. Direct mode — profile the guardrail `bash install_dependencies.sh` (installs promptfoo locally). Then gather: - **Endpoint URL** + **auth** (the scan/classify route — not a model). - **Request schema** — where the candidate text goes (`{messages:[…]}`, `{input:…}`, `{text:…}`, `{prompt:…}`). Edit `promptfooconfig.yaml` `providers[0].config.body`, keeping `{{prompt}}` for the candidate. - **Verdict shape** — how a block is signalled. `adapters/generic_guardrail.js` already recognises `action:block`, `blocked:true`, `flagged:true`, `is_malicious:true` and a `400` status. If the vendor differs, set `GUARDRAIL_BLOCK_STATUSES` (e.g. `400,446`) and/or `GUARDRAIL_BLOCK_FIELD` + `GUARDRAIL_BLOCK_VALUE` (dotted path) in `.env`. Confirm with one curl of a known-bad and a known-good input before a full run. - **Channel:** input or output guardrail? Run them as separate experiments. `cp env.example .env`; fill `GUARDRAIL_URL`, `GUARDRAIL_API_KEY` and any overrides. Validate: `npx promptfoo validate -c promptfooconfig.yaml`. **Depth** (the labelled corpus is bundled in `corpus/`, about 2,900 cases): `smoke` (`--filter-sample 30`), `standard` (`--filter-sample 150`), or `full` (omit the flag). One request per case, no model calls. For a vendor comparison plan one run per vendor with the same depth. ### 2b. In-line mode — profile the gateway ``` uv venv .venv-inspect .venv-inspect/bin/python -m pip install inspect_ai openai 'inspect_evals[agentdojo,cyberseceval_4]' ``` Gather: - **Gateway base URL + key** — the OpenAI-compatible gateway that applies guardrails. - **Connector:** `litellm` (guardrail param = top-level `guardrails:[name]`, the default) or `openai` (configurable field, the base for other gateways). - **Model under test** — the same model on every arm. - **Guardrail names**, exactly as registered in the gateway. - **Judge model** on the same gateway, if a judged benchmark is selected. It is called directly, never through a guardrail, and must not be the model under test. Write `arms.json`: a baseline with no guardrail, then one entry per guardrail: ```json [{"name": "baseline"}, {"name": "vendor-a", "guardrails": ["vendor-a-prompt-injection"]}] ``` **Confirm the guardrail fires before spending a run:** curl the gateway with a known-bad prompt, once without and once with the `guardrails` param. If the guarded call does not block (refusal or 4xx), the name or gateway config is wrong. **Checks** (`--checks`): the `guardrail` profile is `cyse4_mitre` (direct misuse), `cyse4_mitre_frr` (benign security questions refused — the over-block cost), `cyse4_multilingual_prompt_injection` and `agentdojo` (injected tool output). Any check id from `lib/inspect_suite.py` works. **Depth:** `smoke` (25 — wiring only), `standard` (100), `full`, or a number. Show the cost first: ``` python run_inline.py --arms arms.json --checks guardrail --depth standard --plan ``` ### 3. Open an experiment folder `bash new_experiment.sh guardrail-benchmark "