# Spec: Harness Executor ## Status Partially implemented. ## Scope Unlike specs 060-066, `harness` is a built-in Go executor, not a remote action: it ships in the `dagu` binary and never reaches the network to resolve itself. Its purpose is to run a third-party AI coding agent CLI (for example `claude`, `codex`, `aider`, `cursor`, or a custom CLI) as a subprocess, feeding it a prompt and capturing its output like an ordinary command step. This spec covers: - the `action: harness.run` step shape: `with.prompt` (required, becomes the step's command/prompt text), `with.stdin` (optional, piped to the process's stdin), `with.provider` (required, selects a built-in CLI provider or a custom `harnesses:` entry), and `with.fallback` (an ordered list of alternative provider configs tried after the primary fails) - any other `with:` key becomes a CLI flag passed to the provider's binary: a boolean `true` becomes a bare flag, a string or number becomes `--key value`, and an array repeats `--key` once per element - the top-level `harnesses:` block, which defines a named, reusable custom provider (`binary`, `prefix_args`, `prompt_mode`, `prompt_flag`, `prompt_position`, `flag_style`, `option_flags`) that a step selects via `with.provider`, for a CLI the built-in provider catalog does not name - capturing stdout through a declared `output: NAME` and reading it in a downstream step ([Spec 012: Step Outputs](012-step-outputs.md)) - that, unlike the remote actions in specs 060-066, `dagu validate` fully resolves and checks a harness step's provider configuration and any `harnesses:` definitions it references, since both are local to the DAG file and this binary -- the one thing it cannot check is whether the configured binary actually exists, since that is a real filesystem/PATH lookup deferred to run time - fallback behavior: when the primary provider's process exits non-zero, the step retries with the next entry in `with.fallback`, in order, until one succeeds or the list is exhausted; a message naming both providers is written to the step's stderr before each retry This spec also covers the configuration and dispatch failures of containerized harness execution (a step-level `container:` block on a `harness.run` step) that are reachable without a working container daemon or a pulled image -- neither guaranteed in every environment this suite runs in, so a step's *successful* containerized execution is not itself covered here. This spec does not define: - the behavior of any specific built-in CLI provider (`claude`, `codex`, `aider`, and so on) -- none is installed in this conformance environment, so this spec exercises only custom `harnesses:` providers, which share the same invocation-building and fallback code paths - a containerized harness step's successful execution (the actual agent CLI running inside a real container and producing output), the managed OpenCode execution path's successful session, or `type: agent` DAGs' own use of a harness step as one of several actions in a decision loop ([Spec 032: Agent DAGs](032-agent-dag.md)) - prompt engineering, model behavior, or the semantics of any particular agent CLI's own flags ## Goal Workflow authors give an AI coding agent CLI a task and capture its output as an ordinary Dagu step, whether that CLI is one of the built-in provider names or an arbitrary custom CLI wired in through `harnesses:`. ## Behavior ### Step shape and invocation building A harness step is written `action: harness.run` with `with.prompt` required; `with.prompt` becomes the step's command text (equivalent to a `run:` step's command), and `with.stdin`, when set, is piped to the process's stdin as a separate script body. `with.provider` selects either a built-in CLI provider name or a name defined under the DAG's top-level `harnesses:` block; an unknown name fails with `unknown provider`. For a custom (`harnesses:`-defined) provider, the invocation is built as: 1. `prefix_args`, if any. 2. The prompt and the flags built from the step's other `with:` keys, in the order `prompt_position` says (`before_flags`, the default, or `after_flags`), with the prompt delivered according to `prompt_mode`: - `arg`: the prompt is a bare positional argument. - `flag`: the prompt follows `prompt_flag` (required only for this mode; setting it for any other mode is itself an error). - `stdin`: no prompt argument at all; the prompt and `with.stdin` are joined with a blank line (either half omitted if empty) and both piped to the process's stdin instead. 3. `with.stdin`, when set and `prompt_mode` is not `stdin`, is still piped to the process's stdin as a separate stream from the argument-delivered prompt. The input keys `prompt` and `stdin`, and executor options `provider`, `fallback`, and `managed`, do not become CLI flags. Other keys are sorted by key for deterministic ordering. Each key becomes `--key` (or `-key` when the definition's `flag_style` is `single_dash`) unless `option_flags` maps that key to a different literal flag token. A boolean `true` value produces the bare flag with no value (`false` is omitted entirely); a string, integer, or float value produces `flag value`; an array value repeats `flag value` once per element. ### Model output in shell commands Dagu does not escape values inserted into a shell command (Spec 003), and a model's reply can carry whatever text the model read. So a command step whose shell-parsed text inserts the output of a `harness.run`, `chat.completion`, or agent step produces a non-fatal validation warning naming its DAG, step, and reference. A reference that reads a step's exit code or log path does not warn, and neither does an escaped reference. A field that the step's `output_schema` declares with `enum`, or as `boolean`, `integer`, or `number`, does not warn, because validation limits its values. The warning is reported the same way as a working-directory warning. The fix is to pass the value through step `env` and read it from the environment. How to read it safely depends on what runs the command: Dagu leaves `"$NAME"` to a POSIX or Nix shell on the local host, but writes it into the text in other cases (Spec 003), such as a container with a shell, where `"$(printenv NAME)"` reads it safely. ### Working-directory warnings A harness step without an explicitly configured working directory produces a non-fatal validation warning identifying its DAG and step. This includes steps inside `foreach` bodies, lifecycle handlers, and local DAG documents. - For host execution, a step `working_dir`, a root `working_dir` (including an inherited base configuration), or an explicitly configured default satisfies the check. Automatically selected manifest or run directories do not. - For container execution, the effective container configuration must declare `working_dir` to suppress the warning. A step container takes precedence over the root container. A host working directory does not set the agent's directory inside the container; the check also applies to `container.exec`. - Relative paths and runtime expressions count as configuration. Validation does not check directory existence or change directory resolution or creation. `dagu validate` prints the warning and exits successfully when there are no errors. The spec API returns warnings separately from errors, and the DAG editor displays them without blocking saves or execution. Warnings are transient and do not change retry behavior. For example, configure the workspace once for host harness steps: ```yaml working_dir: ./repo steps: - id: review action: harness.run with: provider: claude prompt: Review this repository. ``` For a containerized step, set `container.working_dir: /repo` and mount the intended repository there using the existing container volume configuration. ### Containerized execution A step-level `container:` block runs the resolved provider's binary inside an image-created container, using that binary as its entrypoint. `dagu validate` checks the general harness configuration. The following container-specific restrictions are checked at runtime: - `prompt_mode: stdin` fails with `harness: containerized harness does not support stdin input`. - `container.name` is rejected for image-mode harness steps. Running inside an existing container uses `container.exec`. - `provider: opencode` with `managed: true` fails with `harness: managed OpenCode is not supported inside containers`. ### Fallback `with.fallback` is an array of provider configs (each shaped like the step's own `provider` plus flags), tried in order after the primary config's process exits non-zero. Before each retry, the step's stderr receives a line naming both the failed and the next provider (`harness: attempt / with failed; trying fallback / with `). The step succeeds as a whole the moment any config succeeds, and that config's stdout becomes the step's stdout; each earlier attempt's own stderr remains part of the step's stderr. If every config fails, the step fails with the last config's own error. ### Result A harness step captures the provider process's stdout and stderr. A declared `output: NAME` makes stdout available to downstream steps, following [Spec 012: Step Outputs](012-step-outputs.md). ## Errors ### Validation `dagu validate` resolves a harness step's provider configuration and any `harnesses:` definitions fully -- both are local to the DAG file and this binary, unlike a remote action's schema. It checks: - `with.prompt` missing: `with.prompt is required`. - `with.provider` missing, or naming neither a built-in provider nor a defined `harnesses:` entry: `config.provider is required` or `unknown provider ""`. - `with.binary` or `with.prompt_args` present: both are rejected outright (`config.binary is not supported...` / `config.prompt_args is not supported...`) -- a custom CLI must be defined under `harnesses:` and selected via `with.provider`, not configured inline on the step. - A `harnesses:` entry missing `binary`: `binary is required`. - A `harnesses:` entry's `prompt_mode` not one of `arg`, `flag`, `stdin`: `prompt_mode must be one of: arg, flag, stdin`. - A `harnesses:` entry's `prompt_flag` set when `prompt_mode` is not `flag`, or unset when it is: `prompt_flag is only valid when prompt_mode is flag` / `prompt_flag is required when prompt_mode is flag`. `dagu validate` does not check whether the configured binary actually exists on disk or in `PATH` -- that is deferred to run time. ### Runtime - The configured binary cannot be resolved (it does not exist, or is not on `PATH`): the step fails with `harness: failed to resolve binary "": ...` before any process starts. - The provider's process exits non-zero, with no `with.fallback` configured (or every fallback also fails): the step fails with that process's own real exit code, and its own stdout/stderr are captured as usual -- this is a real process failure, not a wrapper-level validation error. - A containerized step (`container:` set) whose provider's `prompt_mode` is `stdin`, whose `container.name` is set (image mode), or whose `provider: opencode` step sets `managed: true`: each fails as described above in Containerized execution, before any container daemon is contacted. - A containerized step whose configured container daemon cannot be reached (wrong socket, daemon not running): the step fails with `harness: failed to initialize container client: ...`. ## Related Specs - Step outputs and reference syntax, for the standard `output: NAME` mechanism this executor uses instead of a JSON outputs contract: [Spec 012: Step Outputs](012-step-outputs.md) - Agent DAGs, which can use a harness step as one action among several in an LLM-driven decision loop: [Spec 032: Agent DAGs](032-agent-dag.md) ## Examples Define a custom CLI provider and use it with structured flags: ```yaml harnesses: my_agent: binary: /usr/local/bin/my-agent prompt_mode: flag prompt_flag: --task option_flags: max_turns: --turns steps: - id: review action: harness.run with: provider: my_agent prompt: Review this patch for correctness. stdin: | diff --git a/main.go b/main.go ... max_turns: 3 output: REVIEW_RESULT - id: print depends: review run: echo "${REVIEW_RESULT}" ``` Fall back to a second provider if the first fails: ```yaml steps: - id: fix action: harness.run with: provider: claude prompt: Fix the failing test. fallback: - provider: codex ``` ## ACP harness providers ### Local stdio Custom harness definitions select ACP with `protocol: acp`. Omitting `transport` defaults to stdio. Omitting `protocol`, or setting it to `cli`, retains CLI behavior. Install and authenticate the adapter before running the workflow. ```yaml harnesses: coding-agent: protocol: acp binary: codex-acp steps: - action: harness.run with: provider: coding-agent prompt: Review the project without modifying files. output: REVIEW ``` `binary`, `prefix_args`, `flag_style`, and `option_flags` configure local startup. Other `with` options become adapter CLI flags. ACP definitions reject CLI prompt placement options (`prompt_mode`, `prompt_flag`, `prompt_position`); `with.stdin` is appended to the prompt after a blank line instead of sent to the protocol pipe. ### Remote WSS and Streamable HTTP Remote definitions require `url` and an absolute `cwd` on the **agent host**. They do not accept local startup settings (`binary`, `prefix_args`, `flag_style`, `option_flags`). URL, cwd, and header values support workflow variables. ```yaml harnesses: remote-agent: protocol: acp transport: http # websocket for ws:// or wss:// endpoints url: https://acp.example.com/acp # wss://acp.example.com/acp for secure WebSocket cwd: /srv/project # ca_file: /path/to/private-ca.pem headers: Authorization: Bearer ${ACP_TOKEN} steps: - action: harness.run timeout_sec: 120 with: provider: remote-agent prompt: Review the project without modifying files. output: REVIEW ``` Supply tokens through workflow environment/secrets settings. Remote steps accept `provider`, `fallback`, and `acp_permission`, not adapter CLI flags. Of the DAG-level `harness:` options, remote steps inherit only `acp_permission`, also when a variable selects the provider; options set on the step are still rejected. `url` and `cwd` values that reference variables are validated after resolution. Headers sent over `ws://` or `http://` to a non-loopback host produce a warning in the step log. The harness definition's `ca_file` adds PEM roots alongside system certificates for this connection only; relative paths resolve from the local step working directory. Certificate and hostname verification remain enabled. Dagu never creates or translates remote cwd. - **WebSocket:** accepts `ws://` and `wss://`, advertises `acp.v1`, and sends headers in the upgrade request. By default, bearer credentials appear only in the Authorization header. Set `bearer_subprotocol: true` on a WebSocket definition only when the bridge requires `bearer.`; the token must satisfy HTTP subprotocol token syntax. Tokens containing `/` or `=` cannot use that option. Text frames may contain individual or newline- batched messages; outbound NDJSON delimiters are preserved for stdio bridges. Binary frames are rejected. WebSocket control Ping frames keep idle connections alive. - **HTTP:** accepts `http://` and `https://`, following the ACP SDK's experimental Streamable HTTP profile. Initialize returns JSON and `Acp-Connection-Id`. Subsequent POSTs return 202; responses, notifications, and permission requests arrive on connection/session SSE GETs. `Acp-Session-Id` routes session traffic, including callback responses. Cookies preserve affinity; DELETE releases the connection. This is ACP HTTP, not MCP's POST response-stream transport. Remote requests reject redirects and omit credentials/response bodies from network errors. Connection setup and POSTs have bounded deadlines. Use `timeout_sec` for an overall turn deadline. Disconnects do not automatically reconnect or replay work. ### Shared behavior All three transports use the same Go ACP SDK, protocol negotiation, session setup, prompt handling, permissions, output capture, cancellation, and provider fallback. Each execution creates a fresh session. Local processes inherit step environment, working directory, and resource limits; remote agent processes remain server-owned. Assistant text chunks become step stdout and can feed downstream output variables. Protocol JSON, thoughts, and tool events are excluded. Only `end_turn` succeeds. Failed attempts do not contribute text to successful fallback output. `with.acp_permission` defaults to `deny`. `allow` selects only an offered `allow_once` option; it never grants persistent permission. A DAG-level `harness.acp_permission` applies only to steps whose provider uses ACP; CLI steps never receive it as a flag. This policy governs ACP callbacks, not a sandbox around the agent's own tools. A denial selects an offered reject option and fails the attempt. A graceful Stop sends `session/cancel` and waits for the agent to end the turn. A forced Stop, issued once `max_clean_up_time_sec` elapses, and a timeout send `session/cancel` and then interrupt the transport. Local process trees are terminated; remote connections are closed and HTTP connections are deleted. Remote agents must honor cancellation. ACP cancellation skips fallback, including a Stop that arrives before the agent starts. Explicit Stop reports the signal exit code (143 for graceful termination, 137 for forced termination); timeouts report 124. A local adapter that exits on its own after a failed turn keeps its exit code. Session updates report progress at most once per second during long turns. Client filesystem/terminal callbacks, local ACP adapters in containers, session resume, interactive authentication, and MCP injection are not included. Remote steps ignore a DAG-level `container:`. Adapters authenticate beforehand. ### Verification Package tests cover stdio, WSS certificate verification, HTTP cookies and scoped SSE/callback routing, permission decisions, disconnects, cancellation, and cleanup. Binary conformance tests exercise all three transports and downstream output use. Interoperability acceptance also runs the real Dagu binary against Codex via stdio, WSS, and the official TypeScript SDK's Streamable HTTP server profile.