# Dagu

Dagu is a self-contained workflow orchestration engine for running DAGs defined in YAML. It runs as a single binary without requiring an external database or message broker. It stores state locally by default and supports local, queued, and distributed execution modes.

Use this compact reference with an AI agent when authoring, validating, or troubleshooting Dagu workflows, or when operating Dagu through its CLI. It summarizes repository-local workflow and CLI references.

---

# DAG Authoring

Load only the reference file that matches the task.

## Default Approach

- Prefer `type: graph` for new DAGs. It supports both sequential flow via `depends:` and parallel flow.
- Use `type: controller` only when the step order cannot be written down in advance and an LLM must choose it. It requires `llm:` and a `tasks:` list stating when the run is finished.
- Use `type: build` only for local regular-file pipelines whose unchanged transformations should be reused across runs.
- Prefer `id` on every step. Omit `name` unless the display label must differ from the step ID.
- Prefer `dagu schema ...` and `dagu validate ...` over guessing field names or shapes.
- Prefer `action: template.render` when generating text files, prompts, or artifacts instead of assembling them with shell `echo` or heredocs.
- Prefer `file.*` actions for local file operations such as stat, read, write, copy, move, delete, mkdir, and list instead of shelling out to `cp`, `mv`, `rm`, or `mkdir`.
- Prefer `git.worktree.add` and `git.worktree.remove` when steps need isolated branches inside an existing local Git repository. Add an explicit remove step when the workflow should delete the worktree.
- Prefer `stdout.artifact` / `stderr.artifact` when a command stream should become a DAG-run artifact, especially for large reports, JSON, Markdown, logs, or generated files.
- Prefer `artifact.*` actions for explicit artifact reads/writes/lists. Use `DAG_RUN_ARTIFACTS_DIR` only when a tool truly needs a filesystem path inside the step.
- Prefer string-form `output: VAR_NAME` for capturing small stdout values into flat variables.
- Prefer object-form `output:` when downstream steps need structured values via `${step_id.output.*}`.
- Prefer declared step `outputs:` with `$DAGU_OUTPUT_FILE` when a step must publish explicit values for `${steps.<step_id>.outputs.<name>}`.
- Use `action: human.task` when an operator must provide typed input before downstream steps continue. Human task form outputs use `${steps.<step_id>.outputs.<name>}` without an authored `outputs:` field.
- Prefer `stdout.outputs` or `action: outputs.write` when a DAG or remote action needs to return caller-visible values via `${step_id.outputs.*}`.
- Prefer `state.*` actions for small persistent JSON state across DAG runs, such as cursors, checkpoints, and previous-value comparisons.
- Prefer temporary files in the artifacts dir only when downstream steps need file paths; otherwise let commands write large artifact content to stdout and attach it with `stdout.artifact`.
- Prefer scoped Dagu references for named values: `${consts.NAME}`, `${params.NAME}`, and `${env.NAME}`. Avoid unscoped braced names in examples unless the example is intentionally showing shell syntax.
- Declare portable external CLI dependencies in top-level `tools` using aqua shorthand when the binary version affects reproducibility, for example `tools: ["jqlang/jq@jq-1.7.1"]`. Append `#sha256:<64 hex>` to also pin the downloaded artifact content for the run platform, for example `jqlang/jq@jq-1.7.1#sha256:<hex>`.
- For remote actions, put `tools` in the referenced action DAG file, not in `dagu-action.yaml`; caller DAG tools are not inherited across the action boundary.
- Use remote action packages (`dagu-action.yaml`) when reusable logic needs helper files, its own DAG, versioning, or an input/output schema contract.

## High-Signal Rules

- `output:` has two modes:
  - string form captures trimmed stdout into an env-scope variable such as `${env.VERSION}`
  - object form publishes structured step-scoped output for `${step_id.output.*}` access
- Value declarations in step `outputs:` publish explicit values through `${steps.<step_id>.outputs.<name>}`. Write those values to `$DAGU_OUTPUT_FILE`; Dagu captures them only after the command succeeds. Build path declarations publish the final materialization path after commit or reuse.
- `human.task` is a processless root-DAG step with an explicit `id`, a required `with.prompt`, and an optional flat scalar form. A root DAG containing one can run locally or on a distributed worker. Every declared form property is a step output, published when submitted or defaulted, and available as `${steps.<step_id>.outputs.<name>}`.
- `stdout.artifact` / `stderr.artifact` store command stdout/stderr directly as relative artifact paths, for example `stdout: {artifact: reports/report.md}`. Artifact outputs auto-enable artifacts unless `artifacts.enabled: false` is explicitly set, which is invalid.
- `${step_id.stdout}` is a log file path, not stdout content.
- Use `${context.*}` for run metadata in DAG YAML, for example `${context.dag.name}`, `${context.run.id}`, or `${context.paths.artifacts_dir}`. Unavailable context values remain unresolved text instead of becoming empty strings.
- In a build step, `${inputs.<name>}` is the final input path and `${outputs.<name>}` is a fresh attempt staging path. Write file results only to the staging path; dependencies read the committed path as `${steps.<step_id>.outputs.<name>}`.
- Do not read attempt-only `${step_id.stdout}`, `${step_id.stderr}`, or `${step_id.exit_code}` from potentially reusable producers. This also applies to `${step_id.output.<name>}` and `${step_id.outputs.<name>}`, including their whole-value `${step_id.output}` and `${step_id.outputs}` forms. Path-output steps cannot use `continue_on.mark_success`.
- Build workflows are local-only. Path declarations are supported only on host command or shell steps without containers, and stream redirects cannot target declared build inputs or outputs.
- Use `${consts.NAME}`, `${params.NAME}`, and `${env.NAME}` for Dagu-side named values. Use shell `$NAME` or `printenv NAME` only when the target shell or process should read the variable at execution time.
- `consts:` must use list form with one key per item, for example `consts: [{service: api}]`. Const values are resolved while loading the DAG and can reference inherited or earlier consts.
- `env:` should use list-of-maps when values depend on earlier env vars.
- `params:` values arrive as strings. The `params:` field supports JSON schema-like types and validation, check for schema to see how to specify types and validation rules.
- Single-line `run:` values are command-form entries. Array-form `run:` entries run one by one. Multi-line `run:` values are scripts. Dagu does not split pipes, redirects, `&&`, or `;` into separate commands; those stay with the selected shell.
- Do not assume `bash` for `run:` steps. If a script depends on a specific interpreter, add a shebang such as `#!/bin/sh` or `#!/usr/bin/env bash` only after checking that shell exists on the target host or container. Otherwise keep the script portable or set `with.shell:` explicitly.
- `parallel:` currently requires `action: dag.run` to a child DAG.
- Sub-DAGs do not inherit parent env vars; pass what you need via `params:`.
- For arbitrary text inside shell steps, prefer `printenv VAR_NAME` or `action: template.render` over Dagu interpolation such as `${env.VAR_NAME}`.
- `harness.run` supports built-in CLI provider adapters (`claude`, `codex`, `copilot`, `opencode`, `pi`) and custom top-level `harnesses:` entries. It can use top-level `container:` or step-level `container:`.
- Container runtime selection is service-level, not a DAG YAML field. Set `DAGU_CONTAINER_RUNTIME=podman` to use Podman, and set `DAGU_PODMAN_HOST` only when the Podman Docker-compatible socket is not the default.
- DAG/action outputs are collected from string-form `output: VAR_NAME`, `stdout.outputs`, and `action: outputs.write`. Object-form `output:` stays step-scoped for `${step_id.output.*}` unless the workflow explicitly republishes values through `stdout.outputs` or `outputs.write`.
- `state.get`, `state.set`, `state.delete`, `state.list`, and `state.diff` persist small JSON values across DAG runs. State scopes are `dag`, `root_dag`, `global`, and `custom`; use artifacts or external storage for large payloads.
- Git worktree actions discover the repository from the step `working_dir`. Relative worktree paths resolve from the repository root, and the actions never fetch or push.
- Remote action packages define `dagu-action.yaml` with `apiVersion: v1alpha1`, `name`, `dag`, and optional `inputs`/`outputs` JSON Schemas. `inputs` validates caller `with:` before the action DAG starts; `outputs` validates the final action output object after the action DAG returns.
- Remote action manifests do not support `tools`. Declare external CLI tools in the action DAG itself so local and distributed workers prepare the right binaries for that action run.
- In remote action examples, prefer `dag: workflow.yaml` for the action DAG filename. The `dag` field accepts any safe relative file path, but `workflow.yaml` avoids confusing the executable DAG with the `dagu-action.yaml` manifest.
- Object-form `output:` with `decode: json` or `decode: yaml` can act as lightweight runtime validation. Malformed data or an unresolved `select:` path fails the step, so normal `retry_policy` applies.
- Use DAG-level `shell` and `shell_args` only when every inherited `run:` step should use the same shell invocation. Use step-level `with.shell` and `with.shell_args` for a single step.
- Use `dagu schema dag` to check the full list of available fields and their shapes.
- Use `dagu example` to see different DAG patterns and how to express them in YAML.

## Example of Params, template step, and artifacts

```yaml
params:
  type: object
  properties:
    name:
      type: string
      maxLength: 50
    age:
      type: integer
      minimum: 0
      maximum: 120
    favorite_color:
      type: string
  required: [name, age]

steps:
  - id: render
    action: template.render
    with:
      data:
        name: ${params.name}
        age: ${params.age}
        favorite_color: ${params.favorite_color}
      template: |
        Hello, {{ .name }}!
        You are {{ .age }} years old.
        {{- if .favorite_color }}
        Your favorite color is {{ .favorite_color }}.
        {{- end }}
    stdout:
      artifact: greeting.txt
```

## Example of Large Command Output as Artifact

```yaml
steps:
  - id: report
    run: ./generate-report --format markdown
    stdout:
      artifact: reports/report.md
```

## Example of Reproducible External CLI

```yaml
tools:
  - jqlang/jq@jq-1.7.1

steps:
  - id: inspect
    run: jq --version
```

## Example of Object-Form Output

```yaml
steps:
  - id: inspect_build
    run: echo '{"version":"v1.2.3","artifact":{"url":"https://example.test/app.tgz"}}'
    output:
      # decode + select act as a lightweight contract check:
      # malformed JSON or a missing selected field fails the step.
      version:
        from: stdout
        decode: json
        select: .version
      artifact:
        from: stdout
        decode: json
        select: .artifact

  - id: publish
    depends: [inspect_build]
    output:
      versionLabel: "ver - ${inspect_build.output.version}"
      artifactUrl: "${inspect_build.output.artifact.url}"
```

## Example of Action Outputs

```yaml
steps:
  - id: classify
    run: ./classify.sh "${params.INPUT}"
    stdout:
      outputs:
        fields:
          label:
            decode: json
            select: .label
          confidence:
            decode: json
            select: .confidence

  - id: publish
    depends: [classify]
    action: outputs.write
    with:
      values:
        label: ${classify.outputs.label}
        reviewed: false
```

## Example of Human Input

```yaml
steps:
  - id: review
    action: human.task
    with:
      prompt: Choose the deployment target
      form:
        type: object
        properties:
          environment:
            type: string
            enum: [staging, production]
          note:
            type: string
            default: ""
        required: [environment]

  - id: deploy
    depends: [review]
    run: ./deploy --environment '${steps.review.outputs.environment}' --note '${steps.review.outputs.note}'
```

Complete the task from a local CLI context with `dagu human-task complete --run-id=<run-id> --step=review --input environment=production <dag-name>`. A form is optional for acknowledgement-only tasks. Human tasks cannot be used in sub-DAGs; a distributed root run is re-queued through the scheduler after completion.

## Reference Guide

Load only the file you need:

- `references/steptypes.md` when choosing an action or checking action-specific behavior such as `human.task`, `dag.run`, `parallel`, `git.worktree.*`, `jq.filter`, `file.*`, `state.*`, or `template.render`
- `references/dagu-action.md` when creating a reusable `dagu-action.yaml` package or checking action input/output schema behavior
- `references/cli.md` when choosing or using Dagu CLI commands, including workflow inspection, execution, and cleanup operations
- `references/context.md` when using `${context.*}` metadata references or declared step `outputs:`
- `references/build.md` when creating or troubleshooting a `type: build` file workflow, path references, reuse decisions, or `--no-reuse`
- `references/harnesses.md` only when the DAG invokes external CLI harnesses through `harness.run`

---

# Actions

## run: Shell Commands And Scripts

Use top-level `run:` for local shell commands and scripts.

```yaml
steps:
  - id: hello
    run: echo "hello"

  - id: multi_line
    run: |
      echo "step 1"
      echo "step 2"

  - id: ordered
    run:
      - echo "first"
      - echo "second"

  - id: custom_shell
    run: |
      set -euo pipefail
      echo "running in bash"
    with:
      shell: /bin/bash
```

Fields:

- `run` - command string or multi-line shell script
- `with.shell` - shell interpreter, for example `/bin/bash`
- `with.shell_args` - shell interpreter arguments
- `with.shell_packages` - optional packages to install before execution

Notes:

- Single-line `run:` values are command-form entries.
- Array-form `run:` entries run one by one and stop on the first failing entry.
- Multi-line `run:` values are scripts.
- Dagu sends pipes, redirects, `&&`, and `;` to the selected shell. It does not split that shell syntax into separate Dagu commands.
- DAG-level `shell` and `shell_args` provide defaults for inherited `run` steps. Use `with.shell` and `with.shell_args` when one step needs a different shell invocation.
- Dagu resolves `${...}` references before the shell runs. For large or arbitrary text, prefer `printenv VAR_NAME`, reading `${step_id.stdout}` as a file, or `action: template.render`.
- Use scoped Dagu references for named values: `${consts.NAME}`, `${params.NAME}`, and `${env.NAME}`. Use shell `$NAME` only when the target shell should read the variable at execution time.
- When large command output should become an artifact, write it to stdout/stderr and attach the stream directly instead of redirecting inside shell:

```yaml
steps:
  - id: report
    run: ./generate-report --format markdown
    stdout:
      artifact: reports/report.md
```

- Use string-form `output: VAR_NAME` only for small stdout values. Large reports, JSON dumps, Markdown summaries, and logs belong in `stdout.artifact` / `stderr.artifact`.

## docker.run / container.run

Run commands in Docker containers.

```yaml
steps:
  - id: build
    action: docker.run
    with:
      image: golang:1.23
      pull: always
      auto_remove: true
      working_dir: /app
      volumes:
        - /local/src:/app
      command: go build ./...
```

`with` fields: `image`, `container_name`, `pull`, `auto_remove`, `working_dir`, `volumes`, `network`, `platform`, `command`.

Dagu can drive Docker or Podman through a Docker-compatible API. Runtime selection is service-level, not a DAG YAML field. Set `DAGU_CONTAINER_RUNTIME=podman` for Podman. Set `DAGU_PODMAN_HOST` only when the Podman socket is not the default.

## git.worktree.add / git.worktree.remove

Create isolated working directories for branches in an existing local Git repository. The actions discover the repository from the step `working_dir`; they do not clone, fetch, or push.

This example creates a generated branch, runs tests inside its worktree, and then removes the worktree explicitly:

```yaml
working_dir: ./repo

steps:
  - id: worktree
    action: git.worktree.add

  - id: test
    depends: worktree
    working_dir: "${steps.worktree.outputs.path}"
    run: go test ./...

  - id: remove_worktree
    depends: test
    action: git.worktree.remove
    with:
      path: "${steps.worktree.outputs.path}"
```

When `branch` is omitted, Dagu generates a stable branch name for that step and DAG run. The default path is `<repository-root>.worktrees/<branch>`.

To create an explicit branch from a local commit, branch, `origin` remote-tracking branch, or tag:

```yaml
working_dir: ./repo

steps:
  - id: worktree
    action: git.worktree.add
    with:
      branch: feature/api
      create_branch: true
      base: main
      path: ../worktrees/feature-api
```

`git.worktree.add` fields:

- `branch` - local branch to check out. Omit it to let Dagu generate one.
- `path` - worktree directory. Relative paths resolve from the repository root.
- `create_branch` - allow creation of an explicitly named branch. Defaults to `false`.
- `base` - local commit, branch, remote-tracking branch, or tag used when creating the branch. Defaults to repository `HEAD`.

The add action is idempotent. It reuses a matching registered worktree without resetting its branch or discarding local changes. Worktrees remain registered until an explicit remove action or an external Git command removes them.

Use `git.worktree.remove` for explicit removal:

```yaml
working_dir: ./repo

steps:
  - id: worktree
    action: git.worktree.add

  - id: remove_worktree
    depends: worktree
    action: git.worktree.remove
    with:
      path: "${steps.worktree.outputs.path}"
      branch: "${steps.worktree.outputs.branch}"
      delete_branch: true
```

`git.worktree.remove` fields:

- `branch` and `path` - provide either selector or both. When both resolve to a worktree, they must identify the same registration.
- `force` - remove a dirty worktree. Defaults to `false`.
- `delete_branch` - delete the local branch after removing the worktree. Requires `branch`.
- `force_delete_branch` - allow deletion of an unmerged branch. Requires `delete_branch: true`.

`force` and `force_delete_branch` protect different data: `force` permits removal of local worktree changes, while `force_delete_branch` permits deletion of unmerged commits.

Both actions publish fixed outputs. Do not add `output`, `outputs`, or `stdout.outputs` to these steps. Read results through `${steps.<id>.outputs.<field>}`.

- Add outputs: `path`, `branch`, `commit`, `worktree_created`, `branch_created`.
- Remove outputs: `path`, `branch`, `worktree_removed`, `branch_deleted`.

Dagu refuses to remove the primary working tree. Worktree mutations against the same repository are serialized, but Git changes made outside Dagu are not covered by that lock.

## dag.run

Execute another DAG as a child DAG.

```yaml
steps:
  - id: child
    action: dag.run
    with:
      dag: child-workflow
      params:
        input: /data/file.csv
```

Sub-DAGs do not inherit parent env vars. Pass values explicitly via `with.params`.

## human.task

Pause a root DAG run until an operator completes a processless step. A human task does not execute a command and is distinct from an approval gate: completion always succeeds the step, with no reject or rewind operation.

```yaml
params:
  RELEASE: v1.2.3

steps:
  - id: review
    action: human.task
    with:
      prompt: Select a deployment window for ${params.RELEASE}
      form:
        type: object
        title: Deployment review
        properties:
          window:
            type: string
            enum: [morning, evening]
          ticket:
            type: string
            pattern: '^CHG-[0-9]+$'
          notify:
            type: boolean
            default: true
        required: [window, ticket]

  - id: deploy
    depends: [review]
    run: ./deploy --window '${steps.review.outputs.window}' --ticket '${steps.review.outputs.ticket}'
```

`with.prompt` is required and supports normal Dagu value references. `with.form` is optional; omit it for an acknowledgement-only task that accepts no input.

The form is a flat object JSON Schema:

- `type` must be `object`.
- Property names must start with a letter and contain only letters, digits, or `_`.
- Property types are `string`, `integer`, `number`, and `boolean`.
- Supported property constraints include `default`, `enum`, `oneOf` choices, `minimum`, `maximum`, `minLength`, `maxLength`, and `pattern`.
- `additionalProperties` defaults to `false`. Set it explicitly to `true` only when undeclared completion fields are intended.

Dagu derives outputs from form properties; do not add an `outputs:` field to the human task. Every declared property is a step output, published when submitted or defaulted, and available as `${steps.<step_id>.outputs.<name>}`.

Human tasks require an explicit `id` and cannot be used in sub-DAGs, lifecycle handlers, or `foreach.steps`. A root DAG containing human tasks can run locally or on a distributed worker selected by its DAG-level `worker_selector`. Executor, retry, repeat, timeout, container, step-level worker selector, output capture, and approval fields are not supported on the same step.

Complete a waiting task from a local CLI context:

```sh
dagu human-task complete --run-id=<run-id> --step=review --input window=morning --input ticket=CHG-123 <dag-name>
```

Use `--inputs-json` instead of repeated `--input` flags when input types must be preserved exactly.

Completing the last waiting human task resumes a local run directly. A distributed run is re-queued, so its scheduler must be running.

## Declared Value Outputs

Declare value-form `outputs:` when a step should publish named values for later steps as `${steps.<step_id>.outputs.<name>}`. Build file outputs use `path` instead; see `references/build.md`.

```yaml
steps:
  - id: build
    run: |
      printf 'image_tag=v1.2.3\n' >> "$DAGU_OUTPUT_FILE"
      {
        printf 'metadata<<JSON\n'
        printf '{"commit":"abc123"}\n'
        printf 'JSON\n'
      } >> "$DAGU_OUTPUT_FILE"
    outputs:
      - name: image_tag
      - name: metadata
        type: json

  - id: deploy
    depends: [build]
    run: ./deploy.sh '${steps.build.outputs.image_tag}'
```

Rules:

- The step must have an `id`.
- `outputs:` must be a non-empty sequence.
- Each output requires `name`.
- `type` can be `string` or `json`. The default is `string`.
- The step writes output records to `$DAGU_OUTPUT_FILE`.
- Output records use `name=value` or heredoc form: `name<<DELIMITER`, value lines, matching `DELIMITER`.
- The output file must be valid UTF-8.
- Every declared output must be written exactly once.
- Undeclared, duplicate, missing, or invalid JSON outputs fail the step.
- Dagu captures declared outputs only after the command succeeds.

## outputs.write

Publish DAG or remote action outputs assembled from literals, parameters, or prior step values.

```yaml
steps:
  - id: send
    run: ./scripts/notify.sh "${params.text}"
    output:
      response:
        from: stdout
        decode: json

  - id: publish
    depends: [send]
    action: outputs.write
    with:
      values:
        messageId: ${send.output.response.id}
        status: sent
```

Published values are available as `${publish.outputs.messageId}` in the same DAG. When the step runs inside a remote action DAG, the parent action caller reads the final action outputs as `${action_step.outputs.messageId}`.

Notes:

- `values` must be a non-empty object.
- Keep values small and JSON-compatible; use artifacts for files, reports, logs, screenshots, or large JSON payloads.
- If the remote action manifest declares an `outputs` schema, Dagu validates the final collected action output object after the action DAG returns. `outputs.write` itself does not validate the manifest.

## state.get / state.set / state.delete / state.list / state.diff

Read and write persistent JSON state that survives across DAG runs. Use state actions for cursors, checkpoints, and comparing the current result with the previous run. Use artifacts or external storage for large files.

```yaml
steps:
  - id: load_cursor
    action: state.get
    with:
      key: cursors/feed
      default: null

  - id: save_cursor
    action: state.set
    with:
      key: cursors/feed
      value: ${fetch.output.nextCursor}

  - id: detect_change
    action: state.diff
    with:
      key: snapshots/feed
      value: ${fetch.output.items}
      update: true
```

Scope fields:

- `scope` - state scope: `dag` (default), `root_dag`, `global`, or `custom`
- `namespace` - namespace override. For `custom` scope, this is required.

Default namespaces:

- `dag` - current DAG name
- `root_dag` - root DAG name for nested DAG runs
- `global` - `_`
- `custom` - no default; set `namespace`

Operation fields:

- `state.get`: `key`, optional `default`, `required`
- `state.set`: `key`, `value`, optional `expected_version`, `create_only`
- `state.delete`: `key`
- `state.list`: optional `prefix`, `limit`, `include_values`
- `state.diff`: `key`, `value`, optional `expected_version`, `update`

All state actions write JSON to stdout. Common output fields include `operation`, `scope`, `namespace`, and key or prefix information.

- `state.get` returns `found`, and when found, `value`, `version`, and `hash`. If not found and `default` is set, `value` contains the default.
- `state.set` returns `version`, `hash`, and `created`.
- `state.delete` returns `deleted`.
- `state.list` returns `entries`; entry values are omitted unless `include_values` is true.
- `state.diff` returns `changed`, `foundPrevious`, `current`, optional `previous`, and `version` / `hash` when the stored value was written or already exists.

Values must be JSON-serializable. Dagu normalizes state values before storing them and enforces the state payload size limit after normalization.

## parallel

`parallel:` currently works only with `action: dag.run`.

```yaml
steps:
  - id: fan_out
    action: dag.run
    with:
      dag: process-item
    parallel:
      items:
        - item1
        - item2
        - item3
      max_concurrent: 5

  - id: fan_out_dynamic
    action: dag.run
    with:
      dag: process-item
    parallel: ${params.ITEMS}
```

Each child invocation receives the current item as `ITEM`.

## ssh.run / sftp.upload / sftp.download

Remote command execution and file transfer over SSH.

```yaml
steps:
  - id: remote
    action: ssh.run
    with:
      user: deploy
      host: server.example.com
      key: ~/.ssh/id_rsa
      timeout: 60s
      command: systemctl restart app

  - id: upload
    action: sftp.upload
    with:
      user: deploy
      host: server.example.com
      key: ~/.ssh/id_rsa
      source: /local/file.tar.gz
      destination: /remote/file.tar.gz
```

Shared SSH fields: `user`, `host`, `port`, `key`, `password`, `timeout`, `strict_host_key`, `known_host_file`, `shell`, `shell_args`, `bastion`.

## http.request

HTTP requests.

```yaml
steps:
  - id: api_call
    action: http.request
    with:
      method: POST
      url: https://api.example.com/data
      headers:
        Authorization: "Bearer ${env.TOKEN}"
        Content-Type: application/json
      body: '{"key": "value"}'
      json: true
      timeout: 30
```

`with` fields: `method`, `url`, `timeout`, `headers`, `query`, `body`, `silent`, `debug`, `json`, `skip_tls_verify`.

## jq.filter

JSON processing.

```yaml
steps:
  - id: transform
    action: jq.filter
    with:
      filter: ".items[] | {name: .name, count: .quantity}"
      data:
        items:
          - name: a
            quantity: 1

  - id: transform_file
    action: jq.filter
    with:
      filter: .name
      input: ${fetch_json.stdout}
```

Use `with.data` for inline JSON or `with.input` for a JSON file path. Do not set both.

## template.render

Render text using Go `text/template`.

```yaml
steps:
  - id: render
    action: template.render
    with:
      data:
        name: Alice
      template: |
        Hello, {{ .name }}!
    output: RESULT
```

Set exactly one of `with.template` or `with.template_ref`. `with.template` is literal template text. `with.template_ref` must be one complete canonical Dagu reference such as `${env.TEMPLATE}` or `${steps.fetch.outputs.template}`; it resolves once to a non-empty string, and references inside the resulting template remain literal. The selected text is rendered as a template, not executed as shell. `with.output` writes rendered content to a file; top-level `output:` captures or publishes step output.

## file.stat / file.read / file.write / file.copy / file.move / file.delete / file.mkdir / file.list

Local filesystem operations.

```yaml
steps:
  - id: ensure_output_dir
    action: file.mkdir
    with:
      path: ${context.paths.artifacts_dir}/reports

  - id: write_report
    action: file.write
    with:
      path: ${context.paths.artifacts_dir}/reports/summary.txt
      content: "status=ok\n"
      overwrite: true

  - id: copy_report
    action: file.copy
    with:
      source: ${context.paths.artifacts_dir}/reports/summary.txt
      destination: ${context.paths.artifacts_dir}/reports/latest.txt
      overwrite: true

  - id: list_reports
    action: file.list
    with:
      path: ${context.paths.artifacts_dir}/reports
      pattern: "*.txt"
```

Use `path` for `file.stat`, `file.read`, `file.write`, `file.delete`, `file.mkdir`, and `file.list`. Use `source` and `destination` for `file.copy` and `file.move`. `file.write` also requires `content`.

`with` fields: `path`, `source`, `destination`, `content`, `mode`, `format`, `pattern`, `overwrite`, `create_dirs`, `atomic`, `recursive`, `missing_ok`, `dry_run`, `include_dirs`, `follow_symlinks`, `max_bytes`.

Safety defaults:

- `overwrite` defaults to false for write, copy, and move.
- `atomic` defaults to true for file writes.
- `recursive` is required for directory copy and directory delete.
- `file.delete` refuses to delete the filesystem root.
- Copy and move reject the same source and destination, and directory copy rejects destinations inside the source tree.

## postgres.query / sqlite.query / postgres.import / sqlite.import

SQL database queries and imports.

```yaml
steps:
  - id: query
    action: postgres.query
    with:
      dsn: "postgres://user:pass@localhost:5432/db"
      query: "SELECT * FROM users WHERE active = true"
      output_format: json
      timeout: 120
      transaction: true
```

`with` fields include `dsn`, `query`, `params`, `timeout`, `transaction`, `isolation_level`, `output_format`, `headers`, `null_string`, `max_rows`, `streaming`, `output_file`, and `import`.

## redis.<operation>

Redis operations use the operation in the action name.

```yaml
steps:
  - id: cache_set
    action: redis.set
    with:
      url: "redis://localhost:6379"
      key: mykey
      value: myvalue
      ttl: 3600
```

Connection fields: `url`, `host`, `port`, `password`, `username`, `db`, TLS fields, `mode`, `timeout`, `max_retries`.

## s3.upload / s3.download / s3.list / s3.delete

S3 object operations.

```yaml
steps:
  - id: upload
    action: s3.upload
    with:
      region: us-east-1
      bucket: my-bucket
      key: data/output.csv
      source: /local/output.csv
```

Connection fields: `region`, `endpoint`, `access_key_id`, `secret_access_key`, `session_token`, `profile`, `force_path_style`.

## mail.send

Send email.

```yaml
steps:
  - id: notify
    action: mail.send
    with:
      from: noreply@example.com
      to: team@example.com
      subject: "Build Complete"
      message: "The build finished successfully."
```

SMTP server settings come from global configuration.

## archive.create / archive.extract / archive.list

Archive operations.

```yaml
steps:
  - id: compress
    action: archive.create
    with:
      source: /data/output
      destination: /data/output.tar.gz
      format: tar.gz
      exclude:
        - "*.tmp"
```

`with` fields: `source`, `destination`, `format`, `compression_level`, `password`, `overwrite`, `strip_components`, `include`, `exclude`.

## harness.run

Invoke external coding-agent CLIs through built-in provider adapters or custom harness definitions.

```yaml
harnesses:
  gemini:
    binary: gemini
    prefix_args: ["run"]
    prompt_mode: flag
    prompt_flag: --prompt

harness:
  provider: gemini
  model: gemini-2.5-pro
  fallback:
    - provider: claude
      model: sonnet

steps:
  - id: generate_tests
    action: harness.run
    with:
      prompt: "Write unit tests for the auth module"
      yolo: true
    output: RESULT
```

`with.prompt` is required and is passed to the selected provider according to its built-in adapter or custom harness definition. `with.provider` can be a built-in provider adapter (`claude`, `codex`, `copilot`, `opencode`, `pi`) or a top-level `harnesses:` entry. For host subprocess runs, `with.stdin` is piped to stdin as supplementary context.

Harness behavior:

- Built-in provider adapters and custom providers pass non-reserved `with` keys as CLI flags. Built-in adapters normalize `snake_case` keys to kebab-case flags.
- `fallback` is an ordered list of provider configs. Nested fallback is not supported.
- Provider value references must resolve to a concrete provider string before execution. Unresolved `${...}` provider values fail at runtime.
- A harness step is named with `action: harness.run`. A top-level `harness:` config supplies defaults to those steps and does not set the type of any other step, so a step written with `run:`, `exec:`, or `script:` under one stays a local command.

Container support:

- Use root-level `container:` to run compatible harness steps inside the shared DAG-level container.
- Use step-level `container:` when only that step needs a container, or when it needs a different container from the root-level container.
- Step-level `container:` takes precedence for that step.
- The selected provider binary must exist inside the container that runs the step.
- `with.stdin` and custom `prompt_mode: stdin` are rejected for containerized harness steps.
- Do not set `container.name` for step-level image-mode harness steps. Use `container.exec` when the step must run inside an existing container.
- Docker or Podman is selected by the Dagu service process, not by a DAG YAML field.

## router.route

Conditional routing based on expression value. Routes reference existing step IDs.

```yaml
steps:
  - id: check_status
    run: "curl -s -o /dev/null -w '%{http_code}' https://example.com"
    output: STATUS

  - id: route
    action: router.route
    with:
      value: ${env.STATUS}
      routes:
        "200":
          - handle_ok
        "re:5\\d{2}":
          - handle_error
          - send_alert
    depends: [check_status]

  - id: handle_ok
    run: echo "success"

  - id: handle_error
    run: echo "server error occurred"

  - id: send_alert
    run: echo "alerting on-call"
```

Routes are evaluated in priority order: exact matches first, then regex, then catch-all.

---

# Remote Action Packages

Use this reference when creating a reusable package-style action with `dagu-action.yaml`.

Remote actions are different from DAG-local `actions:` templates:

- DAG-local `actions:` are inline wrappers around built-in actions.
- Remote actions are directories or Git repositories that contain a manifest, a DAG entrypoint, and any helper files the action needs.
- Callers use them with `action: owner/repo@version`, `action: name@version`, or `action: source:target@version`.

## Package Layout

```text
dagu-action-notify/
├── dagu-action.yaml
├── workflow.yaml
└── scripts/
    └── notify.sh
```

This reference uses `workflow.yaml` as the recommended entrypoint DAG filename to keep it visually distinct from the `dagu-action.yaml` manifest. The `dag` field can point to any safe relative file path inside the package.

`dagu-action.yaml` supports exactly these fields:

- `apiVersion` - required, currently `v1alpha1`
- `name` - required action name
- `dag` - required relative path to the action DAG file
- `inputs` - optional JSON Schema object for the caller's `with:`
- `outputs` - optional JSON Schema object for the action output object

Unknown manifest keys are rejected. The `dag` path must resolve to a file inside the package.

## Manifest Example

```yaml
apiVersion: v1alpha1
name: notify
dag: workflow.yaml
inputs:
  type: object
  additionalProperties: false
  required: [text]
  properties:
    text:
      type: string
outputs:
  type: object
  additionalProperties: false
  required: [messageId]
  properties:
    messageId:
      type: string
    status:
      type: string
```

`inputs` validates the caller's `with:` object before the action DAG starts. JSON Schema `default` values are validated as schema defaults, but they are not applied to the caller's `with:` object before parameters are passed.

## Action DAG

The action DAG is a normal Dagu workflow. Do not set `working_dir` in the action DAG or local sub-DAGs inside the package; Dagu runs them in the materialized action workspace so relative package files are available.

```yaml
tools:
  - jqlang/jq@jq-1.7.1

params:
  - text
steps:
  - id: send
    run: ./scripts/notify.sh "${params.text}"
    stdout:
      outputs:
        fields:
          messageId:
            decode: json
            select: .id
          status:
            decode: json
            select: .status
```

Scalar `with:` fields are passed as runtime parameters and can be read as `${params.text}`. For structured input, pass an explicit JSON string and decode it in the action DAG; do not assume nested YAML/JSON input objects arrive as structured params.

## Tools

If the action DAG invokes portable external CLIs, declare them with top-level `tools` in the action DAG file. Do not put `tools` in `dagu-action.yaml`; unknown manifest keys are rejected.

Caller DAG tools are not inherited by remote actions. The action DAG is a separate DAG run, and the worker running it prepares that DAG's tools in the worker-local tools cache. Built-in-only actions do not need `tools`, but reusable action packages that call binaries such as `jq`, `yq`, or release helpers should pin those dependencies inside the action DAG.

## Returning Outputs

Use `stdout.outputs` when a command emits the action result on stdout:

```yaml
steps:
  - id: classify
    run: ./classify.sh "${params.text}"
    stdout:
      outputs:
        fields:
          label:
            decode: json
            select: .label
          confidence:
            decode: json
            select: .confidence
```

Use `outputs.write` when the result is assembled from parameters, previous step output, or literals:

```yaml
steps:
  - id: send
    run: ./scripts/notify.sh "${params.text}"
    output:
      response:
        from: stdout
        decode: json

  - id: publish
    depends: [send]
    action: outputs.write
    with:
      values:
        messageId: ${send.output.response.id}
        status: sent
```

Do not use object-form `output:` to return data to the parent DAG. Object-form `output:` is step-scoped inside the action DAG and is read as `${step_id.output.*}`. To cross the action boundary, republish values with `stdout.outputs` or `outputs.write`.

If the manifest declares `outputs`, Dagu validates the final collected action output object after the action DAG returns a run result. Validation failure fails the parent action step. The action executor also writes compact output JSON to the action step stdout for compatibility, but callers should read structured values through `${step.outputs.<field>}`.

Compatibility note: if an action DAG publishes no typed outputs, legacy string-form run outputs from `output: NAME` can be carried as action outputs. New action packages should prefer `stdout.outputs` or `outputs.write` because those define the action boundary explicitly.

## Caller Example

```yaml
params:
  - BUILD_ID: ""

steps:
  - id: notify
    action: acme/dagu-action-notify@v1.2.0
    with:
      text: "Build ${params.BUILD_ID} finished"

  - id: audit
    depends: [notify]
    run: echo "Message ID: ${notify.outputs.messageId}"
```

## References And Workers

Reference formats:

- `name@version` - official Dagu action, resolved as `dagucloud/name`
- `owner/repo@version` - GitHub repository
- `source:target@version` - explicit local path, `file://` path, or Git source

Use immutable tags or commit SHAs for production. Local `source:` paths are useful for development only when the worker executing the action can read the same path. For distributed or heterogeneous workers, prefer GitHub or explicit Git `source:` refs. After resolution, Dagu packages the action workspace and can send that bundle to the worker running the child action DAG.

---

# Dagu CLI Reference

Global flags on all commands: `--config/-c`, `--dagu-home`, `--quiet/-q`, `--cpu-profile`

Advanced and deprecated flags below remain implemented in `internal/cmd/start.go`, `internal/cmd/enqueue.go`, and `internal/cmd/exec.go`, so this reference keeps them documented even when they are mainly used by automation or backward-compatibility paths.

## Core Commands

### dagu start

Execute a DAG.

```sh
dagu start [flags] <dag> [-- params...]
```

Flags:

- `--params/-p` — Parameters (key=value or positional)
- `--name/-N` — Override DAG name
- `--run-id/-r` — Custom run ID
- `--from-run-id` — Historic dag-run ID to use as the template for a new run
- `--labels` — Additional labels (comma-separated key=value or key-only)
- `--tags` — Deprecated alias for `--labels`
- `--default-working-dir` — Default working directory for DAGs without explicit workingDir
- `--no-reuse` — Recompute reusable build steps while preserving staged, atomic publication
- `--worker-id` — Worker ID executing this DAG run; auto-set in distributed mode and defaults to `local`
- `--trigger-type` — Trigger source (`scheduler`, `manual`, `webhook`, `subdag`, `retry`, `catchup`); defaults to `manual`

### dagu enqueue

Enqueue a DAG run for later execution.

```sh
dagu enqueue [flags] <dag> [-- params...]
```

Flags:

- `--params/-p` — Parameters (key=value or positional)
- `--name/-N` — Override DAG name
- `--run-id/-r` — Custom run ID
- `--queue/-u` — Override the DAG-level queue definition
- `--labels` — Additional labels (comma-separated key=value or key-only)
- `--tags` — Deprecated alias for `--labels`
- `--default-working-dir` — Default working directory for DAGs without explicit workingDir
- `--no-reuse` — Recompute reusable build steps when the queued run starts
- `--trigger-type` — Trigger source (`scheduler`, `manual`, `webhook`, `subdag`, `retry`, `catchup`); defaults to `manual`

### dagu exec

Execute a one-off command as a DAG run without a DAG YAML file.

```sh
dagu exec [flags] -- <command> [args...]
```

Flags:

- `--run-id/-r` — Custom run ID
- `--name/-N` — Override DAG name
- `--workdir` — Working directory for the command (defaults to the current directory)
- `--shell` — Override shell binary for the command
- `--base` — Path to a base DAG YAML whose defaults are applied before inline overrides
- `--env/-E` — Environment variable (`KEY=VALUE`) to include in the run; repeatable
- `--dotenv` — Path to a dotenv file to load before execution; repeatable
- `--worker-label` — Worker label selector (`key=value`) for distributed execution; repeatable

### dagu dequeue

Dequeue a DAG run from a queue (marks it as aborted): `dagu dequeue <queue-name> [--dag-run/-d <dag:run-id>]`

### dagu stop

Stop an active DAG run: `dagu stop <dag-name> [--run-id/-r <id>]`

### dagu restart

Stop and restart a DAG run: `dagu restart <dag-name> [--run-id/-r <id>]`

### dagu retry

Retry a previous DAG run using the same run ID.

```sh
dagu retry <dag> --run-id/-r <id> [--step <name>] [--worker-id <id>]
```

### dagu human-task complete

Complete a waiting human task in a root DAG run. The run may be local or distributed, but the command operates on the local Dagu data store.

```sh
dagu human-task complete [flags] <root-dag-name>
```

Flags:

- `--run-id/-r` — Root DAG-run ID containing the human task; required
- `--step` — Human task step ID; required and matched against `id`, not the display name
- `--input` — Form input in `key=value` form; repeatable and coerced using the form schema
- `--inputs-json` — Typed form input as one JSON object

`--input` and `--inputs-json` are mutually exclusive. Omit both for an acknowledgement-only task. Completing one of several waiting human tasks leaves the DAG run waiting; completing the last one starts the run resume automatically. Human tasks cannot be used in sub-DAGs. A distributed run is re-queued, so its scheduler must be running. The command only supports the local context.

```sh
dagu human-task complete --run-id=run-1 --step=review --input environment=production deploy
dagu human-task complete --run-id=run-1 --step=review --inputs-json='{"environment":"production","notify":true}' deploy
```

### dagu dry

Dry-run a DAG without executing commands: `dagu dry [--params/-p] [--name/-N] [--no-reuse] <dag> [-- params...]`

For a build DAG, `--no-reuse` previews the decisions with manifest reuse disabled. Dry-run still creates no locks, staging files, manifests, or run history.

### dagu validate

Validate DAG YAML without executing: `dagu validate <dag>`

### dagu status

Show DAG run status: `dagu status <dag-name> [--run-id/-r <id>] [--sub-run-id/-s <id>]`

### dagu history

Show DAG run history.

```sh
dagu history [dag-name]
```

Flags:

- `--from` — Start date/time in UTC (format: `2006-01-02` or `2006-01-02T15:04:05Z`)
- `--to` — End date/time in UTC (same formats as `--from`)
- `--last` — Relative time period (e.g. `7d`, `24h`, `1w`). Cannot combine with `--from`/`--to`
- `--status` — Filter by status: `running`, `succeeded`, `failed`, `aborted`, `queued`, `waiting`, `rejected`, `not_started`, `partially_succeeded`
- `--run-id` — Filter by run ID (partial match supported)
- `--labels` — Filter by labels (comma-separated key=value or key-only, AND logic)
- `--tags` — Deprecated alias for `--labels`
- `--format/-f` — Output format: `table` (default), `json`, `csv`
- `--limit/-l` — Max results (default 100, max 1000)

Default: shows runs from the last 30 days, newest first.

### dagu ls

List DAG definitions.

This command is local-only. If a remote CLI context is selected, use `--context local`.

```sh
dagu ls [flags] [pattern]
```

Flags:

- `--next/-n` — Show next scheduled run time
- `--last/-l` — Show last run status and time
- `--history/-H` — Show a compact recent-history summary
- `--sort-last/-t` — Sort by last run time, newest first
- `--reverse/-r` — Reverse sort order

### dagu rm

Remove DAG run history and/or the DAG YAML definition. At least one of `--history` or `--definition` is required. Active runs are never deleted from history; definition deletion is refused while the DAG has alive processes. With `--definition`, identify the DAG by filename, stem, or configured path.

```sh
dagu rm [--history|-H] [--definition|-d] [-t <duration>] [-f] [--dry-run] <dag>
```

Flags:

- `--history/-H` — Delete run history
- `--definition/-d` — Delete the DAG YAML definition
- `--older-than/-t` — With `--history`: delete runs older than a duration (e.g. `10d`, `24h`, `1w`). Omitted = delete all history
- `--force/-f` — Skip confirmation prompt
- `--dry-run` — Preview deletions without removing history or the definition

### dagu ps

List running DAG processes.

```sh
dagu ps [-d <dag-name>] [-r <run-id>]
```

`-r`/`--run-id` accepts a partial run ID and matches accordingly.

### dagu cleanup

Remove old DAG run history. Active runs are never deleted.

Deprecated: prefer `dagu rm --history`.

```sh
dagu cleanup <dag-name> [--retention-days <n>] [--dry-run] [--yes/-y]
```

### dagu schema

Show JSON schema documentation. Use a dot-separated path to drill into nested sections.

```sh
dagu schema <dag|config> [path]
```

Examples:

- `dagu schema dag` — All DAG root-level fields
- `dagu schema dag steps` — Step definition structure
- `dagu schema dag steps.container` — Container configuration
- `dagu schema dag steps.retry_policy` — Retry policy fields
- `dagu schema dag steps.harness` — Harness step configuration
- `dagu schema dag handler_on` — Lifecycle event hooks
- `dagu schema config` — All config root-level fields
- `dagu schema config auth` — Authentication configuration

### dagu config

Show resolved configuration paths.

```sh
dagu config
```

## Server & Scheduling

### dagu start-all

Start server + scheduler + optionally coordinator in one process. Coordinator enabled by default (disable with `DAGU_COORDINATOR_ENABLED=false`).

```sh
dagu start-all [--host/-s <host>] [--port/-p <port>] [--dags/-d <dir>]
```

Also accepts `--coordinator.*` and `--peer.*` flags for distributed setup.

### dagu server

Start web UI + REST API.

```sh
dagu server [--host/-s <host>] [--port/-p <port>] [--dags/-d <dir>] [--tunnel/-t]
```

### dagu scheduler

Start cron scheduler. Monitors DAGs and triggers runs on schedule; also processes queued runs.

```sh
dagu scheduler [--dags/-d <dir>]
```

## Distributed Execution

### dagu coordinator

Start gRPC coordinator: `dagu coordinator [--coordinator.host/-H <host>] [--coordinator.port/-P <port>] [--peer.*]`

### dagu worker

Start distributed worker: `dagu worker [--worker.id/-w <id>] [--worker.max-active-runs/-m <n>] [--worker.labels/-l <k=v,...>] [--worker.coordinators <addrs>] [--peer.*]`

## Git Sync

`dagu sync <subcommand>` — Git sync operations for DAG definitions.

| Subcommand | Description |
| ---------- | ----------- |
| `sync status` | Show sync status (repository, branch, per-DAG status) |
| `sync pull` | Pull changes from remote |
| `sync publish [dag] [--message/-m] [--all] [--force/-f]` | Publish local changes to remote |
| `sync discard <dag> [--yes/-y]` | Discard local changes, restore remote version |
| `sync forget <id>... [--yes/-y]` | Remove state entries for missing/untracked items |
| `sync cleanup [--dry-run] [--yes/-y]` | Remove all missing entries from sync state |
| `sync delete <id> [--message/-m] [--force] [--all-missing] [--dry-run] [--yes/-y]` | Delete from remote, local, and sync state |
| `sync mv <old> <new> [--message/-m] [--force] [--dry-run] [--yes/-y]` | Rename across local, remote, and sync state |

## Other Commands

- `dagu example [id]` — Show built-in example DAGs
- `dagu version` — Show version
- `dagu upgrade [--check] [--version/-v <ver>] [--dry-run] [--yes/-y]` — Self-update binary
- `dagu license <activate|deactivate|check>` — Manage license

---

# Context References, Scoped Values, And Step Outputs

Use `${context.*}` when DAG YAML needs metadata about the current DAG run. These values are resolved by Dagu before the step executor runs.

If a supported context value is not available in the current scope, Dagu keeps the original reference text. It does not replace the reference with an empty string.

## Scoped Value References

Use scoped references when Dagu should resolve a named value before execution.

| Reference | Meaning |
| --- | --- |
| `${consts.NAME}` | Top-level `consts:` value |
| `${params.NAME}` | Runtime parameter value |
| `${env.NAME}` | Value in Dagu's runtime environment scope for the current step |
| `${steps.<step_id>.outputs.<name>}` | Declared value output, human-task form value, or committed build path output |

Use shell `$NAME` or `printenv NAME` only when the target shell or process should read the variable at execution time.

`${steps.<step_id>.outputs.<name>}` is only for declared step outputs. Other step properties keep the step-ID form that the resolver supports, such as `${step_id.stdout}`, `${step_id.output.name}`, and `${step_id.outputs.name}`.

## Top-Level `consts:`

Use `consts:` for static DAG values that Dagu should resolve before step execution. Constants are not process environment variables. Use `${consts.NAME}` when Dagu should substitute the value into a DAG field. Copy a constant into `env:` only when a child process needs an environment variable.

```yaml
consts:
  - service: api
  - region: us-east-1
  - endpoint: https://${consts.service}.${consts.region}.example

env:
  - API_ENDPOINT=${consts.endpoint}

steps:
  - id: healthcheck
    run: curl -fsS '${consts.endpoint}/health'
```

Rules:

- `consts:` must use list form. Mapping form is invalid.
- Each list item must be a single-key mapping, such as `- service: api`.
- Names must start with a letter and then contain only letters, digits, or `_`.
- Values must be literal strings, finite numbers, or booleans. `null`, arrays, and objects are invalid.
- A const can reference inherited consts or earlier consts in the same list with `${consts.NAME}`.
- A const cannot read runtime values while it is being defined. References such as `${params.NAME}`, `${env.NAME}`, `${steps.step_id.outputs.name}`, self-references, later const references, and unknown const references remain unresolved text inside the const value.
- Consts from a base config are inherited. A DAG-local const with the same name overrides the inherited value.

Use `consts:` for fixed workflow configuration. Use `params:` for per-run user input. Use `env:` for values that should be available to the running process environment.

## Supported Context References

Run and step metadata:

| Reference | Meaning |
| --- | --- |
| `${context.dag.name}` | Current DAG name |
| `${context.run.id}` | Current DAG run ID |
| `${context.run.status}` | Current run status when status is available, such as lifecycle handler scopes |
| `${context.run.scheduled_at}` | Scheduled time for a scheduled run |
| `${context.run.root_name}` | Root DAG name when this run is part of a nested run |
| `${context.run.root_id}` | Root run ID when this run is part of a nested run |
| `${context.attempt.id}` | Current attempt ID |
| `${context.attempt.started_at}` | Current attempt start time |
| `${context.step.id}` | Current step ID |
| `${context.step.name}` | Current step name |
| `${context.trigger.type}` | Run trigger type when trigger metadata is available |
| `${context.trigger.actor}` | Trigger actor when trigger metadata includes one |

Path metadata:

| Reference | Meaning |
| --- | --- |
| `${context.paths.log_file}` | DAG run log file path |
| `${context.paths.work_dir}` | Per-run working directory when one exists |
| `${context.paths.artifacts_dir}` | Artifact directory when artifacts are enabled |
| `${context.paths.step_stdout_file}` | Current step stdout log file |
| `${context.paths.step_stderr_file}` | Current step stderr log file |
| `${context.paths.step_output_file}` | Current step output file path |

Other scoped metadata:

| Reference | Meaning |
| --- | --- |
| `${context.profile.name}` | Runtime profile name when profile metadata exists |
| `${context.profile.resolved_at}` | Runtime profile resolution time when profile metadata exists |
| `${context.pushback.iteration}` | Push-back iteration when the step is running in push-back flow |
| `${context.pushback.previous_stdout_file}` | Previous stdout file when push-back provides one |

Example:

```yaml
steps:
  - id: summarize
    run: ./summarize.sh '${context.dag.name}' '${context.run.id}'
```

Use single quotes around context references when the shell should receive the resolved value as one argument.

## Declared Value Outputs

A step can declare value outputs and write them during execution. Dagu reads the output file after the command succeeds and publishes the values as `${steps.<step_id>.outputs.<name>}`.

Human task form outputs use the same reference namespace without an authored `outputs:` field. Every declared property is published when submitted or defaulted.

```yaml
steps:
  - id: build
    run: |
      printf 'image_tag=v1.2.3\n' >> "$DAGU_OUTPUT_FILE"
      {
        printf 'metadata<<JSON\n'
        printf '{"commit":"abc123"}\n'
        printf 'JSON\n'
      } >> "$DAGU_OUTPUT_FILE"
    outputs:
      - name: image_tag
      - name: metadata
        type: json

  - id: deploy
    depends: [build]
    run: ./deploy.sh '${steps.build.outputs.image_tag}'
```

Rules:

- A step that declares `outputs:` must have an `id`.
- `outputs:` must be a non-empty sequence.
- Each output must declare `name`.
- `type` is optional. Supported values are `string` and `json`.
- The output file must contain valid UTF-8.
- Use `name=value` for a single-line value.
- Use `name<<DELIMITER`, value lines, and a matching `DELIMITER` for a multi-line value.
- Every declared output must be written exactly once.
- Writing an undeclared output fails the step.
- Writing a duplicate output fails the step.
- For `type: json`, the emitted value must be valid JSON.
- Dagu captures declared outputs only after the command itself succeeds.

Declared step outputs are different from DAG or remote action outputs. To publish caller-visible DAG/action outputs, use `stdout.outputs` or `action: outputs.write`.

In a `type: build` workflow, an `outputs:` entry may declare `path` instead of a value `type`. The owning step writes to `${outputs.<name>}`, and `${steps.<step_id>.outputs.<name>}` publishes the final path only after commit or reuse. See `references/build.md`.

---

# Build Workflows

Use `type: build` for local workflows that transform stable regular files and should reuse unchanged work across DAG runs. Build execution is materialization reuse, not a cache that restores a missing output: a missing or modified final output causes recomputation.

## Minimal Pipeline

```yaml
type: build
working_dir: /srv/build

steps:
  - id: compile
    inputs:
      - name: source
        path: source.c
    outputs:
      - name: binary
        path: app.bin
    run: compiler "${inputs.source}" -o "${outputs.binary}"

  - id: package
    inputs:
      - name: binary
        path: app.bin
    outputs:
      - name: archive
        path: app.tar
    run: packager "${inputs.binary}" "${outputs.archive}"
```

The `package` step does not need `depends: compile`. Its canonical input path matches the `compile` output path, so Dagu infers the dependency. Keep explicit dependencies for ordering that is not represented by a file.

## Declarations and References

- Set a stable `working_dir`, or load the DAG from a file whose directory can anchor relative paths.
- Each input requires `name` and `path`.
- A path-backed output requires `name` and `path`. It cannot also declare a value `type`.
- A step may declare at most one path-backed output.
- Names must start with a letter and contain only letters, digits, or `_`.
- Path declarations are supported only on host command and shell steps without DAG-level or step-level containers.
- Output parent directories must already exist.

| Reference | Meaning |
| --- | --- |
| `${inputs.<name>}` | Absolute final path of the owning step's declared input |
| `${outputs.<name>}` | Fresh, initially absent staging path for the current executor attempt |
| `${steps.<step_id>.outputs.<name>}` | Absolute final output path after commit or reuse |

`${inputs.*}` and `${outputs.*}` are step-scoped. Use `${outputs.<name>}` only in command text, scripts, arguments, and executor environment where an attempt staging path exists. Path declarations may use stable parameters and environment values, but must resolve before execution; do not derive them from step outputs or command substitution.

## Dependency and Path Rules

- Matching canonical producer outputs and consumer inputs create inferred dependencies.
- Each canonical output path must have exactly one producer.
- A step cannot declare the same canonical path as both input and output.
- Explicit and inferred dependencies must remain acyclic.
- Inputs and outputs must be regular, non-symlink files. Inputs must exist when the step is evaluated.

## Reuse and Publication Safety

A reusable step has exactly one path-backed output and a stable host command or shell recipe. Steps with dynamic or scalar output surfaces, secrets, repeat, human tasks, approvals, parallel or foreach bodies, child DAGs, or containers execute normally instead of being reused.

Dagu hashes the resolved recipe, declared input contents, and current output. A matching materialization produces a `reuse` decision and a succeeded node. Changed inputs, recipe, environment, tools, working directory, missing output, or modified output produce an `execute` decision.

The default per-run scratch directory is normalized in the recipe digest, so a different run ID does not prevent reuse by itself. An authored working directory remains part of the recipe and changing it requires execution.

Always write the result to `${outputs.<name>}`. Dagu verifies the staged file and input snapshots before atomically replacing the final output. Writing directly to the final output bypasses that contract. `stdout`, `stderr`, `stdout.artifact`, and `stderr.artifact` destinations cannot target any declared build input or output.

Potentially reusable producers expose downstream data only through `${steps.<step_id>.outputs.<name>}`. Do not read `${step_id.stdout}`, `${step_id.stderr}`, or `${step_id.exit_code}` from them. This also applies to `${step_id.output.<name>}` and `${step_id.outputs.<name>}`, including their whole-value `${step_id.output}` and `${step_id.outputs}` forms, because reuse does not recreate attempt results. Path-output steps also cannot use `continue_on.mark_success`; failed attempts must not expose an old or missing final file as a successful publication.

Each retry receives a new staging path. A failed, timed-out, or aborted attempt removes its staging file and leaves the previous final output and manifest unchanged.
Crash recovery refuses to overwrite a final output that was externally replaced with content matching neither the previous nor the proposed materialization.

## Execution Controls

Build workflows are local-only. Distributed execution is rejected because workers do not share the required materialization fencing.

Preview decisions without executing:

```sh
dagu dry workflow.yaml
```

Disable reuse for one run without disabling staging or commit safety:

```sh
dagu start --no-reuse workflow.yaml
dagu enqueue --no-reuse workflow.yaml
dagu dry --no-reuse workflow.yaml
```

In the Web UI, enable **Disable reuse for this run** in the Start or Enqueue dialog. REST and MCP `dagu_execute` clients can send `noReuse: true` for start or enqueue.

---

# External CLI Harnesses

Use `action: harness.run` to invoke external coding-agent CLIs from DAG steps. Dagu selects and invokes the configured CLI; the CLI itself must be installed on the host or available in the selected container.

## Supported Providers

| Provider | Binary | Invocation |
|----------|---------|------------|
| `claude` | `claude` | `claude -p "<prompt>" [flags]` |
| `codex` | `codex` | `codex exec "<prompt>" [flags]` |
| `copilot` | `copilot` | `copilot -p "<prompt>" [flags]` |
| `opencode` | `opencode` | `opencode run "<prompt>" [flags]` |
| `pi` | `pi` | `pi -p "<prompt>" [flags]` |

Codex defaults to `skip_git_repo_check: true`, so its default invocation includes `--skip-git-repo-check`. Set `skip_git_repo_check: false` or `skip-git-repo-check: false` to omit it.

Claude's `bare` flag skips keychain reads among other things, so a subscription login is invisible to the step and the run fails with `Not logged in`. Set it only when the credential comes from `ANTHROPIC_API_KEY` in the step environment.

For host subprocess runs, built-in provider adapters resolve binaries through `PATH`. Host custom harnesses can use a binary name or an explicit path; relative paths with a path separator are resolved from the step working directory. For containerized runs, the binary is executed inside the selected container and must be valid there.

## Feature Reference

- `with.prompt` is required and is passed to the selected provider according to its built-in adapter or custom harness definition. Multiline prompt text is preserved.
- `with.stdin` is optional supplementary stdin for host subprocess runs. Containerized harness runs reject stdin.
- Built-in provider adapters are `claude`, `codex`, `copilot`, `opencode`, and `pi`. Non-reserved `with` keys become CLI flags.
- Custom providers must be declared under top-level `harnesses:`. Custom names cannot collide with built-in provider names.
- `fallback` is an ordered list of provider configs. Dagu tries the next config only when the previous attempt fails and the run context is still active. Fallback configs cannot contain another `fallback`.
- `provider` may use value references only if they resolve to a concrete provider string before executor creation. If `${...}` remains unresolved at runtime, the harness fails with an unresolved provider template error.
- `provider` and `fallback` are harness control keys. They are not passed as CLI flags.

## How `with` Works

Harness supports built-in provider adapters and named custom harness definitions:

- `with.provider` selects a built-in provider adapter or a custom `harnesses:` entry
- top-level `harnesses.<name>` defines how to invoke a custom harness CLI

For built-in provider adapters and custom providers, non-reserved `with` keys are passed directly as CLI flags:

- `key: "value"` → `--key value`
- `key: true` → `--key`
- `key: false` → omitted
- `key: 123` → `--key 123`
- Arrays repeat the flag once per item
- Built-in provider adapters also normalize `snake_case` keys to kebab-case flags, so `max_turns` becomes `--max-turns`

Reserved keys are `prompt`, `stdin`, `provider`, and `fallback`.

## Custom Harness Registry

Define reusable custom harness adapters once at the DAG level:

```yaml
harnesses:
  gemini:
    binary: gemini
    prefix_args: ["run"]
    prompt_mode: flag
    prompt_flag: --prompt
    option_flags:
      model: --model

steps:
  - id: review
    action: harness.run
    with:
      prompt: "Review the current branch"
      provider: gemini
      model: gemini-2.5-pro
```

Custom harness definition fields:

- `binary` — CLI binary or path
- `prefix_args` — args that always appear before prompt placement and runtime flags
- `prompt_mode` — `arg`, `flag`, or `stdin`
- `prompt_flag` — required when `prompt_mode: flag`
- `prompt_position` — `before_flags` or `after_flags`
- `flag_style` — `gnu_long` or `single_dash`
- `option_flags` — per-option override from `with` key to exact flag token

## DAG-Level Defaults and Fallback

Use top-level `harness:` to define shared defaults for every harness step in the DAG.

```yaml
harness:
  provider: claude
  model: sonnet
  fallback:
    - provider: codex
      full-auto: true
    - provider: copilot
      yolo: true
      silent: true

steps:
  - id: step1
    action: harness.run
    with:
      prompt: "Write tests"

  - id: step2
    action: harness.run
    with:
      prompt: "Fix bugs"
      model: opus
      effort: high

  - id: step3
    action: harness.run
    with:
      prompt: "Generate docs"
      provider: copilot
      fallback:
        - provider: claude
          model: haiku
```

Merge rules:

- DAG-level primary harness config is the base
- Step-level `with` overlays it
- Step-level `with.fallback` replaces DAG-level `fallback`
- Step-level `with.fallback: []` disables inherited fallback
- New DAGs should use `action: harness.run`. Legacy step-level `type: harness` remains loadable for backward compatibility.
- A top-level `harness:` config only supplies defaults. It does not set the type of a step that does not name one, so shell `run:` steps can sit alongside harness steps in the same DAG.
- A step that named its prompt with a bare `command:`, relying on the removed inference, is rejected while loading. Port it to `action: harness.run` with `with.prompt`, and `with.stdin` where it used `script:`.

## Containerized Harness Steps

`container:` is optional for `harness.run`. It can be defined at the DAG root or on a specific harness step.

Use root-level `container:` when all compatible steps should run in the same DAG-level container. Image-mode root containers create a shared container for the run; `container.exec` uses an existing container. A harness step without its own `container:` executes the provider CLI inside that shared container.

```yaml
container:
  image: my-codex-runner:latest
  pull_policy: always
  working_dir: /workspace
  volumes:
    - .:/workspace:rw

steps:
  - id: fix_tests
    action: harness.run
    with:
      provider: codex
      prompt: "Fix the failing tests in this repository"
      sandbox: workspace-write
      skip-git-repo-check: true
    timeout_sec: 600
```

Use step-level `container:` when only that harness step needs a container, or when it needs a different container from the DAG-level one. If both root-level and step-level containers are present, the step-level container is used for that step.

```yaml
steps:
  - id: fix_tests
    action: harness.run
    container:
      image: my-codex-runner:latest
      pull_policy: always
      working_dir: /workspace
      volumes:
        - .:/workspace:rw
    with:
      provider: codex
      prompt: "Fix the failing tests in this repository"
      sandbox: workspace-write
      skip-git-repo-check: true
    timeout_sec: 600
```

Container rules:

- The selected provider binary must exist inside the container that runs the step.
- Built-in provider adapters and custom providers with `prompt_mode: arg` or `prompt_mode: flag` can run in a container.
- `with.stdin` is not supported in a container. Step-level container plus `with.stdin` is rejected during DAG validation; root shared-container stdin is rejected when the harness step runs.
- Custom providers with `prompt_mode: stdin` are not supported in a container.
- A step-level image-mode container creates a container for the step. Dagu uses the provider binary as the container entrypoint and passes provider arguments as the command.
- Do not set `container.name` for image-mode harness steps. Use `container.exec` when the step must execute inside an existing container.
- For a root-level container, Dagu executes the full provider command inside the shared DAG-level container.
- A step-level container inherits user-defined runtime values such as DAG env, step env, params, secrets, and step outputs. The engine process environment is not injected.
- A root-level shared-container harness env filters host-path runtime values such as `PWD`, DAG run log paths, artifact paths, and step stream paths. Do not rely on those host paths inside the shared container.
- Step-level `container.env` overrides inherited values with the same key.
- DAG resource limits are applied to created step-level harness containers and created DAG-level containers when the container runtime supports those limits. Existing-container exec mode cannot change that container's host resources.
- Provider flags still belong under `with:`. For example, Codex `sandbox: workspace-write` configures Codex inside the outer container boundary.
- Docker or Podman is selected by the Dagu service process. This is not configured in the DAG YAML.

## Pattern 1: Single Harness Step

```yaml
params:
  - PROMPT: "Explain the main function in this project"

harness:
  provider: claude
  model: sonnet

steps:
  - id: run_cli
    action: harness.run
    with:
      prompt: "${params.PROMPT}"
    output: RESULT
```

## Pattern 2: Multiple Harness Steps

Chain harness steps that invoke external CLIs, passing output between steps via env-scope `output:` variables or `${step_id.stdout}` file references.

```yaml
type: graph

params:
  - topic: ""

steps:
  - id: research
    action: harness.run
    with:
      prompt: "Research every approach to: ${params.topic}. List all approaches with pros, cons, and when to use each."
      provider: claude
      model: sonnet
    output: RESEARCH

  - id: review
    action: harness.run
    with:
      prompt: "Review the research provided on stdin for completeness and gaps"
      # Interpolated before execution, then piped to the harness CLI on stdin.
      stdin: |
        Review this research for completeness and gaps:

        ${env.RESEARCH}
      provider: codex
      full-auto: true
      skip-git-repo-check: true
    depends: [research]
    output: REVIEW

  - id: refine
    action: harness.run
    with:
      prompt: "Refine this research incorporating the review feedback provided via stdin."
      stdin: |
        === Research ===
        ${env.RESEARCH}

        === Review Feedback ===
        ${env.REVIEW}
      provider: claude
      model: sonnet
    depends: [review]
    output: REFINED
```

`with.prompt` is the prompt. For host subprocess runs, built-in provider adapters and custom `arg`/`flag` harnesses receive `with.stdin` on stdin as supplementary context. For host subprocess custom `stdin` harnesses, stdin receives the prompt, then a blank line, then `with.stdin` when both are present.

## Pattern 3: Parameterized

```yaml
params:
  - PROVIDER: claude
  - MODEL: sonnet
  - PROMPT: "Analyze this codebase"

steps:
  - id: agent
    action: harness.run
    with:
      prompt: "${params.PROMPT}"
      provider: "${params.PROVIDER}"
      model: "${params.MODEL}"
    output: RESULT
```

## Provider Examples

### Claude Code

```yaml
steps:
  - id: task
    action: harness.run
    with:
      prompt: "Write tests for the auth module"
      provider: claude
      model: sonnet
      effort: high
      max-turns: 20
      max-budget-usd: 2.00
      permission-mode: auto
      allowed-tools: "Bash,Read,Edit"
    timeout_sec: 300
    output: RESULT
```

### Codex

```yaml
steps:
  - id: task
    action: harness.run
    with:
      prompt: "Fix failing tests in src/"
      provider: codex
      full-auto: true
      sandbox: workspace-write
      ephemeral: true
      skip-git-repo-check: true
    timeout_sec: 300
```

### Copilot

```yaml
steps:
  - id: task
    action: harness.run
    with:
      prompt: "Refactor the authentication middleware"
      provider: copilot
      autopilot: true
      yolo: true
      silent: true
      no-ask-user: true
      no-auto-update: true
    timeout_sec: 300
```

### OpenCode

```yaml
steps:
  - id: task
    action: harness.run
    with:
      prompt: "Refactor the database layer"
      provider: opencode
      format: json
    timeout_sec: 300
```

### Pi

```yaml
steps:
  - id: task
    action: harness.run
    with:
      prompt: "Design a rate limiting middleware"
      provider: pi
      thinking: high
      tools: read,bash
    timeout_sec: 300
```

## Notes

1. **Model names** — Look up current model names from each provider's documentation. Do not rely on hardcoded names; they change frequently.
2. **Prompt as a parameter** — Expose the prompt via `params:` so users can customize from UI/CLI without editing the DAG.
3. **Timeouts** — Set `timeout_sec:` (300-600s+) on harness steps. External CLI providers can run for minutes.
4. **Retry on transient failures** — Add `retry_policy: { limit: 3, interval_sec: 30 }` to handle rate limits and network errors.
5. **Working directory** — Use `working_dir:` on the step. The CLI operates relative to this directory.
6. **Output capture** — Use string-form `output: VAR_NAME` for small flat values, declared `outputs:` for explicit `${steps.<step_id>.outputs.<name>}` values, object-form `output:` for structured `${step_id.output.*}` access, and `stdout.artifact` / `stderr.artifact` when large provider output, reports, JSON, Markdown, or logs should be stored as DAG-run artifacts. Use `${step_id.stdout}` only when a downstream step needs the stdout log file path.
7. **Exit codes** — `0` means success. Provider process or container failures keep the provider's exit code when available; setup and internal errors use `1`; timeout and cancellation paths use `124`.
8. **Failure output** — Recent stderr is included in the error message on failure. When failed stdout exists, Dagu also includes a recent stdout tail and writes that tail to stderr.
9. **Fallback behavior** — If the primary harness config fails and the context is still active, fallback entries are tried in order. Failed-attempt stdout is not emitted to step stdout; its recent tail may be written to stderr and included in failure diagnostics. Stderr from failed attempts remains visible in logs.