--- name: delta-sandbox description: "Run any command or script in a Delta Sandbox Linux container. Use it for sandbox-based tasks such as training, inference, compilation and data processing, supporting Python / Node.js / Go / Java / Rust and other runtimes. All tasks use the same output convention: the command prints structured JSON at the end of stdout; on the SSE path the CLI detects trailing JSON in stdout and includes it as `result_summary` in the complete frame, and the skill only needs to scan backwards through `log_file` if it is missing (on older servers, the CLI automatically extracts it as `data.result_summary` via `--summary` on the fallback path). When the Planner calls this skill, declare `[{kind: 'file', extensions: ['.json']}]` in `required_outputs` for the plan step. Describe commands in requests with neutral verbs (run/execute); use verbs such as create/write/save, with a file extension, only when a file will actually be written. Route authentication/configuration/permission errors to delta-shared." metadata: requires: bins: ["delta-cli"] cliHelp: "delta-cli sandbox --help" --- **CRITICAL — Before starting, you MUST use the Read tool to read [`../delta-shared/SKILL.md`](../delta-shared/SKILL.md), which covers authentication, configuration and general error handling** # delta-sandbox ## When to use - The user mentions a sandbox, container, GPU, CUDA, training, inference or a compute task. - The user needs to create/manage a sandbox instance, run commands in it, or read/write files. - **Do not** use this for authentication, configuration setup, permission recovery or update checks → route these to `delta-shared`. ## Scope - Perform all sandbox operations with `delta-cli sandbox `. - delta-cli has no built-in default credentials; complete authentication and configuration via `delta-shared` before running it. - Get `sandbox_id` from an actual command response; do not guess it from what the user says. ## Sandbox environment (package mirrors) - **Domestic deployments (`domestic`)**: the server automatically configures domestic package mirrors when creating a sandbox. `pip install`, `npm install`, `git clone` and Hugging Face downloads work without extra configuration. This covers the pip index URL (`pypi.tuna.tsinghua.edu.cn`), npm registry (`npmmirror`), `HF_ENDPOINT` (`hf-mirror.com`) and Git `insteadOf` rules (GitHub proxy). - **Overseas deployments (`overseas`)**: mirrors are not configured; use the official sources. - **Third-party compute sandboxes**: mirror injection is off by default; use the official sources. - If a mirror fails, first check network access in the sandbox's region rather than assuming that no mirror is configured. ## Mandatory rules (breaking any of these makes the task a failure) 1. **You must use delta-cli**: perform all sandbox operations with `delta-cli sandbox `. **Do not** call the HTTP API directly with Python (`urllib`/`httpx`/`requests`) or any SDK. delta-cli provides typed error output (`error.type`), the only reliable way for an AI agent to interpret errors. 2. **You must destroy the sandbox**: after each successful `sandbox create`, you **must** run `sandbox kill ` or `sandbox finish ` to destroy it before the current task's final `step_finish`. Failing to destroy it counts as task failure. 3. **Destroy it even after an error**: if a command inside the sandbox fails, times out or reports an error, you **must still** run `sandbox kill `. An error is no reason to skip cleanup. 4. **Do not rely on external cleanup**: sandboxes **do** have automatic reclamation, but it does not replace destroying them yourself. - The default `--max-life` is 30 minutes. On private compute the timer starts when the create request is accepted and the platform reclaims the sandbox; on third-party compute it starts when the sandbox is ready, and the server-side cleanup reclaims it at max-life plus up to 10 minutes of grace. - The platform may still reclaim a sandbox early (cluster eviction, node failure, etc.; `termination_reason` is `platform_expired` for platform-side expiry, `platform_failed` for a failed Pod, `platform_terminated` for other remote termination, or `platform_gone` when the remote object no longer exists — read `termination_remark` for the underlying reason). - **Automatic reclamation does not write back to OSS**, so artifacts that were not pulled are lost. - Without `kill`, the sandbox keeps occupying GPU resources until it expires, and the reclamation time cannot be used to preserve results. When the task ends you must still run `kill` / `finish` explicitly (only `finish` writes back to OSS). 5. **Complete synchronous tasks in one command; background tasks may span turns**: - Synchronous mode (`sandbox run`): complete the full lifecycle from `create` to `kill` in the same execution. Do not retain a sandbox across tasks. - Background mode (`sandbox run-bg`): you may report `sandbox_id` and `execution_id` to the user during the current execution, then retrieve the result in a later turn with `sandbox status` and `sandbox logs`. You must still run `sandbox kill` when the task is complete. 6. **Do not create duplicates**: only one sandbox may be active for the same task. If a command needs retrying, reuse the existing `sandbox_id` where possible. If a new sandbox really is needed, first run `sandbox kill ` and confirm that the old instance has been destroyed before running another `sandbox create`. You may use `delta-cli sandbox list` to find and **clean up leftovers** from an interrupted previous task, but **do not** use `list` to get around the rule that only one sandbox may be kept for the same task. 7. **Long tasks must run in the background**: for tasks expected to take over 60 seconds (for example, model downloads, training, fine-tuning, long inference runs or large-scale data processing), submit a background task with `delta-cli sandbox run-bg --command "..." --timeout `. Do not use synchronous `sandbox run`. **Recommended: one tool call, wait for completion**: ```bash delta-cli sandbox run-bg --command "" --timeout --wait ``` The CLI follows output in real time over SSE (`/logs/stream`) and passes each raw SSE frame (`data: {...}`) through to stdout, as with `curl -N`. The JSON `type` field identifies each frame: `init` (with `execution_id`), `stdout`, `stderr`, `error` or `complete`. The command ends when `complete` arrives. **The CLI fills in fields in the complete frame**: the server supplies `exit_code`/`execution_id`/`error`; the CLI adds the accumulated streamed `stdout`/`stderr`, a recovered `log_file` as a fallback, and a trailing-JSON `result_summary` when it detects a valid JSON object in the last 50 lines of stdout. Use `exit_code` in the `complete` frame to determine success. **If the command exits non-zero or reports an error, the CLI appends an `error.type: command_failed` envelope and exits with a non-zero code (process exit code 8)**; the envelope's `message` includes the end of stderr, which makes it easier to use with `&&`, branch on `error.type` and scan the failure reason. The CLI also best-effort appends the stderr tail to the `message` of a `type:"error"` frame; it does not append an envelope on success. If no `complete` frame arrives within `--timeout` (300s by default, serving as both the server command timeout and the client wait deadline), the CLI appends a snapshot envelope with `finished=false` after the frames already passed through, indicating that the command is still running — the invocation itself then exits non-zero with `error.type: timeout` (process exit code 11); `error.details.remote_state` tells whether the remote command was cancelled. After `finished`, full stdout is written to `log_file` inside the sandbox (default: `/workspace/{user_id}/sandbox_logs/delta-result-{sandbox_id}-{execution_id}.json`; use the server-returned `log_file` field as authoritative). To read the full output, use `sandbox read --path ` (the trailing JSON summary is already in the complete frame, so this is usually unnecessary). **`--summary`/`--artifacts` enrichment, including hints, is available only on the polling fallback path** (the CLI falls back to polling every 5s and returns a `CommandResult` envelope when the server lacks the `/logs/stream` endpoint, **or when the SSE stream drops mid-flight with a network error after `execution_id` is known**). A dropped stream is not a command outcome: the remote command may still be running, so the CLI reconciles it by polling instead of failing; only non-network stream errors fail immediately. Use `execution_id` for later `sandbox logs` or `cancel` calls. **For automation: `--format json` (same as `--json-result`)** — emits no `data:` frames, just **one** JSON envelope, so you can parse `data` directly instead of stitching stdout frames together: ```bash delta-cli sandbox run --command "" --timeout --format json delta-cli sandbox run-bg --command "" --timeout --wait --format json ``` The `data` fields have the same names and meaning as in the `complete` frame: `execution_id`, `finished`, `exit_code`, `stdout_tail`, `stderr_tail`, `stdout_size`, `stderr_size`, `log_file`, `duration_seconds` and `result_summary`. `stdout_tail`/`stderr_tail` are the last 2000 characters of the accumulated streamed output (newlines preserved), so there is no need to aggregate the `stdout` frames. When the command exits non-zero the CLI emits `{"ok":false,"error":{...},"data":{...}}`, where `data` is the same object (including `exit_code`), and the process exit code is 8. This mode shares field semantics with SSE mode, so you can switch between them at any time. **Live following / manual polling (when you need intermediate progress)**: `sandbox logs --execution-id --stream` follows the task in real time (SSE), passing each raw `data:` frame through to stdout until the `complete` event, with no trailing envelope (`--stream` requires `--execution-id`). Without `--stream`, `sandbox logs --execution-id ` returns a snapshot envelope. You may omit `--execution-id`; if omitted, the server queries only by sandbox and automatically uses ES (Elasticsearch) because `sandbox_id` and `execution_id` were not both supplied (it returns an **array** aggregated by execution ID rather than a single snapshot object): - `finished=true` and `exit_code=0` → complete - `finished=true` and `exit_code!=0` → failed - `running=true` → still running **Troubleshooting**: if `running=true` for three consecutive checks, `stderr_tail`/`stdout_tail`/`cursor` have not changed, and more than 60 seconds have passed since submission, use `sandbox status ` to check whether the sandbox is alive. Note: `cursor` has no `omitempty` and is always present in the response, even when it is 0; `stderr_tail` retains only the **last 200 bytes** of stderr by default, while `stdout_tail` retains the **last 800 bytes** of stdout (`tailString(s, n)` is hard-coded in the CLI and truncates by byte; for ASCII this is n characters, while multi-byte UTF-8 may be cut off). `cursor=0` usually means the server has not yet read any stderr lines from the provider. - After a background task succeeds, you must still destroy the sandbox as required by rules 2/3. 8. **Files are written to the sandbox working directory by default; do not use shell-variable paths**: - ✅ **Recommended**: `delta-cli sandbox write --source train.py` (omit `--path` to write to `/train.py` by default) - ✅ **Explicit path**: `delta-cli sandbox write --path /train.py --source train.py` (relative paths passed to `--path` are also prefixed with the working directory) - ❌ **Do not**: use `--source "$WORKSPACE_ROOT/train.py"` or `--source "$(pwd)/train.py"` (shell expansion can produce an empty file or fail) - ❌ **Do not**: use `--data "a large block of code..."` (shell-escaping problems) - ❌ **Discouraged**: write directly to the working-directory root with a path such as `--path /workspace/train.py`, outside `{user_id}`. Data there is not written back to the shared workspace after `kill` and will not persist. - To find the sandbox working directory, run `delta-cli sandbox working-directory ` (returns `data.path`). When a script is used with `run`/`run-bg`, give its full path, such as `python /train.py`. - For small configuration files (< 20 lines), you may use `--data "..."`, but you must also pass `--path` (the path cannot be inferred from a filename). Use `--mode 755` to set file permissions. 9. **Each `run-bg` creates a separate `execution_id`**: every call to `sandbox run-bg` returns a unique `execution_id`, which distinguishes background tasks. Save each returned `execution_id` and associate it with its task, then query each result with `sandbox logs --execution-id `. Synchronous `sandbox run` does not need an `execution_id`; it returns the result directly. 10. **Use the available shell tool to run delta-cli commands directly; do not wrap `subprocess` in `python_repl`** - `delta-cli` returns structured JSON. Running it directly in the available shell keeps its output clear and makes `log_file` extraction easier. - ❌ **Do not** write `subprocess.run(['delta-cli', ...])` in `python_repl`. This mixes the output into a Python code block, making it harder to read and adding an unnecessary parsing step. - ✅ **Correct**: run `delta-cli sandbox ...` directly with the available shell tool. - Use `python3`/`python_repl` only when **parsing JSON**, for example, extracting `summary` from a `sandbox read` envelope and writing it to `result.json`. 11. **All code/data must enter the sandbox through `delta-cli sandbox write`** - All scripts, configuration files and data files to be run in the sandbox **must** be written with `delta-cli sandbox write --source `. By default, files go to that sandbox's working directory (`/workspace/{user_id}/{sandbox_id}/`, which you can query with `delta-cli sandbox working-directory `), within the OSS sync scope and available for persistence after `kill`. - Do not write a script to the host workspace and rely on synchronisation; files in the host workspace are not visible inside the sandbox. - This rule standardises the entry point as `sandbox write`; it does not depend on any particular host tool. Whatever file operation the host provides, the content must ultimately be sent into the sandbox. 12. **A task is complete when the command has produced a result, not when a script file exists** - The deliverable for this skill is **a summary of the result after the command has run in the sandbox** (the local `result.json` and its `result_summary`), not the `.py`, `.sh` or other script itself. - Writing a script to disk alone does not complete the task. Finish the full lifecycle—`write → run → read log_file → create result.json → kill sandbox`—before calling `step_finish`. - Therefore, do not phrase `skill_request` as “write/save/create a file and run it”, which makes writing the file sound like the milestone. Ask directly to “run the command and return a result summary”. 13. **`result.json` is required before completion** - Before calling `step_finish`, confirm that: 1. `delta-cli sandbox run` / `run-bg --wait` has returned with `finished=true` **and `exit_code` = 0**. `finished=true` alone is not a success signal — a command that fails still finishes. A non-zero exit is reported as `error.type: command_failed` (process exit code 8), an **unknown** exit status as `error.type: unknown_execution_state` (exit code 13), and a wait timeout as `error.type: timeout` (process exit code 11, with `sandbox_id`/`execution_id` in `error.details`); in all three cases the task has **not** succeeded — report the failure instead of claiming a result; 2. (By default) the `result_summary` in the SSE `complete` frame from `sandbox run` / `run-bg --wait` has been written to `result.json`; if it is missing (fallback), read `log_file` with `sandbox read` and parse it with Python. 3. `sandbox kill ` has been run to destroy the sandbox; 4. `final_response` consists of the single line `RESULT: `. 14. **Avoid redundant checks and validation** - If `skill_request` or the user's request already specifies the image and resources, **do not** call `sandbox images` or `sandbox recommend` before `sandbox create`. - After writing a file to the sandbox, it is enough to use `sandbox stat` (optionally) to confirm the size. **Do not** use `sandbox ls` plus `sandbox read` to read the file back to the host and verify its contents; that is an unnecessary round trip. (The `sandbox write` response already carries `server_sha256` and `size_verified`, so a local digest comparison is enough if the bytes matter.) - Do not repeatedly poll with `sandbox status` after a successful `sandbox create`; the sandbox is ready as soon as create returns. ## Request wording conventions To reduce the risk of the host's Phase-10B file-extension check misclassifying a request, follow these conventions when constructing `skill_request`: 1. **Use neutral verbs when asking to run/execute commands** Prefer `run`, `execute` or `call`. - Correct: `Run a PyTorch CUDA self-check script in a GPU sandbox` - Avoid putting code directly in the request and using a verb such as “output”, for example: `Script should output: 1) torch.cuda.is_available(); ...` The regular expression may misread `torch.cuda` as a commitment to produce a `.cuda` file. 2. **Use commitment verbs and file extensions only when a file will actually be created** Use words such as `create`, `write`, `save`, `output` and `generate`, and specify the extension, only when the skill will actually write a file to disk. - Correct: `Create check_cuda.py and run it` - Correct: `Write the metrics to result.json after training` 3. **Do not put a complete code block in `skill_request`** Describe the task goal in natural language; the skill writes the specific code into the sandbox. - Correct: `Run the standard PyTorch CUDA availability check` - Avoid pasting `import torch; print(torch.cuda.is_available())` into `skill_request`. ### Declaring required_outputs When constructing a plan step that calls this skill, you **must** declare that a `.json` file will be written in `required_outputs`: ```yaml required_outputs: - kind: file extensions: [".json"] ``` The reason is that, after running a command, this skill reads `log_file` from the sandbox and writes a local `result.json`. The host's Phase-10B deliverable check uses OR semantics (any required extension satisfied by an actual file is enough); explicitly declaring `.json` prevents code tokens such as `torch.cuda` from being misread as a commitment to produce a `.cuda` file, avoiding duplicate execution caused by `VERIFICATION_FAILED`. ## Quick routing > ⚠️ **Distinguish `list` from `ls`**: `sandbox list` lists the user's **sandbox instances** (like `docker ps`), while `sandbox ls ` lists a **directory inside a sandbox** (like the `ls` command). Take care when using Tab completion. | User goal | Command | |---------|------| | **Discover** | | | View available images (image names) | `sandbox images` — returns `image_name` (public image name), `tags` (purpose tags) and `description` (brief image description) for each image; when there are many, **choose an image that suits the task using `description` and `tags`** (for example, Pytorch → `PyTorch CUDA13 (GPU)`, vLLM → `vLLM …`, or check `supports_shared_workspace` when a shared workspace is needed). On new servers the entry may also carry `source` (always present), `provider_label` and `compatible_gpu_types` (GPU models the compute source currently offers) | | View platform compute inventory | `sandbox resources` — returns the platform's available GPUs, GPU memory and cores grouped by compute source: `compute_sources[].source` is `private` (private compute) or `third_party` (third-party compute); the shape of `gpu_types` varies by group (`private` includes `vgpu`/`core`/`memory_mib`; `third_party` includes `gpu_name`/`total_gpu_num`/`idle_gpu_num`). The compute backend is an internal detail and is selected automatically; it need not be specified. **This is platform inventory, not the current user's remaining hardware quota**; run `delta-cli quota` for the account's own limits (see "Account-level resource limits and GPU access" below). | | View the account's hardware quota and GPU access | `delta-cli quota` — effective limits, current usage and remaining amounts for GPU cards / GPU memory / CPU / memory / concurrent sandboxes / GPU trial duration, plus whether GPU is allowed, the provider, model and image allowlists, and the sub-account model denylist (`denied_gpu_types`). `null` means unlimited, `0` means zero quota; `per_sandbox` is the one-request ceiling, not the remaining amount. | | Get a resource recommendation | `sandbox recommend --cpu N --memory XGi [--gpu N] [--gpu-mem N]` (`--memory`/`--gpu-mem` accept g/m/G/M formats such as `1G`/`512M`; the CLI converts these automatically to Gi/Mi). On new servers the response may also include `source`, `gpu_type` and `provider_label`; `gpu_type` can be reused as `--gpu-type` | | List the current user's sandboxes | `sandbox list [--status ] [--start-time ] [--end-time ] [--sandbox-id ] [--days N]` | | **Lifecycle** | | | Create a sandbox | `sandbox create --image-name [--cpu N --memory XGi --gpu N --gpu-mem N --gpu-type --max-life M]`; `--memory`/`--gpu-mem` accept g/m/G/M formats (for example `1G`/`512M`, including lower-case forms), which the CLI converts to the server's `Gi`/`Mi` units; `--image-name` takes an image name (the `image_name` field returned by `sandbox images`, for example `"PyTorch CUDA13 (GPU)"`); the underlying image identifier is hidden from users. `--gpu-type` selects a GPU model (for example `"RTX 4090"` or `H100`); see `gpu_types[].gpu_name` in the output of `sandbox resources` for available models. If omitted, scheduling is automatic; on plans with a GPU-model allowlist this can auto-select a disallowed model, so pass `--gpu-type` explicitly (see "Account-level resource limits and GPU access" below). Add `--no-auto-cleanup` if you do not want automatic cleanup (only an explicit kill/finish destroys it). `--shared-workspace` defaults to true and enables a shared workspace; `supports_shared_workspace` in `sandbox images` indicates whether that compute source supports one. For quick tasks that do not rely on shared data, use `--shared-workspace=false` to skip workspace seeding/write-back. `--ready-timeout ` sets how long the server waits for the sandbox to become **ready during creation** (distinct from `--max-life`, which controls how long it lives afterwards). The default is 600s. It applies to **all compute sources** (private and third-party), and the accepted range is 10-1800s. Raise it when a cold node must pull a large image for the first time; otherwise creation can fail with `error_code=SandboxReadyTimeout`. | | Shared workspace (persistence across sandboxes) | `--shared-workspace=true` (the default) enables a shared workspace: when creating a sandbox, the platform **seeds** the user's workspace data into `/workspace/{user_id}`; on `kill`/`finish` it **writes the data back**, and the next sandbox receives it automatically, allowing files to be retrieved across sandboxes. **The only persistent directory is `/workspace/{user_id}`**. Files written elsewhere (`/tmp`, `/root`, custom directories, etc.) are lost when the sandbox is destroyed, so put any results you need to keep in this directory. ⚠️ Whether data persists across sandboxes depends on whether the platform detects a pre-installed `rclone` in the image at creation time: with rclone, seeding/write-back works; without it, those steps are skipped (data is not persisted across sandboxes, but creation still succeeds). If you are unsure whether an image includes rclone, write to `/workspace/{user_id}`, destroy the sandbox and create another to check. For quick tasks that do not rely on shared data, use `--shared-workspace=false` to skip seeding/write-back. | | Extend the lifecycle | `sandbox extend --max-life N` — **dynamically extend** a running sandbox's max-life (**extension only** — the new lifetime must exceed the elapsed minutes and be ≤10080 and must push the expiry later, otherwise the server rejects it with `422`; use `kill`/`finish` for early cleanup; sandboxes with `--no-auto-cleanup` do not need renewal). Without `--max-life`, this command reports the current lifecycle (elapsed time, time remaining and expiry); `status` output includes a `data.lifecycle` summary (elapsed time, time remaining and expiry; omitted if the lifecycle endpoint is unavailable). | | Connect to a sandbox | `sandbox connect ` | | View status | `sandbox status ` | | Finish a sandbox | `sandbox finish [--results '{...}']` | | Destroy a sandbox | `sandbox kill ` | | **Command execution** | | | Run a command synchronously (≤60s) | `sandbox run --command "..." [--timeout ] [--format sse\|json] [--json-result] [--summary/--no-summary] [--artifacts]` (`--format json` = a single unified result envelope, recommended for automation). **A non-zero exit is reported as `error.type: command_failed` with process exit code 8; an unknown exit status is reported as `error.type: unknown_execution_state` with exit code 13 — the CLI never reports an unknown outcome as success** | | Run a command in the background (>60s) | `sandbox run-bg --command "..." [--timeout ] [--wait] [--cancel-on-timeout] [--format sse\|json] [--json-result] [--summary/--no-summary] [--artifacts]` (`--wait --format json` = a single unified result envelope). **If the wait times out the CLI exits with `error.type: timeout` (process exit code 11) and keeps `sandbox_id`/`execution_id`/`remote_state` in `error.details`; `remote_state: not_cancelled` means only the local wait stopped and the remote command may still be running — add `--cancel-on-timeout` to cancel it** | | View background logs | `sandbox logs [--execution-id ] [--tail N --grep --context N] [--stream]` (`--execution-id` is optional; if omitted, query ES by sandbox only; only `--stream` requires it) | | Interrupt a background command | `sandbox cancel --execution-id ` | | **Tunnelling (port exposure)** | | | Expose a port publicly | `sandbox expose [--port N]` — expose a service port inside the sandbox to the public internet and return `public_url` (`http://:`, where the server randomly assigns `remote_port` from 20000–29999); **start the service before exposing it** (the server reports “service not ready” if its health check fails); `--port` defaults to 8700 (platform convention) | | List active tunnels | `sandbox exposes ` — list the public tunnels currently active for this sandbox; `data.sandbox_exists=false` means the sandbox is no longer running (the list is empty) — stop polling and create a new sandbox | | Reclaim a tunnel | `sandbox dispose --port N` — explicitly reclaim a single tunnel (stop frpc and release the public port); killing the sandbox also reclaims all its tunnels. Disposing on an already-destroyed sandbox is an idempotent success (`data.already_gone:true`, `data.error_code:sandbox_already_gone`) | | **File operations** | | | Read a file | `sandbox read --path [--output --tail N --grep --offset N --limit N --context N --max-bytes N --parse-json]` — **byte-exact**: a full copy goes through the chunked download (sha256-verified) and a filtered copy is a byte-exact slice of the original. Printed inline, a text file gives `content`; a **non-UTF-8 file gives no `content`** — the exact bytes are in `content_base64` | | Pull a file/directory | `sandbox pull --source --target [--pattern ] [--concurrency N] [--no-resume] [--no-verify]` — pull a single file or **recursively** pull a directory (directories are always recursive), over the **chunked** path: bytes move verbatim (no base64 inflation), binary-safe, sha256-verified, resumable and parallel (the reverse of upload: source=remote, target=local). A `--target` that names an existing local directory receives the file inside it, under its own name | | View the working directory | `sandbox working-directory ` (returns `data.path`, such as `/workspace/{user_id}/{sandbox_id}`) | | Write a file | `sandbox write --source ` (without `--path`, writes to the working directory by default); **every sandbox-side path** — `--path`, the remote part of `write-multiple --entry`, `upload --target`, `pull --source`, `run`/`run-bg --log-file` — is prefixed with the working directory when relative, while an absolute path is used as given. The content is **binary-safe** (raw bytes, with a base64 fallback for older servers) and the server verifies the declared `size`/`sha256` before writing, so the response's `server_sha256`/`size_verified` can be compared with the local file | | Write multiple files | `sandbox write-multiple --entry = [--entry ...]` | | Upload a directory | `sandbox upload --source --target ` | | List a directory | `sandbox ls --path ` (defaults to `.`) | | File metadata | `sandbox stat --path ` | | Move/rename | `sandbox mv --entry [--entry ...]` | | Replace content | `sandbox replace --path --old --new ` | | Change permissions | `sandbox chmod --path --mode ` | | Delete files | `sandbox rm --path [--path ...]` — files only; a directory path is rejected, a missing path is an error | | Delete directories | `sandbox rmdir --path [--path ...]` — directories only; a file path is rejected and the file is left untouched, a missing path is an error | | Create directories | `sandbox mkdir --path [--path ...]` | | Search files | `sandbox search --path --pattern ` — an unterminated glob such as `[` is rejected, not silently empty | **Per-sandbox resource limits**: cpu ≤ 512 / memory ≤ 1024Gi / gpu ≤ 64 / gpuMem ≤ 1024Gi. Requests above these limits fail quickly rather than hanging. **Creation timeout**: `--ready-timeout` (default 600s, range 10-1800s) bounds how long the server waits for a sandbox to become ready *during creation*, and applies to **all compute sources** (private and third-party). A sandbox that is created but never becomes ready fails with `error_code=SandboxReadyTimeout`, which usually means the image was not yet cached on the target node and the first pull exceeded the timeout. `--max-life` is unrelated: it caps the sandbox's lifetime after creation. ## Account-level resource limits and GPU access Besides the per-sandbox limits above, the server enforces limits across **all running sandboxes of the account**. A create request is rejected when the new sandbox would push the account over any of these: | Dimension | How it is counted | Rejection code | |---|---|---| | GPU cards | GPU cards across all running sandboxes | `TIER_GPU_TOTAL_EXCEEDED` | | GPU memory | `gpu × gpu-mem` across all running sandboxes | `TIER_GPU_MEM_TOTAL_EXCEEDED` | | CPU cores | requested cores across all running sandboxes (CPU-only sandboxes included) | `TIER_CPU_TOTAL_EXCEEDED` | | Memory | requested memory across all running sandboxes (CPU-only sandboxes included) | `TIER_MEMORY_TOTAL_EXCEEDED` | | Concurrent sandboxes | number of running sandboxes | `TIER_CONCURRENCY_EXCEEDED` | - `--gpu-mem` is **per card**: `--gpu 2 --gpu-mem 8000` reserves 16000 MiB of account GPU memory. - Typical default limits by plan (account level): | Plan | GPU cards | GPU memory | CPU cores | Memory | Concurrent sandboxes | |---|---|---|---|---|---| | free | 2 | 8192 MiB | 2 | 8192 MiB | 1 | | medium | 4 | 16384 MiB | 4 | 16384 MiB | 2 | | prime | 8 | 32768 MiB | 8 | 32768 MiB | 5 | - Limits follow the account's plan and may be adjusted by an operator; the table shows defaults, not a guarantee. A single sandbox is limited by the smaller of its per-sandbox limit and the account limit. - **On the free plan, `--cpu 4 --memory 16Gi` is rejected** (the free account has only 2 cores / 8 GiB). Use a free-safe configuration such as `--cpu 2 --memory 8Gi --gpu 1 --gpu-mem 8000 --gpu-type "RTX 4090"`. - **Run `delta-cli quota` to see the account's effective limits and remaining amounts** instead of guessing: `per_sandbox` is the one-request ceiling, and `account.` gives `limit`/`used`/`remaining` for GPU cards, GPU memory (`gpu × gpu-mem`), CPU, memory, concurrency and the GPU trial duration. `null` means unlimited; `0` means zero quota. `sandbox resources` still shows platform inventory and `usage` shows credits only. ### GPU model allowlists Some plans restrict GPU models and compute sources, and the effective allowlist may change over time — do not rely on any fixed model list written here. Run `delta-cli quota`: `gpu_access.allowed_gpu_types` is the account's effective model allowlist (empty = no extra plan-level model restriction), and the `account` dimensions show the cumulative GPU trial-duration limit. - **On a plan with an allowlist, always pass `--gpu-type` explicitly.** If it is omitted, automatic scheduling may pick a model outside the allowlist and creation fails with `TIER_GPU_TYPE_NOT_ALLOWED`. - **Sub-accounts may also have a parent-set model denylist.** Run `delta-cli quota` and read `gpu_access.denied_gpu_types` (empty = no extra restriction). The denylist overrides the plan allowlist: any model matching an entry (case-insensitive substring, so `A100` also blocks `A100 80G`) is rejected with `SUBACCOUNT_GPU_TYPE_DENIED`. A model can therefore appear in `allowed_gpu_types` and still be unusable. - `TIER_GPU_TYPE_NOT_ALLOWED` → retry with a model named in the error message; do not keep changing CPU/memory. - `TIER_GPU_NOT_ALLOWED` → the plan has no GPU access; `TIER_GPU_PROVIDER_NOT_ALLOWED` → the selected provider is outside the plan; `TIER_GPU_IMAGE_NOT_ALLOWED` → the image is outside the plan. - `TIER_GPU_DURATION_EXCEEDED` → the cumulative GPU trial time is exhausted and an upgrade is required. ### Sub-account limits Sub-accounts created by a parent account may have their own hardware allocation and GPU-model blacklist: - With no explicit allocation, a sub-account defaults to **2 GPU cards and 16000 MiB of account GPU memory** (the per-sandbox GPU cap is also clamped to 2). - An explicit allocation overrides GPU cards, GPU memory, CPU, memory and concurrency. `max_gpu_cards = 0` means **no GPU**, not unlimited. - `denied_gpu_types` is a per-sub-account blacklist. When it is non-empty, a GPU request **without `--gpu-type` is rejected** (`SUBACCOUNT_GPU_TYPE_DENIED`) so the blacklist cannot be bypassed by omitting the model. - The parent and all its sub-accounts share a **family GPU card pool** (the parent plan's card limit, capped at 8). Exceeding it returns `TIER_GPU_TOTAL_EXCEEDED` with "account family" in the message. - Sub-accounts have no 2-hour trial cap; usage is billed normally. - These sub-account rules take effect only when the deployment enables sub-account quotas; otherwise the parent plan's limits apply. ## Full lifecycle 1. **Choose an image (if needed)**: - The compute backend (provider) is an internal detail selected automatically by the system; **users neither need to nor can specify it**. - If the request already specifies the image (for example, the image name in `--image-name "PyTorch CUDA13 (GPU)"`) and resources (cpu/memory/gpu, plus optional `--gpu-type`), **go straight to step 2 and create the sandbox**. Do not call `sandbox images` or `sandbox recommend` again. - Call these only when the user asks for “a configuration recommendation” or you need to check which images are available: - `delta-cli sandbox images` — list the images supported by the server (returns image names and hides the underlying image identifiers) - `delta-cli sandbox recommend --cpu N --memory XGi [--gpu N]` — recommend a resource configuration 2. **Create**: `delta-cli sandbox create --image-name --cpu 2 --memory 8Gi --gpu 1 --gpu-mem 8000 --gpu-type "RTX 4090" --max-life 120`. **The returned JSON envelope contains `data.sandbox_id`, not `data.id`; all later commands must use this real `sandbox_id`.** (The sandbox is ready immediately after creation; there is no need to connect.) **The response echoes the requested image name/resources (the CLI fills in fields omitted by the server using the request values, while server values take precedence), so you can verify the resource configuration without calling status.** (The echoed image is its display name; the compute backend is auto-selected and surfaces only as a display label — `provider`/`provider_label` and `compute_source` may appear in responses but cannot be chosen.) **If this task already has a `sandbox_id`, do not create another; reuse it first.** This is a free-plan-safe example; increase CPU/memory/GPU only within the account-level limits above. - `--max-life` sets the sandbox's maximum lifetime in minutes; the default is 30. Increase it for long tasks so the sandbox is not reclaimed while a command is running. - **Match `--gpu-mem` (in MiB) to the model being run**: larger models and higher precision need more GPU memory; a large model in too little memory will cause an OOM. The example value 8000 (8GiB) is suitable only for small models. Estimate memory needs and increase it as appropriate before running a 7B/9B/13B model. **Meaning: `--gpu-mem` is GPU memory in MiB per card (`--gpu 2 --gpu-mem 8000` = 8000MiB per card, 16000MiB in total); `--memory` is the container's total CPU memory.** - **`--gpu-type ` (optional) selects a GPU model**: for example, `"RTX 4090"` or `H100`. This schedules the task on that GPU model. Use `sandbox resources` to see available models (`gpu_types[].gpu_name`). Specify a model only when the user explicitly requests one; otherwise, the system schedules automatically. **On a plan with a GPU-model allowlist (notably free), pass `--gpu-type` explicitly** — automatic scheduling can otherwise pick a disallowed model and fail with `TIER_GPU_TYPE_NOT_ALLOWED`. - `--no-auto-cleanup`: when this flag is set, the sandbox **will not be cleaned up automatically** (the create request is sent with no TTL, so platform TTL and the server's periodic cleanup are both skipped). It can be destroyed only by an explicit `sandbox kill`/`finish`, and `status` reports `expires_at`/`remaining` as null. Use this only when the task genuinely needs to survive past periodic cleanup; you must destroy it explicitly when the task is done to avoid leaking resources. - **Do not repeatedly poll with `sandbox status` after a successful create**. The sandbox is ready as soon as `sandbox create` returns; use its returned `data.sandbox_id` directly for `write`/`run`. Extra polling adds tool calls without providing any benefit. 3. **Write code/data**: - **Single file**: `delta-cli sandbox write --source ` — omit `--path` to write to the sandbox's working directory by default (`/workspace/{user_id}/{sandbox_id}/`; query it with `delta-cli sandbox working-directory `). Pass only a local filename to `--source` (for example, `--source train.py`). **Do not** use shell-variable paths such as `"$WORKSPACE_ROOT/train.py"` or `$(pwd)/train.py`. With `--source`, the CLI reads the local file itself without shell expansion, making this the safest option. If you need a specific path, a relative `--path ` is automatically prefixed with the working directory; absolute paths are used as given (the same rule applies to every sandbox-side path: the remote part of `write-multiple --entry`, `upload --target`, `pull --source`, and `run`/`run-bg --log-file`) (**avoid** paths such as `/workspace/` outside `{user_id}`, as they do not persist after `kill`). The returned `size` field is the actual number of bytes on disk (verified with stat), so you can check that the write is complete. For small configuration files (<20 lines), you may use `--data "..."`, but you must also pass `--path`. File paths and extensions depend on the runtime in the image. - **Write multiple files**: `sandbox write-multiple --entry = [--entry ...]` (remote path is on the left of `=`, local path on the right; bulk writes with `--data` are not supported) - **Upload a directory**: `sandbox upload --source --target ` — the CLI packages the local directory as tar.gz and uploads it using multipart/form-data; the server extracts it into the target directory. The response lists each file's path, size and mode. After upload, the CLI checks the local and remote file lists for integrity and warns about mismatched sizes, extra files and similar issues. - **Note**: `--source` is a **local directory path**; `--target` is the **target directory inside the sandbox**. The target directory is created if it does not exist. - Nested directories are supported, and empty directories are created too. - **Optional post-write check**: use `sandbox stat --path ` to confirm that the file exists and its size is as expected. **Do not** use `sandbox ls` plus `sandbox read` to read the just-written file back to the host and compare it character by character; if there is no issue, no read-back is needed. 4. **Run the command**: - **Short task (expected to take ≤ 60 seconds)**: run synchronously with `delta-cli sandbox run --command "" --timeout `. The command's stdout/stderr is passed to stdout over SSE as **raw `data:` frames, one at a time** (with no trailing envelope). Read `exit_code` / `log_file` / `execution_id` / `stdout_tail` / `stderr_tail` / `duration_seconds` from the `complete` frame. For automation, add `--format json` (or `--json-result`) so the CLI emits a single unified result envelope instead of making you stitch frames together yourself. After completion, the full `stdout` is written to the run-envelope log file (the `log_file` field); use `sandbox read` to read it in full or extract trailing JSON. **Do not** then call `sandbox logs`. You can customise the run-envelope log path with `--log-file `; the default is `/workspace/{user_id}/sandbox_logs/delta-result-{sandbox_id}-{execution_id}.json` (use the server-returned `log_file` field as authoritative). A script written with `sandbox write` is in the working directory; run it using its full path, `/` (first query `sandbox working-directory `). Build the command for the runtime in the image. Common examples (where `` is the working directory): - Python: `python /train.py` - Node.js: `node /app.js` - Go: `go run /main.go` - Shell: `bash /run.sh` - **Long task (expected to take > 60 seconds, such as a model download, training, compilation or large-scale data processing)**: submit it in the background with `delta-cli sandbox run-bg --command "" --timeout [--log-file ]`. Once you have the `execution_id`, retrieve the result with one of these commands: - `delta-cli sandbox logs --execution-id --stream` — follow the task in real time over SSE, passing raw `data:` frames through to stdout until the `complete` event (including `exit_code`/`log_file`), with no trailing envelope (`--stream` requires `--execution-id`). - `delta-cli sandbox logs --execution-id ` — returns a snapshot envelope with `stdout_tail` (last 800 bytes), `stderr_tail` (last 200 bytes), `stdout_size`, `stderr_size`, `cursor`, `running`, `finished`, `exit_code` and (after completion) `log_file`. Treat it as complete when `finished=true`; read `log_file` for the full output. You may omit `--execution-id`; if omitted, the server queries by sandbox only and automatically uses ES because `sandbox_id`/`execution_id` were not both supplied, returning an **array** aggregated by execution ID. Do not use synchronous `sandbox run` for long tasks. **Recommended practice**: have the sandbox script print a standalone structured JSON object at the end of `stdout` (for example, `{"status":"ok", ...}`). On the SSE path, the CLI detects this trailing JSON and includes it as `result_summary` in the `complete` frame. **When the task ends, reuse the `result_summary` from the complete frame directly in `result.json`**; there is no need to scan `log_file` again with `sandbox read`. Only if `result_summary` is missing from the complete frame (no valid trailing JSON) should you read `sandbox read --path ` and scan backwards as described in step 5, avoiding a request for the model to “read the whole log and summarise it”. 5. **Create result.json** (prefer the SSE path: reuse `result_summary` from the complete frame; only read and parse `log_file` if it is missing): - **Get the `log_file` path**: the SSE `complete` frame from `run` / `run-bg --wait` contains a `log_file` field (the CLI fills it in as a fallback); the `sandbox logs --execution-id ` snapshot also returns `log_file` once `finished=true`. Parse it as below only when you need to read it: - `sandbox read` returns a CLI envelope `{"ok":true,"data":{"content":"..."}}`; the actual result JSON is in the `data.content` field. - **Do not write the CLI envelope or the full, lengthy `stdout` verbatim to local `result.json`**; parse it and create a concise structured summary. - **Recommended approach (to avoid heredoc quoting failures)**: first use the available shell to save the envelope to a temporary file, then use `python3` (or `python_repl`) to read that file and create `result.json`. **Do not** pipe the output of `delta-cli sandbox read` directly into an inline heredoc; combining a pipe and heredoc is very likely to fail due to shell escaping/quoting. 1. Save the envelope in the available shell (Bash syntax example): ```bash delta-cli sandbox read --path > /tmp/_delta_result_envelope.json ``` 2. `python3` / `python_repl` creates `result.json`: ```python import json, os env = json.load(open("/tmp/_delta_result_envelope.json", encoding="utf-8")) raw = json.loads(env["data"]["content"]) # Reduce this to the minimal fields the user cares about, to avoid truncation in the host's read_file preview concise = { "exit_code": raw.get("exit_code"), "finished": raw.get("finished"), "summary": {}, "error": raw.get("error"), } # Prefer the standalone JSON object at the end of stdout as the summary stdout = raw.get("stdout", "") for line in reversed(stdout.strip().splitlines()): s = line.strip() if s.startswith("{") and s.endswith("}"): try: concise["summary"] = json.loads(s) break except Exception: pass # Build result_summary as a one-line string and use it directly in the final RESULT: line parts = [f"exit_code={concise['exit_code']}", f"finished={concise['finished']}"] for key, value in concise["summary"].items(): # Avoid putting long text in result_summary if isinstance(value, (str, int, float, bool)) and len(str(value)) < 200: parts.append(f"{key}={value}") concise["result_summary"] = ", ".join(parts) ws = os.environ.get("WORKSPACE_ROOT", ".") with open(os.path.join(ws, "result.json"), "w", encoding="utf-8") as f: json.dump(concise, f, indent=2, ensure_ascii=False) print(concise["result_summary"]) ``` 3. If `python3` is not available in the current environment, use `jq`, `node` or any JSON-capable tool to implement **the same logic**: 1) read `/tmp/_delta_result_envelope.json`; 2) unwrap the CLI envelope and get `data.content`; 3) parse the inner JSON; 4) extract the summary JSON from the end of `stdout`; 5) build `result_summary` and write `result.json`. - If `stdout` is very large (training or download logs, etc.), do not put all of it in `result.json`; keep only key metrics and a preview. - The complete raw output remains in the sandbox's `log_file` and can be retrieved again with `sandbox read` if needed. - The purpose of writing a local copy is to let the host's deliverable check find an actual `.json` file. Since the check uses OR semantics, a `.json` file takes precedence over a `.cuda` file commitment mistakenly inferred from tokens such as `torch.cuda`. 6. **Destroy the sandbox**: `delta-cli sandbox kill ` (if you need to save results, use `sandbox finish --results '{...}'` instead of kill; finish destroys the sandbox automatically) For the detailed steps, see [lifecycle.md](references/lifecycle.md). ## Tunnelling (port exposure) Expose a service port inside the sandbox to the public internet so the user can access it directly through `public_url`. How it works: inside the sandbox, `frpc` opens an outbound connection to the public `frps` server, which forwards TCP traffic from a randomly assigned public port (20000–29999) back to the service in the sandbox. **No per-sandbox port configuration is needed on the host** (this uses a reverse connection). Available commands: | Command | Purpose | |------|------| | `sandbox expose [--port N]` | Expose a port and return `public_url`; `--port` defaults to **8700** (platform convention), so you can omit it | | `sandbox exposes ` | List the tunnels currently active for this sandbox (service_port / remote_port / public_url). `data.sandbox_exists=false` means the sandbox is no longer running — do not keep polling, create a new one | | `sandbox dispose --port N` | Explicitly reclaim one tunnel (stop frpc and release the public port). Idempotent when the sandbox is already destroyed (`data.already_gone:true`, `data.error_code:sandbox_already_gone`) | Follow this order and these conventions: 1. **Start the service before exposing it**. Before running `expose`, the service must be listening inside the sandbox (bound to `0.0.0.0`, not only `127.0.0.1`), and the server checks `127.0.0.1:{port}` for health — **any HTTP response (including 4xx/5xx) counts as alive**; only a connection failure or timeout reports “service not ready”. 2. **Keep long-running services in the foreground**: with `run-bg`, make the service process the task's foreground process, **Do not** background it with `nohup ... &` (the background process is reclaimed when the task ends, so the service disappears): ```bash delta-cli sandbox run-bg --command "cd && exec python3 -m http.server 8700 --bind 0.0.0.0" --timeout 3600 delta-cli sandbox expose # --port defaults to 8700; no need to pass it ``` 3. **Tunnels are reclaimed automatically with the sandbox lifecycle**: when the sandbox is killed, a server hook marks the tunnel record as closed, and frps automatically releases the public port when the frpc connection closes; use `dispose` to reclaim a tunnel without destroying the sandbox. **Exposed ports are fully open to the public internet, so the service itself must require authentication**; do not leave it unprotected. 4. If the public URL is unreachable, troubleshoot in this order: (1) check whether the service is listening inside the sandbox (`curl 127.0.0.1:{port}`; you can test first with `sandbox run`); (2) check whether the `frpc` process is alive (`ps aux | grep frpc`); (3) check connectivity to the public `frps` host; (4) check whether the security group allows the service port range; (5) confirm that the frpc and frps tokens match. ## General structured-output convention This convention applies to **all** `delta-sandbox` tasks (GPU/CPU, training/inference/compilation/data processing, etc.). Its purpose is to let the Runner read the result directly from `result.json` without parsing long logs again. ### What sandbox scripts/commands should output After the command finishes, it is recommended to print a standalone line of **structured JSON** at the end of `stdout`. For example: - CUDA check: `{"status":"ok","torch_version":"2.5.1+cu121","cuda_available":true,"device_name":"NVIDIA H100 80GB HBM3"}` - Training task: `{"status":"ok","epochs":10,"final_loss":0.12,"final_accuracy":0.94,"model_file":"/model.pt"}` - Data processing: `{"status":"ok","input_rows":10000,"output_rows":9876,"output_file":"/output.csv"}` This lets the skill extract the result into the `summary` field of `result.json` with the same code, then produce the `RESULT:` line. - **Do not** print redundant marker lines such as `FINAL_ANSWER: ...` to stdout; they may cause the Runner to mistake ordinary text for a result field. - Do **not** use schema-duplicate keys such as `final_answer` in `summary`. Use fields that describe the result, for example `generated_text`/`output`/`result` for inference, `final_loss`/`final_accuracy` for training, and `output_file`/`output_rows` for data processing. ### General local `result.json` schema ```json { "exit_code": 0, "finished": true, "summary": {}, "result_summary": "exit_code=0, status=ok, torch_version=..., ...", "error": null } ``` Key fields: - `summary`: the structured result object printed by the command. - `result_summary`: a **one-line string** matching the final answer's `RESULT:` line, so the Runner can read it directly. - `error`: error information when the command fails; usually `null` on success. - **Do not include the full `stdout`**. The complete log remains in the sandbox's `log_file`. ## Reading output and final-answer format 1. **On the SSE path, the `complete` frame from `sandbox run` / `run-bg --wait` includes `result_summary`** (injected by the CLI when it detects trailing JSON) and `stdout` / `stderr` / `log_file` (filled in by the CLI as a fallback). Read `exit_code` / `log_file` / `execution_id` from the complete frame. On command failure, an `error.type: command_failed` envelope is appended after the complete frame (process exit code 8). The `log_file` path can also be obtained from a `sandbox logs --execution-id ` snapshot after `finished=true`. 2. **Extract the summary**: prefer reusing `result_summary` from the complete frame directly. If it is missing, read the run-envelope log file with `sandbox read --path `, parse it and scan backwards through trailing stdout JSON to produce `result_summary` (see step 5 in “Full lifecycle”). - The run-envelope log file contains the full `stdout`, `stderr`, `exit_code`, `finished`, `command` and `error`. 3. **Create a concise local `result.json`** (see step 5 in “Full lifecycle” and “General structured-output convention”). The local copy should contain only key fields and a summary, **not the full, lengthy `stdout`**. 4. **The final answer must be a single line beginning with `RESULT:`.** - Prefer reusing the `result_summary` field in `result.json` directly: ```text RESULT: ``` - If `result_summary` is missing, build one line from the `summary` object, for example: ```text RESULT: exit_code=0, status=ok, torch_version=2.7.0+cu128, cuda_available=True, device_name=NVIDIA GeForce RTX 3090. ``` 5. **`final_response` must begin with `RESULT:`** - **The first character must be `R`, immediately followed by the `ESULT:` prefix**; the entire response must be a single line, `RESULT: `, with no preceding natural language, blank lines, transition or progress report. - Reason: the host Runner and downstream LLM recognise a `final_response` beginning with `RESULT:` by default, then pass the content after it to the user as the task result. Any preceding text—including neutral status updates such as “Sandbox destroyed”, “Task complete” or “Full lifecycle complete”—makes the Runner treat `final_response` as a process extract rather than the final result. It may then show a failure message such as “No specific result.json content was included” and lose the actual result. - Correct: `RESULT: exit_code=0, cuda_available=True, torch_version=2.5.1+cu121, device_name=NVIDIA H100 80GB HBM3` - Incorrect (prefix contamination; a real example): `Sandbox destroyed successfully. The full lifecycle is complete: create → write → run → generate result.json → destroy.\n\nRESULT: torch_version=2.5.1+cu121, cuda_available=True, ...` — after seeing the “Sandbox destroyed successfully...” prefix, the Runner stopped reading before the later `RESULT:` line, and the user saw “No specific result.json content was included”. - Incorrect: `Sandbox killed successfully. Final answer with result summary. RESULT: ...` - Incorrect: `Sandbox killed. Task done. Need final answer exactly RESULT: line from result_summary. RESULT: ...` - Incorrect: `The user wants me to verify CUDA... RESULT: ...` - Incorrect: `Summary: ... RESULT: ...` - Incorrect: adding extra sentences after `RESULT:`, such as `Let me know if you need anything else.` 6. **Do not rely on `stdout`/`stderr` fields that may be truncated**; key conclusions must come from `summary` / `result_summary` in `result.json`. 7. **Do not call any more tools after `sandbox kill` succeeds**. A kill response of `{ok:true}` means the task is finished; the **next turn's `final_response` must strictly follow rule 5 and begin with `RESULT:`**. Do not put any transition or progress report before `RESULT:`, including “Sandbox destroyed successfully”, “Full lifecycle complete”, “Task complete” or “Sandbox killed”; the Runner does not need these details, and they would push `RESULT:` later in `final_response`, risking the lost-result problem described in rule 5. Do not call any shell, `read_file` or `ls` tool again, and do not start with an internal monologue such as “Need final answer exactly...”. ## Host / Planner call guidance This skill is designed to make the conclusion self-contained: when execution ends, **`result.json`'s `result_summary` and `final_response` already contain the user-facing conclusion**. Therefore: - When the **Planner** calls this skill in a plan step, `required_outputs` must include `.json` (see “Request wording conventions” above). - After receiving the skill result, the **host (host / Runner / finalize stage)** should use `final_response` or `result_summary` in `result.json` directly as the final answer. **Do not ask the model to read `result.json`, verify its contents or reason again.** - If the host is concerned that the content may be untrusted, it may inspect `result.json`, but the **final output must still be a single line beginning with `RESULT:`**. Do not expose reasoning such as “I need to read the file to verify it” or “Wait, I don't have...” to the user. ## Common recovery | Error / symptom | Recovery action | |-------------|----------| | `error.type: not_found` | Check whether `sandbox_id` is correct and whether the sandbox has been killed | | `error.type: network` | Check `base_url` and network connectivity | | `error.type: validation` | Check parameter formats and required flags. This also covers a file path used where a directory is required (or the reverse) and an invalid glob; `error.hint` names what to change | | `error.type: auth / permission` | Follow the permission-denied guidance in delta-shared | | `error.type: api` (5xx) | Server error; check the API service status | | `sandbox create` failed | Read `error.hint` — it states the cause and the next step (for example, the image was not cached on the target node, so the initial pull exceeded the readiness timeout: retry once, or pick a GPU type whose node already has the image). A failed create leaves no sandbox behind, so retrying is safe. `error.request_id` identifies the failure in the server log. On new servers `error.details` adds `stage`/`retryable`/`suggested_actions` (and nested hints such as `details.suggested_gpu_types`) for a machine-readable retry decision. | | `sandbox create` rejected with `TIER_*` / `SUBACCOUNT_*` | Account/plan quota rejection — retrying the same request will not help. `TIER_GPU_TOTAL_EXCEEDED` / `TIER_GPU_MEM_TOTAL_EXCEEDED` / `TIER_CPU_TOTAL_EXCEEDED` / `TIER_MEMORY_TOTAL_EXCEEDED` / `TIER_CONCURRENCY_EXCEEDED` → stop or shrink an existing sandbox, or lower the request; `TIER_GPU_TYPE_NOT_ALLOWED` / `TIER_GPU_PROVIDER_NOT_ALLOWED` / `TIER_GPU_IMAGE_NOT_ALLOWED` → choose an allowed model/provider/image; `SUBACCOUNT_GPU_TYPE_DENIED` → run `delta-cli quota`, read `gpu_access.denied_gpu_types`, choose a model outside that blacklist **and pass `--gpu-type` explicitly** (the denylist overrides the allowlist, so a model listed in `allowed_gpu_types` may still be blocked); `TIER_GPU_DURATION_EXCEEDED` → trial time exhausted, upgrade required. | | `sandbox operation` reports not found | The `request_id` is wrong or the operation has aged out. Run `delta-cli sandbox operation` (no argument) to list recent operations; do not treat it as a missing sandbox. A failed ledger record carries a `diagnostics` summary (`error_code`/`stage`/`hint`) for quick triage. | | Command execution failed (`exit_code ≠ 0`) | Check stdout/stderr returned by `run` | | Command timed out | Increase `--timeout` or use `run-bg` | | `error.type: internal` | Client-side internal error; check the delta-cli version or contact the maintainer |