--- name: agents description: Build voice AI agents with ElevenLabs. Use when creating voice assistants, customer service bots, interactive voice characters, or any real-time voice conversation experience, and when configuring an agent's tools, workflows, or procedures, including creating, editing, and publishing free-form and structured procedure drafts on an agent branch over the SDKs or REST API. license: MIT compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY). metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}} --- # ElevenLabs Agents Platform Build voice AI agents with natural conversations, multiple LLM providers, custom tools, and easy web embedding. > **Setup:** See [Installation Guide](references/installation.md) for CLI and SDK setup. ## Quick Start with CLI The ElevenLabs CLI is the recommended way to create and manage agents: ```bash # Install CLI and authenticate npm install -g @elevenlabs/cli elevenlabs auth login # Initialize project and create an agent elevenlabs agents init elevenlabs agents add "My Assistant" --template complete # Push to ElevenLabs platform elevenlabs agents push ``` **Available templates:** `complete`, `minimal`, `voice-only`, `text-only`, `customer-service`, `assistant` ### Python ```python from elevenlabs import ElevenLabs client = ElevenLabs() agent = client.conversational_ai.agents.create( name="My Assistant", conversation_config={ "agent": { "first_message": "Hello! How can I help?", "language": "en", "prompt": { "prompt": "You are a helpful assistant. Be concise and friendly.", "llm": "gemini-2.0-flash", "temperature": 0.7 } }, "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"} } ) ``` ### JavaScript ```javascript import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js"; const client = new ElevenLabsClient(); const agent = await client.conversationalAi.agents.create({ name: "My Assistant", conversationConfig: { agent: { firstMessage: "Hello! How can I help?", language: "en", prompt: { prompt: "You are a helpful assistant.", llm: "gemini-2.0-flash", temperature: 0.7 } }, tts: { voiceId: "JBFqnCBsd6RMkjVDRZzb" } } }); ``` ### CLI The CLI reads `ELEVENLABS_API_KEY` from the environment automatically: ```bash elevenlabs agents create \ --json '{"name": "My Assistant", "conversation_config": {"agent": {"first_message": "Hello!", "language": "en", "prompt": {"prompt": "You are helpful.", "llm": "gemini-2.0-flash"}}, "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}}}' ``` ## Starting Conversations **Authenticated WebRTC:** Request a session token from your backend. The response includes both the token and the conversation ID: ```python session = client.conversational_ai.conversations.get_webrtc_token( agent_id="your-agent-id", ) print(session.token, session.conversation_id) ``` **Server-side (Python):** Get signed URL for client connection: ```python signed_url = client.conversational_ai.conversations.get_signed_url( agent_id="your-agent-id", environment="staging", ) ``` **Client-side (JavaScript):** ```javascript import { Conversation } from "@elevenlabs/client"; const conversation = await Conversation.startSession({ agentId: "your-agent-id", environment: "staging", overrides: { asr: { keywords: ["ElevenLabs", "TechCorp"] } }, onMessage: (msg) => console.log("Agent:", msg.message), onUserTranscript: (t) => console.log("User:", t.message), onPing: (event) => console.log("Estimated latency:", event.ping_ms), onContextUsage: ({ model, context_tokens, context_limit_tokens }) => console.log(`${model}: ${context_tokens}/${context_limit_tokens} context tokens`), onError: (e) => console.error(e) }); ``` **React Hook:** Wrap hook consumers in `ConversationProvider`. Prefer granular hooks such as `useConversationControls` and `useConversationStatus` for session controls and UI state; `useConversation` remains available as the convenience all-in-one hook. Pass provider-level callbacks such as `onError` when you want React to handle conversation errors in one place. ```typescript import { ConversationProvider, useConversationControls, useConversationStatus, } from "@elevenlabs/react"; function Agent({ signedUrl }: { signedUrl: string }) { const { startSession, endSession } = useConversationControls(); const { status } = useConversationStatus(); if (status === "connected") { return ; } return ( ); } function App({ signedUrl }: { signedUrl: string }) { return ( console.error("Conversation error:", error)} onPing={(event) => console.log("Estimated latency:", event.ping_ms)} onContextUsage={({ model, context_tokens, context_limit_tokens }) => console.log(`${model}: ${context_tokens}/${context_limit_tokens} context tokens`) } > ); } ``` ## Configuration | Provider | Models | |----------|--------| | OpenAI | `gpt-6.1-sol`, `gpt-6-sol`, `gpt-6-luna`, `gpt-6-astra`, `gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna`, `gpt-5.5`, `gpt-5.5-2026-04-23`, `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-5.4-2026-03-05`, `gpt-5.4-mini-2026-03-17`, `gpt-5.4-nano-2026-03-17`, `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `gpt-4.1`, `gpt-4.1-mini`, `gpt-4.1-nano`, `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo` | | Anthropic | `claude-opus-5-5`, `claude-opus-5`, `claude-sonnet-5-5`, `claude-opus-4-7`, `claude-sonnet-4-6`, `claude-sonnet-4-5`, `claude-sonnet-4`, `claude-haiku-4-5`, `claude-3-7-sonnet`, `claude-3-5-sonnet`, `claude-3-haiku` | | Google | `gemini-3.8-flash`, `gemini-3.7-flash`, `gemini-3.6-flash`, `gemini-3.1-flash-lite-preview`, `gemini-3.1-pro-preview`, `gemini-3-pro-preview`, `gemini-3-flash-preview`, `gemini-2.5-flash`, `gemini-2.5-flash-lite`, `gemini-2.0-flash`, `gemini-2.0-flash-lite` | | ElevenLabs | `glm-52`, `deepseek-v41-flash`, `glm-45-air-fp8`, `qwen3-30b-a3b`, `qwen36-35b-a3b`, `qwen35-35b-a3b`, `qwen35-397b-a17b`, `gpt-oss-120b` | | Custom | `custom-llm` (bring your own endpoint) | Use `GET /v1/convai/llm/list` to inspect the current model catalog, including deprecation state, token/context limits, capability flags such as image-input support, and model-specific reasoning effort support. **Popular voices:** `JBFqnCBsd6RMkjVDRZzb` (George), `EXAVITQu4vr4xnSDxMaL` (Sarah), `onwK4e9ZLuTAKqWW03F9` (Daniel), `XB0fDUnXU5powFXDhCwa` (Charlotte) **Turn eagerness:** `patient` (waits longer for user to finish), `normal`, or `eager` (responds quickly) See [Agent Configuration](references/agent-configuration.md) for all options. ## System Prompt Structure Section the prompt with markdown headings — the model prioritizes and interprets instructions more reliably ([prompting guide](https://elevenlabs.io/docs/eleven-agents/best-practices/prompting-guide)): ``` # Personality – named character, 2-3 traits # Environment – where they work, who they talk to # Tone – vocal style as 4-5 bullets # Goal – what success looks like (numbered for multi-step flows) ``` Keep instructions short and action-based. Mark critical steps with "This step is important." For critical refusal/safety rules, include concise instructions in the prompt and also configure independent custom Guardrails via `platform_settings.guardrails` (see [Guardrails](#guardrails)). ## Tools Extend agents with webhook, client, or built-in system tools. Tools are defined inside `conversation_config.agent.prompt`: Workspace environment variables can resolve per-environment server tool URLs, headers, and auth connections, and runtime system variables such as `{{system__conversation_history}}` can pass full conversation context into tool calls when needed. ```python "prompt": { "prompt": "You are a helpful assistant that can check the weather.", "llm": "gemini-2.0-flash", "tools": [ # Webhook: server-side API call {"type": "webhook", "name": "get_weather", "description": "Get weather", "api_schema": {"url": "https://api.example.com/weather", "method": "POST", "request_body_schema": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}}, # Client: runs in the browser {"type": "client", "name": "show_product", "description": "Display a product", "parameters": {"type": "object", "properties": {"productId": {"type": "string"}}, "required": ["productId"]}} ], "built_in_tools": { "end_call": {}, "transfer_to_number": {"transfers": [{"transfer_destination": {"type": "phone", "phone_number": "+1234567890"}, "condition": "User asks for human support"}]} } } ``` **Client tools** run in browser: ```javascript clientTools: { show_product: async ({ productId }) => { document.getElementById("product").src = `/products/${productId}`; return { success: true }; } } ``` See [Client Tools Reference](references/client-tools.md) for complete documentation. ### Built-in System Tools Set under `conversation_config.agent.prompt.built_in_tools`. `{}` enables defaults; provide `description` to customize; omit to disable. | Tool | Enable for | |------|------------| | `end_call` | All agents | | `language_detection` | Multilingual agents | | `transfer_to_number` | Phone-based human escalation | | `transfer_to_agent` | Multi-agent workflows | | `skip_turn` | Tutoring / coaching (silent listening) | | `voicemail_detection` | Outbound calling | | `play_keypad_touch_tone` | IVR navigation | `start_procedure` and `end_procedure` are not configured here. The platform adds them automatically whenever the agent has at least one procedure (see [Procedures](#procedures)). `run_subagent` is a system tool for delegating a task to another configured agent. Add it to `conversation_config.agent.prompt.tools` with `params.system_tool_type: "run_subagent"` and an `agents` array. Each entry requires `agent_id` and `description`; `branch_id` and a JSON-schema `parameters` object are optional. `knowledge_base` is a system tool for letting the model choose how to inspect attached knowledge. Add it to `conversation_config.agent.prompt.tools` with `type: "system"`, a `name`, and `params.system_tool_type: "knowledge_base"`. Use `enabled_strategies` to expose any combination of `cat`, `keyword`, `semantic`, and `ls`: ```json { "type": "system", "name": "knowledge_base", "description": "Search the attached knowledge base.", "params": { "system_tool_type": "knowledge_base", "enabled_strategies": ["semantic", "keyword"] } } ``` ### Integration Tools Pre-built connectors managed by the platform. Create a connection with credentials, then attach via `tool_ids`: | Integration | Use case | |-------------|----------| | `calcom` | Scheduling appointments | | `salesforce` | CRM lookups, case creation | | `hubspot` | CRM, marketing, contacts | | `zendesk` | Support ticketing | Three-step flow: `POST /v1/convai/api-integrations/{id}/connections` → `GET /v1/convai/api-integrations/{id}/tools` → `POST /v1/convai/tools` with `api_integration_id` and `api_integration_connection_id`. Attach to the agent with `"prompt": {"tool_ids": ["tool_xxxx"]}`. Inline `tools` and `tool_ids` can coexist — prefer an integration over a duplicate custom webhook. ### Public-API Webhook Examples No-auth APIs useful for prototypes (URLs must be HTTPS): | Tool | URL | Purpose | |------|-----|---------| | `get_weather` | `https://wttr.in/{location}?format=j1` | Current weather | | `search_wikipedia` | `https://en.wikipedia.org/api/rest_v1/page/summary/{topic}` | Topic summary | | `get_exchange_rate` | `https://open.er-api.com/v6/latest/{base_currency}` | FX rates | ## Workflows Route conversations through discrete steps with branching logic. Define under the agent's top-level `workflow` field. Reference: [Agent Workflows](https://elevenlabs.io/docs/eleven-agents/customization/agent-workflows). **Node types:** `start` (ID must be `"start_node"`), `end`, `override_agent` (subagent step with `label` + `additional_prompt`), `dispatch_tool` (executes a tool with success/failure routing), `agent_transfer`, `transfer_to_number`. **Edge types:** `unconditional`, `llm` (natural-language condition), `expression` (deterministic data check). Tool nodes have separate success/failure edges. **Scope tools per step** with `additional_tool_ids` on a node — prevents the wrong tool firing at the wrong step. Set `additional_tool_ids: []` on conversational routing nodes such as greeting and `classify_intent` so they only converse: ```json { "type": "override_agent", "label": "Book Appointment", "additional_prompt": "Discuss preferred dates and doctors. Show the booking form once agreed.", "entry_behavior": "wait_for_user", "additional_tool_ids": ["show_booking_form", "display_appointment_card"], "position": {"x": 0, "y": 400} } ``` Include `position` (`{x, y}`) on every node so the editor renders cleanly. Start at `y=0`, put `end` at the bottom, and space branches horizontally at `x=-150` and `x=150`; suggested spacing is 200px vertical between levels and 300px horizontal between branches. Keep workflows to 4-7 nodes and always have a path to `end`. Use `entry_behavior` on `override_agent` nodes to choose whether a sub-agent speaks immediately (`generate_immediately`), waits for user input (`wait_for_user`), or lets the platform decide (`auto`). For nested agent transfers, set `enable_nesting` on a `standalone_agent` node and `return_when_nested` on an `end` node that should return control to the parent workflow. ## Procedures Reusable instruction blocks an agent runs when a trigger matches. A procedure is `free_form` (markdown guidance the agent adapts, and the only type that can reference knowledge base documents) or `deterministic` (called "structured" in the dashboard: typed steps that run in a fixed order, for flows that must happen the same way every time). See [Using the Procedure API](references/using-procedure-api.md) for the full CLI and SDK flow, and [Writing Procedures](references/writing-procedures.md) for the step reference, validation rules, and authoring guidance. Procedures live on an agent branch, and every write stages a per-user draft: | Operation | Call | |-----------|------| | List, create, read, update, discard, remove | `/v1/convai/agents/{agent_id}/branches/{branch_id}/procedures...` (`procedures.*` and `procedures.drafts.*` in the SDKs) | | Publish | `PATCH /v1/convai/agents/{agent_id}?branch_id=...` (`agents.update`) | Semantics worth knowing before writing any of these calls: - Nothing reaches the live agent until you publish. Publishing is not a procedure endpoint; one PATCH on the agent versions every changed procedure draft on the branch. - `GET .../procedures/{procedure_id}` reads branch HEAD and returns `404` until that procedure's first publish. Read the `/draft` variant to see a procedure you just created; do not retry the create. - Turning a structured procedure's steps into the form the agent executes is called compiling. When you publish, the platform validates every structured procedure on the branch, compiles them, and stores the result with the new version. You do not compile anything yourself and do not send a `workflow` in the request; `agents.update` with `branch_id` publishes, and compilation happens as part of that. The compiled result is currently visible as read-only nodes in the dashboard's Workflow tab. A failed publish writes nothing, so publishing is also the validation step. To validate without publishing, save an agent draft with `POST /v1/convai/agents/{agent_id}/drafts?branch_id=...`, sending the agent's current `name`, `conversation_config`, `platform_settings`, and `workflow` unchanged, then discard that agent draft with `DELETE` on the same path; see [Using the Procedure API](references/using-procedure-api.md#validate-without-publishing). - If a structured procedure is invalid, the publish or draft save returns `400` with `status` `procedure_validation_failed` and `errors` keyed by procedure ID, each entry carrying the `path` of the offending field and a message. Nothing is written; repair the procedure draft and publish again. - A draft update replaces the whole body. Read the draft first, then resend `name`, `type`, and `trigger` alongside the new `content`. `type` cannot change after creation. - `content` is markdown for a `free_form` procedure, and a JSON-encoded object with a `steps` array for a `deterministic` one. The trigger is the top-level `trigger` field in both cases, not part of `content`. Serialize it; do not hand-escape quotes. - A third `type`, `folder`, groups procedures in the dashboard. Folders carry no `content` or `trigger`; move a procedure into one with `POST .../procedures/{procedure_id}/move`. - The agent keeps the five most recently started procedures in context. When more have been started, the content, inline tools, and knowledge base documents of the oldest free-form ones drop out of the prompt, although the procedures remain active. Keep procedures focused and use sub-procedures so that few are active at once. - Routing is driven by the `trigger` text, not the procedure name. Write concrete, non-overlapping triggers that cover the phrasings a user would actually say. The model sees only a numbered menu of triggers; it never sees procedure names or IDs. - To restrict the starting agent to selected procedures for one conversation, enable `platform_settings.overrides.enable_procedure_ids_from_client`, then pass their IDs as `procedure_ids` in conversation initiation data. An empty list disables all procedures for that starting agent. - Procedure APIs require `elevenlabs` (Python) or `@elevenlabs/elevenlabs-js` at `2.60.0` or newer. ## Guardrails Layered safety enforcement that runs independently of the LLM — configured under `platform_settings.guardrails`, not in the system prompt. Reference: [Guardrails](https://elevenlabs.io/docs/eleven-agents/best-practices/guardrails). ```json "platform_settings": { "guardrails": { "version": "1", "focus": {"is_enabled": true}, "prompt_injection": {"is_enabled": true}, "content": {"config": {"harassment": {"is_enabled": true, "threshold": 0.5}}}, "custom": { "config": { "configs": [{ "is_enabled": true, "name": "No medical diagnoses", "prompt": "Block the agent from providing medical diagnoses or treatment advice.", "execution_mode": "blocking", "model": "gemini-2.5-flash-lite", "history_message_count": 1, "trigger_action": {"type": "retry", "feedback": "Reason: {{trigger_reason}}"} }] } } } } ``` **Types:** `focus` (on-topic), `prompt_injection` (manipulation defense), `content` (category filters), `custom` (LLM-evaluated domain rules). Content categories include `harassment`, `profanity`, `sexual`, `violence`, `self_harm`, and `medical_and_legal_information` — threshold range `0.0`–`1.0` (default `0.3`). Custom rules use `execution_mode: "blocking"` with a `model`, `history_message_count`, and `trigger_action` (e.g., `retry` with feedback). Custom guardrails evaluate in parallel and fail-open. **Per vertical:** healthcare/finance/legal → enable `medical_and_legal_information`; education/youth → `sexual`/`violence`/`self_harm`/`profanity`; support/sales → `harassment`/`profanity`. All agents benefit from `focus` + `prompt_injection` + 2-4 custom rules. ## Testing Agents Three test types via `POST /v1/convai/agent-testing/create`, then attached with PATCH on the agent. Reference: [Agent Testing](https://elevenlabs.io/docs/eleven-agents/customization/agent-testing). | Type | Purpose | |------|---------| | `llm` | Scenario test — does the agent respond appropriately to a message? | | `tool` | Tool-call test — right tool, right parameters? | | `simulation` | Multi-turn flow with a simulated user persona | ```json // Tool-call test (snake_case throughout; chat_history role is "user" or "agent") { "name": "Books with correct doctor and date", "type": "tool", "chat_history": [ {"role": "user", "message": "Dr. Smith on March 5 at 2pm", "time_in_call_secs": 10} ], "tool_call_parameters": { "referenced_tool": {"id": "show_booking_form", "type": "client"}, "parameters": [ {"path": "doctor_name", "eval": {"type": "llm", "description": "Should reference Dr. Smith"}}, {"path": "date", "eval": {"type": "regex", "pattern": "2025-03-05|March 5"}} ] } } ``` Eval strategies: `exact`, `regex`, `llm`. Prompt evaluation criteria can use binary scoring or numeric scoring with `scoring_mode: "numeric_uniform"`, `max_score`, and `score_instructions`; numeric scores are normalized into the aggregate conversation success percentage. Attach via an agent update: ```bash elevenlabs agents update --agent-id "your-agent-id" \ --json '{"platform_settings": {"testing": {"attached_tests": [{"test_id": "test_xxxx"}]}}}' ``` Run selected tests with `POST /v1/convai/agents/{agent_id}/run-tests`. The request body requires `tests` and accepts `repeat_count` from `1` to `50` for repeated runs. Simulation tests can define up to 30 `success_conditions` prompts; all criteria are evaluated and merged into the final result. Simulation tests can also define `tool_mock_overrides`, keyed by tool ID, to replace shared response mocks for one test. Each override is an array of mocks with a required `mock_result`; set `is_error: true` to exercise a tool-failure path. Overrides only apply to tools enabled for mocking through `tool_mock_config`. For completed conversations, rerun one evaluation criterion with `POST /v1/convai/conversations/{conversation_id}/analysis/evaluations/run` and a request body containing `evaluation_id`. ## Widget Embedding ```html ``` Customize with attributes: `avatar-image-url`, `action-text`, `start-call-text`, `end-call-text`. See [Widget Embedding Reference](references/widget-embedding.md) for all options. ## Outbound Calls Make outbound phone calls using your agent via Twilio or Exotel integration: The examples below use Twilio. See the reference for Exotel usage. ### Python ```python response = client.conversational_ai.twilio.outbound_call( agent_id="your-agent-id", agent_phone_number_id="your-phone-number-id", to_number="+1234567890", call_recording_enabled=True ) print(f"Call initiated: {response.conversation_id}") ``` ### JavaScript ```javascript const response = await client.conversationalAi.twilio.outboundCall({ agentId: "your-agent-id", agentPhoneNumberId: "your-phone-number-id", toNumber: "+1234567890", callRecordingEnabled: true, }); ``` ### CLI ```bash elevenlabs agents twilio outbound_call \ --agent-id "your-agent-id" \ --agent-phone-number-id "your-phone-number-id" \ --to-number "+1234567890" \ --call-recording-enabled true ``` See [Outbound Calls Reference](references/outbound-calls.md) for provider-specific endpoints, configuration overrides, and dynamic variables. ## Managing Agents ### Using CLI (Recommended) ```bash # List agents and check status elevenlabs agents list elevenlabs agents status # Import agents from platform to local config elevenlabs agents pull # Import all agents elevenlabs agents pull --agent # Import specific agent # Push local changes to platform elevenlabs agents push # Upload configurations elevenlabs agents push --dry-run # Preview changes first # Add tools elevenlabs tools add-webhook "Weather API" elevenlabs tools add-client "UI Tool" ``` ### Project Structure The CLI creates a project structure for managing agents: ``` your_project/ ├── agents.json # Agent definitions ├── tools.json # Tool configurations ├── tests.json # Test configurations ├── agent_configs/ # Individual agent configs ├── tool_configs/ # Individual tool configs └── test_configs/ # Individual test configs ``` ### SDK Examples ```python # List agents = client.conversational_ai.agents.list() # Get agent = client.conversational_ai.agents.get(agent_id="your-agent-id") # Update (partial - only include fields to change) client.conversational_ai.agents.update(agent_id="your-agent-id", name="New Name") client.conversational_ai.agents.update(agent_id="your-agent-id", conversation_config={ "agent": {"prompt": {"prompt": "New instructions", "llm": "claude-sonnet-4"}} }) # Delete client.conversational_ai.agents.delete(agent_id="your-agent-id") ``` See [Agent Configuration](references/agent-configuration.md) for all configuration options and SDK examples. ## Error Handling ```python try: agent = client.conversational_ai.agents.create(...) except Exception as e: print(f"API error: {e}") ``` Common errors: **401** (invalid key), **404** (not found), **422** (invalid config), **429** (rate limit) ## References - [Installation Guide](references/installation.md) - SDK setup and migration - [Agent Configuration](references/agent-configuration.md) - All config options and CRUD examples - [Client Tools](references/client-tools.md) - Webhook, client, and system tools - [Using the Procedure API](references/using-procedure-api.md) - Procedure CLI and SDK flow, drafts and publish - [Writing Procedures](references/writing-procedures.md) - Trigger and content authoring, structured step reference and rules - [Widget Embedding](references/widget-embedding.md) - Website integration - [Outbound Calls](references/outbound-calls.md) - Phone call integrations