asyncapi: '2.6.0' id: 'urn:com:retellai:llm-websocket' info: title: Retell AI Custom LLM WebSocket version: '1.0.0' description: | AsyncAPI 2.6 description of Retell AI's real-time **Custom LLM WebSocket** surface, documented at https://docs.retellai.com/api-references/llm-websocket. Unlike a one-way HTTP stream, this is a true bidirectional WebSocket. When an agent whose `response_engine.type` is `custom-llm` starts a call, the Retell server **connects out to your** WebSocket server at `{your-llm-websocket-url}/{call_id}`. Over that single connection Retell pushes transcript/turn-taking events and requests agent responses, and your server pushes back the agent's speech content, tool-call bookkeeping, live agent-config updates, and interrupts. A separate WebRTC/WebSocket audio channel carries the actual call audio and is handled by the Retell Web Client / mobile SDKs (opened with the `access_token` returned by `POST /v2/create-web-call`); that media transport is not modeled at message level here. Direction convention in this document: * Retell server -> your server = the `receive` (publish) messages * your server -> Retell server = the `send` (subscribe) messages contact: name: API Evangelist email: kin@apievangelist.com url: https://apievangelist.com license: name: API documentation - Retell AI Terms of Service url: https://www.retellai.com/terms-of-service x-transport-notes: transport: WebSocket protocol: wss direction: bidirectional connectedBy: 'Retell server dials OUT to your WebSocket server' urlPattern: '{your-llm-websocket-url}/{call_id}' appliesTo: 'Agents with response_engine.type = custom-llm' source: https://docs.retellai.com/api-references/llm-websocket defaultContentType: application/json servers: custom-llm: url: '{yourLlmWebsocketUrl}/{callId}' protocol: wss description: | Your WebSocket endpoint. Retell establishes the connection to this URL, appending the call_id as the final path segment, at the start of every call served by a custom-llm agent. variables: yourLlmWebsocketUrl: description: The base wss:// URL you configured on the custom-llm response engine. default: wss://your-server.example.com/llm-websocket callId: description: Unique id of the call, appended by Retell as the final path segment. default: call_abc123 channels: /llm-websocket/{call_id}: parameters: call_id: description: Unique id of the call. schema: type: string description: | The single bidirectional channel for one call. Retell sends ping_pong / call_details / update_only / response_required / reminder_required events to your server; your server replies with config / ping_pong / response / agent_interrupt / tool_call_invocation / tool_call_result / update_agent / metadata events. publish: operationId: sendToRetell summary: Messages your server sends to Retell. description: | Your server -> Retell. Primarily `response` messages that stream the agent's spoken content back for a given `response_id`, plus optional config, keep-alive, tool bookkeeping, live agent-config updates, and frontend metadata. message: oneOf: - $ref: '#/components/messages/ConfigResponse' - $ref: '#/components/messages/PingPongResponse' - $ref: '#/components/messages/Response' - $ref: '#/components/messages/AgentInterrupt' - $ref: '#/components/messages/ToolCallInvocation' - $ref: '#/components/messages/ToolCallResult' - $ref: '#/components/messages/UpdateAgent' - $ref: '#/components/messages/Metadata' subscribe: operationId: receiveFromRetell summary: Messages Retell sends to your server. description: | Retell -> your server. Keep-alive, initial call details, transcript / turn-taking updates, and requests for a required response or reminder. message: oneOf: - $ref: '#/components/messages/PingPongRequest' - $ref: '#/components/messages/CallDetails' - $ref: '#/components/messages/UpdateOnly' - $ref: '#/components/messages/ResponseRequired' - $ref: '#/components/messages/ReminderRequired' components: messages: PingPongRequest: name: ping_pong title: Ping (Retell -> your server) summary: Keep-alive ping sent by Retell. contentType: application/json payload: $ref: '#/components/schemas/PingPong' examples: - name: ping payload: interaction_type: ping_pong timestamp: 1751328000000 CallDetails: name: call_details title: Call details summary: Initial call metadata, sent once at connection if enabled in config. contentType: application/json payload: $ref: '#/components/schemas/CallDetails' UpdateOnly: name: update_only title: Update only summary: Transcript and turn-taking update that does not require a response. contentType: application/json payload: $ref: '#/components/schemas/UpdateOnly' ResponseRequired: name: response_required title: Response required summary: Retell requests an agent response for the given response_id. contentType: application/json payload: $ref: '#/components/schemas/ResponseRequired' examples: - name: responseRequired payload: interaction_type: response_required response_id: 3 transcript: - role: user content: I would like to check my balance. ReminderRequired: name: reminder_required title: Reminder required summary: Retell requests a reminder utterance after user silence. contentType: application/json payload: $ref: '#/components/schemas/ResponseRequired' ConfigResponse: name: config title: Config summary: Optional initial config your server sends to Retell. contentType: application/json payload: $ref: '#/components/schemas/ConfigResponse' examples: - name: config payload: response_type: config config: auto_reconnect: true call_details: true transcript_with_tool_calls: true PingPongResponse: name: ping_pong title: Pong (your server -> Retell) summary: Keep-alive acknowledgment. contentType: application/json payload: $ref: '#/components/schemas/PingPong' Response: name: response title: Response summary: Streams the agent's spoken content for a response_id. contentType: application/json payload: $ref: '#/components/schemas/Response' examples: - name: response payload: response_type: response response_id: 3 content: 'Sure, let me pull that up for you.' content_complete: false no_interruption_allowed: false - name: finalWithEndCall payload: response_type: response response_id: 3 content: ' Your balance is one hundred dollars. Goodbye!' content_complete: true end_call: true AgentInterrupt: name: agent_interrupt title: Agent interrupt summary: Injects an immediate agent utterance out of the normal response cycle. contentType: application/json payload: $ref: '#/components/schemas/AgentInterrupt' ToolCallInvocation: name: tool_call_invocation title: Tool call invocation summary: Records that the agent invoked a tool (for the transcript/logs). contentType: application/json payload: $ref: '#/components/schemas/ToolCallInvocation' ToolCallResult: name: tool_call_result title: Tool call result summary: Records the result returned from a previously invoked tool. contentType: application/json payload: $ref: '#/components/schemas/ToolCallResult' UpdateAgent: name: update_agent title: Update agent summary: Applies runtime agent-config changes mid-call. contentType: application/json payload: $ref: '#/components/schemas/UpdateAgent' Metadata: name: metadata title: Metadata summary: Arbitrary JSON forwarded to the frontend / web client. contentType: application/json payload: $ref: '#/components/schemas/MetadataMessage' schemas: PingPong: type: object required: - interaction_type - timestamp properties: interaction_type: type: string enum: - ping_pong response_type: type: string enum: - ping_pong timestamp: type: integer description: Unix timestamp in milliseconds. Utterance: type: object properties: role: type: string enum: - agent - user content: type: string CallDetails: type: object required: - interaction_type properties: interaction_type: type: string enum: - call_details call: type: object description: The full call object (call_id, agent_id, from_number, to_number, metadata, dynamic variables, etc.). UpdateOnly: type: object required: - interaction_type properties: interaction_type: type: string enum: - update_only transcript: type: array items: $ref: '#/components/schemas/Utterance' transcript_with_tool_calls: type: array items: type: object turntaking: type: string enum: - agent_turn - user_turn ResponseRequired: type: object required: - interaction_type - response_id properties: interaction_type: type: string enum: - response_required - reminder_required response_id: type: integer description: Echo this id back on your `response` message(s). transcript: type: array items: $ref: '#/components/schemas/Utterance' transcript_with_tool_calls: type: array items: type: object ConfigResponse: type: object required: - response_type properties: response_type: type: string enum: - config config: type: object properties: auto_reconnect: type: boolean call_details: type: boolean transcript_with_tool_calls: type: boolean Response: type: object required: - response_type - response_id properties: response_type: type: string enum: - response response_id: type: integer description: Must match the response_id from the response_required event. content: type: string description: A slice of the agent's spoken content (may be streamed across many messages). content_complete: type: boolean description: True on the final content slice for this response_id. no_interruption_allowed: type: boolean end_call: type: boolean description: Ends the call after this content is spoken. transfer_number: type: string show_transferee_as_caller: type: boolean digit_to_press: type: string AgentInterrupt: type: object required: - response_type - interrupt_id properties: response_type: type: string enum: - agent_interrupt interrupt_id: type: integer content: type: string content_complete: type: boolean no_interruption_allowed: type: boolean end_call: type: boolean transfer_number: type: string digit_to_press: type: string ToolCallInvocation: type: object required: - response_type - tool_call_id - name properties: response_type: type: string enum: - tool_call_invocation tool_call_id: type: string name: type: string arguments: type: string description: JSON-encoded tool arguments. ToolCallResult: type: object required: - response_type - tool_call_id properties: response_type: type: string enum: - tool_call_result tool_call_id: type: string content: type: string UpdateAgent: type: object required: - response_type properties: response_type: type: string enum: - update_agent agent_config: type: object properties: responsiveness: type: number interruption_sensitivity: type: number reminder_trigger_ms: type: integer reminder_max_count: type: integer MetadataMessage: type: object required: - response_type properties: response_type: type: string enum: - metadata metadata: type: object description: Arbitrary JSON forwarded to the frontend.