openapi: 3.2.0 info: title: Elasticsearch Request & Response Specification Inference API license: name: Apache 2.0 url: https://github.com/elastic/elasticsearch-specification/blob/main/LICENSE version: '' tags: - name: inference paths: /_inference/chat_completion/{inference_id}/_stream: post: tags: - inference summary: Perform chat completion inference on the service description: 'The chat completion inference API enables real-time responses for chat completion tasks by delivering answers incrementally, reducing response times during computation. It only works with the `chat_completion` task type. NOTE: The `chat_completion` task type is only available within the _stream API and only supports streaming. The Chat completion inference API and the Stream inference API differ in their response structure and capabilities. The Chat completion inference API provides more comprehensive customization options through more fields and function calling support. To determine whether a given inference service supports this task type, please see the page for that service.' operationId: inference-chat-completion-unified parameters: - in: path name: inference_id description: The inference Id required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference request to complete. deprecated: false schema: default: 120s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: $ref: '#/components/schemas/inference._types.RequestChatCompletion' examples: PostChatCompletionRequestExample1: summary: A chat completion task description: Run `POST _inference/chat_completion/openai-completion/_stream` to perform a chat completion on the example question with streaming. value: "{\n \"model\": \"gpt-4o\",\n \"messages\": [\n {\n \"role\": \"user\",\n \"content\": \"What is Elastic?\"\n }\n ]\n}" PostChatCompletionRequestExample2: summary: A chat completion task with tool_calls description: Run `POST _inference/chat_completion/openai-completion/_stream` to perform a chat completion using an Assistant message with `tool_calls`. value: "{\n \"messages\": [\n {\n \"role\": \"assistant\",\n \"content\": \"Let's find out what the weather is\",\n \"tool_calls\": [ \n {\n \"id\": \"call_KcAjWtAww20AihPHphUh46Gd\",\n \"type\": \"function\",\n \"function\": {\n \"name\": \"get_current_weather\",\n \"arguments\": \"{\\\"location\\\":\\\"Boston, MA\\\"}\"\n }\n }\n ]\n },\n { \n \"role\": \"tool\",\n \"content\": \"The weather is cold\",\n \"tool_call_id\": \"call_KcAjWtAww20AihPHphUh46Gd\"\n }\n ]\n}" PostChatCompletionRequestExample3: summary: A chat completion task with tools and tool_calls description: Run `POST _inference/chat_completion/openai-completion/_stream` to perform a chat completion using a User message with `tools` and `tool_choice`. value: "{\n \"messages\": [\n {\n \"role\": \"user\",\n \"content\": [\n {\n \"type\": \"text\",\n \"text\": \"What's the price of a scarf?\"\n }\n ]\n }\n ],\n \"tools\": [\n {\n \"type\": \"function\",\n \"function\": {\n \"name\": \"get_current_price\",\n \"description\": \"Get the current price of a item\",\n \"parameters\": {\n \"type\": \"object\",\n \"properties\": {\n \"item\": {\n \"id\": \"123\"\n }\n }\n }\n }\n }\n ],\n \"tool_choice\": {\n \"type\": \"function\",\n \"function\": {\n \"name\": \"get_current_price\"\n }\n }\n}" PostChatCompletionRequestExample4: summary: A chat completion task with `effort` parameter defined reasoning description: Run `POST _inference/chat_completion/reasoning-chat-completion/_stream` to perform a chat completion task, using both `effort` parameter based reasoning configuration and including reasoning generated by the model on previous step. value: "{\n \"messages\": [{\n \"role\": \"user\",\n \"content\": [{\n \"type\": \"text\",\n \"text\": \"Barber shaves all those, who do not shave themselves. Who shaves the barber?\"\n }\n ]\n }, {\n \"role\": \"assistant\",\n \"content\": [{\n \"type\": \"text\",\n \"text\": \"This is the barber paradox. Such a barber cannot logically exist.\"\n }\n ],\n \"reasoning\": \"If the barber shaves himself, he should not; if he does not, he should.\",\n \"reasoning_details\": [{\n \"type\": \"reasoning.encrypted\",\n \"data\": \"[REDACTED]\"\n }, {\n \"type\": \"reasoning.summary\",\n \"summary\": \"Barber shaving himself creates contradiction\"\n }, {\n \"type\": \"reasoning.text\",\n \"text\": \"If the barber shaves himself, he should not; if he does not, he should.\",\n \"signature\": \"sig_123\"\n }\n ]\n }, {\n \"role\": \"user\",\n \"content\": [{\n \"type\": \"text\",\n \"text\": \"What if there are 2 barbers?\"\n }\n ]\n }\n ],\n \"reasoning\": {\n \"effort\": \"high\",\n \"summary\": \"detailed\",\n \"exclude\": false\n }\n}" PostChatCompletionRequestExample5: summary: A chat completion task with `enabled` parameter defined reasoning description: Run `POST _inference/chat_completion/reasoning-chat-completion/_stream` to perform a chat completion task, using both `enabled` parameter based reasoning configuration and including reasoning generated by the model on previous step. value: "{\n \"messages\": [{\n \"role\": \"user\",\n \"content\": [{\n \"type\": \"text\",\n \"text\": \"Barber shaves all those, who do not shave themselves. Who shaves the barber?\"\n }\n ]\n }, {\n \"role\": \"assistant\",\n \"content\": [{\n \"type\": \"text\",\n \"text\": \"This is the barber paradox. Such a barber cannot logically exist.\"\n }\n ],\n \"reasoning\": \"If the barber shaves himself, he should not; if he does not, he should.\",\n \"reasoning_details\": [{\n \"type\": \"reasoning.encrypted\",\n \"data\": \"[REDACTED]\"\n }, {\n \"type\": \"reasoning.summary\",\n \"summary\": \"Barber shaving himself creates contradiction\"\n }, {\n \"type\": \"reasoning.text\",\n \"text\": \"If the barber shaves himself, he should not; if he does not, he should.\",\n \"signature\": \"sig_123\"\n }\n ]\n }, {\n \"role\": \"user\",\n \"content\": [{\n \"type\": \"text\",\n \"text\": \"What if there are 2 barbers?\"\n }\n ]\n }\n ],\n \"reasoning\": {\n \"enabled\": true,\n \"summary\": \"detailed\",\n \"exclude\": false\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/_types.StreamResult' examples: PostChatCompletionResponseExample1: description: A successful response when performing a chat completion task using a User message with `tools` and `tool_choice`. value: "event: message\ndata: {\"chat_completion\":{\"id\":\"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3\",\"choices\":[{\"delta\":{\"content\":\"\",\"role\":\"assistant\"},\"index\":0}],\"model\":\"gpt-4o-2024-08-06\",\"object\":\"chat.completion.chunk\"}}\n\nevent: message\ndata: {\"chat_completion\":{\"id\":\"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3\",\"choices\":[{\"delta\":{\"content\":Elastic\"},\"index\":0}],\"model\":\"gpt-4o-2024-08-06\",\"object\":\"chat.completion.chunk\"}}\n\nevent: message\ndata: {\"chat_completion\":{\"id\":\"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3\",\"choices\":[{\"delta\":{\"content\":\" is\"},\"index\":0}],\"model\":\"gpt-4o-2024-08-06\",\"object\":\"chat.completion.chunk\"}}\n\n(...)\n\nevent: message\ndata: {\"chat_completion\":{\"id\":\"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3\",\"choices\":[],\"model\":\"gpt-4o-2024-08-06\",\"object\":\"chat.completion.chunk\",\"usage\":{\"completion_tokens\":28,\"prompt_tokens\":16,\"total_tokens\":44}}} \n\nevent: message\ndata: [DONE]" PostChatCompletionResponseExample2: description: A successful response when performing a chat completion task with response-level reasoning data. value: 'event: message data: {"chat_completion":{"id":"chatcmpl-910TWsy2VPnSfBbv5UztnSdYUJA10","choices":[{"delta":{"content":"With two barbers, the paradox disappears.","role":"assistant"},"index":0,"reasoning":"The contradiction only occurs when a barber must determine whether to shave himself.","reasoning_details":[{"type":"reasoning.encrypted","data":"[REDACTED]"},{"type":"reasoning.summary","summary":"Two barbers can shave each other."},{"type":"reasoning.text","text":"The contradiction only occurs when a barber must determine whether to shave himself.","signature":"sig_example"}]}],"model":"openai-gpt-oss-120b","object":"chat.completion.chunk"}} event: message data: {"chat_completion":{"id":"chatcmpl-910TWsy2VPnSfBbv5UztnSdYUJA10","choices":[{"delta":{"summary":"Each barber can shave the other, so neither needs to shave himself.","role":"assistant"},"index":0,"reasoning":"With two barbers, they can shave each other.","reasoning_details":[{"type":"reasoning.encrypted","data":"[REDACTED]"},{"type":"reasoning.summary","summary":"avoiding the self-reference paradox"},{"type":"reasoning.text","format":"some_text_reasoning_detail_format","text":"With two barbers, they can shave each other.","signature":"sig_example"}]}],"model":"openai-gpt-oss-120b","object":"chat.completion.chunk"}} (...) event: message data: {"chat_completion":{"id":"chatcmpl-910TWsy2VPnSfBbv5UztnSdYUJA10","choices":[],"model":"openai-gpt-oss-120b","object":"chat.completion.chunk","usage":{"completion_tokens":28,"prompt_tokens":16,"total_tokens":44,"completion_tokens_details":{"reasoning_tokens":10}}}} event: message data: [DONE]' x-state: Generally available; Added in 8.18.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/completion/{inference_id}: post: tags: - inference summary: Perform completion inference on the service description: 'Get responses for completion tasks. This API works only with the completion task type. IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs. This API requires the `monitor_inference` cluster privilege (the built-in `inference_admin` and `inference_user` roles grant this privilege).' operationId: inference-completion parameters: - in: path name: inference_id description: The inference Id required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference request to complete. deprecated: false schema: default: 120s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: input: description: 'Inference input. Either a string or an array of strings.' oneOf: - type: string - type: array items: type: string task_settings: description: Task settings for the individual inference request. These settings are specific to the you specified and override the task settings specified when initializing the service. allOf: - $ref: '#/components/schemas/inference._types.TaskSettings' required: - input examples: CompletionRequestExample1: summary: Completion task description: Run `POST _inference/completion/openai_completions` to perform a completion on the example question. value: "{\n \"input\": \"What is Elastic?\"\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.CompletionInferenceResult' examples: CompletionResponseExample1: summary: Completion task description: 'A successful response from `POST _inference/completion/openai_completions`. ' value: "{\n \"completion\": [\n {\n \"result\": \"Elastic is a company that provides a range of software solutions for search, logging, security, and analytics. Their flagship product is Elasticsearch, an open-source, distributed search engine that allows users to search, analyze, and visualize large volumes of data in real-time. Elastic also offers products such as Kibana, a data visualization tool, and Logstash, a log management and pipeline tool, as well as various other tools and solutions for data analysis and management.\"\n }\n ]\n}" x-state: Generally available; Added in 8.11.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{inference_id}: get: tags: - inference summary: Get an inference endpoint description: This API requires the `monitor_inference` cluster privilege (the built-in `inference_admin` and `inference_user` roles grant this privilege). operationId: inference-get-1 parameters: - $ref: '#/components/parameters/inference.get-inference_id' responses: '200': $ref: '#/components/responses/inference.get-200' x-state: Generally available; Added in 8.11.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name put: tags: - inference summary: Create an inference endpoint description: 'IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Mistral, Azure OpenAI, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs. The following integrations are available through the inference API. You can find the available task types next to the integration name: * AI21 (`chat_completion`, `completion`) * AlibabaCloud AI Search (`completion`, `rerank`, `sparse_embedding`, `text_embedding`) * Amazon Bedrock (`chat_completion`, `completion`, `text_embedding`) * Amazon SageMaker (`chat_completion`, `completion`, `rerank`, `sparse_embedding`, `text_embedding`) * Anthropic (`chat_completion`, `completion`) * Azure AI Studio (`completion`, `rerank`, `text_embedding`) * Azure OpenAI (`chat_completion`, `completion`, `text_embedding`) * Cohere (`completion`, `rerank`, `text_embedding`) * DeepSeek (`chat_completion`, `completion`) * Elasticsearch (`rerank`, `sparse_embedding`, `text_embedding` - this service is for built-in models and models uploaded through Eland) * ELSER (`sparse_embedding`) * Fireworks AI (`chat_completion`, `completion`, `text_embedding`) * Google AI Studio (`completion`, `text_embedding`) * Google Vertex AI (`chat_completion`, `completion`, `rerank`, `text_embedding`) * Groq (`chat_completion`) * Hugging Face (`chat_completion`, `completion`, `rerank`, `text_embedding`) * JinaAI (`embedding`, `rerank`, `text_embedding`) * Llama (`chat_completion`, `completion`, `text_embedding`) * Mistral (`chat_completion`, `completion`, `text_embedding`) * Nvidia (`chat_completion`, `completion`, `text_embedding`, `rerank`) * OpenAI (`chat_completion`, `completion`, `text_embedding`) * OpenShift AI (`chat_completion`, `completion`, `rerank`, `text_embedding`) * VoyageAI (`rerank`, `text_embedding`) * Watsonx (`chat_completion`, `completion`, `rerank`, `text_embedding`) ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put parameters: - $ref: '#/components/parameters/inference.put-inference_id' - $ref: '#/components/parameters/inference.put-timeout' requestBody: $ref: '#/components/requestBodies/inference.put' responses: '200': $ref: '#/components/responses/inference.put-200' x-state: Generally available; Added in 8.11.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name post: tags: - inference summary: Perform inference on the service description: 'This API enables you to use machine learning models to perform specific tasks on data that you provide as an input. It returns a response with the results of the tasks. The inference endpoint you use can perform one specific task that has been defined when the endpoint was created with the create inference API. For details about using this API with a service, such as Amazon Bedrock, Anthropic, or HuggingFace, refer to the service-specific documentation. > info > The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs. ## Required authorization * Cluster privileges: `monitor_inference` ' operationId: inference-inference parameters: - $ref: '#/components/parameters/inference.inference-inference_id' - $ref: '#/components/parameters/inference.inference-timeout' requestBody: $ref: '#/components/requestBodies/inference.inference' responses: '200': $ref: '#/components/responses/inference.inference-200' x-state: Generally available; Added in 8.11.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name delete: tags: - inference summary: Delete an inference endpoint description: This API requires the manage_inference cluster privilege (the built-in `inference_admin` role grants this privilege). operationId: inference-delete parameters: - $ref: '#/components/parameters/inference.delete-inference_id' - $ref: '#/components/parameters/inference.delete-dry_run' - $ref: '#/components/parameters/inference.delete-force' responses: '200': $ref: '#/components/responses/inference.delete-200' x-state: Generally available; Added in 8.11.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{inference_id}: get: tags: - inference summary: Get an inference endpoint description: This API requires the `monitor_inference` cluster privilege (the built-in `inference_admin` and `inference_user` roles grant this privilege). operationId: inference-get-2 parameters: - $ref: '#/components/parameters/inference.get-task_type' - $ref: '#/components/parameters/inference.get-inference_id' responses: '200': $ref: '#/components/responses/inference.get-200' x-state: Generally available; Added in 8.11.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name put: tags: - inference summary: Create an inference endpoint description: 'IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Mistral, Azure OpenAI, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs. The following integrations are available through the inference API. You can find the available task types next to the integration name: * AI21 (`chat_completion`, `completion`) * AlibabaCloud AI Search (`completion`, `rerank`, `sparse_embedding`, `text_embedding`) * Amazon Bedrock (`chat_completion`, `completion`, `text_embedding`) * Amazon SageMaker (`chat_completion`, `completion`, `rerank`, `sparse_embedding`, `text_embedding`) * Anthropic (`chat_completion`, `completion`) * Azure AI Studio (`completion`, `rerank`, `text_embedding`) * Azure OpenAI (`chat_completion`, `completion`, `text_embedding`) * Cohere (`completion`, `rerank`, `text_embedding`) * DeepSeek (`chat_completion`, `completion`) * Elasticsearch (`rerank`, `sparse_embedding`, `text_embedding` - this service is for built-in models and models uploaded through Eland) * ELSER (`sparse_embedding`) * Fireworks AI (`chat_completion`, `completion`, `text_embedding`) * Google AI Studio (`completion`, `text_embedding`) * Google Vertex AI (`chat_completion`, `completion`, `rerank`, `text_embedding`) * Groq (`chat_completion`) * Hugging Face (`chat_completion`, `completion`, `rerank`, `text_embedding`) * JinaAI (`embedding`, `rerank`, `text_embedding`) * Llama (`chat_completion`, `completion`, `text_embedding`) * Mistral (`chat_completion`, `completion`, `text_embedding`) * Nvidia (`chat_completion`, `completion`, `text_embedding`, `rerank`) * OpenAI (`chat_completion`, `completion`, `text_embedding`) * OpenShift AI (`chat_completion`, `completion`, `rerank`, `text_embedding`) * VoyageAI (`rerank`, `text_embedding`) * Watsonx (`chat_completion`, `completion`, `rerank`, `text_embedding`) ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-1 parameters: - $ref: '#/components/parameters/inference.put-task_type' - $ref: '#/components/parameters/inference.put-inference_id' - $ref: '#/components/parameters/inference.put-timeout' requestBody: $ref: '#/components/requestBodies/inference.put' responses: '200': $ref: '#/components/responses/inference.put-200' x-state: Generally available; Added in 8.11.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name post: tags: - inference summary: Perform inference on the service description: 'This API enables you to use machine learning models to perform specific tasks on data that you provide as an input. It returns a response with the results of the tasks. The inference endpoint you use can perform one specific task that has been defined when the endpoint was created with the create inference API. For details about using this API with a service, such as Amazon Bedrock, Anthropic, or HuggingFace, refer to the service-specific documentation. > info > The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs. ## Required authorization * Cluster privileges: `monitor_inference` ' operationId: inference-inference-1 parameters: - $ref: '#/components/parameters/inference.inference-task_type' - $ref: '#/components/parameters/inference.inference-inference_id' - $ref: '#/components/parameters/inference.inference-timeout' requestBody: $ref: '#/components/requestBodies/inference.inference' responses: '200': $ref: '#/components/responses/inference.inference-200' x-state: Generally available; Added in 8.11.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name delete: tags: - inference summary: Delete an inference endpoint description: This API requires the manage_inference cluster privilege (the built-in `inference_admin` role grants this privilege). operationId: inference-delete-1 parameters: - $ref: '#/components/parameters/inference.delete-task_type' - $ref: '#/components/parameters/inference.delete-inference_id' - $ref: '#/components/parameters/inference.delete-dry_run' - $ref: '#/components/parameters/inference.delete-force' responses: '200': $ref: '#/components/responses/inference.delete-200' x-state: Generally available; Added in 8.11.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/_region_policy: get: tags: - inference summary: Get the inference region policy description: ' ## Required authorization * Cluster privileges: `monitor_inference` ' operationId: inference-get-region-policy responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.RegionPolicyDoc' examples: GetRegionPolicyResponseExample1: summary: The configured region policy description: A successful response when retrieving the currently configured region policy. value: "{\n \"region_policy\": {\n \"allowed_geos\": [\"us\", \"eu\"]\n },\n \"created_at\": \"2026-07-13T12:00:00.000Z\",\n \"created_by\": \"inference_user\",\n \"updated_at\": \"2026-07-13T13:30:00.000Z\",\n \"updated_by\": \"inference_user\"\n}" x-state: Generally available; Added in 9.5.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name put: tags: - inference summary: Create or update the inference region policy description: 'The region policy restricts inference to a set of allowed geographic areas or cloud service provider regions. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-region-policy parameters: - in: query name: force description: If `true`, the region policy is applied even if it would deny access to inference endpoints that are currently in use by ingest pipeline or indices. deprecated: false schema: default: false type: boolean style: form requestBody: content: application/json: schema: type: object properties: region_policy: description: The region policy configuration. allOf: - $ref: '#/components/schemas/inference._types.RegionPolicy' required: - region_policy examples: PutRegionPolicyRequestExample1: summary: A region policy allowing specific cloud service provider regions description: Run `PUT _inference/_region_policy` to restrict inference to a specific set of cloud service provider regions. value: "{\n \"region_policy\": {\n \"allowed_regions\": [\n { \"csp\": \"aws\", \"region\": \"us-east-1\" },\n { \"csp\": \"aws\", \"region\": \"eu-west-1\" }\n ]\n }\n}" PutRegionPolicyRequestExample2: summary: A region policy allowing specific geographic areas description: Run `PUT _inference/_region_policy` to restrict inference to a specific set of geographic areas. value: "{\n \"region_policy\": {\n \"allowed_geos\": [\"us\", \"eu\"]\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.RegionPolicyDoc' examples: PutRegionPolicyResponseExample1: summary: A region policy allowing specific cloud service provider regions description: A successful response when creating a region policy that allows specific cloud service provider regions. value: "{\n \"region_policy\": {\n \"allowed_regions\": [\n { \"csp\": \"aws\", \"region\": \"us-east-1\" },\n { \"csp\": \"aws\", \"region\": \"eu-west-1\" }\n ]\n },\n \"created_at\": \"2026-07-13T12:00:00.000Z\",\n \"created_by\": \"inference_user\"\n}" x-state: Generally available; Added in 9.5.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name delete: tags: - inference summary: Delete the inference region policy description: ' ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-delete-region-policy responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/_types.AcknowledgedResponseBase' examples: DeleteRegionPolicyResponseExample1: summary: A deleted region policy description: A successful response when deleting the region policy. value: "{\n \"acknowledged\": true\n}" x-state: Generally available; Added in 9.5.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/embedding/{inference_id}: post: tags: - inference summary: Perform dense embedding inference on the service operationId: inference-embedding parameters: - in: path name: inference_id description: The inference Id required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference request to complete. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: $ref: '#/components/schemas/inference._types.RequestEmbedding' examples: EmbeddingRequestExample1: summary: Multimodal embedding task description: Run `POST _inference/embedding/my-multimodal-endpoint` to generate embeddings from the example text and image value: "{\n \"input\": [\n {\n \"content\": {\n \"type\": \"image\",\n \"format\": \"base64\",\n \"value\": \"data:image/jpeg;base64,...\"\n }\n },\n {\n \"content\": {\n \"type\": \"text\",\n \"value\": \"Some text to create an embedding\"\n }\n }\n ]\n}" EmbeddingRequestExample2: summary: Text-only embedding task description: Run `POST _inference/embedding/my-text-only-endpoint` to generate embeddings from the example text value: "{\n \"input\": [\"The first text\", \"The second text\"]\n}" EmbeddingRequestExample3: summary: Embedding task using multiple items grouped under a single content object description: Run `POST _inference/embedding/my-multimodal-endpoint` to generate a single embedding from the example text and image. Multiple items in a single `content` object are available in Elasticsearch 9.5.0 and later. value: "{\n \"input\": [\n {\n \"content\": [\n {\n \"type\": \"image\",\n \"format\": \"base64\",\n \"value\": \"data:image/jpeg;base64,...\"\n },\n {\n \"type\": \"text\",\n \"value\": \"Some text to create an embedding\"\n }\n ]\n }\n ]\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.EmbeddingInferenceResult' examples: EmbeddingResponseExample1: summary: Multimodal embedding task description: 'An abbreviated response from `POST _inference/embedding/my-multimodal-endpoint`. ' value: "{\n \"embeddings\": [\n {\n \"embedding\": [\n -0.0189209,\n -0.04174805,\n 0.00854492,\n 0.01556396,\n 0.01928711,\n -0.00616455,\n -0.00460815,\n 0.01477051,\n -0.00656128,\n 0.05419922\n ]\n },\n {\n \"embedding\": [\n -0.01379395,\n -0.02368164,\n 0.01068115,\n 0.0279541,\n 0.01043701,\n -7.7057E-4,\n 0.04150391,\n 0.00836182,\n -0.01135254,\n 0.0246582\n ]\n }\n ]\n}" EmbeddingResponseExample2: summary: Text-only embedding task description: 'An abbreviated response from `POST _inference/embedding/my-text-only-endpoint`. ' value: "{\n \"embeddings\": [\n {\n \"embedding\": [\n 0.00854492,\n -0.00616455,\n -0.0189209,\n 0.01556396,\n -0.00460815,\n 0.01477051,\n -0.04174805,\n 0.01928711,\n -0.00656128,\n 0.05419922\n ]\n },\n {\n \"embedding\": [\n -0.01135254,\n 0.0279541,\n -0.02368164,\n 0.01068115,\n 0.01043701,\n 0.04150391,\n 0.00836182,\n -7.7057E-4,\n -0.01379395,\n 0.0246582\n ]\n }\n ]\n}" x-state: Experimental; Added in 9.4.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference: get: tags: - inference summary: Get an inference endpoint description: This API requires the `monitor_inference` cluster privilege (the built-in `inference_admin` and `inference_user` roles grant this privilege). operationId: inference-get responses: '200': $ref: '#/components/responses/inference.get-200' x-state: Generally available; Added in 8.11.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/_all: get: tags: - inference summary: Get an inference endpoint description: This API requires the `monitor_inference` cluster privilege (the built-in `inference_admin` and `inference_user` roles grant this privilege). operationId: inference-get-3 parameters: - $ref: '#/components/parameters/inference.get-task_type' responses: '200': $ref: '#/components/responses/inference.get-200' x-state: Generally available; Added in 8.11.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{ai21_inference_id}: put: tags: - inference summary: Create a AI21 inference endpoint description: 'Create an inference endpoint to perform an inference task with the `ai21` service. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-ai21 parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.Ai21TaskType' style: simple - in: path name: ai21_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: service: description: The type of service supported for the specified task type. In this case, `ai21`. allOf: - $ref: '#/components/schemas/inference._types.Ai21ServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `ai21` service. allOf: - $ref: '#/components/schemas/inference._types.Ai21ServiceSettings' required: - service - service_settings examples: PutAi21RequestExample1: description: Run `PUT _inference/completion/ai21-completion` to create an AI21 inference endpoint that performs a `completion` task. value: "{\n \"service\": \"ai21\",\n \"service_settings\": {\n \"api_key\": \"ai21-api-key\",\n \"model_id\": \"jamba-large\" \n }\n}" PutAi21RequestExample2: description: Run `PUT _inference/chat-completion/ai21-chat-completion` to create a AI21 inference endpoint that performs a `chat_completion` task. value: "{\n \"service\": \"ai21\",\n \"service_settings\": {\n \"api_key\": \"ai21-api-key\",\n \"model_id\": \"jamba-mini\" \n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoAi21' x-state: Generally available; Added in 9.2.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{alibabacloud_inference_id}: put: tags: - inference summary: Create an AlibabaCloud AI Search inference endpoint description: 'Create an inference endpoint to perform an inference task with the `alibabacloud-ai-search` service. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-alibabacloud parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.AlibabaCloudTaskType' style: simple - in: path name: alibabacloud_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `sparse_embedding` or `text_embedding` task types. Not applicable to the `rerank` or `completion` task types.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `alibabacloud-ai-search`. allOf: - $ref: '#/components/schemas/inference._types.AlibabaCloudServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `alibabacloud-ai-search` service. allOf: - $ref: '#/components/schemas/inference._types.AlibabaCloudServiceSettings' task_settings: description: 'Settings to configure the inference task. These settings are specific to the task type you specified.' allOf: - $ref: '#/components/schemas/inference._types.AlibabaCloudTaskSettings' required: - service - service_settings examples: PutAlibabaCloudRequestExample1: summary: A completion task description: Run `PUT _inference/completion/alibabacloud_ai_search_completion` to create an inference endpoint that performs a completion task. value: "{\n \"service\": \"alibabacloud-ai-search\",\n \"service_settings\": {\n \"host\" : \"default-j01.platform-cn-shanghai.opensearch.aliyuncs.com\",\n \"api_key\": \"AlibabaCloud-API-Key\",\n \"service_id\": \"ops-qwen-turbo\",\n \"workspace\" : \"default\"\n }\n}" PutAlibabaCloudRequestExample2: summary: A rerank task description: Run `PUT _inference/rerank/alibabacloud_ai_search_rerank` to create an inference endpoint that performs a rerank task. value: "{\n \"service\": \"alibabacloud-ai-search\",\n \"service_settings\": {\n \"api_key\": \"AlibabaCloud-API-Key\",\n \"service_id\": \"ops-bge-reranker-larger\",\n \"host\": \"default-j01.platform-cn-shanghai.opensearch.aliyuncs.com\",\n \"workspace\": \"default\"\n }\n}" PutAlibabaCloudRequestExample3: summary: A sparse embedding task description: Run `PUT _inference/sparse_embedding/alibabacloud_ai_search_sparse` to create an inference endpoint that performs perform a sparse embedding task. value: "{\n \"service\": \"alibabacloud-ai-search\",\n \"service_settings\": {\n \"api_key\": \"AlibabaCloud-API-Key\",\n \"service_id\": \"ops-text-sparse-embedding-001\",\n \"host\": \"default-j01.platform-cn-shanghai.opensearch.aliyuncs.com\",\n \"workspace\": \"default\"\n }\n}" PutAlibabaCloudRequestExample4: summary: A text embedding task description: Run `PUT _inference/text_embedding/alibabacloud_ai_search_embeddings` to create an inference endpoint that performs a text embedding task. value: "{\n \"service\": \"alibabacloud-ai-search\",\n \"service_settings\": {\n \"api_key\": \"AlibabaCloud-API-Key\",\n \"service_id\": \"ops-text-embedding-001\",\n \"host\": \"default-j01.platform-cn-shanghai.opensearch.aliyuncs.com\",\n \"workspace\": \"default\"\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoAlibabaCloudAI' x-state: Generally available; Added in 8.16.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{amazonbedrock_inference_id}: put: tags: - inference summary: Create an Amazon Bedrock inference endpoint description: 'Create an inference endpoint to perform an inference task with the `amazonbedrock` service. >info > You need to provide the access and secret keys only once, during the inference model creation. The get inference API does not retrieve your access or secret keys. After creating the inference model, you cannot change the associated key pairs. If you want to use a different access and secret key pair, delete the inference model and recreate it with the same name and the updated keys. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-amazonbedrock parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.AmazonBedrockTaskType' style: simple - in: path name: amazonbedrock_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `text_embedding` task type. Not applicable to the `chat_completion` and `completion` task types.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `amazonbedrock`. allOf: - $ref: '#/components/schemas/inference._types.AmazonBedrockServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `amazonbedrock` service. allOf: - $ref: '#/components/schemas/inference._types.AmazonBedrockServiceSettings' task_settings: description: 'Settings to configure the inference task. These settings are specific to the task type you specified.' allOf: - $ref: '#/components/schemas/inference._types.AmazonBedrockTaskSettings' required: - service - service_settings examples: PutAmazonBedrockRequestExample1: summary: A text embedding task description: Run `PUT _inference/text_embedding/amazon_bedrock_embeddings` to create an inference endpoint that performs a text embedding task. value: "{\n \"service\": \"amazonbedrock\",\n \"service_settings\": {\n \"access_key\": \"AWS-access-key\",\n \"secret_key\": \"AWS-secret-key\",\n \"region\": \"us-east-1\",\n \"provider\": \"amazontitan\",\n \"model\": \"amazon.titan-embed-text-v2:0\"\n }\n}" PutAmazonBedrockRequestExample2: summary: A completion task description: Run `PUT _inference/completion/amazon_bedrock_completion` to create an inference endpoint to perform a completion task. value: "{\n \"service\": \"amazonbedrock\",\n \"service_settings\": {\n \"access_key\": \"AWS-access-key\",\n \"secret_key\": \"AWS-secret-key\",\n \"region\": \"us-east-1\",\n \"provider\": \"amazontitan\",\n \"model\": \"amazon.titan-text-premier-v1:0\"\n }\n}" PutAmazonBedrockRequestExample3: summary: A chat completion task description: Run `PUT _inference/chat_completion/amazon_bedrock_chat_completion` to create an inference endpoint to perform a chat completion task. value: "{\n \"service\": \"amazonbedrock\",\n \"service_settings\": {\n \"access_key\": \"AWS-access-key\",\n \"secret_key\": \"AWS-secret-key\",\n \"region\": \"us-east-1\",\n \"provider\": \"amazontitan\",\n \"model\": \"amazon.titan-text-premier-v1:0\"\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoAmazonBedrock' x-state: Generally available; Added in 8.12.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{amazonsagemaker_inference_id}: put: tags: - inference summary: Create an Amazon SageMaker inference endpoint description: 'Create an inference endpoint to perform an inference task with the `amazon_sagemaker` service. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-amazonsagemaker parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.TaskTypeAmazonSageMaker' style: simple - in: path name: amazonsagemaker_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `sparse_embedding` or `text_embedding` task types. Not applicable to the `rerank`, `completion`, or `chat_completion` task types.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `amazon_sagemaker`. allOf: - $ref: '#/components/schemas/inference._types.AmazonSageMakerServiceType' service_settings: description: 'Settings used to install the inference model. These settings are specific to the `amazon_sagemaker` service and `service_settings.api` you specified.' allOf: - $ref: '#/components/schemas/inference._types.AmazonSageMakerServiceSettings' task_settings: description: 'Settings to configure the inference task. These settings are specific to the task type and `service_settings.api` you specified.' allOf: - $ref: '#/components/schemas/inference._types.AmazonSageMakerTaskSettings' required: - service - service_settings examples: PutAmazonSageMakerRequestExample1: summary: A text embedding task description: Run `PUT _inference/text_embedding/amazon_sagemaker_embeddings` to create an inference endpoint that performs a text embedding task. value: "{\n \"service\": \"amazon_sagemaker\",\n \"service_settings\": {\n \"access_key\": \"AWS-access-key\",\n \"secret_key\": \"AWS-secret-key\",\n \"region\": \"us-east-1\",\n \"api\": \"elastic\",\n \"endpoint_name\": \"my-endpoint\",\n \"dimensions\": 384,\n \"element_type\": \"float\"\n }\n}" PutAmazonSageMakerRequestExample2: summary: A completion task description: Run `PUT _inference/completion/amazon_sagemaker_completion` to create an inference endpoint that performs a completion task. value: "{\n \"service\": \"amazon_sagemaker\",\n \"service_settings\": {\n \"access_key\": \"AWS-access-key\",\n \"secret_key\": \"AWS-secret-key\",\n \"region\": \"us-east-1\",\n \"api\": \"elastic\",\n \"endpoint_name\": \"my-endpoint\"\n }\n}" PutAmazonSageMakerRequestExample3: summary: A chat completion task description: Run `PUT _inference/chat_completion/amazon_sagemaker_chat_completion` to create an inference endpoint that performs a chat completion task. value: "{\n \"service\": \"amazon_sagemaker\",\n \"service_settings\": {\n \"access_key\": \"AWS-access-key\",\n \"secret_key\": \"AWS-secret-key\",\n \"region\": \"us-east-1\",\n \"api\": \"elastic\",\n \"endpoint_name\": \"my-endpoint\"\n }\n}" PutAmazonSageMakerRequestExample4: summary: A sparse embedding task description: Run `PUT _inference/sparse_embedding/amazon_sagemaker_sparse_embedding` to create an inference endpoint that performs a sparse embedding task. value: "{\n \"service\": \"amazon_sagemaker\",\n \"service_settings\": {\n \"access_key\": \"AWS-access-key\",\n \"secret_key\": \"AWS-secret-key\",\n \"region\": \"us-east-1\",\n \"api\": \"elastic\",\n \"endpoint_name\": \"my-endpoint\"\n }\n}" PutAmazonSageMakerRequestExample5: summary: A rerank task description: Run `PUT _inference/rerank/amazon_sagemaker_rerank` to create an inference endpoint that performs a rerank task. value: "{\n \"service\": \"amazon_sagemaker\",\n \"service_settings\": {\n \"access_key\": \"AWS-access-key\",\n \"secret_key\": \"AWS-secret-key\",\n \"region\": \"us-east-1\",\n \"api\": \"elastic\",\n \"endpoint_name\": \"my-endpoint\"\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoAmazonSageMaker' x-state: Generally available; Added in 9.1.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{anthropic_inference_id}: put: tags: - inference summary: Create an Anthropic inference endpoint description: 'Create an inference endpoint to perform an inference task with the `anthropic` service. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-anthropic parameters: - in: path name: task_type description: 'The task type. The valid task types for the model to perform are `completion` and `chat_completion`. NOTE: The `chat_completion` task type only supports streaming and only through the _stream API.' required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.AnthropicTaskType' style: simple - in: path name: anthropic_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: service: description: The type of service supported for the specified task type. In this case, `anthropic`. allOf: - $ref: '#/components/schemas/inference._types.AnthropicServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `anthropic` service. allOf: - $ref: '#/components/schemas/inference._types.AnthropicServiceSettings' task_settings: description: 'Settings to configure the inference task. These settings are specific to the task type you specified.' allOf: - $ref: '#/components/schemas/inference._types.AnthropicTaskSettings' required: - service - service_settings examples: PutAnthropicChatCompletionRequestExample1: description: Run `PUT _inference/chat_completion/anthropic_chat_completion` to create an inference endpoint that performs a chat completion task. value: "{\n \"service\": \"anthropic\",\n \"service_settings\": {\n \"api_key\": \"Anthropic-Api-Key\",\n \"model_id\": \"claude-sonnet-4-5\"\n },\n \"task_settings\": {\n \"max_tokens\": 1024\n }\n}" PutAnthropicRequestExample1: description: Run `PUT _inference/completion/anthropic_completion` to create an inference endpoint that performs a completion task. value: "{\n \"service\": \"anthropic\",\n \"service_settings\": {\n \"api_key\": \"Anthropic-Api-Key\",\n \"model_id\": \"Model-ID\"\n },\n \"task_settings\": {\n \"max_tokens\": 1024\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoAnthropic' x-state: Generally available; Added in 8.16.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{azureaistudio_inference_id}: put: tags: - inference summary: Create an Azure AI studio inference endpoint description: 'Create an inference endpoint to perform an inference task with the `azureaistudio` service. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-azureaistudio parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.AzureAiStudioTaskType' style: simple - in: path name: azureaistudio_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `text_embedding` task type. Not applicable to the `rerank` or `completion` task types.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `azureaistudio`. allOf: - $ref: '#/components/schemas/inference._types.AzureAiStudioServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `azureaistudio` service. allOf: - $ref: '#/components/schemas/inference._types.AzureAiStudioServiceSettings' task_settings: description: 'Settings to configure the inference task. These settings are specific to the task type you specified.' allOf: - $ref: '#/components/schemas/inference._types.AzureAiStudioTaskSettings' required: - service - service_settings examples: PutAzureAiStudioRequestExample1: summary: A text embedding task description: Run `PUT _inference/text_embedding/azure_ai_studio_embeddings` to create an inference endpoint that performs a text_embedding task. Note that you do not specify a model here, as it is defined already in the Azure AI Studio deployment. value: "{\n \"service\": \"azureaistudio\",\n \"service_settings\": {\n \"api_key\": \"Azure-AI-Studio-API-key\",\n \"target\": \"Target-Uri\",\n \"provider\": \"openai\",\n \"endpoint_type\": \"token\"\n }\n}" PutAzureAiStudioRequestExample2: summary: A completion task description: Run `PUT _inference/completion/azure_ai_studio_completion` to create an inference endpoint that performs a completion task. value: "{\n \"service\": \"azureaistudio\",\n \"service_settings\": {\n \"api_key\": \"Azure-AI-Studio-API-key\",\n \"target\": \"Target-URI\",\n \"provider\": \"databricks\",\n \"endpoint_type\": \"realtime\"\n }\n}" PutAzureAiStudioRequestExample3: summary: A rerank task description: Run `PUT _inference/rerank/azure_ai_studio_rerank` to create an inference endpoint that performs a rerank task. value: "{\n \"service\": \"azureaistudio\",\n \"service_settings\": {\n \"api_key\": \"Azure-AI-Studio-API-key\",\n \"target\": \"Target-URI\",\n \"provider\": \"cohere\",\n \"endpoint_type\": \"token\"\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoAzureAIStudio' x-state: Generally available; Added in 8.14.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{azureopenai_inference_id}: put: tags: - inference summary: Create an Azure OpenAI inference endpoint description: 'Create an inference endpoint to perform an inference task with the `azureopenai` service. The list of chat completion models that you can choose from in your Azure OpenAI deployment include: * [GPT-4 and GPT-4 Turbo models](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=global-standard%2Cstandard-chat-completions#gpt-4-and-gpt-4-turbo-models) * [GPT-3.5](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=global-standard%2Cstandard-chat-completions#gpt-35) The list of embeddings models that you can choose from in your deployment can be found in the [Azure models documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=global-standard%2Cstandard-chat-completions#embeddings). ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-azureopenai parameters: - in: path name: task_type description: 'The type of the inference task that the model will perform. NOTE: The `chat_completion` task type only supports streaming and only through the _stream API.' required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.AzureOpenAITaskType' style: simple - in: path name: azureopenai_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `text_embedding` task type. Not applicable to the `completion` and `chat_completion` task types.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `azureopenai`. allOf: - $ref: '#/components/schemas/inference._types.AzureOpenAIServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `azureopenai` service. allOf: - $ref: '#/components/schemas/inference._types.AzureOpenAIServiceSettings' task_settings: description: 'Settings to configure the inference task. These settings are specific to the task type you specified.' allOf: - $ref: '#/components/schemas/inference._types.AzureOpenAITaskSettings' required: - service - service_settings examples: PutAzureOpenAiRequestExample1: summary: A text embedding task description: Run `PUT _inference/text_embedding/azure_openai_embeddings` to create an inference endpoint that performs a `text_embedding` task. You do not specify a model, as it is defined already in the Azure OpenAI deployment. value: "{\n \"service\": \"azureopenai\",\n \"service_settings\": {\n \"api_key\": \"Api-Key\",\n \"resource_name\": \"Resource-name\",\n \"deployment_id\": \"Deployment-id\",\n \"api_version\": \"2024-02-01\"\n }\n}" PutAzureOpenAiRequestExample2: summary: A completion task description: Run `PUT _inference/completion/azure_openai_completion` to create an inference endpoint that performs a `completion` task. value: "{\n \"service\": \"azureopenai\",\n \"service_settings\": {\n \"api_key\": \"Api-Key\",\n \"resource_name\": \"Resource-name\",\n \"deployment_id\": \"Deployment-id\",\n \"api_version\": \"2024-02-01\"\n }\n}" PutAzureOpenAiRequestExample3: summary: A chat completion task description: Run `PUT _inference/chat_completion/azure_openai_chat_completion` to create an inference endpoint that performs a `chat_completion` task. value: "{\n \"service\": \"azureopenai\",\n \"service_settings\": {\n \"api_key\": \"Api-Key\",\n \"resource_name\": \"Resource-name\",\n \"deployment_id\": \"Deployment-id\",\n \"api_version\": \"2024-02-01\"\n }\n}" PutAzureOpenAiRequestExample4: summary: A text embedding task with OAuth 2.0 authorization and custom headers description: Run `PUT _inference/text_embedding/azure_openai_embeddings` to create an inference endpoint that performs a `text_embedding` task. You do not specify a model, as it is defined already in the Azure OpenAI deployment. value: "{\n \"service\": \"azureopenai\",\n \"service_settings\": {\n \"api_version\": \"2023-05-15\",\n \"client_id\": \"12345\",\n \"client_secret\": \"secret\",\n \"deployment_id\": \"Deployment-id\",\n \"resource_name\": \"Resource-name\",\n \"scopes\": [\"Scope-1\", \"Scope-2\"],\n \"tenant_id\": \"tenant id\"\n },\n \"task_settings\": {\n \"headers\": {\n \"Custom-Header-Key\": \"Custom header value\"\n }\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoAzureOpenAI' examples: PutAzureOpenAiResponseExample1: summary: A text embedding task description: A successful response when creating an Azure OpenAI `text_embedding` inference endpoint. value: "{\n \"inference_id\": \"azure_openai_embeddings\",\n \"task_type\": \"text_embedding\",\n \"service\": \"azureopenai\",\n \"service_settings\": {\n \"resource_name\": \"Resource-name\",\n \"deployment_id\": \"Deployment-id\",\n \"api_version\": \"2024-02-01\",\n \"rate_limit\": {\n \"requests_per_minute\": 1140\n },\n \"dimensions\": 1536,\n \"similarity\": \"dot_product\"\n },\n \"chunking_settings\": {\n \"strategy\": \"sentence\",\n \"max_chunk_size\": 250,\n \"sentence_overlap\": 1\n }\n}" PutAzureOpenAiResponseExample2: summary: A completion task description: A successful response when creating an Azure OpenAI `completion` inference endpoint. value: "{\n \"inference_id\": \"azure_openai_completion\",\n \"task_type\": \"completion\",\n \"service\": \"azureopenai\",\n \"service_settings\": {\n \"resource_name\": \"Resource-name\",\n \"deployment_id\": \"Deployment-id\",\n \"api_version\": \"2024-02-01\",\n \"rate_limit\": {\n \"requests_per_minute\": 120\n }\n }\n}" PutAzureOpenAiResponseExample3: summary: A chat completion task description: A successful response when creating an Azure OpenAI `chat_completion` inference endpoint. value: "{\n \"inference_id\": \"azure_openai_chat_completion\",\n \"task_type\": \"chat_completion\",\n \"service\": \"azureopenai\",\n \"service_settings\": {\n \"resource_name\": \"Resource-name\",\n \"deployment_id\": \"Deployment-id\",\n \"api_version\": \"2024-02-01\",\n \"rate_limit\": {\n \"requests_per_minute\": 120\n }\n }\n}" x-state: Generally available; Added in 8.14.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{cohere_inference_id}: put: tags: - inference summary: Create a Cohere inference endpoint description: 'Create an inference endpoint to perform an inference task with the `cohere` service. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-cohere parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.CohereTaskType' style: simple - in: path name: cohere_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `text_embedding` task type. Not applicable to the `rerank` or `completion` task type.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `cohere`. allOf: - $ref: '#/components/schemas/inference._types.CohereServiceType' service_settings: description: 'Settings used to install the inference model. These settings are specific to the `cohere` service.' allOf: - $ref: '#/components/schemas/inference._types.CohereServiceSettings' task_settings: description: 'Settings to configure the inference task. These settings are specific to the task type you specified.' allOf: - $ref: '#/components/schemas/inference._types.CohereTaskSettings' required: - service - service_settings examples: PutCohereRequestExample1: summary: A text embedding task description: Run `PUT _inference/text_embedding/cohere-embeddings` to create an inference endpoint that performs a text embedding task. value: "{\n \"service\": \"cohere\",\n \"service_settings\": {\n \"api_key\": \"Cohere-Api-key\",\n \"model_id\": \"embed-english-light-v3.0\",\n \"embedding_type\": \"byte\"\n }\n}" PutCohereRequestExample2: summary: A rerank task description: Run `PUT _inference/rerank/cohere-rerank` to create an inference endpoint that performs a rerank task. value: "{\n \"service\": \"cohere\",\n \"service_settings\": {\n \"api_key\": \"Cohere-API-key\",\n \"model_id\": \"rerank-english-v3.0\"\n },\n \"task_settings\": {\n \"top_n\": 10,\n \"return_documents\": true\n }\n}" PutCohereRequestExample3: summary: A completion task description: Run `PUT _inference/completion/cohere-completion` to create an inference endpoint that performs a completion task. value: "{\n \"service\": \"cohere\",\n \"service_settings\": {\n \"api_key\": \"Cohere-API-key\",\n \"model_id\": \"command-a-03-2025\"\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoCohere' x-state: Generally available; Added in 8.13.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{contextualai_inference_id}: put: tags: - inference summary: Create an Contextual AI inference endpoint description: 'Create an inference endpoint to perform an inference task with the `contexualai` service. To review the available `rerank` models, refer to . ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-contextualai parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.TaskTypeContextualAI' style: simple - in: path name: contextualai_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: service: description: The type of service supported for the specified task type. In this case, `contextualai`. allOf: - $ref: '#/components/schemas/inference._types.ContextualAIServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `contextualai` service. allOf: - $ref: '#/components/schemas/inference._types.ContextualAIServiceSettings' task_settings: description: 'Settings to configure the inference task. These settings are specific to the task type you specified.' allOf: - $ref: '#/components/schemas/inference._types.ContextualAITaskSettings' required: - service - service_settings examples: PutContextualAiRequestExample1: summary: A rerank task description: Run `PUT _inference/rerank/contextualai-rerank` to create an inference endpoint for rerank tasks using the Contextual AI service. value: "{\n \"service\": \"contextualai\",\n \"service_settings\": {\n \"api_key\": \"ContextualAI-Api-key\",\n \"model_id\": \"ctxl-rerank-v2-instruct-multilingual-mini\"\n },\n \"task_settings\": {\n \"instruction\": \"Rerank the following documents based on their relevance to the query.\",\n \"top_k\": 3\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoContextualAi' x-state: Generally available; Added in 9.2.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{custom_inference_id}: put: tags: - inference summary: Create a custom inference endpoint description: "The custom service gives more control over how to interact with external inference services that aren't explicitly supported through dedicated integrations.\nThe custom service gives you the ability to define the headers, url, query parameters, request body, and secrets.\nThe custom service supports the template replacement functionality, which enables you to define a template that can be replaced with the value associated with that key.\nTemplates are portions of a string that start with `${` and end with `}`.\nThe parameters `secret_parameters` and `task_settings` are checked for keys for template replacement. Template replacement is supported in the `request`, `headers`, `url`, and `query_parameters`.\nIf the definition (key) is not found for a template, an error message is returned.\nIn case of an endpoint definition like the following:\n```\nPUT _inference/text_embedding/test-text-embedding\n{\n \"service\": \"custom\",\n \"service_settings\": {\n \"secret_parameters\": {\n \"api_key\": \"\"\n },\n \"url\": \"...endpoints.huggingface.cloud/v1/embeddings\",\n \"headers\": {\n \"Authorization\": \"Bearer ${api_key}\",\n \"Content-Type\": \"application/json\"\n },\n \"request\": \"{\\\"input\\\": ${input}}\",\n \"response\": {\n \"json_parser\": {\n \"text_embeddings\":\"$.data[*].embedding[*]\"\n }\n }\n }\n}\n```\nTo replace `${api_key}` the `secret_parameters` and `task_settings` are checked for a key named `api_key`.\n\n> info\n> Templates should not be surrounded by quotes.\n\nPre-defined templates:\n* `${input}` refers to the array of input strings that comes from the `input` field of the subsequent inference requests.\n* `${input_type}` refers to the input type translation values.\n* `${query}` refers to the query field used specifically for reranking tasks.\n* `${top_n}` refers to the `top_n` field available when performing rerank requests.\n* `${return_documents}` refers to the `return_documents` field available when performing rerank requests.\n\n## Required authorization\n\n* Cluster privileges: `manage_inference`\n" operationId: inference-put-custom parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.CustomTaskType' style: simple - in: path name: custom_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `sparse_embedding` or `text_embedding` task types. Not applicable to the `rerank` or `completion` task types.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `custom`. allOf: - $ref: '#/components/schemas/inference._types.CustomServiceType' service_settings: description: 'Settings used to install the inference model. These settings are specific to the `custom` service.' allOf: - $ref: '#/components/schemas/inference._types.CustomServiceSettings' task_settings: description: 'Settings to configure the inference task. These settings are specific to the task type you specified.' allOf: - $ref: '#/components/schemas/inference._types.CustomTaskSettings' required: - service - service_settings examples: PutCustomRequestExample1: summary: Custom text embedding task (OpenAI) description: Run `PUT _inference/text_embedding/custom-embeddings` to create an inference endpoint that performs a text embedding task. value: "{\n \"service\": \"custom\",\n \"service_settings\": {\n \"secret_parameters\": {\n \"api_key\": \"\"\n },\n \"url\": \"https://api.openai.com/v1/embeddings\",\n \"headers\": {\n \"Authorization\": \"Bearer ${api_key}\",\n \"Content-Type\": \"application/json;charset=utf-8\"\n },\n \"request\": \"{\\\"input\\\": ${input}, \\\"model\\\": \\\"text-embedding-3-small\\\"}\",\n \"response\": {\n \"json_parser\": {\n \"text_embeddings\": \"$.data[*].embedding[*]\"\n }\n }\n }\n}" PutCustomRequestExample2: summary: Custom rerank task (Cohere APIv2) description: Run `PUT _inference/rerank/custom-rerank` to create an inference endpoint that performs a rerank task. value: "{\n \"service\": \"custom\",\n \"service_settings\": {\n \"secret_parameters\": {\n \"api_key\": \"\"\n },\n \"url\": \"https://api.cohere.com/v2/rerank\",\n \"headers\": {\n \"Authorization\": \"bearer ${api_key}\",\n \"Content-Type\": \"application/json\"\n },\n \"request\": \"{\\\"documents\\\": ${input}, \\\"query\\\": ${query}, \\\"model\\\": \\\"rerank-v3.5\\\"}\",\n \"response\": {\n \"json_parser\": {\n \"reranked_index\":\"$.results[*].index\",\n \"relevance_score\":\"$.results[*].relevance_score\"\n }\n }\n }\n}" PutCustomRequestExample3: summary: Custom text embedding task (Cohere APIv2) description: Run `PUT _inference/text_embedding/custom-text-embedding` to create an inference endpoint that performs a text embedding task. value: "{\n \"service\": \"custom\",\n \"service_settings\": {\n \"secret_parameters\": {\n \"api_key\": \"\"\n },\n \"url\": \"https://api.cohere.com/v2/embed\",\n \"headers\": {\n \"Authorization\": \"bearer ${api_key}\",\n \"Content-Type\": \"application/json\"\n },\n \"request\": \"{\\\"texts\\\": ${input}, \\\"model\\\": \\\"embed-v4.0\\\", \\\"input_type\\\": ${input_type}}\",\n \"response\": {\n \"json_parser\": {\n \"text_embeddings\":\"$.embeddings.float[*]\"\n }\n },\n \"input_type\": {\n \"translation\": {\n \"ingest\": \"search_document\",\n \"search\": \"search_query\"\n },\n \"default\": \"search_document\"\n }\n }\n}" PutCustomRequestExample4: summary: Custom rerank task (Jina AI) description: Run `PUT _inference/rerank/custom-rerank-jina` to create an inference endpoint that performs a rerank task. value: "{\n \"service\": \"custom\",\n \"service_settings\": {\n \"secret_parameters\": {\n \"api_key\": \"\"\n }, \n \"url\": \"https://api.jina.ai/v1/rerank\",\n \"headers\": {\n \"Content-Type\": \"application/json\",\n \"Authorization\": \"Bearer ${api_key}\"\n },\n \"request\": \"{\\\"model\\\": \\\"jina-reranker-v2-base-multilingual\\\",\\\"query\\\": ${query},\\\"documents\\\":${input}}\",\n \"response\": {\n \"json_parser\": {\n \"relevance_score\": \"$.results[*].relevance_score\",\n \"reranked_index\": \"$.results[*].index\"\n }\n }\n }\n}" PutCustomRequestExample5: summary: Custom text embedding task (Hugging Face) description: Run `PUT _inference/text_embedding/custom-text-embedding-hf` to create an inference endpoint that performs a text embedding task by using the Qwen/Qwen3-Embedding-8B model. value: "{\n \"service\": \"custom\",\n \"service_settings\": {\n \"secret_parameters\": {\n \"api_key\": \"\"\n },\n \"url\": \"/v1/embeddings\",\n \"headers\": {\n \"Authorization\": \"Bearer ${api_key}\",\n \"Content-Type\": \"application/json\"\n },\n \"request\": \"{\\\"input\\\": ${input}}\",\n \"response\": {\n \"json_parser\": {\n \"text_embeddings\":\"$.data[*].embedding[*]\"\n }\n }\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoCustom' x-state: Generally available; Added in 8.19.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{deepseek_inference_id}: put: tags: - inference summary: Create a DeepSeek inference endpoint description: 'Create an inference endpoint to perform an inference task with the `deepseek` service. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-deepseek parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.TaskTypeDeepSeek' style: simple - in: path name: deepseek_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: service: description: The type of service supported for the specified task type. In this case, `deepseek`. allOf: - $ref: '#/components/schemas/inference._types.DeepSeekServiceType' service_settings: description: 'Settings used to install the inference model. These settings are specific to the `deepseek` service.' allOf: - $ref: '#/components/schemas/inference._types.DeepSeekServiceSettings' required: - service - service_settings required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoDeepSeek' x-state: Generally available; Added in 9.1.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{elasticsearch_inference_id}: put: tags: - inference summary: Create an Elasticsearch inference endpoint description: 'Create an inference endpoint to perform an inference task with the `elasticsearch` service. > info > Your Elasticsearch deployment contains preconfigured ELSER and E5 inference endpoints, you only need to create the enpoints using the API if you want to customize the settings. If you use the ELSER or the E5 model through the `elasticsearch` service, the API request will automatically download and deploy the model if it isn''t downloaded yet. > info > You might see a 502 bad gateway error in the response when using the Kibana Console. This error usually just reflects a timeout, while the model downloads in the background. You can check the download progress in the Machine Learning UI. If using the Python client, you can set the timeout parameter to a higher value. After creating the endpoint, wait for the model deployment to complete before using it. To verify the deployment status, use the get trained model statistics API. Look for `"state": "fully_allocated"` in the response and ensure that the `"allocation_count"` matches the `"target_allocation_count"`. Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-elasticsearch parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.ElasticsearchTaskType' style: simple - in: path name: elasticsearch_inference_id description: 'The unique identifier of the inference endpoint. The must not match the `model_id`.' required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `sparse_embedding` and `text_embedding` task types. Not applicable to the `rerank` task type.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `elasticsearch`. allOf: - $ref: '#/components/schemas/inference._types.ElasticsearchServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `elasticsearch` service. allOf: - $ref: '#/components/schemas/inference._types.ElasticsearchServiceSettings' task_settings: description: 'Settings to configure the inference task. These settings are specific to the task type you specified.' allOf: - $ref: '#/components/schemas/inference._types.ElasticsearchTaskSettings' required: - service - service_settings examples: PutElasticsearchRequestExample1: summary: ELSER sparse embedding task description: Run `PUT _inference/sparse_embedding/my-elser-model` to create an inference endpoint that performs a `sparse_embedding` task. The `model_id` must be the ID of one of the built-in ELSER models. The API will automatically download the ELSER model if it isn't already downloaded and then deploy the model. value: "{\n \"service\": \"elasticsearch\",\n \"service_settings\": {\n \"adaptive_allocations\": { \n \"enabled\": true,\n \"min_number_of_allocations\": 1,\n \"max_number_of_allocations\": 4\n },\n \"num_threads\": 1,\n \"model_id\": \".elser_model_2\" \n }\n}" PutElasticsearchRequestExample2: summary: Elastic rerank task description: Run `PUT _inference/rerank/my-elastic-rerank` to create an inference endpoint that performs a rerank task using the built-in Elastic Rerank cross-encoder model. The `model_id` must be `.rerank-v1`, which is the ID of the built-in Elastic Rerank model. The API will automatically download the Elastic Rerank model if it isn't already downloaded and then deploy the model. Once deployed, the model can be used for semantic re-ranking with a `text_similarity_reranker` retriever. value: "{\n \"service\": \"elasticsearch\",\n \"service_settings\": {\n \"model_id\": \".rerank-v1\", \n \"num_threads\": 1,\n \"adaptive_allocations\": { \n \"enabled\": true,\n \"min_number_of_allocations\": 1,\n \"max_number_of_allocations\": 4\n }\n }\n}" PutElasticsearchRequestExample3: summary: E5 text embedding task description: Run `PUT _inference/text_embedding/my-e5-model` to create an inference endpoint that performs a `text_embedding` task. The `model_id` must be the ID of one of the built-in E5 models. The API will automatically download the E5 model if it isn't already downloaded and then deploy the model. value: "{\n \"service\": \"elasticsearch\",\n \"service_settings\": {\n \"num_allocations\": 1,\n \"num_threads\": 1,\n \"model_id\": \".multilingual-e5-small\" \n }\n}" PutElasticsearchRequestExample4: summary: Eland text embedding task description: Run `PUT _inference/text_embedding/my-msmarco-minilm-model` to create an inference endpoint that performs a `text_embedding` task with a model that was uploaded by Eland. value: "{\n \"service\": \"elasticsearch\",\n \"service_settings\": {\n \"num_allocations\": 1,\n \"num_threads\": 1,\n \"model_id\": \"msmarco-MiniLM-L12-cos-v5\" \n }\n}" PutElasticsearchRequestExample5: summary: Adaptive allocation description: Run `PUT _inference/text_embedding/my-e5-model` to create an inference endpoint that performs a `text_embedding` task and to configure adaptive allocations. The API request will automatically download the E5 model if it isn't already downloaded and then deploy the model. value: "{\n \"service\": \"elasticsearch\",\n \"service_settings\": {\n \"adaptive_allocations\": {\n \"enabled\": true,\n \"min_number_of_allocations\": 3,\n \"max_number_of_allocations\": 10\n },\n \"num_threads\": 1,\n \"model_id\": \".multilingual-e5-small\"\n }\n}" PutElasticsearchRequestExample6: summary: Existing model deployment description: Run `PUT _inference/sparse_embedding/use_existing_deployment` to use an already existing model deployment when creating an inference endpoint. value: "{\n \"service\": \"elasticsearch\",\n \"service_settings\": {\n \"deployment_id\": \".elser_model_2\"\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoElasticsearch' examples: PutElasticsearchResponseExample1: description: 'A successful response from `PUT _inference/sparse_embedding/use_existing_deployment`. It contains the model ID and the threads and allocations settings from the model deployment. ' value: "{\n \"inference_id\": \"use_existing_deployment\",\n \"task_type\": \"sparse_embedding\",\n \"service\": \"elasticsearch\",\n \"service_settings\": {\n \"num_allocations\": 2,\n \"num_threads\": 1,\n \"model_id\": \".elser_model_2\",\n \"deployment_id\": \".elser_model_2\"\n },\n \"chunking_settings\": {\n \"strategy\": \"sentence\",\n \"max_chunk_size\": 250,\n \"sentence_overlap\": 1\n }\n}" x-state: Generally available; Added in 8.13.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{elser_inference_id}: put: tags: - inference summary: Create an ELSER inference endpoint description: 'Create an inference endpoint to perform an inference task with the `elser` service. You can also deploy ELSER by using the Elasticsearch inference integration. > info > Your Elasticsearch deployment contains a preconfigured ELSER inference endpoint, you only need to create the enpoint using the API if you want to customize the settings. The API request will automatically download and deploy the ELSER model if it isn''t already downloaded. > info > You might see a 502 bad gateway error in the response when using the Kibana Console. This error usually just reflects a timeout, while the model downloads in the background. You can check the download progress in the Machine Learning UI. If using the Python client, you can set the timeout parameter to a higher value. After creating the endpoint, wait for the model deployment to complete before using it. To verify the deployment status, use the get trained model statistics API. Look for `"state": "fully_allocated"` in the response and ensure that the `"allocation_count"` matches the `"target_allocation_count"`. Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-elser parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.ElserTaskType' style: simple - in: path name: elser_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Note that for ELSER endpoints, the max_chunk_size may not exceed `300`.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `elser`. allOf: - $ref: '#/components/schemas/inference._types.ElserServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `elser` service. allOf: - $ref: '#/components/schemas/inference._types.ElserServiceSettings' required: - service - service_settings examples: PutElserRequestExample1: summary: A sparse embedding task description: Run `PUT _inference/sparse_embedding/my-elser-model` to create an inference endpoint that performs a `sparse_embedding` task. The request will automatically download the ELSER model if it isn't already downloaded and then deploy the model. value: "{\n \"service\": \"elser\",\n \"service_settings\": {\n \"num_allocations\": 1,\n \"num_threads\": 1\n }\n}" PutElserRequestExample2: summary: Adaptive allocations description: Run `PUT _inference/sparse_embedding/my-elser-model` to create an inference endpoint that performs a `sparse_embedding` task with adaptive allocations. When adaptive allocations are enabled, the number of allocations of the model is set automatically based on the current load. value: "{\n \"service\": \"elser\",\n \"service_settings\": {\n \"adaptive_allocations\": {\n \"enabled\": true,\n \"min_number_of_allocations\": 3,\n \"max_number_of_allocations\": 10\n },\n \"num_threads\": 1\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoELSER' examples: PutElserResponseExample1: description: A successful response when creating an ELSER inference endpoint. value: "{\n \"inference_id\": \"my-elser-model\",\n \"task_type\": \"sparse_embedding\",\n \"service\": \"elser\",\n \"service_settings\": {\n \"num_allocations\": 1,\n \"num_threads\": 1\n },\n \"task_settings\": {}\n}" deprecated: true x-state: Generally available; Added in 8.11.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{fireworksai_inference_id}: put: tags: - inference summary: Create a Fireworks AI inference endpoint description: 'Create an inference endpoint to perform an inference task with the `fireworksai` service. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-fireworksai parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.FireworksAITaskType' style: simple - in: path name: fireworksai_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `text_embedding` task type. Not applicable to the `completion` or `chat_completion` task types.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `fireworksai`. allOf: - $ref: '#/components/schemas/inference._types.FireworksAIServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `fireworksai` service. allOf: - $ref: '#/components/schemas/inference._types.FireworksAIServiceSettings' task_settings: description: 'Settings to configure the inference task. Applies only to the `completion` or `chat_completion` task types. Not applicable to the `text_embedding` task type. These settings are specific to the task type you specified.' allOf: - $ref: '#/components/schemas/inference._types.FireworksAITaskSettings' required: - service - service_settings examples: PutFireworksAiRequestExample1: summary: Create an endpoint for text embedding tasks description: Run `PUT _inference/text_embedding/my-fireworks-embeddings` to create an inference endpoint that performs a `text_embedding` task. value: "{\n \"service\": \"fireworksai\",\n \"service_settings\": {\n \"api_key\": \"your-api-key\",\n \"model_id\": \"fireworks/qwen3-embedding-8b\"\n }\n}" PutFireworksAiRequestExample2: summary: Create an endpoint for text embedding tasks with optional settings description: Create a Fireworks AI text embedding endpoint with custom dimensions, similarity, and rate limit. value: "{\n \"service\": \"fireworksai\",\n \"service_settings\": {\n \"api_key\": \"your-api-key\",\n \"model_id\": \"fireworks/qwen3-embedding-8b\",\n \"dimensions\": 1024,\n \"similarity\": \"cosine\",\n \"rate_limit\": {\n \"requests_per_minute\": 6000\n }\n }\n}" PutFireworksAiRequestExample3: summary: Create an endpoint for chat completion tasks description: Run `PUT _inference/chat_completion/my-fireworks-chat` to create an inference endpoint that performs a `chat_completion` task. value: "{\n \"service\": \"fireworksai\",\n \"service_settings\": {\n \"api_key\": \"your-api-key\",\n \"model_id\": \"accounts/fireworks/models/deepseek-v3p1\"\n }\n}" PutFireworksAiRequestExample4: summary: Create an endpoint for completion tasks description: Run `PUT _inference/completion/my-fireworks-completion` to create an inference endpoint that performs a `completion` task. value: "{\n \"service\": \"fireworksai\",\n \"service_settings\": {\n \"api_key\": \"your-api-key\",\n \"model_id\": \"accounts/fireworks/models/deepseek-v3p1\"\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoFireworksAI' examples: PutFireworksAiResponseExample1: summary: Response for created text embedding endpoint description: A successful response when creating a Fireworks AI `text_embedding` inference endpoint. value: "{\n \"inference_id\": \"my-fireworks-embeddings\",\n \"task_type\": \"text_embedding\",\n \"service\": \"fireworksai\",\n \"service_settings\": {\n \"model_id\": \"fireworks/qwen3-embedding-8b\",\n \"url\": \"https://api.fireworks.ai/inference/v1/embeddings\",\n \"similarity\": \"cosine\",\n \"dimensions\": 4096,\n \"rate_limit\": {\n \"requests_per_minute\": 6000\n }\n },\n \"chunking_settings\": {\n \"strategy\": \"sentence\",\n \"max_chunk_size\": 250,\n \"sentence_overlap\": 1\n }\n}" PutFireworksAiResponseExample2: summary: Response for created completion endpoint description: A successful response when creating a Fireworks AI `chat_completion` inference endpoint. value: "{\n \"inference_id\": \"my-fireworks-chat\",\n \"task_type\": \"chat_completion\",\n \"service\": \"fireworksai\",\n \"service_settings\": {\n \"model_id\": \"accounts/fireworks/models/deepseek-v3p1\",\n \"rate_limit\": {\n \"requests_per_minute\": 6000\n }\n }\n}" x-state: Generally available; Added in 9.4.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{googleaistudio_inference_id}: put: tags: - inference summary: Create an Google AI Studio inference endpoint description: 'Create an inference endpoint to perform an inference task with the `googleaistudio` service. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-googleaistudio parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.GoogleAiStudioTaskType' style: simple - in: path name: googleaistudio_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `text_embedding` task type. Not applicable to the `completion` task type.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `googleaistudio`. allOf: - $ref: '#/components/schemas/inference._types.GoogleAiServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `googleaistudio` service. allOf: - $ref: '#/components/schemas/inference._types.GoogleAiStudioServiceSettings' required: - service - service_settings examples: PutGoogleAiStudioRequestExample1: summary: A completion task description: Run `PUT _inference/completion/google_ai_studio_completion` to create an inference endpoint to perform a `completion` task type. value: "{\n \"service\": \"googleaistudio\",\n \"service_settings\": {\n \"api_key\": \"api-key\",\n \"model_id\": \"model-id\"\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoGoogleAIStudio' x-state: Generally available; Added in 8.15.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{googlevertexai_inference_id}: put: tags: - inference summary: Create a Google Vertex AI inference endpoint description: 'Create an inference endpoint to perform an inference task with the `googlevertexai` service. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-googlevertexai parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.GoogleVertexAITaskType' style: simple - in: path name: googlevertexai_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `text_embedding` task type. Not applicable to the `rerank`, `completion`, or `chat_completion` task types.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `googlevertexai`. allOf: - $ref: '#/components/schemas/inference._types.GoogleVertexAIServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `googlevertexai` service. allOf: - $ref: '#/components/schemas/inference._types.GoogleVertexAIServiceSettings' task_settings: description: 'Settings to configure the inference task. These settings are specific to the task type you specified.' allOf: - $ref: '#/components/schemas/inference._types.GoogleVertexAITaskSettings' required: - service - service_settings examples: PutGoogleVertexAiRequestExample1: summary: A text embedding task description: Run `PUT _inference/text_embedding/google_vertex_ai_embeddings` to create an inference endpoint to perform a `text_embedding` task type. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"service_account_json\": \"service-account-json\",\n \"model_id\": \"model-id\",\n \"location\": \"location\",\n \"project_id\": \"project-id\"\n }\n}" PutGoogleVertexAiRequestExample10: summary: A chat_completion task for Google Model Garden Meta shared endpoint with single streaming URL provided description: Run `PUT _inference/chat_completion/google_model_garden_meta_chat_completion` to create an inference endpoint to perform a `chat_completion` task using Meta's model hosted on Google Model Garden shared endpoint with single streaming URL provided. See the endpoint's `Sample request` page for the variable values used in the URL. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"meta\",\n \"service_account_json\": \"service-account-json\",\n \"streaming_url\": \"https://%LOCATION_ID%-aiplatform.googleapis.com/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/endpoints/%ENDPOINT_ID%/chat/completions\"\n }\n}" PutGoogleVertexAiRequestExample11: summary: A completion task for Google Model Garden Hugging Face dedicated endpoint with single URL provided for both streaming and non-streaming tasks description: Run `PUT _inference/completion/google_model_garden_hugging_face_completion` to create an inference endpoint to perform a `completion` task using Hugging Face's model hosted on Google Model Garden dedicated endpoint with single URL provided for both streaming and non-streaming tasks. See the endpoint's `Sample request` page for the variable values used in the URL. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"hugging_face\",\n \"service_account_json\": \"service-account-json\",\n \"url\": \"https://%ENDPOINT_ID%.%LOCATION_ID%-%PROJECT_ID%.prediction.vertexai.goog/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/endpoints/%ENDPOINT_ID%/chat/completions\"\n }\n}" PutGoogleVertexAiRequestExample12: summary: A chat_completion task for Google Model Garden Hugging Face dedicated endpoint with single streaming URL provided description: Run `PUT _inference/chat_completion/google_model_garden_hugging_face_chat_completion` to create an inference endpoint to perform a `chat_completion` task using Hugging Face's model hosted on Google Model Garden dedicated endpoint with single streaming URL provided. See the endpoint's `Sample request` page for the variable values used in the URL. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"hugging_face\",\n \"service_account_json\": \"service-account-json\",\n \"streaming_url\": \"https://%ENDPOINT_ID%.%LOCATION_ID%-%PROJECT_ID%.prediction.vertexai.goog/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/endpoints/%ENDPOINT_ID%/chat/completions\"\n }\n}" PutGoogleVertexAiRequestExample13: summary: A completion task for Google Model Garden Hugging Face shared endpoint with single URL provided for both streaming and non-streaming tasks description: Run `PUT _inference/completion/google_model_garden_hugging_face_completion` to create an inference endpoint to perform a `completion` task using Hugging Face's model hosted on Google Model Garden shared endpoint with single URL provided for both streaming and non-streaming tasks. See the endpoint's `Sample request` page for the variable values used in the URL. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"hugging_face\",\n \"service_account_json\": \"service-account-json\",\n \"url\": \"https://%LOCATION_ID%-aiplatform.googleapis.com/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/endpoints/%ENDPOINT_ID%/chat/completions\"\n }\n}" PutGoogleVertexAiRequestExample14: summary: A chat_completion task for Google Model Garden Hugging Face shared endpoint with single streaming URL provided description: Run `PUT _inference/chat_completion/google_model_garden_hugging_face_chat_completion` to create an inference endpoint to perform a `chat_completion` task using Hugging Face's model hosted on Google Model Garden shared endpoint with single streaming URL provided. See the endpoint's `Sample request` page for the variable values used in the URL. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"hugging_face\",\n \"service_account_json\": \"service-account-json\",\n \"streaming_url\": \"https://%LOCATION_ID%-aiplatform.googleapis.com/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/endpoints/%ENDPOINT_ID%/chat/completions\"\n }\n}" PutGoogleVertexAiRequestExample15: summary: A completion task for Google Model Garden Mistral serverless endpoint with separate URLs for streaming and non-streaming tasks description: Run `PUT _inference/completion/google_model_garden_mistral_completion` to create an inference endpoint to perform a `completion` task using Mistral's serverless model hosted on Google Model Garden with separate URLs for streaming and non-streaming tasks. See the Mistral model documentation for instructions on how to construct URLs. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"mistral\",\n \"model_id\": \"mistral-small-2503\",\n \"service_account_json\": \"service-account-json\",\n \"url\": \"https://%LOCATION_ID%-aiplatform.googleapis.com/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/publishers/mistralai/models/%MODEL_ID%:rawPredict\",\n \"streaming_url\": \"https://%LOCATION_ID%-aiplatform.googleapis.com/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/publishers/mistralai/models/%MODEL_ID%:streamRawPredict\"\n }\n}" PutGoogleVertexAiRequestExample16: summary: A chat_completion task for Google Model Garden Mistral serverless endpoint with single streaming URL provided description: Run `PUT _inference/chat_completion/google_model_garden_mistral_chat_completion` to create an inference endpoint to perform a `chat_completion` task using Mistral's serverless model hosted on Google Model Garden with single streaming URL provided. See the Mistral model documentation for instructions on how to construct the URL. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"mistral\",\n \"model_id\": \"mistral-small-2503\",\n \"service_account_json\": \"service-account-json\",\n \"streaming_url\": \"https://%LOCATION_ID%-aiplatform.googleapis.com/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/publishers/mistralai/models/%MODEL_ID%:streamRawPredict\"\n }\n}" PutGoogleVertexAiRequestExample17: summary: A completion task for Google Model Garden Mistral dedicated endpoint with single URL provided for both streaming and non-streaming tasks description: Run `PUT _inference/completion/google_model_garden_mistral_completion` to create an inference endpoint to perform a `completion` task using Mistral's model hosted on Google Model Garden dedicated endpoint with single URL provided for both streaming and non-streaming tasks. See the endpoint's `Sample request` page for the variable values used in the URL. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"mistral\",\n \"service_account_json\": \"service-account-json\",\n \"url\": \"https://%ENDPOINT_ID%.%LOCATION_ID%-%PROJECT_ID%.prediction.vertexai.goog/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/endpoints/%ENDPOINT_ID%/chat/completions\"\n }\n}" PutGoogleVertexAiRequestExample18: summary: A chat_completion task for Google Model Garden Mistral dedicated endpoint with single streaming URL provided description: Run `PUT _inference/chat_completion/google_model_garden_mistral_chat_completion` to create an inference endpoint to perform a `chat_completion` task using Mistral's model hosted on Google Model Garden dedicated endpoint with single streaming URL provided. See the endpoint's `Sample request` page for the variable values used in the URL. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"mistral\",\n \"service_account_json\": \"service-account-json\",\n \"streaming_url\": \"https://%ENDPOINT_ID%.%LOCATION_ID%-%PROJECT_ID%.prediction.vertexai.goog/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/endpoints/%ENDPOINT_ID%/chat/completions\"\n }\n}" PutGoogleVertexAiRequestExample19: summary: A completion task for Google Model Garden Mistral shared endpoint with single URL provided for both streaming and non-streaming tasks description: Run `PUT _inference/completion/google_model_garden_mistral_completion` to create an inference endpoint to perform a `completion` task using Mistral's model hosted on Google Model Garden shared endpoint with single URL provided for both streaming and non-streaming tasks. See the endpoint's `Sample request` page for the variable values used in the URL. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"mistral\",\n \"service_account_json\": \"service-account-json\",\n \"url\": \"https://%LOCATION_ID%-aiplatform.googleapis.com/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/endpoints/%ENDPOINT_ID%/chat/completions\"\n }\n}" PutGoogleVertexAiRequestExample2: summary: A rerank task description: Run `PUT _inference/rerank/google_vertex_ai_rerank` to create an inference endpoint to perform a `rerank` task type. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"service_account_json\": \"service-account-json\",\n \"project_id\": \"project-id\"\n }\n}" PutGoogleVertexAiRequestExample20: summary: A chat_completion task for Google Model Garden Mistral shared endpoint with single streaming URL provided description: Run `PUT _inference/chat_completion/google_model_garden_mistral_chat_completion` to create an inference endpoint to perform a `chat_completion` task using Mistral's model hosted on Google Model Garden shared endpoint with single streaming URL provided. See the endpoint's `Sample request` page for the variable values used in the URL. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"mistral\",\n \"service_account_json\": \"service-account-json\",\n \"streaming_url\": \"https://%LOCATION_ID%-aiplatform.googleapis.com/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/endpoints/%ENDPOINT_ID%/chat/completions\"\n }\n}" PutGoogleVertexAiRequestExample21: summary: A completion task for Google Model Garden AI21 serverless endpoint with separate URLs for streaming and non-streaming tasks description: Run `PUT _inference/completion/google_model_garden_ai21_completion` to create an inference endpoint to perform a `completion` task using AI21's model hosted on Google Model Garden serverless endpoint with separate URLs for streaming and non-streaming tasks. See the AI21 model documentation for instructions on how to construct URLs. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"ai21\",\n \"service_account_json\": \"service-account-json\",\n \"url\": \"https://%LOCATION_ID%-aiplatform.googleapis.com/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/publishers/ai21/models/%MODEL_ID%:rawPredict\",\n \"streaming_url\": \"https://%LOCATION_ID%-aiplatform.googleapis.com/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/publishers/ai21/models/%MODEL_ID%:streamRawPredict\"\n }\n}" PutGoogleVertexAiRequestExample22: summary: A chat_completion task for Google Model Garden AI21 serverless endpoint with single streaming URL provided description: Run `PUT _inference/chat_completion/google_model_garden_ai21_chat_completion` to create an inference endpoint to perform a `chat_completion` task using AI21's model hosted on Google Model Garden serverless endpoint with single streaming URL provided. See the AI21 model documentation for instructions on how to construct URLs. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"ai21\",\n \"service_account_json\": \"service-account-json\",\n \"streaming_url\": \"https://%LOCATION_ID%-aiplatform.googleapis.com/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/publishers/ai21/models/%MODEL_ID%:streamRawPredict\"\n }\n}" PutGoogleVertexAiRequestExample23: summary: A text embedding task targeting the global endpoint description: Run `PUT _inference/text_embedding/google_vertex_ai_embeddings_global` to create an inference endpoint that performs a `text_embedding` task against the Google Vertex AI global endpoint. Omit `location` to target the global endpoint (`https://aiplatform.googleapis.com`) instead of a regional one. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"service_account_json\": \"service-account-json\",\n \"model_id\": \"model-id\",\n \"project_id\": \"project-id\"\n }\n}" PutGoogleVertexAiRequestExample3: summary: A completion task for Google Model Garden Anthropic serverless endpoint with separate URLs for streaming and non-streaming tasks description: Run `PUT _inference/completion/google_model_garden_anthropic_completion` to create an inference endpoint to perform a `completion` task using Anthropic's serverless model hosted on Google Model Garden with separate URLs for streaming and non-streaming tasks. See the Anthropic model documentation for instructions on how to construct URLs. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"anthropic\",\n \"service_account_json\": \"service-account-json\",\n \"url\": \"https://%LOCATION_ID%-aiplatform.googleapis.com/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/publishers/anthropic/models/%MODEL_ID%:rawPredict\",\n \"streaming_url\": \"https://%LOCATION_ID%-aiplatform.googleapis.com/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/publishers/anthropic/models/%MODEL_ID%:streamRawPredict\"\n },\n \"task_settings\": {\n \"max_tokens\": 128\n }\n}" PutGoogleVertexAiRequestExample4: summary: A chat_completion task for Google Model Garden Anthropic serverless endpoint with single streaming URL provided description: Run `PUT _inference/chat_completion/google_model_garden_anthropic_chat_completion` to create an inference endpoint to perform a `chat_completion` task using Anthropic's serverless model hosted on Google Model Garden with single streaming URL provided. See the Anthropic model documentation for instructions on how to construct the URL. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"anthropic\",\n \"service_account_json\": \"service-account-json\",\n \"streaming_url\": \"https://%LOCATION_ID%-aiplatform.googleapis.com/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/publishers/anthropic/models/%MODEL_ID%:streamRawPredict\"\n },\n \"task_settings\": {\n \"max_tokens\": 128\n }\n}" PutGoogleVertexAiRequestExample5: summary: A completion task for Google Model Garden Meta serverless endpoint with single URL provided for both streaming and non-streaming tasks description: Run `PUT _inference/completion/google_model_garden_meta_completion` to create an inference endpoint to perform a `completion` task using Meta's serverless model hosted on Google Model Garden with single URL provided for both streaming and non-streaming tasks. See the Meta model documentation for instructions on how to construct the URL. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"meta\",\n \"model_id\": \"meta/llama-3.3-70b-instruct-maas\",\n \"service_account_json\": \"service-account-json\",\n \"url\": \"https://%LOCATION_ID%-aiplatform.googleapis.com/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/endpoints/openapi/chat/completions\"\n }\n}" PutGoogleVertexAiRequestExample6: summary: A chat_completion task for Google Model Garden Meta serverless endpoint with single streaming URL provided description: Run `PUT _inference/chat_completion/google_model_garden_meta_chat_completion` to create an inference endpoint to perform a `chat_completion` task using Meta's serverless model hosted on Google Model Garden with single streaming URL provided. See the Meta model documentation for instructions on how to construct the URL. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"meta\",\n \"model_id\": \"meta/llama-3.3-70b-instruct-maas\",\n \"service_account_json\": \"service-account-json\",\n \"streaming_url\": \"https://%LOCATION_ID%-aiplatform.googleapis.com/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/endpoints/openapi/chat/completions\"\n }\n}" PutGoogleVertexAiRequestExample7: summary: A completion task for Google Model Garden Meta dedicated endpoint with single URL provided for both streaming and non-streaming tasks description: Run `PUT _inference/completion/google_model_garden_meta_completion` to create an inference endpoint to perform a `completion` task using Meta's model hosted on Google Model Garden dedicated endpoint with single URL provided for both streaming and non-streaming tasks. See the endpoint's `Sample request` page for the variable values used in the URL. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"meta\",\n \"service_account_json\": \"service-account-json\",\n \"url\": \"https://%ENDPOINT_ID%.%LOCATION_ID%-fasttryout.prediction.vertexai.goog/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/endpoints/%ENDPOINT_ID%/chat/completions\"\n }\n}" PutGoogleVertexAiRequestExample8: summary: A chat_completion task for Google Model Garden Meta dedicated endpoint with single streaming URL provided description: Run `PUT _inference/chat_completion/google_model_garden_meta_chat_completion` to create an inference endpoint to perform a `chat_completion` task using Meta's model hosted on Google Model Garden dedicated endpoint with single streaming URL provided. See the endpoint's `Sample request` page for the variable values used in the URL. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"meta\",\n \"service_account_json\": \"service-account-json\",\n \"streaming_url\": \"https://%ENDPOINT_ID%.%LOCATION_ID%-fasttryout.prediction.vertexai.goog/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/endpoints/%ENDPOINT_ID%/chat/completions\"\n }\n}" PutGoogleVertexAiRequestExample9: summary: A completion task for Google Model Garden Meta shared endpoint with single URL provided for both streaming and non-streaming tasks description: Run `PUT _inference/completion/google_model_garden_meta_completion` to create an inference endpoint to perform a `completion` task using Meta's model hosted on Google Model Garden shared endpoint with single URL provided for both streaming and non-streaming tasks. See the endpoint's `Sample request` page for the variable values used in the URL. value: "{\n \"service\": \"googlevertexai\",\n \"service_settings\": {\n \"provider\": \"meta\",\n \"service_account_json\": \"service-account-json\",\n \"url\": \"https://%LOCATION_ID%-aiplatform.googleapis.com/v1/projects/%PROJECT_ID%/locations/%LOCATION_ID%/endpoints/%ENDPOINT_ID%/chat/completions\"\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoGoogleVertexAI' x-state: Generally available; Added in 8.15.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{groq_inference_id}: put: tags: - inference summary: Create a Groq inference endpoint description: 'Create an inference endpoint to perform an inference task with the `groq` service. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-groq parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.GroqTaskType' style: simple - in: path name: groq_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: service: description: The type of service supported for the specified task type. In this case, `groq`. allOf: - $ref: '#/components/schemas/inference._types.GroqServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `groq` service. allOf: - $ref: '#/components/schemas/inference._types.GroqServiceSettings' required: - service - service_settings examples: PutGroqRequestExample1: description: Run `PUT _inference/chat_completion/groq-chat-completion` to create a Groq inference endpoint that performs a `chat_completion` task. value: "{\n \"service\": \"groq\",\n \"service_settings\": {\n \"api_key\": \"groq-api-key\",\n \"model_id\": \"llama-3.3-70b-versatile\" \n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoGroq' x-state: Generally available; Added in 9.3.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{huggingface_inference_id}: put: tags: - inference summary: Create a Hugging Face inference endpoint description: 'Create an inference endpoint to perform an inference task with the `hugging_face` service. Supported tasks include: `text_embedding`, `completion`, and `chat_completion`. To configure the endpoint, first visit the Hugging Face Inference Endpoints page and create a new endpoint. Select a model that supports the task you intend to use. For Elastic''s `text_embedding` task: The selected model must support the `Sentence Embeddings` task. On the new endpoint creation page, select the `Sentence Embeddings` task under the `Advanced Configuration` section. After the endpoint has initialized, copy the generated endpoint URL. Recommended models for `text_embedding` task: * `all-MiniLM-L6-v2` * `all-MiniLM-L12-v2` * `all-mpnet-base-v2` * `e5-base-v2` * `e5-small-v2` * `multilingual-e5-base` * `multilingual-e5-small` For Elastic''s `chat_completion` and `completion` tasks: The selected model must support the `Text Generation` task and expose OpenAI API. HuggingFace supports both serverless and dedicated endpoints for `Text Generation`. When creating dedicated endpoint select the `Text Generation` task. After the endpoint is initialized (for dedicated) or ready (for serverless), ensure it supports the OpenAI API and includes `/v1/chat/completions` part in URL. Then, copy the full endpoint URL for use. Recommended models for `chat_completion` and `completion` tasks: * `Mistral-7B-Instruct-v0.2` * `QwQ-32B` * `Phi-3-mini-128k-instruct` For Elastic''s `rerank` task: The selected model must support the `sentence-ranking` task and expose OpenAI API. HuggingFace supports only dedicated (not serverless) endpoints for `Rerank` so far. After the endpoint is initialized, copy the full endpoint URL for use. Tested models for `rerank` task: * `bge-reranker-base` * `jina-reranker-v1-turbo-en-GGUF` ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-hugging-face parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.HuggingFaceTaskType' style: simple - in: path name: huggingface_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `text_embedding` task type. Not applicable to the `rerank`, `completion`, or `chat_completion` task types.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `hugging_face`. allOf: - $ref: '#/components/schemas/inference._types.HuggingFaceServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `hugging_face` service. allOf: - $ref: '#/components/schemas/inference._types.HuggingFaceServiceSettings' task_settings: description: 'Settings to configure the inference task. These settings are specific to the task type you specified.' allOf: - $ref: '#/components/schemas/inference._types.HuggingFaceTaskSettings' required: - service - service_settings examples: PutHuggingFaceRequestExample1: summary: A text embedding task description: Run `PUT _inference/text_embedding/hugging-face-embeddings` to create an inference endpoint that performs a `text_embedding` task type. value: "{\n \"service\": \"hugging_face\",\n \"service_settings\": {\n \"api_key\": \"hugging-face-access-token\", \n \"url\": \"url-endpoint\" \n }\n}" PutHuggingFaceRequestExample2: summary: A rerank task description: Run `PUT _inference/rerank/hugging-face-rerank` to create an inference endpoint that performs a `rerank` task type. value: "{\n \"service\": \"hugging_face\",\n \"service_settings\": {\n \"api_key\": \"hugging-face-access-token\", \n \"url\": \"url-endpoint\" \n },\n \"task_settings\": {\n \"return_documents\": true,\n \"top_n\": 3\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoHuggingFace' x-state: Generally available; Added in 8.12.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{jinaai_inference_id}: put: tags: - inference summary: Create an JinaAI inference endpoint description: 'Create an inference endpoint to perform an inference task with the `jinaai` service. To review the available `rerank` models, refer to . To review the available `embedding` and `text_embedding` models, refer to . ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-jinaai parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.JinaAITaskType' style: simple - in: path name: jinaai_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `embedding` and text_embedding` task types. Not applicable to the `rerank` task type.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `jinaai`. allOf: - $ref: '#/components/schemas/inference._types.JinaAIServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `jinaai` service. allOf: - $ref: '#/components/schemas/inference._types.JinaAIServiceSettings' task_settings: description: 'Settings to configure the inference task. These settings are specific to the task type you specified.' allOf: - $ref: '#/components/schemas/inference._types.JinaAITaskSettings' required: - service - service_settings examples: PutJinaAiRequestExample1: summary: A text embedding task description: Run `PUT _inference/text_embedding/jinaai-embeddings` to create an inference endpoint for text embedding tasks using the JinaAI service. value: "{\n \"service\": \"jinaai\",\n \"service_settings\": {\n \"model_id\": \"jina-embeddings-v3\",\n \"api_key\": \"JinaAi-Api-key\"\n }\n}" PutJinaAiRequestExample2: summary: A rerank task description: Run `PUT _inference/rerank/jinaai-rerank` to create an inference endpoint for rerank tasks using the JinaAI service. value: "{\n \"service\": \"jinaai\",\n \"service_settings\": {\n \"api_key\": \"JinaAI-Api-key\",\n \"model_id\": \"jina-reranker-v2-base-multilingual\"\n },\n \"task_settings\": {\n \"top_n\": 10,\n \"return_documents\": true\n }\n}" PutJinaAiRequestExample3: summary: An embedding task using a multimodal model description: Run `PUT _inference/embedding/jinaai-embeddings-multimodal` to create an inference endpoint for embedding tasks using an embedding model that supports multimodal inputs (e.g. `jina-embeddings-v4`) via the JinaAI service. value: "{\n \"service\": \"jinaai\",\n \"service_settings\": {\n \"model_id\": \"jina-embeddings-v4\",\n \"api_key\": \"JinaAi-Api-key\"\n }\n}" PutJinaAiRequestExample4: summary: An embedding task using a non-multimodal model description: Run `PUT _inference/embedding/jinaai-embeddings-text-only` to create an inference endpoint for embedding tasks using an embedding model that does not support multimodal inputs (e.g. `jina-embeddings-v3`) via the JinaAI service. value: "{\n \"service\": \"jinaai\",\n \"service_settings\": {\n \"model_id\": \"jina-embeddings-v3\",\n \"api_key\": \"JinaAi-Api-key\",\n \"multimodal_model\": false\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoJinaAi' examples: PutJinaAiResponseExample1: summary: A text embedding task description: A successful response when creating a JinaAI `text_embedding` inference endpoint. value: "{\n \"inference_id\": \"jinaai-embeddings\",\n \"task_type\": \"text_embedding\",\n \"service\": \"jinaai\",\n \"service_settings\": {\n \"model_id\": \"jina-embeddings-v3\",\n \"rate_limit\": {\n \"requests_per_minute\": 2000\n },\n \"dimensions\": 1024,\n \"embedding_type\": \"float\",\n \"similarity\": \"dot_product\"\n },\n \"chunking_settings\": {\n \"strategy\": \"sentence\",\n \"max_chunk_size\": 250,\n \"sentence_overlap\": 1\n }\n}" PutJinaAiResponseExample2: summary: A rerank task description: A successful response when creating a JinaAI `rerank` inference endpoint. value: "{\n \"inference_id\": \"jinaai-rerank\",\n \"task_type\": \"rerank\",\n \"service\": \"jinaai\",\n \"service_settings\": {\n \"model_id\": \"jina-reranker-v2-base-multilingual\",\n \"rate_limit\": {\n \"requests_per_minute\": 2000\n }\n },\n \"task_settings\": {\n \"top_n\": 10,\n \"return_documents\": true\n }\n}" PutJinaAiResponseExample3: summary: An embedding task using a multimodal model description: A successful response when creating a JinaAI `embedding` inference endpoint. value: "{\n \"inference_id\": \"jinaai-embeddings-multimodal\",\n \"task_type\": \"embedding\",\n \"service\": \"jinaai\",\n \"service_settings\": {\n \"model_id\": \"jina-embeddings-v4\",\n \"rate_limit\": {\n \"requests_per_minute\": 2000\n },\n \"dimensions\": 2048,\n \"embedding_type\": \"float\",\n \"similarity\": \"dot_product\",\n \"multimodal_model\": true\n },\n \"chunking_settings\": {\n \"strategy\": \"sentence\",\n \"max_chunk_size\": 250,\n \"sentence_overlap\": 1\n }\n}" PutJinaAiResponseExample4: summary: An embedding task using a non-multimodal model description: A successful response when creating a JinaAI `embedding` inference endpoint with a model that does not support multimodal inputs. value: "{\n \"inference_id\": \"jinaai-embeddings-text-only\",\n \"task_type\": \"embedding\",\n \"service\": \"jinaai\",\n \"service_settings\": {\n \"model_id\": \"jina-embeddings-v3\",\n \"rate_limit\": {\n \"requests_per_minute\": 2000\n },\n \"dimensions\": 1024,\n \"embedding_type\": \"float\",\n \"similarity\": \"dot_product\",\n \"multimodal_model\": false\n },\n \"chunking_settings\": {\n \"strategy\": \"sentence\",\n \"max_chunk_size\": 250,\n \"sentence_overlap\": 1\n }\n}" x-state: Generally available; Added in 8.18.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{llama_inference_id}: put: tags: - inference summary: Create a Llama inference endpoint description: 'Create an inference endpoint to perform an inference task with the `llama` service. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-llama parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.LlamaTaskType' style: simple - in: path name: llama_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `text_embedding` task type. Not applicable to the `completion` or `chat_completion` task types.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `llama`. allOf: - $ref: '#/components/schemas/inference._types.LlamaServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `llama` service. allOf: - $ref: '#/components/schemas/inference._types.LlamaServiceSettings' required: - service - service_settings examples: PutLlamaRequestExample1: description: Run `PUT _inference/text_embedding/llama-text-embedding` to create a Llama inference endpoint that performs a `text_embedding` task. value: "{\n \"service\": \"llama\",\n \"service_settings\": {\n \"url\": \"http://localhost:8321/v1/inference/embeddings\",\n \"dimensions\": 384,\n \"model_id\": \"all-MiniLM-L6-v2\" \n }\n}" PutLlamaRequestExample2: description: Run `PUT _inference/completion/llama-completion` to create a Llama inference endpoint that performs a `completion` task. value: "{\n \"service\": \"llama\",\n \"service_settings\": {\n \"url\": \"http://localhost:8321/v1/openai/v1/chat/completions\",\n \"model_id\": \"llama3.2:3b\" \n }\n}" PutLlamaRequestExample3: description: Run `PUT _inference/chat_completion/llama-chat-completion` to create a Llama inference endpoint that performs a `chat_completion` task. value: "{\n \"service\": \"llama\",\n \"service_settings\": {\n \"url\": \"http://localhost:8321/v1/openai/v1/chat/completions\",\n \"model_id\": \"llama3.2:3b\" \n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoLlama' x-state: Generally available; Added in 9.2.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{mistral_inference_id}: put: tags: - inference summary: Create a Mistral inference endpoint description: 'Create an inference endpoint to perform an inference task with the `mistral` service. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-mistral parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.MistralTaskType' style: simple - in: path name: mistral_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `text_embedding` task type. Not applicable to the `completion` or `chat_completion` task types.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `mistral`. allOf: - $ref: '#/components/schemas/inference._types.MistralServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `mistral` service. allOf: - $ref: '#/components/schemas/inference._types.MistralServiceSettings' required: - service - service_settings examples: PutMistralRequestExample1: description: Run `PUT _inference/text_embedding/mistral-embeddings-test` to create a Mistral inference endpoint that performs a text embedding task. value: "{\n \"service\": \"mistral\",\n \"service_settings\": {\n \"api_key\": \"Mistral-API-Key\",\n \"model\": \"mistral-embed\" \n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoMistral' x-state: Generally available; Added in 8.15.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{nvidia_inference_id}: put: tags: - inference summary: Create an Nvidia inference endpoint description: 'Create an inference endpoint to perform an inference task with the `nvidia` service. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-nvidia parameters: - in: path name: task_type description: 'The type of the inference task that the model will perform. NOTE: The `chat_completion` task type only supports streaming and only through the _stream API.' required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.NvidiaTaskType' style: simple - in: path name: nvidia_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `text_embedding` task type. Not applicable to the `rerank`, `completion`, or `chat_completion` task types.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `nvidia`. allOf: - $ref: '#/components/schemas/inference._types.NvidiaServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `nvidia` service. allOf: - $ref: '#/components/schemas/inference._types.NvidiaServiceSettings' task_settings: description: 'Settings to configure the inference task. Applies only to the `text_embedding` task type. Not applicable to the `rerank`, `completion`, or `chat_completion` task types. These settings are specific to the task type you specified.' allOf: - $ref: '#/components/schemas/inference._types.NvidiaTaskSettings' required: - service - service_settings examples: PutNvidiaRequestExample1: summary: A text embedding task description: Run `PUT _inference/text_embedding/nvidia-text-embedding` to create an inference endpoint that performs a `text_embedding` task. value: "{\n \"service\": \"nvidia\",\n \"service_settings\": {\n \"url\": \"nvidia-embeddings-url\",\n \"api_key\": \"nvidia-embeddings-token\",\n \"model_id\": \"nvidia/llama-3.2-nv-embedqa-1b-v2\"\n }\n}" PutNvidiaRequestExample2: summary: A text embedding task with custom `task_settings` and no `url` parameter description: Run `PUT _inference/text_embedding/nvidia-text-embedding` to create an inference endpoint that performs a `text_embedding` task, specifying custom `task_settings` and omitting the `url` parameter if model is accessible via default NVIDIA endpoint. value: "{\n \"service\": \"nvidia\",\n \"service_settings\": {\n \"model_id\": \"nvidia/llama-3.2-nv-embedqa-1b-v2\",\n \"api_key\": \"nvidia-text-embeddings-token\"\n },\n \"task_settings\": {\n \"input_type\": \"ingest\",\n \"truncate\": \"start\"\n }\n}" PutNvidiaRequestExample3: summary: A completion task description: Run `PUT _inference/completion/nvidia-completion` to create an inference endpoint that performs a `completion` task. value: "{\n \"service\": \"nvidia\",\n \"service_settings\": {\n \"url\": \"nvidia-completion-url\",\n \"api_key\": \"nvidia-completion-token\",\n \"model_id\": \"microsoft/phi-3-mini-128k-instruct\"\n }\n}" PutNvidiaRequestExample4: summary: A completion task without `url` parameter description: Run `PUT _inference/completion/nvidia-completion` to create an inference endpoint that performs a `completion` task, omitting the `url` parameter if model is accessible via default NVIDIA endpoint. value: "{\n \"service\": \"nvidia\",\n \"service_settings\": {\n \"api_key\": \"nvidia-completion-token\",\n \"model_id\": \"microsoft/phi-3-mini-128k-instruct\"\n }\n}" PutNvidiaRequestExample5: summary: A chat completion task description: Run `PUT _inference/chat_completion/nvidia-chat-completion` to create an inference endpoint that performs a `chat_completion` task. value: "{\n \"service\": \"nvidia\",\n \"service_settings\": {\n \"url\": \"nvidia-chat-completion-url\",\n \"api_key\": \"nvidia-chat-completion-token\",\n \"model_id\": \"microsoft/phi-3-mini-128k-instruct\"\n }\n}" PutNvidiaRequestExample6: summary: A chat completion task without `url` parameter description: Run `PUT _inference/chat_completion/nvidia-chat-completion` to create an inference endpoint that performs a `chat_completion` task, omitting the `url` parameter if model is accessible via default NVIDIA endpoint. value: "{\n \"service\": \"nvidia\",\n \"service_settings\": {\n \"api_key\": \"nvidia-chat-completion-token\",\n \"model_id\": \"microsoft/phi-3-mini-128k-instruct\"\n }\n}" PutNvidiaRequestExample7: summary: A rerank task description: Run `PUT _inference/rerank/nvidia-rerank` to create an inference endpoint that performs a `rerank` task. value: "{\n \"service\": \"nvidia\",\n \"service_settings\": {\n \"url\": \"nvidia-rerank-url\",\n \"api_key\": \"nvidia-rerank-token\",\n \"model_id\": \"nv-rerank-qa-mistral-4b:1\"\n }\n}" PutNvidiaRequestExample8: summary: A rerank task without `url` parameter description: Run `PUT _inference/rerank/nvidia-rerank` to create an inference endpoint that performs a `rerank` task, omitting the `url` parameter if model is accessible via default NVIDIA endpoint. value: "{\n \"service\": \"nvidia\",\n \"service_settings\": {\n \"api_key\": \"nvidia-rerank-token\",\n \"model_id\": \"nv-rerank-qa-mistral-4b:1\"\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoNvidia' examples: PutNvidiaResponseExample1: summary: A text embedding task description: A successful response when creating an Nvidia `text_embedding` inference endpoint. value: "{\n \"inference_id\": \"nvidia-text-embedding\",\n \"task_type\": \"text_embedding\",\n \"service\": \"nvidia\",\n \"service_settings\": {\n \"model_id\": \"nvidia/llama-3.2-nv-embedqa-1b-v2\",\n \"url\": \"nvidia-embeddings-url\",\n \"rate_limit\": {\n \"requests_per_minute\": 3000\n },\n \"dimensions\": 2048,\n \"similarity\": \"dot_product\"\n },\n \"chunking_settings\": {\n \"strategy\": \"sentence\",\n \"max_chunk_size\": 250,\n \"sentence_overlap\": 1\n }\n}" PutNvidiaResponseExample2: summary: A text embedding task with custom `task_settings` and no `url` parameter description: A successful response when creating an Nvidia `text_embedding` inference endpoint with custom `task_settings` and no `url` parameter. value: "{\n \"inference_id\": \"nvidia-text-embedding\",\n \"task_type\": \"text_embedding\",\n \"service\": \"nvidia\",\n \"service_settings\": {\n \"model_id\": \"nvidia/llama-3.2-nv-embedqa-1b-v2\",\n \"rate_limit\": {\n \"requests_per_minute\": 3000\n },\n \"dimensions\": 2048,\n \"similarity\": \"dot_product\"\n },\n \"task_settings\": {\n \"input_type\": \"ingest\",\n \"truncate\": \"start\"\n },\n \"chunking_settings\": {\n \"strategy\": \"sentence\",\n \"max_chunk_size\": 250,\n \"sentence_overlap\": 1\n }\n}" PutNvidiaResponseExample3: summary: A completion task description: A successful response when creating an Nvidia `completion` inference endpoint. value: "{\n \"inference_id\": \"nvidia-completion\",\n \"task_type\": \"completion\",\n \"service\": \"nvidia\",\n \"service_settings\": {\n \"model_id\": \"microsoft/phi-3-mini-128k-instruct\",\n \"url\": \"nvidia-completion-url\",\n \"rate_limit\": {\n \"requests_per_minute\": 3000\n }\n }\n}" PutNvidiaResponseExample4: summary: A completion task without `url` parameter description: A successful response when creating an Nvidia `completion` inference endpoint without `url` parameter. value: "{\n \"inference_id\": \"nvidia-completion\",\n \"task_type\": \"completion\",\n \"service\": \"nvidia\",\n \"service_settings\": {\n \"model_id\": \"microsoft/phi-3-mini-128k-instruct\",\n \"rate_limit\": {\n \"requests_per_minute\": 3000\n }\n }\n}" PutNvidiaResponseExample5: summary: A chat completion task description: A successful response when creating an Nvidia `chat_completion` inference endpoint. value: "{\n \"inference_id\": \"nvidia-chat-completion\",\n \"task_type\": \"chat_completion\",\n \"service\": \"nvidia\",\n \"service_settings\": {\n \"model_id\": \"microsoft/phi-3-mini-128k-instruct\",\n \"url\": \"nvidia-chat-completion-url\",\n \"rate_limit\": {\n \"requests_per_minute\": 3000\n }\n }\n}" PutNvidiaResponseExample6: summary: A chat completion task without `url` parameter description: A successful response when creating an Nvidia `chat_completion` inference endpoint without `url` parameter. value: "{\n \"inference_id\": \"nvidia-chat-completion\",\n \"task_type\": \"chat_completion\",\n \"service\": \"nvidia\",\n \"service_settings\": {\n \"model_id\": \"microsoft/phi-3-mini-128k-instruct\",\n \"rate_limit\": {\n \"requests_per_minute\": 3000\n }\n }\n}" PutNvidiaResponseExample7: summary: A rerank task description: A successful response when creating an Nvidia `rerank` inference endpoint. value: "{\n \"inference_id\": \"nvidia-rerank\",\n \"task_type\": \"rerank\",\n \"service\": \"nvidia\",\n \"service_settings\": {\n \"model_id\": \"nv-rerank-qa-mistral-4b:1\",\n \"url\": \"nvidia-rerank-url\",\n \"rate_limit\": {\n \"requests_per_minute\": 3000\n }\n }\n}" PutNvidiaResponseExample8: summary: A rerank task without `url` parameter description: A successful response when creating an Nvidia `rerank` inference endpoint without `url` parameter. value: "{\n \"inference_id\": \"nvidia-rerank\",\n \"task_type\": \"rerank\",\n \"service\": \"nvidia\",\n \"service_settings\": {\n \"model_id\": \"nv-rerank-qa-mistral-4b:1\",\n \"rate_limit\": {\n \"requests_per_minute\": 3000\n }\n }\n}" x-state: Generally available; Added in 9.3.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{openai_inference_id}: put: tags: - inference summary: Create an OpenAI inference endpoint description: 'Create an inference endpoint to perform an inference task with the `openai` service or `openai` compatible APIs. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-openai parameters: - in: path name: task_type description: 'The type of the inference task that the model will perform. NOTE: The `chat_completion` task type only supports streaming and only through the _stream API.' required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.OpenAITaskType' style: simple - in: path name: openai_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `text_embedding` task type. Not applicable to the `completion` or `chat_completion` task types.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `openai`. allOf: - $ref: '#/components/schemas/inference._types.OpenAIServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `openai` service. allOf: - $ref: '#/components/schemas/inference._types.OpenAIServiceSettings' task_settings: description: 'Settings to configure the inference task. These settings are specific to the task type you specified.' allOf: - $ref: '#/components/schemas/inference._types.OpenAITaskSettings' required: - service - service_settings examples: PutOpenAiRequestExample1: summary: A text embedding task description: Run `PUT _inference/text_embedding/openai-embeddings` to create an inference endpoint that performs a `text_embedding` task. The embeddings created by requests to this endpoint will have 128 dimensions. value: "{\n \"service\": \"openai\",\n \"service_settings\": {\n \"api_key\": \"OpenAI-API-Key\",\n \"model_id\": \"text-embedding-3-small\",\n \"dimensions\": 128\n }\n}" PutOpenAiRequestExample2: summary: A completion task description: Run `PUT _inference/completion/openai-completion` to create an inference endpoint to perform a `completion` task type. value: "{\n \"service\": \"openai\",\n \"service_settings\": {\n \"api_key\": \"OpenAI-API-Key\",\n \"model_id\": \"gpt-3.5-turbo\"\n }\n}" PutOpenAiRequestExample3: summary: A text embedding task with OAuth 2.0 authorization description: Run `PUT _inference/text_embedding/openai-embeddings` to create an inference endpoint that performs a `text_embedding` task. The requests to this endpoint will use OAuth 2.0 authorization. value: "{\n \"service\": \"openai\",\n \"service_settings\": {\n \"model_id\": \"text-embedding-3-small\",\n \"client_id\": \"test_client_id\",\n \"scopes\": [\"scope1\", \"scope2\"],\n \"client_secret\": \"secret\",\n \"token_url\": \"http://localhost:8080/token\",\n \"url\": \"https://api.openai.com/v1/embeddings\"\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoOpenAI' x-state: Generally available; Added in 8.12.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{openshiftai_inference_id}: put: tags: - inference summary: Create an OpenShift AI inference endpoint description: 'Create an inference endpoint to perform an inference task with the `openshift_ai` service. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-openshift-ai parameters: - in: path name: task_type description: 'The type of the inference task that the model will perform. NOTE: The `chat_completion` task type only supports streaming and only through the _stream API.' required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.OpenShiftAiTaskType' style: simple - in: path name: openshiftai_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `text_embedding` task type. Not applicable to the `rerank`, `completion`, or `chat_completion` task types.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `openshift_ai`. allOf: - $ref: '#/components/schemas/inference._types.OpenShiftAiServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `openshift_ai` service. allOf: - $ref: '#/components/schemas/inference._types.OpenShiftAiServiceSettings' task_settings: description: 'Settings to configure the inference task. Applies only to the `rerank` task type. Not applicable to the `text_embedding`, `completion`, or `chat_completion` task types. These settings are specific to the task type you specified.' allOf: - $ref: '#/components/schemas/inference._types.OpenShiftAiTaskSettings' required: - service - service_settings examples: PutOpenShiftAiRequestExample1: summary: A text embedding task description: Run `PUT _inference/text_embedding/openshift-ai-text-embedding` to create an inference endpoint that performs a `text_embedding` task. value: "{\n \"service\": \"openshift_ai\",\n \"service_settings\": {\n \"url\": \"openshift-ai-embeddings-url\",\n \"api_key\": \"openshift-ai-embeddings-token\",\n \"model_id\": \"gritlm-7b\"\n }\n}" PutOpenShiftAiRequestExample2: summary: A completion task description: Run `PUT _inference/completion/openshift-ai-completion` to create an inference endpoint that performs a `completion` task. value: "{\n \"service\": \"openshift_ai\",\n \"service_settings\": {\n \"url\": \"openshift-ai-completion-url\",\n \"api_key\": \"openshift-ai-completion-token\",\n \"model_id\": \"llama-31-8b-instruct\"\n }\n}" PutOpenShiftAiRequestExample3: summary: A chat completion task description: Run `PUT _inference/chat_completion/openshift-ai-chat-completion` to create an inference endpoint that performs a `chat_completion` task. value: "{\n \"service\": \"openshift_ai\",\n \"service_settings\": {\n \"url\": \"openshift-ai-chat-completion-url\",\n \"api_key\": \"openshift-ai-chat-completion-token\",\n \"model_id\": \"llama-31-8b-instruct\"\n }\n}" PutOpenShiftAiRequestExample4: summary: A rerank task description: Run `PUT _inference/rerank/openshift-ai-rerank` to create an inference endpoint that performs a `rerank` task. value: "{\n \"service\": \"openshift_ai\",\n \"service_settings\": {\n \"url\": \"openshift-ai-rerank-url\",\n \"api_key\": \"openshift-ai-rerank-token\",\n \"model_id\": \"bge-reranker-v2-m3\"\n }\n}" PutOpenShiftAiRequestExample5: summary: A rerank task with custom `task_settings` and omitted `model_id` description: Run `PUT _inference/rerank/openshift-ai-rerank` to create an inference endpoint that performs a `rerank` task, specifying custom `task_settings` and omitting the `model_id` if deployed model doesn't require it. value: "{\n \"service\": \"openshift_ai\",\n \"service_settings\": {\n \"url\": \"openshift-ai-rerank-url\",\n \"api_key\": \"openshift-ai-rerank-token\"\n },\n \"task_settings\": {\n \"return_documents\": true,\n \"top_n\": 2\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoOpenShiftAi' examples: PutOpenShiftAiResponseExample1: summary: A text embedding task description: A successful response when creating an OpenShift AI `text_embedding` inference endpoint. value: "{\n \"inference_id\": \"openshift-ai-text-embedding\",\n \"task_type\": \"text_embedding\",\n \"service\": \"openshift_ai\",\n \"service_settings\": {\n \"model_id\": \"gritlm-7b\",\n \"url\": \"openshift-ai-embeddings-url\",\n \"rate_limit\": {\n \"requests_per_minute\": 3000\n },\n \"dimensions\": 4096,\n \"similarity\": \"dot_product\",\n \"dimensions_set_by_user\": false\n },\n \"chunking_settings\": {\n \"strategy\": \"sentence\",\n \"max_chunk_size\": 250,\n \"sentence_overlap\": 1\n }\n}" PutOpenShiftAiResponseExample2: summary: A completion task description: A successful response when creating an OpenShift AI `completion` inference endpoint. value: "{\n \"inference_id\": \"openshift-ai-completion\",\n \"task_type\": \"completion\",\n \"service\": \"openshift_ai\",\n \"service_settings\": {\n \"model_id\": \"llama-31-8b-instruct\",\n \"url\": \"openshift-ai-completion-url\",\n \"rate_limit\": {\n \"requests_per_minute\": 3000\n }\n }\n}" PutOpenShiftAiResponseExample3: summary: A chat completion task description: A successful response when creating an OpenShift AI `chat_completion` inference endpoint. value: "{\n \"inference_id\": \"openshift-ai-chat-completion\",\n \"task_type\": \"chat_completion\",\n \"service\": \"openshift_ai\",\n \"service_settings\": {\n \"model_id\": \"llama-31-8b-instruct\",\n \"url\": \"openshift-ai-chat-completion-url\",\n \"rate_limit\": {\n \"requests_per_minute\": 3000\n }\n }\n}" PutOpenShiftAiResponseExample4: summary: A rerank task description: A successful response when creating an OpenShift AI `rerank` inference endpoint. value: "{\n \"inference_id\": \"openshift-ai-rerank\",\n \"task_type\": \"rerank\",\n \"service\": \"openshift_ai\",\n \"service_settings\": {\n \"model_id\": \"bge-reranker-v2-m3\",\n \"url\": \"openshift-ai-rerank-url\",\n \"rate_limit\": {\n \"requests_per_minute\": 3000\n }\n }\n}" PutOpenShiftAiResponseExample5: summary: A rerank task with custom `task_settings` and omitted `model_id` description: A successful response when creating an OpenShift AI `rerank` inference endpoint with custom `task_settings` and omitted `model_id` value: "{\n \"inference_id\": \"openshift-ai-rerank\",\n \"task_type\": \"rerank\",\n \"service\": \"openshift_ai\",\n \"service_settings\": {\n \"url\": \"openshift-ai-rerank-url\",\n \"rate_limit\": {\n \"requests_per_minute\": 3000\n }\n },\n \"task_settings\": {\n \"return_documents\": true,\n \"top_n\": 2\n }\n}" x-state: Generally available; Added in 9.3.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{voyageai_inference_id}: put: tags: - inference summary: Create a VoyageAI inference endpoint description: 'Create an inference endpoint to perform an inference task with the `voyageai` service. Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-voyageai parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.VoyageAITaskType' style: simple - in: path name: voyageai_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `text_embedding` task type. Not applicable to the `rerank` task type.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `voyageai`. allOf: - $ref: '#/components/schemas/inference._types.VoyageAIServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `voyageai` service. allOf: - $ref: '#/components/schemas/inference._types.VoyageAIServiceSettings' task_settings: description: 'Settings to configure the inference task. These settings are specific to the task type you specified.' allOf: - $ref: '#/components/schemas/inference._types.VoyageAITaskSettings' required: - service - service_settings examples: PutVoyageAIRequestExample1: summary: A text embedding task description: Run `PUT _inference/text_embedding/voyageai-embeddings` to create an inference endpoint that performs a `text_embedding` task. The embeddings created by requests to this endpoint will have 512 dimensions. value: "{\n \"service\": \"voyageai\",\n \"service_settings\": {\n \"model_id\": \"voyage-3-large\",\n \"dimensions\": 512\n }\n}" PutVoyageAIRequestExample2: summary: A rerank task description: Run `PUT _inference/rerank/voyageai-rerank` to create an inference endpoint that performs a `rerank` task. value: "{\n \"service\": \"voyageai\",\n \"service_settings\": {\n \"model_id\": \"rerank-2\"\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoVoyageAI' x-state: Generally available; Added in 8.19.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{watsonx_inference_id}: put: tags: - inference summary: Create a Watsonx inference endpoint description: 'Create an inference endpoint to perform an inference task with the `watsonxai` service. You need an IBM Cloud Databases for Elasticsearch deployment to use the `watsonxai` inference service. You can provision one through the IBM catalog, the Cloud Databases CLI plug-in, the Cloud Databases API, or Terraform. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-put-watsonx parameters: - in: path name: task_type description: The type of the inference task that the model will perform. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.WatsonxTaskType' style: simple - in: path name: watsonx_inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference endpoint to be created. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: chunking_settings: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#infer-chunking-config description: 'The chunking configuration object. Applies only to the `text_embedding` task type. Not applicable to the `rerank`, `completion` or `chat_completion` task types.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The type of service supported for the specified task type. In this case, `watsonxai`. allOf: - $ref: '#/components/schemas/inference._types.WatsonxServiceType' service_settings: description: Settings used to install the inference model. These settings are specific to the `watsonxai` service. allOf: - $ref: '#/components/schemas/inference._types.WatsonxServiceSettings' required: - service - service_settings examples: PutWatsonxRequestExample1: description: Run `PUT _inference/text_embedding/watsonx-embeddings` to create an Watonsx inference endpoint that performs a text embedding task. value: "{\n \"service\": \"watsonxai\",\n \"service_settings\": {\n \"api_key\": \"Watsonx-API-Key\", \n \"url\": \"Wastonx-URL\", \n \"model_id\": \"ibm/slate-30m-english-rtrvr\",\n \"project_id\": \"IBM-Cloud-ID\", \n \"api_version\": \"2024-03-14\"\n }\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfoWatsonx' x-state: Generally available; Added in 8.16.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/rerank/{inference_id}: post: tags: - inference summary: Perform reranking inference on the service description: ' ## Required authorization * Cluster privileges: `monitor_inference` ' operationId: inference-rerank parameters: - in: path name: inference_id description: The unique identifier for the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: The amount of time to wait for the inference request to complete. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: query: description: "Query input.\nThe query can be specified as a single string, or as an object.\nThe object form additionally allows specifying non-text inputs, such as images.\n\n> info\n> Only the `elastic` service currently supports non-text queries for the `rerank` task. For all other services, the query must be a string.\n\nstring example:\n```\n\"query\": \"some query text\"\n```\nobject example:\n```\n\"query\": {\n \"type\": \"image\",\n \"format\": \"base64\",\n \"value\": \"data:image/jpeg;base64,...\"\n}\n```" allOf: - $ref: '#/components/schemas/inference.rerank.RerankQuery' input: description: "The documents to rank.\nThe input can be specified as a single string or an array of strings, or as an object or an array of objects.\nThe object form additionally allows specifying non-text inputs, such as images.\n\n> info\n> Only the `elastic` service currently supports non-text inputs for the `rerank` task. For all other services, the input must be a string or an array of strings.\n\nstring example:\n```\n\"input\": \"some document text\"\n```\nstring array example:\n```\n\"input\": [\"some document text\", \"some more document text\"]\n```\nobject example:\n```\n\"input\": {\n \"type\": \"image\",\n \"format\": \"base64\",\n \"value\": \"data:image/jpeg;base64,...\"\n}\n```\nobject array example:\n```\n\"input\": [\n {\n \"type\": \"text\",\n \"format\": \"text\",\n \"value\": \"some document text\"\n },\n {\n \"type\": \"image\",\n \"format\": \"base64\",\n \"value\": \"data:image/jpeg;base64,...\"\n }\n]\n```" allOf: - $ref: '#/components/schemas/inference.rerank.RerankInput' return_documents: description: Include the document text in the response. type: boolean top_n: description: Limit the response to the top N documents. type: number task_settings: description: 'Task settings for the individual inference request. These settings are specific to the task type you specified and override the task settings specified when initializing the service.' allOf: - $ref: '#/components/schemas/inference._types.TaskSettings' required: - query - input examples: RerankRequestExample1: summary: Rerank task description: Run `POST _inference/rerank/cohere_rerank` to perform reranking on the example input. value: "{\n \"input\": [\"luke\", \"like\", \"leia\", \"chewy\",\"r2d2\", \"star\", \"wars\"],\n \"query\": \"star wars main character\"\n}" RerankRequestExample2: summary: Rerank task description: Run `POST _inference/rerank/bge-reranker-base-mkn` to perform reranking on the example input via Hugging Face value: "{\n \"input\": [\"luke\", \"like\", \"leia\", \"chewy\",\"r2d2\", \"star\", \"wars\"],\n \"query\": \"star wars main character\",\n \"return_documents\": false,\n \"top_n\": 2\n}" RerankRequestExample3: summary: Rerank task description: Run `POST _inference/rerank/bge-reranker-base-mkn` to perform reranking on the example input via Hugging Face value: "{\n \"input\": [\"luke\", \"like\", \"leia\", \"chewy\",\"r2d2\", \"star\", \"wars\"],\n \"query\": \"star wars main character\",\n \"return_documents\": true,\n \"top_n\": 3\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.RerankedInferenceResult' examples: RerankResponseExample1: summary: Rerank task description: 'A successful response from `POST _inference/rerank/cohere_rerank`. ' value: "{\n \"rerank\": [\n {\n \"index\": \"2\",\n \"relevance_score\": \"0.011597361\",\n \"text\": \"leia\"\n },\n {\n \"index\": \"0\",\n \"relevance_score\": \"0.006338922\",\n \"text\": \"luke\"\n },\n {\n \"index\": \"5\",\n \"relevance_score\": \"0.0016166499\",\n \"text\": \"star\"\n },\n {\n \"index\": \"4\",\n \"relevance_score\": \"0.0011695103\",\n \"text\": \"r2d2\"\n },\n {\n \"index\": \"1\",\n \"relevance_score\": \"5.614787E-4\",\n \"text\": \"like\"\n },\n {\n \"index\": \"6\",\n \"relevance_score\": \"3.7850367E-4\",\n \"text\": \"wars\"\n },\n {\n \"index\": \"3\",\n \"relevance_score\": \"1.2508839E-5\",\n \"text\": \"chewy\"\n }\n ]\n}" RerankResponseExample2: summary: Rerank task description: 'A successful response from `POST _inference/rerank/bge-reranker-base-mkn`. ' value: "{\n \"rerank\": [\n {\n \"index\": 6,\n \"relevance_score\": 0.50955844\n },\n {\n \"index\": 5,\n \"relevance_score\": 0.084341794\n }\n ]\n}" RerankResponseExample3: summary: Rerank task description: 'A successful response from `POST _inference/rerank/bge-reranker-base-mkn`. ' value: "{\n \"rerank\": [\n {\n \"index\": 6,\n \"relevance_score\": 0.50955844,\n \"text\": \"wars\"\n },\n {\n \"index\": 5,\n \"relevance_score\": 0.084341794,\n \"text\": \"star\"\n },\n {\n \"index\": 3,\n \"relevance_score\": 0.004520818,\n \"text\": \"chewy\"\n }\n ]\n}" x-state: Generally available; Added in 8.11.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/sparse_embedding/{inference_id}: post: tags: - inference summary: Perform sparse embedding inference on the service operationId: inference-sparse-embedding parameters: - in: path name: inference_id description: The inference Id required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference request to complete. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: input: description: 'Inference input. Either a string or an array of strings.' oneOf: - type: string - type: array items: type: string task_settings: description: Task settings for the individual inference request. These settings are specific to the you specified and override the task settings specified when initializing the service. allOf: - $ref: '#/components/schemas/inference._types.TaskSettings' required: - input examples: SparseEmbeddingRequestExample1: summary: Sparse embedding task description: Run `POST _inference/sparse_embedding/my-elser-model` to perform sparse embedding on the example sentence. value: "{\n \"input\": \"The sky above the port was the color of television tuned to a dead channel.\"\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.SparseEmbeddingInferenceResult' examples: SparseEmbeddingResponseExample1: summary: Sparse embedding task description: 'An abbreviated response from `POST _inference/sparse_embedding/my-elser-model`. ' value: "{\n \"sparse_embedding\": [\n {\n \"is_truncated\": false,\n \"embedding\": {\n \"port\": 2.1259406,\n \"sky\": 1.7073475,\n \"color\": 1.6922266,\n \"dead\": 1.6247464,\n \"television\": 1.3525393,\n \"above\": 1.2425821,\n \"tuned\": 1.1440028,\n \"colors\": 1.1218185,\n \"tv\": 1.0111054,\n \"ports\": 1.0067928,\n \"poem\": 1.0042328,\n \"channel\": 0.99471164,\n \"tune\": 0.96235967,\n \"scene\": 0.9020516\n }\n }\n ]\n}" x-state: Generally available; Added in 8.11.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/completion/{inference_id}/_stream: post: tags: - inference summary: Perform streaming completion inference on the service description: 'Get real-time responses for completion tasks by delivering answers incrementally, reducing response times during computation. This API works only with the completion task type. IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs. This API requires the `monitor_inference` cluster privilege (the built-in `inference_admin` and `inference_user` roles grant this privilege). You must use a client that supports streaming. ## Required authorization * Cluster privileges: `monitor_inference` ' operationId: inference-stream-completion parameters: - in: path name: inference_id description: The unique identifier for the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: The amount of time to wait for the inference request to complete. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: input: description: 'The text on which you want to perform the inference task. It can be a single string or an array. NOTE: Inference endpoints for the completion task type currently only support a single string as input.' oneOf: - type: string - type: array items: type: string task_settings: description: Task settings for the individual inference request. These settings are specific to the you specified and override the task settings specified when initializing the service. allOf: - $ref: '#/components/schemas/inference._types.TaskSettings' required: - input examples: StreamInferenceRequestExample1: summary: Perform a completion task description: Run `POST _inference/completion/openai-completion/_stream` to perform a completion on the example question with streaming. value: "{\n \"input\": \"What is Elastic?\"\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/_types.StreamResult' x-state: Generally available; Added in 8.16.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/text_embedding/{inference_id}: post: tags: - inference summary: Perform text embedding inference on the service operationId: inference-text-embedding parameters: - in: path name: inference_id description: The inference Id required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple - in: query name: timeout description: Specifies the amount of time to wait for the inference request to complete. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form requestBody: content: application/json: schema: type: object properties: input: description: 'Inference input. Either a string or an array of strings.' oneOf: - type: string - type: array items: type: string input_type: description: 'The input data type for the text embedding model. Possible values include: * `SEARCH` * `INGEST` * `CLASSIFICATION` * `CLUSTERING` Not all services support all values. Unsupported values will trigger a validation exception. Accepted values depend on the configured inference service, refer to the relevant service-specific documentation for more info. > info > The `input_type` parameter specified on the root level of the request body will take precedence over the `input_type` parameter specified in `task_settings`.' type: string task_settings: description: Task settings for the individual inference request. These settings are specific to the you specified and override the task settings specified when initializing the service. allOf: - $ref: '#/components/schemas/inference._types.TaskSettings' required: - input examples: TextEmbeddingRequestExample1: summary: Text embedding task description: Run `POST _inference/text_embedding/my-cohere-endpoint` to perform text embedding on the example sentence using the Cohere integration, value: "{\n \"input\": \"The sky above the port was the color of television tuned to a dead channel.\",\n \"input_type\": \"ingest\"\n}" required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.TextEmbeddingInferenceResult' examples: TextEmbeddingResponseExample1: summary: Text embedding task description: 'An abbreviated response from `POST _inference/text_embedding/my-text-embedding-endpoint`. ' value: "{\n \"text_embedding\": [\n {\n \"embedding\": [\n 0.018569946,\n -0.036895752,\n 0.01486969,\n -0.0045204163,\n -0.04385376,\n 0.0075950623,\n 0.04260254,\n -0.004005432,\n 0.007865906,\n 0.030792236,\n -0.050476074,\n 0.011795044,\n -0.011642456,\n -0.010070801\n ]\n }\n ]\n}" x-state: Generally available; Added in 8.11.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{inference_id}/_update: put: tags: - inference summary: Update an inference endpoint description: 'Modify `task_settings`, secrets (within `service_settings`), or `num_allocations` for an inference endpoint, depending on the specific endpoint service and `task_type`. IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-update parameters: - $ref: '#/components/parameters/inference.update-inference_id' - $ref: '#/components/parameters/inference.update-timeout' requestBody: $ref: '#/components/requestBodies/inference.update' responses: '200': $ref: '#/components/responses/inference.update-200' x-state: Generally available; Added in 8.17.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name /_inference/{task_type}/{inference_id}/_update: put: tags: - inference summary: Update an inference endpoint description: 'Modify `task_settings`, secrets (within `service_settings`), or `num_allocations` for an inference endpoint, depending on the specific endpoint service and `task_type`. IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs. ## Required authorization * Cluster privileges: `manage_inference` ' operationId: inference-update-1 parameters: - $ref: '#/components/parameters/inference.update-task_type' - $ref: '#/components/parameters/inference.update-inference_id' - $ref: '#/components/parameters/inference.update-timeout' requestBody: $ref: '#/components/requestBodies/inference.update' responses: '200': $ref: '#/components/responses/inference.update-200' x-state: Generally available; Added in 8.17.0 x-metaTags: - content: Elasticsearch, Machine Learning name: product_name components: schemas: inference._types.FireworksAITaskType: type: string enum: - chat_completion - completion - text_embedding inference._types.ThinkingConfig: type: object properties: thinking_budget: description: Indicates the desired thinking budget in tokens. type: number inference._types.AmazonBedrockTaskSettings: type: object properties: max_new_tokens: description: For `chat_completion` and `completion` tasks, it sets the maximum number for the output tokens to be generated. default: 64.0 type: number temperature: description: 'For `chat_completion` and `completion` tasks, it is a number between 0.0 and 1.0 that controls the apparent creativity of the results. At temperature 0.0 the model is most deterministic, at temperature 1.0 most random. It should not be used if `top_p` or `top_k` is specified.' type: number top_k: description: 'For `chat_completion` and `completion` tasks, it limits samples to the top-K most likely words, balancing coherence and variability. It is only available for anthropic, cohere, and mistral providers. It is an alternative to `temperature`; it should not be used if `temperature` is specified.' type: number top_p: description: 'For `chat_completion` and `completion` tasks, it is a number in the range of 0.0 to 1.0, to eliminate low-probability tokens. Top-p uses nucleus sampling to select top tokens whose sum of likelihoods does not exceed a certain value, ensuring both variety and coherence. It is an alternative to `temperature`; it should not be used if `temperature` is specified.' type: number inference._types.CohereSimilarityType: type: string enum: - cosine - dot_product - l2_norm inference._types.DenseEmbeddingResult: description: The dense embedding result object for float representation type: object properties: embedding: allOf: - $ref: '#/components/schemas/inference._types.DenseVector' required: - embedding _types.AcknowledgedResponseBase: type: object properties: acknowledged: description: For a successful response, this value is always true. On failure, an exception is returned instead. type: boolean required: - acknowledged inference._types.CohereTruncateType: type: string enum: - END - NONE - START inference._types.AmazonBedrockServiceSettings: type: object properties: access_key: description: A valid AWS access key that has permissions to use Amazon Bedrock and access to models for inference requests. type: string model: externalDocs: url: https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html description: 'The base model ID or an ARN to a custom model based on a foundational model. The base model IDs can be found in the Amazon Bedrock documentation. Note that the model ID must be available for the provider chosen and your IAM user must have access to the model.' type: string provider: description: 'The model provider for your deployment. Note that some providers may support only certain task types. Supported providers include: * `amazontitan` - available for `text_embedding` and `completion` task types * `anthropic` - available for `chat_completion` and `completion` task types * `ai21labs` - available for `chat_completion` and `completion` task types * `cohere` - available for `chat_completion`, `completion` and `text_embedding` task types * `meta` - available for `chat_completion` and `completion` task types * `mistral` - available for `chat_completion` and `completion` task types' type: string region: externalDocs: url: https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html description: 'The region that your model or ARN is deployed in. The list of available regions per model can be found in the Amazon Bedrock documentation.' type: string rate_limit: description: 'This setting helps to minimize the number of rate limit errors returned from Amazon Bedrock. By default, the `amazonbedrock` service sets the number of requests allowed per minute to 240.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' secret_key: externalDocs: url: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html description: 'A valid AWS secret key that is paired with the `access_key`. For informationg about creating and managing access and secret keys, refer to the AWS documentation.' type: string required: - access_key - model - region - secret_key inference._types.TaskTypeAmazonBedrock: type: string enum: - chat_completion - completion - text_embedding inference.rerank.RerankInputObject: description: An object describing a single input for the `rerank` task, which additionally allows specifying non-text inputs, such as images. type: object properties: type: description: The type of input. Not all services and models support all input types. allOf: - $ref: '#/components/schemas/inference.rerank.RerankInputType' format: description: 'The format of the input. For the `text` type this must be `text`. For the `image` type this must be `base64`. If not specified, this defaults to `text` for the `text` type and `base64` for the `image` type.' allOf: - $ref: '#/components/schemas/inference.rerank.RerankInputFormat' value: description: The value of the input. For images, this must be a base64-encoded data URI, that is, "data:content/type;base64,...". type: string required: - type - value inference._types.InferenceEndpointInfoElasticsearch: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeElasticsearch' required: - inference_id - task_type inference._types.ContextualAIServiceType: type: string enum: - contextualai inference._types.AzureAiStudioServiceSettings: type: object properties: api_key: externalDocs: url: https://ai.azure.com/ description: 'A valid API key of your Azure AI Studio model deployment. This key can be found on the overview page for your deployment in the management section of your Azure AI Studio account. IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key.' type: string endpoint_type: externalDocs: url: https://learn.microsoft.com/en-us/azure/ai-foundry/concepts/deployments-overview#billing-for-deploying-and-inferencing-llms-in-azure-ai-studio description: 'The type of endpoint that is available for deployment through Azure AI Studio: `token` or `realtime`. The `token` endpoint type is for "pay as you go" endpoints that are billed per token. The `realtime` endpoint type is for "real-time" endpoints that are billed per hour of usage.' type: string target: description: 'The target URL of your Azure AI Studio model deployment. This can be found on the overview page for your deployment in the management section of your Azure AI Studio account.' type: string provider: description: 'The model provider for your deployment. Note that some providers may support only certain task types. Supported providers include: * `cohere` - available for `text_embedding`, `rerank` and `completion` task types * `databricks` - available for `completion` task type only * `meta` - available for `completion` task type only * `microsoft_phi` - available for `completion` task type only * `mistral` - available for `completion` task type only * `openai` - available for `text_embedding` and `completion` task types' type: string rate_limit: description: 'This setting helps to minimize the number of rate limit errors returned from Azure AI Studio. By default, the `azureaistudio` service sets the number of requests allowed per minute to 240.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' required: - api_key - endpoint_type - target - provider inference._types.CompletionToolFunction: description: The completion tool function definition. type: object properties: description: description: 'A description of what the function does. This is used by the model to choose when and how to call the function.' type: string name: description: The name of the function. type: string parameters: description: The parameters the functional accepts. This should be formatted as a JSON object. type: object strict: description: Whether to enable schema adherence when generating the function call. type: boolean required: - name inference._types.VoyageAIServiceSettings: type: object properties: dimensions: externalDocs: url: https://docs.voyageai.com/docs/embeddings description: 'The number of dimensions for resulting output embeddings. This setting maps to `output_dimension` in the VoyageAI documentation. Only for the `text_embedding` task type.' type: number model_id: externalDocs: url: https://docs.voyageai.com/docs/reranker description: 'The name of the model to use for the inference task. Refer to the VoyageAI documentation for the list of available text embedding and rerank models.' type: string rate_limit: description: 'This setting helps to minimize the number of rate limit errors returned from VoyageAI. The `voyageai` service sets a default number of requests allowed per minute depending on the task type. For both `text_embedding` and `rerank`, it is set to `2000`.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' embedding_type: externalDocs: url: https://docs.voyageai.com/docs/embeddings description: 'The data type for the embeddings to be returned. This setting maps to `output_dtype` in the VoyageAI documentation. Permitted values: float, int8, bit. `int8` is a synonym of `byte` in the VoyageAI documentation. `bit` is a synonym of `binary` in the VoyageAI documentation. Only for the `text_embedding` task type.' type: number required: - model_id inference._types.GoogleAiStudioTaskType: type: string enum: - completion - text_embedding _types.StreamResult: type: object _types.EpochTimeUnitMillis: allOf: - $ref: '#/components/schemas/_types.UnitMillis' inference._types.EncryptedReasoningDetail: description: The reasoning detail with encrypted reasoning data that may be redacted or protected. allOf: - $ref: '#/components/schemas/inference._types.BaseReasoningDetail' - type: object properties: type: type: string enum: - reasoning.encrypted data: description: 'The encrypted reasoning data generated by the model, which may be redacted or protected based on the model''s configuration and the nature of the reasoning information.' type: string required: - type - data inference._types.InferenceEndpointInfoOpenShiftAi: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeOpenShiftAi' required: - inference_id - task_type inference._types.OpenShiftAiServiceType: type: string enum: - openshift_ai inference._types.HuggingFaceServiceSettings: type: object properties: api_key: externalDocs: url: https://huggingface.co/settings/tokens description: 'A valid access token for your HuggingFace account. You can create or find your access tokens on the HuggingFace settings page. IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key.' type: string rate_limit: description: 'This setting helps to minimize the number of rate limit errors returned from Hugging Face. By default, the `hugging_face` service sets the number of requests allowed per minute to 3000 for all supported tasks. Hugging Face does not publish a universal rate limit โ€” actual limits may vary. It is recommended to adjust this value based on the capacity and limits of your specific deployment environment.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' url: externalDocs: url: https://huggingface.co/docs/inference-providers/en/tasks/chat-completion#conversational-large-language-models-llms description: 'The URL endpoint to use for the requests. For `completion` and `chat_completion` tasks, the deployed model must be compatible with the Hugging Face Chat Completion interface (see the linked external documentation for details). The endpoint URL for the request must include `/v1/chat/completions`. If the model supports the OpenAI Chat Completion schema, a toggle should appear in the interface. Enabling this toggle doesn''t change any model behavior, it reveals the full endpoint URL needed (which should include `/v1/chat/completions`) when configuring the inference endpoint in Elasticsearch. If the model doesn''t support this schema, the toggle may not be shown.' type: string model_id: description: 'The name of the HuggingFace model to use for the inference task. For `completion` and `chat_completion` tasks, this field is optional but may be required for certain models โ€” particularly when using serverless inference endpoints. For the `text_embedding` task, this field should not be included. Otherwise, the request will fail.' type: string required: - api_key - url inference._types.TaskTypeOpenShiftAi: type: string enum: - text_embedding - chat_completion - completion - rerank inference._types.TaskTypeCustom: type: string enum: - text_embedding - sparse_embedding - rerank - completion inference._types.ContentObject: description: An object style representation of a single portion of a conversation. type: object properties: type: description: 'The type of content. Must be one of `text`, `image_url` or `file`. Not all services/models support content types other than "text"' allOf: - $ref: '#/components/schemas/inference._types.ContentType' text: description: The text content. Only applicable for the `text` type type: string image_url: description: The image content. Only applicable for the `image_url` type allOf: - $ref: '#/components/schemas/inference._types.ImageUrl' file: description: The file content. Only applicable for the `file` type allOf: - $ref: '#/components/schemas/inference._types.FileContent' required: - type - text - image_url - file inference._types.AzureAiStudioTaskType: type: string enum: - completion - rerank - text_embedding inference._types.ElserServiceType: type: string enum: - elser inference._types.CohereEmbeddingType: type: string enum: - binary - bit - byte - float - int8 inference._types.GoogleModelGardenProvider: type: string enum: - google - anthropic - meta - hugging_face - mistral - ai21 inference._types.NvidiaServiceSettings: type: object properties: api_key: description: 'A valid API key for your Nvidia endpoint. Can be found in `API Keys` section of Nvidia account settings.' type: string url: description: 'The URL of the Nvidia model endpoint. If not provided, the default endpoint URL is used depending on the task type: * For `text_embedding` task - `https://integrate.api.nvidia.com/v1/embeddings`. * For `completion` and `chat_completion` tasks - `https://integrate.api.nvidia.com/v1/chat/completions`. * For `rerank` task - `https://ai.api.nvidia.com/v1/retrieval/nvidia/reranking`.' type: string model_id: description: 'The name of the model to use for the inference task. Refer to the model''s documentation for the name if needed. Service has been tested and confirmed to be working with the following models: * For `text_embedding` task - `nvidia/llama-3.2-nv-embedqa-1b-v2`. * For `completion` and `chat_completion` tasks - `microsoft/phi-3-mini-128k-instruct`. * For `rerank` task - `nv-rerank-qa-mistral-4b:1`. Service doesn''t support `text_embedding` task `baai/bge-m3` and `nvidia/nvclip` models due to them not recognizing the `input_type` parameter.' type: string max_input_tokens: description: For a `text_embedding` task, the maximum number of tokens per input. Inputs exceeding this value are truncated prior to sending to the Nvidia API. type: number similarity: description: For a `text_embedding` task, the similarity measure. One of cosine, dot_product, l2_norm. allOf: - $ref: '#/components/schemas/inference._types.NvidiaSimilarityType' rate_limit: description: 'This setting helps to minimize the number of rate limit errors returned from the Nvidia API. By default, the `nvidia` service sets the number of requests allowed per minute to 3000.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' required: - api_key - model_id inference._types.EmbeddingContentObjectItem: description: An object containing the input data for a single item for the model to embed. type: object properties: type: description: 'The type of input to embed. Not all models support all input types. The `audio`, `video`, and `pdf` types are available in Elasticsearch 9.5.0 and later.' allOf: - $ref: '#/components/schemas/inference._types.EmbeddingContentType' format: description: 'The format of the input. For the `text` type this must be `text`. For all other types, this must be `base64`. If not specified, this will default to `text` for the `text` type and `base64` for all other types.' allOf: - $ref: '#/components/schemas/inference._types.EmbeddingContentFormat' value: description: The value of the input to embed. For images, this must be a base64-encoded data URI, i.e. "data:content/type;base64,..." type: string required: - type - value inference._types.FireworksAITaskSettings: type: object properties: user: description: 'For a `completion` or`chat_completion` task, specify the user issuing the request. This information can be used for abuse detection.' type: string headers: description: "For a `completion` or`chat_completion` task. Specifies custom HTTP header parameters.\nFor example:\n```\n\"headers\": {\n \"Custom-Header\": \"Some-Value\",\n \"Another-Custom-Header\": \"Another-Value\"\n}\n```" type: object additionalProperties: type: string inference._types.CohereTaskType: type: string enum: - completion - rerank - text_embedding _types.byte: type: number inference._types.InferenceEndpoint: description: Configuration options when storing the inference endpoint type: object properties: chunking_settings: description: 'The chunking configuration object. Applies only to the `embedding`, `sparse_embedding` and `text_embedding` task types. Not applicable to the `rerank`, `completion`, or `chat_completion` task types.' allOf: - $ref: '#/components/schemas/inference._types.InferenceChunkingSettings' service: description: The service type type: string service_settings: description: Settings specific to the service allOf: - $ref: '#/components/schemas/inference._types.ServiceSettings' task_settings: description: Task settings specific to the service and task type allOf: - $ref: '#/components/schemas/inference._types.TaskSettings' required: - service - service_settings inference._types.TaskTypeContextualAI: type: string enum: - rerank inference._types.TaskType: type: string enum: - sparse_embedding - text_embedding - rerank - completion - chat_completion - embedding _types.DateTime: description: 'A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.' oneOf: - type: string - $ref: '#/components/schemas/_types.EpochTimeUnitMillis' inference._types.TaskTypeFireworksAI: type: string enum: - chat_completion - completion - text_embedding inference._types.InferenceEndpointInfoJinaAi: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeJinaAi' required: - inference_id - task_type inference.rerank.RerankQuery: description: 'Query input for the `rerank` task. Either a string, or an object. The object form additionally allows specifying non-text inputs, such as images. > info > Only the `elastic` service currently supports non-text queries for the `rerank` task.' oneOf: - type: string - $ref: '#/components/schemas/inference.rerank.RerankInputObject' inference._types.FireworksAIServiceSettings: type: object properties: api_key: externalDocs: url: https://docs.fireworks.ai/ description: 'A valid API key for your Fireworks AI account. You can find or create your API keys in the Fireworks AI dashboard. IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key.' type: string model_id: externalDocs: url: https://docs.fireworks.ai/guides/recommended-models description: 'The name of the model to use for the inference task. Refer to the Fireworks AI documentation for the list of available models for chat completion, completion, and text embedding. For text embedding, supported models include the Qwen3 embedding family (e.g. `fireworks/qwen3-embedding-8b`) and other models in the Fireworks model library.' type: string url: description: 'The URL endpoint to use for the requests. If not provided, the default Fireworks AI API endpoint is used.' type: string dimensions: description: 'For a `text_embedding` task, the number of dimensions the resulting output embeddings should have. Variable-length embeddings are supported via this parameter.' type: number similarity: externalDocs: url: https://docs.fireworks.ai/guides/querying-embeddings-models description: For a `text_embedding` task, the similarity measure. One of cosine, dot_product, l2_norm. default: cosine allOf: - $ref: '#/components/schemas/inference._types.FireworksAISimilarityType' rate_limit: externalDocs: url: https://docs.fireworks.ai/guides/quotas_usage/rate-limits description: 'This setting helps to minimize the number of rate limit errors returned from the Fireworks AI API. Rate limit grouping is per API key only. By default, the `fireworksai` service sets the number of requests allowed per minute to 6000.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' required: - api_key - model_id inference._types.RankedDocument: description: 'The rerank result object representing a single ranked document id: the original index of the document in the request relevance_score: the relevance_score of the document relative to the query text: Optional, the text of the document, if requested' type: object properties: index: type: number relevance_score: type: number text: type: string required: - index - relevance_score inference._types.CohereServiceSettings: type: object properties: api_key: externalDocs: url: https://dashboard.cohere.com/api-keys description: 'A valid API key for your Cohere account. You can find or create your Cohere API keys on the Cohere API key settings page. IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key.' type: string embedding_type: description: 'For a `text_embedding` task, the types of embeddings you want to get back. Use `binary` for binary embeddings, which are encoded as bytes with signed int8 precision. Use `bit` for binary embeddings, which are encoded as bytes with signed int8 precision (this is a synonym of `binary`). Use `byte` for signed int8 embeddings (this is a synonym of `int8`). Use `float` for the default float embeddings. Use `int8` for signed int8 embeddings.' default: float allOf: - $ref: '#/components/schemas/inference._types.CohereEmbeddingType' model_id: description: 'For a `completion`, `rerank`, or `text_embedding` task, the name of the model to use for the inference task. * For the available `completion` models, refer to the [Cohere command docs](https://docs.cohere.com/docs/models#command). * For the available `rerank` models, refer to the [Cohere rerank docs](https://docs.cohere.com/reference/rerank-1). * For the available `text_embedding` models, refer to [Cohere embed docs](https://docs.cohere.com/reference/embed).' type: string rate_limit: description: 'This setting helps to minimize the number of rate limit errors returned from Cohere. By default, the `cohere` service sets the number of requests allowed per minute to 10000.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' similarity: description: 'The similarity measure. If the `embedding_type` is `float`, the default value is `dot_product`. If the `embedding_type` is `int8` or `byte`, the default value is `cosine`.' allOf: - $ref: '#/components/schemas/inference._types.CohereSimilarityType' required: - api_key - model_id inference._types.RerankedInferenceResult: description: Defines the response for a rerank request. type: object properties: rerank: type: array items: $ref: '#/components/schemas/inference._types.RankedDocument' required: - rerank inference._types.ToolCallFunction: description: The function that the model called. type: object properties: arguments: description: The arguments to call the function with in JSON format. type: string name: description: The name of the function to call. type: string required: - arguments - name inference._types.InferenceResult: description: InferenceResult is an aggregation of mutually exclusive variants type: object properties: embeddings_bytes: type: array items: $ref: '#/components/schemas/inference._types.DenseEmbeddingByteResult' embeddings_bits: type: array items: $ref: '#/components/schemas/inference._types.DenseEmbeddingByteResult' embeddings: type: array items: $ref: '#/components/schemas/inference._types.DenseEmbeddingResult' text_embedding_bytes: type: array items: $ref: '#/components/schemas/inference._types.DenseEmbeddingByteResult' text_embedding_bits: type: array items: $ref: '#/components/schemas/inference._types.DenseEmbeddingByteResult' text_embedding: type: array items: $ref: '#/components/schemas/inference._types.DenseEmbeddingResult' sparse_embedding: type: array items: $ref: '#/components/schemas/inference._types.SparseEmbeddingResult' completion: type: array items: $ref: '#/components/schemas/inference._types.CompletionResult' rerank: type: array items: $ref: '#/components/schemas/inference._types.RankedDocument' minProperties: 1 maxProperties: 1 inference._types.LlamaSimilarityType: type: string enum: - cosine - dot_product - l2_norm inference._types.AnthropicServiceType: type: string enum: - anthropic inference._types.TextReasoningDetail: description: The reasoning text detail includes plaintext reasoning with optional signature verification. allOf: - $ref: '#/components/schemas/inference._types.BaseReasoningDetail' - type: object properties: type: type: string enum: - reasoning.text signature: description: 'The signature of the reasoning text, which can be used to verify the authenticity and integrity of the reasoning information provided by the model.' type: string text: description: 'The plaintext reasoning generated by the model, which provides a detailed explanation of the model''s reasoning process in human-readable form.' type: string required: - type inference._types.DenseByteVector: description: 'Dense Embedding results containing bytes are represented as Dense Vectors of bytes.' type: array items: $ref: '#/components/schemas/_types.byte' inference._types.RequestEmbedding: type: object properties: input: description: "Inference input.\nEither a string, an array of strings, a `content` object, or an array of `content` objects.\n`content` objects may contain a single item or an array of items. Models that support multiple items per `content`\nobject will return a single embedding for each `content` object, regardless of how many items it contains.\nSupport for multiple items in a single `content` object is available in Elasticsearch 9.5.0 and later.\n\nstring example:\n```\n\"input\": \"Some text\"\n```\nstring array example:\n```\n\"input\": [\"Some text\", \"Some more text\"]\n```\n`content` object example:\n```\n\"input\": {\n \"content\": {\n \"type\": \"image\",\n \"format\": \"base64\",\n \"value\": \"data:image/jpeg;base64,...\"\n }\n }\n```\n`content` object array example:\n```\n\"input\": [\n {\n \"content\": {\n \"type\": \"text\",\n \"format\": \"text\",\n \"value\": \"Some text to generate an embedding\"\n }\n },\n {\n \"content\": {\n \"type\": \"image\",\n \"format\": \"base64\",\n \"value\": \"data:image/jpeg;base64,...\"\n }\n }\n]\n```\nMultiple items in one `content` object example (available in Elasticsearch 9.5.0 and later):\n```\n\"input\": [\n {\n \"content\": [\n {\n \"type\": \"image\",\n \"format\": \"base64\",\n \"value\": \"data:image/jpeg;base64,...\"\n },\n {\n \"type\": \"text\",\n \"value\": \"Some text to create an embedding\"\n }\n ]\n }\n]\n```" allOf: - $ref: '#/components/schemas/inference._types.EmbeddingInput' input_type: description: 'The input data type for the embedding model. Possible values include: * `SEARCH` * `INGEST` * `CLASSIFICATION` * `CLUSTERING` Not all models support all values. Unsupported values will trigger a validation exception. Accepted values depend on the configured inference service, refer to the relevant service-specific documentation for more info. > info > The `input_type` parameter specified on the root level of the request body will take precedence over the `input_type` parameter specified in `task_settings`.' type: string task_settings: description: Task settings for the individual inference request. These settings are specific to the you specified and override the task settings specified when initializing the service. allOf: - $ref: '#/components/schemas/inference._types.TaskSettings' required: - input inference._types.JinaAITextEmbeddingTask: type: string enum: - classification - clustering - ingest - search inference._types.EmbeddingContentObjectGroup: description: 'Allows specifying one or multiple items for the `embedding` task, which should result in a single embedding vector. Support for multiple items is available in Elasticsearch 9.5.0 and later.' oneOf: - $ref: '#/components/schemas/inference._types.EmbeddingContentObjectItem' - type: array items: $ref: '#/components/schemas/inference._types.EmbeddingContentObjectItem' inference._types.InferenceEndpointInfoAlibabaCloudAI: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeAlibabaCloudAI' required: - inference_id - task_type inference._types.DeleteInferenceEndpointResult: description: Acknowledged response. For dry_run, contains the list of pipelines which reference the inference endpoint allOf: - $ref: '#/components/schemas/_types.AcknowledgedResponseBase' - type: object properties: pipelines: type: array items: type: string required: - pipelines inference._types.CompletionToolType: oneOf: - type: string - $ref: '#/components/schemas/inference._types.CompletionToolChoice' inference._types.CompletionToolChoiceFunction: description: The tool choice function. type: object properties: name: description: The name of the function to call. type: string required: - name inference._types.JinaAITaskSettings: type: object properties: return_documents: description: For a `rerank` task, return the doc text within the results. type: boolean input_type: description: 'For an `embedding` or `text_embedding` task, the task passed to the model. Valid values are: * `classification`: Use it for embeddings passed through a classifier. * `clustering`: Use it for the embeddings run through a clustering algorithm. * `ingest`: Use it for storing document embeddings in a vector database. * `search`: Use it for storing embeddings of search queries run against a vector database to find relevant documents.' allOf: - $ref: '#/components/schemas/inference._types.JinaAITextEmbeddingTask' late_chunking: description: 'For an `embedding` or `text_embedding` task, controls when text is split into chunks. When set to `true`, a request from Elasticsearch contains only chunks related to a single document. Instead of batching chunks across documents, Elasticsearch sends them in separate requests. This ensures that chunk embeddings retain context from the entire document, improving semantic quality. If a document exceeds the model''s context limits, or if the document contains non-text inputs (relevant when using the multimodal `embedding` task), late chunking is automatically disabled for that document only and standard chunking is used instead. If not specified, defaults to `false`.' type: boolean top_n: description: 'For a `rerank` task, the number of most relevant documents to return. It defaults to the number of the documents. If this inference endpoint is used in a `text_similarity_reranker` retriever query and `top_n` is set, it must be greater than or equal to `rank_window_size` in the query.' type: number inference._types.GoogleVertexAITaskType: type: string enum: - rerank - text_embedding - completion - chat_completion inference._types.ContextualAIServiceSettings: type: object properties: api_key: description: 'A valid API key for your Contexutual AI account. IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key.' type: string model_id: description: 'The name of the model to use for the inference task. Refer to the Contextual AI documentation for the list of available rerank models.' type: string rate_limit: description: 'This setting helps to minimize the number of rate limit errors returned from Contextual AI. The `contextualai` service sets a default number of requests allowed per minute depending on the task type. For `rerank`, it is set to `1000`.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' required: - api_key - model_id inference._types.InferenceEndpointInfoELSER: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeELSER' required: - inference_id - task_type inference._types.CustomServiceType: type: string enum: - custom inference._types.CohereServiceType: type: string enum: - cohere _types.Duration: externalDocs: url: https://www.elastic.co/docs/reference/elasticsearch/rest-apis/api-conventions#time-units description: 'A duration. Units can be `nanos`, `micros`, `ms` (milliseconds), `s` (seconds), `m` (minutes), `h` (hours) and `d` (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.' oneOf: - type: string - type: string enum: - '-1' - type: string enum: - '0' inference._types.TaskTypeHuggingFace: type: string enum: - chat_completion - completion - rerank - text_embedding inference._types.WatsonxTaskType: type: string enum: - text_embedding - rerank - chat_completion - completion inference._types.InferenceEndpointInfo: description: Represents an inference endpoint as returned by the GET API allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskType' required: - inference_id - task_type inference._types.CohereTaskSettings: type: object properties: input_type: description: 'For a `text_embedding` task, the type of input passed to the model. Valid values are: * `classification`: Use it for embeddings passed through a text classifier. * `clustering`: Use it for the embeddings run through a clustering algorithm. * `ingest`: Use it for storing document embeddings in a vector database. * `search`: Use it for storing embeddings of search queries run against a vector database to find relevant documents. IMPORTANT: The `input_type` field is required when using embedding models `v3` and higher.' allOf: - $ref: '#/components/schemas/inference._types.CohereInputType' return_documents: description: For a `rerank` task, return doc text within the results. type: boolean top_n: description: 'For a `rerank` task, the number of most relevant documents to return. It defaults to the number of the documents. If this inference endpoint is used in a `text_similarity_reranker` retriever query and `top_n` is set, it must be greater than or equal to `rank_window_size` in the query.' type: number truncate: description: 'For a `text_embedding` task, the method to handle inputs longer than the maximum token length. Valid values are: * `END`: When the input exceeds the maximum input token length, the end of the input is discarded. * `NONE`: When the input exceeds the maximum input token length, an error is returned. * `START`: When the input exceeds the maximum input token length, the start of the input is discarded.' allOf: - $ref: '#/components/schemas/inference._types.CohereTruncateType' required: - input_type inference._types.OpenShiftAiTaskSettings: type: object properties: return_documents: description: For a `rerank` task, whether to return the source documents in the response. type: boolean top_n: description: For a `rerank` task, the number of most relevant documents to return. type: number inference._types.OpenShiftAiServiceSettings: type: object properties: api_key: description: 'A valid API key for your OpenShift AI endpoint. Can be found in `Token authentication` section of model related information.' type: string url: description: The URL of the OpenShift AI hosted model endpoint. type: string model_id: description: 'The name of the model to use for the inference task. Refer to the hosted model''s documentation for the name if needed. Service has been tested and confirmed to be working with the following models: * For `text_embedding` task - `gritlm-7b`. * For `completion` and `chat_completion` tasks - `llama-31-8b-instruct`. * For `rerank` task - `bge-reranker-v2-m3`.' type: string max_input_tokens: description: For a `text_embedding` task, the maximum number of tokens per input before chunking occurs. type: number similarity: description: 'For a `text_embedding` task, the similarity measure. One of cosine, dot_product, l2_norm. If not specified, the default dot_product value is used.' allOf: - $ref: '#/components/schemas/inference._types.OpenShiftAiSimilarityType' rate_limit: description: 'This setting helps to minimize the number of rate limit errors returned from the OpenShift AI API. By default, the `openshift_ai` service sets the number of requests allowed per minute to 3000.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' required: - api_key - url inference._types.HuggingFaceTaskSettings: type: object properties: return_documents: description: For a `rerank` task, return doc text within the results. type: boolean top_n: description: 'For a `rerank` task, the number of most relevant documents to return. It defaults to the number of the documents.' type: number inference._types.ContentType: type: string enum: - text - image_url - file inference._types.ContextualAITaskSettings: type: object properties: instruction: description: 'Instructions for the reranking model. Refer to Only for the `rerank` task type.' type: string top_k: description: 'The number of most relevant documents to return. If not specified, the reranking results of all documents will be returned. Only for the `rerank` task type.' type: number inference._types.TaskTypeVoyageAI: type: string enum: - text_embedding - rerank inference._types.RequestChatCompletion: type: object properties: messages: description: 'A list of objects representing the conversation. Requests should generally only add new messages from the user (role `user`). The other message roles (`assistant`, `system`, or `tool`) should generally only be copied from the response to a previous completion request, such that the messages array is built up throughout a conversation.' type: array items: $ref: '#/components/schemas/inference._types.Message' model: description: The ID of the model to use. By default, the model ID is set to the value included when creating the inference endpoint. type: string max_completion_tokens: description: The upper bound limit for the number of tokens that can be generated for a completion request. type: number reasoning: description: "The reasoning configuration for the completion request.\nThis controls the model's reasoning process in one of two ways:\n\n* By specifying the modelโ€™s reasoning effort level with the `effort` field.\n* By enabling reasoning with default settings by setting `enabled` field to `true`.\n\nIt also includes optional settings to control:\n\n* The level of detail in the summary returned in the response with the `summary` field.\n* Whether reasoning details are included in the response at all with the `exclude` field.\n\nExample (effort):\n```\n{\n \"reasoning\": {\n \"effort\": \"high\",\n \"summary\": \"concise\",\n \"exclude\": false\n }\n}\n```\nExample (enabled):\n```\n{\n \"reasoning\": {\n \"enabled\": true,\n \"summary\": \"concise\",\n \"exclude\": false\n }\n}\n```\nCurrently supported only for `elastic` provider." allOf: - $ref: '#/components/schemas/inference._types.Reasoning' stop: description: A sequence of strings to control when the model should stop generating additional tokens. type: array items: type: string temperature: description: The sampling temperature to use. type: number tool_choice: description: "Controls which tool is called by the model.\nString representation: One of `auto`, `none`, or `requrired`. `auto` allows the model to choose between calling tools and generating a message. `none` causes the model to not call any tools. `required` forces the model to call one or more tools.\nExample (object representation):\n```\n{\n \"tool_choice\": {\n \"type\": \"function\",\n \"function\": {\n \"name\": \"get_current_weather\"\n }\n }\n}\n```" allOf: - $ref: '#/components/schemas/inference._types.CompletionToolType' tools: description: "A list of tools that the model can call.\nExample:\n```\n{\n \"tools\": [\n {\n \"type\": \"function\",\n \"function\": {\n \"name\": \"get_price_of_item\",\n \"description\": \"Get the current price of an item\",\n \"parameters\": {\n \"type\": \"object\",\n \"properties\": {\n \"item\": {\n \"id\": \"12345\"\n },\n \"unit\": {\n \"type\": \"currency\"\n }\n }\n }\n }\n }\n ]\n}\n```" type: array items: $ref: '#/components/schemas/inference._types.CompletionTool' top_p: description: Nucleus sampling, an alternative to sampling with temperature. type: number required: - messages inference._types.TaskTypeCohere: type: string enum: - text_embedding - rerank - completion inference._types.TaskTypeGoogleAIStudio: type: string enum: - text_embedding - completion inference._types.TaskTypeDeepSeek: type: string enum: - completion - chat_completion inference._types.InferenceEndpointInfoOpenAI: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeOpenAI' required: - inference_id - task_type inference._types.InferenceEndpointInfoNvidia: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference ID type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeNvidia' required: - inference_id - task_type inference._types.InferenceEndpointInfoAmazonBedrock: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeAmazonBedrock' required: - inference_id - task_type inference._types.CustomServiceQueryParameter: type: array items: type: string inference._types.TaskTypeLlama: type: string enum: - text_embedding - chat_completion - completion inference._types.BaseReasoningDetail: description: The base reasoning detail that includes common fields across different types of reasoning details. type: object properties: format: description: The format of the reasoning detail. type: string id: description: The identifier of the reasoning detail. type: string index: description: 'The index of the reasoning detail, which indicates its position in the sequence of reasoning details generated by the model.' type: number inference._types.NvidiaTaskType: type: string enum: - chat_completion - completion - rerank - text_embedding inference._types.OpenAISimilarityType: type: string enum: - cosine - dot_product - l2_norm inference._types.EmbeddingInput: description: 'Inference input. Either a string, an array of strings, a `content` object, or an array of `content` objects.' oneOf: - $ref: '#/components/schemas/inference._types.EmbeddingStringInput' - $ref: '#/components/schemas/inference._types.EmbeddingContentInput' inference._types.CustomTaskType: type: string enum: - text_embedding - sparse_embedding - rerank - completion inference._types.AmazonSageMakerServiceType: type: string enum: - amazon_sagemaker inference._types.ImageUrl: type: object properties: url: description: The base64 encoded image data as a data URI type: string detail: description: Specifies the detail level of the image allOf: - $ref: '#/components/schemas/inference._types.ImageUrlDetail' required: - url inference.rerank.RerankInputFormat: type: string enum: - text - base64 inference._types.CompletionResult: description: The completion result object type: object properties: result: type: string required: - result inference._types.InferenceEndpointInfoAmazonSageMaker: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeAmazonSageMaker' required: - inference_id - task_type inference._types.InferenceEndpointInfoFireworksAI: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeFireworksAI' required: - inference_id - task_type inference._types.AzureOpenAIServiceType: type: string enum: - azureopenai inference._types.InferenceEndpointInfoVoyageAI: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeVoyageAI' required: - inference_id - task_type inference._types.EmbeddingInferenceResult: description: EmbeddingInferenceResult is an aggregation of mutually exclusive embeddings variants type: object properties: embeddings_bytes: type: array items: $ref: '#/components/schemas/inference._types.DenseEmbeddingByteResult' embeddings_bits: type: array items: $ref: '#/components/schemas/inference._types.DenseEmbeddingByteResult' embeddings: type: array items: $ref: '#/components/schemas/inference._types.DenseEmbeddingResult' minProperties: 1 maxProperties: 1 inference._types.InferenceEndpointInfoLlama: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeLlama' required: - inference_id - task_type inference._types.EmbeddingContentObject: description: A wrapper object which contains the fields required to specify multimodal inputs type: object properties: content: description: An object or an array of objects containing the input data for the model to embed allOf: - $ref: '#/components/schemas/inference._types.EmbeddingContentObjectGroup' required: - content inference._types.FireworksAIServiceType: type: string enum: - fireworksai inference._types.WatsonxServiceSettings: type: object properties: api_key: externalDocs: url: https://cloud.ibm.com/iam/apikeys description: 'A valid API key of your Watsonx account. You can find your Watsonx API keys or you can create a new one on the API keys page. IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key.' type: string api_version: externalDocs: url: https://cloud.ibm.com/apidocs/watsonx-ai#active-version-dates description: 'A version parameter that takes a version date in the format of `YYYY-MM-DD`. For the active version data parameters, refer to the Wastonx documentation.' type: string model_id: externalDocs: url: https://www.ibm.com/products/watsonx-ai/foundation-models description: 'The name of the model to use for the inference task. Refer to the IBM Embedding Models section in the Watsonx documentation for the list of available text embedding models. Refer to the IBM library - Foundation models in Watsonx.ai.' type: string project_id: description: The identifier of the IBM Cloud project to use for the inference task. type: string rate_limit: description: 'This setting helps to minimize the number of rate limit errors returned from Watsonx. By default, the `watsonxai` service sets the number of requests allowed per minute to 120.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' url: description: The URL of the inference endpoint that you created on Watsonx. type: string required: - api_key - api_version - model_id - project_id - url inference._types.InferenceEndpointInfoMistral: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeMistral' required: - inference_id - task_type inference._types.InferenceEndpointInfoCohere: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeCohere' required: - inference_id - task_type inference._types.ServiceSettings: type: object inference._types.MistralServiceSettings: type: object properties: api_key: externalDocs: url: https://console.mistral.ai/api-keys/ description: 'A valid API key of your Mistral account. You can find your Mistral API keys or you can create a new one on the API Keys page. IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key.' type: string max_input_tokens: description: The maximum number of tokens per input before chunking occurs. type: number model: externalDocs: url: https://docs.mistral.ai/getting-started/models/ description: 'The name of the model to use for the inference task. Refer to the Mistral models documentation for the list of available models.' type: string rate_limit: description: 'This setting helps to minimize the number of rate limit errors returned from the Mistral API. By default, the `mistral` service sets the number of requests allowed per minute to 240.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' required: - api_key - model inference._types.CompletionInferenceResult: description: Defines the completion result. type: object properties: completion: type: array items: $ref: '#/components/schemas/inference._types.CompletionResult' required: - completion inference._types.InferenceChunkingSettings: description: Chunking configuration object type: object properties: max_chunk_size: description: 'The maximum size of a chunk in words. This value cannot be lower than `20` (for `sentence` strategy) or `10` (for `word` strategy). This value should not exceed the window size for the associated model.' default: 250.0 type: number overlap: description: 'The number of overlapping words for chunks. It is applicable only to a `word` chunking strategy. This value cannot be higher than half the `max_chunk_size` value.' default: 100.0 type: number sentence_overlap: description: 'The number of overlapping sentences for chunks. It is applicable only for a `sentence` chunking strategy. It can be either `1` or `0`.' default: 1.0 type: number separator_group: description: 'Only applicable to the `recursive` strategy and required when using it. Sets a predefined list of separators in the saved chunking settings based on the selected text type. Values can be `markdown` or `plaintext`. Using this parameter is an alternative to manually specifying a custom `separators` list.' type: string separators: description: 'Only applicable to the `recursive` strategy and required when using it. A list of strings used as possible split points when chunking text. Each string can be a plain string or a regular expression (regex) pattern. The system tries each separator in order to split the text, starting from the first item in the list. After splitting, it attempts to recombine smaller pieces into larger chunks that stay within the `max_chunk_size` limit, to reduce the total number of chunks generated.' type: array items: type: string strategy: externalDocs: url: https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#chunking-strategies description: "The chunking strategy: `sentence`, `word`, `none` or `recursive`.\n\n * If `strategy` is set to `recursive`, you must also specify:\n\n- `max_chunk_size`\n- either `separators` or`separator_group`\n\nLearn more about different chunking strategies in the linked documentation." default: sentence type: string inference._types.TaskTypeOpenAI: type: string enum: - text_embedding - chat_completion - completion - embedding inference._types.InferenceEndpointInfoCustom: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeCustom' required: - inference_id - task_type inference._types.Ai21TaskType: type: string enum: - completion - chat_completion inference.rerank.RerankInput: description: 'The documents to rank for the `rerank` task. Either a string, an array of strings, an object, or an array of objects. The object form additionally allows specifying non-text inputs, such as images. > info > Only the `elastic` service currently supports non-text inputs for the `rerank` task.' oneOf: - $ref: '#/components/schemas/inference.rerank.RerankStringInput' - $ref: '#/components/schemas/inference.rerank.RerankObjectInput' inference._types.ElasticsearchTaskSettings: type: object properties: return_documents: description: For a `rerank` task, return the document instead of only the index. default: true type: boolean inference._types.SparseEmbeddingInferenceResult: description: The response format for the sparse embedding request. type: object properties: sparse_embedding: type: array items: $ref: '#/components/schemas/inference._types.SparseEmbeddingResult' required: - sparse_embedding inference._types.GoogleVertexAIServiceSettings: type: object properties: provider: description: 'The name of the Google Model Garden Provider for `completion` and `chat_completion` tasks. In order for a Google Model Garden endpoint to be used `provider` must be defined and be other than `google`. Modes: - Google Model Garden (third-party models): set `provider` to a supported non-`google` value and provide `url` and/or `streaming_url`. - Google Vertex AI: omit `provider` or set it to `google`. In this mode, do not set `url` or `streaming_url` and Elastic will construct the endpoint url from `location`, `model_id`, and `project_id` parameters.' allOf: - $ref: '#/components/schemas/inference._types.GoogleModelGardenProvider' url: description: 'The URL for non-streaming `completion` requests to a Google Model Garden provider endpoint. If both `url` and `streaming_url` are provided, each is used for its respective mode. If `streaming_url` is not provided, `url` is also used for streaming `completion` and `chat_completion`. If `provider` is not provided or set to `google` (Google Vertex AI), do not set `url` (or `streaming_url`). At least one of `url` or `streaming_url` must be provided for Google Model Garden endpoint usage. Certain providers require separate URLs for streaming and non-streaming operations (e.g., Anthropic, Mistral, AI21). Others support both operation types through a single URL (e.g., Meta, Hugging Face). Information on constructing the URL for various providers can be found in the Google Model Garden documentation for the model, or on the endpointโ€™s `Sample request` page. The request examples also illustrate the proper formatting for the `url`.' type: string streaming_url: description: 'The URL for streaming `completion` and `chat_completion` requests to a Google Model Garden provider endpoint. If both `streaming_url` and `url` are provided, each is used for its respective mode. If `url` is not provided, `streaming_url` is also used for non-streaming `completion` requests. If `provider` is not provided or set to `google` (Google Vertex AI), do not set `streaming_url` (or `url`). At least one of `streaming_url` or `url` must be provided for Google Model Garden endpoint usage. Certain providers require separate URLs for streaming and non-streaming operations (e.g., Anthropic, Mistral, AI21). Others support both operation types through a single URL (e.g., Meta, Hugging Face). Information on constructing the URL for various providers can be found in the Google Model Garden documentation for the model, or on the endpointโ€™s `Sample request` page. The request examples also illustrate the proper formatting for the `streaming_url`.' type: string location: externalDocs: url: https://cloud.google.com/vertex-ai/generative-ai/docs/learn/locations description: 'The name of the location to use for the inference task for the Google Vertex AI inference task. For Google Vertex AI, when `provider` is omitted or `google`, `location` is optional. When `location` is omitted, the request targets the Google Vertex AI global endpoint (`https://aiplatform.googleapis.com`); when it is provided, the corresponding regional endpoint (`https://-aiplatform.googleapis.com`) is used. The field must be omitted to select the global endpoint; an empty string is not a valid `location`. For Google Model Garden''s `completion` and `chat_completion` tasks, when `provider` is a supported non-`google` value - `location` is ignored. Refer to the Google documentation for the list of supported locations.' type: string model_id: externalDocs: url: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api description: 'The name of the model to use for the inference task. For Google Vertex AI `model_id` is mandatory. For Google Model Garden''s `completion` and `chat_completion` tasks, when `provider` is a supported non-`google` value - `model_id` will be used for some providers that require it, otherwise - ignored. Refer to the Google documentation for the list of supported models for Google Vertex AI.' type: string project_id: description: 'The name of the project to use for the Google Vertex AI inference task. For Google Vertex AI `project_id` is mandatory. For Google Model Garden''s `completion` and `chat_completion` tasks, when `provider` is a supported non-`google` value - `project_id` is ignored.' type: string rate_limit: description: 'This setting helps to minimize the number of rate limit errors returned from Google Vertex AI. By default, the `googlevertexai` service sets the number of requests allowed per minute to 30.000.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' service_account_json: description: A valid service account in JSON format for the Google Vertex AI API. type: string dimensions: externalDocs: url: https://cloud.google.com/vertex-ai/generative-ai/docs/embeddings/get-text-embeddings description: 'For a `text_embedding` task, the number of dimensions the resulting output embeddings should have. By default, the model''s standard output dimension is used. Refer to the Google documentation for more information.' type: number max_batch_size: description: 'Only applicable for the `text_embedding` task type. Controls the batch size of chunked inference requests sent to Google Vertex AI. Setting this parameter lower reduces the risk of exceeding token limits but may result in more API calls. Setting it higher increases throughput but may risk hitting token limits. To estimate a safe `max_batch_size` value, you can use it together with the `max_chunk_size` parameter using the following formula: `max_batch_size โ‰ˆ max_chunk_size ร— 1.3 ร— 512 รท 20000` Where: - `1.3` is an approximate tokens-per-word ratio - `512` is the maximum number of chunks that can be generated per document - `20000` is the Google Vertex AI token limit per request This estimate assumes the worst-case scenario with a document generating the maximum 512 chunks.' default: 250.0 type: number required: - service_account_json inference._types.JinaAISimilarityType: type: string enum: - cosine - dot_product - l2_norm inference.rerank.RerankStringInput: description: Allows specifying text-only documents to rank for the `rerank` task. oneOf: - type: string - type: array items: type: string inference._types.JinaAIElementType: type: string enum: - binary - bit - float inference._types.CustomResponseParams: type: object properties: json_parser: description: "Specifies the JSON parser that is used to parse the response from the custom service.\nDifferent task types require different json_parser parameters.\nFor example:\n```\n# text_embedding\n# For a response like this:\n\n{\n \"object\": \"list\",\n \"data\": [\n {\n \"object\": \"embedding\",\n \"index\": 0,\n \"embedding\": [\n 0.014539449,\n -0.015288644\n ]\n }\n ],\n \"model\": \"text-embedding-ada-002-v2\",\n \"usage\": {\n \"prompt_tokens\": 8,\n \"total_tokens\": 8\n }\n}\n\n# the json_parser definition should look like this:\n\n\"response\":{\n \"json_parser\":{\n \"text_embeddings\":\"$.data[*].embedding[*]\"\n }\n}\n\n# Elasticsearch supports the following embedding types:\n* float\n* byte\n* bit (or binary)\n\nTo specify the embedding type for the response, the `embedding_type`\nfield should be added in the `json_parser` object. Here's an example:\n\"response\":{\n \"json_parser\":{\n \"text_embeddings\":\"$.data[*].embedding[*]\",\n \"embedding_type\":\"bit\"\n }\n}\n\nIf `embedding_type` is not specified, it defaults to `float`.\n\n# sparse_embedding\n# For a response like this:\n\n{\n \"request_id\": \"75C50B5B-E79E-4930-****-F48DBB392231\",\n \"latency\": 22,\n \"usage\": {\n \"token_count\": 11\n },\n \"result\": {\n \"sparse_embeddings\": [\n {\n \"index\": 0,\n \"embedding\": [\n {\n \"token_id\": 6,\n \"weight\": 0.101\n },\n {\n \"token_id\": 163040,\n \"weight\": 0.28417\n }\n ]\n }\n ]\n }\n}\n\n# the json_parser definition should look like this:\n\n\"response\":{\n \"json_parser\":{\n \"token_path\":\"$.result.sparse_embeddings[*].embedding[*].token_id\",\n \"weight_path\":\"$.result.sparse_embeddings[*].embedding[*].weight\"\n }\n}\n\n# rerank\n# For a response like this:\n\n{\n \"results\": [\n {\n \"index\": 3,\n \"relevance_score\": 0.999071,\n \"document\": \"abc\"\n },\n {\n \"index\": 4,\n \"relevance_score\": 0.7867867,\n \"document\": \"123\"\n },\n {\n \"index\": 0,\n \"relevance_score\": 0.32713068,\n \"document\": \"super\"\n }\n ],\n}\n\n# the json_parser definition should look like this:\n\n\"response\":{\n \"json_parser\":{\n \"reranked_index\":\"$.result.scores[*].index\", // optional\n \"relevance_score\":\"$.result.scores[*].score\",\n \"document_text\":\"xxx\" // optional\n }\n}\n\n# completion\n# For a response like this:\n\n{\n \"id\": \"chatcmpl-B9MBs8CjcvOU2jLn4n570S5qMJKcT\",\n \"object\": \"chat.completion\",\n \"created\": 1741569952,\n \"model\": \"gpt-4.1-2025-04-14\",\n \"choices\": [\n {\n \"index\": 0,\n \"message\": {\n \"role\": \"assistant\",\n \"content\": \"Hello! How can I assist you today?\",\n \"refusal\": null,\n \"annotations\": []\n },\n \"logprobs\": null,\n \"finish_reason\": \"stop\"\n }\n ]\n}\n\n# the json_parser definition should look like this:\n\n\"response\":{\n \"json_parser\":{\n \"completion_result\":\"$.choices[*].message.content\"\n }\n}" type: object additionalProperties: type: string required: - json_parser inference._types.InferenceEndpointInfoHuggingFace: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeHuggingFace' required: - inference_id - task_type inference._types.GroqServiceType: type: string enum: - groq inference._types.AmazonBedrockServiceType: type: string enum: - amazonbedrock inference._types.EmbeddingContentFormat: type: string enum: - text - base64 inference._types.TaskTypeGroq: type: string enum: - chat_completion _types.Id: type: string inference._types.TaskTypeAi21: type: string enum: - completion - chat_completion inference._types.CustomServiceSettings: type: object properties: batch_size: externalDocs: url: https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/semantic-text#auto-text-chunking description: 'Specifies the batch size used for the semantic_text field. If the field is not provided, the default is 10. The batch size is the maximum number of inputs in a single request to the upstream service. The chunk within the batch are controlled by the selected chunking strategy for the semantic_text field.' type: number headers: description: "Specifies the HTTP header parameters โ€“ such as `Authorization` or `Content-Type` โ€“ that are required to access the custom service.\nFor example:\n```\n\"headers\": {\n \"Authorization\": \"Bearer ${api_key}\",\n \"Content-Type\": \"application/json;charset=utf-8\"\n}\n```" type: object additionalProperties: type: string input_type: description: "Specifies the input type translation values that are used to replace the `${input_type}` template in the request body.\nFor example:\n```\n\"input_type\": {\n \"translation\": {\n \"ingest\": \"do_ingest\",\n \"search\": \"do_search\"\n },\n \"default\": \"a_default\"\n},\n```\nIf the subsequent inference requests come from a search context, the `search` key will be used and the template will be replaced with `do_search`.\nIf it comes from the ingest context `do_ingest` is used. If it's a different context that is not specified, the default value will be used. If no default is specified an empty string is used.\n`translation` can be:\n* `classification`\n* `clustering`\n* `ingest`\n* `search`" type: object additionalProperties: type: string query_parameters: description: "Specifies the query parameters as a list of tuples. The arrays inside the `query_parameters` must have two items, a key and a value.\nFor example:\n```\n\"query_parameters\":[\n [\"param_key\", \"some_value\"],\n [\"param_key\", \"another_value\"],\n [\"other_key\", \"other_value\"]\n]\n```\nIf the base url is `https://www.elastic.co` it results in: `https://www.elastic.co?param_key=some_value¶m_key=another_value&other_key=other_value`." type: array items: $ref: '#/components/schemas/inference._types.CustomServiceQueryParameter' request: description: The request configuration object. allOf: - $ref: '#/components/schemas/inference._types.CustomRequestParams' response: description: The response configuration object. allOf: - $ref: '#/components/schemas/inference._types.CustomResponseParams' secret_parameters: description: "Specifies secret parameters, like `api_key` or `api_token`, that are required to access the custom service.\nFor example:\n```\n\"secret_parameters\":{\n \"api_key\":\"\"\n}\n```" type: object additionalProperties: type: string url: description: The URL endpoint to use for the requests. type: string required: - request - response - secret_parameters inference._types.AdaptiveAllocations: type: object properties: enabled: description: Turn on `adaptive_allocations`. default: false type: boolean max_number_of_allocations: description: 'The maximum number of allocations to scale to. If set, it must be greater than or equal to `min_number_of_allocations`.' type: number min_number_of_allocations: description: 'The minimum number of allocations to scale to. If set, it must be greater than or equal to 0. If not defined, the deployment scales to 0.' type: number inference._types.TaskTypeJinaAi: type: string enum: - embedding - text_embedding - rerank inference._types.NvidiaSimilarityType: type: string enum: - cosine - dot_product - l2_norm inference._types.FileContent: type: object properties: file_data: description: The base64 encoded file data type: string filename: description: The name of the file type: string required: - file_data - filename inference._types.CustomTaskParameter: oneOf: - type: string - type: number - type: number - type: number - type: boolean inference._types.InferenceEndpointInfoAi21: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeAi21' required: - inference_id - task_type inference._types.CustomRequestParams: type: object properties: content: description: 'The body structure of the request. It requires passing in the string-escaped result of the JSON format HTTP request body. For example: ``` "request": "{\"input\":${input}}" ``` > info > The content string needs to be a single line except when using the Kibana console.' type: string required: - content inference._types.TaskTypeAzureAIStudio: type: string enum: - text_embedding - completion - rerank inference._types.WatsonxServiceType: type: string enum: - watsonxai inference._types.FireworksAISimilarityType: type: string enum: - cosine - dot_product - l2_norm inference._types.LlamaTaskType: type: string enum: - text_embedding - completion - chat_completion inference._types.SparseVector: description: 'Sparse Embedding tokens are represented as a dictionary of string to double.' type: object additionalProperties: type: number inference._types.TaskTypeElasticsearch: type: string enum: - sparse_embedding - text_embedding - rerank inference._types.JinaAIServiceSettings: type: object properties: api_key: externalDocs: url: https://jina.ai/embeddings/ description: 'A valid API key of your JinaAI account. IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key.' type: string model_id: description: The name of the model to use for the inference task. type: string rate_limit: externalDocs: url: https://jina.ai/contact-sales/#rate-limit description: 'This setting helps to minimize the number of rate limit errors returned from JinaAI. By default, the `jinaai` service sets the number of requests allowed per minute to 2000 for all task types.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' similarity: description: 'For an `embedding` or `text_embedding` task, the similarity measure. One of cosine, dot_product, l2_norm. The default values varies with the embedding type. For example, a float embedding type uses a `dot_product` similarity measure by default.' allOf: - $ref: '#/components/schemas/inference._types.JinaAISimilarityType' dimensions: externalDocs: url: https://jina.ai/embeddings/ description: 'For an `embedding` or `text_embedding` task, the number of dimensions the resulting output embeddings should have. By default, the model''s standard output dimension is used. Refer to the Jina documentation for more information.' type: number embedding_type: description: 'For an `embedding` or `text_embedding` task, the data type returned by the model. Use `bit` for binary embeddings, which are encoded as bytes with signed int8 precision. Use `binary` for binary embeddings, which are encoded as bytes with signed int8 precision (this is a synonym of `bit`). Use `float` for the default float embeddings.' default: float allOf: - $ref: '#/components/schemas/inference._types.JinaAIElementType' multimodal_model: description: 'For the `embedding` task, whether the model supports multimodal inputs. If true, requests sent to the Jina model will use the multimodal request format (a list of objects). If false, requests sent to the model will use the same format as the `text_embedding` task (a list of strings). Setting this to `false` allows the `embedding` task to be used with models that do not support multimodal requests.' default: true type: boolean required: - api_key - model_id inference._types.OpenAIServiceSettings: type: object properties: api_key: externalDocs: url: https://platform.openai.com/api-keys description: 'A valid API key of your OpenAI account. You can find your OpenAI API keys in your OpenAI account under the API keys section. IMPORTANT: You must specify either `api_key` or `client_secret`. If you do not provide one or you provide more than one of them, you will receive an error when you try to create your endpoint.' type: string client_id: description: 'For OAuth 2.0 authorization using the client credentials grant flow. The application ID that''s assigned to your app. IMPORTANT: To configure OAuth 2.0, you must specify `client_id`, `scopes`, `token_url`, and `client_secret` together. If one of the fields is missing, you will receive an error when you try to create your endpoint.' type: string client_secret: description: 'For OAuth 2.0 authorization using the client credentials grant flow. The application secret that you created for your app. IMPORTANT: You must specify either `api_key` or `client_secret`. If you do not provide one or you provide more than one of them, you will receive an error when you try to create your endpoint. IMPORTANT: To configure OAuth 2.0, you must specify `client_id`, `scopes`, `token_url`, and `client_secret` together. If one of the fields is missing, you will receive an error when you try to create your endpoint.' type: string dimensions: description: 'For a `text_embedding` or `embedding` task, the number of dimensions the resulting output embeddings should have. It is supported only in `text-embedding-3` and later models. If it is not set, the OpenAI defined default for the model is used.' type: number model_id: externalDocs: url: https://platform.openai.com/docs/guides/embeddings/what-are-embeddings description: 'The name of the model to use for the inference task. Refer to the OpenAI documentation for the list of available text embedding models.' type: string organization_id: description: 'The unique identifier for your organization. You can find the Organization ID in your OpenAI account under *Settings > Organizations*.' type: string rate_limit: description: 'This setting helps to minimize the number of rate limit errors returned from OpenAI. The `openai` service sets a default number of requests allowed per minute depending on the task type. For `text_embedding` and `embedding`, it is set to `3000`. For `completion` and `chat_completion`, it is set to `500`.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' scopes: description: "For OAuth 2.0 authorization using the client credentials grant flow.\nThe resource identifier of the resource you want.\nFor example:\n```\n\"scopes\": [\n \"scope1\",\n \"scope2\"\n]\n```\n\nIMPORTANT: To configure OAuth 2.0, you must specify `client_id`, `scopes`, `token_url`, and `client_secret` together.\nIf one of the fields is missing, you will receive an error when you try to create your endpoint." type: array items: type: string similarity: description: For a `text_embedding` or `embedding` task, the similarity measure. One of `cosine`, `dot_product`, `l2_norm`. Defaults to `dot_product`. allOf: - $ref: '#/components/schemas/inference._types.OpenAISimilarityType' token_url: description: 'For OAuth 2.0 authorization using the client credentials grant flow. An OAuth2 token endpoint where Elasticsearch sends a request to exchange client credentials for an access token. IMPORTANT: To configure OAuth 2.0, you must specify `client_id`, `scopes`, `token_url`, and `client_secret` together. If one of the fields is missing, you will receive an error when you try to create your endpoint.' type: string url: description: 'The URL endpoint to use for the requests. It can be changed for testing purposes. Default value is `https://api.openai.com/v1/embeddings` for a `text_embedding` or `embedding` task, `https://api.openai.com/v1/chat/completions` for a `completion` or `chat_completion` task.' type: string required: - model_id inference._types.DeepSeekServiceType: type: string enum: - deepseek inference._types.SparseEmbeddingResult: type: object properties: is_truncated: description: Indicates if the text input was truncated in the request sent to the service type: boolean embedding: allOf: - $ref: '#/components/schemas/inference._types.SparseVector' required: - is_truncated - embedding inference._types.TaskTypeAzureOpenAI: type: string enum: - text_embedding - completion - chat_completion inference._types.AmazonSageMakerElementType: type: string enum: - byte - float - bit inference._types.DenseEmbeddingByteResult: description: The dense embedding result object for byte representation type: object properties: embedding: allOf: - $ref: '#/components/schemas/inference._types.DenseByteVector' required: - embedding inference._types.TaskTypeMistral: type: string enum: - text_embedding - chat_completion - completion inference._types.CompletionTool: description: A list of tools that the model can call. type: object properties: type: description: The type of tool. type: string function: description: The function definition. allOf: - $ref: '#/components/schemas/inference._types.CompletionToolFunction' required: - type - function inference._types.OpenAITaskType: type: string enum: - chat_completion - completion - text_embedding - embedding inference._types.CohereInputType: type: string enum: - classification - clustering - ingest - search inference._types.AnthropicTaskSettings: type: object properties: max_tokens: description: For a `completion` or `chat_completion` task, it is the maximum number of tokens to generate before stopping. type: number temperature: externalDocs: url: https://docs.anthropic.com/en/api/messages description: 'For a `completion` or `chat_completion` task, it is the amount of randomness injected into the response. For more details about the supported range, refer to Anthropic documentation.' type: number top_k: description: 'For a `completion` or `chat_completion` task, it specifies to only sample from the top K options for each subsequent token. It is recommended for advanced use cases only. You usually only need to use `temperature`.' type: number top_p: description: 'For a `completion` or `chat_completion` task, it specifies to use Anthropic''s nucleus sampling. In nucleus sampling, Anthropic computes the cumulative distribution over all the options for each subsequent token in decreasing probability order and cuts it off once it reaches the specified probability. You should either alter `temperature` or `top_p`, but not both. It is recommended for advanced use cases only. You usually only need to use `temperature`.' type: number required: - max_tokens inference._types.RegionPolicy: description: 'The region policy configuration. Specify exactly one of `allowed_geos` or `allowed_regions`.' type: object properties: allowed_geos: description: 'The list of allowed geographic areas. Mutually exclusive with `allowed_regions`.' type: array items: type: string allowed_regions: description: 'The list of allowed cloud service provider regions. Mutually exclusive with `allowed_geos`.' type: array items: $ref: '#/components/schemas/inference._types.CspRegion' minProperties: 1 maxProperties: 1 inference._types.ElasticsearchServiceType: type: string enum: - elasticsearch inference._types.GoogleAiServiceType: type: string enum: - googleaistudio inference._types.MessageContent: oneOf: - type: string - type: array items: $ref: '#/components/schemas/inference._types.ContentObject' inference._types.Reasoning: description: 'The reasoning configuration to use for the completion request. Currently supported only for `elastic` provider.' type: object properties: effort: description: 'The level of effort the model should put into reasoning. This is a hint that guides the model in how much effort to put into reasoning, with `xhigh` being the most effort and `none` being no effort.' allOf: - $ref: '#/components/schemas/inference._types.ReasoningEffort' enabled: description: 'Whether to enable reasoning with default settings. This is a shortcut for enabling reasoning without having to specify the other parameters. If `enabled` is set to `true`, then reasoning at the `medium` effort level is enabled. Ignored if `effort` is specified, in which case that parameter will control the reasoning process instead.' type: boolean exclude: description: 'Whether to exclude reasoning information from the response. If `true`, the response will not include any reasoning details.' type: boolean summary: description: 'The level of detail included in the reasoning summary returned in the response. This is a hint on how much detail to include in the summary of the reasoning that is returned in the response, with `auto` being the default level of detail, `concise` being less detail, and `detailed` being more detail.' allOf: - $ref: '#/components/schemas/inference._types.ReasoningSummary' inference._types.OpenAITaskSettings: type: object properties: user: description: 'Specifies the user issuing the request. This information can be used for abuse detection.' type: string headers: description: "Specifies custom HTTP header parameters.\nFor example:\n```\n\"headers\": {\n \"Custom-Header\": \"Some-Value\",\n \"Another-Custom-Header\": \"Another-Value\"\n}\n```" type: object additionalProperties: type: string inference._types.MistralTaskType: type: string enum: - text_embedding - completion - chat_completion inference._types.InferenceEndpointInfoGoogleAIStudio: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeGoogleAIStudio' required: - inference_id - task_type inference._types.AzureOpenAIServiceSettings: type: object properties: api_key: externalDocs: url: https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#authentication description: 'A valid API key for your Azure OpenAI account. IMPORTANT: You must specify either `api_key`, `entra_id`, or `client_secret`. If you do not provide one or you provide more than one of them, you will receive an error when you try to create your endpoint.' type: string api_version: description: 'The Azure API version ID to use. It is recommended to use the latest supported non-preview version.' type: string client_id: externalDocs: url: https://learn.microsoft.com/en-us/azure/api-management/api-management-authenticate-authorize-ai-apis#oauth-20-authorization-by-using-identity-provider description: 'For OAuth 2.0 authorization using the client credentials grant flow. The application ID that''s assigned to your app. IMPORTANT: To configure OAuth 2.0, you must specify `client_id`, `scopes`, `tenant_id`, and `client_secret` together. If one of the fields is missing, you will receive an error when you try to create your endpoint.' type: string client_secret: externalDocs: url: https://learn.microsoft.com/en-us/azure/api-management/api-management-authenticate-authorize-ai-apis#oauth-20-authorization-by-using-identity-provider description: 'For OAuth 2.0 authorization using the client credentials grant flow. The application secret that you created in the Microsoft app registration portal for your app. IMPORTANT: You must specify either `api_key`, `entra_id`, or `client_secret`. If you do not provide one or you provide more than one of them, you will receive an error when you try to create your endpoint. IMPORTANT: To configure OAuth 2.0, you must specify `client_id`, `scopes`, `tenant_id`, and `client_secret` together. If one of the fields is missing, you will receive an error when you try to create your endpoint.' type: string deployment_id: externalDocs: url: https://oai.azure.com/ description: 'The deployment name of your deployed models. Your Azure OpenAI deployments can be found though the Azure OpenAI Studio portal that is linked to your subscription.' type: string entra_id: externalDocs: url: https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#authentication description: 'A valid Microsoft Entra token. IMPORTANT: You must specify either `api_key`, `entra_id`, or `client_secret`. If you do not provide one or you provide more than one of them, you will receive an error when you try to create your endpoint.' type: string rate_limit: externalDocs: url: https://learn.microsoft.com/en-us/azure/ai-services/openai/quotas-limits description: 'This setting helps to minimize the number of rate limit errors returned from Azure. The `azureopenai` service sets a default number of requests allowed per minute depending on the task type. For `text_embedding`, it is set to `1440`. For `completion` and `chat_completion`, it is set to `120`.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' resource_name: externalDocs: url: https://portal.azure.com/#view/HubsExtension/BrowseAll description: 'The name of your Azure OpenAI resource. You can find this from the list of resources in the Azure Portal for your subscription.' type: string scopes: externalDocs: url: https://learn.microsoft.com/en-us/entra/identity-platform/scopes-oidc description: "For OAuth 2.0 authorization using the client credentials grant flow.\nThe resource identifier (application ID URI) of the resource you want, suffixed with .default\nFor example:\n```\n\"scopes\": [\n \"https://cognitiveservices.azure.com/.default\"\n]\n```\n\nIMPORTANT: To configure OAuth 2.0, you must specify `client_id`, `scopes`, `tenant_id`, and `client_secret` together.\nIf one of the fields is missing, you will receive an error when you try to create your endpoint." type: array items: type: string tenant_id: externalDocs: url: https://learn.microsoft.com/en-us/azure/api-management/api-management-authenticate-authorize-ai-apis#oauth-20-authorization-by-using-identity-provider description: 'For OAuth 2.0 authorization using the client credentials grant flow. The directory tenant the application plans to operate against. IMPORTANT: To configure OAuth 2.0, you must specify `client_id`, `scopes`, `tenant_id`, and `client_secret` together. If one of the fields is missing, you will receive an error when you try to create your endpoint.' type: string required: - api_version - deployment_id - resource_name inference._types.CustomTaskSettings: type: object properties: parameters: description: "Specifies parameters that are required to run the custom service. The parameters depend on the model your custom service uses.\nFor example:\n```\n\"task_settings\":{\n \"parameters\":{\n \"input_type\":\"query\",\n \"return_token\":true\n }\n}\n```\n\n> warn\n> The `task_settings.parameters` cannot contain the same keys as `secret_parameters`. If they do, an error will be returned. This applies to PUT requests and POST requests." type: object additionalProperties: $ref: '#/components/schemas/inference._types.CustomTaskParameter' inference._types.InferenceEndpointInfoAnthropic: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeAnthropic' required: - inference_id - task_type inference._types.ElasticsearchTaskType: type: string enum: - rerank - sparse_embedding - text_embedding inference._types.ElasticsearchServiceSettings: type: object properties: adaptive_allocations: description: 'Adaptive allocations configuration details. If `enabled` is true, the number of allocations of the model is set based on the current load the process gets. When the load is high, a new model allocation is automatically created, respecting the value of `max_number_of_allocations` if it''s set. When the load is low, a model allocation is automatically removed, respecting the value of `min_number_of_allocations` if it''s set. If `enabled` is true, do not set the number of allocations manually.' allOf: - $ref: '#/components/schemas/inference._types.AdaptiveAllocations' deployment_id: description: 'The deployment identifier for a trained model deployment. When `deployment_id` is used the `model_id` is optional.' type: string model_id: externalDocs: url: https://www.elastic.co/docs/explore-analyze/machine-learning/nlp/ml-nlp-import-model#ml-nlp-import-script description: 'The name of the model to use for the inference task. It can be the ID of a built-in model (for example, `.multilingual-e5-small` for E5) or a text embedding model that was uploaded by using the Eland client.' type: string num_allocations: description: 'The total number of allocations that are assigned to the model across machine learning nodes. Increasing this value generally increases the throughput. If adaptive allocations are enabled, do not set this value because it''s automatically set.' type: number num_threads: description: 'The number of threads used by each model allocation during inference. This setting generally increases the speed per inference request. The inference process is a compute-bound process; `threads_per_allocations` must not exceed the number of available allocated processors per node. The value must be a power of 2. The maximum value is 32.' type: number long_document_strategy: description: 'Available only for the `rerank` task type using the Elastic reranker model. Controls the strategy used for processing long documents during inference. Possible values: - `truncate` (default): Processes only the beginning of each document. - `chunk`: Splits long documents into smaller parts (chunks) before inference. When `long_document_strategy` is set to `chunk`, Elasticsearch splits each document into smaller parts but still returns a single score per document. That score reflects the highest relevance score among all chunks.' x-state: Experimental type: string max_chunks_per_doc: description: 'Only for the `rerank` task type. Limits the number of chunks per document that are sent for inference when chunking is enabled. If not set, all chunks generated for the document are processed.' x-state: Experimental type: number required: - model_id - num_threads inference._types.NvidiaInputType: type: string enum: - ingest - search inference._types.CspRegion: description: A cloud service provider region. type: object properties: csp: description: The cloud service provider, for example `aws`, `gcp`, or `azure`. type: string region: description: The region of the cloud service provider, for example `us-east-1`. type: string required: - csp - region inference._types.DeepSeekServiceSettings: type: object properties: api_key: externalDocs: url: https://api-docs.deepseek.com/ description: 'A valid API key for your DeepSeek account. You can find or create your DeepSeek API keys on the DeepSeek API key page. IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key.' type: string model_id: description: 'For a `completion` or `chat_completion` task, the name of the model to use for the inference task. For the available `completion` and `chat_completion` models, refer to the [DeepSeek Models & Pricing docs](https://api-docs.deepseek.com/quick_start/pricing).' type: string url: description: The URL endpoint to use for the requests. Defaults to `https://api.deepseek.com/chat/completions`. type: string required: - api_key - model_id inference._types.EmbeddingStringInput: description: Allows specifying text-only inputs for the `embedding` task. oneOf: - type: string - type: array items: type: string inference._types.HuggingFaceServiceType: type: string enum: - hugging_face inference._types.AzureAiStudioServiceType: type: string enum: - azureaistudio inference._types.AzureOpenAITaskSettings: type: object properties: user: description: 'Specifies the user issuing the request. This information can be used for abuse detection.' type: string headers: description: "Specifies custom HTTP header parameters.\nFor example:\n```\n\"headers\": {\n \"Custom-Header\": \"Some-Value\",\n \"Another-Custom-Header\": \"Another-Value\"\n}\n```" type: object additionalProperties: type: string inference._types.ReasoningSummary: type: string enum: - auto - concise - detailed inference._types.OpenShiftAiSimilarityType: type: string enum: - cosine - dot_product - l2_norm inference._types.EmbeddingContentInput: description: Allows specifying multimodal inputs for the `embedding` task. oneOf: - $ref: '#/components/schemas/inference._types.EmbeddingContentObject' - type: array items: $ref: '#/components/schemas/inference._types.EmbeddingContentObject' inference._types.AmazonSageMakerSimilarity: type: string enum: - cosine - dot_product - l2_norm inference._types.GroqServiceSettings: type: object properties: model_id: externalDocs: url: https://console.groq.com/docs/models description: 'The name of the model to use for the inference task. Refer to the Groq model documentation for the list of supported models and versions. Service has been tested and confirmed to be working for `completion` and `chat_completion` tasks with the following models: * `llama-3.3-70b-versatile`' type: string api_key: description: 'A valid API key for accessing Groq API. IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key.' type: string rate_limit: externalDocs: url: https://console.groq.com/docs/rate-limits description: 'This setting helps to minimize the number of rate limit errors returned from the Groq API. By default, the `groq` service sets the number of requests allowed per minute to 1000. Refer to Groq documentation for more details.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' required: - model_id inference._types.VoyageAITaskType: type: string enum: - text_embedding - rerank inference._types.AmazonSageMakerTaskSettings: type: object properties: custom_attributes: externalDocs: url: https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpoint.html description: 'The AWS custom attributes passed verbatim through to the model running in the SageMaker Endpoint. Values will be returned in the `X-elastic-sagemaker-custom-attributes` header.' type: string enable_explanations: externalDocs: url: https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpoint.html description: The optional JMESPath expression used to override the EnableExplanations provided during endpoint creation. type: string inference_id: externalDocs: url: https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpoint.html description: The capture data ID when enabled in the endpoint. type: string session_id: externalDocs: url: https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpoint.html description: 'The stateful session identifier for a new or existing session. New sessions will be returned in the `X-elastic-sagemaker-new-session-id` header. Closed sessions will be returned in the `X-elastic-sagemaker-closed-session-id` header.' type: string target_variant: externalDocs: url: https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpoint.html description: Specifies the variant when running with multi-variant Endpoints. type: string inference._types.InferenceEndpointInfoAzureOpenAI: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeAzureOpenAI' required: - inference_id - task_type inference._types.JinaAITaskType: type: string enum: - embedding - rerank - text_embedding inference._types.Message: description: An object representing part of the conversation. type: object properties: content: description: "The content of the message.\n\nString example:\n```\n{\n \"content\": \"Some string\"\n}\n```\n\nText example:\n```\n{\n \"content\": [\n {\n \"text\": \"Some text\",\n \"type\": \"text\"\n }\n ]\n}\n```\n\nImage example:\n```\n{\n \"content\": [\n {\n \"image_url\": {\n \"url\": \"data:image/jpeg;base64,...\"\n },\n \"type\": \"image_url\"\n }\n ]\n}\n```\n\nFile example:\n```\n{\n \"content\": [\n {\n \"file\": {\n \"file_data\": \"data:application/pdf;base64,...\",\n \"filename\": \"somePDF\"\n },\n \"type\": \"file\"\n }\n ]\n}\n```" allOf: - $ref: '#/components/schemas/inference._types.MessageContent' role: description: The role of the message author. Valid values are `user`, `assistant`, `system`, and `tool`. type: string tool_call_id: description: Only for `tool` role messages. The tool call that this message is responding to. allOf: - $ref: '#/components/schemas/_types.Id' tool_calls: description: "Only for `assistant` role messages. The tool calls generated by the model. If it's specified, the `content` field is optional.\nExample:\n```\n{\n \"tool_calls\": [\n {\n \"id\": \"call_KcAjWtAww20AihPHphUh46Gd\",\n \"type\": \"function\",\n \"function\": {\n \"name\": \"get_current_weather\",\n \"arguments\": \"{\\\"location\\\":\\\"Boston, MA\\\"}\"\n }\n }\n ]\n}\n```" type: array items: $ref: '#/components/schemas/inference._types.ToolCall' reasoning: description: 'Only for `assistant` role messages. The reasoning details generated by the model as plaintext. Currently supported only for `elastic` provider.' type: string reasoning_details: description: 'Only for `assistant` role messages. The reasoning details generated by the model as structured data. Currently supported only for `elastic` provider.' type: array items: $ref: '#/components/schemas/inference._types.ReasoningDetail' required: - role inference._types.AlibabaCloudTaskType: type: string enum: - completion - rerank - sparse_embedding - text_embedding inference.rerank.RerankObjectInput: description: Allows specifying documents to rank as objects, which additionally supports non-text inputs, such as images. oneOf: - $ref: '#/components/schemas/inference.rerank.RerankInputObject' - type: array items: $ref: '#/components/schemas/inference.rerank.RerankInputObject' inference._types.GroqTaskType: type: string enum: - chat_completion inference.rerank.RerankInputType: type: string enum: - text - image inference._types.AmazonBedrockTaskType: type: string enum: - chat_completion - completion - text_embedding inference._types.TaskTypeWatsonx: type: string enum: - text_embedding - chat_completion - completion inference._types.NvidiaTaskSettings: type: object properties: input_type: description: 'For a `text_embedding` task, type of input sent to the Nvidia endpoint. Valid values are: * `ingest`: Mapped to Nvidia''s `passage` value in request. Used when generating embeddings during indexing. * `search`: Mapped to Nvidia''s `query` value in request. Used when generating embeddings during querying. IMPORTANT: For Nvidia endpoints, if the `input_type` field is not specified, it defaults to `query`.' allOf: - $ref: '#/components/schemas/inference._types.NvidiaInputType' truncate: description: 'For a `text_embedding` task, the method used by the Nvidia model to handle inputs longer than the maximum token length. Valid values are: * `END`: When the input exceeds the maximum input token length, the end of the input is discarded. * `NONE`: When the input exceeds the maximum input token length, an error is returned. * `START`: When the input exceeds the maximum input token length, the start of the input is discarded.' allOf: - $ref: '#/components/schemas/inference._types.CohereTruncateType' inference._types.RateLimitSetting: description: This setting helps to minimize the number of rate limit errors returned from the service. type: object properties: requests_per_minute: description: 'The number of requests allowed per minute. By default, the number of requests allowed per minute is set by each service as follows: * `alibabacloud-ai-search` service: `1000` * `amazonbedrock` service: `240` * `anthropic` service: `50` * `azureaistudio` service: `240` * `azureopenai` service and task type `text_embedding`: `1440` * `azureopenai` service and task types `completion` or `chat_completion`: `120` * `cohere` service: `10000` * `contextualai` service: `1000` * `elastic` service and task type `chat_completion`: `240` * `fireworksai` service: `6000` * `googleaistudio` service: `360` * `googlevertexai` service: `30000` * `hugging_face` service: `3000` * `jinaai` service: `2000` * `llama` service: `3000` * `mistral` service: `240` * `openai` service and task type `text_embedding`: `3000` * `openai` service and task type `completion`: `500` * `openshift_ai` service: `3000` * `voyageai` service: `2000` * `watsonxai` service: `120`' type: number inference._types.GoogleVertexAIServiceType: type: string enum: - googlevertexai inference._types.GoogleAiStudioServiceSettings: type: object properties: api_key: description: A valid API key of your Google Gemini account. type: string model_id: externalDocs: url: https://ai.google.dev/gemini-api/docs/models description: 'The name of the model to use for the inference task. Refer to the Google documentation for the list of supported models.' type: string rate_limit: description: 'This setting helps to minimize the number of rate limit errors returned from Google AI Studio. By default, the `googleaistudio` service sets the number of requests allowed per minute to 360.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' required: - api_key - model_id inference._types.ToolCall: description: A tool call generated by the model. type: object properties: id: description: The identifier of the tool call. allOf: - $ref: '#/components/schemas/_types.Id' function: description: The function that the model called. allOf: - $ref: '#/components/schemas/inference._types.ToolCallFunction' type: description: The type of the tool call. type: string required: - id - function - type inference._types.Ai21ServiceSettings: type: object properties: model_id: externalDocs: url: https://docs.ai21.com/docs/jamba-foundation-models description: 'The name of the model to use for the inference task. Refer to the AI21 models documentation for the list of supported models and versions. Service has been tested and confirmed to be working for `completion` and `chat_completion` tasks with the following models: * `jamba-mini` * `jamba-large`' type: string api_key: description: 'A valid API key for accessing AI21 API. IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key.' type: string rate_limit: externalDocs: url: https://docs.ai21.com/reference/api-rate-limits description: 'This setting helps to minimize the number of rate limit errors returned from the AI21 API. By default, the `ai21` service sets the number of requests allowed per minute to 200. Please refer to AI21 documentation for more details.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' required: - model_id inference._types.InferenceEndpointInfoGroq: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeGroq' required: - inference_id - task_type inference._types.OpenAIServiceType: type: string enum: - openai inference._types.AnthropicTaskType: type: string enum: - completion - chat_completion inference._types.AmazonSageMakerServiceSettings: type: object properties: access_key: description: A valid AWS access key that has permissions to use Amazon SageMaker and access to models for invoking requests. type: string endpoint_name: externalDocs: url: https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpoint.html description: The name of the SageMaker endpoint. type: string api: description: 'The API format to use when calling SageMaker. Elasticsearch will convert the POST _inference request to this data format when invoking the SageMaker endpoint.' allOf: - $ref: '#/components/schemas/inference._types.AmazonSageMakerApi' region: externalDocs: url: https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpoint.html description: 'The region that your endpoint or Amazon Resource Name (ARN) is deployed in. The list of available regions per model can be found in the Amazon SageMaker documentation.' type: string secret_key: externalDocs: url: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html description: 'A valid AWS secret key that is paired with the `access_key`. For information about creating and managing access and secret keys, refer to the AWS documentation.' type: string similarity: description: 'Required when `api` is `elastic` and task type is `text_embedding`. The similarity measure used when invoking the `text_embedding` task type.' allOf: - $ref: '#/components/schemas/inference._types.AmazonSageMakerSimilarity' element_type: description: 'Required when `api` is `elastic` and task type is `text_embedding`. The data type returned by the text embedding model. This value is used when parsing the response back to Elasticsearch data structures.' allOf: - $ref: '#/components/schemas/inference._types.AmazonSageMakerElementType' target_model: externalDocs: url: https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpoint.html description: The model ID when calling a multi-model endpoint. type: string target_container_hostname: externalDocs: url: https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpoint.html description: The container to directly invoke when calling a multi-container endpoint. type: string inference_component_name: externalDocs: url: https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpoint.html description: The inference component to directly invoke when calling a multi-component endpoint. type: string batch_size: description: 'The maximum number of inputs in each batch. This value is used by inference ingestion pipelines when processing semantic values. It correlates to the number of times the SageMaker endpoint is invoked (one per batch of input).' default: 256.0 type: number dimensions: description: 'The number of dimensions returned by the text embedding models. If this value is not provided, then it is guessed by making invoking the endpoint for the `text_embedding` task.' type: number required: - access_key - endpoint_name - api - region - secret_key inference._types.VoyageAITaskSettings: type: object properties: input_type: description: 'Type of the input text. Permitted values: `ingest` (maps to `document` in the VoyageAI documentation), `search` (maps to `query` in the VoyageAI documentation). Only for the `text_embedding` task type.' type: string return_documents: description: 'Whether to return the source documents in the response. Only for the `rerank` task type.' default: false type: boolean top_k: description: 'The number of most relevant documents to return. If not specified, the reranking results of all documents will be returned. Only for the `rerank` task type.' type: number truncation: description: Whether to truncate the input texts to fit within the context length. default: true type: boolean inference._types.InferenceEndpointInfoAzureAIStudio: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeAzureAIStudio' required: - inference_id - task_type inference._types.AlibabaCloudTaskSettings: type: object properties: input_type: description: 'For a `sparse_embedding` or `text_embedding` task, specify the type of input passed to the model. Valid values are: * `ingest` for storing document embeddings in a vector database. * `search` for storing embeddings of search queries run against a vector database to find relevant documents.' type: string return_token: description: 'For a `sparse_embedding` task, it affects whether the token name will be returned in the response. It defaults to `false`, which means only the token ID will be returned in the response.' type: boolean inference._types.TaskTypeNvidia: type: string enum: - chat_completion - completion - rerank - text_embedding inference._types.AnthropicServiceSettings: type: object properties: api_key: description: A valid API key for the Anthropic API. type: string model_id: description: 'The name of the model to use for the inference task. Refer to the Anthropic documentation for the list of supported models.' type: string rate_limit: description: 'This setting helps to minimize the number of rate limit errors returned from Anthropic. By default, the `anthropic` service sets the number of requests allowed per minute to 50.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' required: - api_key - model_id inference._types.TaskTypeAmazonSageMaker: type: string enum: - text_embedding - completion - chat_completion - sparse_embedding - rerank inference._types.HuggingFaceTaskType: type: string enum: - chat_completion - completion - rerank - text_embedding inference._types.RegionPolicyDoc: description: The stored region policy document. type: object properties: region_policy: allOf: - $ref: '#/components/schemas/inference._types.RegionPolicy' created_at: description: The date and time the region policy was created. allOf: - $ref: '#/components/schemas/_types.DateTime' created_by: description: The user who created the region policy. type: string updated_at: description: The date and time the region policy was last updated. allOf: - $ref: '#/components/schemas/_types.DateTime' updated_by: description: The user who last updated the region policy. type: string required: - region_policy - created_at inference._types.LlamaServiceType: type: string enum: - llama inference._types.TaskTypeAnthropic: type: string enum: - completion - chat_completion inference._types.AlibabaCloudServiceType: type: string enum: - alibabacloud-ai-search inference._types.LlamaServiceSettings: type: object properties: url: description: 'The URL endpoint of the Llama stack endpoint. URL must contain: * For `text_embedding` task - `/v1/inference/embeddings`. * For `completion` and `chat_completion` tasks - `/v1/openai/v1/chat/completions`.' type: string model_id: externalDocs: url: https://llama-stack.readthedocs.io/en/latest/references/llama_cli_reference/download_models.html/ description: 'The name of the model to use for the inference task. Refer to the Llama downloading models documentation for different ways of getting a list of available models and downloading them. Service has been tested and confirmed to be working with the following models: * For `text_embedding` task - `all-MiniLM-L6-v2`. * For `completion` and `chat_completion` tasks - `llama3.2:3b`.' type: string max_input_tokens: description: For a `text_embedding` task, the maximum number of tokens per input before chunking occurs. type: number similarity: description: For a `text_embedding` task, the similarity measure. One of cosine, dot_product, l2_norm. allOf: - $ref: '#/components/schemas/inference._types.LlamaSimilarityType' rate_limit: description: 'This setting helps to minimize the number of rate limit errors returned from the Llama API. By default, the `llama` service sets the number of requests allowed per minute to 3000.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' required: - url - model_id inference._types.InferenceEndpointInfoContextualAi: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeContextualAI' required: - inference_id - task_type inference._types.EmbeddingContentType: type: string enum: - text - image - audio - video - pdf inference._types.VoyageAIServiceType: type: string enum: - voyageai inference._types.ReasoningDetail: description: 'Type representing the different types of reasoning details that can be included in the response from the model. Currently supported only for `elastic` provider.' discriminator: propertyName: type mapping: reasoning.encrypted: '#/components/schemas/inference._types.EncryptedReasoningDetail' reasoning.summary: '#/components/schemas/inference._types.SummaryReasoningDetail' reasoning.text: '#/components/schemas/inference._types.TextReasoningDetail' oneOf: - $ref: '#/components/schemas/inference._types.EncryptedReasoningDetail' - $ref: '#/components/schemas/inference._types.SummaryReasoningDetail' - $ref: '#/components/schemas/inference._types.TextReasoningDetail' inference._types.CompletionToolChoice: description: Controls which tool is called by the model. type: object properties: type: description: The type of the tool. type: string function: description: The tool choice function. allOf: - $ref: '#/components/schemas/inference._types.CompletionToolChoiceFunction' required: - type - function inference._types.InferenceEndpointInfoDeepSeek: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeDeepSeek' required: - inference_id - task_type inference._types.InferenceEndpointInfoGoogleVertexAI: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeGoogleVertexAI' required: - inference_id - task_type inference._types.DenseVector: description: 'Dense Embedding results are represented as Dense Vectors of floats.' type: array items: type: number inference._types.GoogleVertexAITaskSettings: type: object properties: auto_truncate: description: For a `text_embedding` task, truncate inputs longer than the maximum token length automatically. type: boolean top_n: description: For a `rerank` task, the number of the top N documents that should be returned. type: number thinking_config: externalDocs: url: https://ai.google.dev/gemini-api/docs/thinking#set-budget description: 'For a `completion` or `chat_completion` task, allows configuration of the thinking features for the model. Refer to the Google documentation for the allowable configurations for each model type.' allOf: - $ref: '#/components/schemas/inference._types.ThinkingConfig' max_tokens: externalDocs: url: https://docs.claude.com/en/api/messages#body-max-tokens description: 'For `completion` and `chat_completion` tasks, specifies the `max_tokens` value for requests sent to the Google Model Garden `anthropic` provider. If `provider` is not set to `anthropic`, this field is ignored. If `max_tokens` is specified - it must be a positive integer. If not specified, the default value of 1024 is used. Anthropic models require `max_tokens` to be set for each request. Please refer to the Anthropic documentation for more information.' type: number inference._types.ElserServiceSettings: type: object properties: adaptive_allocations: description: 'Adaptive allocations configuration details. If `enabled` is true, the number of allocations of the model is set based on the current load the process gets. When the load is high, a new model allocation is automatically created, respecting the value of `max_number_of_allocations` if it''s set. When the load is low, a model allocation is automatically removed, respecting the value of `min_number_of_allocations` if it''s set. If `enabled` is true, do not set the number of allocations manually.' allOf: - $ref: '#/components/schemas/inference._types.AdaptiveAllocations' num_allocations: description: 'The total number of allocations this model is assigned across machine learning nodes. Increasing this value generally increases the throughput. If adaptive allocations is enabled, do not set this value because it''s automatically set.' type: number num_threads: description: 'The number of threads used by each model allocation during inference. Increasing this value generally increases the speed per inference request. The inference process is a compute-bound process; `threads_per_allocations` must not exceed the number of available allocated processors per node. The value must be a power of 2. The maximum value is 32. > info > If you want to optimize your ELSER endpoint for ingest, set the number of threads to 1. If you want to optimize your ELSER endpoint for search, set the number of threads to greater than 1.' type: number required: - num_allocations - num_threads inference._types.Ai21ServiceType: type: string enum: - ai21 inference._types.TaskTypeAlibabaCloudAI: type: string enum: - text_embedding - rerank - completion - sparse_embedding inference._types.SummaryReasoningDetail: description: The reasoning summary detail includes a high-level summary of the model's reasoning process. allOf: - $ref: '#/components/schemas/inference._types.BaseReasoningDetail' - type: object properties: type: type: string enum: - reasoning.summary summary: description: 'The summary of the reasoning process generated by the model, which provides an overview of the key points and conclusions reached during the reasoning process.' type: string required: - type - summary inference._types.AzureAiStudioTaskSettings: type: object properties: do_sample: description: 'For a `completion` task, instruct the inference process to perform sampling. It has no effect unless `temperature` or `top_p` is specified.' type: number max_new_tokens: description: For a `completion` task, provide a hint for the maximum number of output tokens to be generated. type: number temperature: description: 'For a `completion` task, control the apparent creativity of generated completions with a sampling temperature. It must be a number in the range of 0.0 to 2.0. It should not be used if `top_p` is specified.' type: number top_p: description: 'For a `completion` task, make the model consider the results of the tokens with nucleus sampling probability. It is an alternative value to `temperature` and must be a number in the range of 0.0 to 2.0. It should not be used if `temperature` is specified.' type: number user: description: 'For a `text_embedding` task, specify the user issuing the request. This information can be used for abuse detection.' type: string return_documents: description: For a `rerank` task, return doc text within the results. type: boolean top_n: description: 'For a `rerank` task, the number of most relevant documents to return. It defaults to the number of the documents.' type: number inference._types.OpenShiftAiTaskType: type: string enum: - text_embedding - completion - chat_completion - rerank inference._types.AzureOpenAITaskType: type: string enum: - completion - chat_completion - text_embedding inference._types.ReasoningEffort: type: string enum: - xhigh - high - medium - low - minimal - none inference._types.InferenceEndpointInfoWatsonx: allOf: - $ref: '#/components/schemas/inference._types.InferenceEndpoint' - type: object properties: inference_id: description: The inference Id type: string task_type: description: The task type allOf: - $ref: '#/components/schemas/inference._types.TaskTypeWatsonx' required: - inference_id - task_type inference._types.TextEmbeddingInferenceResult: description: TextEmbeddingInferenceResult is an aggregation of mutually exclusive text_embedding variants type: object properties: text_embedding_bytes: type: array items: $ref: '#/components/schemas/inference._types.DenseEmbeddingByteResult' text_embedding_bits: type: array items: $ref: '#/components/schemas/inference._types.DenseEmbeddingByteResult' text_embedding: type: array items: $ref: '#/components/schemas/inference._types.DenseEmbeddingResult' minProperties: 1 maxProperties: 1 inference._types.TaskSettings: type: object inference._types.NvidiaServiceType: type: string enum: - nvidia inference._types.MistralServiceType: type: string enum: - mistral inference._types.AmazonSageMakerApi: type: string enum: - openai - elastic inference._types.JinaAIServiceType: type: string enum: - jinaai inference._types.TaskTypeGoogleVertexAI: type: string enum: - chat_completion - completion - text_embedding - rerank inference._types.ElserTaskType: type: string enum: - sparse_embedding inference._types.ImageUrlDetail: type: string enum: - auto - low - high _types.UnitMillis: description: Time unit for milliseconds type: number inference._types.AlibabaCloudServiceSettings: type: object properties: api_key: description: A valid API key for the AlibabaCloud AI Search API. type: string host: externalDocs: url: https://opensearch.console.aliyun.com/cn-shanghai/rag/api-key description: 'The name of the host address used for the inference task. You can find the host address in the API keys section of the documentation.' type: string rate_limit: description: 'This setting helps to minimize the number of rate limit errors returned from AlibabaCloud AI Search. By default, the `alibabacloud-ai-search` service sets the number of requests allowed per minute to `1000`.' allOf: - $ref: '#/components/schemas/inference._types.RateLimitSetting' service_id: description: 'The name of the model service to use for the inference task. The following service IDs are available for the `completion` task: * `ops-qwen-turbo` * `qwen-turbo` * `qwen-plus` * `qwen-max รท qwen-max-longcontext` The following service ID is available for the `rerank` task: * `ops-bge-reranker-larger` The following service ID is available for the `sparse_embedding` task: * `ops-text-sparse-embedding-001` The following service IDs are available for the `text_embedding` task: `ops-text-embedding-001` `ops-text-embedding-zh-001` `ops-text-embedding-en-001` `ops-text-embedding-002`' type: string workspace: description: The name of the workspace used for the inference task. type: string required: - api_key - host - service_id - workspace inference._types.TaskTypeELSER: type: string enum: - sparse_embedding responses: inference.get-200: description: '' content: application/json: schema: type: object properties: endpoints: type: array items: $ref: '#/components/schemas/inference._types.InferenceEndpointInfo' required: - endpoints inference.inference-200: description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceResult' inference.put-200: description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfo' inference.delete-200: description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.DeleteInferenceEndpointResult' inference.update-200: description: '' content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpointInfo' requestBodies: inference.put: content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpoint' examples: InferencePutExample1: description: An example body for a `PUT _inference/rerank/my-rerank-model` request. value: "{\n \"service\": \"cohere\",\n \"service_settings\": {\n \"model_id\": \"rerank-english-v3.0\",\n \"api_key\": \"{{COHERE_API_KEY}}\"\n }\n}" required: true inference.inference: content: application/json: schema: type: object properties: query: description: 'The query input, which is required only for the `rerank` task. It is not required for other tasks.' type: string input: description: 'The text on which you want to perform the inference task. It can be a single string or an array. > info > Inference endpoints for the `completion` task type currently only support a single string as input.' oneOf: - type: string - type: array items: type: string input_type: description: 'Specifies the input data type for the embedding model. The `input_type` parameter only applies to Inference Endpoints with the `embedding` or `text_embedding` task type. Possible values include: * `SEARCH` * `INGEST` * `CLASSIFICATION` * `CLUSTERING` Not all services support all values. Unsupported values will trigger a validation exception. Accepted values depend on the configured inference service, refer to the relevant service-specific documentation for more info. > info > The `input_type` parameter specified on the root level of the request body will take precedence over the `input_type` parameter specified in `task_settings`.' type: string task_settings: description: 'Task settings for the individual inference request. These settings are specific to the task type you specified and override the task settings specified when initializing the service.' allOf: - $ref: '#/components/schemas/inference._types.TaskSettings' required: - input required: true inference.update: content: application/json: schema: $ref: '#/components/schemas/inference._types.InferenceEndpoint' examples: InferenceUpdateExample1: description: An example body for a `PUT _inference/my-inference-endpoint/_update` request. value: "{\n \"service_settings\": {\n \"api_key\": \"\"\n },\n\"service\": \"example-service\"\n}" required: true parameters: inference.put-inference_id: in: path name: inference_id description: The inference Id required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple inference.put-timeout: in: query name: timeout description: 'Specifies the amount of time to wait for the inference endpoint to be created. The default depends on the task type: 120s for `completion` and `chat_completion`, and 30s for all other task types.' deprecated: false schema: $ref: '#/components/schemas/_types.Duration' style: form inference.update-inference_id: in: path name: inference_id description: The unique identifier of the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple inference.delete-task_type: in: path name: task_type description: The task type required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.TaskType' style: simple inference.update-timeout: in: query name: timeout description: 'Specifies the amount of time to wait for the inference endpoint to be updated. The default depends on the task type: 120s for `completion` and `chat_completion`, and 30s for all other task types.' deprecated: false schema: $ref: '#/components/schemas/_types.Duration' x-state: Generally available; Added in 9.5.0 style: form inference.get-task_type: in: path name: task_type description: The task type of the endpoint to return required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.TaskType' style: simple inference.inference-inference_id: in: path name: inference_id description: The unique identifier for the inference endpoint. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple inference.put-task_type: in: path name: task_type description: The task type. Refer to the integration list in the API description for the available task types. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.TaskType' style: simple inference.get-inference_id: in: path name: inference_id description: 'The inference Id of the endpoint to return. Using `_all` or `*` will return all endpoints with the specified `task_type` if one is specified, or all endpoints for all task types if no `task_type` is specified' required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple inference.update-task_type: in: path name: task_type description: The type of inference task that the model performs. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.TaskType' style: simple inference.inference-timeout: in: query name: timeout description: The amount of time to wait for the inference request to complete. deprecated: false schema: default: 30s allOf: - $ref: '#/components/schemas/_types.Duration' style: form inference.delete-inference_id: in: path name: inference_id description: The inference identifier. required: true deprecated: false schema: $ref: '#/components/schemas/_types.Id' style: simple inference.inference-task_type: in: path name: task_type description: The type of inference task that the model performs. required: true deprecated: false schema: $ref: '#/components/schemas/inference._types.TaskType' style: simple inference.delete-dry_run: in: query name: dry_run description: When true, checks the semantic_text fields and inference processors that reference the endpoint and returns them in a list, but does not delete the endpoint. deprecated: false schema: default: false type: boolean style: form inference.delete-force: in: query name: force description: When true, the inference endpoint is forcefully deleted even if it is still being used by ingest processors or semantic text fields. deprecated: false schema: default: false type: boolean style: form