{ "openapi": "3.0.3", "info": { "title": "Multimodal Features", "version": "2.0", "description": "Your project description" }, "paths": { "/multimodal/chat/": { "post": { "operationId": "multimodal_multimodal_chat_create", "description": "
Available Providers\n\n\n\n|Provider|Model|Version|Price|Billing unit|\n|----|----|-------|-----|------------|\n|**anthropic**|**claude-3-5-haiku-latest**|`bedrock-2023-05-31`|4.0 (per 1000000 token)|1 token\n|**anthropic**|-|`bedrock-2023-05-31`|15.0 (per 1000000 token)|1 token\n|**anthropic**|**claude-3-7-sonnet-20250219**|`bedrock-2023-05-31`|15.0 (per 1000000 token)|1 token\n|**google**|**gemini-1.5-flash**|`v1`|0.6 (per 1000000 token)|1 token\n|**google**|**gemini-1.5-pro**|`v1`|10.0 (per 1000000 token)|1 token\n|**google**|**gemini-1.5-flash-latest**|`v1`|0.6 (per 1000000 token)|1 token\n|**google**|-|`v1`|0.6 (per 1000000 token)|1 token\n|**google**|**gemini-2.0-flash-lite**|`v1`|0.3 (per 1000000 token)|1 token\n|**google**|**gemini-2.5-flash**|`v1`|2.5 (per 1000000 token)|1 token\n|**google**|**gemini-3-pro-preview**|`v1`|18.0 (per 1000000 token)|1 token\n|**google**|**gemini-2.0-flash**|`v1`|0.4 (per 1000000 token)|1 token\n|**google**|**gemini-2.5-pro**|`v1`|15.0 (per 1000000 token)|1 token\n|**google**|**gemini-2.5-pro-preview-03-25**|`v1`|15.0 (per 1000000 token)|1 token\n|**google**|**gemini-1.5-pro-latest**|`v1`|10.0 (per 1000000 token)|1 token\n|**google**|**gemini-2.0-flash-exp**|`v1`|10.0 (per 1000000 token)|1 token\n|**google**|**gemini-2.5-pro-exp-03-25**|`v1`|0.0 (per 1000000 token)|1 token\n|**openai**|**o1**|`v1Beta`|60.0 (per 1000000 token)|1 token\n|**openai**|**gpt-4o**|`v1Beta`|10.0 (per 1000000 token)|1 token\n|**openai**|**gpt-4-turbo**|`v1Beta`|30.0 (per 1000000 token)|1 token\n|**openai**|-|`v1Beta`|10.0 (per 1000000 token)|1 token\n|**openai**|**gpt-4.1-2025-04-14**|`v1Beta`|8.0 (per 1000000 token)|1 token\n|**openai**|**gpt-4.1-mini-2025-04-14**|`v1Beta`|1.6 (per 1000000 token)|1 token\n|**openai**|**gpt-4.1-nano-2025-04-14**|`v1Beta`|0.4 (per 1000000 token)|1 token\n|**openai**|**gpt-4o-mini**|`v1Beta`|0.6 (per 1000000 token)|1 token\n|**openai**|**gpt-5.2**|`v1Beta`|14.0 (per 1000000 token)|1 token\n|**openai**|**gpt-5**|`v1Beta`|10.0 (per 1000000 token)|1 token\n|**openai**|**gpt-5-mini**|`v1Beta`|2.0 (per 1000000 token)|1 token\n|**openai**|**gpt-5-nano**|`v1Beta`|0.4 (per 1000000 token)|1 token\n|**openai**|**gpt-5-chat-latest**|`v1Beta`|10.0 (per 1000000 token)|1 token\n|**openai**|**o1-mini**|`v1Beta`|12.0 (per 1000000 token)|1 token\n|**openai**|**o1-2024-12-17**|`v1Beta`|60.0 (per 1000000 token)|1 token\n|**xai**|-|`v1`|10.0 (per 1000000 token)|1 token\n|**xai**|**grok-2-vision**|`v1`|10.0 (per 1000000 token)|1 token\n|**xai**|**grok-2-vision-latest**|`v1`|10.0 (per 1000000 token)|1 token\n|**xai**|**grok-2-vision-1212**|`v1`|10.0 (per 1000000 token)|1 token\n|**amazon**|-|`boto3 (v1.29.6)`|3.2 (per 1000000 token)|1 token\n|**amazon**|**amazon.nova-pro-v1:0**|`boto3 (v1.29.6)`|3.2 (per 1000000 token)|1 token\n|**mistral**|-|`v0.0.1`|6.0 (per 1000000 token)|1 token\n|**mistral**|**pixtral-large-latest**|`v0.0.1`|6.0 (per 1000000 token)|1 token\n|**microsoft**|**gpt-4o**|`Azure AI Foundry`|15.0 (per 1000000 token)|1 token\n\n\n
\n\n

Supported Models

\n\n
Default Models\n\n\n\n\n\n|Name|Value|\n|----|-----|\n|**anthropic**|`claude-3-5-sonnet-latest`|\n|**google**|`gemini-2.0-flash`|\n|**openai**|`gpt-4o`|\n|**xai**|`grok-2-vision-latest`|\n|**amazon**|`amazon.nova-pro-v1:0`|\n|**mistral**|`pixtral-large-latest`|\n|**microsoft**|`gpt-4o`|\n\n
", "summary": "Chat", "tags": [ "Chat" ], "requestBody": { "content": { "application/json": { "schema": { "$ref": "#/components/schemas/multimodalchatChatRequest" }, "examples": { "RequestExample": { "value": { "providers": "google,microsoft,openai,anthropic,xai,amazon,mistral", "messages": [ { "role": "user", "content": [ { "type": "text", "content": { "text": "Describe this image please ! " } }, { "type": "media_url", "content": { "media_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg", "media_type": "image/jpeg" } } ] } ], "chatbot_global_action": "", "temperature": 0, "max_tokens": 200, "stop_sequences": [], "top_k": 0, "top_p": 0 }, "summary": "Request Example" } } } }, "required": true }, "security": [ { "FeatureApiAuth": [] } ], "responses": { "200": { "content": { "application/json": { "schema": { "$ref": "#/components/schemas/multimodalchatResponseModel" }, "examples": { "ResponseExample": { "value": { "google": { "generated_text": "Ahoy, matey! Shiver me timbers, that there's a picture of a fine wooden boardwalk cuttin' through a field o' tall, green grass under a sky as blue as a mermaid's eye! The sun be shinin' bright, and there ain't a cloud in sight, except for a few fluffy ones driftin' lazily about. Looks like a peaceful spot fer a stroll, aye? Avast ye, and enjoy the view!\n", "messages": [ { "role": "user", "content": [ { "type": "text", "content": { "media_url": null, "media_base64": null, "text": "Describe this image please ! ", "media_type": null } }, { "type": "media_url", "content": { "media_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg", "media_base64": null, "text": null, "media_type": "image/jpeg" } } ] }, { "role": "assistant", "content": [ { "type": "text", "content": { "media_url": null, "media_base64": null, "text": "Ahoy, matey! Shiver me timbers, that there's a picture of a fine wooden boardwalk cuttin' through a field o' tall, green grass under a sky as blue as a mermaid's eye! The sun be shinin' bright, and there ain't a cloud in sight, except for a few fluffy ones driftin' lazily about. Looks like a peaceful spot fer a stroll, aye? Avast ye, and enjoy the view!\n", "media_type": null } } ] } ], "usage": { "completion_tokens": 101, "prompt_tokens": 270, "total_tokens": 371, "completion_tokens_details": null, "prompt_tokens_details": null }, "cost": 0.0 }, "microsoft": { "generated_text": "Arrr, matey! What we have here be a fine scene of nature's bounty! A wooden plank path stretches straight ahead, leadin' through a lush green meadow filled with tall grass swayin' in the gentle breeze. The sky above be painted with a brilliant blue, streaked with soft, wispy clouds, as if the heavens themselves be smilin' down on this tranquil land. In the distance, ye can spy a line of trees and bushes, addin' depth to this peaceful landscape. Aye, 'tis a place where a pirate might rest his weary bones and dream of treasure!", "messages": [ { "role": "user", "content": [ { "type": "text", "content": { "media_url": null, "media_base64": null, "text": "Describe this image please ! ", "media_type": null } }, { "type": "media_url", "content": { "media_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg", "media_base64": null, "text": null, "media_type": "image/jpeg" } } ] }, { "role": "assistant", "content": [ { "type": "text", "content": { "media_url": null, "media_base64": null, "text": "Arrr, matey! What we have here be a fine scene of nature's bounty! A wooden plank path stretches straight ahead, leadin' through a lush green meadow filled with tall grass swayin' in the gentle breeze. The sky above be painted with a brilliant blue, streaked with soft, wispy clouds, as if the heavens themselves be smilin' down on this tranquil land. In the distance, ye can spy a line of trees and bushes, addin' depth to this peaceful landscape. Aye, 'tis a place where a pirate might rest his weary bones and dream of treasure!", "media_type": null } } ] } ], "usage": { "completion_tokens": 124, "prompt_tokens": 930, "total_tokens": 1054, "completion_tokens_details": { "accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 0, "rejected_prediction_tokens": 0 }, "prompt_tokens_details": { "audio_tokens": 0, "cached_tokens": 0 } }, "cost": 0.0 }, "openai": { "generated_text": "Ahoy! What we have here be a splendid vista, me hearty! A wooden plank path cuts through a sea of lush green grass, leading ye towards the horizon under a sky painted with soft clouds. The land be flanked by wild shrubbery and trees, a true haven for any landlubber or pirate lookin' for a bit o' peace away from the roarin' seas. It's a fine day with the sun shinin' bright, makin' the greens look vibrant and the path invitin' for a leisurely stroll or a treasure hunt, should ye be so inclined!", "messages": [ { "role": "user", "content": [ { "type": "text", "content": { "media_url": null, "media_base64": null, "text": "Describe this image please ! ", "media_type": null } }, { "type": "media_url", "content": { "media_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg", "media_base64": null, "text": null, "media_type": "image/jpeg" } } ] }, { "role": "assistant", "content": [ { "type": "text", "content": { "media_url": null, "media_base64": null, "text": "Ahoy! What we have here be a splendid vista, me hearty! A wooden plank path cuts through a sea of lush green grass, leading ye towards the horizon under a sky painted with soft clouds. The land be flanked by wild shrubbery and trees, a true haven for any landlubber or pirate lookin' for a bit o' peace away from the roarin' seas. It's a fine day with the sun shinin' bright, makin' the greens look vibrant and the path invitin' for a leisurely stroll or a treasure hunt, should ye be so inclined!", "media_type": null } } ] } ], "usage": { "completion_tokens": 124, "prompt_tokens": 1094, "total_tokens": 1218, "completion_tokens_details": { "accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 0, "rejected_prediction_tokens": 0 }, "prompt_tokens_details": { "audio_tokens": 0, "cached_tokens": 0 } }, "cost": 0.0 }, "anthropic": { "generated_text": "Arrr matey! Feast yer eyes on this magnificent sight! \n\n'Tis a wooden boardwalk, straight as an arrow, cuttin' through a sea of emerald grass that be swayin' in the breeze like waves on the ocean! The planks be weathered and gray, just like me old ship's deck, stretching far into the distance.\n\nAbove, the sky be a brilliant blue, with wispy white clouds sailin' across it like ghost ships on the horizon. The sunlight be makin' the grass glow like treasure, and off in the distance, ye can spot trees standin' guard like silent sentinels.\n\nThis here path would make for a fine walk for any landlubber, though it ain't quite as excitin' as walkin' the plank on me ship! The marsh grass be tall and thick on both sides, remindin' me of the reeds", "messages": [ { "role": "user", "content": [ { "type": "text", "content": { "media_url": null, "media_base64": null, "text": "Describe this image please ! ", "media_type": null } }, { "type": "media_url", "content": { "media_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg", "media_base64": null, "text": null, "media_type": "image/jpeg" } } ] }, { "role": "assistant", "content": [ { "type": "text", "content": { "media_url": null, "media_base64": null, "text": "Arrr matey! Feast yer eyes on this magnificent sight! \n\n'Tis a wooden boardwalk, straight as an arrow, cuttin' through a sea of emerald grass that be swayin' in the breeze like waves on the ocean! The planks be weathered and gray, just like me old ship's deck, stretching far into the distance.\n\nAbove, the sky be a brilliant blue, with wispy white clouds sailin' across it like ghost ships on the horizon. The sunlight be makin' the grass glow like treasure, and off in the distance, ye can spot trees standin' guard like silent sentinels.\n\nThis here path would make for a fine walk for any landlubber, though it ain't quite as excitin' as walkin' the plank on me ship! The marsh grass be tall and thick on both sides, remindin' me of the reeds", "media_type": null } } ] } ], "usage": { "completion_tokens": 200, "prompt_tokens": 1590, "total_tokens": 1790, "completion_tokens_details": null, "prompt_tokens_details": { "audio_tokens": null, "cached_tokens": 0 }, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0 }, "cost": 0.0 }, "xai": { "generated_text": "Arrr, matey! This here be a fine image o' a wooden boardwalk stretchin' out into a vast sea o' green grass, leadin' ye towards the horizon. The sky above be a brilliant blue, dotted with fluffy white clouds, like sails on the high seas. The path be straight and true, invitin' ye to walk the plank into the heart o' this lush meadow. Aye, 'tis a peaceful sight, perfect fer a pirate lookin' fer a bit o' tranquility on land!", "messages": [ { "role": "user", "content": [ { "type": "text", "content": { "media_url": null, "media_base64": null, "text": "Describe this image please ! ", "media_type": null } }, { "type": "media_url", "content": { "media_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg", "media_base64": null, "text": null, "media_type": "image/jpeg" } } ] }, { "role": "assistant", "content": [ { "type": "text", "content": { "media_url": null, "media_base64": null, "text": "Arrr, matey! This here be a fine image o' a wooden boardwalk stretchin' out into a vast sea o' green grass, leadin' ye towards the horizon. The sky above be a brilliant blue, dotted with fluffy white clouds, like sails on the high seas. The path be straight and true, invitin' ye to walk the plank into the heart o' this lush meadow. Aye, 'tis a peaceful sight, perfect fer a pirate lookin' fer a bit o' tranquility on land!", "media_type": null } } ] } ], "usage": { "completion_tokens": 109, "prompt_tokens": 277, "total_tokens": 386, "completion_tokens_details": null, "prompt_tokens_details": { "audio_tokens": 0, "cached_tokens": 0 } }, "cost": 0.0 }, "amazon": { "generated_text": "Ahoy, matey! Shiver me timbers, what a sight for these ol' pirate eyes!\n\nYe be lookin' at a right fine stretch o' land, as green as the emerald seas! There be a wooden walkway, straight as an arrow, cuttin' through a sea o' tall grass that'd tickle yer chin if ye were standin' in it.\n\nThe sky above, she's a beauty! Blue as a sapphire with wispy white clouds stretchin' across like sails on the horizon. Ye can almost feel the warm breeze blowin' through yer whiskers.\n\nOff in the distance, ye can spy some trees standin' tall, like a crew of landlubbers waitin' to greet ye at the end of yer journey. The whole scene be as peaceful as a calm day at sea, with nary a storm cloud in sight.\n\nThis", "messages": [ { "role": "user", "content": [ { "type": "text", "content": { "media_url": null, "media_base64": null, "text": "Describe this image please ! ", "media_type": null } }, { "type": "media_url", "content": { "media_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg", "media_base64": null, "text": null, "media_type": "image/jpeg" } } ] }, { "role": "assistant", "content": [ { "type": "text", "content": { "media_url": null, "media_base64": null, "text": "Ahoy, matey! Shiver me timbers, what a sight for these ol' pirate eyes!\n\nYe be lookin' at a right fine stretch o' land, as green as the emerald seas! There be a wooden walkway, straight as an arrow, cuttin' through a sea o' tall grass that'd tickle yer chin if ye were standin' in it.\n\nThe sky above, she's a beauty! Blue as a sapphire with wispy white clouds stretchin' across like sails on the horizon. Ye can almost feel the warm breeze blowin' through yer whiskers.\n\nOff in the distance, ye can spy some trees standin' tall, like a crew of landlubbers waitin' to greet ye at the end of yer journey. The whole scene be as peaceful as a calm day at sea, with nary a storm cloud in sight.\n\nThis", "media_type": null } } ] } ], "usage": { "completion_tokens": 200, "prompt_tokens": 1590, "total_tokens": 1790, "completion_tokens_details": null, "prompt_tokens_details": null }, "cost": 0.0 }, "mistral": { "generated_text": "Arr matey, this here image be a sight for sore eyes! It shows a wooden plank walkway, stretchin' into the horizon, invitin' us to explore what lies beyond. The path be flanked by tall, green grasses, swayin' gently in the breeze. The sky above be a clear blue, with wisps of clouds scattered about, like a hidden treasure map. In the distance, ye can see a line of trees standin' tall and proud, guardin' the secrets of the land. The sun be castin' a warm glow over the scene, makin' it feel like a true paradise. It be a perfect place to bury some treasure, savvy?", "messages": [ { "role": "user", "content": [ { "type": "text", "content": { "media_url": null, "media_base64": null, "text": "Describe this image please ! ", "media_type": null } }, { "type": "media_url", "content": { "media_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg", "media_base64": null, "text": null, "media_type": "image/jpeg" } } ] }, { "role": "assistant", "content": [ { "type": "text", "content": { "media_url": null, "media_base64": null, "text": "Arr matey, this here image be a sight for sore eyes! It shows a wooden plank walkway, stretchin' into the horizon, invitin' us to explore what lies beyond. The path be flanked by tall, green grasses, swayin' gently in the breeze. The sky above be a clear blue, with wisps of clouds scattered about, like a hidden treasure map. In the distance, ye can see a line of trees standin' tall and proud, guardin' the secrets of the land. The sun be castin' a warm glow over the scene, makin' it feel like a true paradise. It be a perfect place to bury some treasure, savvy?", "media_type": null } } ] } ], "usage": { "completion_tokens": 149, "prompt_tokens": 2746, "total_tokens": 2895, "completion_tokens_details": null, "prompt_tokens_details": null }, "cost": 0.0 } }, "summary": "Response Example" } } } }, "description": "" }, "400": { "content": { "application/json": { "schema": { "$ref": "#/components/schemas/BadRequest" } } }, "description": "" }, "500": { "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } }, "description": "" }, "403": { "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } }, "description": "" }, "404": { "content": { "application/json": { "schema": { "$ref": "#/components/schemas/NotFoundResponse" } } }, "description": "" } } } } }, "components": { "schemas": { "BadRequest": { "type": "object", "properties": { "error": { "$ref": "#/components/schemas/NestedBadRequest" } }, "required": [ "error" ] }, "ChatMessage": { "properties": { "type": { "allOf": [ { "$ref": "#/components/schemas/TypeEnum" } ], "title": "Type" }, "content": { "$ref": "#/components/schemas/ChatMessageContent" } }, "required": [ "type", "content" ], "title": "ChatMessage", "type": "object" }, "ChatMessageContent": { "properties": { "media_url": { "default": null, "title": "Media Url", "type": "string" }, "media_base64": { "default": null, "title": "Media Base64", "type": "string" }, "text": { "default": null, "title": "Text", "type": "string" }, "media_type": { "default": null, "title": "Media Type", "type": "string" } }, "title": "ChatMessageContent", "type": "object" }, "ChatMessageDataClass": { "properties": { "role": { "title": "Role", "type": "string" }, "content": { "items": { "$ref": "#/components/schemas/ChatMessage" }, "title": "Content", "type": "array" } }, "required": [ "role" ], "title": "ChatMessageDataClass", "type": "object" }, "Error": { "type": "object", "properties": { "error": { "$ref": "#/components/schemas/NestedError" } }, "required": [ "error" ] }, "FieldError": { "type": "object", "properties": { "": { "type": "array", "items": { "type": "string" } } }, "required": [ "" ] }, "NestedBadRequest": { "type": "object", "properties": { "type": { "type": "string" }, "message": { "$ref": "#/components/schemas/FieldError" } }, "required": [ "message", "type" ] }, "NestedError": { "type": "object", "properties": { "type": { "type": "string" }, "message": { "type": "string" } }, "required": [ "message", "type" ] }, "NotFoundResponse": { "type": "object", "properties": { "details": { "type": "string", "default": "Not Found" } } }, "ReasoningEffortEnum": { "enum": [ "low", "medium", "high" ], "type": "string", "description": "**Choices**:\n- 'low': Minimal reasoning, quick responses\n- 'medium': Balanced reasoning approach\n- 'high': In-depth, comprehensive reasoning\n\n**Example**: 'high' for complex problem-solving tasks\n\n* `low` - low\n* `medium` - medium\n* `high` - high" }, "StatusEnum": { "enum": [ "sucess", "fail" ], "type": "string" }, "TypeEnum": { "enum": [ "media_url", "media_base64", "text" ], "type": "string" }, "multimodalchatChatDataClass": { "properties": { "generated_text": { "title": "Generated Text", "type": "string" }, "messages": { "items": { "$ref": "#/components/schemas/ChatMessageDataClass" }, "title": "Messages", "type": "array" }, "original_response": { "default": null, "description": "original response sent by the provider, hidden by default, show it by passing the `show_original_response` field to `true` in your request", "title": "Original Response" }, "status": { "allOf": [ { "$ref": "#/components/schemas/StatusEnum" } ], "title": "Status" } }, "required": [ "generated_text", "status" ], "title": "multimodalchatChatDataClass", "type": "object" }, "multimodalchatChatRequest": { "type": "object", "properties": { "settings": { "type": "string", "default": {}, "description": "A dictionnary or a json object to specify specific models to use for some providers.
It can be in the following format: {\"google\" : \"google_model\", \"ibm\": \"ibm_model\"...}.\n " }, "providers": { "type": "array", "items": { "type": "string", "minLength": 1 }, "description": "It can be one (ex: **'amazon'** or **'google'**) or multiple provider(s) (ex: **'amazon,microsoft,google'**) that the data will be redirected to in order to get the processed results.
Providers can also be invoked with specific models (ex: providers: **'amazon/model1, amazon/model2, google/model3'**)" }, "fallback_providers": { "type": "array", "items": { "type": "string" }, "default": [], "description": "Providers in this list will be used as fallback if the call to provider in `providers` parameter fails.\n To use this feature, you must input **only one** provider in the `providers` parameter. but you can put up to 5 fallbacks.\n\nThey will be tried in the same order they are input, and it will stop to the first provider who doesn't fail.\n\n\n*Doesn't work with async subfeatures.*\n ", "maxItems": 5 }, "response_as_dict": { "type": "boolean", "default": true, "description": "Optional : When set to **true** (default), the response is an object of responses with providers names as keys :
\n ``` {\"google\" : { \"status\": \"success\", ... }, } ```
\n When set to **false** the response structure is a list of response objects :
\n ``` [{\"status\": \"success\", \"provider\": \"google\" ... }, ] ```.
\n " }, "attributes_as_list": { "type": "boolean", "default": false, "description": "Optional : When set to **false** (default) the structure of the extracted items is list of objects having different attributes :
\n ```{'items': [{\"attribute_1\": \"x1\",\"attribute_2\": \"y2\"}, ... ]}```
\n When it is set to **true**, the response contains an object with each attribute as a list :
\n ```{ \"attribute_1\": [\"x1\",\"x2\", ...], \"attribute_2\": [y1, y2, ...]}``` " }, "show_base_64": { "type": "boolean", "default": true }, "show_original_response": { "type": "boolean", "default": false, "description": "Optional : Shows the original response of the provider.
\n When set to **true**, a new attribute *original_response* will appear in the response object." }, "messages": { "type": "array", "items": { "type": "object", "additionalProperties": {} }, "description": "A list containing all the conversations between the user and the assistant. Each item in the list should be a dictionary with two keys: 'role' and 'message'.\n\n**role**: Specifies the role of the speaker and can have the values 'user' or 'assistant'.\n\n**message**: A list of dictionaries. Each dictionary in the 'message' list must contain the keys 'type' and 'content'.\n\n#### Structure\n- **type**: Specifies the type of content and can be 'media_url', 'media_base64', or 'text'.\n- **content**: A dictionary with the actual content based on the 'type':\n - If 'type' is 'media_url', 'content' must contain 'media_url' and must not contain 'media_base64' or 'text'.\n - If 'type' is 'media_base64', 'content' must contain 'media_base64' and must not contain 'media_url' or 'text'.\n - If 'type' is 'text', 'content' must contain 'text' and must not contain 'media_url' or 'media_base64'.\n\n#### Example\n```json\n[\n {\n 'role': 'user',\n 'content': [\n {\n 'type': 'text',\n 'content': {'text': 'Describe this image'}\n },\n {\n 'type': 'media_url',\n 'content': {\n 'media_url': 'https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg',\n 'media_type': 'image/jpeg'}\n },\n ]\n }\n]\n```" }, "chatbot_global_action": { "type": "string", "nullable": true, "description": "A system message that helps set the behavior of the assistant. For example, 'You are a helpful assistant'." }, "temperature": { "type": "number", "format": "double", "maximum": 2, "minimum": 0, "default": 0.0, "description": "Controls the creativity of the model's responses. Higher values (up to 2) make the output more random, while lower values make it more focused and deterministic. A value of 0 (argmax sampling) is useful for scenarios requiring precise answers." }, "max_tokens": { "type": "integer", "maximum": 300000, "minimum": 1, "default": 2048, "description": "The maximum number of tokens to generate in the completion. This value, combined with the token count of your prompt, cannot exceed the model's context length." }, "stop_sequences": { "type": "array", "items": { "type": "string" }, "default": [], "description": "A list of sequences where the model will stop generating further tokens. Useful for controlling response length and format." }, "top_k": { "type": "integer", "maximum": 500, "minimum": 0, "description": "Limits the sampling pool to the top K options for each token. Setting this to a lower value can make the output more focused and deterministic." }, "top_p": { "type": "number", "format": "double", "maximum": 1, "minimum": 0, "description": "Enables nucleus sampling, where the model considers the smallest number of tokens whose cumulative probability is at least top_p. This allows for a dynamic selection of tokens based on probability, offering a balance between focus and creativity." }, "reasoning_effort": { "allOf": [ { "$ref": "#/components/schemas/ReasoningEffortEnum" } ], "description": "Optional parameter to control the model's reasoning depth. \nAllows specifying the level of analytical effort in generating responses. \n\n**Choices**:\n- 'low': Minimal reasoning, quick responses\n- 'medium': Balanced reasoning approach\n- 'high': In-depth, comprehensive reasoning\n\n**Example**: 'high' for complex problem-solving tasks\n\n* `low` - low\n* `medium` - medium\n* `high` - high" } }, "required": [ "messages", "providers" ] }, "multimodalchatResponseModel": { "properties": { "amazon": { "$ref": "#/components/schemas/multimodalchatChatDataClass", "default": null }, "mistral": { "$ref": "#/components/schemas/multimodalchatChatDataClass", "default": null }, "openai": { "$ref": "#/components/schemas/multimodalchatChatDataClass", "default": null }, "microsoft": { "$ref": "#/components/schemas/multimodalchatChatDataClass", "default": null }, "xai": { "$ref": "#/components/schemas/multimodalchatChatDataClass", "default": null }, "google": { "$ref": "#/components/schemas/multimodalchatChatDataClass", "default": null }, "anthropic": { "$ref": "#/components/schemas/multimodalchatChatDataClass", "default": null } }, "title": "multimodalchatResponseModel", "type": "object" } }, "securitySchemes": { "FeatureApiAuth": { "type": "http", "scheme": "bearer", "bearerFormat": "JWT" } } }, "servers": [ { "url": "https://api.edenai.run/v2" } ] }