[whatmodel.sh](../../README.md) / Mistral Medium 3.1 (2508) # Mistral Medium 3.1 (2508) Mistral · released 2025-08 · closed weights **score 63.8** ±2.2 · comparison only: no current run option verified · from 4 benchmark results across 3 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks Mistral Medium 3.1 (2508) among every model on that task or test, worst to best. 
task                vs the others          rank  counts  tests
agent tasks         ▁▁▂▆█▄●▄▄▃▁▁   #124 of 354*     11%      1
chat & writing      ▁▁▁▃▅▄▇█▇●▇▃   #123 of 474*     51%      1
vision & documents  ▁▁▂▃▂▃▅●▆██▄   #119 of 189*     34%      1
agent tasks · #124 of 354 · 11% of its score Beats [o3-pro](o3-pro.md), [Grok 4.3](grok-4-3.md), [Qwen3.6 Plus](qwen3-6-plus.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Devstral 2 (2512)](devstral-2.md), [Nova 2 Pro (Preview)](nova-2-pro.md) here, though they rank lower overall. 
test                                result  vs the others in its run  difficulty      counts  why
Galileo Agent Leaderboard v2 (act…   61.0%  ▅▃·▃▃▅▅█▃█▅● beats 18/19  hard · avg 26%     11%  at its level
chat & writing · #123 of 474 · 51% of its score Beats [LongCat 2.0 (1.6T-A48B)](longcat-2-0.md), [Inkling Small (266B-A12B)](inkling-small.md), [GPT-5.4 nano](gpt-5-4-nano.md) here, though they rank higher overall. Trails [Mistral Large 3 (675B-A41B)](mistral-large-3.md), [Gemma 4 12B](gemma-4-12b.md), [GPT-4.1](gpt-4-1.md) here, though they rank lower overall. 
test            result  vs the others in its run    difficulty  counts  why
LMArena (text)  1407.8  ▁▁▁▂▃▃▃▆▆●█▃ beats 252/385  avg 1260       51%  blunt
vision & documents · #119 of 189 · 34% of its score Beats [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md) here, though they rank higher overall. Trails [GPT-4o (2024-11-20)](gpt-4o-2024-11-20.md), [Qwen3-VL 30B-A3B Thinking](qwen3-vl-30b-a3b-thinking.md), [Mistral Small 4 (119B-A6.5B)](mistral-small-4.md) here, though they rank lower overall. 
test                             result  vs the others in its run   difficulty  counts  why
Arena vision (style controlled)  1159.5  ▁▁▂▂▂▂▃●▅▅█▇ beats 53/152  avg 1061       34%  blunt
other tests (1) 
test                                result  vs the others in its run   difficulty           counts  why
Vectara hallucination leaderboard…   88.6%  ▁▁▂●▄▄█▆▃▂▂▁ beats 18/140  very easy · avg 94%    1.7%  blunt · maxed out · noisy
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                             results  share of the score
LMArena leaderboard dataset              2   85% ▇▇▇▇▇▇▇▇▇▇
Galileo Agent Leaderboard                1   11% ▇
Vectara hallucination leaderboard        1    2% ▇
prior: pull toward the middle                 2% ▇
weighted average                            = 63.8
Licences: LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Galileo Agent Leaderboard ([Apache-2.0](../../licenses/galileo-agent-leaderboard-Apache-2.0.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/mistral-medium-3-1) · [JSON](https://whatmodel.sh/api?format=json&all=1)