[whatmodel.sh](../../README.md) / Mistral Medium 3 (2505) # Mistral Medium 3 (2505) Mistral · released 2025-05 · proprietary **score 60.0** ±1.2 · comparison only: no current run option verified · from 8 benchmark results across 5 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks Mistral Medium 3 (2505) among every model on that task or test, worst to best. 
task                vs the others          rank  counts  tests
agent tasks         ▁▁▂▆●▄▅▄▄▃▁▁   #238 of 354*    6.8%      1
coding              ▁▁▁▃▄▆█●▅▄▃▁   #210 of 539*    8.4%      1
chat & writing      ▁▁▁▃▅▄▇█▇●▇▃   #149 of 474*     14%      1
math                ▁▁▃▆██●█▃▁▁▁   #211 of 530*     34%      2
vision & documents  ▁▁▂▃▂▃▅●▆██▄   #120 of 189*    9.4%      1
agent tasks · #238 of 354 · 6.8% of its score Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md), [DeepSeek R1 (Jan '25)](deepseek-r1.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Mellum2 12B-A2.5B Instruct](mellum2-12b-a2-5b-instruct.md) here, though they rank lower overall. 
test                             result  vs the others in its run  difficulty      counts  why
BFCL V4 (December 2025 archive)   37.6%  ▁▁▁▂██●▅▆▂▂▂ beats 43/77  hard · avg 31%    6.8%  at its level
coding · #210 of 539 · 8.4% of its score Beats [Qwen3.5 9B](qwen3-5-9b.md), [Gemini 2.5 Flash](gemini-2-5-flash.md), [Qwen3-VL 32B Thinking](qwen3-vl-32b-thinking.md) here, though they rank higher overall. Trails [QwQ-32B-Preview](qwq-32b-preview.md), [Ministral 3 8B Reasoning 2512](ministral-3-8b-reasoning.md), [Phi-4-reasoning 14B](phi-4-reasoning.md) here, though they rank lower overall. 
test                                result  vs the others in its run  difficulty      counts  why
WebApp1K-Duo-React pass@1 (onekq-…   53.7%  ▁··▁·▁▁▁●█▇█ beats 7/45   hard · avg 29%    8.4%  at its level
chat & writing · #149 of 474 · 14% of its score Beats [LongCat 2.0 (1.6T-A48B)](longcat-2-0.md), [o3-mini](o3-mini.md), [MiMo-V2-Flash (309B-A15B)](mimo-v2-flash.md) here, though they rank higher overall. Trails [LFM2.5 2.6B](lfm2-5-2-6b.md), [Nemotron-Cascade-8B](nemotron-cascade-8b.md) here, though they rank lower overall. 
test            result  vs the others in its run    difficulty  counts  why
LMArena (text)  1387.3  ▁▁▁▂▃▃▃▆▆●█▃ beats 227/385  avg 1260       14%  blunt
math · #211 of 530 · 34% of its score Beats [MiniMax-M2.1](minimax-m2-1.md), [DeepSeek V3 0324](deepseek-v3-0324.md) here, though they rank higher overall. Trails [Phi-4-mini-reasoning 3.8B](phi-4-mini-reasoning.md), [OpenThinker3 7B](openthinker3-7b.md), [Olmo 3 7B Think](olmo-3-7b-think.md) here, though they rank lower overall. 
test                      result  vs the others in its run   difficulty            counts  why
MATH level 5               81.6%  ▂▂▅▆▄█▆●▅▂▃▄ beats 66/92   hard · avg 28%           23%  sharp
OTIS mock AIME 2024–2025   32.2%  ▃▃▄▃▃●▇▇█▃▂▅ beats 68/197  very hard · avg 7.2%     11%  sharp · at its level · noisy
vision & documents · #120 of 189 · 9.4% of its score Beats [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md), [GPT-5 nano](gpt-5-nano.md) here, though they rank higher overall. Trails [GPT-4o (2024-11-20)](gpt-4o-2024-11-20.md), [Qwen3-VL 30B-A3B Thinking](qwen3-vl-30b-a3b-thinking.md), [Mistral Small 4 (119B-A6.5B)](mistral-small-4.md) here, though they rank lower overall. 
test                             result  vs the others in its run   difficulty  counts  why
Arena vision (style controlled)  1155.4  ▁▁▂▂▂▂▃●▅▅█▇ beats 51/152  avg 1061      9.4%  blunt
other tests (2) 
test                                result  vs the others in its run   difficulty      counts  why
GPQA Diamond                         59.5%  ▁▁▁▁▂▄▆●▅▇█▅ beats 87/211  hard · avg 39%     24%  sharp · at its level
DiagnosticIQ industrial maintenan…   60.3%  ▁··▃▁▁▁●█▅▁▅ beats 9/24    hard · avg 51%    3.0%  blunt · at its level
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
Epoch AI                                        3   58% ▇▇▇▇▇▇▇
LMArena leaderboard dataset                     2   23% ▇▇▇
WebApp1K models leaderboard                     1    8% ▇
Berkeley Function Calling Leaderboard r…        1    7% ▇
DiagnosticIQ: A Benchmark for LLM-Based…        1    3% ▇
prior: pull toward the middle                        1% ▇
weighted average                                   = 60.0
Licences: Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · WebApp1K models leaderboard ([MIT](../../licenses/webapp1k-leaderboard-MIT.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/mistral-medium-2505) · [JSON](https://whatmodel.sh/api?format=json&all=1)