[whatmodel.sh](../../README.md) / Mistral Medium (2312) # Mistral Medium (2312) Mistral · released 2023-12 · closed weights **score 47.1** ±1.1 · comparison only: no current run option verified · from 6 benchmark results across 5 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks Mistral Medium (2312) among every model on that task or test, worst to best.}

task            vs the others          rank  counts  tests
agent tasks     ▁▁▂●█▄▅▄▄▃▁▁   #297 of 354*    2.8%      1
chat & writing  ▁▁▁▃▅●▇█▇▇▇▃    #353 of 474     97%      5

agent tasks · #297 of 354 · 2.8% of its score

Beats [DeepSeek R1 0528 Qwen3 8B](deepseek-r1-0528-qwen3-8b.md), [Magistral Small 1.0 (2506)](magistral-small-2506.md), [Llama 4 Scout (109B-A17B)](llama-4-scout.md) here, though they rank higher overall. Trails [Granite 4.0 H 1B](granite-4-0-h-1b.md), [Granite 4.0 1B](granite-4-0-1b.md), [SmolLM3 3B](smollm3-3b.md) here, though they rank lower overall.

test     result  vs the others in its run   difficulty        counts  why
BFCL v3   51.4%  ▁▁▁▁▁▃●█▅▄▁▁ beats 39/114  medium · avg 51%    2.8%  blunt · at its level

chat & writing · #353 of 474 · 97% of its score

Beats [Ministral 3 14B Reasoning 2512](ministral-3-14b-reasoning.md), [Gemma 4 E4B](gemma-4-e4b.md), [OpenThinker3 7B](openthinker3-7b.md) here, though they rank higher overall. Trails [EXAONE 3.5 2.4B Instruct](exaone-3-5-2-4b.md), [LFM2 700M](lfm2-700m.md), [Flan-T5 XXL](flan-t5-xxl.md) here, though they rank lower overall.

test                                result  vs the others in its run   difficulty           counts  why
AlpacaEval 2 length-controlled       28.6%  ▁▂▂▃▆▆●▆▄▄▁▁ beats 76/122  medium · avg 35%        44%  sharp · at its level
Arena-Hard (v0.1, GPT-4-1106 judg…   31.9%  ▂▁▁▅█●▄▃▃▅▃▁ beats 27/60   medium · avg 47%        28%  sharp · at its level
LMArena (text)                      1222.8  ▁▁▁▂▃●▃▆▆▇█▃ beats 77/385  avg 1260                12%  blunt
Polyrating rating (bias-aware re-…  1153.0  ▂▁▂▁▃▇█▆●▄▂▂ beats 68/95   avg 1187                11%  typical
AlpacaEval 1.0 win rate (vs text-…   96.8%  ▂▂·▆·▇▃▆█▆▄● beats 44/45   very easy · avg 94%    1.2%  sharp · noisy

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
AlpacaEval 1.0 and 2.0 leaderboards             2   45% ▇▇▇▇▇
Arena-Hard-Auto v0.1 leaderboard                1   28% ▇▇▇
LMArena leaderboard dataset                     1   12% ▇
Polyrating: A Cost-Effective and Bias-A…        1   11% ▇
Berkeley Function Calling Leaderboard r…        1    3% ▇
prior: pull toward the middle                        1% ▇
weighted average                                   = 47.1

_{Licences: AlpacaEval 1.0 and 2.0 leaderboards ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Arena-Hard-Auto v0.1 leaderboard ([Apache-2.0](../../licenses/arena-hard-auto-v0-1-Apache-2.0.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt))} ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/mistral-medium-2312) · [JSON](https://whatmodel.sh/api?format=json&all=1)