[whatmodel.sh](../../README.md) / Mistral Large (2402) # Mistral Large (2402) Mistral · released 2024-02 · proprietary **score 47.8** ±0.6 · comparison only: no current run option verified · from 14 benchmark results across 9 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks Mistral Large (2402) among every model on that task or test, worst to best. 
task            vs the others          rank  counts  tests
coding          ▁▁▁▃▄●█▇▅▄▃▁    #379 of 539     43%      3
chat & writing  ▁▁▁▃▅▄●█▇▇▇▃    #339 of 474     26%      4
math            ▁▁▃▆●█▅█▃▁▁▁   #403 of 530*    5.1%      2
coding · #379 of 539 · 43% of its score Beats [Solar Open 100B (102B-A12B)](solar-open-100b.md), [Qwen3 4B Instruct 2507](qwen3-4b-instruct-2507.md), [Qwen3-VL 8B Instruct](qwen3-vl-8b-instruct.md) here, though they rank higher overall. Trails [DeepSeek LLM 67B Chat](deepseek-llm-67b-chat.md), [WizardLM 70B V1.0](wizardlm-70b-v1-0.md), [Code Tulu 2 34B](code-tulu-2-34b.md) here, though they rank lower overall. 
test        result  vs the others in its run  difficulty      counts  why
HumanEval    69.5%  ▁·▁▁▃▄▆●█▃▁▁ beats 56/99  easy · avg 76%     21%  sharp · at its level
HumanEval+   62.2%  ▁·▁▁▁▃▄▄●█▃▁ beats 55/99  easy · avg 70%     18%  sharp · at its level
MBPP+        59.5%  ▁···▁▁▂▅●█▆▂ beats 36/75  easy · avg 66%    3.6%  at its level
chat & writing · #339 of 474 · 26% of its score Beats [Ministral 3 14B Reasoning 2512](ministral-3-14b-reasoning.md), [Ministral 3 8B](ministral-3-8b.md), [Gemma 4 E4B](gemma-4-e4b.md) here, though they rank higher overall. Trails [EXAONE 3.5 2.4B Instruct](exaone-3-5-2-4b.md), [LFM2 700M](lfm2-700m.md), [Flan-T5 XXL](flan-t5-xxl.md) here, though they rank lower overall. 
test                                result  vs the others in its run   difficulty        counts  why
AlpacaEval 2 length-controlled       32.6%  ▁▂▂▃▆▆█●▄▄▁▁ beats 85/122  medium · avg 35%     12%  sharp · at its level
Arena-Hard (v0.1, GPT-4-1106 judg…   37.7%  ▂▁▁▅█▅●▃▃▅▃▁ beats 32/60   medium · avg 47%    7.7%  sharp · at its level
LMArena (text)                      1242.6  ▁▁▁▂▃▃●▆▆▇█▃ beats 90/385  avg 1260            3.3%  blunt
Polyrating rating (bias-aware re-…  1163.0  ▂▁▂▁▃▇█▆●▄▂▂ beats 72/95   avg 1187            3.0%  typical
math · #403 of 530 · 5.1% of its score Beats [GPT-4 (0613)](gpt-4-0613.md), [Athene-70B](athene-70b.md), [Llama 3 70B Instruct](llama-3-70b.md) here, though they rank higher overall. Trails [Gemma 2B (base)](gemma-2b-base.md), [Stable Code 3B](stable-code-3b.md), [Zephyr 7B beta](zephyr-7b-beta.md) here, though they rank lower overall. 
test                      result  vs the others in its run   difficulty            counts  why
MATH level 5               24.5%  ▂▂▅●▄█▆▃▅▂▃▄ beats 22/92   hard · avg 28%          4.7%  sharp
OTIS mock AIME 2024–2025    1.9%  ▃●▄▃▃▃▇▇█▃▂▅ beats 14/197  very hard · avg 7.2%    0.5%  sharp · too hard for it · noisy
other tests (5) 
test                                          result  vs the others in its run   difficulty           counts  why
LiveBench 2024-07-26                           39.2%  ▁▁▁▂▂▃▂█●▃▅▃ beats 43/61   medium · avg 38%        13%  at its level
BigCodeBench (Instruct)                        29.6%  ▃▂·●▆██▆▅▃▃▃ beats 6/33    medium · avg 39%       3.7%  at its level
BigCodeBench (Complete)                        37.9%  ▂▁▁▄▂●▇█▆▂▂▂ beats 21/58   medium · avg 48%       3.7%  at its level
WildBench v2 WB-Score (task macro)  38.8937 WB-Score  ▁·▁▁·▃█▄●▆▅▆ beats 27/52   avg 43.019 WB-Score    3.5%  sharp
GPQA Diamond                                   38.8%  ▁▁▁▁▂●▆▄▅▇█▅ beats 36/211  hard · avg 39%         2.7%  sharp
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
EvalPlus / EvalPerf published evaluatio…        3   43% ▇▇▇▇▇
LiveBench model judgments                       1   13% ▇▇
AlpacaEval 1.0 and 2.0 leaderboards             1   12% ▇
Epoch AI                                        3    8% ▇
Arena-Hard-Auto v0.1 leaderboard                1    8% ▇
BigCodeBench paper                              2    7% ▇
WildBench v2 leaderboard data                   1    3% ▇
LMArena leaderboard dataset                     1    3% ▇
Polyrating: A Cost-Effective and Bias-A…        1    3% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 47.8
Licences: EvalPlus / EvalPerf published evaluation runs ([Apache-2.0](../../licenses/evalplus-published-runs-Apache-2.0.txt)) · LiveBench model judgments ([Apache-2.0 (stated in the LiveBench datasheet)](https://www.apache.org/licenses/LICENSE-2.0)) · AlpacaEval 1.0 and 2.0 leaderboards ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Arena-Hard-Auto v0.1 leaderboard ([Apache-2.0](../../licenses/arena-hard-auto-v0-1-Apache-2.0.txt)) · BigCodeBench paper ([CC BY 4.0](http://creativecommons.org/licenses/by/4.0/)) · WildBench v2 leaderboard data ([Apache-2.0](https://github.com/allenai/WildBench/blob/d6b8dcaf377d173d031980f97c16e1a82618c03d/LICENSE)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/mistral-large-2402) · [JSON](https://whatmodel.sh/api?format=json&all=1)