[whatmodel.sh](../../README.md) / Ministral 3B (2410) # Ministral 3B (2410) Mistral · released 2024-10 · proprietary **score 43.0** ±0.8 · comparison only: no current run option verified · from 24 benchmark results across 4 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks Ministral 3B (2410) among every model on that task or test, worst to best.}

task            vs the others          rank  counts  tests
coding          ▄··········●       #2 of 3*    4.4%      2
chat & writing  ▁▁▁▃▅▄●█▇▇▇▃   #294 of 474*    1.8%      1
math            ▁▁▃●██▅█▃▁▁▁    #440 of 530     36%      5

coding · #2 of 3 · 4.4% of its score

test               result  vs the others in its run    difficulty      counts  why
HumanEval (maker)   77.4%  ▁▁▁▁▂▂▆▆█●▂▁ beats 168/205  easy · avg 80%    2.4%  sharp · at its level · noisy · small run
MBPP (maker)        67.7%  ▁▁▁▄▅█▆●▄▁▂▁ beats 65/86    easy · avg 74%    2.1%  at its level · small run

chat & writing · #294 of 474 · 1.8% of its score

Beats [Mistral Small 4 (119B-A6.5B)](mistral-small-4.md), [Qwen2.5 Coder 32B Instruct](qwen2-5-coder-32b.md), [Ministral 3 14B Reasoning 2512](ministral-3-14b-reasoning.md) here, though they rank higher overall. Trails [Flan-T5 XXL](flan-t5-xxl.md), [Granite 4.0 H 1B](granite-4-0-h-1b.md) here, though they rank lower overall.

test                                result  vs the others in its run  difficulty      counts  why
Arena-Hard (version/judge unspeci…   64.3%  ▃▃▂▅▃▆●█▅▅▂▃ beats 14/30  easy · avg 70%    1.8%  at its level · small run

math · #440 of 530 · 36% of its score

Beats [GPT-3.5 Turbo (0125)](gpt-3-5-turbo-0125.md), [DBRX Instruct](dbrx-instruct.md), [Command R Plus (04-2024)](command-r-plus-04-2024.md) here, though they rank higher overall. Trails [OPT-2.7B](opt-2-7b.md), [Gemma 2B (base)](gemma-2b-base.md), [Stable Code 3B](stable-code-3b.md) here, though they rank lower overall.

test                               result  vs the others in its run  difficulty           counts  why
MGSM                                44.6%  ●▃▃█·▅···▃·▃ beats 0/10   easy · avg 72%          13%  sharp · at its level
GSM8K                               80.1%  █·●▄▄█···█·▄ beats 2/10   very easy · avg 90%     13%  sharp
MATH                                41.8%  ●▄██···▄▄▄·▄ beats 1/10   medium · avg 61%       4.3%  at its level
MATH level 5                        14.4%  ▂▂●▆▄█▆▃▅▂▃▄ beats 11/92  hard · avg 28%         4.1%  sharp
MATH (subset unspecified) (maker)   51.7%  ▂▄▄▃●▇▅▆▄▃▄▁ beats 28/80  medium · avg 63%       1.6%  at its level · noisy

other tests (16)

test                                result  vs the others in its run   difficulty           counts  why
ARC Challenge                        80.3%  ▂▂●█▂···▄··▂ beats 2/10    very easy · avg 89%     12%  sharp · at its level
MMLU                                 60.8%  ●▄█▄▄▄·▄▄··▄ beats 0/10    easy · avg 75%         9.4%  at its level
BIG-Bench Hard                       51.2%  ●█▄·█▄·▄▄··▄ beats 0/10    medium · avg 71%       7.3%  sharp · at its level
OpenBookQA                           79.8%  ▂··▄●▂·▂···▄ beats 3/10    easy · avg 85%         6.3%  at its level
MMLU-Pro                             35.3%  ●▄▄·█▄▄▄·▄·▄ beats 0/10    medium · avg 54%       5.1%  at its level
Multilingual MMLU (Microsoft)        46.4%  ●▄▄██··█···▄ beats 0/10    medium · avg 63%       4.7%  at its level
GPQA Diamond                         25.3%  ●▁▁▁▂▄▆▄▅▇█▅ beats 0/211   hard · avg 39%         2.8%  sharp · too hard for it
HellaSwag                            74.6%  ▃▅●·▃▃▃···▃▃ beats 4/10    easy · avg 82%         2.4%  at its level
WinoGrande                           59.8%  ▃····●▃█▃·▅▅ beats 1/10    medium · avg 71%       1.9%  at its level
Social IQA                           73.9%  ▃·▃▅●█·▃···▃ beats 4/10    medium · avg 76%       1.8%  blunt · at its level
BoolQ                                79.4%  ▃··▃··●█▃·▃▃ beats 4/10    easy · avg 83%         1.6%  at its level
GPQA (subset unspecified)            24.4%  ●··▄▂··▂▂··▄ beats 0/10    very hard · avg 32%    1.0%  sharp · too hard for it · noisy
PIQA                                 73.2%  ▃·●·▃█·▃▅··▃ beats 2/10    medium · avg 82%       0.6%  blunt · at its level
Vectara hallucination leaderboard…   91.7%  ▁▁▂▃●▄█▆▃▂▂▁ beats 35/140  very easy · avg 94%    0.5%  blunt · noisy
TruthfulQA MC2                       62.9%  ▂·●▂▂▄····▂▂ beats 1/10    medium · avg 64%       0.3%  blunt · at its level
Vectara hallucination leaderboard…   92.7%  ▁▂▁▂▅█▂●▄▃▁▁ beats 68/98   very easy · avg 90%    0.0%  blunt

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
microsoft/Phi-4-mini-instruct model card       16   84% ▇▇▇▇▇▇▇▇▇▇
Mistral AI vendor-reported benchmark sc…        4    8% ▇
Epoch AI                                        2    7% ▇
Vectara hallucination leaderboard               2    1% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 43.0

_{Licences: microsoft/Phi-4-mini-instruct model card ([MIT](../../licenses/card-microsoft-phi-4-mini-instruct.txt)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))} ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/ministral-3b-2410) · [JSON](https://whatmodel.sh/api?format=json&all=1)