[whatmodel.sh](../../README.md) / GPT-4.1 mini
# GPT-4.1 mini
OpenAI · released 2025-04 · closed weights
**score 60.2** ±0.6 · #197 of 732 · from 43 benchmark results across 19 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks GPT-4.1 mini among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█●▅▅▄▄▂▁ #175 of 313 16% 10
coding ▁▁▁▃▅▇█●▅▄▃▁ #183 of 469* 10% 2
chat & writing ▁▁▁▃▆▄▆█●▇▇▃ #134 of 402 8.6% 3
math ▁▁▃▅█▆●█▄▂▁▁ #179 of 431 18% 5
vision & documents ▁▁▂▂▁▃▄▂●▇█▄ #83 of 153 7.2% 3
web search ▁▁▁▅▄█▃▄·●·▁ #5 of 84* 2.0% 1
agent tasks · #175 of 313 · 16% of its score
Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md), [DeepSeek R1 0528](deepseek-r1-0528.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Jamba 1.5 Large](jamba-1-5-large.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
τ²-bench Telecom (maker) 44.0% ▃·▅●▃▃··█▃▃▅ beats 4/13 very hard · avg 13% 3.8% sharp · at its level · several runs
BrowserGym WebArena (AgentLab Gen… 30.7% █··▄·▄●·▄··▄ beats 4/7 hard · avg 17% 2.2% at its level
τ²-bench Retail 61.4% ▁····▂▁▃●█▃▁ beats 12/44 medium · avg 40% 2.2% at its level
Finance Agent Benchmark v1 (537 q… 20.3% █·▃▃·█▅●▅▅▃▅ beats 11/18 very hard · avg 5.5% 2.0% sharp
BFCL V4 (December 2025 archive) 50.4% ▁▁▁▂██▅▅●▂▂▂ beats 57/77 hard · avg 31% 1.7% at its level
τ²-bench Airline 48.7% ▁▁·▂▁▃●▁█▅▂▃ beats 12/42 hard · avg 28% 1.4% at its level
BFCL v3 65.2% ▁▁▁▁▁▃▅█▅●▁▁ beats 98/114 medium · avg 51% 0.7% blunt · at its level
Galileo Agent Leaderboard v2 (act… 56.0% ▅▃·▃▃▅▅█▃█●▅ beats 17/19 hard · avg 26% 0.7% at its level
Galileo Agent Leaderboard v1 (too… 91.0% ▃▃▃▃▆▅▆▅●▂▅▂ beats 25/31 easy · avg 78% 0.5% typical
KlinikeBench strict pass@1 (333 i… 2.1% ▁▁·●▁▁▁▅▃█▅█ beats 2/27 very hard · avg 2.4% 0.4% too hard for it
coding · #183 of 469 · 10% of its score
Beats [Qwen3.5 9B](qwen3-5-9b.md), [Gemini 2.5 Flash](gemini-2-5-flash.md), [Qwen3-VL 32B Thinking](qwen3-vl-32b-thinking.md) here, though they rank higher overall. Trails [QwQ-32B-Preview](qwq-32b-preview.md), [Ministral 3 8B Reasoning 2512](ministral-3-8b-reasoning.md), [Phi-4-reasoning 14B](phi-4-reasoning.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Aider polyglot (225 exercises, tw… 32.4% ▂·▂▃▅●▃█▆▄▂▂ beats 16/49 very hard · avg 7.6% 9.0% sharp · at its level
HumanEval (simple-evals) (maker) 93.8% ▆█▆●▂··▂···▂ beats 12/15 easy · avg 83% 0.9% typical
chat & writing · #134 of 402 · 8.6% of its score
Beats [LongCat 2.0 (1.6T-A48B)](longcat-2-0.md), [o3-mini](o3-mini.md), [K-EXAONE (236B-A23B)](k-exaone-236b-a23b.md) here, though they rank higher overall. Trails [LFM2.5 8B-A1B](lfm2-5-8b-a1b.md), [LFM2.5 2.6B](lfm2-5-2-6b.md), [Nemotron-Cascade-8B](nemotron-cascade-8b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena-Hard v2.0 hard prompts (sty… 46.9% ▂·▃▅▅·▃●▆▂▂▂ beats 13/22 very hard · avg 12% 4.2% sharp · at its level
LMArena (text) 1382.8 ▁▁▁▂▃▃▃▆●▇█▃ beats 223/385 avg 1260 3.6% blunt
Arena-Hard v2.0 creative writing … 28.2% ▆▂▂●▂▂▃█▂▃·▂ beats 9/22 hard · avg 19% 0.9% sharp · at its level · noisy
math · #179 of 431 · 18% of its score
Beats [MiniMax-M2.1](minimax-m2-1.md), [Mistral Medium 3.5 (128B)](mistral-medium-3-5.md), [GPT-4.1](gpt-4-1.md) here, though they rank higher overall. Trails [OpenThinker3 7B](openthinker3-7b.md), [Olmo 3 7B Think](olmo-3-7b-think.md), [OpenThinker2 32B](openthinker2-32b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
FrontierMath tiers 1–3 v2 6.7% ▂··●▁▃▄▄█▆▄▃ beats 5/86 very hard · avg 1.2% 5.8% sharp
MATH level 5 87.3% ▂▂▅▆▄█▆▃●▂▃▄ beats 74/92 hard · avg 28% 5.6% sharp
OTIS mock AIME 2024–2025 44.7% ▃▃▄▃▃●▇▇█▃▂▅ beats 74/197 very hard · avg 7.2% 2.9% sharp · at its level · noisy
MATH-500 (simple-evals) (maker) 81.4% ▃··●▅▃···▃·█ beats 1/9 medium · avg 61% 2.5% sharp
MGSM (simple-evals) (maker) 88.2% ▁····▁▅●█▁▃▁ beats 6/16 easy · avg 80% 1.1% typical
vision & documents · #83 of 153 · 7.2% of its score
Beats [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md), [GPT-5.4 nano](gpt-5-4-nano.md), [GPT-5 mini](gpt-5-mini.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
CharXiv descriptive questions (ma… 88.4% ▄▄·····▄·▄●▄ beats 4/6 easy · avg 71% 2.5% typical
Arena vision (style controlled) 1200.7 ▁▁▂▂▂▂▃▄●▅█▇ beats 74/152 avg 1061 2.4% blunt
CharXiv (reasoning questions) (ma… 56.8% ▄··▂●··▂···▄ beats 6/9 hard · avg 32% 2.3% at its level
web search · #5 of 84 · 2.0% of its score
Beats [GPT-6 Astra](gpt-6-astra.md), [Claude Opus 5.5](claude-opus-5-5.md), [GPT-6.1 Sol](gpt-6-1-sol.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
Finance Agent Benchmark v1 (537 q… 20.3% █·▃▃·█▅●▅▅▃▅ beats 11/18 very hard · avg 5.5% 2.0% sharp
other tests (19)
test result vs the others in its run difficulty counts why
GPQA Diamond (simple-evals) (make… 65.0% █▃▃█·▃●▅▃▃▃▅ beats 9/16 hard · avg 39% 6.5% sharp · at its level
GPQA Diamond 65.8% ▁▁▁▁▂▄▆▄●▇█▅ beats 98/211 hard · avg 39% 6.1% sharp · at its level
CLIcK Korean (Daekeun Kim) 72.1% ▆▄▄▄●▄▂·▄█·▄ beats 10/23 medium · avg 56% 4.5% typical
HAE-RAE 1.0 (Daekeun Kim) 78.9% ▄▄▄▂▂●▂▄▂█▂▄ beats 11/22 medium · avg 61% 3.6% typical
KMMLU-HARD (Daekeun Kim, zero-sho… 35.6% █···▁●▁▂▃·▆▂ beats 9/23 very hard · avg 29% 3.4% sharp
KCL-Essay vanilla 30.9% ▁···▁▃▂●▂▃▃▂ beats 8/23 hard · avg 17% 2.5% at its level
MMLU (simple-evals, zero-shot CoT… 87.5% ▂▂·█▄●▂·█▂·▂ beats 9/16 easy · avg 79% 2.2% typical
EQ-Bench 3 (emotional intelligenc… 1171.0 ▁·▂▁▄▅●▃█▄▁▁ beats 26/62 avg 1034 2.2% typical
ARC-AGI-1 public evaluation set (… 7.2% ▁·●▃▁·█▇▁·▇▃ beats 2/27 very hard · avg 4.0% 2.1% sharp · at its level · noisy
HindiGen-v1 3C3H score (Inception… 65.0% ▁▁▁▁·▂·█●█▇▁ beats 16/31 medium · avg 46% 1.7% at its level
Open Persian LLM Leaderboard v2 a… 70.1% ▂▄▂▃█▄▃▆▃●▂▂ beats 74/81 medium · avg 57% 1.4% blunt · at its level
KCL-MCQA vanilla 29.4% ▂···●▂▆▆██▆▂ beats 4/23 very hard · avg 29% 1.2% typical
SimpleQA Verified 12.7% ▁▁●▂▃▃▆▆█▄▂▅ beats 7/81 very hard · avg 8.0% 0.9% typical
Chess Puzzles (Epoch) 7.0% █▁●▁▁▂▃▃▃▂▁▁ beats 51/137 very hard · avg 5.8% 0.6% sharp · too hard for it · noisy
ARC-AGI-2 public evaluation set (… 0.0% ●··▂▅▁▃▁▃·▁▃ beats 1/24 very hard · avg 0.3% 0.5% sharp · too hard for it · noisy
Mystery Game Puzzles (Epoch) 7.0% ●··▂▃▇▄█▂▄▂▁ beats 6/77 very hard · avg 9.4% 0.5% sharp · too hard for it · noisy
SimpleQA (simple-evals) (maker) 16.8% ▄·▁●▁▁··█▁·▁ beats 4/14 hard · avg 17% 0.3% at its level · noisy
DROP F1, 3-shot (simple-evals) (m… 81.0% ▂▂▄●▆█·▂···▄ beats 6/16 easy · avg 78% 0.3% blunt
Vectara hallucination leaderboard… 97.8% ▁▁▂▃▄▄█▆●▂▂▁ beats 120/140 very easy · avg 94% 0.1% blunt · maxed out · noisy
source results share of the score
Epoch AI 7 22% ▇▇▇
OpenAI simple-evals README benchmark re… 7 14% ▇▇
Daekeun Kim Korean LLM benchmark runs 3 11% ▇
Aider polyglot benchmark runs 1 9% ▇
OpenAI vendor-reported benchmark scores… 3 6% ▇
LMArena leaderboard dataset 2 6% ▇
τ²/τ³-bench leaderboard: Sierra's own e… 3 6% ▇
Arena-Hard-Auto v2.0-Preview leaderboard 2 5% ▇
Korean Canonical Legal Benchmark: Towar… 2 4% ▇
ARC Prize Foundation model_baseline out… 2 3% ▇
Berkeley Function Calling Leaderboard r… 2 2% ▇
ServiceNow BrowserGym leaderboard 1 2% ▇
EQ-Bench 3 canonical leaderboard results 1 2% ▇
Finance Agent Benchmark 1 2% ▇
Inception Arabic/Hindi leaderboards 1 2% ▇
Open Persian LLM Leaderboard v2 1 1% ▇
Galileo Agent Leaderboard 2 1% ▇
KlinikeBench: Evaluating Language Model… 1 0% ▇
Vectara hallucination leaderboard 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 60.2
Licences: Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · OpenAI simple-evals README benchmark results ([MIT](../../licenses/openai-simple-evals-MIT.txt)) · Daekeun Kim Korean LLM benchmark runs ([MIT](../../licenses/korean-personal-benchmark-runs-MIT.txt)) · Aider polyglot benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · τ²/τ³-bench leaderboard: Sierra's own evaluation runs ([MIT](https://opensource.org/license/mit)) · Arena-Hard-Auto v2.0-Preview leaderboard ([Apache-2.0](../../licenses/arena-hard-auto-Apache-2.0.txt)) · Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities ([CC BY 4.0](../../licenses/cards-final-NOTICE.txt)) · ARC Prize Foundation model\_baseline outputs on ARC-AGI-1/2 public evaluation sets ([MIT](../../licenses/arcprize-model-baseline-outputs-MIT.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · ServiceNow BrowserGym leaderboard ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · EQ-Bench 3 canonical leaderboard results ([MIT](../../licenses/eqbench3-MIT.txt)) · Finance Agent Benchmark ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Galileo Agent Leaderboard ([Apache-2.0](../../licenses/galileo-agent-leaderboard-Apache-2.0.txt)) · KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
### Through an API
api (OpenAI) · $0.70/M blended · $0.4 in / $1.6 out per 1M tokens · [platform.openai.com](https://platform.openai.com/docs/models)
[price source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/providers/openai/models/gpt-4.1-mini.toml): Standard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/models/openai/gpt-4.1-mini.toml)
[interactive version](https://whatmodel.sh/model/gpt-4-1-mini) · [JSON](https://whatmodel.sh/api?format=json&all=1)