[whatmodel.sh](../../README.md) / GPT-5 mini
# GPT-5 mini
OpenAI · released 2025-08 · closed weights
**score 66.7** ±0.5 · #116 of 732 · from 53 benchmark results across 22 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks GPT-5 mini among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█▄●▅▄▄▂▁ #129 of 313 35% 17
coding ▁▁▁▃▅▇█▇●▄▃▁ #119 of 469 19% 5
chat & writing ▁▁▁▃▆▄▆█▆●▇▃ #127 of 402* 3.1% 1
math ▁▁▃▅█▆▄█●▂▁▁ #63 of 431 21% 7
vision & documents ▁▁▂▂▁▃▄▂●▇█▄ #91 of 153 7.8% 7
web search ▁▁●▅▄█▃▄·▁·▁ #77 of 84* 1.8% 1
agent tasks · #129 of 313 · 35% of its score
Beats [o3](o3.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Large](jamba-1-5-large.md), [Devstral 2 (2512)](devstral-2.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 64.7% ▁··▁·▁●▁▄█▄▁ beats 6/29 hard · avg 19% 8.8% sharp · at its level · several runs
τ²-bench aggregate 69.8% ▁▁▁·▁▁▃●▃▄█▁ beats 15/39 medium · avg 41% 2.6% several runs
Terminal-Bench 2.0 with Terminus … 24.0% ▃··▃·▃▃●█▅▃▅ beats 5/14 very hard · avg 6.1% 2.4% sharp · at its level
BrowserGym WorkArena-L2 (AgentLab… 47.7% ▃▅▃·▅▃▃··●·▃ beats 10/11 very hard · avg 3.6% 2.4% sharp · at its level · noisy
τ²-bench Telecom (maker) 74.1% ▃·▅▅▃●··█▃▃▅ beats 6/13 hard · avg 16% 2.1% sharp · at its level · noisy · several …
Terminal-Bench 2.0 (maker) 38.2% ●█···██··███ beats 0/6 hard · avg 15% 1.9% at its level
Humanity's Last Exam with tools (… 31.6% ●·▃·▅····▅·█ beats 0/8 very hard · avg 12% 1.8% at its level
Toolathlon (Tool Decathlon) (make… 26.9% ●··▃··▃··▃█▅ beats 0/8 very hard · avg 13% 1.8% at its level
BrowserGym WorkArena-L1 (AgentLab… 60.6% █···█▄▄·▄█·● beats 9/10 hard · avg 28% 1.8% at its level
τ²-bench Retail 73.4% █▄··▄··▄·●·▄ beats 5/6 medium · avg 36% 1.8% at its level
BFCL V4 (December 2025 archive) 55.5% ▁▁▁▂██▅▅●▂▂▂ beats 65/77 hard · avg 31% 1.5% at its level
τ²-bench Airline 59.3% ▄▄······▄█●▄ beats 5/6 hard · avg 28% 1.4% at its level
OSWorld-Verified (maker) 42.0% █●·····▄·▄▄▄ beats 2/6 hard · avg 16% 1.4% at its level
BrowserGym MiniWoB (AgentLab Gene… 71.0% ▅····▅▅▃▃·▃● beats 10/11 medium · avg 53% 1.0% blunt · at its level
Vibe Code Bench (test split, 50 w… 14.2% ▃··▃·▃●█▃▃▃█ beats 3/14 very hard · avg 2.8% 1.0% sharp · noisy
SWE-bench Pro (maker) 45.7% ●·····▄·▄▄▄█ beats 0/6 hard · avg 29% 0.8% blunt · at its level
τ²-Bench-Verified (GPT-5.1 user) 69.8% ▂··▄···●▄▄█▄ beats 3/13 medium · avg 45% 0.5% at its level
coding · #119 of 469 · 19% of its score
Beats [Grok 4.3](grok-4-3.md), [Qwen3.5 Plus](qwen3-5-plus.md), [DeepSeek V3.2](deepseek-v3-2.md) here, though they rank higher overall. Trails [Llama 3.1 Nemotron Ultra 253B v1](llama-3-1-nemotron-ultra-253b.md), [Solar Pro 4](solar-pro-4.md), [Laguna XS 2.1 (33B-A3B)](laguna-xs-2-1.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 64.7% ▁··▁·▁●▁▄█▄▁ beats 6/29 hard · avg 19% 8.8% sharp · at its level · several runs
Aider polyglot (225 exercises, tw… 71.6% █▄····▄··●·▄ beats 5/6 very hard · avg 7.3% 7.1% sharp · at its level
VisEditBench strict pass rate (1,… 60.0% ▄··█·▄··●█·▄ beats 4/7 hard · avg 33% 1.6% at its level
Vibe Code Bench (test split, 50 w… 14.2% ▃··▃·▃●█▃▃▃█ beats 3/14 very hard · avg 2.8% 1.0% sharp · noisy
SWE-bench Pro (maker) 45.7% ●·····▄·▄▄▄█ beats 0/6 hard · avg 29% 0.8% blunt · at its level
chat & writing · #127 of 402 · 3.1% of its score
Beats [LongCat 2.0 (1.6T-A48B)](longcat-2-0.md) here, though they rank higher overall. Trails [LFM2.5 2.6B](lfm2-5-2-6b.md), [Nemotron-Cascade-8B](nemotron-cascade-8b.md), [DeepSeek V3 0324](deepseek-v3-0324.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LMArena (text) 1389.8 ▁▁▁▂▃▃▃▆▆●█▃ beats 232/385 avg 1260 3.1% blunt
math · #63 of 431 · 21% of its score
Beats [Qwen3.6 Max Preview](qwen3-6-max-preview.md), [Claude Sonnet 4.6](claude-sonnet-4-6.md), [Claude Opus 4.5](claude-opus-4-5.md) here, though they rank higher overall. Trails [INTELLECT-3](intellect-3.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
FrontierMath tiers 1–3 v2 46.7% ▂··▁▁▃▄●█▆▄▃ beats 34/86 very hard · avg 1.2% 12% sharp · at its level
FrontierMath tier 4 v2 12.2% ▁▂▁●█▅▂▃▂▁▁▁ beats 14/67 very hard · avg 0.1% 3.3% sharp · too hard for it
MATH level 5 97.8% ▂▂▅▆▄█▆▃▅▂▃● beats 91/92 hard · avg 28% 1.8% sharp · maxed out
OTIS mock AIME 2024–2025 86.7% ▃▃▄▃▃▃▇▇●▃▂▅ beats 133/197 very hard · avg 7.2% 1.7% sharp · noisy
MathNet-Solve-Test micro-average … 57.0% ▃▃···█▃·●·▃▃ beats 6/8 very hard · avg 15% 1.0% at its level
MATH-Vision 71.9% ▁▁·▁··▄▁▃●▇█ beats 10/24 medium · avg 48% 0.9% typical
FormalProofBench (200 graduate-le… 7.0% ▃▅·●··▅▃█··▃ beats 3/10 very hard · avg 3.1% 0.4% noisy
vision & documents · #91 of 153 · 7.8% of its score
Beats [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md) here, though they rank higher overall. Trails [Qwen3.5 4B](qwen3-5-4b.md), [GPT-4.1 mini](gpt-4-1-mini.md), [Mistral Large 3 (675B-A41B)](mistral-large-3.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
MMMU Pro (maker) 67.5% ●·····▃·▃·█▃ beats 1/7 hard · avg 40% 2.5% at its level
Arena vision (style controlled) 1184.3 ▁▁▂▂▂▂▃▄●▅█▇ beats 63/152 avg 1061 2.0% blunt
VisEditBench strict pass rate (1,… 60.0% ▄··█·▄··●█·▄ beats 4/7 hard · avg 33% 1.6% at its level
MATH-Vision 71.9% ▁▁·▁··▄▁▃●▇█ beats 10/24 medium · avg 48% 0.9% typical
Molmo2-VideoCount accuracy (val) 29.8% ▃··●·▃····█▃ beats 1/6 hard · avg 22% 0.3% blunt · at its level
Ai2 Molmo2 human preference Elo (… 1076.0 ▄··▄··▄··●█▄ beats 3/6 avg 968 0.2% blunt
Molmo2-CapTest dense video captio… 56.6% █···█·████·● beats 6/6 hard · avg 31% 0.1% blunt · at its level
web search · #77 of 84 · 1.8% of its score
Trails [GPT-4.1 nano](gpt-4-1-nano-2025-04-14.md), [GPT-4o mini (2024-07-18)](gpt-4o-mini-2024-07-18.md), [Mistral Small 3.1 24B (2503)](mistral-small-3-1.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Humanity's Last Exam with tools (… 31.6% ●·▃·▅····▅·█ beats 0/8 very hard · avg 12% 1.8% at its level
other tests (18)
test result vs the others in its run difficulty counts why
KMMLU-HARD (Daekeun Kim, zero-sho… 60.6% █···▁▁▁▂●·▆▂ beats 15/23 very hard · avg 29% 4.3% sharp · at its level
GPQA Diamond 75.0% ▁▁▁▁▂▄▆▄●▇█▅ beats 112/211 hard · avg 39% 4.2% sharp · at its level
CLIcK Korean (Daekeun Kim) 82.2% ▆▄▄▄█▄●·▄█·▄ beats 15/23 medium · avg 56% 2.8% typical
Humanity's Last Exam (maker) 18.3% ●··▃·▃▃···▃█ beats 0/7 very hard · avg 4.4% 2.5% typical
KCL-Essay vanilla 40.3% ▁···▁▃▂█●▃▃▂ beats 15/23 hard · avg 17% 2.4% at its level
HAE-RAE 1.0 (Daekeun Kim) 84.4% ▄▄▄▂▂█●▄▂█▂▄ beats 12/22 medium · avg 61% 2.2% typical
Mystery Game Puzzles (Epoch) 5.0% ●··▂▃▇▄█▂▄▂▁ beats 12/77 very hard · avg 9.4% 1.5% sharp · too hard for it · noisy
Chess Puzzles (Epoch) 30.0% █▁▁▁▁▂▃▃●▂▁▁ beats 104/137 very hard · avg 5.8% 1.4% sharp · noisy
KCL-MCQA vanilla 40.9% ▂···█▂▆●██▆▂ beats 11/23 very hard · avg 29% 1.3% at its level
SimpleQA Verified 21.6% ▁▁▃▂●▃▆▆█▄▂▅ beats 14/81 very hard · avg 8.0% 1.0% at its level
SimpleQA (maker) 22.0% ▄▄▄▆█▆█●█·▂█ beats 20/32 very hard · avg 9.2% 0.7% at its level · noisy
Multi-IF (aggregation unspecified) 85.4% ▄·····▄█▄●·▄ beats 5/6 easy · avg 66% 0.6% typical
SOOHAK Challenge (340 problems, A… 18.8% █·█████··●·█ beats 6/7 very hard · avg 4.3% 0.5% typical
SOOHAK-Mini (702 problems, Avg@3) 67.1% ▄··▄·▄·▄▄●·▄ beats 6/7 medium · avg 36% 0.4% blunt · at its level
HealthBench Hard (maker) 40.3% ▁····▁▁▂█·●▁ beats 12/13 hard · avg 16% 0.4% blunt · at its level · small run
Open PT-LLM Leaderboard average (… 83.9% ▃▁▁▁▁▁▁▁▂█▅● beats 273/276 medium · avg 72% 0.4% noisy
Vectara hallucination leaderboard… 96.8% ▁▁▂▃▄▄█●▃▂▂▁ beats 99/140 very easy · avg 94% 0.1% blunt · maxed out · noisy
Vectara hallucination leaderboard… 87.1% ▁▂▁▂●█▂▃▄▃▁▁ beats 17/98 very easy · avg 90% 0.0% blunt
source results share of the score
Epoch AI 9 34% ▇▇▇▇
OpenAI vendor-reported benchmark scores… 8 14% ▇▇
ServiceNow-AI/Apriel-1.6-15b-Thinker mo… 5 13% ▇▇
Daekeun Kim Korean LLM benchmark runs 3 9% ▇
ServiceNow BrowserGym leaderboard 3 5% ▇
LMArena leaderboard dataset 2 5% ▇
Korean Canonical Legal Benchmark: Towar… 2 4% ▇
Terminal-Bench: Benchmarking Agents on … 1 2% ▇
Qwen/Qwen3.5-27B model card 2 2% ▇
VisEditBench: Can Vision-Language Model… 1 2% ▇
Qwen/Qwen3.5-122B-A10B model card 1 1% ▇
Berkeley Function Calling Leaderboard r… 1 1% ▇
OpenAI (submitted / maker-reported) 2 1% ▇
Vibe Code Bench 1 1% ▇
MathNet: a Global Multimodal Benchmark … 1 1% ▇
Soohak: A Mathematician-Curated Benchma… 2 1% ▇
SEATauBench: Progressively Adapting Too… 1 1% ▇
Molmo2: Open Weights and Data for Visio… 3 1% ▇
Amazon AGI 1 1% ▇
FormalProofBench 1 0% ▇
Open Portuguese LLM Leaderboard 1 0% ▇
Vectara hallucination leaderboard 2 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 66.7
Licences: Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · ServiceNow-AI/Apriel-1.6-15b-Thinker model card ([MIT](../../licenses/cards-final-NOTICE.txt)) · Daekeun Kim Korean LLM benchmark runs ([MIT](../../licenses/korean-personal-benchmark-runs-MIT.txt)) · ServiceNow BrowserGym leaderboard ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities ([CC BY 4.0](../../licenses/cards-final-NOTICE.txt)) · Terminal-Bench: Benchmarking Agents on Hard ([CC BY 4.0](../../licenses/paper-2601.11868v1-CC-BY-4.0.txt)) · Qwen/Qwen3.5-27B model card ([apache-2.0](../../licenses/Qwen--Qwen3.5-27B.txt)) · VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback? ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Qwen/Qwen3.5-122B-A10B model card ([apache-2.0](../../licenses/Qwen--Qwen3.5-122B-A10B.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · Vibe Code Bench ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval ([CC BY 4.0](../../licenses/paper-2604.18584v2-CC-BY-4.0.txt)) · Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs ([CC BY-SA 4.0](../../licenses/paper-2605.09063v3-CC-BY-SA-4.0.txt)) · SEATauBench: Progressively Adapting Tool-Agent-User Evaluation Into Low-Resource Southeast Asian Languages ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding ([CC BY 4.0](http://creativecommons.org/licenses/by/4.0/)) · Amazon AGI ([MIT](../../licenses/tau2-verified-MIT.txt)) · FormalProofBench ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
### Through an API
api (OpenAI) · $0.69/M blended · $0.25 in / $2 out per 1M tokens · [platform.openai.com](https://platform.openai.com/docs/models)
[price source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/providers/openai/models/gpt-5-mini.toml): Standard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/models/openai/gpt-5-mini.toml)
[interactive version](https://whatmodel.sh/model/gpt-5-mini) · [JSON](https://whatmodel.sh/api?format=json&all=1)