[whatmodel.sh](../../README.md) / o3-mini
# o3-mini
OpenAI · released 2025-01 · proprietary
**score 65.0** ±0.5 · #143 of 732 · from 50 benchmark results across 19 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks o3-mini among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃●█▄▅▅▄▄▂▁ #251 of 313 8.4% 7
coding ▁▁▁▃▅▇█▇●▄▃▁ #127 of 469 27% 10
chat & writing ▁▁▁▃▆▄▆█●▇▇▃ #146 of 402 8.5% 5
math ▁▁▃▅█▆▄●▄▂▁▁ #121 of 431 22% 9
web search ▁▁▁▅▄█▃●·▁·▁ #8 of 84* 2.2% 1
agent tasks · #251 of 313 · 8.4% of its score
Trails [Granite 4.0 1B](granite-4-0-1b.md), [SmolLM3 3B](smollm3-3b.md), [Qwen3-VL 2B Instruct](qwen3-vl-2b-instruct.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Finance Agent Benchmark v1 (537 q… 12.8% █·▃▃·●▅▅▅▅▃▅ beats 7/18 very hard · avg 5.5% 2.2% sharp · at its level
τ²-bench Retail 75.7% █▄··▄··▄·▄·● beats 6/6 medium · avg 36% 1.7% at its level
SWE-bench Verified 22.6% ●·█▄·······█ beats 0/6 very hard · avg 10% 1.5% sharp · at its level
τ²-bench Airline 61.3% ▄▄······▄█▄● beats 6/6 hard · avg 28% 1.4% at its level
Multi-SWE-bench, 7 languages (Mag… 3.6% ▄···●▄·▄▄·▄█ beats 1/7 very hard · avg 2.6% 0.7% blunt · too hard for it
BFCL v3 51.3% ▁▁▁▁▁▃●█▅▄▁▁ beats 38/114 medium · avg 51% 0.6% blunt · at its level
Galileo Agent Leaderboard v1 (too… 84.7% ▃▃▃▃▆●▆▅█▂▅▂ beats 14/31 easy · avg 78% 0.3% typical
coding · #127 of 469 · 27% of its score
Beats [Qwen3.5 Plus](qwen3-5-plus.md), [Qwen3-Coder-Next (80B-A3B)](qwen3-coder-next.md), [Gemini 2.5 Pro](gemini-2-5-pro.md) here, though they rank higher overall. Trails [Llama 3.1 Nemotron Ultra 253B v1](llama-3-1-nemotron-ultra-253b.md), [Solar Pro 4](solar-pro-4.md), [Nemotron 3 Super 120B-A12B](nemotron-3-super-120b-a12b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Aider polyglot (225 exercises, tw… 60.4% ▂·▂▃▅▂▃█●▄▂▂ beats 35/49 very hard · avg 7.3% 15% sharp · at its level · several runs
SciCode main problems (official l… 9.2% ▃···█··▂·▃·● beats 17/17 very hard · avg 1.0% 3.7% sharp · too hard for it
LiveCodeBench official (Aug 1, 20… 67.4% ▃▃▅▃··▃▅█●▅▃ beats 11/15 hard · avg 24% 1.9% sharp · at its level
WebApp1K-Duo-React pass@1 (onekq-… 69.6% ▁··▁·▁▁▁▆█●█ beats 30/45 hard · avg 29% 1.6% at its level
SWE-bench Verified 22.6% ●·█▄·······█ beats 0/6 very hard · avg 10% 1.5% sharp · at its level
HumanEval+ 94.0% ▁·▁▁▂▃▄▄█▅▄● beats 143/145 easy · avg 77% 1.2% sharp · maxed out
HumanEval (Vector Institute, Insp… 98.2% ▄·▄·█·█·█·▄● beats 9/9 easy · avg 77% 1.1% sharp · maxed out
Multi-SWE-bench, 7 languages (Mag… 3.6% ▄···●▄·▄▄·▄█ beats 1/7 very hard · avg 2.6% 0.7% blunt · too hard for it
HumanEval (simple-evals) (maker) 97.6% ▆█▆▆▂··●···▂ beats 14/15 easy · avg 83% 0.6% maxed out
WebApp1K-React pass@1 (onekq-ai) 96.1% ▃▁▁█▁▄▅▃▄▇▁● beats 31/31 medium · avg 40% 0.5% sharp · noisy
chat & writing · #146 of 402 · 8.5% of its score
Trails [LFM2.5 1.2B Thinking](lfm2-5-1-2b-thinking.md), [LFM2.5 8B-A1B](lfm2-5-8b-a1b.md), [LFM2.5 2.6B](lfm2-5-2-6b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena-Hard v2.0 hard prompts (sty… 66.1% ▂·▃▅▅·▃█▆●▂▂ beats 20/22 very hard · avg 12% 3.4% sharp · at its level
LMArena (text) 1363.6 ▁▁▁▂▃▃▃▆●▇█▃ beats 205/385 avg 1260 2.9% blunt
Arena-Hard (version/judge unspeci… 81.9% ▃▃▂▅▃▆▅█▅▅●▃ beats 28/30 medium · avg 59% 1.2% typical
Arena-Hard v2.0 creative writing … 43.0% ▆▂▂▆▂●▃█▂▃·▂ beats 11/22 hard · avg 19% 0.7% sharp · at its level · noisy
IFEval (aggregation unspecified) 91.5% ▁▂▄▂▂▄▆██●▂▁ beats 121/145 easy · avg 74% 0.3% typical
math · #121 of 431 · 22% of its score
Beats [Claude Sonnet 4.6](claude-sonnet-4-6.md), [MiniMax-M3](minimax-m3.md), [MiMo-V2.5 (310B-A15B)](mimo-v2-5.md) here, though they rank higher overall. Trails [Olmo 3.1 32B Think](olmo-3-1-32b-think.md), [Granite 4.2 8B](granite-4-2-8b.md), [gpt-oss-120b](gpt-oss-120b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
FrontierMath tiers 1–3 v2 18.6% ▂··▁▁●▄▄█▆▄▃ beats 11/86 very hard · avg 1.2% 10% sharp
MATH level 5 96.5% ▂▂▅▆▄█▆▃▅▂●▄ beats 85/92 hard · avg 28% 2.3% sharp
FrontierMath tier 4 v2 0.0% ●▂▁▂█▅▂▃▂▁▁▁ beats 0/67 very hard · avg 0.1% 2.1% sharp · too hard for it
MATH (Vector Institute, Inspect E… 96.9% ▄··██▄··▄·▄● beats 8/9 medium · avg 50% 2.0% sharp
OTIS mock AIME 2024–2025 76.9% ▃▃▄▃▃▃▇●█▃▂▅ beats 110/197 very hard · avg 7.2% 1.9% sharp · at its level · noisy
MATH-500 (simple-evals) (maker) 97.9% ▃··▅▅▃···▃·● beats 7/9 medium · avg 61% 1.3% sharp
AIME 2025 82.5% █··▄··█··▄·● beats 6/6 hard · avg 32% 1.1% sharp · noisy
MGSM (simple-evals) (maker) 92.0% ▁····▁▅▁█▁●▁ beats 14/16 easy · avg 80% 0.7% typical
AIME 2024 88.0% ▄···▄█·▄▄··● beats 6/6 medium · avg 43% 0.7% sharp · noisy
web search · #8 of 84 · 2.2% of its score
Beats [GPT-6 Astra](gpt-6-astra.md), [Claude Opus 5.5](claude-opus-5-5.md), [GPT-6.1 Sol](gpt-6-1-sol.md) here, though they rank higher overall. Trails [GPT-4o (2024-08-06)](gpt-4o-2024-08-06.md), [GPT-4.1 mini](gpt-4-1-mini.md), [GPT-4.1](gpt-4-1.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Finance Agent Benchmark v1 (537 q… 12.8% █·▃▃·●▅▅▅▅▃▅ beats 7/18 very hard · avg 5.5% 2.2% sharp · at its level
other tests (19)
test result vs the others in its run difficulty counts why
GPQA Diamond 77.0% ▁▁▁▁▂▄▆▄▅●█▅ beats 121/211 hard · avg 39% 8.2% sharp · at its level · several runs
ZebraLogic (ZeroEval, 1,000 puzzl… 91.7% ▁▁▃▆██▁▁·▁▁● beats 57/57 hard · avg 15% 6.3% sharp · at its level
GPQA Diamond (simple-evals) (make… 77.2% █▃▃█·▃▃▅▃▃●▅ beats 14/16 hard · avg 39% 5.0% sharp · at its level
GPQA Diamond (Vector Institute, I… 73.7% ▃··▃▃▃▃·▃·▃● beats 7/9 hard · avg 40% 3.7% sharp · at its level
MMLU-Pro (Vector Institute, Inspe… 79.2% ▃····▃▅·▃●·█ beats 6/9 medium · avg 58% 1.8% typical
AraGen-v3 3C3H score (Inception, … 59.8% ▁▁·▁▂▂▄█▄●▄▄ beats 50/66 hard · avg 29% 1.6% at its level
MMLU (simple-evals, zero-shot CoT… 86.9% ▂▂·█●▄▂·█▂·▂ beats 7/16 easy · avg 79% 1.5% typical
MMLU-Pro 79.4% ▁▁·▁▁▁▃▂▄●█▁ beats 124/188 medium · avg 59% 1.3% typical
HindiGen-v1 3C3H score (Inception… 55.1% ▁▁▁▁·▂·●▆█▇▁ beats 10/31 medium · avg 46% 1.1% typical
Chess Puzzles (Epoch) 17.0% █▁▁▁▁▂●▃▃▂▁▁ beats 71/137 very hard · avg 5.8% 1.0% sharp · noisy
Mystery Game Puzzles (Epoch) 7.0% ●··▂▃▇▄█▂▄▂▁ beats 2/77 very hard · avg 9.4% 1.0% sharp · too hard for it · noisy
SimpleQA Verified 15.3% ▁▁▃●▃▃▆▆█▄▂▅ beats 8/81 very hard · avg 8.0% 0.9% at its level
EuroEval HellaSwag (English), val… 75.8% ▂··▂▂·▂▆●█·▄ beats 8/15 medium · avg 63% 0.8% typical
Multi-IF (aggregation unspecified) 87.3% ▄·····▄█▄▄·● beats 6/6 easy · avg 66% 0.7% typical
FlenQA (3K-token subset) 96.8% ▄···▄····●·█ beats 2/4 easy · avg 86% 0.5% sharp · maxed out · noisy
SimpleQA (simple-evals) (maker) 13.8% ▄·●▁▁▁··█▁·▁ beats 3/14 hard · avg 17% 0.3% at its level · noisy
DROP F1, 3-shot (simple-evals) (m… 80.6% ▂▂▄●▆█·▂···▄ beats 5/16 easy · avg 78% 0.2% blunt
Omni-MATH 74.6% █·█·█·●█··██ beats 3/6 medium · avg 64% 0.1% blunt · noisy
Vectara hallucination leaderboard… 99.2% ▁▁▂▃▄▄█▆▃▂▂● beats 138/140 very easy · avg 94% 0.1% blunt · maxed out · noisy
source results share of the score
Epoch AI 8 23% ▇▇▇
ServiceNow-AI/Apriel-1.6-15b-Thinker mo… 5 13% ▇▇
Microsoft 9 10% ▇
OpenAI simple-evals README benchmark re… 7 10% ▇
Vector Institute Evaluation Leaderboard 4 9% ▇
Aider polyglot benchmark runs 1 7% ▇
WildEval ZeroEval published results 1 6% ▇
Arena-Hard-Auto v2.0-Preview leaderboard 2 4% ▇
SciCode leaderboard 1 4% ▇
LMArena leaderboard dataset 1 3% ▇
Inception Arabic/Hindi leaderboards 2 3% ▇
Finance Agent Benchmark 1 2% ▇
WebApp1K models leaderboard 2 2% ▇
LiveCodeBench official website results 1 2% ▇
EuroEval leaderboards 1 1% ▇
Multi-SWE-bench experiments 1 1% ▇
Berkeley Function Calling Leaderboard r… 1 1% ▇
Galileo Agent Leaderboard 1 0% ▇
Vectara hallucination leaderboard 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 65.0
Licences: Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · ServiceNow-AI/Apriel-1.6-15b-Thinker model card ([MIT](../../licenses/cards-final-NOTICE.txt)) · Microsoft ([CC BY 4.0](../../licenses/report-2504.21318v1.txt)) · OpenAI simple-evals README benchmark results ([MIT](../../licenses/openai-simple-evals-MIT.txt)) · Vector Institute Evaluation Leaderboard ([Apache-2.0](../../licenses/vector-inspect-evals-leaderboard-Apache-2.0.txt)) · Aider polyglot benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) · WildEval ZeroEval published results ([Apache-2.0](../../licenses/zeroeval-published-results-Apache-2.0.txt)) · Arena-Hard-Auto v2.0-Preview leaderboard ([Apache-2.0](../../licenses/arena-hard-auto-Apache-2.0.txt)) · SciCode leaderboard ([Apache-2.0](../../licenses/scicode-Apache-2.0.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Finance Agent Benchmark ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · WebApp1K models leaderboard ([MIT](../../licenses/webapp1k-leaderboard-MIT.txt)) · LiveCodeBench official website results ([CC-BY-SA-4.0](../../licenses/lcb-sharealike-NOTICE.txt)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · Multi-SWE-bench experiments ([Apache-2.0](../../licenses/multi-swe-bench-experiments-Apache-2.0.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · Galileo Agent Leaderboard ([Apache-2.0](../../licenses/galileo-agent-leaderboard-Apache-2.0.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
### Through an API
api (OpenAI) · $1.93/M blended · $1.1 in / $4.4 out per 1M tokens · [developers.openai.com](https://developers.openai.com/api/docs/models/o3-mini)
[price source](https://developers.openai.com/api/docs/models/o3-mini): USD per 1M tokens, OpenAI API standard processing, model page 'o3-mini' whose default snapshot is o3-mini-2025-01-31: input $1.1, output $4.4; cached-input, Batch and other tiers excluded. Currency: USD. OpenAI official model page, observed 2026-09-28. · [context source](https://developers.openai.com/api/docs/models/o3-mini): OpenAI model page for o3-mini (default snapshot o3-mini-2025-01-31): 200,000 context window. Observed 2026-09-28.
[interactive version](https://whatmodel.sh/model/o3-mini) · [JSON](https://whatmodel.sh/api?format=json&all=1)