[whatmodel.sh](../../README.md) / GPT-4o mini (2024-07-18)
# GPT-4o mini (2024-07-18)
OpenAI · released 2024-07 · proprietary
**score 54.0** ±0.3 · #302 of 732 · from 107 benchmark results across 38 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks GPT-4o mini (2024-07-18) among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆●▄▅▅▄▄▂▁ #220 of 313 3.1% 9
coding ▁▁▁▃▅▇●▇▅▄▃▁ #222 of 469 21% 10
chat & writing ▁▁▁▃▆▄▆●▆▇▇▃ #208 of 402 12% 7
math ▁▁▃▅●▆▄█▄▂▁▁ #260 of 431 13% 13
vision & documents ▁▁▂▂●▃▄▂▆▇█▄ #124 of 153 7.1% 12
web search ▁▁▁▅▄█▃●·▁·▁ #11 of 84* 0.4% 1
agent tasks · #220 of 313 · 3.1% of its score
Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md), [DeepSeek R1 (Jan '25)](deepseek-r1.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Mellum2 12B-A2.5B Instruct](mellum2-12b-a2-5b-instruct.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
BrowserGym WorkArena-L1 (AgentLab… 27.0% ●···█▄▄·▄█·█ beats 0/10 hard · avg 28% 0.7% at its level
BrowserGym WebArena (AgentLab Gen… 17.4% ●··▄·▄█·▄··▄ beats 0/7 hard · avg 17% 0.6% typical
BrowserGym MiniWoB (AgentLab Gene… 56.6% ●····▅▅▃▃·▃█ beats 0/11 medium · avg 53% 0.5% blunt · at its level
Finance Agent Benchmark v1 (537 q… 10.8% █·▃▃·●▅▅▅▅▃▅ beats 6/18 very hard · avg 5.5% 0.4% sharp
BrowserGym WorkArena-L2 (AgentLab… 1.3% ●▅▃·▅▃▃··█·▃ beats 0/11 very hard · avg 3.6% 0.3% sharp · too hard for it · noisy
BFCL V4 (2025-11-03 archive, mode… 42.1% ▃··▅▃●·▅·▃·█ beats 4/10 hard · avg 33% 0.3% at its level
BFCL v3 64.0% ▁▁▁▁▁▃▅█▅●▁▁ beats 93/114 medium · avg 51% 0.3% blunt · at its level
Galileo Agent Leaderboard v1 (too… 83.2% ▃▃▃▃▆●▆▅█▂▅▂ beats 13/31 easy · avg 78% 0.2% typical
KlinikeBench strict pass@1 (333 i… 0.6% ●▁·▁▁▁▁▅▃█▅█ beats 0/27 very hard · avg 2.4% 0.1% too hard for it
coding · #222 of 469 · 21% of its score
Beats [Qwen3.5 4B](qwen3-5-4b.md), [Qwen3-VL 32B Instruct](qwen3-vl-32b-instruct.md), [Solar Open 100B (102B-A12B)](solar-open-100b.md) here, though they rank higher overall. Trails [QwQ-32B-Preview](qwq-32b-preview.md), [Ministral 3 3B Reasoning 2512](ministral-3-3b-reasoning.md), [s1.1 32B](s1-1-32b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
HumanEval+ 83.5% ▁·▁▁▁▃▄▄▄█▃● beats 95/99 easy · avg 70% 5.6% sharp · several runs
HumanEval 88.4% ▁·▁▁▃▄▆▅█●▁▁ beats 94/99 easy · avg 76% 4.8% sharp
CRUXEval (ZeroEval, 800 problems) 75.9% ▁▂▆▃▆█●▃▂·▁▁ beats 39/47 medium · avg 57% 2.8% sharp · at its level
Aider code editing (133 Exercism … 55.6% ▁·▁▂▄▁●▄█▆▂▂ beats 23/62 medium · avg 51% 2.1% sharp · at its level
Aider polyglot (225 exercises, tw… 3.6% ●·▂▃▅▂▃█▆▄▂▂ beats 0/49 very hard · avg 7.6% 1.6% sharp
MBPP+ 72.2% ▁···▁▁▂▅▇█●▂ beats 67/75 easy · avg 66% 1.1% at its level
HumanEval (Vector Institute, Insp… 86.0% ▄·▄·●·█·█·▄▄ beats 2/9 easy · avg 77% 1.0% sharp
HumanEval (simple-evals) (maker) 87.2% ▆●▆▆▂··▂···▂ beats 3/15 easy · avg 83% 0.7% several runs
LiveCodeBench official (Aug 1, 20… 27.5% ▃●▅▃··▃▅█▅▅▃ beats 1/15 hard · avg 24% 0.7% sharp · at its level
WebApp1K-React pass@1 (onekq-ai) 82.7% ▃▁▁█▁▄▅▃●▇▁▄ beats 21/31 medium · avg 40% 0.5% sharp · at its level · noisy
chat & writing · #208 of 402 · 12% of its score
Beats [Gemma 2 9B IT SimPO](gemma-2-9b-simpo.md), [Granite 4.2 8B](granite-4-2-8b.md), [Mistral Small 4 (119B-A6.5B)](mistral-small-4.md) here, though they rank higher overall. Trails [LFM2.5 1.2B Instruct](lfm2-5-1-2b-instruct.md), [Gemma 3n E4B](gemma-3n-e4b.md), [LFM2.5 1.2B Thinking](lfm2-5-1-2b-thinking.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
AlpacaEval 2 length-controlled 50.7% ▁▂▂▃▆▆█▆▄●▁▁ beats 107/122 medium · avg 35% 5.9% sharp · at its level · several runs
Arena-Hard (v0.1, GPT-4-1106 judg… 74.9% ▂▁▁▅█▅▄▃▃●▃▁ beats 47/60 medium · avg 47% 2.7% sharp · at its level
LMArena (text) 1317.6 ▁▁▁▂▃▃▃●▆▇█▃ beats 149/385 avg 1260 1.2% blunt
Arabic IFEval prompt-level (Incep… 68.1% ▁·▁▁▂▁▄▄▆▄█● beats 37/38 medium · avg 46% 0.9% sharp · at its level
Arena-Hard (version/judge unspeci… 76.2% ▃▃▂▅▃▆▅█●▅▂▃ beats 24/30 medium · avg 65% 0.5% at its level
IFEval prompt-level loose 83.5% ▁▃▁▃▁▄▁▁●▃▃▁ beats 17/22 easy · avg 78% 0.2% typical
IFEval (aggregation unspecified) 80.0% ▁▂▄▂▂▄●██▅▂▁ beats 52/145 easy · avg 76% 0.2% typical
math · #260 of 431 · 13% of its score
Beats [GPT-4o (2024-11-20)](gpt-4o-2024-11-20.md), [GPT-4 Turbo (2024-04-09)](gpt-4-turbo-2024-04-09.md), [Llama 3.1 405B Instruct](llama-3-1-405b.md) here, though they rank higher overall. Trails [DeepSeek R1 Distill Qwen 1.5B](deepseek-r1-distill-qwen-1-5b.md), [Gemma 7B (base)](gemma-7b-base.md), [CodeGemma 7B (v1.0)](codegemma-7b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
MATH level 5 52.6% ▂▂▅▆▄●▆▃▅▂▃▄ beats 46/92 hard · avg 28% 2.6% sharp · at its level
MATH (Vector Institute, Inspect E… 63.3% ▄··●█▄··▄·▄█ beats 2/9 medium · avg 50% 1.8% sharp · at its level
MGSM 81.7% ▅▃▃█·▅···▃·● beats 10/10 easy · avg 72% 1.6% sharp
MATH Level 5 (ZeroEval, 721 probl… 52.1% ▁▁▁▆█▂▇●▄··▂ beats 28/38 hard · avg 30% 1.6% sharp · at its level
GSM8K 83.0% ▂▃··▃▂▅●▆▃▂▃ beats 9/22 very easy · avg 90% 1.3% sharp · maxed out · several runs
GSM8K (ZeroEval, zero-shot) 94.2% ▁▄·▁▄█▅▅▅█●█ beats 33/43 very easy · avg 87% 0.9% sharp · maxed out
MGSM (simple-evals) (maker) 87.0% ▁····▁●▁█▁▃▁ beats 4/16 easy · avg 80% 0.8% several runs
MATH 70.2% █▄██···▄▄▄·● beats 10/10 medium · avg 61% 0.7% at its level
FrontierMath tiers 1–3 v2 0.7% ●··▁▁▃▄▄█▆▄▃ beats 2/86 very hard · avg 1.2% 0.6% sharp · too hard for it
OTIS mock AIME 2024–2025 6.9% ▃▃▄●▃▃▇▇█▃▂▅ beats 41/197 very hard · avg 7.2% 0.5% sharp · noisy
MATH (simple-evals, original test… 70.2% █····●··▄··█ beats 2/5 medium · avg 62% 0.5% at its level
MATH (subset unspecified) 67.9% ▃▃█▂●▆·····▁ beats 15/22 medium · avg 58% 0.4% at its level
MathVista (testmini) 38.8% ▁··●▄▄▆▇▄▅█▄ beats 1/48 medium · avg 52% 0.2% at its level · noisy
vision & documents · #124 of 153 · 7.1% of its score
Beats [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md) here, though they rank higher overall. Trails [Qwen3.5 2B](qwen3-5-2b.md), [Qwen3-VL 4B Instruct](qwen3-vl-4b-instruct.md), [Gemma 4 E4B](gemma-4-e4b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
CharXiv descriptive questions 74.9% ▄█▄▄▄·█●·▄·█ beats 9/12 easy · avg 69% 2.2% several runs
CharXiv (reasoning questions) 34.1% ▂▂▂█●▂·▂▂··▂ beats 9/13 hard · avg 34% 1.5% at its level · several runs
Arena vision (style controlled) 1096.9 ▁▁▂▂▂▂●▄▅▅█▇ beats 30/152 avg 1061 0.8% blunt
MMMU val (VHELM, zero-shot exact … 54.8% ▄▄··▄●█··▄▄█ beats 3/9 medium · avg 50% 0.7% at its level
MathVista testmini (VHELM, zero-s… 47.5% ▃·▅···●▃▃·▃█ beats 3/9 medium · avg 42% 0.6% at its level
CapArena-Auto detailed image capt… 9.3300 ▃▃▃·▃█·●▃▃▃▃ beats 7/12 avg -4.498 0.5% typical
MMMU (val) 52.1% ▁····▁▁▃●▆▆▅ beats 13/53 medium · avg 51% 0.4% at its level
MathVista (testmini) 38.8% ▁··●▄▄▆▇▄▅█▄ beats 1/48 medium · avg 52% 0.2% at its level · noisy
PhysBench physical-world understa… 43.1% ▅▃█··●▃····▃ beats 6/9 hard · avg 42% 0.2% blunt · at its level
TextVQA 70.9% ▁▁▁▄●▁▃·▃▃▃█ beats 8/24 easy · avg 75% 0.1% blunt
AI2D test 75.2% ▁▁▅●▃▇·▄·█·▃ beats 8/28 easy · avg 82% 0.0% blunt
ChartQA 54.5% ▁··▁▁●▁▁▂▅▅█ beats 3/44 easy · avg 78% 0.0% blunt · noisy
web search · #11 of 84 · 0.4% of its score
Beats [Claude Opus 5.5](claude-opus-5-5.md), [Claude Fable 5.1](claude-fable-5-1.md), [Claude Sonnet 5.5](claude-sonnet-5-5.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
Finance Agent Benchmark v1 (537 q… 10.8% █·▃▃·●▅▅▅▅▃▅ beats 6/18 very hard · avg 5.5% 0.4% sharp
other tests (42)
test result vs the others in its run difficulty counts why
LiveBench 2024-07-26 44.3% ▁▁▁▂▂▃▂█▆●▅▃ beats 47/61 medium · avg 38% 5.0% at its level
MMLU-Redux (ZeroEval, 2,778 quest… 81.5% ▁·▁▁▄▄▄▅●▄▂▁ beats 32/53 easy · avg 75% 4.2% typical
MMLU-Pro 63.1% ▃▂▅█▃·●▃▅··▂ beats 15/23 medium · avg 54% 3.7% at its level · several runs
MMLU 82.2% ▁▂·▁·▁▁█·▃●▅ beats 17/22 easy · avg 75% 2.8% several runs
GPQA Diamond (simple-evals) (make… 40.2% ●▃▃█·▃▃▅▃▃▃▅ beats 0/16 hard · avg 39% 2.7% sharp · at its level · several runs
BIG-Bench Hard 65.9% ▁▄·▁▁·●█·▄▄▁ beats 8/22 medium · avg 72% 2.5% sharp · at its level · several runs
ZebraLogic (ZeroEval, 1,000 puzzl… 20.1% ▁▁▃▆●█▁▁·▁▁▁ beats 30/57 hard · avg 15% 2.1% sharp · at its level
CLIcK Korean (Daekeun Kim) 68.5% ▆▄▄●█▄▂·▄█·▄ beats 8/23 medium · avg 56% 1.8% at its level
GPQA Diamond 37.7% ▁▁▁▁▂●▆▄▅▇█▅ beats 33/211 hard · avg 39% 1.6% sharp · at its level
HAE-RAE 1.0 (Daekeun Kim) 76.4% ▄▄▄▂▂●▂▄▂█▂▄ beats 8/22 medium · avg 61% 1.5% at its level
MMLU (simple-evals, zero-shot CoT… 82.0% ▂●·█▄▄▂·█▂·▂ beats 1/16 easy · avg 79% 1.3% several runs
GPQA Diamond (Vector Institute, I… 38.4% ▃··●▃▃▃·▃·▃█ beats 1/9 hard · avg 40% 1.3% sharp · at its level
WildBench v2 WB-Score (task macro) 57.1369 WB-Score ▁·▁▁·▃█▄▇▆▅● beats 50/52 avg 43.019 WB-Score 1.2% sharp
ARC Challenge 93.5% ▂▂▂█▂···▄··● beats 10/10 very easy · avg 89% 1.1% sharp
MMLU-Pro (Vector Institute, Inspe… 64.0% ▃····●▅·▃▅·█ beats 1/9 medium · avg 58% 1.0% at its level
Multilingual MMLU (Microsoft) 72.9% █▄▄██··█···● beats 10/10 medium · avg 63% 0.8% at its level
OpenBookQA 90.0% ▂··▄█▂·▂···● beats 9/10 easy · avg 85% 0.7% typical
AraGen-v3 3C3H score (Inception, … 50.7% ▁▁·▁▂▂▄█▄●▄▄ beats 45/66 hard · avg 29% 0.7% at its level
KCL-Essay vanilla 16.1% ▁···●▃▂█▂▃▃▂ beats 1/23 hard · avg 17% 0.6% typical
HindiGen-v1 3C3H score (Inception… 65.5% ▁▁▁▁·▂·█●█▇▁ beats 17/31 medium · avg 46% 0.6% at its level
Polish EQ-Bench v2 (SpeakLeash) 71.2% ▁··▁▁▂▂▆▅▆●▆ beats 55/70 medium · avg 62% 0.6% at its level
EvalPerf correctness (118 tasks, … 85.5% ▂·▂▂▂▆▅▆▆▆●█ beats 23/29 easy · avg 74% 0.6% typical
KMMLU-HARD (Daekeun Kim, zero-sho… 24.6% ●···▁▁▁▂▃·▆▂ beats 1/23 very hard · avg 29% 0.6% sharp
GPQA (subset unspecified) 41.1% █··▄▂··▂▂··● beats 10/10 very hard · avg 32% 0.5% sharp · noisy
Open Persian LLM Leaderboard v2 a… 68.9% ▂▄▂▃█▄▃▆▃●▂▂ beats 69/81 medium · avg 57% 0.5% blunt · at its level
MMMLU (maker) 70.5% ▂▁▁▂▃▂▄▄●██▃ beats 20/46 medium · avg 64% 0.5% at its level · small run
WinoGrande 76.9% ▃····▃▃█▃·▅● beats 10/10 medium · avg 71% 0.4% at its level
Judgemark v2 (LLM-as-judge for cr… 44.0% ▃●··▃▅▅···█▃ beats 1/10 medium · avg 53% 0.4% at its level
PopQA 39.0% ▁▁▁▃▃▃▁█▁●·▄ beats 18/21 hard · avg 31% 0.4% at its level
HellaSwag 88.7% ▃▅█·▃▃▃···▃● beats 10/10 easy · avg 82% 0.3% typical
Social IQA 82.9% ▃·▃▅▅█·▃···● beats 10/10 medium · avg 76% 0.3% blunt · at its level
KCL-MCQA vanilla 29.9% ▂···█●▆▆██▆▂ beats 5/23 very hard · avg 29% 0.3% typical
BoolQ 88.7% ▃··▃··██▃·▃● beats 10/10 easy · avg 83% 0.3% at its level
Open PT-LLM Leaderboard average (… 76.8% ▃▁▁▁▁▁▁▁▂█●▁ beats 236/276 medium · avg 72% 0.3% at its level · noisy
SimpleQA Verified 8.3% ▁●▃▂▃▃▆▆█▄▂▅ beats 1/81 very hard · avg 8.0% 0.2% typical
DROP F1, 3-shot (simple-evals) (m… 79.7% ▂▂●▆▆█·▂···▄ beats 3/16 easy · avg 78% 0.2% blunt · several runs
PIQA 88.7% ▃·▅·▃█·▃▅··● beats 10/10 medium · avg 82% 0.1% blunt · at its level
TruthfulQA MC2 64.8% ▃▃▂·▃▃·▅●▂▅▂ beats 15/21 medium · avg 64% 0.1% blunt · at its level · several runs
SimpleQA (simple-evals) (maker) 9.5% ●·▁▁▁▁··█▁·▁ beats 2/14 hard · avg 17% 0.1% noisy
Chess Puzzles (Epoch) 0.0% ●▁▁▁▁▂▃▃▃▂▁▁ beats 12/137 very hard · avg 5.8% 0.1% sharp · too hard for it · noisy
Vectara hallucination leaderboard… 98.3% ▁▁▂▃▄▄█▆▃●▂▁ beats 128/140 very easy · avg 94% 0.1% blunt · maxed out · noisy
Mystery Game Puzzles (Epoch) 12.0% █··●▃▇▄█▂▄▂▁ beats 16/77 very hard · avg 9.4% 0.0% sharp · too hard for it · noisy
source results share of the score
WildEval ZeroEval published results 5 12% ▇
microsoft/Phi-4-mini-instruct model card 16 11% ▇
EvalPlus / EvalPerf published evaluatio… 4 11% ▇
Epoch AI 7 6% ▇
Vector Institute Evaluation Leaderboard 4 5% ▇
AlpacaEval 1.0 and 2.0 leaderboards 1 5% ▇
LiveBench model judgments 1 5% ▇
2 OLMo 2 Furious 8 5% ▇
Microsoft Research 9 4% ▇
OpenAI simple-evals README benchmark re… 7 4% ▇
Daekeun Kim Korean LLM benchmark runs 3 4% ▇
Arena-Hard-Auto v0.1 leaderboard 1 3% ▇
Inception Arabic/Hindi leaderboards 3 2% ▇
Aider code editing benchmark runs 1 2% ▇
ServiceNow BrowserGym leaderboard 4 2% ▇
LMArena leaderboard dataset 2 2% ▇
CharXiv leaderboard 2 2% ▇
MMLU-Pro author-released evaluation runs 1 2% ▇
OpenAI vendor-reported benchmark scores… 2 2% ▇
Aider polyglot benchmark runs 1 2% ▇
WildBench v2 leaderboard data 1 1% ▇
VHELM: A Holistic Evaluation of Vision … 2 1% ▇
Korean Canonical Legal Benchmark: Towar… 2 1% ▇
LiveCodeBench official website results 1 1% ▇
Microsoft Phi-3.5-vision-instruct model… 5 1% ▇
SpeakLeash Polish EQ-Bench leaderboard 1 1% ▇
WebApp1K models leaderboard 1 1% ▇
Open Persian LLM Leaderboard v2 1 1% ▇
Berkeley Function Calling Leaderboard r… 2 1% ▇
CapArena: Benchmarking and Analyzing De… 1 0% ▇
OpenAI (submitted / maker-reported) 1 0% ▇
Judgemark v2 results 1 0% ▇
Finance Agent Benchmark 1 0% ▇
Open Portuguese LLM Leaderboard 1 0% ▇
Galileo Agent Leaderboard 1 0% ▇
PhysBench: Benchmarking and Enhancing V… 1 0% ▇
KlinikeBench: Evaluating Language Model… 1 0% ▇
Vectara hallucination leaderboard 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 54.0
Licences: WildEval ZeroEval published results ([Apache-2.0](../../licenses/zeroeval-published-results-Apache-2.0.txt)) · microsoft/Phi-4-mini-instruct model card ([MIT](../../licenses/card-microsoft-phi-4-mini-instruct.txt)) · EvalPlus / EvalPerf published evaluation runs ([Apache-2.0](../../licenses/evalplus-published-runs-Apache-2.0.txt)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Vector Institute Evaluation Leaderboard ([Apache-2.0](../../licenses/vector-inspect-evals-leaderboard-Apache-2.0.txt)) · AlpacaEval 1.0 and 2.0 leaderboards ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · LiveBench model judgments ([Apache-2.0 (stated in the LiveBench datasheet)](https://www.apache.org/licenses/LICENSE-2.0)) · 2 OLMo 2 Furious ([CC BY 4.0](../../licenses/paper-2501.00656v3-CC-BY-4.0.txt)) · Microsoft Research ([CC BY 4.0](../../licenses/report-2412.08905v1.txt)) · OpenAI simple-evals README benchmark results ([MIT](../../licenses/openai-simple-evals-MIT.txt)) · Daekeun Kim Korean LLM benchmark runs ([MIT](../../licenses/korean-personal-benchmark-runs-MIT.txt)) · Arena-Hard-Auto v0.1 leaderboard ([Apache-2.0](../../licenses/arena-hard-auto-v0-1-Apache-2.0.txt)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Aider code editing benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) · ServiceNow BrowserGym leaderboard ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · CharXiv leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · MMLU-Pro author-released evaluation runs ([Apache-2.0](../../licenses/mmlu-pro-author-runs-Apache-2.0.txt)) · Aider polyglot benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) · WildBench v2 leaderboard data ([Apache-2.0](https://github.com/allenai/WildBench/blob/d6b8dcaf377d173d031980f97c16e1a82618c03d/LICENSE)) · VHELM: A Holistic Evaluation of Vision Language Models ([CC BY 4.0](http://creativecommons.org/licenses/by/4.0/)) · Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities ([CC BY 4.0](../../licenses/cards-final-NOTICE.txt)) · LiveCodeBench official website results ([CC-BY-SA-4.0](../../licenses/lcb-sharealike-NOTICE.txt)) · Microsoft Phi-3.5-vision-instruct model card: Benchmarks table ([MIT](https://huggingface.co/microsoft/Phi-3.5-vision-instruct/blob/12b77fb40b63a2c73c68243d3f767aab688a1b2a/LICENSE)) · SpeakLeash Polish EQ-Bench leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · WebApp1K models leaderboard ([MIT](../../licenses/webapp1k-leaderboard-MIT.txt)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · CapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Judgemark v2 results ([MIT](../../licenses/judgemark-v2-MIT.txt)) · Finance Agent Benchmark ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Galileo Agent Leaderboard ([Apache-2.0](../../licenses/galileo-agent-leaderboard-Apache-2.0.txt)) · PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
### Through an API
api (OpenAI) · $0.26/M blended · $0.15 in / $0.6 out per 1M tokens · [developers.openai.com](https://developers.openai.com/api/docs/models/gpt-4o-mini)
[price source](https://developers.openai.com/api/docs/models/gpt-4o-mini): USD per 1M tokens, OpenAI API standard processing, model page 'gpt-4o-mini' whose default snapshot is gpt-4o-mini-2024-07-18: input $0.15, output $0.6; cached-input, Batch and other tiers excluded. Currency: USD. OpenAI official model page, observed 2026-09-28. · [context source](https://developers.openai.com/api/docs/models/gpt-4o-mini): OpenAI model page for gpt-4o-mini (default snapshot gpt-4o-mini-2024-07-18): 128,000 context window. Observed 2026-09-28.
[interactive version](https://whatmodel.sh/model/gpt-4o-mini-2024-07-18) · [JSON](https://whatmodel.sh/api?format=json&all=1)