[whatmodel.sh](../../README.md) / GPT-4o (2024-11-20)
# GPT-4o (2024-11-20)
OpenAI · released 2024-11 · proprietary
**score 57.6** ±0.6 · #234 of 732 · from 31 benchmark results across 13 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks GPT-4o (2024-11-20) among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█●▅▅▄▄▂▁ #173 of 313 41% 10
coding ▁▁▁▃▅▇●▇▅▄▃▁ #219 of 469 28% 5
math ▁▁▃▅●▆▄█▄▂▁▁ #269 of 431 13% 3
vision & documents ▁▁▂▂▁▃▄●▆▇█▄ #97 of 153* 6.3% 2
agent tasks · #173 of 313 · 41% of its score
Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md), [DeepSeek R1 0528](deepseek-r1-0528.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Jamba 1.5 Large](jamba-1-5-large.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
τ²-bench Telecom 24.1% █··●█··██··█ beats 1/5 very hard · avg 9.9% 14% sharp · at its level · several runs
SWE-bench Verified 31.0% ●··▁·▁▂▁▄█▄▁ beats 0/29 hard · avg 19% 10% sharp · at its level
τ-bench Retail 60.3% ▁····▃·▂●█▂▆ beats 10/28 medium · avg 40% 4.8% at its level · small run · several runs
τ²-bench Retail 66.7% ▁····▂▁▃●█▃▁ beats 21/44 medium · avg 44% 3.2% at its level · small run · several runs
τ²-bench Airline 42.0% ▁▁·▂●▃▂▁█▅▂▃ beats 6/42 hard · avg 25% 2.7% at its level · small run · several runs
τ-bench Airline 42.8% ▁▁▁·▅▄▁●▇▁▁▇ beats 11/28 hard · avg 21% 2.6% at its level · several runs
BFCL V4 (2025-11-03 archive, mode… 52.2% ▃··▅▃▃·●·▃·█ beats 5/10 hard · avg 33% 1.0% at its level
BFCL v3 71.7% ▁▁▁▁▁▃▅█▅▄●▁ beats 110/114 medium · avg 51% 0.9% blunt · at its level
Multi-SWE-bench, 7 languages (Mag… 3.8% ▄···▄●·▄▄·▄█ beats 2/7 very hard · avg 2.6% 0.8% blunt · too hard for it
Galileo Agent Leaderboard v1 (too… 90.0% ▃▃▃▃▆▅▆▅●▂▅▂ beats 23/31 easy · avg 78% 0.6% typical
coding · #219 of 469 · 28% of its score
Beats [Qwen3.5 4B](qwen3-5-4b.md), [Qwen3-VL 32B Instruct](qwen3-vl-32b-instruct.md), [Solar Open 100B (102B-A12B)](solar-open-100b.md) here, though they rank higher overall. Trails [QwQ-32B-Preview](qwq-32b-preview.md), [Ministral 3 3B Reasoning 2512](ministral-3-3b-reasoning.md), [s1.1 32B](s1-1-32b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 31.0% ●··▁·▁▂▁▄█▄▁ beats 0/29 hard · avg 19% 10% sharp · at its level
Aider polyglot (225 exercises, tw… 18.2% ▂·▂▃●▂▃█▆▄▂▂ beats 10/49 very hard · avg 7.6% 9.2% sharp · at its level
Aider code editing (133 Exercism … 71.4% ▁·▁▂▄▁▆▄█●▂▂ beats 50/62 medium · avg 51% 6.9% sharp · at its level
HumanEval (simple-evals) (maker) 90.2% ▆█●▆▂··▂···▂ beats 8/15 easy · avg 83% 1.3% typical
Multi-SWE-bench, 7 languages (Mag… 3.8% ▄···▄●·▄▄·▄█ beats 2/7 very hard · avg 2.6% 0.8% blunt · too hard for it
math · #269 of 431 · 13% of its score
Trails [DeepSeek R1 Distill Qwen 1.5B](deepseek-r1-distill-qwen-1-5b.md), [Gemma 7B (base)](gemma-7b-base.md), [CodeGemma 7B (v1.0)](codegemma-7b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
MATH level 5 49.8% ▂▂▅▆▄●▆▃▅▂▃▄ beats 41/92 hard · avg 28% 8.7% sharp · at its level
OTIS mock AIME 2024–2025 6.3% ▃▃●▃▃▃▇▇█▃▂▅ beats 34/197 very hard · avg 7.2% 3.0% sharp · at its level · noisy
MGSM (simple-evals) (maker) 90.3% ▁····▁▅▁●▁▃▁ beats 12/16 easy · avg 80% 1.5% typical
vision & documents · #97 of 153 · 6.3% of its score
Beats [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md), [GPT-5 nano](gpt-5-nano.md), [Qwen3-VL 30B-A3B Thinking](qwen3-vl-30b-a3b-thinking.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
CharXiv descriptive questions (ma… 85.3% ▄▄·····●·▄█▄ beats 2/6 easy · avg 71% 3.5% typical
CharXiv (reasoning questions) (ma… 52.7% ▄··●█··▂···▄ beats 2/9 hard · avg 32% 2.8% at its level
other tests (7)
test result vs the others in its run difficulty counts why
GPQA Diamond (simple-evals) (make… 46.0% █●▃█·▃▃▅▃▃▃▅ beats 3/16 hard · avg 39% 7.6% sharp · at its level
GPQA Diamond 47.9% ▁▁▁▁▂▄●▄▅▇█▅ beats 62/211 hard · avg 39% 7.1% sharp · at its level
MMLU (simple-evals, zero-shot CoT… 85.7% ▂▂·●▄▄▂·█▂·▂ beats 5/16 easy · avg 79% 3.0% typical
HindiGen-v1 3C3H score (Inception… 72.4% ▁▁▁▁·▂·█▆●▇▁ beats 22/31 medium · avg 46% 2.2% at its level
Judgemark v2 (LLM-as-judge for cr… 82.5% ▃▃··▃▅▅···█● beats 10/10 medium · avg 53% 1.4% at its level
DROP F1, 3-shot (simple-evals) (m… 81.5% ▂▂▄▆●█·▂···▄ beats 8/16 easy · avg 78% 0.4% blunt
SimpleQA (simple-evals) (maker) 38.8% ▄·▁▁▁▁··●▁·▁ beats 7/14 hard · avg 17% 0.4% at its level · noisy
source results share of the score
Epoch AI 4 29% ▇▇▇
OpenAI simple-evals README benchmark re… 6 14% ▇▇
Qwen/Qwen3-235B-A22B-Instruct-2507 mode… 5 10% ▇
Aider polyglot benchmark runs 1 9% ▇
Qwen/Qwen3-30B-A3B-Instruct-2507 model … 5 9% ▇
MUA-RL: Multi-turn User-interacting Age… 1 8% ▇
Aider code editing benchmark runs 1 7% ▇
OpenAI vendor-reported benchmark scores… 2 6% ▇
Inception Arabic/Hindi leaderboards 1 2% ▇
Berkeley Function Calling Leaderboard r… 2 2% ▇
Judgemark v2 results 1 1% ▇
Multi-SWE-bench experiments 1 1% ▇
Galileo Agent Leaderboard 1 1% ▇
prior: pull toward the middle 0% ▇
weighted average = 57.6
Licences: Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · OpenAI simple-evals README benchmark results ([MIT](../../licenses/openai-simple-evals-MIT.txt)) · Qwen/Qwen3-235B-A22B-Instruct-2507 model card ([apache-2.0](../../licenses/Qwen--Qwen3-235B-A22B-Instruct-2507.txt)) · Aider polyglot benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) · Qwen/Qwen3-30B-A3B-Instruct-2507 model card ([apache-2.0](../../licenses/Qwen--Qwen3-30B-A3B-Instruct-2507.txt)) · MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Aider code editing benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · Judgemark v2 results ([MIT](../../licenses/judgemark-v2-MIT.txt)) · Multi-SWE-bench experiments ([Apache-2.0](../../licenses/multi-swe-bench-experiments-Apache-2.0.txt)) · Galileo Agent Leaderboard ([Apache-2.0](../../licenses/galileo-agent-leaderboard-Apache-2.0.txt))
## How to run it
### Through an API
api (OpenAI) · $4.38/M blended · $2.5 in / $10 out per 1M tokens · [developers.openai.com](https://developers.openai.com/api/docs/models/gpt-4o)
[price source](https://github.com/anomalyco/models.dev/blob/6b32416340759c52202fef2b0e69cd89adb74bbe/providers/openai/models/gpt-4o-2024-11-20.toml): Standard base token rate from models.dev (revision 6b32416, observed 2026-10-07). Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://developers.openai.com/api/docs/models/gpt-4o): Context limit from the maker's documentation, observed 2026-10-06.
[interactive version](https://whatmodel.sh/model/gpt-4o-2024-11-20) · [JSON](https://whatmodel.sh/api?format=json&all=1)