[whatmodel.sh](../../README.md) / GPT-5.1
# GPT-5.1
OpenAI · released 2025-11 · closed weights
**score 69.6** ±0.7 · #81 of 732 · from 30 benchmark results across 14 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks GPT-5.1 among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█▄▅●▄▄▂▁ #100 of 313 25% 7
coding ▁▁▁▃▅▇█▇●▄▃▁ #97 of 469 22% 3
chat & writing ▁▁▁▃▆▄▆█▆▇●▃ #48 of 402* 5.1% 1
math ▁▁▃▅█▆▄●▄▂▁▁ #77 of 431* 1.7% 1
vision & documents ▁▁▂▂▁▃▄▂▆●█▄ #51 of 153 7.9% 3
web search ▁▁▁▅●█▃▄·▁·▁ #48 of 84* 1.8% 1
agent tasks · #100 of 313 · 25% of its score
Beats [Grok 4.20 (0309 reasoning)](grok-4-20.md) here, though they rank higher overall. Trails [Mistral Medium 3.5 (128B)](mistral-medium-3-5.md), [Hy3 preview (295B-A21B)](hy3-preview.md), [MiniMax-M2.7](minimax-m2-7.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 68.0% ▁··▁·▁▂●▄█▄▁ beats 7/29 hard · avg 19% 10% sharp · at its level
τ²-bench Telecom (maker) 95.6% ▃·▅▅▃▃··█●▃▅ beats 10/13 very hard · avg 12% 4.4% sharp · at its level · noisy · several …
ContractBench success rate (33 ob… 48.5% ▆··▆▄▂▄▂●▆█▄ beats 12/23 very hard · avg 13% 3.9% sharp · at its level
τ²-bench Retail 78.1% ▁····▂▁▃▆█●▁ beats 39/44 medium · avg 37% 2.0% typical
τ²-bench Airline 64.0% ▁▁·▂▁▃▂▁█▅●▃ beats 36/42 hard · avg 29% 1.9% at its level
KlinikeBench strict pass@1 (333 i… 15.3% ▁▁·▁▁▁▁▅▃●▅█ beats 13/27 very hard · avg 2.4% 1.1% typical
τ²-Bench-Verified (GPT-5.1 user) 76.8% ▂··▄···▂▄●█▄ beats 7/13 medium · avg 45% 0.8% typical
coding · #97 of 469 · 22% of its score
Beats [Grok 4.20 (0309 reasoning)](grok-4-20.md) here, though they rank higher overall. Trails [Llama 3.1 Nemotron Ultra 253B v1](llama-3-1-nemotron-ultra-253b.md), [Qwen3 235B-A22B Thinking 2507](qwen3-235b-a22b-thinking-2507.md), [o4-mini](o4-mini.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 68.0% ▁··▁·▁▂●▄█▄▁ beats 7/29 hard · avg 19% 10% sharp · at its level
Arena WebDev (Code Arena) 1395.6 ▁▁▃▂▅●▃▆▅▅▁▂ beats 42/129 avg 981 9.0% sharp
WebApp1K-Duo-React pass@1 (onekq-… 76.5% ▁··▁·▁▁▁▆█▇● beats 42/45 hard · avg 29% 2.2% typical
chat & writing · #48 of 402 · 5.1% of its score
Beats [MiMo-V2.6-Flash (309B-A15B)](mimo-v2-6-flash.md), [Claude Haiku 5.5](claude-haiku-5-5.md), [GPT-6 Luna](gpt-6-luna.md) here, though they rank higher overall. Trails [Gemini 3.5 Flash-Lite](gemini-3-5-flash-lite.md), [MiMo-V2.5-Pro (1T-A42B)](mimo-v2-5-pro.md), [GLM-5](glm-5.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LMArena (text) 1455.7 ▁▁▁▂▃▃▃▆▆▇●▃ beats 330/385 avg 1260 5.1% blunt
math · #77 of 431 · 1.7% of its score
Beats [Claude Sonnet 4.6](claude-sonnet-4-6.md), [Claude Opus 4.5](claude-opus-4-5.md), [Qwen3.7 Plus](qwen3-7-plus.md) here, though they rank higher overall. Trails [Granite 4.2 30B](granite-4-2-30b.md), [INTELLECT-3](intellect-3.md), [Qwen3 235B-A22B Thinking 2507](qwen3-235b-a22b-thinking-2507.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
OTIS mock AIME 2024–2025 88.6% ▃▃▄▃▃▃▇▇●▃▂▅ beats 138/197 very hard · avg 7.2% 1.7% sharp · noisy
vision & documents · #51 of 153 · 7.9% of its score
Beats [MiMo-V2.6-Pro](mimo-v2-6-pro.md), [MiMo-V2.6-Flash (309B-A15B)](mimo-v2-6-flash.md), [GPT-6 Luna](gpt-6-luna.md) here, though they rank higher overall. Trails [Qwen3.6 Flash](qwen3-6-flash.md), [Gemma 4 31B](gemma-4-31b.md), [Gemini 3.5 Flash-Lite](gemini-3-5-flash-lite.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena vision (style controlled) 1251.2 ▁▁▂▂▂▂▃▄▅▅●▇ beats 108/152 avg 1061 3.4% blunt
CharXiv (reasoning questions) (ma… 67.0% ▄··▂█··●···▄ beats 7/9 hard · avg 32% 2.7% at its level
Arena document (style controlled) 1415.8 ▁●▃▃▄▃▇█▅▇▄▅ beats 1/37 avg 1331 1.8% blunt
web search · #48 of 84 · 1.8% of its score
Beats [Gemini 3.1 Pro (Preview)](gemini-3-1-pro.md), [Qwen3.7 Max](qwen3-7-max.md), [GPT-5.2](gpt-5-2.md) here, though they rank higher overall. Trails [GPT-4o mini (2024-07-18)](gpt-4o-mini-2024-07-18.md), [Mistral Small 3.1 24B (2503)](mistral-small-3-1.md), [Llama 4 Scout (109B-A17B)](llama-4-scout.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena search systems (style contr… 1183.6 ▁·▁▁▂▂●█▆▃▂▂ beats 10/36 avg 1069 1.8% blunt
other tests (14)
test result vs the others in its run difficulty counts why
KMMLU-HARD (Daekeun Kim, zero-sho… 72.8% █···▁▁▁▂▃·●▂ beats 16/23 very hard · avg 29% 7.3% sharp · at its level
GPQA Diamond 87.6% ▁▁▁▁▂▄▆▄▅▇●▅ beats 158/211 hard · avg 39% 5.6% sharp
ARC-AGI-2 public evaluation set (… 18.3% █··▂▅▁●▁▃·▁▃ beats 16/24 very hard · avg 0.3% 4.1% sharp · noisy
Mystery Game Puzzles (Epoch) 16.0% █··▂●▇▄█▂▄▂▁ beats 24/77 very hard · avg 9.4% 4.1% sharp · noisy
CLIcK Korean (Daekeun Kim) 89.8% ▆▄▄▄█▄▂·●█·▄ beats 16/23 medium · avg 56% 3.9% typical
Medmarks-V weighted mean win rate… 63.9% ▂▃▅▂▆▃█▅█▅▃● beats 34/34 medium · avg 41% 3.8% blunt · at its level
ARC-AGI-1 public evaluation set (… 77.1% ▁·▇▃▁·█●▁·▇▃ beats 19/27 very hard · avg 4.0% 3.3% sharp · at its level · noisy
Chess Puzzles (Epoch) 32.0% █▁▁▁▁▂▃▃●▂▁▁ beats 108/137 very hard · avg 5.8% 3.2% sharp · noisy
EQ-Bench 3 (emotional intelligenc… 1314.4 ▁·▂▁▄▅▅▃█●▁▁ beats 54/62 avg 1034 3.1% typical
HAE-RAE 1.0 (Daekeun Kim) 90.6% ▄▄▄▂▂█▂▄●█▂▄ beats 15/22 medium · avg 61% 3.0% typical
KOPA-Bench (pass@1) 37.1% ▄·█··▄··█●·█ beats 6/8 very hard · avg 14% 2.9% at its level
SimpleQA Verified 48.0% ▁▁▃▂▃▃▆▆●▄▂▅ beats 50/81 very hard · avg 8.0% 1.9% at its level
BFCL multi-turn (version not stat… 38.1% ▄▄···●·▄█▄▄▄ beats 2/8 hard · avg 26% 1.2% at its level
Vectara hallucination leaderboard… 87.9% ▁▂▁▂●█▂▃▄▃▁▁ beats 22/98 very easy · avg 90% 0.0% blunt
source results share of the score
Epoch AI 6 27% ▇▇▇
LMArena leaderboard dataset 5 21% ▇▇▇
Daekeun Kim Korean LLM benchmark runs 3 14% ▇▇
ARC Prize Foundation model_baseline out… 2 7% ▇
HyperCLOVA X 32B Think 3 7% ▇
OpenAI vendor-reported benchmark scores… 2 4% ▇
Multi-Step Tool-Calling over Korean Ope… 2 4% ▇
ContractBench: Can LLM Agents Preserve … 1 4% ▇
Medmarks: A Comprehensive Open-Source L… 1 4% ▇
EQ-Bench 3 canonical leaderboard results 1 3% ▇
WebApp1K models leaderboard 1 2% ▇
KlinikeBench: Evaluating Language Model… 1 1% ▇
Amazon AGI 1 1% ▇
Vectara hallucination leaderboard 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 69.6
Licences: Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Daekeun Kim Korean LLM benchmark runs ([MIT](../../licenses/korean-personal-benchmark-runs-MIT.txt)) · ARC Prize Foundation model\_baseline outputs on ARC-AGI-1/2 public evaluation sets ([MIT](../../licenses/arcprize-model-baseline-outputs-MIT.txt)) · HyperCLOVA X 32B Think ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Multi-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · ContractBench: Can LLM Agents Preserve Observation Contracts? ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks ([CC BY 4.0](../../licenses/paper-2605.01417v1-CC-BY-4.0.txt)) · EQ-Bench 3 canonical leaderboard results ([MIT](../../licenses/eqbench3-MIT.txt)) · WebApp1K models leaderboard ([MIT](../../licenses/webapp1k-leaderboard-MIT.txt)) · KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Amazon AGI ([MIT](../../licenses/tau2-verified-MIT.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
### Through an API
api (OpenAI) · $3.44/M blended · $1.25 in / $10 out per 1M tokens · [platform.openai.com](https://platform.openai.com/docs/models)
[price source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/providers/openai/models/gpt-5.1.toml): Standard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/models/openai/gpt-5.1.toml)
[interactive version](https://whatmodel.sh/model/gpt-5-1) · [JSON](https://whatmodel.sh/api?format=json&all=1)