[whatmodel.sh](../../README.md) / GPT-5.6 Sol
# GPT-5.6 Sol
OpenAI · released 2026-07 · closed weights
**score 79.6** ±0.5 · #13 of 732 · from 55 benchmark results across 20 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks GPT-5.6 Sol among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█▄▅▅▄▄●▁ #15 of 313 27% 14
coding ▁▁▁▃▅▇█▇▅▄●▁ #27 of 469 11% 6
chat & writing ▁▁▁▃▆▄▆█▆▇▇● #17 of 402* 2.2% 1
math ▁▁▃▅█▆▄█▄▂●▁ #8 of 431 31% 7
vision & documents ▁▁▂▂▁▃▄▂▆▇█● #11 of 153 4.4% 6
web search ▁▁▁▅▄█▃●·▁·▁ #7 of 84* 2.2% 2
agent tasks · #15 of 313 · 27% of its score
Trails [Claude Mythos Preview (limited preview)](claude-mythos-preview.md), [Claude Fable 5](claude-fable-5.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Terminal-Bench 4.0 37.3% █▅▅▅▅·●▃▃▅▅▅ beats 12/20 very hard · avg 0.4% 5.9% sharp · at its level · small run · seve…
Arena Agent (October 2026 aggrega… 0.0479 IPS ▁▁▁▃▃▂█●▂▂▁▁ beats 45/59 avg -0.379 IPS 5.7% sharp
SWE-bench Pro (split unspecified)… 64.6% ▂·▁▁▁▁▂▃█●▂▁ beats 49/58 very hard · avg 4.9% 1.9% sharp · at its level
GameLogicBench solve rate (72 God… 41.7% █▄·█·▄·█●▄·▄ beats 9/11 very hard · avg 5.1% 1.7% at its level
τ³-bench banking_knowledge (v1.0.… 46.9% ▃▃▂▃·▆▃▂█▅●▂ beats 22/25 very hard · avg 4.3% 1.7% at its level
Terminal-Bench 2.1 (maker) 88.8% ▁·▁·▁▁▄▃▆█▅● beats 60/63 hard · avg 23% 1.5% sharp
BrowseComp (maker) 90.4% ▄··█▄··▄·▄●▄ beats 7/8 medium · avg 38% 1.4% typical
Toolathlon (Tool Decathlon) (make… 58.0% ▃··▃··▃··▃█● beats 8/8 very hard · avg 13% 1.4% at its level
E2E-SWE (build a working codebase… 61.2% ▃··▅··▃█▃▅●▅ beats 10/12 very hard · avg 3.7% 1.4% sharp · at its level
DeepSWE v1.1 (resolved) (maker) 72.7% █··▃▃····●·▃ beats 5/6 hard · avg 22% 1.1% at its level
Video2World V2WScore (coding agen… 31.0% █··▄●·▄····▄ beats 4/6 very hard · avg 7.3% 1.0% at its level
Thinkingbox-bench pass@1 (507 sta… 61.9% ▅··▃·▃·█▃▅█● beats 13/15 very hard · avg 5.1% 1.0% sharp · at its level
KlinikeBench strict pass@1 (333 i… 23.7% ▁▁·▁▁▁▁▅▃█▅● beats 24/27 very hard · avg 2.4% 0.8% typical
EEGAgentBench overall (tool-based… 63.1% ▂·▁▁··▇▇█▂·● beats 25/25 hard · avg 28% 0.7% blunt · at its level
coding · #27 of 469 · 11% of its score
Trails [Grok 4.7](grok-4-7.md), [Qwen3.8-Flash-Next](qwen3-8-flash-next.md), [Qwen3.8 Max (0902)](qwen3-8-max-0902.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena WebDev (Code Arena) 1615.8 ▁▁▃▂▅█▃▆●▅▁▂ beats 102/129 avg 981 4.0% sharp
SWE-bench Pro (split unspecified)… 64.6% ▂·▁▁▁▁▂▃█●▂▁ beats 49/58 very hard · avg 4.9% 1.9% sharp · at its level
GameLogicBench solve rate (72 God… 41.7% █▄·█·▄·█●▄·▄ beats 9/11 very hard · avg 5.1% 1.7% at its level
E2E-SWE (build a working codebase… 61.2% ▃··▅··▃█▃▅●▅ beats 10/12 very hard · avg 3.7% 1.4% sharp · at its level
Luau Leetcode fully solved (Torpe… 84.6% ▁▁▁▁·▁▂▃▅▃█● beats 43/49 hard · avg 17% 1.1% sharp
DeepSWE v1.1 (resolved) (maker) 72.7% █··▃▃····●·▃ beats 5/6 hard · avg 22% 1.1% at its level
chat & writing · #17 of 402 · 2.2% of its score
Beats [GPT-6 Astra](gpt-6-astra.md), [GPT-6.1 Sol](gpt-6-1-sol.md), [Claude Sonnet 5.5](claude-sonnet-5-5.md) here, though they rank higher overall. Trails [Muse Spark 1.2](muse-spark-1-2.md), [Muse Spark 1.1](muse-spark-1-1.md), [Claude Opus 4.6](claude-opus-4-6.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LMArena (text) 1484.9 ▁▁▁▂▃▃▃▆▆▇█● beats 367/385 avg 1260 2.2% blunt
math · #8 of 431 · 31% of its score
Beats [Claude Sonnet 5.5](claude-sonnet-5-5.md), [Claude Opus 5](claude-opus-5.md), [GPT-5.5 Pro](gpt-5-5-pro.md) here, though they rank higher overall. Trails [Claude Fable 5](claude-fable-5.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
FrontierMath tier 4 v2 80.5% ▁▂▁▂█▅▂●▂▁▁▁ beats 60/67 very hard · avg 0.1% 15% sharp · at its level · several runs
FrontierMath tiers 1–3 v2 89.1% ▂··▁▁▃▄▄█▆▄● beats 80/86 very hard · avg 1.2% 9.1% sharp · several runs
MathArena ArXivMath June 2026 (48… 88.5% ▂···▄·██▆▆●▂ beats 17/19 very hard · avg 5.1% 3.4% sharp
MathArena ArXivLean June 2026 (46… 39.1% ▄····▄▄██●·▄ beats 8/9 very hard · avg 0.9% 1.7% sharp · at its level
MathArena BrokenArXiv June 2026 (… 67.3% ▄··█▂▂▂▆●··▂ beats 12/14 very hard · avg 1.3% 1.2% sharp · at its level · noisy
FrontierMath: Erdős problems 0.0% ●··········▆ beats 1/6 very hard · avg 0.0% 0.3% sharp · too hard for it · noisy
OTIS mock AIME 2024–2025 100.0% ▃▃▄▃▃▃▇▇█▃▂● beats 195/197 very hard · avg 7.2% 0.1% sharp · maxed out · noisy
vision & documents · #11 of 153 · 4.4% of its score
Beats [GPT-6 Astra](gpt-6-astra.md), [Claude Fable 5.1](claude-fable-5-1.md), [Claude Sonnet 5.5](claude-sonnet-5-5.md) here, though they rank higher overall. Trails [Muse Spark 1.2](muse-spark-1-2.md), [Claude Opus 4.7](claude-opus-4-7.md), [Gemini 3.7 Flash](gemini-3-7-flash.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena vision (style controlled) 1286.2 ▁▁▂▂▂▂▃▄▅▅█● beats 138/152 avg 1061 1.5% blunt
MMMU Pro (maker) 83.0% ▅·····▃·▃·█● beats 7/7 hard · avg 40% 1.2% typical
Arena document (style controlled) 1488.7 ▁▁▃▃▄▃▇█▅▇●▅ beats 32/37 avg 1331 0.8% blunt
SciDocBench scientific-document w… 61.0% ▃▃···█▃▅▃·▃● beats 10/11 hard · avg 25% 0.7% blunt · at its level
ScreenSpot-Pro (maker) 76.9% ▂▃▅▆██▅●···▂ beats 23/24 medium · avg 49% 0.2% blunt · small run
PureDocBench document parsing (Av… 69.2% ▂·▁·▂▂▅●·▁▂▂ beats 12/22 medium · avg 62% 0.0% blunt · at its level
web search · #7 of 84 · 2.2% of its score
Beats [GPT-6 Astra](gpt-6-astra.md), [Claude Opus 5.5](claude-opus-5-5.md), [GPT-6.1 Sol](gpt-6-1-sol.md) here, though they rank higher overall. Trails [GPT-4o (2024-08-06)](gpt-4o-2024-08-06.md), [GPT-4.1 mini](gpt-4-1-mini.md), [GPT-4.1](gpt-4-1.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
BrowseComp (maker) 90.4% ▄··█▄··▄·▄●▄ beats 7/8 medium · avg 38% 1.4% typical
Arena search systems (style contr… 1257.9 ▁·▁▁▂▂▂█▆▃▂● beats 36/36 avg 1069 0.8% blunt
other tests (21)
test result vs the others in its run difficulty counts why
EBR-bench (Epoch) 44.8% ▄▂·▆█▂█▆●▆·▆ beats 18/25 very hard · avg 0.3% 5.7% sharp · at its level
Mystery Game Puzzles (Epoch) 58.0% █··▂▃▇▄█▂●▂▁ beats 70/77 very hard · avg 9.4% 4.9% sharp · at its level · noisy
Furniture Assembly (Epoch) 56.7% █▆▆▆▆▂●▂▂▂▂▅ beats 24/32 very hard · avg 1.2% 4.1% sharp · at its level
Chess Puzzles (Epoch) 64.0% █▁▁▁▁▂▃▃▃▂●▁ beats 136/137 very hard · avg 5.8% 2.5% sharp · at its level · noisy
KMMLU-HARD (Daekeun Kim, zero-sho… 85.0% █···▁▁▁▂▃·▆● beats 22/23 very hard · avg 29% 1.7% sharp
KoBALT-700 (Daekeun Kim) 84.3% ▄▄█·····●··▄ beats 4/5 hard · avg 39% 0.9% typical
CLIcK Korean (Daekeun Kim) 95.5% ▆▄▄▄█▄▂·▄█·● beats 22/23 medium · avg 56% 0.9% maxed out
SimpleQA Verified 69.7% ▁▁▃▂▃▃▆▆█▄▂● beats 74/81 very hard · avg 8.0% 0.8% at its level
HealthBench Professional (maker) 60.5% ▄·▄··▄▄·●█·█ beats 4/8 hard · avg 20% 0.8% blunt · at its level
Icelandic LLM Leaderboard average… 65.3% ▁▁▃▂▂▄▁▃▄▄▄● beats 31/38 very hard · avg 14% 0.8% at its level
GPQA Diamond 93.5% ▁▁▁▁▂▄▆▄▅▇█● beats 200/211 hard · avg 39% 0.8% sharp · maxed out
MirrorCode (Epoch) 20.0% ▄·█▄●▄██·▄▄█ beats 4/13 very hard · avg 1.6% 0.7% sharp · at its level · noisy
AutomationBench public set (Zapie… 45.8% ▄······●█··▄ beats 1/4 very hard · avg 11% 0.7% at its level
KMMLU-Pro (Daekeun Kim) 93.6% ▄·▄█·····●·▄ beats 4/5 medium · avg 43% 0.7% typical
HAE-RAE 1.0 (Daekeun Kim) 94.6% ▄▄▄▂▂█▂▄▂█▂● beats 21/22 medium · avg 61% 0.6% maxed out
HealthBench Hard (maker) 31.1% ▂··▂····▂●·█ beats 3/8 very hard · avg 9.6% 0.5% blunt · at its level
MuSR Korean (Daekeun Kim) 85.3% █·····███·●█ beats 4/5 medium · avg 45% 0.5% typical
HealthBench (maker) 55.6% ▃·▃·····●▃▃▅ beats 3/8 hard · avg 24% 0.5% blunt · at its level
RobloxQA v2.0 multiple choice (To… 97.9% ▁▁▃▁█▄▃▃▆▂●▂ beats 45/49 easy · avg 77% 0.4% maxed out
NTCIR SciClaimEval Subtask 1 pair… 70.2% ▃▃···▃·▃▃●·▃ beats 7/8 medium · avg 45% 0.3% blunt · at its level
Vectara hallucination leaderboard… 87.6% ▁▂▁▂●█▂▃▄▃▁▁ beats 19/98 very easy · avg 90% 0.0% blunt
source results share of the score
Epoch AI 11 33% ▇▇▇▇
OpenAI vendor-reported benchmark scores… 13 25% ▇▇▇
LMArena leaderboard dataset 6 15% ▇▇
MathArena ArXivMath / BrokenArXiv / ArX… 3 6% ▇
Daekeun Kim Korean LLM benchmark runs 6 5% ▇
Terminal-Bench 4.0 official leaderboard… 1 3% ▇
GameLogicBench: Evaluating Coding Agent… 1 2% ▇
τ²/τ³-bench leaderboard: Sierra's own e… 1 2% ▇
Roblox LLM Leaderboard 2 1% ▇
E2E-SWE: Benchmarking LLMs on Building … 1 1% ▇
Video2World: Benchmarking Coding Agents… 1 1% ▇
One Success Isn't Reliability: Thinking… 1 1% ▇
KlinikeBench: Evaluating Language Model… 1 1% ▇
Miðeind Icelandic LLM Leaderboard 1 1% ▇
SciDocBench: A Workflow-Centered Benchm… 1 1% ▇
Zapier AutomationBench README: public-s… 1 1% ▇
EEGAgentBench: Benchmarking LLM Agents … 1 1% ▇
SciTrue: Reliable Scientific Claim Vali… 1 0% ▇
How Far Is Document Parsing from Solved… 1 0% ▇
Vectara hallucination leaderboard 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 79.6
Licences: Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets ([CC BY-SA 4.0](../../licenses/matharena-sharealike-NOTICE.txt)) · Daekeun Kim Korean LLM benchmark runs ([MIT](../../licenses/korean-personal-benchmark-runs-MIT.txt)) · Terminal-Bench 4.0 official leaderboard submissions ([Apache-2.0](../../licenses/terminal-bench-leaderboard-submissions-Apache-2.0.txt)) · GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · τ²/τ³-bench leaderboard: Sierra's own evaluation runs ([MIT](https://opensource.org/license/mit)) · Roblox LLM Leaderboard ([MIT](../../licenses/roblox-llm-leaderboard-MIT.txt)) · E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos ([CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/)) · One Success Isn't Reliability: Thinkingbox ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Miðeind Icelandic LLM Leaderboard ([CC BY 4.0](../../licenses/mideind-icelandic-llm-leaderboard-CC-BY-4.0.txt)) · SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Zapier AutomationBench README: public-set pass rates ([MIT](../../licenses/zapier-automationbench-readme-MIT.txt)) · EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · SciTrue: Reliable Scientific Claim Validation with Frontier and Open Language Models at the NTCIR SciClaimEval Task ([CC BY 4.0](../../licenses/paper-2609.00654v1-CC-BY-4.0.txt)) · How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
### Through an API
api (OpenAI) · $8.00/M blended · $4 in / $20 out per 1M tokens · [platform.openai.com](https://platform.openai.com/docs/models)
[price source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/providers/openai/models/gpt-5.6-sol.toml): Standard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/models/openai/gpt-5.6-sol.toml)
[interactive version](https://whatmodel.sh/model/gpt-5-6-sol) · [JSON](https://whatmodel.sh/api?format=json&all=1)