[whatmodel.sh](../../README.md) / Claude Sonnet 4.5
# Claude Sonnet 4.5
Anthropic · released 2025-09 · closed weights
**score 67.7** ±0.5 · #106 of 732 · from 51 benchmark results across 25 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Claude Sonnet 4.5 among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█▄▅●▄▄▂▁ #93 of 313 23% 14
coding ▁▁▁▃▅▇█▇●▄▃▁ #100 of 469 23% 5
chat & writing ▁▁▁▃▆▄▆█▆▇●▃ #51 of 402* 2.9% 1
math ▁▁▃▅█▆▄●▄▂▁▁ #109 of 431 21% 6
vision & documents ▁▁▂▂▁▃▄▂●▇█▄ #71 of 153 4.0% 6
web search ▁▁▁▅●█▃▄·▁·▁ #50 of 84* 1.1% 1
agent tasks · #93 of 313 · 23% of its score
Beats [Grok 4.20 (0309 reasoning)](grok-4-20.md), [GPT-5.1](gpt-5-1.md), [o3-pro](o3-pro.md) here, though they rank higher overall. Trails [Hy3 preview (295B-A21B)](hy3-preview.md), [MiniMax-M2.7](minimax-m2-7.md), [Step 3.5 Flash (196B-A11B)](step-3-5-flash.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 71.3% ▁··▁·▁▂▁●█▄▁ beats 9/29 hard · avg 19% 8.2% sharp · at its level · several runs
Terminal-Bench 2.0 with Terminus … 42.8% ▃··▃·▃▃██▅●▅ beats 12/14 very hard · avg 6.1% 2.4% sharp · at its level
ContractBench success rate (33 ob… 69.7% ▆··▆▄▂▄▂▆▆●▄ beats 19/23 very hard · avg 13% 2.4% sharp · at its level
τ²-bench Retail 69.8% █▄··▄··●·▄·▄ beats 4/6 medium · avg 36% 1.6% at its level
τ³-bench banking_knowledge (v1.0.… 25.3% ▃▃▂▃·●▃▂█▅▅▂ beats 9/25 very hard · avg 4.3% 1.5% typical
BFCL V4 (December 2025 archive) 73.2% ▁▁▁▂██▅▅▆▂▂● beats 76/77 hard · avg 31% 1.4% at its level
τ²-bench Airline 58.0% ▄▄······▄●▄▄ beats 3/6 hard · avg 28% 1.4% at its level
τ³-bench airline (tau2-bench ≥ v1… 72.0% ●······▃·▃█▃ beats 0/6 medium · avg 51% 1.3% typical
τ³-bench retail (tau2-bench ≥ v1.… 72.4% ●▄·█··▄·▄··▄ beats 0/6 medium · avg 48% 1.1% at its level
τ³-bench telecom (tau2-bench ≥ v1… 84.9% ●·▄█▄······▄ beats 0/6 medium · avg 64% 1.0% typical
KlinikeBench strict pass@1 (333 i… 9.9% ▁▁·▁▁▁▁▅●█▅█ beats 10/27 very hard · avg 2.4% 0.6% typical
τ²-Bench-Verified (GPT-5.1 user) 73.3% ▂··▄···▂●▄█▄ beats 5/13 medium · avg 45% 0.5% at its level
T1-Bench multi-domain conversatio… 33.7% ▂▂··▂·▂·▂▂●▄ beats 6/11 hard · avg 25% 0.0% blunt · at its level · noisy
τ²-bench Telecom (maker) 98.0% ▁·▅█▆▅▂▃▃▂●▁ beats 82/89 very hard · avg 8.8% — not counted yet: no other model in the …
coding · #100 of 469 · 23% of its score
Beats [Grok 4.20 (0309 reasoning)](grok-4-20.md), [Grok 4.3](grok-4-3.md), [Muse Glimmer 30B](muse-glimmer-30b.md) here, though they rank higher overall. Trails [Llama 3.1 Nemotron Ultra 253B v1](llama-3-1-nemotron-ultra-253b.md), [Qwen3 235B-A22B Thinking 2507](qwen3-235b-a22b-thinking-2507.md), [o4-mini](o4-mini.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 71.3% ▁··▁·▁▂▁●█▄▁ beats 9/29 hard · avg 19% 8.2% sharp · at its level · several runs
Aider polyglot (225 exercises, tw… 78.0% █▄····▄··█·● beats 6/6 very hard · avg 7.3% 6.3% sharp · at its level
Arena WebDev (Code Arena) 1384.9 ▁▁▃▂▅●▃▆▅▅▁▂ beats 39/129 avg 981 5.2% sharp
VisEditBench strict pass rate (1,… 62.7% ▄··█·▄··▄●·▄ beats 5/7 hard · avg 33% 1.5% at its level
WebApp1K-Duo-React pass@1 (onekq-… 78.4% ▁··▁·▁▁▁▆█▇● beats 44/45 hard · avg 29% 1.4% at its level
chat & writing · #51 of 402 · 2.9% of its score
Beats [MiMo-V2.6-Flash (309B-A15B)](mimo-v2-6-flash.md), [Claude Haiku 5.5](claude-haiku-5-5.md), [GPT-6 Luna](gpt-6-luna.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
LMArena (text) 1455.1 ▁▁▁▂▃▃▃▆▆▇●▃ beats 326/385 avg 1260 2.9% blunt
math · #109 of 431 · 21% of its score
Beats [Claude Sonnet 4.6](claude-sonnet-4-6.md), [MiniMax-M3](minimax-m3.md), [Nemotron 3 Ultra 550B-A55B](nemotron-3-ultra-550b-a55b.md) here, though they rank higher overall. Trails [Gemini 2.5 Flash](gemini-2-5-flash.md), [GLM-4.5-Air](glm-4-5-air.md), [Granite 4.2 30B](granite-4-2-30b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
FrontierMath tiers 1–3 v2 23.9% ▂··▁▁▃●▄█▆▄▃ beats 15/86 very hard · avg 1.2% 13% sharp · at its level
FrontierMath tier 4 v2 2.4% ▁●▁▂█▅▂▃▂▁▁▁ beats 6/67 very hard · avg 0.1% 4.1% sharp
MATH level 5 97.7% ▂▂▅▆▄█▆▃▅▂▃● beats 88/92 hard · avg 28% 1.4% sharp · maxed out
OTIS mock AIME 2024–2025 77.8% ▃▃▄▃▃▃▇●█▃▂▅ beats 112/197 very hard · avg 7.2% 1.4% sharp · noisy
MATH-Vision 71.1% ▁▁·▁··▄▁▃●▇█ beats 9/24 medium · avg 48% 0.8% typical
FormalProofBench (200 graduate-le… 18.0% ▃▅·▃··▅▃●··▃ beats 8/10 very hard · avg 3.1% 0.5% noisy
vision & documents · #71 of 153 · 4.0% of its score
Beats [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md), [GPT-5](gpt-5.md), [o3](o3.md) here, though they rank higher overall. Trails [Gemma 4 26B-A4B](gemma-4-26b-a4b.md), [Claude Haiku 4.5](claude-haiku-4-5.md), [Qwen3.6 Flash](qwen3-6-flash.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
VisEditBench strict pass rate (1,… 62.7% ▄··█·▄··▄●·▄ beats 5/7 hard · avg 33% 1.5% at its level
Arena document (style controlled) 1449.1 ▁▁▃▃▄●▇█▅▇▄▅ beats 10/37 avg 1331 1.0% blunt
MATH-Vision 71.1% ▁▁·▁··▄▁▃●▇█ beats 9/24 medium · avg 48% 0.8% typical
Molmo2-VideoCount accuracy (val) 27.2% ●··▃·▃····█▃ beats 0/6 hard · avg 22% 0.3% blunt · at its level
Ai2 Molmo2 human preference Elo (… 1008.0 ●··▄··▄··▄█▄ beats 0/6 avg 968 0.2% blunt
Molmo2-CapTest dense video captio… 26.0% ●···█·████·█ beats 0/6 hard · avg 31% 0.1% blunt · at its level
web search · #50 of 84 · 1.1% of its score
Beats [Gemini 3.1 Pro (Preview)](gemini-3-1-pro.md), [Qwen3.7 Max](qwen3-7-max.md), [GPT-5.2](gpt-5-2.md) here, though they rank higher overall. Trails [GPT-4o mini (2024-07-18)](gpt-4o-mini-2024-07-18.md), [Mistral Small 3.1 24B (2503)](mistral-small-3-1.md), [Llama 4 Scout (109B-A17B)](llama-4-scout.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena search systems (style contr… 1176.0 ▁·▁▁▂▂●█▆▃▂▂ beats 9/36 avg 1069 1.1% blunt
other tests (21)
test result vs the others in its run difficulty counts why
Medmarks-V mean win rate (19 veri… 0.6258 ▁▄▃▃▇▄█▃▅▁●▁ beats 29/30 avg 0.418 5.5% sharp
GPQA Diamond 82.3% ▁▁▁▁▂▄▆▄▅●█▅ beats 134/211 hard · avg 39% 3.8% sharp
WebCompass (overall) 57.1% ▅·▃▃▃···●·▃█ beats 5/9 hard · avg 18% 3.0% at its level
KCL-Essay vanilla 47.6% ▁···▁▃▂█▂▃●▂ beats 19/23 hard · avg 17% 2.3% at its level
Medmarks-V weighted mean win rate… 62.6% ▂▃▅▂▆▃█▅█▅▃● beats 33/34 medium · avg 41% 2.3% blunt · at its level
EBR-bench (Epoch) 2.4% ●▂·▆█▂█▆▄▆·▆ beats 0/25 very hard · avg 0.3% 2.2% sharp · too hard for it
ARC-AGI-1 public evaluation set (… 73.8% ▁·▇▃▁·█●▁·▇▃ beats 17/27 very hard · avg 4.0% 2.2% sharp · at its level · noisy
ARC-AGI-2 public evaluation set (… 14.7% █··▂▅▁●▁▃·▁▃ beats 15/24 very hard · avg 0.3% 1.8% sharp · noisy
EQ-Bench 3 (emotional intelligenc… 1293.1 ▁·▂▁▄▅▅▃●▄▁▁ beats 44/62 avg 1034 1.8% typical
Mystery Game Puzzles (Epoch) 17.0% █··▂▃●▄█▂▄▂▁ beats 26/77 very hard · avg 9.4% 1.7% sharp · too hard for it · noisy
Chess Puzzles (Epoch) 12.0% █▁▁▁▁●▃▃▃▂▁▁ beats 60/137 very hard · avg 5.8% 1.5% sharp · noisy
AraGen-v3 3C3H score (Inception, … 78.2% ▁▁·▁▂▂▄█▄▅▄● beats 61/66 hard · avg 29% 1.4% at its level
KCL-MCQA vanilla 54.9% ▂···█▂▆▆██●▂ beats 21/23 very hard · avg 29% 1.3% at its level
RPC-Bench overall, text input (Z.… 20.1% ●··▄▄·▄▄█·▄▄ beats 0/8 very hard · avg 12% 1.2% at its level
LongSocialBench (full-context, 4-… 51.4% ▆▂·▄█·▂▂·●▄▂ beats 12/16 very hard · avg 36% 1.1% blunt · at its level
SimpleQA Verified 30.7% ▁▁▃▂▃●▆▆█▄▂▅ beats 20/81 very hard · avg 8.0% 1.0% at its level
Multi-IF (aggregation unspecified) 84.3% ▄·····▄█●▄·▄ beats 4/6 easy · avg 66% 0.6% typical
SOOHAK Challenge (340 problems, A… 5.7% ●·█████··█·█ beats 0/7 very hard · avg 4.3% 0.5% typical
SOOHAK-Mini (702 problems, Avg@3) 40.9% ●··▄·▄·▄▄█·▄ beats 0/7 medium · avg 36% 0.4% blunt · at its level
Vectara hallucination leaderboard… 94.5% ▁▁▂▃▄●█▆▃▂▂▁ beats 55/140 very easy · avg 94% 0.1% blunt · maxed out · noisy
Vectara hallucination leaderboard… 88.0% ▁▂▁▂●█▂▃▄▃▁▁ beats 25/98 very easy · avg 90% 0.0% blunt
source results share of the score
Epoch AI 10 36% ▇▇▇▇
ServiceNow-AI/Apriel-1.6-15b-Thinker mo… 5 11% ▇
LMArena leaderboard dataset 4 10% ▇
Medmarks: A Comprehensive Open-Source L… 2 8% ▇
τ²/τ³-bench leaderboard: Sierra's own e… 4 5% ▇
ARC Prize Foundation model_baseline out… 2 4% ▇
Korean Canonical Legal Benchmark: Towar… 2 4% ▇
WebCompass: Towards Multimodal Web Codi… 1 3% ▇
Terminal-Bench: Benchmarking Agents on … 1 2% ▇
ContractBench: Can LLM Agents Preserve … 1 2% ▇
EQ-Bench 3 canonical leaderboard results 1 2% ▇
VisEditBench: Can Vision-Language Model… 1 2% ▇
Inception Arabic/Hindi leaderboards 1 1% ▇
WebApp1K models leaderboard 1 1% ▇
Berkeley Function Calling Leaderboard r… 1 1% ▇
RPC-Bench leaderboard 1 1% ▇
LongSocialBench: Do Long-Context LLMs U… 1 1% ▇
Qwen/Qwen3.5-27B model card 1 1% ▇
Soohak: A Mathematician-Curated Benchma… 2 1% ▇
Molmo2: Open Weights and Data for Visio… 3 1% ▇
KlinikeBench: Evaluating Language Model… 1 1% ▇
Amazon AGI 1 0% ▇
FormalProofBench 1 0% ▇
Vectara hallucination leaderboard 2 0% ▇
T1-Bench: Benchmarking Multi-Scenario A… 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 67.7
Licences: Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · ServiceNow-AI/Apriel-1.6-15b-Thinker model card ([MIT](../../licenses/cards-final-NOTICE.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks ([CC BY 4.0](../../licenses/paper-2605.01417v1-CC-BY-4.0.txt)) · τ²/τ³-bench leaderboard: Sierra's own evaluation runs ([MIT](https://opensource.org/license/mit)) · ARC Prize Foundation model\_baseline outputs on ARC-AGI-1/2 public evaluation sets ([MIT](../../licenses/arcprize-model-baseline-outputs-MIT.txt)) · Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities ([CC BY 4.0](../../licenses/cards-final-NOTICE.txt)) · WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models ([CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/)) · Terminal-Bench: Benchmarking Agents on Hard ([CC BY 4.0](../../licenses/paper-2601.11868v1-CC-BY-4.0.txt)) · ContractBench: Can LLM Agents Preserve Observation Contracts? ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · EQ-Bench 3 canonical leaderboard results ([MIT](../../licenses/eqbench3-MIT.txt)) · VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback? ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · WebApp1K models leaderboard ([MIT](../../licenses/webapp1k-leaderboard-MIT.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · RPC-Bench leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · LongSocialBench: Do Long-Context LLMs Understand Online Discussion Threads? ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Qwen/Qwen3.5-27B model card ([apache-2.0](../../licenses/Qwen--Qwen3.5-27B.txt)) · Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs ([CC BY-SA 4.0](../../licenses/paper-2605.09063v3-CC-BY-SA-4.0.txt)) · Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding ([CC BY 4.0](http://creativecommons.org/licenses/by/4.0/)) · KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Amazon AGI ([MIT](../../licenses/tau2-verified-MIT.txt)) · FormalProofBench ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/))
Not counted (3)
source test score why
τ-bench leaderboard: … τ²-bench Retail 86.2% only model in this source's run
τ-bench leaderboard: … τ²-bench Airline 70.0% only model in this source's run
τ-bench leaderboard: … τ²-bench Telecom 98.0% too few models have it yet