[whatmodel.sh](../../README.md) / Claude 3.7 Sonnet
# Claude 3.7 Sonnet
Anthropic · released 2025-02 · proprietary
**score 64.8** ±0.7 · comparison only: no current run option verified · from 35 benchmark results across 22 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Claude 3.7 Sonnet among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▂▆●▄▅▄▄▃▁▁ #208 of 354 28% 9
coding ▁▁▁▃▄▆█▇●▄▃▁ #142 of 539 27% 4
chat & writing ▁▁▁▃▅▄▇█●▇▇▃ #158 of 474 13% 4
math ▁▁▃▆██●█▃▁▁▁ #175 of 530 10% 5
vision & documents ▁▁▂▃▂▃▅●▆██▄ #107 of 189 6.4% 4
web search ▂▁▂▆▅█▅▄▁▂▁● #1 of 96* 3.5% 1
agent tasks · #208 of 354 · 28% of its score
Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md), [DeepSeek R1 0528](deepseek-r1-0528.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Jamba 1.5 Large](jamba-1-5-large.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 61.0% ▁··▁·▁●▁▄█▄▁ beats 4/29 hard · avg 19% 11% sharp · at its level
Finance Agent Benchmark v1 (537 q… 45.9% █·▃▃·█▅▅▅▅▃● beats 17/18 very hard · avg 5.5% 3.5% sharp · at its level
BrowserGym WebArena (AgentLab Gen… 44.6% █··▄·▄█·▄··● beats 7/7 hard · avg 17% 3.1% at its level
τ²-bench Telecom 49.0% ▁·▅█●▅▂▃▃▂▃▁ beats 43/89 very hard · avg 13% 3.0% sharp · at its level
τ²-bench Retail 72.1% ▁····▂▁▃▆●▃▁ beats 32/44 medium · avg 40% 2.6% at its level
τ²-bench Airline 64.2% ▁▁·▂▁▃▂▁█▅●▃ beats 37/42 hard · avg 28% 1.9% at its level
Multi-SWE-bench, 7 languages (Mag… 5.1% ▄···▄▄·▄●·▄█ beats 4/7 very hard · avg 2.6% 1.1% blunt · too hard for it
BFCL v3 58.6% ▁▁▁▁▁▃▅█●▄▁▁ beats 77/114 medium · avg 51% 0.9% blunt · at its level
Galileo Agent Leaderboard v1 (too… 95.3% ▃▃▃▃▆▅▆▅█▂▅● beats 31/31 easy · avg 78% 0.5% typical
coding · #142 of 539 · 27% of its score
Beats [Qwen3.5 Plus](qwen3-5-plus.md), [Qwen3-Coder-Next (80B-A3B)](qwen3-coder-next.md), [Gemini 2.5 Pro](gemini-2-5-pro.md) here, though they rank higher overall. Trails [Llama 3.1 Nemotron Ultra 253B v1](llama-3-1-nemotron-ultra-253b.md), [Solar Pro 4](solar-pro-4.md), [Nemotron 3 Super 120B-A12B](nemotron-3-super-120b-a12b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Aider polyglot (225 exercises, tw… 64.9% ▂·▂▃▅▂▃█●▄▂▂ beats 38/49 very hard · avg 7.6% 12% sharp · at its level
SWE-bench Verified 61.0% ▁··▁·▁●▁▄█▄▁ beats 4/29 hard · avg 19% 11% sharp · at its level
WebApp1K-Duo-React pass@1 (onekq-… 66.6% ▁··▁·▁▁▁▆●▇█ beats 23/45 hard · avg 29% 2.5% at its level
Multi-SWE-bench, 7 languages (Mag… 5.1% ▄···▄▄·▄●·▄█ beats 4/7 very hard · avg 2.6% 1.1% blunt · too hard for it
chat & writing · #158 of 474 · 13% of its score
Beats [LongCat 2.0 (1.6T-A48B)](longcat-2-0.md), [o3-mini](o3-mini.md) here, though they rank higher overall. Trails [LFM2.5 8B-A1B](lfm2-5-8b-a1b.md), [LFM2.5 2.6B](lfm2-5-2-6b.md), [Nemotron-Cascade-8B](nemotron-cascade-8b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena-Hard v2.0 hard prompts (sty… 59.8% ▂·▃▅▅·▃█●▂▂▂ beats 18/22 very hard · avg 12% 5.4% sharp · at its level
LMArena (text) 1372.5 ▁▁▁▂▃▃▃▆●▇█▃ beats 213/385 avg 1260 4.7% blunt
IFEval prompt-level strict 87.3% ▄▄··▄·▄··█·● beats 6/6 easy · avg 73% 1.6% typical
Arena-Hard v2.0 creative writing … 63.9% ▆▂▂▆▂▂▃●▂▃·▂ beats 18/22 hard · avg 19% 1.1% sharp · at its level · noisy
math · #175 of 530 · 10% of its score
Beats [MiMo-V2.5 (310B-A15B)](mimo-v2-5.md), [MiniMax-M2.7](minimax-m2-7.md) here, though they rank higher overall. Trails [Olmo 3 7B Think](olmo-3-7b-think.md), [Qwen3-VL 4B Thinking](qwen3-vl-4b-thinking.md), [Nemotron 3 Nano 30B-A3B](nemotron-3-nano-30b-a3b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
MATH level 5 91.2% ▂▂▅▆▄█▆▃▅●▃▄ beats 79/92 hard · avg 28% 3.8% sharp
OTIS mock AIME 2024–2025 57.8% ▃▃▄▃▃▃●▇█▃▂▅ beats 87/197 very hard · avg 7.2% 3.2% sharp · at its level · noisy
AIME 2025 53.0% ●··▄··█··▄·▄ beats 1/6 hard · avg 32% 1.8% sharp · noisy
AIME 2024 55.3% ●···▄█·▄▄··▄ beats 0/6 medium · avg 43% 1.1% sharp · noisy
THINK-Bench final-answer accuracy… 94.3% ▄█·▄····▄▄·● beats 6/7 easy · avg 76% 0.5% typical
vision & documents · #107 of 189 · 6.4% of its score
Beats [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md) here, though they rank higher overall. Trails [Qwen3.5 4B](qwen3-5-4b.md), [GPT-4.1 mini](gpt-4-1-mini.md), [Mistral Large 3 (675B-A41B)](mistral-large-3.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena vision (style controlled) 1175.1 ▁▁▂▂▂▂▃▄●▅█▇ beats 60/152 avg 1061 3.1% blunt
CharXiv (reasoning questions) 70.9% ▄·····▄▄·▄·● beats 4/5 hard · avg 31% 1.6% at its level
CharXiv descriptive questions 93.3% ▄····▄···█▄● beats 5/5 easy · avg 67% 0.8% noisy
MSEarth-Bench Earth-science figur… 58.0% █·██····█●·█ beats 4/5 hard · avg 48% 0.8% blunt · at its level
web search · #1 of 96 · 3.5% of its score
Beats [GPT-6 Astra](gpt-6-astra.md), [Claude Opus 5.5](claude-opus-5-5.md), [GPT-6.1 Sol](gpt-6-1-sol.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
Finance Agent Benchmark v1 (537 q… 45.9% █·▃▃·█▅▅▅▅▃● beats 17/18 very hard · avg 5.5% 3.5% sharp · at its level
other tests (10)
test result vs the others in its run difficulty counts why
GPQA Diamond 79.7% ▁▁▁▁▂▄▆▄▅●█▅ beats 127/211 hard · avg 39% 13% sharp · at its level · several runs
KCL-Essay vanilla 43.6% ▁···▁▃▂█▂●▃▂ beats 17/23 hard · avg 17% 3.6% at its level
EQ-Bench 3 (emotional intelligenc… 1169.2 ▁·▂▁▄▅●▃█▄▁▁ beats 25/62 avg 1034 2.8% typical
AraGen-v3 3C3H score (Inception, … 78.2% ▁▁·▁▂▂▄█▄▅▄● beats 60/66 hard · avg 29% 2.5% at its level
KCL-MCQA vanilla 50.4% ▂···█▂▆▆█●▆▂ beats 18/23 very hard · avg 29% 1.9% at its level
HindiGen-v1 3C3H score (Inception… 70.8% ▁▁▁▁·▂·█▆●▇▁ beats 20/31 medium · avg 46% 1.8% typical
DiagnosticIQ industrial maintenan… 70.6% ▁··▃▁▁▁▅█▅▁● beats 21/24 hard · avg 51% 1.0% blunt · at its level
Open PT-LLM Leaderboard average (… 84.5% ▃▁▁▁▁▁▁▁▂█▅● beats 274/276 medium · avg 72% 0.6% noisy
Omni-MATH 54.6% ●·█·█·██··██ beats 0/6 medium · avg 64% 0.2% blunt · noisy
Vectara hallucination leaderboard… 95.5% ▁▁▂▃▄▄●▆▃▂▂▁ beats 75/140 very easy · avg 94% 0.1% blunt · maxed out · noisy
source results share of the score
Epoch AI 4 25% ▇▇▇
Aider polyglot benchmark runs 1 12% ▇
Microsoft 4 9% ▇
LMArena leaderboard dataset 2 8% ▇
τ²/τ³-bench leaderboard: Sierra's own e… 3 7% ▇
Arena-Hard-Auto v2.0-Preview leaderboard 2 7% ▇
Korean Canonical Legal Benchmark: Towar… 2 5% ▇
Inception Arabic/Hindi leaderboards 2 4% ▇
Finance Agent Benchmark 1 3% ▇
ServiceNow BrowserGym leaderboard 1 3% ▇
EQ-Bench 3 canonical leaderboard results 1 3% ▇
WebApp1K models leaderboard 1 3% ▇
ServiceNow-AI/Apriel-1.6-15b-Thinker mo… 2 2% ▇
OpenCompass CompassAcademic leaderboard 1 2% ▇
Multi-SWE-bench experiments 1 1% ▇
DiagnosticIQ: A Benchmark for LLM-Based… 1 1% ▇
Berkeley Function Calling Leaderboard r… 1 1% ▇
MSEarth: A Multimodal Benchmark for Ear… 1 1% ▇
Open Portuguese LLM Leaderboard 1 1% ▇
THINK-Bench: Evaluating Thinking Effici… 1 1% ▇
Galileo Agent Leaderboard 1 0% ▇
Vectara hallucination leaderboard 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 64.8
Licences: Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Aider polyglot benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) · Microsoft ([CC BY 4.0](../../licenses/report-2504.21318v1.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · τ²/τ³-bench leaderboard: Sierra's own evaluation runs ([MIT](https://opensource.org/license/mit)) · Arena-Hard-Auto v2.0-Preview leaderboard ([Apache-2.0](../../licenses/arena-hard-auto-Apache-2.0.txt)) · Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities ([CC BY 4.0](../../licenses/cards-final-NOTICE.txt)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Finance Agent Benchmark ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · ServiceNow BrowserGym leaderboard ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · EQ-Bench 3 canonical leaderboard results ([MIT](../../licenses/eqbench3-MIT.txt)) · WebApp1K models leaderboard ([MIT](../../licenses/webapp1k-leaderboard-MIT.txt)) · ServiceNow-AI/Apriel-1.6-15b-Thinker model card ([MIT](../../licenses/cards-final-NOTICE.txt)) · OpenCompass CompassAcademic leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Multi-SWE-bench experiments ([Apache-2.0](../../licenses/multi-swe-bench-experiments-Apache-2.0.txt)) · DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · MSEarth: A Multimodal Benchmark for Earth Science Phenomenon Discovery with MLLMs ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · THINK-Bench: Evaluating Thinking Efficiency and Chain-of-Thought Quality of Large Reasoning Models ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Galileo Agent Leaderboard ([Apache-2.0](../../licenses/galileo-agent-leaderboard-Apache-2.0.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
No verified API or local download yet.
[interactive version](https://whatmodel.sh/model/claude-3-7-sonnet-20250219) · [JSON](https://whatmodel.sh/api?format=json&all=1)