[whatmodel.sh](../../README.md) / Claude Opus 4 # Claude Opus 4 Anthropic · released 2025-05 · proprietary **score 66.5** ±0.8 · comparison only: no current run option verified · from 18 benchmark results across 10 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks Claude Opus 4 among every model on that task or test, worst to best. 
task                vs the others          rank  counts  tests
agent tasks         ▁▁▂▆█▄●▄▄▃▁▁    #140 of 354     31%      4
coding              ▁▁▁▃▄▆█▇●▄▃▁    #118 of 539     42%      4
chat & writing      ▁▁▁▃▅▄▇█▇●▇▃   #110 of 474*    7.3%      1
math                ▁▁▃▆██●█▃▁▁▁   #198 of 530*    8.4%      2
vision & documents  ▁▁▂▃▂▃▅▄●██▄   #102 of 189*    4.8%      1
web search          ▂▁▂●▅█▅▄▁▂▁▁     #68 of 96*    2.6%      1
agent tasks · #140 of 354 · 31% of its score Beats [o3](o3.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Large](jamba-1-5-large.md), [Devstral 2 (2512)](devstral-2.md) here, though they rank lower overall. 
test                     result  vs the others in its run  difficulty            counts  why
SWE-bench Verified        70.7%  ▁··▁·▁▂▁●█▄▁ beats 8/29   hard · avg 19%           17%  sharp · at its level
τ²-bench Telecom          57.0%  ██··█·██··●█ beats 5/6    very hard · avg 8.9%    7.2%  sharp · at its level
τ-bench Retail (maker)    81.4%  ▁····▃·▂▇█▂● beats 27/28  medium · avg 45%        4.0%  small run
τ-bench Airline (maker)   59.6%  ▁▁▁·▅▄▁█▇▁▁● beats 24/28  hard · avg 26%          2.8%  at its level · small run
coding · #118 of 539 · 42% of its score Beats [Grok 4.20 (0309 reasoning)](grok-4-20.md), [Grok 4.3](grok-4-3.md), [Muse Glimmer 30B](muse-glimmer-30b.md) here, though they rank higher overall. Trails [Llama 3.1 Nemotron Ultra 253B v1](llama-3-1-nemotron-ultra-253b.md), [Nemotron Cascade 2 30B-A3B](nemotron-cascade-2-30b-a3b.md), [MiniMax-M2.1](minimax-m2-1.md) here, though they rank lower overall. 
test                                result  vs the others in its run  difficulty            counts  why
SWE-bench Verified                   70.7%  ▁··▁·▁▂▁●█▄▁ beats 8/29   hard · avg 19%           17%  sharp · at its level
Aider polyglot (225 exercises, tw…   72.0%  ▂·▂▃▅▂▃█▆●▂▂ beats 40/49  very hard · avg 7.6%     17%  sharp · at its level
LiveCodeBench official (Aug 1, 20…   56.6%  ▃▃▅▃··▃●█▅▅▃ beats 6/15   hard · avg 24%          4.4%  sharp · at its level
WebApp1K-Duo-React pass@1 (onekq-…   76.3%  ▁··▁·▁▁▁▆█▇● beats 40/45  hard · avg 29%          3.7%  at its level
chat & writing · #110 of 474 · 7.3% of its score Beats [LongCat 2.0 (1.6T-A48B)](longcat-2-0.md), [Inkling Small (266B-A12B)](inkling-small.md), [GPT-5.4 nano](gpt-5-4-nano.md) here, though they rank higher overall. Trails [Gemma 4 12B](gemma-4-12b.md), [Kimi K2 Instruct (0905)](kimi-k2-0905.md), [Gemma 4 26B-A4B](gemma-4-26b-a4b.md) here, though they rank lower overall. 
test            result  vs the others in its run    difficulty  counts  why
LMArena (text)  1414.3  ▁▁▁▂▃▃▃▆▆●█▃ beats 262/385  avg 1260      7.3%  blunt
math · #198 of 530 · 8.4% of its score Trails [OpenThinker3 7B](openthinker3-7b.md), [Olmo 3 7B Think](olmo-3-7b-think.md), [OpenThinker2 32B](openthinker2-32b.md) here, though they rank lower overall. 
test                      result  vs the others in its run   difficulty            counts  why
MATH level 5               85.0%  ▂▂▅▆▄█▆▃●▂▃▄ beats 72/92   hard · avg 28%          4.3%  sharp · maxed out
OTIS mock AIME 2024–2025   64.4%  ▃▃▄▃▃▃●▇█▃▂▅ beats 92/197  very hard · avg 7.2%    4.1%  sharp · noisy
vision & documents · #102 of 189 · 4.8% of its score Beats [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md), [GPT-5 mini](gpt-5-mini.md) here, though they rank higher overall. Trails [Qwen3.5 4B](qwen3-5-4b.md), [GPT-4.1 mini](gpt-4-1-mini.md), [Mistral Large 3 (675B-A41B)](mistral-large-3.md) here, though they rank lower overall. 
test                             result  vs the others in its run   difficulty  counts  why
Arena vision (style controlled)  1188.9  ▁▁▂▂▂▂▃▄●▅█▇ beats 68/152  avg 1061      4.8%  blunt
web search · #68 of 96 · 2.6% of its score Beats [Kimi K2.6](kimi-k2-6.md), [GPT-5 Pro](gpt-5-pro.md), [DeepSeek V4 Flash (Preview, Apr 2026)](deepseek-v4-flash-preview.md) here, though they rank higher overall. Trails [GPT-4o mini (2024-07-18)](gpt-4o-mini-2024-07-18.md), [Mistral Small 3.1 24B (2503)](mistral-small-3-1.md), [Llama 4 Scout (109B-A17B)](llama-4-scout.md) here, though they rank lower overall. 
test                                result  vs the others in its run  difficulty  counts  why
Arena search systems (style contr…  1151.8  ▁·▁▁●▂▂█▆▃▂▂ beats 5/36   avg 1069      2.6%  blunt
other tests (6) 
test                                result  vs the others in its run    difficulty           counts  why
GPQA Diamond                         76.3%  ▁▁▁▁▂▄▆▄●▇█▅ beats 117/211  hard · avg 39%          10%  sharp · at its level
EQ-Bench 3 (emotional intelligenc…  1260.4  ▁·▂▁▄▅▅▃●▄▁▁ beats 37/62    avg 1034               4.4%  typical
AraGen-v3 3C3H score (Inception, …   81.0%  ▁▁·▁▂▂▄█▄▅▄● beats 63/66    hard · avg 29%         3.7%  at its level
HindiGen-v1 3C3H score (Inception…   74.5%  ▁▁▁▁·▂·█▆█●▁ beats 27/31    medium · avg 46%       2.6%  typical
Vectara hallucination leaderboard…   95.2%  ▁▁▂▃▄▄●▆▃▂▂▁ beats 65/140   very easy · avg 94%    0.2%  blunt · maxed out · noisy
Vectara hallucination leaderboard…   88.0%  ▁▂▁▂●█▂▃▄▃▁▁ beats 26/98    very easy · avg 90%    0.0%  blunt
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
Epoch AI                                        4   35% ▇▇▇▇
Aider polyglot benchmark runs                   1   17% ▇▇
LMArena leaderboard dataset                     3   15% ▇▇
Moonshot AI Kimi-K2-Instruct model card         1    7% ▇
τ-bench leaderboard: Anthropic's own su…        2    7% ▇
Inception Arabic/Hindi leaderboards             2    6% ▇
EQ-Bench 3 canonical leaderboard results        1    4% ▇
LiveCodeBench official website results          1    4% ▇
WebApp1K models leaderboard                     1    4% ▇
Vectara hallucination leaderboard               2    0% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 66.5
Licences: Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Aider polyglot benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Moonshot AI Kimi-K2-Instruct model card ([Modified MIT License (Kimi K2)](https://huggingface.co/moonshotai/Kimi-K2-Instruct/blob/fd1984e2b7a3350dbf7305fe73a4ede25c14de50/LICENSE)) · τ-bench leaderboard: Anthropic's own submissions ([MIT](https://opensource.org/license/mit)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · EQ-Bench 3 canonical leaderboard results ([MIT](../../licenses/eqbench3-MIT.txt)) · LiveCodeBench official website results ([CC-BY-SA-4.0](../../licenses/lcb-sharealike-NOTICE.txt)) · WebApp1K models leaderboard ([MIT](../../licenses/webapp1k-leaderboard-MIT.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/claude-opus-4-20250514) · [JSON](https://whatmodel.sh/api?format=json&all=1)