[whatmodel.sh](../../README.md) / Claude Opus 4.8
# Claude Opus 4.8
Anthropic · released 2026-05 · closed weights
**score 76.4** ±0.4 · #24 of 732 · from 62 benchmark results across 17 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Claude Opus 4.8 among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█▄▅▅▄▄●▁ #17 of 313 29% 19
coding ▁▁▁▃▅▇█▇▅▄●▁ #39 of 469 15% 7
chat & writing ▁▁▁▃▆▄▆█▆▇▇● #23 of 402* 2.1% 2
math ▁▁▃▅█▆▄█▄●▁▁ #19 of 431 35% 16
vision & documents ▁▁▂▂▁▃▄▂▆▇●▄ #16 of 153 3.5% 4
web search ▁▁▁▅▄●▃▄·▁·▁ #29 of 84 3.9% 5
agent tasks · #17 of 313 · 29% of its score
Beats [Muse Wonder](muse-wonder.md), [GPT-5.5](gpt-5-5.md), [MiMo-V2.6-Pro](mimo-v2-6-pro.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
Arena Agent (October 2026 aggrega… 0.0664 IPS ▁▁▁▃▃▂█●▂▂▁▁ beats 47/59 avg -0.379 IPS 4.8% sharp
SWE-bench Pro (split unspecified)… 69.2% ▄█·●·▄██···▄ beats 3/9 very hard · avg 7.4% 4.5% sharp · at its level
SWE-bench Multilingual (maker) 84.4% ▃··▃·●▃▃█··▃ beats 3/9 hard · avg 31% 2.1% typical
Terminal-Bench 4.0 23.6% █▅▅▅●·▅▃▃▅▅▅ beats 9/20 very hard · avg 0.5% 2.0% sharp · at its level
SWE-bench Verified (maker) 88.6% ▁▁▁▁▁▁▂▄█▆●▁ beats 135/138 hard · avg 28% 1.9% sharp
τ³-bench banking_knowledge (v1.0.… 39.7% ▃▃▂▃·▆▃▂●▅▅▂ beats 17/25 very hard · avg 4.3% 1.4% at its level
Chartography with tools (maker) 75.0% ▃········●██ beats 1/7 very hard · avg 6.5% 1.3% sharp · at its level
Terminal-Bench 2.1 (maker) 82.7% ▁·▁·▁▁▄▃▆█●█ beats 46/63 hard · avg 19% 1.3% sharp
OSWorld-Verified (maker) 83.4% █▃·▂▅▂▃▂▅▃▆● beats 24/27 hard · avg 28% 1.2% typical
Humanity's Last Exam with tools (… 57.9% ▁······▁·●·█ beats 4/10 very hard · avg 14% 1.2% at its level
E2E-SWE (build a working codebase… 51.9% ▃··▅··▃█▃●▃▅ beats 9/12 very hard · avg 3.7% 1.2% sharp · at its level
Toolathlon (Tool Decathlon) (make… 59.9% █···█··█·█●█ beats 4/5 hard · avg 16% 1.0% at its level
DeepSWE v1.1 (resolved) (maker) 59.0% ●·····████·█ beats 0/5 hard · avg 19% 1.0% at its level
MCP-Atlas (public set) 77.8% █··█·██·███● beats 7/7 hard · avg 30% 0.9% typical
BrowseComp with context managemen… 84.3% ▄·▄▂·█▂·●▄·▂ beats 11/15 medium · avg 47% 0.8% typical
Legal Research Bench (agentic, al… 42.9% ▄█··▄▄▄██·▄● beats 12/12 very hard · avg 6.3% 0.6% at its level
KlinikeBench strict pass@1 (333 i… 16.8% ▁▁·▁▁▁▁▅▃●▅█ beats 17/27 very hard · avg 2.4% 0.6% typical
RWSearch 77.3% █●██··█····█ beats 1/5 medium · avg 45% 0.6% typical
Toolathlon-Verified (maker) 79.9% ▃·····█···●▃ beats 4/5 medium · avg 37% 0.5% blunt · at its level
coding · #39 of 469 · 15% of its score
Beats [GPT-5.5](gpt-5-5.md), [GPT-5.6 Terra](gpt-5-6-terra.md) here, though they rank higher overall. Trails [Qwen3.8 27B](qwen3-8-27b.md), [GLM-5.2](glm-5-2.md), [GLM-5.3-Flash](glm-5-3-flash.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Pro (split unspecified)… 69.2% ▄█·●·▄██···▄ beats 3/9 very hard · avg 7.4% 4.5% sharp · at its level
Arena WebDev (Code Arena) 1555.8 ▁▁▃▂▅█▃●▅▅▁▂ beats 89/129 avg 981 3.3% sharp
SWE-bench Multilingual (maker) 84.4% ▃··▃·●▃▃█··▃ beats 3/9 hard · avg 31% 2.1% typical
SWE-bench Verified (maker) 88.6% ▁▁▁▁▁▁▂▄█▆●▁ beats 135/138 hard · avg 28% 1.9% sharp
E2E-SWE (build a working codebase… 51.9% ▃··▅··▃█▃●▃▅ beats 9/12 very hard · avg 3.7% 1.2% sharp · at its level
Luau Leetcode fully solved (Torpe… 79.8% ▁▁▁▁·▁▂▃▅▃●▇ beats 33/49 hard · avg 17% 1.1% sharp
DeepSWE v1.1 (resolved) (maker) 59.0% ●·····████·█ beats 0/5 hard · avg 19% 1.0% at its level
chat & writing · #23 of 402 · 2.1% of its score
Beats [GPT-6 Astra](gpt-6-astra.md), [Claude Sonnet 5.5](claude-sonnet-5-5.md), [GPT-6 Sol](gpt-6-sol.md) here, though they rank higher overall. Trails [Gemini 3.6 Flash](gemini-3-6-flash.md), [Muse Spark 1.2](muse-spark-1-2.md), [Muse Spark 1.1](muse-spark-1-1.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LMArena (text) 1481.7 ▁▁▁▂▃▃▃▆▆▇█● beats 364/385 avg 1260 1.9% blunt
IFEval (aggregation unspecified) 86.0% ●·██·█···█·█ beats 0/5 easy · avg 72% 0.2% maxed out
math · #19 of 431 · 35% of its score
Beats [DeepSeek V4.1 Flash 1017](deepseek-v4-1-flash-1017.md), [Qwen3.8 Max (2.4T-A95B)](qwen3-8-2-4t-a95b.md) here, though they rank higher overall. Trails [GPT-5.6 Luna](gpt-5-6-luna.md), [GPT-6 Luna](gpt-6-luna.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
FrontierMath tier 4 v2 56.1% ▁▂▁▂█▅●▃▂▁▁▁ beats 51/67 very hard · avg 0.1% 8.4% sharp · at its level
FrontierMath tiers 1–3 v2 80.0% ▂··▁▁▃▄▄█▆●▃ beats 70/86 very hard · avg 1.2% 6.8% sharp
MathArena ArXivMath April 2026 (4… 62.5% ▁·····▅█●▄·▁ beats 14/20 very hard · avg 7.6% 3.6% sharp · at its level
MathArena ArXivMath June 2026 (48… 70.0% ▂···▄·██●▆▄▂ beats 11/19 very hard · avg 5.1% 3.6% sharp · at its level
MathArena ArXivMath February 2026… 60.2% ▁▁▁▁█▇▄▃▁●·▃ beats 21/24 very hard · avg 9.4% 2.2% at its level
MathArena ArXivMath March 2026 (3… 75.0% █▃▅·█▃▃·▅·●▃ beats 13/14 hard · avg 16% 1.6% at its level
MathArena BrokenArXiv April 2026 … 34.4% ▂▂▆█●▃▂▂▂··▂ beats 12/18 very hard · avg 0.8% 1.5% sharp · at its level · noisy
MathArena ArXivMath May 2026 (40 … 65.0% ▁··▁▄█●▁·▃·▁ beats 12/17 very hard · avg 11% 1.2% at its level
MathArena ArXivLean June 2026 (46… 23.9% ▄····▄▄█●█·▄ beats 6/9 very hard · avg 0.9% 1.2% sharp
MathArena BrokenArXiv June 2026 (… 41.7% ▄··█▂▂●▆▄··▂ beats 8/14 very hard · avg 1.3% 1.0% sharp · at its level · noisy
MathArena BrokenArXiv May 2026 (5… 23.0% ██·▄▄●····▄▄ beats 6/8 very hard · avg 1.9% 1.0% sharp · at its level
MathArena BrokenArXiv March 2026 … 35.7% █·▆█▂·●▂···▂ beats 12/14 very hard · avg 1.0% 0.9% sharp · at its level · noisy
MathArena BrokenArXiv February 20… 35.5% ▃▂▂▃█▃▂·●··▂ beats 14/16 very hard · avg 0.9% 0.9% sharp · at its level · noisy
IMO-AnswerBench 75.3% ▁▁·▁▁▁▄●█▄▂▃ beats 16/44 hard · avg 25% 0.6% typical
USAMO 2026 (maker) 96.7% █·██······●█ beats 3/4 hard · avg 19% 0.4% sharp · noisy
OTIS mock AIME 2024–2025 98.3% ▃▃▄▃▃▃▇▇█▃●▅ beats 175/197 very hard · avg 7.2% 0.2% sharp · maxed out · noisy
vision & documents · #16 of 153 · 3.5% of its score
Beats [Claude Sonnet 5.5](claude-sonnet-5-5.md), [GPT-6 Sol](gpt-6-sol.md), [GPT-5.5](gpt-5-5.md) here, though they rank higher overall. Trails [Muse Spark 1.2](muse-spark-1-2.md), [Qwen3.8-Flash-Next](qwen3-8-flash-next.md), [Qwen3.8-Omni-Flash](qwen3-8-omni-flash.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena vision (style controlled) 1285.7 ▁▁▂▂▂▂▃▄▅▅█● beats 137/152 avg 1061 1.2% blunt
CharXiv (reasoning questions) (ma… 80.5% ███··█·●█··█ beats 4/6 hard · avg 28% 1.1% typical
Arena document (style controlled) 1485.9 ▁▁▃▃▄▃▇█▅▇●▅ beats 31/37 avg 1331 0.6% blunt
SciDocBench scientific-document w… 53.4% ▃▃···█▃●▃·▃▅ beats 6/11 hard · avg 25% 0.6% blunt · at its level
web search · #29 of 84 · 3.9% of its score
Trails [GPT-4o mini (2024-07-18)](gpt-4o-mini-2024-07-18.md), [Mistral Small 3.1 24B (2503)](mistral-small-3-1.md), [GPT-4o (2024-08-06)](gpt-4o-2024-08-06.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Humanity's Last Exam with tools (… 57.9% ▁······▁·●·█ beats 4/10 very hard · avg 14% 1.2% at its level
BrowseComp with context managemen… 84.3% ▄·▄▂·█▂·●▄·▂ beats 11/15 medium · avg 47% 0.8% typical
Arena search systems (style contr… 1195.2 ▁·▁▁▂▂▂●▆▃▂▂ beats 18/36 avg 1069 0.7% blunt
Legal Research Bench (agentic, al… 42.9% ▄█··▄▄▄██·▄● beats 12/12 very hard · avg 6.3% 0.6% at its level
RWSearch 77.3% █●██··█····█ beats 1/5 medium · avg 45% 0.6% typical
other tests (17)
test result vs the others in its run difficulty counts why
EBR-bench (Epoch) 28.6% ▄▂·▆█▂●▆▄▆·▆ beats 13/25 very hard · avg 0.3% 4.0% sharp · at its level
Mystery Game Puzzles (Epoch) 36.0% █··▂▃▇▄●▂▄▂▁ beats 58/77 very hard · avg 9.4% 3.5% sharp · at its level · noisy
Furniture Assembly (Epoch) 42.5% █▆▆▆●▂▆▂▂▂▂▅ beats 18/32 very hard · avg 1.2% 3.3% sharp · at its level
GPQA Diamond 91.0% ▁▁▁▁▂▄▆▄▅▇●▅ beats 184/211 hard · avg 39% 2.2% sharp · small run · several runs
Humanity's Last Exam (maker) 49.8% ▂·····▂·▄●█▄ beats 4/10 very hard · avg 5.9% 2.2% at its level
ARC-AGI-2 public evaluation set (… 76.9% █··▂▅▁▃▁▃·▁● beats 23/24 very hard · avg 0.3% 2.1% sharp · at its level · noisy
Chess Puzzles (Epoch) 34.0% █▁▁▁▁▂▃▃●▂▁▁ beats 111/137 very hard · avg 5.8% 1.9% sharp · at its level · noisy
Legal Research Bench (single-shot… 8.7% ▃▃█▃·▅···█▃● beats 12/12 very hard · avg 0.8% 0.9% too hard for it
AgentWorldBench (open-ended, over… 56.6% ▄▄··▄▄·▄█▄●█ beats 9/11 hard · avg 26% 0.9% blunt · at its level
SimpleQA Verified 53.0% ▁▁▃▂▃▃▆▆●▄▂▅ beats 60/81 very hard · avg 8.0% 0.7% at its level
Icelandic LLM Leaderboard average… 63.9% ▁▁▃▂▂▄▁▃▄▄●█ beats 29/38 very hard · avg 14% 0.7% at its level
FORTE (AGI-Eval) 77.2% ▄···█·····●█ beats 3/5 medium · avg 38% 0.6% typical
WritingBench 85.2% ▁·▁▂▁··▅●▂▂▁ beats 13/20 medium · avg 54% 0.6% typical
ARC-AGI-1 public evaluation set (… 95.5% ▁·▇▃▁·█▇▁·●▃ beats 24/27 very hard · avg 4.0% 0.5% sharp · noisy
HealthBench (maker) 58.8% ●·▄·····▄▄▄▄ beats 0/6 hard · avg 30% 0.5% blunt · at its level
RobloxQA v2.0 multiple choice (To… 96.7% ▁▁▃▁█▄▃▃●▂▂▂ beats 38/49 easy · avg 77% 0.4% maxed out
NTCIR SciClaimEval Subtask 1 pair… 67.6% ▃▃···▃·▃●█·▃ beats 4/8 medium · avg 45% 0.3% blunt · at its level
source results share of the score
Epoch AI 9 30% ▇▇▇▇
Anthropic system cards and model announ… 15 21% ▇▇▇
MathArena ArXivMath / BrokenArXiv / ArX… 11 19% ▇▇
LMArena leaderboard dataset 6 12% ▇
meituan-longcat/LongCat-2.0 model card 6 4% ▇
ARC Prize Foundation model_baseline out… 2 3% ▇
Terminal-Bench 4.0 official leaderboard… 1 2% ▇
Legal Research Bench 2 2% ▇
Roblox LLM Leaderboard 2 2% ▇
τ²/τ³-bench leaderboard: Sierra's own e… 1 1% ▇
E2E-SWE: Benchmarking LLMs on Building … 1 1% ▇
Qwen/Qwen-AgentWorld-35B-A3B model card 1 1% ▇
zai-org/GLM-5.2 model card 1 1% ▇
Miðeind Icelandic LLM Leaderboard 1 1% ▇
SciDocBench: A Workflow-Centered Benchm… 1 1% ▇
KlinikeBench: Evaluating Language Model… 1 1% ▇
SciTrue: Reliable Scientific Claim Vali… 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 76.4
Licences: Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets ([CC BY-SA 4.0](../../licenses/matharena-sharealike-NOTICE.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · meituan-longcat/LongCat-2.0 model card ([mit](../../licenses/meituan-longcat--LongCat-2.0.txt)) · ARC Prize Foundation model\_baseline outputs on ARC-AGI-1/2 public evaluation sets ([MIT](../../licenses/arcprize-model-baseline-outputs-MIT.txt)) · Terminal-Bench 4.0 official leaderboard submissions ([Apache-2.0](../../licenses/terminal-bench-leaderboard-submissions-Apache-2.0.txt)) · Legal Research Bench ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Roblox LLM Leaderboard ([MIT](../../licenses/roblox-llm-leaderboard-MIT.txt)) · τ²/τ³-bench leaderboard: Sierra's own evaluation runs ([MIT](https://opensource.org/license/mit)) · E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Qwen/Qwen-AgentWorld-35B-A3B model card ([Apache-2.0](https://huggingface.co/Qwen/Qwen-AgentWorld-35B-A3B/blob/60d2b0434a53d2e62a7c00a489586815d94ebffb/README.md)) · zai-org/GLM-5.2 model card ([mit](../../licenses/zai-org--GLM-5.2.txt)) · Miðeind Icelandic LLM Leaderboard ([CC BY 4.0](../../licenses/mideind-icelandic-llm-leaderboard-CC-BY-4.0.txt)) · SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · SciTrue: Reliable Scientific Claim Validation with Frontier and Open Language Models at the NTCIR SciClaimEval Task ([CC BY 4.0](../../licenses/paper-2609.00654v1-CC-BY-4.0.txt))
## How to run it
### Through an API
api (Anthropic) · $10.00/M blended · $5 in / $25 out per 1M tokens · [docs.anthropic.com](https://docs.anthropic.com/en/docs/about-claude/models)
[price source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/providers/anthropic/models/claude-opus-4-8.toml): Standard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/models/anthropic/claude-opus-4-8.toml)
[interactive version](https://whatmodel.sh/model/claude-opus-4-8) · [JSON](https://whatmodel.sh/api?format=json&all=1)