[whatmodel.sh](../../README.md) / Claude 2.0 # Claude 2.0 Anthropic · released 2023-07 · proprietary **score 47.2** ±0.6 · comparison only: no current run option verified · from 16 benchmark results across 8 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks Claude 2.0 among every model on that task or test, worst to best. 
task            vs the others         rank  counts  tests
coding          ▁▁▁▃▄●█▇▅▄▃▁   #378 of 539     51%      2
chat & writing  ▁▁▁▃▅●▇█▇▇▇▃   #357 of 474     30%      5
math            ▁▁▃●██▅█▃▁▁▁   #452 of 530    9.0%      3
coding · #378 of 539 · 51% of its score Beats [Solar Open 100B (102B-A12B)](solar-open-100b.md), [Qwen3 4B Instruct 2507](qwen3-4b-instruct-2507.md), [Qwen3-VL 8B Instruct](qwen3-vl-8b-instruct.md) here, though they rank higher overall. Trails [DeepSeek LLM 67B Chat](deepseek-llm-67b-chat.md), [WizardLM 70B V1.0](wizardlm-70b-v1-0.md), [Code Tulu 2 34B](code-tulu-2-34b.md) here, though they rank lower overall. 
test        result  vs the others in its run  difficulty      counts  why
HumanEval    69.5%  ▁·▁▁▃▄▆●█▃▁▁ beats 55/99  easy · avg 76%     29%  sharp · at its level · several runs
HumanEval+   61.6%  ▁·▁▁▁▃▄▄●█▃▁ beats 54/99  easy · avg 70%     22%  sharp · at its level
chat & writing · #357 of 474 · 30% of its score Beats [Ministral 3 14B Reasoning 2512](ministral-3-14b-reasoning.md), [Gemma 4 E4B](gemma-4-e4b.md), [OpenThinker3 7B](openthinker3-7b.md) here, though they rank higher overall. Trails [EXAONE 3.5 2.4B Instruct](exaone-3-5-2-4b.md), [LFM2 700M](lfm2-700m.md), [Flan-T5 XXL](flan-t5-xxl.md) here, though they rank lower overall. 
test                                                      result  vs the others in its run   difficulty                counts  why
AlpacaEval 2 length-controlled                             28.2%  ▁▂▂▃▆▆●▆▄▄▁▁ beats 75/122  medium · avg 35%             14%  sharp · at its level
Arena-Hard (v0.1, GPT-4-1106 judg…                         24.0%  ▂▁▁▅●▅▄▃▃▅▃▁ beats 22/60   medium · avg 47%            9.0%  sharp · at its level
Polyrating rating (bias-aware re-…                        1164.0  ▂▁▂▁▃▇█▆●▄▂▂ beats 73/95   avg 1187                    3.6%  typical
MT-Bench (GPT-4 single-answer gra…  8.0600 MT-Bench score (1-10)  ▆▂▂▆▆▄██▂●▄▆ beats 22/28   avg 8.539 MT-Bench scor…    2.8%  sharp · noisy
AlpacaEval 1.0 win rate (vs text-…                         91.4%  ▂▂·▆·▇▃▆●▆▄▂ beats 33/45   very easy · avg 94%         0.4%  sharp · noisy
math · #452 of 530 · 9.0% of its score Beats [GPT-3.5 Turbo (0125)](gpt-3-5-turbo-0125.md) here, though they rank higher overall. Trails [OPT-1.3B](opt-1-3b.md), [OPT-2.7B](opt-2-7b.md), [Gemma 2B (base)](gemma-2b-base.md) here, though they rank lower overall. 
test                      result  vs the others in its run    difficulty            counts  why
MATH level 5               11.7%  ▂▂●▆▄█▆▃▅▂▃▄ beats 10/92    hard · avg 28%          5.1%  sharp
GSM8K (maker)              88.0%  ▂▂▂▂▃▄▅▇██●▄ beats 171/206  very easy · avg 90%     3.4%  sharp
OTIS mock AIME 2024–2025    2.5%  ▃●▄▃▃▃▇▇█▃▂▅ beats 20/197   very hard · avg 7.2%    0.5%  sharp · too hard for it · noisy
other tests (5) 
test                                result  vs the others in its run    difficulty           counts  why
GPQA Diamond                         34.7%  ▁▁▁▁▂●▆▄▅▇█▅ beats 27/211   hard · avg 39%         3.0%  sharp
ARC Challenge (maker)                91.0%  ▄▁▁▂▁▄█▇▃▂▃● beats 138/140  very easy · avg 90%    2.7%  sharp · small run
MMLU (maker)                         78.5%  ▁▁▁▁▁▂▂▅██●▄ beats 163/214  easy · avg 81%         2.5%  at its level · small run
TriviaQA (maker)                     87.5%  ▁▁▂▂▄▄▃▆▅█▅● beats 80/80    easy · avg 85%         1.2%  small run
Vectara hallucination leaderboard…   82.6%  ▁▁●▃▄▄█▆▃▂▂▁ beats 7/140    very easy · avg 94%    0.3%  blunt · maxed out · noisy
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
EvalPlus / EvalPerf published evaluatio…        2   49% ▇▇▇▇▇▇
AlpacaEval 1.0 and 2.0 leaderboards             2   14% ▇▇
Anthropic system cards and model announ…        5   13% ▇▇
Arena-Hard-Auto v0.1 leaderboard                1    9% ▇
Epoch AI                                        3    9% ▇
Polyrating: A Cost-Effective and Bias-A…        1    4% ▇
LMSYS Chatbot Arena leaderboard Space: …        1    3% ▇
Vectara hallucination leaderboard               1    0% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 47.2
Licences: EvalPlus / EvalPerf published evaluation runs ([Apache-2.0](../../licenses/evalplus-published-runs-Apache-2.0.txt)) · AlpacaEval 1.0 and 2.0 leaderboards ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Arena-Hard-Auto v0.1 leaderboard ([Apache-2.0](../../licenses/arena-hard-auto-v0-1-Apache-2.0.txt)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/claude-2-0) · [JSON](https://whatmodel.sh/api?format=json&all=1)