[whatmodel.sh](../../README.md) / Claude 2.1
# Claude 2.1
Anthropic · released 2023-11 · proprietary
**score 46.5** ±0.8 · comparison only: no current run option verified · from 9 benchmark results across 6 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Claude 2.1 among every model on that task or test, worst to best.
task vs the others rank counts tests
coding ▁▁▁▃▄●█▇▅▄▃▁ #376 of 539* 27% 1
chat & writing ▁▁▁▃▅●▇█▇▇▇▃ #364 of 474 49% 5
math ▁▁▃▆●█▅█▃▁▁▁ #396 of 530* 0.7% 1
coding · #376 of 539 · 27% of its score
Beats [Solar Open 100B (102B-A12B)](solar-open-100b.md), [Qwen3 4B Instruct 2507](qwen3-4b-instruct-2507.md), [Qwen3-VL 8B Instruct](qwen3-vl-8b-instruct.md) here, though they rank higher overall. Trails [DeepSeek LLM 67B Chat](deepseek-llm-67b-chat.md), [WizardLM 70B V1.0](wizardlm-70b-v1-0.md), [Code Tulu 2 34B](code-tulu-2-34b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
HumanEval 66.5% ▁█▄▅▇█·▄●▅▄▄ beats 29/42 easy · avg 75% 27% sharp · at its level
chat & writing · #364 of 474 · 49% of its score
Beats [Ministral 3 14B Reasoning 2512](ministral-3-14b-reasoning.md), [Gemma 4 E4B](gemma-4-e4b.md), [OpenThinker3 7B](openthinker3-7b.md) here, though they rank higher overall. Trails [LFM2 350M](lfm2-350m.md), [EXAONE 3.5 2.4B Instruct](exaone-3-5-2-4b.md), [LFM2 700M](lfm2-700m.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
AlpacaEval 2 length-controlled 25.3% ▁▂▂▃▆▆●▆▄▄▁▁ beats 64/122 medium · avg 35% 23% sharp · at its level
Arena-Hard (v0.1, GPT-4-1106 judg… 22.8% ▂▁▁▅●▅▄▃▃▅▃▁ beats 17/60 medium · avg 47% 14% sharp · at its level
Polyrating rating (bias-aware re-… 1156.0 ▂▁▂▁▃▇█▆●▄▂▂ beats 69/95 avg 1187 6.1% typical
MT-Bench (GPT-4 single-answer gra… 8.1800 MT-Bench score (1-10) ▆▂▂▆▆▄██▂●▄▆ beats 23/28 avg 8.539 MT-Bench scor… 4.7% sharp · noisy
AlpacaEval 1.0 win rate (vs text-… 87.1% ▂▂·▆·▇▃●█▆▄▂ beats 25/45 very easy · avg 94% 0.7% sharp · noisy
math · #396 of 530 · 0.7% of its score
Beats [GPT-4 (0613)](gpt-4-0613.md), [Athene-70B](athene-70b.md), [Gemma 2 27B IT](gemma-2-27b.md) here, though they rank higher overall. Trails [Stable Code 3B](stable-code-3b.md), [Qwen3 0.6B](qwen3-0-6b.md), [Falcon 40B (base)](falcon-40b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
OTIS mock AIME 2024–2025 1.9% ▃●▄▃▃▃▇▇█▃▂▅ beats 16/197 very hard · avg 7.2% 0.7% sharp · too hard for it · noisy
other tests (2)
test result vs the others in its run difficulty counts why
EvoEval (overall) 46.2% ▂▂▃▄▄█▁▄●▃▁▃ beats 31/42 medium · avg 52% 19% at its level
GPQA Diamond 33.0% ▁▁▁▁●▄▆▄▅▇█▅ beats 23/211 hard · avg 39% 4.6% sharp
source results share of the score
Top Leaderboard Ranking = Top Coding Pr… 2 46% ▇▇▇▇▇▇
AlpacaEval 1.0 and 2.0 leaderboards 2 23% ▇▇▇
Arena-Hard-Auto v0.1 leaderboard 1 14% ▇▇
Polyrating: A Cost-Effective and Bias-A… 1 6% ▇
Epoch AI 2 5% ▇
LMSYS Chatbot Arena leaderboard Space: … 1 5% ▇
prior: pull toward the middle 0% ▇
weighted average = 46.5
Licences: Top Leaderboard Ranking = Top Coding Proficiency ([CC BY 4.0](../../licenses/paper-2403.19114v1-CC-BY-4.0.txt)) · AlpacaEval 1.0 and 2.0 leaderboards ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Arena-Hard-Auto v0.1 leaderboard ([Apache-2.0](../../licenses/arena-hard-auto-v0-1-Apache-2.0.txt)) · Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · LMSYS Chatbot Arena leaderboard Space: MT-Bench column of the published leaderboard table ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
No verified API or local download yet.
[interactive version](https://whatmodel.sh/model/claude-2-1) · [JSON](https://whatmodel.sh/api?format=json&all=1)