[whatmodel.sh](../../README.md) / Claude Instant 1.2 # Claude Instant 1.2 Anthropic · released 2023-08 · proprietary **score 44.1** ±0.7 · comparison only: no current run option verified · from 5 benchmark results across 3 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks Claude Instant 1.2 among every model on that task or test, worst to best.}

task            vs the others          rank  counts  tests
coding          ▁▁▁▃▄●█▇▅▄▃▁    #421 of 539     78%      2
chat & writing  ▁▁▁▃▅●▇█▇▇▇▃   #356 of 474*     16%      1
math            ▁▁▃▆●█▅█▃▁▁▁   #342 of 530*    5.7%      1

coding · #421 of 539 · 78% of its score

Beats [LFM2.5 8B-A1B](lfm2-5-8b-a1b.md), [Qwen3-VL 2B Thinking](qwen3-vl-2b-thinking.md), [DeepSeek R1 Distill Qwen 7B](deepseek-r1-distill-qwen-7b.md) here, though they rank higher overall. Trails [Code Tulu 2 13B](code-tulu-2-13b.md), [Phi-3-mini-128k-instruct (Apr 2024)](phi-3-mini-128k-instruct.md), [DeepSeek LLM 67B Chat](deepseek-llm-67b-chat.md) here, though they rank lower overall.

test        result  vs the others in its run  difficulty      counts  why
HumanEval    57.3%  ▁·▁▁▃▄●▅█▃▁▁ beats 45/99  easy · avg 76%     45%  sharp · at its level · several runs
HumanEval+   50.6%  ▁·▁▁▁▃▄●▄█▃▁ beats 41/99  easy · avg 70%     33%  sharp · at its level

chat & writing · #356 of 474 · 16% of its score

Beats [Ministral 3 14B Reasoning 2512](ministral-3-14b-reasoning.md), [Gemma 4 E4B](gemma-4-e4b.md), [OpenThinker3 7B](openthinker3-7b.md) here, though they rank higher overall. Trails [EXAONE 3.5 2.4B Instruct](exaone-3-5-2-4b.md), [LFM2 700M](lfm2-700m.md), [Flan-T5 XXL](flan-t5-xxl.md) here, though they rank lower overall.

test                            result  vs the others in its run   difficulty        counts  why
AlpacaEval 2 length-controlled   25.6%  ▁▂▂▃▆▆●▆▄▄▁▁ beats 68/122  medium · avg 35%     16%  sharp

math · #342 of 530 · 5.7% of its score

Beats [Molmo 2 8B](molmo-2-8b.md), [Llama 3.3 70B Instruct](llama-3-3-70b.md), [DeepSeek-V2.5](deepseek-v2-5.md) here, though they rank higher overall. Trails [DeepSeek R1 Distill Qwen 1.5B](deepseek-r1-distill-qwen-1-5b.md), [Gemma 7B (base)](gemma-7b-base.md), [Mistral 7B v0.1 (base)](mistral-7b-v0-1.md) here, though they rank lower overall.

test           result  vs the others in its run    difficulty           counts  why
GSM8K (maker)   86.7%  ▂▂▂▂▃▄▅▇█●▆▄ beats 159/206  very easy · avg 90%    5.7%  sharp

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
EvalPlus / EvalPerf published evaluatio…        2   74% ▇▇▇▇▇▇▇▇▇
AlpacaEval 1.0 and 2.0 leaderboards             1   16% ▇▇
Anthropic system cards and model announ…        2   10% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 44.1

_{Licences: EvalPlus / EvalPerf published evaluation runs ([Apache-2.0](../../licenses/evalplus-published-runs-Apache-2.0.txt)) · AlpacaEval 1.0 and 2.0 leaderboards ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))} ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/claude-instant-1-2) · [JSON](https://whatmodel.sh/api?format=json&all=1)