[whatmodel.sh](../../README.md) / GPT-3.5 text-davinci-002
# GPT-3.5 text-davinci-002
OpenAI · released 2022-03 · proprietary
**score 44.4** ±0.4 · comparison only: no current run option verified · from 67 benchmark results across 28 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks GPT-3.5 text-davinci-002 among every model on that task or test, worst to best.
task vs the others rank counts tests
math ▁▁▃▆●█▅█▃▁▁▁ #315 of 530 25% 2
math · #315 of 530 · 25% of its score
Beats [GPT-4o (2024-11-20)](gpt-4o-2024-11-20.md), [GPT-4 Turbo (2024-04-09)](gpt-4-turbo-2024-04-09.md), [Llama 3.1 405B Instruct](llama-3-1-405b.md) here, though they rank higher overall. Trails [DeepSeek R1 Distill Qwen 1.5B](deepseek-r1-distill-qwen-1-5b.md), [Gemma 7B (base)](gemma-7b-base.md), [CodeGemma 7B (v1.0)](codegemma-7b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
GSM8K 46.9% █·▅▃▃▃·▃·▃●▅ beats 10/12 easy · avg 67% 22% sharp · at its level · several runs
MGSM 23.7% ▆·▂·▄▂▂▂●█▄▆ beats 9/18 medium · avg 48% 2.5% sharp · at its level
other tests (38)
test result vs the others in its run difficulty counts why
SVAMP 68.9% ▃▅█···▅·▃▃●▅ beats 10/12 easy · avg 82% 14% sharp · at its level · several runs
AQuA-RAT 35.8% █▁·▁▁·····●▁ beats 11/12 medium · avg 48% 12% sharp · several runs
CommonsenseQA 79.5% █······▁·▁▁● beats 11/12 easy · avg 75% 9.0% sharp · at its level · several runs
MAWPS (CoT-paper union) 87.1% ▅▃█···▅·▃▃●▅ beats 10/12 very easy · avg 94% 5.8% sharp
ASDiv 71.3% ▅▃▃▅··▃▃·▃▃● beats 10/12 easy · avg 83% 5.2% sharp · at its level
MMLU 63.1% ▁···▁··▃▁▅●█ beats 11/18 easy · avg 74% 4.5% at its level · several runs
BIG-Bench Hard 48.6% ▂▂▂▂·▅·▃▂●▃▃ beats 11/18 medium · avg 60% 3.8% sharp · at its level · several runs
New Yorker Caption Contest matchi… 57.2% █·····▃··●·▃ beats 4/6 medium · avg 71% 3.4% sharp · at its level
BIG-bench Sports Understanding 69.6% █·····▃·▁·●▁ beats 9/12 medium · avg 81% 2.6% sharp · at its level
BIG-bench Date Understanding 43.8% ▃·▅·▅▃▅·▃▃·● beats 10/12 medium · avg 52% 2.2% at its level · several runs
StrategyQA 65.9% █·····▅▃▃▅▃● beats 10/12 medium · avg 70% 2.0% at its level · several runs
OpenBookQA 75.4% ▅▁▁▁▂▂▃█▃▁●▁ beats 95/107 easy · avg 78% 1.3% at its level · small run · several runs
TyDiQA-GoldP (1-shot, EM, 8-langu… 41.1% ▂··▂·▂▄▄·●██ beats 8/18 medium · avg 61% 1.0% at its level
HaluEval knowledge-grounded dialo… 60.8% █·········●▄ beats 2/3 medium · avg 68% 0.8% sharp · noisy
EQ-Bench v1 (60 questions, paper … 39.4% ▂··▄·▂●▄▂█·▂ beats 6/14 medium · avg 53% 0.8% at its level
MultiArith 90.8% █·······●·██ beats 1/3 very easy · avg 95% 0.8% sharp · small run
WinoGrande 62.8% ▁▁▁▁▂▁▁▅▇●▃▁ beats 104/131 medium · avg 73% 0.7% at its level · several runs
ARC Challenge 68.9% ▄▁▁▂▁▄█▇●▂▃▂ beats 115/140 medium · avg 72% 0.7% sharp · at its level · small run
COPA 94.4% ▁·▁▁▁▅▅█▆▂●▁ beats 62/67 very easy · avg 96% 0.5% several runs
HellaSwag 71.7% ▁▁▁▂▂▂▄▆●▇▄▂ beats 74/135 easy · avg 77% 0.5% at its level · several runs
Anthropic HH helpful preference c… 61.0% █········●·█ beats 1/2 hard · avg 65% 0.4% small run
ACE05-E event argument extraction… 56.5% ●·····█····█ beats 0/2 medium · avg 65% 0.4% at its level · small run
BoolQ 58.8% ▁····▁●█▆▅▆▄ beats 8/103 medium · avg 75% 0.4% at its level · small run
CoNLL-2003 English NER, few-shot … 81.0% ▃···▅·▃▅··▃● beats 7/9 easy · avg 84% 0.4% small run
PopQA 34.7% ▁▂▁▃▃█▂█▆●▁▆ beats 32/40 hard · avg 31% 0.4% typical
SummEdits factual-consistency det… 60.1% ▄····▄·●···▄ beats 3/4 hard · avg 65% 0.4% noisy
CSK-PN boolean commonsense QA (10… 84.4% ▃·▃·····▃▃▃● beats 6/7 easy · avg 87% 0.4% blunt · at its level
SelfAware unanswerable-question d… 47.5% ●▄▄▄·······▄ beats 1/5 medium · avg 63% 0.3% blunt · at its level
Zero-shot relation extraction, va… 43.2% █····●····█▄ beats 2/5 medium · avg 50% 0.2% blunt · at its level
CREPE event likelihood change (3-… 53.3% ██····█····● beats 3/3 medium · avg 52% 0.2% blunt · at its level
HaluEval QA hallucination recogni… 60.1% █··········● beats 3/3 very hard · avg 56% 0.1% too hard for it · noisy
Social IQA 65.3% ▁·▁██▄▅▁●▃▇▁ beats 44/59 medium · avg 65% 0.1% blunt · at its level · small run
LMentry accuracy (zero-shot) 80.6% ▂▆·▄▄█▂·▂··● beats 14/14 medium · avg 65% 0.1% blunt · at its level
ARC Easy 85.8% ▁▃▂▄▆█▅▃●··▁ beats 96/100 easy · avg 85% 0.1% blunt · at its level · noisy · small run
Story Cloze Test 95.6% ▃▃▃▂▅█▅▂▁··● beats 49/50 easy · avg 90% 0.1% blunt · at its level · noisy · small run
PIQA 84.2% ▁▁▁▂▂▃▅█▅●▁▁ beats 118/123 easy · avg 85% 0.1% blunt · at its level · noisy · small run
SciQ 86.0% ▂·▁▂▂▃●▃▇▆█▂ beats 15/51 very easy · avg 97% 0.0% blunt · maxed out · noisy · small run
HaluEval summarization hallucinat… 47.8% ●··········▃ beats 2/3 very hard · avg 50% 0.0% blunt · too hard for it · noisy
source results share of the score
Chain-of-Thought Prompting Elicits Reas… 9 41% ▇▇▇▇▇
Large Language Models Are Reasoning Tea… 5 11% ▇
Scaling Instruction-Finetuned Language … 4 9% ▇
Faithful Chain-of-Thought Reasoning 3 8% ▇
INFORM : Information eNtropy based mult… 1 4% ▇
Do Androids Laugh at Electric Sheep? Hu… 1 3% ▇
Making Language Models Better Reasoners… 2 3% ▇
Vera: A General-Purpose Plausibility Es… 12 3% ▇
Complexity-Based Prompting for Multi-St… 4 3% ▇
Tab-CoT: Zero-shot Tabular Chain of Tho… 3 3% ▇
Augmentation-Adapted Retriever Improves… 2 2% ▇
Challenging BIG-Bench Tasks and Whether… 1 1% ▇
Towards Understanding Chain-of-Thought … 1 1% ▇
MoT: Memory-of-Thought Enables ChatGPT … 2 1% ▇
Improving Mathematics Tutoring With A C… 1 1% ▇
HaluEval: A Large-Scale Hallucination E… 3 1% ▇
EQ-Bench: An Emotional Intelligence Ben… 1 1% ▇
Exploring the Curious Case of Code Prom… 2 1% ▇
Is ChatGPT a Good Causal Reasoner? A Co… 1 0% ▇
trlX: A Framework for Large Scale Reinf… 1 0% ▇
Code4Struct: Code Generation for Few-Sh… 1 0% ▇
CodeIE: Large Code Generation Models ar… 1 0% ▇
SummEdits: Measuring LLM Ability at Fac… 1 0% ▇
Say What You Mean! Large Language Model… 1 0% ▇
Do Large Language Models Know What They… 1 0% ▇
Aligning Instruction Tasks Unlocks Larg… 1 0% ▇
Causal Reasoning of Entities and Events… 1 0% ▇
LMentry: A Language Model Benchmark of … 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 44.4
Licences: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models ([CC BY 4.0](../../licenses/paper-2201.11903v6-CC-BY-4-0.txt)) · Large Language Models Are Reasoning Teachers ([CC BY 4.0](../../licenses/acl-2023-acl-long-830-CC-BY-4-0.txt)) · Scaling Instruction-Finetuned Language Models ([CC BY 4.0](../../licenses/paper-2210.11416v5-CC-BY-4.0.txt)) · Faithful Chain-of-Thought Reasoning ([CC BY 4.0](../../licenses/acl-2023-ijcnlp-main-20-CC-BY-4-0.txt)) · INFORM : Information eNtropy based multi-step reasoning FOR large language Models ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-216-CC-BY-4-0.txt)) · Do Androids Laugh at Electric Sheep? Humor \`\`Understanding'' Benchmarks from The New Yorker Caption Contest ([CC BY 4.0](../../licenses/acl-2023-acl-long-41-CC-BY-4-0.txt)) · Making Language Models Better Reasoners with Step-Aware Verifier ([CC BY 4.0](../../licenses/acl-2023-acl-long-291-CC-BY-4-0.txt)) · Vera: A General-Purpose Plausibility Estimation Model for Commonsense Statements ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-81-CC-BY-4-0.txt)) · Complexity-Based Prompting for Multi-Step Reasoning ([CC BY 4.0](../../licenses/paper-2210.00720v2-CC-BY-4-0.txt)) · Tab-CoT: Zero-shot Tabular Chain of Thought ([CC BY 4.0](../../licenses/acl-2023-findings-acl-651-CC-BY-4-0.txt)) · Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In ([CC BY 4.0](../../licenses/acl-2023-acl-long-136-CC-BY-4-0.txt)) · Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them ([CC BY 4.0](../../licenses/acl-2023-findings-acl-824-bbh-CC-BY-4-0.txt)) · Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters ([CC BY 4.0](../../licenses/acl-2023-acl-long-153-CC-BY-4-0.txt)) · MoT: Memory-of-Thought Enables ChatGPT to Self-Improve ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-392-CC-BY-4-0.txt)) · Improving Mathematics Tutoring With A Code Scratchpad ([CC BY 4.0](../../licenses/acl-2023-bea-1-2-CC-BY-4-0.txt)) · HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-397-CC-BY-4-0.txt)) · EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models ([CC BY 4.0](../../licenses/paper-2312.06281v2-CC-BY-4.0.txt)) · Exploring the Curious Case of Code Prompts ([CC BY 4.0](../../licenses/acl-2023-nlrse-1-2-CC-BY-4-0.txt)) · Is ChatGPT a Good Causal Reasoner? A Comprehensive Evaluation ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-743-CC-BY-4-0.txt)) · trlX: A Framework for Large Scale Reinforcement Learning from Human Feedback ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-530-CC-BY-4-0.txt)) · Code4Struct: Code Generation for Few-Shot Event Structure Prediction ([CC BY 4.0](../../licenses/acl-2023-acl-long-202-CC-BY-4-0.txt)) · CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors ([CC BY 4.0](../../licenses/acl-2023-acl-long-855-CC-BY-4-0.txt)) · SummEdits: Measuring LLM Ability at Factual Reasoning Through The Lens of Summarization ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-600-CC-BY-4-0.txt)) · Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge ([CC BY 4.0](../../licenses/acl-2023-acl-long-550-CC-BY-4-0.txt)) · Do Large Language Models Know What They Don't Know? ([CC BY 4.0](../../licenses/acl-2023-findings-acl-551-CC-BY-4-0.txt)) · Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors ([CC BY 4.0](../../licenses/acl-2023-findings-acl-50-CC-BY-4-0.txt)) · Causal Reasoning of Entities and Events in Procedural Texts ([CC BY 4.0](../../licenses/acl-2023-findings-eacl-31-CC-BY-4-0.txt)) · LMentry: A Language Model Benchmark of Elementary Language Tasks ([CC BY 4.0](../../licenses/acl-2023-findings-acl-666-CC-BY-4-0.txt))
Not counted (2)
source test score why
Strength in Numbers: … SciQ 30.0% only model in this source's run
Strength in Numbers: … Social IQA 35.0% only model in this source's run