[whatmodel.sh](../../README.md) / GPT-3.5 text-davinci-003
# GPT-3.5 text-davinci-003
OpenAI · released 2022-11 · proprietary
**score 46.1** ±0.5 · comparison only: no current run option verified · from 70 benchmark results across 40 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks GPT-3.5 text-davinci-003 among every model on that task or test, worst to best.
task vs the others rank counts tests
chat & writing ▁▁●▃▅▄▇█▇▇▇▃ #460 of 474* 0.3% 1
math ▁▁▃▆█●▅█▃▁▁▁ #283 of 530 29% 2
chat & writing · #460 of 474 · 0.3% of its score
Trails [Gemma 3 270M IT](gemma-3-270m.md), [SmolLM2 360M Instruct](smollm2-360m.md), [Granite 4.0 350M](granite-4-0-350m.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
AlpacaEval 1.0 win rate (vs text-… 50.0% ▂▂·●·▇▃▆█▆▄▂ beats 9/45 very easy · avg 94% 0.3% sharp · noisy
math · #283 of 530 · 29% of its score
Beats [GPT-4o (2024-08-06)](gpt-4o-2024-08-06.md), [Command A (03-2025)](command-a.md), [GPT-4o (2024-05-13)](gpt-4o-2024-05-13.md) here, though they rank higher overall. Trails [Gemma 7B (base)](gemma-7b-base.md), [CodeGemma 7B (v1.0)](codegemma-7b.md), [Phi-2 (2.7B)](phi-2.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
GSM8K 56.5% █··█······▄● beats 5/5 easy · avg 67% 23% sharp · at its level · several runs
MGSM 36.0% ▆·▂·▄▂▂▂▂●▄▆ beats 13/18 medium · avg 48% 6.4% sharp · at its level · several runs
other tests (41)
test result vs the others in its run difficulty counts why
AQuA-RAT 46.9% █··▁▁▁··▁●▁▁ beats 25/27 medium · avg 56% 16% sharp · at its level · several runs
CommonsenseQA 62.8% █·····▂▂▃▂▄● beats 34/38 easy · avg 80% 8.7% sharp · at its level · small run · seve…
SVAMP 79.5% █▆▆▅▂▅▂▅▂▂▅● beats 29/30 easy · avg 84% 6.4% sharp · several runs
MMLU 64.8% ▁···▁··▃▁▅●█ beats 12/18 easy · avg 74% 6.2% at its level · several runs
BIG-Bench Hard 50.9% ▂▂▂▂·▅·▃▂●▃▃ beats 14/18 medium · avg 60% 4.1% sharp · at its level
OpenBookQA 81.6% ▅▁▁▁▂▂▃█▃▁●▁ beats 103/107 easy · avg 78% 3.5% at its level · small run · several runs
StrategyQA 71.5% █····▂▂▁▂▂▂● beats 23/25 medium · avg 72% 2.2% at its level · small run · several runs
BoolQ 67.7% ▁····▁▂█▆▅●▄ beats 89/103 medium · avg 75% 1.7% at its level · small run · several runs
TyDiQA-GoldP (1-shot, EM, 8-langu… 43.7% ▂··▂·▂▄▄·●██ beats 9/18 medium · avg 61% 1.7% at its level
COPA 93.2% ▁·▁▁▁▅▅█▆▂●▁ beats 63/67 very easy · avg 96% 1.5% several runs
HaluEval knowledge-grounded dialo… 68.4% █·········▄● beats 3/3 medium · avg 68% 1.4% sharp · at its level · noisy
Integer-sequence completion, unam… 66.0% ▄········●·█ beats 1/3 easy · avg 73% 1.4% at its level
EQ-Bench v1 (60 questions, paper … 43.7% ▂··▄·▂▆●▂█·▂ beats 7/14 medium · avg 53% 1.4% at its level
clembench v1.0 clemscore (self-pl… 15.8% ▄······●·█·▄ beats 1/4 medium · avg 38% 1.3% sharp · at its level
Integer-sequence explanation as P… 31.2% █······●··██ beats 1/3 medium · avg 47% 1.2% at its level
JEEBench total score (zero-shot, … 12.2% █··●····█··█ beats 1/3 very hard · avg 20% 0.9% too hard for it
ZeroSCROLLS Qasper (zero-shot F1) 52.7% ▂▂··▂·▂··▂█● beats 9/10 medium · avg 56% 0.8% blunt · at its level
CoNLL-2003 English NER, few-shot … 77.2% ▃···▅·▃▅··●█ beats 6/9 easy · avg 83% 0.8% small run
ZeroSCROLLS QuALITY (zero-shot ac… 69.0% ▃····▃·▃▃●▅▅ beats 5/10 easy · avg 81% 0.8% at its level
Anthropic HH helpful preference c… 63.0% █········█·● beats 2/2 hard · avg 65% 0.7% at its level · small run
SciTab 3-class claim verification… 41.4% █···▄▄·▄▄▄·● beats 7/7 medium · avg 51% 0.7% blunt · at its level
ACE05-E event argument extraction… 60.4% █·····█····● beats 2/2 medium · avg 65% 0.7% at its level · small run
WinoGrande 73.5% ▁▁▁▁▂▁▁▅▇●▃▁ beats 108/131 medium · avg 73% 0.7% at its level
SummEdits factual-consistency det… 70.7% ▄····▄·█···● beats 4/4 hard · avg 65% 0.6% noisy
RadNLI (3-class radiology NLI), z… 55.9% █·●········█ beats 1/2 easy · avg 65% 0.6% at its level · small run
CSK-PN boolean commonsense QA (10… 84.2% ▃·▃·····▃▃▃● beats 5/7 easy · avg 87% 0.6% blunt · at its level
HellaSwag 71.4% ▁▁▁▂▂▂▄▆●▇▄▂ beats 71/135 easy · avg 77% 0.5% at its level
SelfAware unanswerable-question d… 51.4% █▄●▄·······▄ beats 3/5 medium · avg 63% 0.4% blunt · at its level
Social IQA 73.3% ▁·▁██▄▅▁▂▃●▁ beats 50/59 medium · avg 70% 0.4% blunt · at its level · small run
ZeroSCROLLS NarrativeQA (zero-sho… 24.6% ▃··▃·█▃·●▃·▃ beats 6/10 hard · avg 29% 0.4% blunt · at its level
HateCheck functional tests, zero-… 76.1% █·····●·█··█ beats 1/3 easy · avg 84% 0.4% blunt · at its level
Zero-shot relation extraction, va… 47.6% █····▄····█● beats 5/5 medium · avg 50% 0.3% blunt · at its level
PIQA 76.9% ▁▁▁▂▂▃▅●▅▃▁▁ beats 60/123 easy · avg 84% 0.3% blunt · at its level · small run
ConvRe Re2Text, normal relation, … 85.4% ▂▂▄··▂▂●▄··▂ beats 9/12 easy · avg 90% 0.3% at its level · noisy
CREPE event likelihood change (3-… 42.3% ██····●····█ beats 2/3 medium · avg 52% 0.3% blunt · at its level
HaluEval QA hallucination recogni… 49.6% ●··········▃ beats 1/3 very hard · avg 56% 0.2% noisy
FANToM BeliefQ [Choice], short co… 17.7% ●·····▂····▁ beats 1/11 very hard · avg 55% 0.2% sharp · too hard for it · noisy
StoryAnalogy multiple-choice stor… 37.1% █·······●··█ beats 1/2 hard · avg 42% 0.2% blunt · noisy · small run
ZeroSCROLLS MuSiQue (zero-shot F1) 33.5% ▄·█··●█·▄▄▄▄ beats 4/10 medium · avg 43% 0.1% blunt · at its level
HaluEval summarization hallucinat… 48.1% ●··········▃ beats 1/3 very hard · avg 50% 0.0% blunt · too hard for it · noisy
Natural Questions (closed book) 29.9% ▂▁▁▁▂▄▄▃▅█●▁ beats 60/68 medium · avg 48% — not counted yet: no other model in the …
source results share of the score
Scaling Instruction-Finetuned Language … 4 15% ▇▇
Faithful Chain-of-Thought Reasoning 3 13% ▇▇
Is ChatGPT a General-Purpose Natural La… 7 10% ▇
INFORM : Information eNtropy based mult… 1 7% ▇
Batch Prompting: Efficient Inference wi… 3 6% ▇
Unnatural Error Correction: GPT-4 Can A… 1 4% ▇
Increasing Probability Mass on Answer C… 3 3% ▇
PAL: Program-aided Language Models 1 3% ▇
Examining Inter-Consistency of Large La… 5 3% ▇
Self-Knowledge Guided Retrieval Augment… 2 3% ▇
Explanation Selection Using Unlabeled D… 2 3% ▇
Self-Consistency of Large Language Mode… 2 3% ▇
Small Language Models Fine-tuned to Coo… 1 2% ▇
ZeroSCROLLS: A Zero-Shot Benchmark for … 4 2% ▇
Not All Languages Are Created Equal in … 1 2% ▇
Towards Understanding Chain-of-Thought … 1 2% ▇
MoT: Memory-of-Thought Enables ChatGPT … 2 2% ▇
HaluEval: A Large-Scale Hallucination E… 3 2% ▇
In-Context Demonstration Selection with… 2 1% ▇
Bridging the Resource Gap: Exploring th… 1 1% ▇
EQ-Bench: An Emotional Intelligence Ben… 1 1% ▇
clembench: Using Game Play to Evaluate … 1 1% ▇
Exploring the Curious Case of Code Prom… 2 1% ▇
Have LLMs Advanced Enough? A Challengin… 1 1% ▇
Large Language Model Is Not a Good Few-… 1 1% ▇
Is ChatGPT a Good Causal Reasoner? A Co… 1 1% ▇
trlX: A Framework for Large Scale Reinf… 1 1% ▇
SCITAB: A Challenging Benchmark for Com… 1 1% ▇
Code4Struct: Code Generation for Few-Sh… 1 1% ▇
SummEdits: Measuring LLM Ability at Fac… 1 1% ▇
Exploring the Boundaries of GPT-4 in Ra… 1 1% ▇
Say What You Mean! Large Language Model… 1 1% ▇
Do Large Language Models Know What They… 1 0% ▇
InstructSafety: A Unified Framework for… 1 0% ▇
AlpacaEval 1.0 and 2.0 leaderboards 1 0% ▇
Aligning Instruction Tasks Unlocks Larg… 1 0% ▇
An Investigation of LLMs' Inefficacy in… 1 0% ▇
Causal Reasoning of Entities and Events… 1 0% ▇
FANToM: A Benchmark for Stress-testing … 1 0% ▇
StoryAnalogy: Deriving Story-level Anal… 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 46.1
Licences: Scaling Instruction-Finetuned Language Models ([CC BY 4.0](../../licenses/paper-2210.11416v5-CC-BY-4.0.txt)) · Faithful Chain-of-Thought Reasoning ([CC BY 4.0](../../licenses/acl-2023-ijcnlp-main-20-CC-BY-4-0.txt)) · Is ChatGPT a General-Purpose Natural Language Processing Task Solver? ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-85-CC-BY-4-0.txt)) · INFORM : Information eNtropy based multi-step reasoning FOR large language Models ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-216-CC-BY-4-0.txt)) · Batch Prompting: Efficient Inference with Large Language Model APIs ([CC BY 4.0](../../licenses/acl-2023-emnlp-industry-74-CC-BY-4-0.txt)) · Unnatural Error Correction: GPT-4 Can Almost Perfectly Handle Unnatural Scrambled Text ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-550-CC-BY-4-0.txt)) · Increasing Probability Mass on Answer Choices Does Not Always Improve Accuracy ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-522-CC-BY-4-0.txt)) · PAL: Program-aided Language Models ([CC0 1.0](../../licenses/paper-2211.10435v2-pal-CC0-1-0.txt)) · Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-508-CC-BY-4-0.txt)) · Self-Knowledge Guided Retrieval Augmentation for Large Language Models ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-691-CC-BY-4-0.txt)) · Explanation Selection Using Unlabeled Data for Chain-of-Thought Prompting ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-41-CC-BY-4-0.txt)) · Self-Consistency of Large Language Models under Ambiguity ([CC BY 4.0](../../licenses/acl-2023-blackboxnlp-1-7-CC-BY-4-0.txt)) · Small Language Models Fine-tuned to Coordinate Larger Language Models improve Complex Reasoning ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-225-CC-BY-4-0.txt)) · ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-536-CC-BY-4-0.txt)) · Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-826-CC-BY-4-0.txt)) · Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters ([CC BY 4.0](../../licenses/acl-2023-acl-long-153-CC-BY-4-0.txt)) · MoT: Memory-of-Thought Enables ChatGPT to Self-Improve ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-392-CC-BY-4-0.txt)) · HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-397-CC-BY-4-0.txt)) · In-Context Demonstration Selection with Cross Entropy Difference ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-81-CC-BY-4-0.txt)) · Bridging the Resource Gap: Exploring the Efficacy of English and Multilingual LLMs for Swedish ([CC BY 4.0](../../licenses/acl-2023-resourceful-1-13-CC-BY-4-0.txt)) · EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models ([CC BY 4.0](../../licenses/paper-2312.06281v2-CC-BY-4.0.txt)) · clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-689-CC-BY-4-0.txt)) · Exploring the Curious Case of Code Prompts ([CC BY 4.0](../../licenses/acl-2023-nlrse-1-2-CC-BY-4-0.txt)) · Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-468-CC-BY-4-0.txt)) · Large Language Model Is Not a Good Few-shot Information Extractor ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-710-CC-BY-4-0.txt)) · Is ChatGPT a Good Causal Reasoner? A Comprehensive Evaluation ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-743-CC-BY-4-0.txt)) · trlX: A Framework for Large Scale Reinforcement Learning from Human Feedback ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-530-CC-BY-4-0.txt)) · SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-483-CC-BY-4-0.txt)) · Code4Struct: Code Generation for Few-Shot Event Structure Prediction ([CC BY 4.0](../../licenses/acl-2023-acl-long-202-CC-BY-4-0.txt)) · SummEdits: Measuring LLM Ability at Factual Reasoning Through The Lens of Summarization ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-600-CC-BY-4-0.txt)) · Exploring the Boundaries of GPT-4 in Radiology ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-891-CC-BY-4-0.txt)) · Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge ([CC BY 4.0](../../licenses/acl-2023-acl-long-550-CC-BY-4-0.txt)) · Do Large Language Models Know What They Don't Know? ([CC BY 4.0](../../licenses/acl-2023-findings-acl-551-CC-BY-4-0.txt)) · InstructSafety: A Unified Framework for Building Multidimensional and Explainable Safety Detector through Instruction Tuning ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-700-CC-BY-4-0.txt)) · AlpacaEval 1.0 and 2.0 leaderboards ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors ([CC BY 4.0](../../licenses/acl-2023-findings-acl-50-CC-BY-4-0.txt)) · An Investigation of LLMs' Inefficacy in Understanding Converse Relations ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-429-CC-BY-4-0.txt)) · Causal Reasoning of Entities and Events in Procedural Texts ([CC BY 4.0](../../licenses/acl-2023-findings-eacl-31-CC-BY-4-0.txt)) · FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-890-CC-BY-4-0.txt)) · StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-706-CC-BY-4-0.txt))
Not counted (18)
source test score why
Plan-and-Solve Prompt… GSM8K 58.4% only model in this source's run
Plan-and-Solve Prompt… AQuA-RAT 48.4% only model in this source's run
Plan-and-Solve Prompt… SVAMP 80.3% only model in this source's run
Plan-and-Solve Prompt… CommonsenseQA 78.3% only model in this source's run
Plan-and-Solve Prompt… StrategyQA 71.2% only model in this source's run
Evaluating Open-Domai… Natural Questions (closed boo… 29.9% too few models have it yet
IAG: Induction-Augmen… StrategyQA 71.5% only model in this source's run
INFORM : Information … AQuA-RAT 43.7% only model in this source's run
INFORM : Information … CommonsenseQA 77.5% only model in this source's run
INFORM : Information … StrategyQA 68.2% only model in this source's run
Auto-Instruct: Automa… BIG-Bench Hard 50.3% only model in this source's run
Self-Polish: Enhance … GSM8K 56.1% only model in this source's run
Self-Polish: Enhance … AQuA-RAT 44.9% only model in this source's run
Self-Polish: Enhance … SVAMP 80.3% only model in this source's run
CAR: Conceptualizatio… CommonsenseQA 68.9% only model in this source's run
CAR: Conceptualizatio… PIQA 67.8% only model in this source's run
CAR: Conceptualizatio… Social IQA 68.0% only model in this source's run
CAR: Conceptualizatio… WinoGrande 60.7% only model in this source's run