[whatmodel.sh](../../README.md) / GPT-3 davinci (API base model)
# GPT-3 davinci (API base model)
OpenAI · released 2020-06 · proprietary
**score 31.5** ±0.7 · comparison only: no current run option verified · from 54 benchmark results across 19 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks GPT-3 davinci (API base model) among every model on that task or test, worst to best.
task vs the others rank counts tests
math ▁▁●▆██▅█▃▁▁▁ #488 of 530 6.6% 2
math · #488 of 530 · 6.6% of its score
Beats [Command R (03-2024)](command-r-03-2024.md), [Molmo 7B-D (0924)](molmo-7b-d.md), [Mistral 7B Instruct v0.3](mistral-7b-instruct-v0-3.md) here, though they rank higher overall. Trails [GPT-Neo 125M](gpt-neo-125m.md), [OPT-125M](opt-125m.md), [GPT-2 XL (1.5B)](gpt-2-xl.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
GSM8K 0.0% ▂▂▂▂●▄▅▇██▆▄ beats 34/206 medium · avg 62% 4.4% sharp · small run · several runs
MGSM 5.7% ▆·▂·▄●▂▂▂█▄▆ beats 6/18 medium · avg 48% 2.2% sharp · too hard for it
other tests (36)
test result vs the others in its run difficulty counts why
ARC Challenge 43.5% ▄····▄···█▄● beats 5/5 easy · avg 78% 14% sharp · several runs
WinoGrande 72.0% ▄·▄·▄▄▄··█·● beats 7/7 easy · avg 87% 10% at its level · several runs
HellaSwag 78.0% ▄·█·█··▄▄··● beats 7/7 easy · avg 83% 8.2% at its level · several runs
CommonsenseQA 58.2% █·····▂▂▃●▄▄ beats 26/38 very easy · avg 93% 8.0% sharp · at its level · small run · seve…
MMLU 39.7% ▁···▁··●▁▅▅█ beats 3/18 easy · avg 74% 7.2% several runs
OpenBookQA 33.6% ▅·······█··● beats 5/5 hard · avg 49% 4.8% several runs
DMath (English) few-shot CoT accu… 17.1% █·●······▄·▄ beats 2/4 medium · avg 64% 2.9% typical
TriviaQA 40.9% █···█·███··● beats 5/5 easy · avg 68% 2.8% at its level
BIG-Bench Hard 33.6% ▂▂▂●·▅·▃▂█▃▃ beats 3/18 medium · avg 60% 2.5% sharp
ARC Easy 76.2% ▄···▄··█·▄·● beats 5/5 easy · avg 87% 2.4% blunt · at its level · several runs
LogiQA (harness) 22.7% ······●····· beats 0/5 very hard · avg 25% 2.4% too hard for it
Integer-sequence completion, unam… 20.2% ●········▄·█ beats 0/3 easy · avg 73% 2.2% at its level
BoolQ 62.0% ▁····▁▂●▆▅▆▄ beats 24/103 medium · avg 78% 2.1% at its level
LAMBADA (OpenAI version), last-wo… 75.0% █·▄··█··█··● beats 7/7 very easy · avg 88% 2.1% blunt · several runs
ANLI Round 1 (prompted) 36.3% █··▂·······● beats 5/5 hard · avg 46% 2.1% sharp · too hard for it · noisy
TyDiQA-GoldP (1-shot, EM, 8-langu… 21.6% ▂··▂·▂▄●·███ beats 5/18 medium · avg 61% 1.9% typical
PIQA 80.0% █··▃··▃▃·▃·● beats 7/7 easy · avg 88% 1.7% blunt · at its level · several runs
COPA 80.0% ▁·▁▁▁▅▅●▆▂▁▁ beats 30/67 easy · avg 92% 1.6% at its level
Integer-sequence explanation as P… 6.0% ●······█··██ beats 0/3 medium · avg 47% 1.2% typical
Natural Questions (closed book) 0.0% ●▁▁▁▂▄▄▃▅█▅▁ beats 0/68 very hard · avg 13% 1.2% too hard for it
ANLI Round 3 (prompted) 36.9% ▃······▃█··● beats 5/5 hard · avg 43% 1.1% too hard for it
CSK-PN boolean commonsense QA (10… 70.8% ▃·●·····▃▃▃█ beats 1/7 easy · avg 87% 1.1% blunt · at its level
ANLI Round 2 (prompted) 37.5% ██·███·····● beats 5/5 very hard · avg 43% 1.0% too hard for it · noisy
Super-NaturalInstructions English… 45.0% █····●··█··█ beats 1/3 easy · avg 71% 1.0% at its level
API-Bank evaluation system, zero-… 0.6% ●······█··▄▄ beats 0/4 medium · avg 50% 1.0% typical
HaluEval knowledge-grounded dialo… 50.0% ●·········▄▄ beats 1/3 medium · avg 68% 1.0% sharp · too hard for it · noisy
SciQ 94.9% ██····███··● beats 5/5 very easy · avg 98% 1.0% blunt · maxed out
PROST (harness, micro accuracy) 26.7% ▃······●···▅ beats 1/5 very hard · avg 35% 0.8% blunt · too hard for it
QA4MRE 2013 (harness) 42.6% ▄····▄█▄···● beats 5/5 medium · avg 57% 0.8% blunt · at its level
SelfAware unanswerable-question d… 45.7% ●▄▄▄·······▄ beats 0/5 medium · avg 63% 0.8% blunt · at its level
Super-NaturalInstructions English… 6.8% ●·▄······▄▄█ beats 0/5 medium · avg 63% 0.6% at its level · noisy · small run
HEAD-QA (English, harness) 35.6% █··█····█··● beats 3/3 medium · avg 58% 0.6% blunt · at its level
SuperGLUE WSC (prompted) 54.8% █··········● beats 4/5 hard · avg 62% 0.6% blunt
LMentry accuracy (zero-shot) 38.0% ▂▆·●▄█▂·▂··▂ beats 5/14 medium · avg 65% 0.3% blunt · at its level
HaluEval QA hallucination recogni… 49.2% ●··········▃ beats 2/3 very hard · avg 56% 0.2% too hard for it · noisy
HaluEval summarization hallucinat… 51.2% █··········● beats 3/3 very hard · avg 50% 0.0% blunt · too hard for it · noisy
source results share of the score
GPT-NeoX-20B: An Open-Source Autoregres… 17 33% ▇▇▇▇
UPRISE: Universal Prompt Retrieval for … 8 18% ▇▇
Scaling Instruction-Finetuned Language … 4 12% ▇
EleutherAI GPT-J-6B model card 4 7% ▇
Making Language Models Better Reasoners… 2 6% ▇
GLM-130B: An Open Bilingual Pre-trained… 1 4% ▇
Self-Consistency of Large Language Mode… 2 3% ▇
Increasing Probability Mass on Answer C… 2 3% ▇
It Ain't Over: A Multi-aspect Diverse M… 1 3% ▇
WinoDict: Probing language models for i… 1 2% ▇
Exploring the Curious Case of Code Prom… 2 2% ▇
Bridging the Resource Gap: Exploring th… 1 2% ▇
HaluEval: A Large-Scale Hallucination E… 3 1% ▇
Say What You Mean! Large Language Model… 1 1% ▇
Super-NaturalInstructions: Generalizati… 1 1% ▇
API-Bank: A Comprehensive Benchmark for… 1 1% ▇
Do Large Language Models Know What They… 1 1% ▇
Self-Instruct: Aligning Language Models… 1 1% ▇
LMentry: A Language Model Benchmark of … 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 31.5
Licences: GPT-NeoX-20B: An Open-Source Autoregressive Language Model ([CC BY 4.0](../../licenses/paper-2204.06745v1-CC-BY-4.0.txt)) · UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-758-CC-BY-4-0.txt)) · Scaling Instruction-Finetuned Language Models ([CC BY 4.0](../../licenses/paper-2210.11416v5-CC-BY-4.0.txt)) · EleutherAI GPT-J-6B model card ([Apache-2.0](../../licenses/eleutherai-gpt-j-6b-card-Apache-2.0.txt)) · Making Language Models Better Reasoners with Step-Aware Verifier ([CC BY 4.0](../../licenses/acl-2023-acl-long-291-CC-BY-4-0.txt)) · GLM-130B: An Open Bilingual Pre-trained Model ([CC BY 4.0](../../licenses/paper-2210.02414v2-glm130b-CC-BY-4-0.txt)) · Self-Consistency of Large Language Models under Ambiguity ([CC BY 4.0](../../licenses/acl-2023-blackboxnlp-1-7-CC-BY-4-0.txt)) · Increasing Probability Mass on Answer Choices Does Not Always Improve Accuracy ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-522-CC-BY-4-0.txt)) · It Ain't Over: A Multi-aspect Diverse Math Word Problem Dataset ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-927-CC-BY-4-0.txt)) · WinoDict: Probing language models for in-context word acquisition ([CC BY 4.0](../../licenses/acl-2023-eacl-main-7-CC-BY-4-0.txt)) · Exploring the Curious Case of Code Prompts ([CC BY 4.0](../../licenses/acl-2023-nlrse-1-2-CC-BY-4-0.txt)) · Bridging the Resource Gap: Exploring the Efficacy of English and Multilingual LLMs for Swedish ([CC BY 4.0](../../licenses/acl-2023-resourceful-1-13-CC-BY-4-0.txt)) · HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-397-CC-BY-4-0.txt)) · Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge ([CC BY 4.0](../../licenses/acl-2023-acl-long-550-CC-BY-4-0.txt)) · Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks ([CC BY 4.0](../../licenses/acl-2022-emnlp-main-340-CC-BY-4-0.txt)) · API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-187-CC-BY-4-0.txt)) · Do Large Language Models Know What They Don't Know? ([CC BY 4.0](../../licenses/acl-2023-findings-acl-551-CC-BY-4-0.txt)) · Self-Instruct: Aligning Language Models with Self-Generated Instructions ([CC BY 4.0](../../licenses/acl-2023-acl-long-754-CC-BY-4-0.txt)) · LMentry: A Language Model Benchmark of Elementary Language Tasks ([CC BY 4.0](../../licenses/acl-2023-findings-acl-666-CC-BY-4-0.txt))
## How to run it
No verified API or local download yet.
[interactive version](https://whatmodel.sh/model/gpt-3-davinci) · [JSON](https://whatmodel.sh/api?format=json&all=1)