[whatmodel.sh](../../README.md) / GPT-3 text-curie-001
# GPT-3 text-curie-001
OpenAI · released 2022-01 · proprietary
**score 12.0** ±1.4 · comparison only: no current run option verified · from 18 benchmark results across 9 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks GPT-3 text-curie-001 among every model on that task or test, worst to best.
task vs the others rank counts tests
math ▁●▃▆██▅█▃▁▁▁ #513 of 530* 5.6% 1
math · #513 of 530 · 5.6% of its score
Beats [Flan-T5 Large](flan-t5-large.md), [OPT-66B](opt-66b.md), [Flan-T5 Base](flan-t5-base.md) here, though they rank higher overall. Trails [GPT-Neo 125M](gpt-neo-125m.md), [OPT-125M](opt-125m.md), [GPT-2 XL (1.5B)](gpt-2-xl.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
GSM8K 2.4% █·▅●▃▃·▃·▃▃▅ beats 5/12 easy · avg 67% 5.6% sharp · too hard for it · several runs
other tests (16)
test result vs the others in its run difficulty counts why
MAWPS (CoT-paper union) 3.5% ▅▃●···▅·▃▃▃▅ beats 4/12 very easy · avg 94% 32% sharp · too hard for it
ASDiv 3.6% ▅▃●▅··▃▃·▃▃█ beats 3/12 easy · avg 83% 17% sharp · too hard for it
SVAMP 3.1% ▃▅●···▅·▃▃▃▅ beats 4/12 easy · avg 82% 14% sharp · too hard for it
BIG-bench Date Understanding 8.9% ▃·▅·●▃▅·▃▃·█ beats 4/12 medium · avg 52% 6.3% too hard for it
CSK-PN boolean commonsense QA (10… 67.9% ●·▃·····▃▃▃█ beats 0/7 easy · avg 87% 3.9% blunt · at its level
New Yorker Caption Contest matchi… 20.4% ●·····▃··▅·▃ beats 0/6 medium · avg 71% 3.7% sharp · too hard for it
CommonsenseQA 19.0% ●······▁·▁▁▃ beats 3/12 easy · avg 88% 3.5% sharp · too hard for it
OpenBookQA 30.6% ▅▁▁▁●▂▃█▃▁▃▁ beats 30/107 medium · avg 70% 3.0% too hard for it · small run
BIG-bench Sports Understanding 0.0% ●·····▃·▁·▄▁ beats 3/12 medium · avg 81% 2.5% sharp · too hard for it
StrategyQA 53.6% █·····●▃▃▅▃█ beats 4/12 medium · avg 70% 2.3% too hard for it
BoolQ 66.2% ▁····▁▂█●▅▆▄ beats 42/103 easy · avg 88% 2.2% noisy · small run
LMentry accuracy (zero-shot) 41.7% ▂▆·▄●█▂·▂··▂ beats 7/14 medium · avg 65% 1.2% blunt · at its level
ConvRe Re2Text, normal relation, … 53.5% ▂▂●··▂▂█▄··▂ beats 3/12 easy · avg 90% 1.2% noisy
CREPE event likelihood change (3-… 35.6% █●····█····█ beats 1/3 medium · avg 52% 0.9% blunt
AQuA-RAT 13.4% ●▁·▁▁·····▂▁ beats 4/12 medium · avg 48% 0.5% sharp · too hard for it
FANToM BeliefQ [Choice], short co… 21.0% ●·····▂····▁ beats 2/11 very hard · avg 55% 0.0% sharp · too hard for it · noisy
source results share of the score
Chain-of-Thought Prompting Elicits Reas… 9 82% ▇▇▇▇▇▇▇▇▇▇
In-Context Demonstration Selection with… 2 5% ▇
Say What You Mean! Large Language Model… 1 4% ▇
Do Androids Laugh at Electric Sheep? Hu… 1 4% ▇
LMentry: A Language Model Benchmark of … 1 1% ▇
An Investigation of LLMs' Inefficacy in… 1 1% ▇
Causal Reasoning of Entities and Events… 1 1% ▇
Bridging the Resource Gap: Exploring th… 1 1% ▇
FANToM: A Benchmark for Stress-testing … 1 0% ▇
prior: pull toward the middle 1% ▇
weighted average = 12.0
Licences: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models ([CC BY 4.0](../../licenses/paper-2201.11903v6-CC-BY-4-0.txt)) · In-Context Demonstration Selection with Cross Entropy Difference ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-81-CC-BY-4-0.txt)) · Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge ([CC BY 4.0](../../licenses/acl-2023-acl-long-550-CC-BY-4-0.txt)) · Do Androids Laugh at Electric Sheep? Humor \`\`Understanding'' Benchmarks from The New Yorker Caption Contest ([CC BY 4.0](../../licenses/acl-2023-acl-long-41-CC-BY-4-0.txt)) · LMentry: A Language Model Benchmark of Elementary Language Tasks ([CC BY 4.0](../../licenses/acl-2023-findings-acl-666-CC-BY-4-0.txt)) · An Investigation of LLMs' Inefficacy in Understanding Converse Relations ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-429-CC-BY-4-0.txt)) · Causal Reasoning of Entities and Events in Procedural Texts ([CC BY 4.0](../../licenses/acl-2023-findings-eacl-31-CC-BY-4-0.txt)) · Bridging the Resource Gap: Exploring the Efficacy of English and Multilingual LLMs for Swedish ([CC BY 4.0](../../licenses/acl-2023-resourceful-1-13-CC-BY-4-0.txt)) · FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-890-CC-BY-4-0.txt))
## How to run it
No verified API or local download yet.
[interactive version](https://whatmodel.sh/model/text-curie-001) · [JSON](https://whatmodel.sh/api?format=json&all=1)