[whatmodel.sh](../../README.md) / GPT-3 text-babbage-001 # GPT-3 text-babbage-001 OpenAI · released 2022-01 · proprietary **score 3.8** ±2.4 · comparison only: no current run option verified · from 12 benchmark results across 4 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks GPT-3 text-babbage-001 among every model on that task or test, worst to best. 
task  vs the others          rank  counts  tests
math  ●▁▃▆██▅█▃▁▁▁   #527 of 530*    4.6%      1
math · #527 of 530 · 4.6% of its score Trails [GPT-2 Small (124M)](gpt-2-small.md), [GPT-Neo 125M](gpt-neo-125m.md), [OPT-125M](opt-125m.md) here, though they rank lower overall. 
test   result  vs the others in its run  difficulty      counts  why
GSM8K    0.5%  ●·▅▃▃▃·▃·▃▃▅ beats 1/12   easy · avg 67%    4.6%  sharp · too hard for it
other tests (11) 
test                                result  vs the others in its run  difficulty           counts  why
MAWPS (CoT-paper union)               1.7%  ▅●█···▅·▃▃▃▅ beats 2/12   very easy · avg 94%     29%  sharp · too hard for it
ASDiv                                 1.4%  ▅●▃▅··▃▃·▃▃█ beats 2/12   easy · avg 83%          20%  sharp · too hard for it
SVAMP                                 1.7%  ▃●█···▅·▃▃▃▅ beats 2/12   easy · avg 82%          14%  sharp · too hard for it
BIG-bench Date Understanding          4.0%  ▃·●·▅▃▅·▃▃·█ beats 1/12   medium · avg 52%        11%  too hard for it
New Yorker Caption Contest matchi…   19.0%  ●·····▃··▅·▃ beats 1/6    medium · avg 71%       4.0%  sharp · too hard for it
StrategyQA                           45.8%  ●·····▅▃▃▅▃█ beats 1/12   medium · avg 70%       4.0%  too hard for it
CommonsenseQA                        12.0%  ●······▁·▁▁▃ beats 4/12   easy · avg 88%         3.0%  sharp · too hard for it
LMentry accuracy (zero-shot)         34.2%  ▂▆·●▄█▂·▂··▂ beats 4/14   medium · avg 65%       2.9%  blunt · at its level
BIG-bench Sports Understanding        0.0%  ●·····▃·▁·▄▁ beats 4/12   medium · avg 81%       2.7%  sharp · too hard for it
ConvRe Re2Text, normal relation, …   52.4%  ▂▂●··▂▂█▄··▂ beats 2/12   easy · avg 90%         1.9%  too hard for it · noisy
AQuA-RAT                              4.3%  ●▁·▁▁·····▂▁ beats 5/12   medium · avg 48%       0.3%  sharp · too hard for it
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
Chain-of-Thought Prompting Elicits Reas…        9   89% ▇▇▇▇▇▇▇▇▇▇▇
Do Androids Laugh at Electric Sheep? Hu…        1    4% ▇
LMentry: A Language Model Benchmark of …        1    3% ▇
An Investigation of LLMs' Inefficacy in…        1    2% ▇
prior: pull toward the middle                        3% ▇
weighted average                                   = 3.8
Licences: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models ([CC BY 4.0](../../licenses/paper-2201.11903v6-CC-BY-4-0.txt)) · Do Androids Laugh at Electric Sheep? Humor \`\`Understanding'' Benchmarks from The New Yorker Caption Contest ([CC BY 4.0](../../licenses/acl-2023-acl-long-41-CC-BY-4-0.txt)) · LMentry: A Language Model Benchmark of Elementary Language Tasks ([CC BY 4.0](../../licenses/acl-2023-findings-acl-666-CC-BY-4-0.txt)) · An Investigation of LLMs' Inefficacy in Understanding Converse Relations ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-429-CC-BY-4-0.txt)) ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/text-babbage-001) · [JSON](https://whatmodel.sh/api?format=json&all=1)