[whatmodel.sh](../../README.md) / GPT-3 ada (API base model) # GPT-3 ada (API base model) OpenAI · released 2020-06 · proprietary **score 7.5** ±1.4 · comparison only: no current run option verified · from 30 benchmark results across 6 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks GPT-3 ada (API base model) among every model on that task or test, worst to best. 
task  vs the others          rank  counts  tests
math  ▁●▃▆██▅█▃▁▁▁   #511 of 530*    3.7%      2
math · #511 of 530 · 3.7% of its score Beats [Flan-T5 Large](flan-t5-large.md), [OPT-66B](opt-66b.md), [Flan-T5 Base](flan-t5-base.md) here, though they rank higher overall. Trails [GPT-Neo 125M](gpt-neo-125m.md), [OPT-125M](opt-125m.md), [GPT-2 XL (1.5B)](gpt-2-xl.md) here, though they rank lower overall. 
test    result  vs the others in its run  difficulty        counts  why
GSM8K     1.7%  ▅·▂●·······▂ beats 5/9    medium · avg 62%    1.9%  sharp · too hard for it · noisy
MathQA   24.3%  ▃·▃▃█●··▃·▃▃ beats 6/10   hard · avg 34%      1.9%  blunt · too hard for it
other tests (23) 
test                                result  vs the others in its run  difficulty           counts  why
HellaSwag                            43.4%  ●·█·█··▄▄··▄ beats 0/7    very easy · avg 91%     15%  at its level · several runs
WinoGrande                           52.9%  ●·▄·▄▄▄··█·▄ beats 0/7    easy · avg 87%          12%  several runs
LAMBADA (OpenAI version), last-wo…   51.6%  ●·▄··█··█··▄ beats 1/7    very easy · avg 88%     11%  blunt · at its level · several runs
LogiQA (harness)                     21.8%  ······●····· beats 3/5    very hard · avg 25%     10%  too hard for it
SciQ                                 84.3%  ●█····███··█ beats 0/5    very easy · avg 98%    8.3%  blunt
PIQA                                 70.5%  ●··▃··▃▃·▃·▃ beats 0/7    easy · avg 88%         6.7%  blunt · at its level · several runs
ARC Easy                             51.4%  ●···▄··█·▄·▄ beats 0/5    easy · avg 87%         5.4%  blunt · at its level
ARC Challenge                        22.5%  ●····▄···█▄▄ beats 0/5    easy · avg 78%         4.4%  sharp · too hard for it
TriviaQA                              5.0%  ●···█·███··█ beats 0/5    easy · avg 68%         4.1%  too hard for it
CommonsenseQA                        18.4%  ●····▁·····▁ beats 7/9    easy · avg 75%         2.4%  sharp · too hard for it
SVAMP                                 2.0%  █·●▂▄▂▂····▂ beats 6/12   easy · avg 70%         2.2%  sharp · too hard for it
QA4MRE 2013 (harness)                32.0%  ●····▄█▄···▄ beats 0/5    medium · avg 57%       2.0%  blunt
OpenBookQA                           17.2%  ●·······█··▃ beats 1/5    hard · avg 49%         1.9%  too hard for it
HEAD-QA (English, harness)           24.5%  ●··█····█··█ beats 0/3    medium · avg 58%       1.7%  blunt
ANLI Round 3 (prompted)              35.4%  ▃······▃●··▃ beats 2/5    hard · avg 43%         1.3%  too hard for it
LMentry accuracy (zero-shot)         20.3%  ●▆·▄▄█▂·▂··▂ beats 0/14   medium · avg 65%       1.2%  blunt · at its level
PROST (harness, micro accuracy)      25.4%  ●······█···▅ beats 0/5    very hard · avg 35%    1.1%  blunt · too hard for it
PubMedQA                             52.8%  ▂·····●█▅▆▆█ beats 2/24   medium · avg 74%       1.1%  blunt · at its level
StrategyQA                           43.0%  ●········▁▃▁ beats 5/9    hard · avg 62%         0.9%  too hard for it · noisy
SuperGLUE WSC (prompted)             37.5%  ●··········▄ beats 1/5    hard · avg 62%         0.9%  blunt · too hard for it
ANLI Round 2 (prompted)              34.2%  ██·█●█·····█ beats 3/5    very hard · avg 43%    0.8%  too hard for it · noisy
ANLI Round 1 (prompted)              33.4%  ●··▂·······▂ beats 2/5    hard · avg 46%         0.6%  sharp · too hard for it · noisy
AQuA-RAT                             15.8%  ●·····▁·▁··▁ beats 6/9    medium · avg 53%       0.0%  sharp · too hard for it · noisy
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
GPT-NeoX-20B: An Open-Source Autoregres…       17   62% ▇▇▇▇▇▇▇
EleutherAI GPT-J-6B model card                  4   23% ▇▇▇
Large Language Models Are Reasoning Tea…        5    7% ▇
EleutherAI GPT-Neo 2.7B model card              2    3% ▇
WinoDict: Probing language models for i…        1    2% ▇
LMentry: A Language Model Benchmark of …        1    1% ▇
prior: pull toward the middle                        1% ▇
weighted average                                   = 7.5
Licences: GPT-NeoX-20B: An Open-Source Autoregressive Language Model ([CC BY 4.0](../../licenses/paper-2204.06745v1-CC-BY-4.0.txt)) · EleutherAI GPT-J-6B model card ([Apache-2.0](../../licenses/eleutherai-gpt-j-6b-card-Apache-2.0.txt)) · Large Language Models Are Reasoning Teachers ([CC BY 4.0](../../licenses/acl-2023-acl-long-830-CC-BY-4-0.txt)) · EleutherAI GPT-Neo 2.7B model card ([MIT](../../licenses/eleutherai-gpt-neo-2-7b-card-MIT.txt)) · WinoDict: Probing language models for in-context word acquisition ([CC BY 4.0](../../licenses/acl-2023-eacl-main-7-CC-BY-4-0.txt)) · LMentry: A Language Model Benchmark of Elementary Language Tasks ([CC BY 4.0](../../licenses/acl-2023-findings-acl-666-CC-BY-4-0.txt)) ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/gpt-3-ada) · [JSON](https://whatmodel.sh/api?format=json&all=1)