[whatmodel.sh](../../README.md) / OPT-175B # OPT-175B Meta · released 2022-05 · 175B params · opt-175b-license-noncommercial **score 32.8** ±1.0 · comparison only: no current run option verified · from 26 benchmark results across 4 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks OPT-175B among every model on that task or test, worst to best.
other tests (18) 
test                                result  vs the others in its run   difficulty           counts  why
ARC Challenge                        43.9%  █▃··▃··▅▃█▅● beats 13/14   easy · avg 81%          39%  sharp · at its level · several runs
WinoGrande                           72.5%  ▃·▃▃▃▅·●···▃ beats 7/9     easy · avg 87%          12%  at its level · several runs
ARC Easy                             71.0%  ▅▃▃▃▃▃▃▃▅·█● beats 14/14   easy · avg 83%         8.0%  blunt · at its level · several runs
Story Cloze Test                     79.8%  ▂▂▄▂·▂·▆·▂█● beats 14/14   easy · avg 89%         7.7%  blunt · at its level · several runs
MMLU (maker)                         34.2%  ▁▁▁▁▁●▂▅██▆▄ beats 18/214  medium · avg 59%       4.8%  small run
BIG-Bench Hard (maker)               30.2%  ▁▁▁●▂▄▅▆█▃▃▁ beats 11/155  medium · avg 59%       4.6%  sharp · noisy · small run
PIQA                                 81.1%  ▄·█▄·█▄▄▄██● beats 14/14   easy · avg 88%         4.0%  blunt · at its level · several runs
VEC embodied concepts, zero-shot …   55.8%  ▁····▁▁·▁·▃● beats 8/11    hard · avg 63%         3.7%  typical
RTE (prompted) (maker)               60.3%  ▅▃·▃··▅·▅●▃█ beats 8/13    medium · avg 75%       3.3%  small run
WiC (prompted) (maker)               56.6%  █·▂▁······●▁ beats 12/13   medium · avg 71%       3.0%  sharp · too hard for it · noisy · small…
LAMBADA (OpenAI version), last-wo…   75.6%  ▄▄▄█▄·█·▄██● beats 14/14   very easy · avg 87%    2.4%  blunt · at its level
COPA (maker)                         87.0%  ▁·▁▁▁▅▅█●▂▁▁ beats 50/67   very easy · avg 95%    2.0%  small run
BoolQ (maker)                        60.1%  ▁····▁▂●▆▅▆▄ beats 12/103  easy · avg 84%         1.8%  at its level · small run
OpenBookQA (maker)                   58.4%  ▅▁▁▁▂▂▃█●▁▃▁ beats 90/107  easy · avg 74%         1.7%  at its level · noisy · small run
Winograd Schema Challenge (WSC273…   85.3%  █▃·▅▃▃█▅█●▅▃ beats 16/21   easy · avg 92%         1.4%  blunt · at its level · small run
CommitmentBank (prompted) (maker)    46.4%  ▂▃·▂█●▂▂·▂·▂ beats 9/13    medium · avg 60%       0.4%  blunt · at its level · small run
SuperGLUE WSC (prompted) (maker)     51.4%  █··●·▃·▄▂▁▅▃ beats 8/26    medium · avg 70%       0.4%  blunt · noisy · small run
VEC visual concepts, zero-shot pe…   69.9%  ▅▅▃█·▃··▃·▃● beats 11/11   hard · avg 66%         0.1%  blunt · at its level
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
GPTQ: Accurate Post-Training Quantizati…        5   42% ▇▇▇▇▇
OPT-IML: Scaling Language Model Instruc…       15   36% ▇▇▇▇
Outlier Suppression+: Accurate quantiza…        4   19% ▇▇
Can Language Models Understand Physical…        2    4% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 32.8
Licences: GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers ([CC BY 4.0](../../licenses/paper-2210.17323v2-gptq-CC-BY-4-0.txt)) · OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization ([CC BY 4.0](../../licenses/paper-2212.12017v3-opt-iml-CC-BY-4-0.txt)) · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-102-CC-BY-4-0.txt)) · Can Language Models Understand Physical Concepts? ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-726-CC-BY-4-0.txt)) ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/opt-175b) · [JSON](https://whatmodel.sh/api?format=json&all=1)