[whatmodel.sh](../../README.md) / OPT-125M
# OPT-125M
Meta · released 2022-05 · 0.125B params · opt-175b-license-noncommercial
**score -3.2** ±1.2 · #728 of 732 · from 31 benchmark results across 7 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks OPT-125M among every model on that task or test, worst to best.
task vs the others rank counts tests
math ▁▁●▅█▆▄█▄▂▁▁ #396 of 431* 1.1% 1
math · #396 of 431 · 1.1% of its score
Beats [Yi 34B Chat](yi-34b-chat.md), [Command R (03-2024)](command-r-03-2024.md), [Molmo 7B-D (0924)](molmo-7b-d.md) here, though they rank higher overall. Trails [GPT-Neo 125M](gpt-neo-125m.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
MathQA 22.0% ●·▃▃█▅··▃·▃▃ beats 0/10 hard · avg 33% 1.1% blunt · too hard for it
other tests (16)
test result vs the others in its run difficulty counts why
ARC Easy 43.5% ▃·●▃▃▅·██▃██ beats 1/19 easy · avg 87% 23% blunt · several runs
SciQ 75.1% ▂··▂●▂▂▆▃▃█▂ beats 3/19 very easy · avg 98% 18% blunt · at its level · several runs
PIQA 63.0% ▂·●▄·▄▂▆▂▄█▄ beats 2/19 easy · avg 87% 14% blunt · at its level · several runs
LAMBADA (OpenAI version), last-wo… 37.9% ▃··▅●▃▅▅▅███ beats 3/19 very easy · avg 87% 12% blunt · at its level · several runs
WinoGrande 50.3% ●····▄▆▂▄▆▆█ beats 0/19 easy · avg 88% 7.0% too hard for it · several runs
ImageNetVC visual commonsense, te… 28.5% ●·▄·▂█▄▄█▄▂▆ beats 0/21 easy · avg 74% 5.3% blunt
Story Cloze Test 60.0% ●▂▄▂·▂·▆·▂█▂ beats 0/14 easy · avg 89% 5.1% blunt
COPA 66.0% ▁·▁▁▁●▅█▆▂▁▁ beats 8/67 very easy · avg 96% 4.2% at its level
ARC Challenge 18.9% ●··▁·▁▁·▁▂▁▂ beats 2/19 easy · avg 81% 4.0% sharp · too hard for it · several runs
HellaSwag 29.2% ●·······▄·█▄ beats 1/5 easy · avg 82% 3.5% too hard for it
VEC embodied concepts, zero-shot … 50.2% ●····▁▁·▁·▃█ beats 0/11 hard · avg 63% 0.6% too hard for it
PubMedQA 37.1% ●·····▅█▅▆▆█ beats 0/24 medium · avg 73% 0.4% blunt · at its level
Open PT-LLM Leaderboard average (… 24.4% ●▁▁▁▁▁▁▁▂█▅▁ beats 16/276 medium · avg 72% 0.2% too hard for it · noisy
SuperGLUE WSC (prompted) 36.5% ●·······▂·▂▃ beats 3/8 medium · avg 68% 0.2% blunt · too hard for it · noisy
VEC visual concepts, zero-shot pe… 62.5% ▅●▃█·▃··▃·▃▃ beats 2/11 hard · avg 66% 0.1% blunt
LogiQA (harness) 22.7% ●··········▁ beats 7/14 very hard · avg 25% 0.0% too hard for it · noisy
source results share of the score
Pythia: A Suite for Analyzing Large Lan… 6 33% ▇▇▇▇
EleutherAI lm-evaluation-harness result… 8 22% ▇▇▇
GPTQ: Accurate Post-Training Quantizati… 5 21% ▇▇▇
Preserving Privacy Through Dememorizati… 8 17% ▇▇
ImageNetVC: Zero- and Few-Shot Visual C… 1 5% ▇
Can Language Models Understand Physical… 2 1% ▇
Open Portuguese LLM Leaderboard 1 0% ▇
prior: pull toward the middle 1% ▇
weighted average = -3.2
Licences: Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling ([CC BY-SA 4.0](../../licenses/paper-2304.01373v2-CC-BY-SA-4.0.txt)) · EleutherAI lm-evaluation-harness results folder ([MIT](../../licenses/lm-eval-harness-results-2023-05-MIT.txt)) · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers ([CC BY 4.0](../../licenses/paper-2210.17323v2-gptq-CC-BY-4-0.txt)) · Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-265-CC-BY-4-0.txt)) · ImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-133-CC-BY-4-0.txt)) · Can Language Models Understand Physical Concepts? ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-726-CC-BY-4-0.txt)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt))
## How to run it
### Other ways to run it
1. Run it as published (fp16, needs ~0.25 GB, from [huggingface.co/facebook](https://huggingface.co/facebook/opt-125m/tree/27dcfa74d334bc871f3234de431e71c6eeba5dd6))
```sh
uv run --with torch --with 'transformers<5' python -c "from transformers import pipeline; print(pipeline('text-generation', model='facebook/opt-125m')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"
```
Model repository and documentation: [huggingface.co/facebook](https://huggingface.co/facebook/opt-125m)
[interactive version](https://whatmodel.sh/model/opt-125m) · [JSON](https://whatmodel.sh/api?format=json&all=1)