[whatmodel.sh](../../README.md) / OPT-350M
# OPT-350M
Meta · released 2022-05 · 0.35B params · opt-175b-license-noncommercial
**score 1.0** ±1.3 · #723 of 732 · from 23 benchmark results across 7 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks OPT-350M among every model on that task or test, worst to best.
other tests (13)
test result vs the others in its run difficulty counts why
ARC Easy 44.0% ▃·▅●▃▅·██▃██ beats 3/19 easy · avg 87% 24% blunt · at its level · several runs
SciQ 74.8% ▂··▂●▂▂▆▃▃█▂ beats 2/19 very easy · avg 98% 19% blunt · at its level · several runs
PIQA 64.4% ▂·▄●·▄▂▆▂▄█▄ beats 4/19 easy · avg 87% 16% blunt · at its level · several runs
LAMBADA (OpenAI version), last-wo… 45.2% ▃··▅▃▃●▅▅███ beats 5/19 very easy · avg 87% 11% blunt · at its level · several runs
WinoGrande 52.3% ▄····●▆▂▄▆▆█ beats 2/19 easy · avg 88% 8.0% too hard for it · several runs
ImageNetVC visual commonsense, te… 32.3% ▂·●·▂█▄▄█▄▂▆ beats 1/21 easy · avg 74% 6.6% blunt
Story Cloze Test 63.2% ▂▂●▂·▂·▆·▂█▂ beats 2/14 easy · avg 89% 6.1% blunt
ARC Challenge 20.7% ●··▁·▁▁·▁▂▁▂ beats 1/19 easy · avg 81% 5.6% sharp · too hard for it · several runs
COPA 72.0% ▁·▁▁▁▅●█▆▂▁▁ beats 18/67 very easy · avg 96% 2.4% at its level · small run
MC-TACO as yes/no plausibility cl… 65.0% █··········● beats 3/4 easy · avg 75% 0.4% blunt · at its level
Open PT-LLM Leaderboard average (… 23.3% ●▁▁▁▁▁▁▁▂█▅▁ beats 13/276 medium · avg 72% 0.3% too hard for it · noisy
SuperGLUE WSC (prompted) 36.5% ●·······▂·▂▃ beats 2/8 medium · avg 68% 0.2% blunt · too hard for it · noisy
LogiQA (harness) 21.0% ●··········▁ beats 6/14 very hard · avg 25% 0.0% too hard for it · noisy
source results share of the score
Pythia: A Suite for Analyzing Large Lan… 6 38% ▇▇▇▇▇
EleutherAI lm-evaluation-harness result… 8 26% ▇▇▇
GPTQ: Accurate Post-Training Quantizati… 5 25% ▇▇▇
ImageNetVC: Zero- and Few-Shot Visual C… 1 7% ▇
Revisiting Block-based Quantisation: Wh… 1 2% ▇
Do Language Models Have a Common Sense … 1 0% ▇
Open Portuguese LLM Leaderboard 1 0% ▇
prior: pull toward the middle 1% ▇
weighted average = 1.0
Licences: Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling ([CC BY-SA 4.0](../../licenses/paper-2304.01373v2-CC-BY-SA-4.0.txt)) · EleutherAI lm-evaluation-harness results folder ([MIT](../../licenses/lm-eval-harness-results-2023-05-MIT.txt)) · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers ([CC BY 4.0](../../licenses/paper-2210.17323v2-gptq-CC-BY-4-0.txt)) · ImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-133-CC-BY-4-0.txt)) · Revisiting Block-based Quantisation: What is Important for Sub-8-bit LLM Inference? ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-617-CC-BY-4-0.txt)) · Do Language Models Have a Common Sense regarding Time? Revisiting Temporal Commonsense Reasoning in the Era of Large Language Models ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-418-CC-BY-4-0.txt)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt))
## How to run it
### Other ways to run it
1. Run it as published (fp16, needs ~0.66 GB, from [huggingface.co/facebook](https://huggingface.co/facebook/opt-350m/tree/08ab08cc4b72ff5593870b5d527cf4230323703c))
```sh
uv run --with torch --with 'transformers<5' python -c "from transformers import pipeline; print(pipeline('text-generation', model='facebook/opt-350m')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"
```
Model repository and documentation: [huggingface.co/facebook](https://huggingface.co/facebook/opt-350m)
[interactive version](https://whatmodel.sh/model/opt-350m) · [JSON](https://whatmodel.sh/api?format=json&all=1)