[whatmodel.sh](../../README.md) / GPT-Neo 125M
# GPT-Neo 125M
EleutherAI · released 2021-03 · 0.125B params · mit
**score -3.4** ±2.2 · #729 of 732 · from 20 benchmark results across 6 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks GPT-Neo 125M among every model on that task or test, worst to best.
task vs the others rank counts tests
math ▁▁▃●█▆▄█▄▂▁▁ #381 of 431* 3.5% 1
math · #381 of 431 · 3.5% of its score
Beats [Yi 34B Chat](yi-34b-chat.md), [Llama 3 8B Instruct](llama-3-8b.md), [Command R (03-2024)](command-r-03-2024.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
MathQA 22.8% ▃·●▃█▅··▃·▃▃ beats 1/10 hard · avg 33% 3.5% blunt · too hard for it
other tests (19)
test result vs the others in its run difficulty counts why
COPA 64.0% ▁·▁▁●▅▅█▆▂▁▁ beats 6/67 very easy · avg 96% 14% at its level
COPEN concept identification in c… 28.8% ●···▃▃·▃·▃·█ beats 0/7 easy · avg 81% 12% too hard for it
HellaSwag 28.7% ●·······▄·█▄ beats 0/5 easy · avg 82% 11% too hard for it
ARC Easy 43.7% ●·······▄▄·█ beats 1/5 easy · avg 87% 11% blunt
CoNLL-2003 English NER, few-shot … 15.5% ●···▅·▃▅··▃█ beats 0/9 very easy · avg 85% 11% typical
LAMBADA (OpenAI version), last-wo… 37.4% ▁·····▁▁●▃██ beats 5/86 very easy · avg 87% 8.1% blunt · at its level
UD English-GUM UPOS tagging, few-… 64.3% ●·▄▄·······█ beats 0/4 very easy · avg 92% 6.5% blunt · at its level
WinoGrande 50.4% ▁●▁▁▂▁▁▅▇█▃▁ beats 4/131 easy · avg 85% 4.8% too hard for it
Natural Questions (closed book) 0.5% ●·▁·▁▂▁▂·▅▂█ beats 0/28 medium · avg 48% 4.0% too hard for it
PIQA 63.0% ▁●▁▂▂▃▅█▅▃▁▁ beats 4/123 easy · avg 87% 3.9% blunt · small run
ARC Challenge 19.1% ●·········▂▂ beats 3/5 medium · avg 60% 2.5% sharp · too hard for it · noisy
COPEN conceptual similarity judgm… 7.9% ▄···●···██·▄ beats 1/7 hard · avg 22% 1.5% blunt · too hard for it
PubMedQA 55.1% ▂·····▅●▅▆▆█ beats 4/24 medium · avg 73% 1.3% blunt · at its level
TruthfulQA MC2 45.6% ▅▃█●▁▂▅···▁▁ beats 21/31 medium · avg 57% 1.3% blunt · at its level
TriviaQA 1.6% ●··▁█▃▃▁▄▅▄█ beats 0/28 easy · avg 73% 1.1% too hard for it · noisy
Open PT-LLM Leaderboard average (… 25.6% ●▁▁▁▁▁▁▁▂█▅▁ beats 21/276 medium · avg 72% 0.7% too hard for it · noisy
TruthfulQA generation, % true (hu… 43.6% ██····█·██·● beats 5/5 medium · avg 42% 0.1% blunt · at its level
HaluEval QA hallucination detecti… 46.5% ●▁▁·▁▁·▁▁▁▁▁ beats 14/27 hard · avg 66% 0.1% too hard for it · noisy
COPEN conceptual property judgmen… 47.6% ●·······▁··▁ beats 2/7 very hard · avg 52% 0.1% blunt · too hard for it · noisy
source results share of the score
Preserving Privacy Through Dememorizati… 9 60% ▇▇▇▇▇▇▇
Prompting Language Models for Linguisti… 2 17% ▇▇
COPEN 3 13% ▇▇
Hallucinations Leaderboard results 4 6% ▇
Open Portuguese LLM Leaderboard 1 1% ▇
TruthfulQA: Measuring How Models Mimic … 1 0% ▇
prior: pull toward the middle 2% ▇
weighted average = -3.4
Licences: Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-265-CC-BY-4-0.txt)) · Prompting Language Models for Linguistic Structure ([CC BY 4.0](../../licenses/acl-2023-acl-long-367-CC-BY-4-0.txt)) · COPEN ([CC BY 4.0](../../licenses/copen-emnlp2022-zero-shot-CC-BY-4.0.txt)) · Hallucinations Leaderboard results ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · TruthfulQA: Measuring How Models Mimic Human Falsehoods ([CC BY 4.0](../../licenses/acl-2022-acl-long-229-CC-BY-4-0.txt))
## How to run it
### Other ways to run it
1. Run it as published (fp32, needs ~0.53 GB, from [huggingface.co/EleutherAI](https://huggingface.co/EleutherAI/gpt-neo-125m/tree/21def0189f5705e2521767faed922f1f15e7d7db))
```sh
uv run --with torch --with 'transformers<5' python -c "from transformers import pipeline; print(pipeline('text-generation', model='EleutherAI/gpt-neo-125m')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"
```
Model repository and documentation: [huggingface.co/EleutherAI](https://huggingface.co/EleutherAI/gpt-neo-125m)
[interactive version](https://whatmodel.sh/model/gpt-neo-125m) · [JSON](https://whatmodel.sh/api?format=json&all=1)