[whatmodel.sh](../../README.md) / GPT-2 Large (774M)
# GPT-2 Large (774M)
OpenAI · released 2019-08 · 0.774B params · mit
**score -3.0** ±1.3 · #727 of 732 · from 35 benchmark results across 9 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks GPT-2 Large (774M) among every model on that task or test, worst to best.
task vs the others rank counts tests
math ●▁▃▅█▆▄█▄▂▁▁ #431 of 431* 0.4% 1
math · #431 of 431 · 0.4% of its score
Trails [GPT-2 Small (124M)](gpt-2-small.md), [GPT-2 Medium (355M)](gpt-2-medium.md), [GPT-Neo 125M](gpt-neo-125m.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
GSM8K 0.0% ●·▂█·······▂ beats 0/9 medium · avg 62% 0.4% sharp · too hard for it · noisy
other tests (21)
test result vs the others in its run difficulty counts why
OpenBookQA 19.4% ●▁▁▁▂▂▃█▃▁▃▁ beats 11/107 very hard · avg 26% 20% too hard for it · small run · several r…
ARC Challenge 23.1% ●▁▁▂▁▄█▇▃▂▃▂ beats 3/140 very hard · avg 27% 19% sharp · too hard for it · several runs
COPA 69.8% ▁·▁▁▁●▅█▆▂▁▁ beats 16/67 very easy · avg 97% 13% at its level · several runs
HellaSwag 35.0% ▁▁▁●▂▂▄▆█▇▄▂ beats 8/135 easy · avg 87% 10% several runs
ARC Easy 52.7% ▁▃▂●▆█▅▃▁··▁ beats 18/100 very easy · avg 90% 9.1% blunt · at its level · several runs
Story Cloze Test 66.1% ▃▃●▂▅█▅▂▁··▂ beats 10/50 easy · avg 91% 8.4% blunt · at its level · several runs
PROST physical reasoning (zero-sh… 26.4% █···█●·█···█ beats 2/4 easy · avg 86% 5.5% too hard for it
COPEN concept identification in c… 33.8% ▃···▃▃·●·▃·█ beats 3/7 easy · avg 81% 3.8% too hard for it
12-dataset text classification, 4… 36.4% ●▂··▂·█···▂▂ beats 0/8 easy · avg 79% 2.8% blunt · at its level
Social IQA 42.0% ▁·▁█●▄▅▁▂▃▇▁ beats 17/59 medium · avg 71% 2.3% blunt
PIQA 70.3% ▁▁▁▂●▃▅█▅▃▁▁ beats 20/123 easy · avg 89% 1.8% blunt · at its level
BoolQ 61.2% ▁····▁▂●▆▅▆▄ beats 19/103 easy · avg 91% 1.7% noisy
COPEN conceptual similarity judgm… 11.3% ▄···█···█●·▄ beats 5/7 hard · avg 22% 0.5% blunt · too hard for it
SVAMP 0.0% ●·▆▂▄▂▂····▂ beats 0/12 easy · avg 70% 0.4% sharp · too hard for it
CommonsenseQA 0.2% ●····▁·····▁ beats 0/9 easy · avg 75% 0.4% sharp · too hard for it · several runs
StrategyQA 35.1% ●········▁▃▁ beats 0/9 hard · avg 62% 0.4% too hard for it · noisy
Open PT-LLM Leaderboard average (… 27.4% ●▁▁▁▁▁▁▁▂█▅▁ beats 25/276 medium · avg 72% 0.2% too hard for it · noisy
EuroEval Life in the UK (English … 26.5% ▁▁●▁▁▁▁▂▄█▄▃ beats 3/146 easy · avg 82% 0.2% too hard for it · noisy
EuroEval HellaSwag (English), tes… 25.0% ●·▁▁▂▁▂▅█▇▄▁ beats 2/100 medium · avg 67% 0.0% sharp · too hard for it · noisy
COPEN conceptual property judgmen… 47.2% ●·······▁··▁ beats 4/7 very hard · avg 52% 0.0% blunt · too hard for it · noisy
AQuA-RAT 0.4% ●·····▁·▁··▁ beats 0/9 medium · avg 53% 0.0% sharp · too hard for it · noisy
source results share of the score
Coherence boosting: When your pretraine… 9 34% ▇▇▇▇
Surface Form Competition 7 26% ▇▇▇
CASE: Commonsense-Augmented Score with … 6 25% ▇▇▇
PROST 1 6% ▇
COPEN 3 4% ▇
Generative Calibration for In-context L… 1 3% ▇
Large Language Models Are Reasoning Tea… 5 2% ▇
EuroEval leaderboards 2 0% ▇
Open Portuguese LLM Leaderboard 1 0% ▇
prior: pull toward the middle 1% ▇
weighted average = -3.0
Licences: Coherence boosting: When your pretrained language model is not paying enough attention ([CC BY 4.0](../../licenses/acl-2022-acl-long-565-CC-BY-4-0.txt)) · Surface Form Competition ([CC BY 4.0](../../licenses/acl-2021-emnlp-main-564-CC-BY-4.0.txt)) · CASE: Commonsense-Augmented Score with an Expanded Answer Space ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-180-CC-BY-4-0.txt)) · PROST ([CC BY 4.0](../../licenses/_cc-by-4.0-legalcode.txt)) · COPEN ([CC BY 4.0](../../licenses/copen-emnlp2022-zero-shot-CC-BY-4.0.txt)) · Generative Calibration for In-context Learning ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-152-CC-BY-4-0.txt)) · Large Language Models Are Reasoning Teachers ([CC BY 4.0](../../licenses/acl-2023-acl-long-830-CC-BY-4-0.txt)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt))
## How to run it
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run GPT-2 Large (774M) (Q4_K_M, needs ~0.5 GB, from [huggingface.co/mradermacher](https://huggingface.co/mradermacher/gpt2-large-GGUF/tree/c83630987bcee4945f3e947b1977c70e6b2760a1))
```sh
llama cli -hf mradermacher/gpt2-large-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf mradermacher/gpt2-large-GGUF:Q4_K_M
```
Other versions: Q8_0 (~0.9 GB) `llama cli -hf mradermacher/gpt2-large-GGUF:Q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run GPT-2 Large (774M) (Q4_K_M, needs ~0.5 GB, from [huggingface.co/mradermacher](https://huggingface.co/mradermacher/gpt2-large-GGUF/tree/c83630987bcee4945f3e947b1977c70e6b2760a1))
```sh
lms get https://huggingface.co/mradermacher/gpt2-large-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~0.9 GB) `lms get https://huggingface.co/mradermacher/gpt2-large-GGUF@Q8_0 -y` · fp16 (~1.5 GB, Mac only) `lms get https://huggingface.co/MCES10/gpt2-large-mlx-fp16 -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run GPT-2 Large (774M) (Q4_K_M, needs ~0.5 GB, from [huggingface.co/mradermacher](https://huggingface.co/mradermacher/gpt2-large-GGUF/tree/c83630987bcee4945f3e947b1977c70e6b2760a1))
```sh
ollama run hf.co/mradermacher/gpt2-large-GGUF:Q4_K_M
```
Other versions: Q8_0 (~0.9 GB) `ollama run hf.co/mradermacher/gpt2-large-GGUF:Q8_0`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run GPT-2 Large (774M) (fp16, needs ~1.5 GB, Mac only, from [huggingface.co/MCES10](https://huggingface.co/MCES10/gpt2-large-mlx-fp16/tree/2ae98124ef6dcda29918ea2a73f189eecfeb417a))
```sh
mlx_lm.chat --model MCES10/gpt2-large-mlx-fp16
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model MCES10/gpt2-large-mlx-fp16
```
### Other ways to run it
1. Run it as published (fp32, needs ~3.25 GB, from [huggingface.co/openai-community](https://huggingface.co/openai-community/gpt2-large/tree/32b71b12589c2f8d625668d2335a01cac3249519))
```sh
uv run --with torch --with 'transformers<5' python -c "from transformers import pipeline; print(pipeline('text-generation', model='openai-community/gpt2-large')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"
```
Model repository and documentation: [huggingface.co/openai-community](https://huggingface.co/openai-community/gpt2-large)
## Published speed measurements
Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine.
hardware precision speed engine and settings
Apple M2 Pro (16-core GPU),… Q4_K_M 137.11 tok/s llama.cpp · whatmodel bench.py 3.…
Each speed links to its source, which has the full settings.
[interactive version](https://whatmodel.sh/model/gpt-2-large) · [JSON](https://whatmodel.sh/api?format=json&all=1)