[whatmodel.sh](../../README.md) / gpt-oss-20b
# gpt-oss-20b
OpenAI · released 2025-08 · 21B params (3.6B active) · apache-2.0
**score 58.2** ±0.5 · #221 of 732 · from 65 benchmark results across 22 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks gpt-oss-20b among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆●▄▅▅▄▄▂▁ #215 of 313 11% 10
coding ▁▁▁▃▅▇█●▅▄▃▁ #173 of 469 12% 6
chat & writing ▁▁▁▃▆▄▆●▆▇▇▃ #205 of 402* 3.5% 2
math ▁▁▃▅█▆▄●▄▂▁▁ #145 of 431 21% 5
web search ●··········█ #2 of 2* 0.7% 1
agent tasks · #215 of 313 · 11% of its score
Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md), [DeepSeek R1 (Jan '25)](deepseek-r1.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Mellum2 12B-A2.5B Instruct](mellum2-12b-a2-5b-instruct.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
BrowserGym WorkArena-L1 (AgentLab… 38.5% █···●▄▄·▄█·█ beats 2/10 hard · avg 28% 1.8% at its level
SWE-bench Verified (maker) 60.7% ▁▁▁▁▁▁▂▄●▆▁▁ beats 59/138 hard · avg 30% 1.6% sharp · at its level · small run
Terminal-Bench 2.0 with Terminus … 3.1% ●··▃·▃▃██▅▃▅ beats 0/14 very hard · avg 6.1% 1.4% sharp
Terminal-Bench Hard 9.0% ▂▅▆▂▂●▂▂█▅▅▂ beats 12/27 very hard · avg 4.7% 1.3% sharp · small run
τ-bench Retail (maker) 54.8% ▁····▃·▂●█▂▆ beats 6/28 medium · avg 37% 1.2% at its level · small run
BrowserGym WorkArena-L2 (AgentLab… 2.6% ▃●▃·▅▃▃··█·▃ beats 2/11 very hard · avg 3.6% 1.2% sharp · noisy
BrowserGym MiniWoB (AgentLab Gene… 64.0% ▅····●▅▃▃·▃█ beats 3/11 medium · avg 53% 1.1% blunt · at its level
τ-bench Airline (maker) 38.0% ▁▁▁·▅●▁█▇▁▁▇ beats 8/28 hard · avg 25% 0.8% at its level · small run
Humanity's Last Exam with tools (… 17.3% ▁·▁●▂▁▄▄▇█▄▅ beats 4/62 very hard · avg 11% 0.7% small run
T1-Bench multi-domain conversatio… 8.6% ▂●··▂·▂·▂▂█▄ beats 1/11 hard · avg 25% 0.0% blunt · at its level · noisy
coding · #173 of 469 · 12% of its score
Beats [Qwen3-Coder-Next (80B-A3B)](qwen3-coder-next.md), [Gemini 2.5 Pro](gemini-2-5-pro.md), [Grok 4.1 Fast (Reasoning)](grok-4-1-fast-reasoning.md) here, though they rank higher overall. Trails [QwQ-32B-Preview](qwq-32b-preview.md), [Ministral 3 8B Reasoning 2512](ministral-3-8b-reasoning.md), [Phi-4-reasoning 14B](phi-4-reasoning.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Aider polyglot (225 exercises, tw… 34.2% ▂▁▁▃▆▃●▆█▃▄▂ beats 29/71 very hard · avg 8.3% 3.8% sharp · at its level · small run
Luau Leetcode fully solved (Torpe… 52.4% ▁▁▁▁·▁●▃▅▃█▇ beats 10/49 hard · avg 17% 2.4% sharp · at its level
LiveCodeBench v6 (window unspecif… 72.0% █·▄·▄····▄·● beats 5/5 medium · avg 41% 2.0% at its level
SWE-bench Verified (maker) 60.7% ▁▁▁▁▁▁▂▄●▆▁▁ beats 59/138 hard · avg 30% 1.6% sharp · at its level · small run
LiveCodeBench v6 (Feb–May 2025) 63.8% ▁▁▃▁█▅●▇▅▅▃█ beats 15/38 medium · avg 41% 1.3% at its level
SciCode main problems 6.2% ▄··▄····●▄·▄ beats 2/5 very hard · avg 3.8% 0.4% too hard for it
chat & writing · #205 of 402 · 3.5% of its score
Beats [Gemma 2 9B IT SimPO](gemma-2-9b-simpo.md), [Granite 4.2 8B](granite-4-2-8b.md), [Mistral Small 4 (119B-A6.5B)](mistral-small-4.md) here, though they rank higher overall. Trails [LFM2.5 1.2B Instruct](lfm2-5-1-2b-instruct.md), [LFM2.5 1.2B Thinking](lfm2-5-1-2b-thinking.md), [LFM2.5 8B-A1B](lfm2-5-8b-a1b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LMArena (text) 1317.3 ▁▁▁▂▃▃▃●▆▇█▃ beats 147/385 avg 1260 2.9% blunt
IFEval (aggregation unspecified) 92.8% ▁▂▄▂▂▄▆██●▂▁ beats 130/145 easy · avg 80% 0.6% typical
math · #145 of 431 · 21% of its score
Beats [Claude Sonnet 4.6](claude-sonnet-4-6.md), [MiMo-V2.5 (310B-A15B)](mimo-v2-5.md), [MiniMax-M2.7](minimax-m2-7.md) here, though they rank higher overall. Trails [Olmo 3 32B Think](olmo-3-32b-think.md), [Nemotron-Cascade-8B](nemotron-cascade-8b.md), [Olmo 3.1 32B Think](olmo-3-1-32b-think.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
AIME 2024 83.3% █▄···●·····▄ beats 3/5 medium · avg 62% 7.2% sharp · several runs
AIME 2025 84.4% ▄·█··▄····▄● beats 5/5 medium · avg 52% 7.1% sharp · several runs
HMMT 2025 (edition unspecified) 64.8% ▄··▄▄·▄····● beats 4/5 hard · avg 29% 3.3% sharp · at its level
OTIS mock AIME 2024–2025 50.8% ▃▃▄▃▃▃●▇█▃▂▅ beats 77/197 very hard · avg 7.2% 2.1% sharp · at its level · noisy
MATH-500 94.8% ●·▄·····█··█ beats 0/5 very easy · avg 92% 1.3% maxed out
web search · #2 of 2 · 0.7% of its score
test result vs the others in its run difficulty counts why
Humanity's Last Exam with tools (… 17.3% ▁·▁●▂▁▄▄▇█▄▅ beats 4/62 very hard · avg 11% 0.7% small run
other tests (30)
test result vs the others in its run difficulty counts why
GPQA Diamond 46.0% ▁▁▁▁▂▄●▄▅▇█▅ beats 52/211 hard · avg 39% 15% sharp · at its level · several runs
Medmarks-V mean win rate (19 veri… 0.5198 ▁▄▃▃▇▄●▃▅▁▁▁ beats 21/30 avg 0.418 5.4% sharp
Humanity's Last Exam 9.8% █··██···█·●█ beats 4/5 very hard · avg 3.7% 4.3% too hard for it · several runs
MMLU 81.7% ▁▁▁▁▁▂▂▅██●▄ beats 189/214 medium · avg 70% 3.6% several runs
MMLU-Pro 75.6% █··█··██●··█ beats 4/5 medium · avg 57% 3.6% at its level · several runs
Medmarks-V weighted mean win rate… 53.6% ▂▃▅▂▆▃█▅●▅▃▆ beats 21/34 medium · avg 41% 2.4% blunt · at its level
MMMLU 77.4% ▂▁▁▂▃▂▄▄●██▃ beats 25/46 medium · avg 64% 2.3% at its level · several runs
BRIDGE clinical text, zero-shot a… 29.0% ▁▁▁▁▁▂▂●▄▆█▃ beats 28/97 hard · avg 31% 1.8% blunt · at its level
EQ-Bench 3 (emotional intelligenc… 1111.5 ▁·▂▁▄●▅▃█▄▁▁ beats 14/62 avg 1034 1.8% typical
AraGen-v3 3C3H score (Inception, … 30.6% ▁▁·▁▂▂▄●▄▅▄▄ beats 27/66 hard · avg 29% 1.8% at its level
RobloxQA v2.0 multiple choice (To… 88.7% ▁▁▃●█▄▃▃▆▂▂▂ beats 8/49 easy · avg 77% 1.6% typical
SimpleQA 6.7% ▄▄▄●█▆███·▂█ beats 7/32 very hard · avg 6.0% 1.5% small run · several runs
Open Persian LLM Leaderboard v2 a… 45.5% ▂▄▂●█▄▃▆▃▅▂▂ beats 19/81 medium · avg 57% 1.2% blunt · at its level
SuperGPQA 50.1% ▂▁▂▂▅▆▃●▇▇█▃ beats 19/46 medium · avg 37% 1.0% at its level
Next Steps data preparation (bala… 77.8% ▄·▄···█·●▄·▄ beats 4/6 medium · avg 52% 0.9% at its level
LongSocialBench (full-context, 4-… 36.3% ●▂·▄█·▂▂·▄▄▂ beats 1/16 very hard · avg 36% 0.8% blunt
KOR-Bench 72.3% ▄·······█··● beats 3/3 medium · avg 55% 0.8% at its level
SurgiQ surgical MCQ (zero-shot, l… 57.6% ▂···▂▂·▃▂▃●█ beats 9/17 medium · avg 53% 0.7% at its level
MMLU-ProX 69.1% ▁·▁▁▁▂●▃▃█▅▄ beats 7/36 medium · avg 56% 0.7% at its level · small run
RULER 128k 78.7% ▃··▅▄█▅●▇·▃▃ beats 24/33 easy · avg 72% 0.5% typical
Open PT-LLM Leaderboard average (… 42.8% ▃▁▁▁▁▁●▁▂█▅▁ beats 51/276 medium · avg 72% 0.5% at its level · noisy
IFBench (aggregation unspecified) 69.4% ███···██···● beats 5/5 hard · avg 34% 0.5% at its level
EuroEval HellaSwag (English), tes… 67.7% ▃·▁▁▂▁▂▅●▇▄▁ beats 44/100 medium · avg 67% 0.4% sharp · noisy
Chess Puzzles (Epoch) 0.0% ●▁▁▁▁▂▃▃▃▂▁▁ beats 19/137 very hard · avg 5.8% 0.4% sharp · too hard for it · noisy
BeyondAIME 69.0% ▄·▂·▂▄··▄▄▄● beats 12/15 medium · avg 37% 0.3% at its level · noisy
HealthBench (maker) 42.5% ●▂·▂·▅█▃▂·▃▃ beats 0/17 hard · avg 32% 0.3% blunt · at its level · small run
EuroEval Life in the UK (English … 72.7% ▁▁▁▁▁▁▁▂●█▄▃ beats 42/146 easy · avg 82% 0.2% noisy
HealthBench Hard (maker) 10.8% ●····▁▁▂█·▁▁ beats 0/13 very hard · avg 13% 0.2% blunt · small run
Vectara hallucination leaderboard… 96.3% ▁▁▂▃▄▄█●▃▂▂▁ beats 95/140 very easy · avg 94% 0.1% blunt · maxed out · noisy
miniF2F (pass@1) 12.0% ▃·●▅·····▅·▃ beats 3/8 very hard · avg 14% 0.1% blunt · noisy · small run
source results share of the score
Falcon-H1R: Pushing the Reasoning Front… 10 22% ▇▇▇
ByteDance-Seed/Seed-OSS-36B-Instruct mo… 14 20% ▇▇
OpenAI (submitted / maker-reported) 14 19% ▇▇
Medmarks: A Comprehensive Open-Source L… 2 8% ▇
Epoch AI 3 7% ▇
Roblox LLM Leaderboard 2 4% ▇
ServiceNow BrowserGym leaderboard 3 4% ▇
LMArena leaderboard dataset 1 3% ▇
BRIDGE Medical Leaderboard 1 2% ▇
EQ-Bench 3 canonical leaderboard results 1 2% ▇
Inception Arabic/Hindi leaderboards 1 2% ▇
Terminal-Bench: Benchmarking Agents on … 1 1% ▇
Multiverse Computing HyperNova 60B 2602… 1 1% ▇
Open Persian LLM Leaderboard v2 1 1% ▇
Agentic coding without the cloud: evalu… 1 1% ▇
LongSocialBench: Do Long-Context LLMs U… 1 1% ▇
NVIDIA 2 1% ▇
SurgiQ: A Large-Scale Multi-Domain Benc… 1 1% ▇
EuroEval leaderboards 2 1% ▇
Open Portuguese LLM Leaderboard 1 1% ▇
Vectara hallucination leaderboard 1 0% ▇
T1-Bench: Benchmarking Multi-Scenario A… 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 58.2
Licences: Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling ([CC BY 4.0](../../licenses/report-2601.02346v1.txt)) · ByteDance-Seed/Seed-OSS-36B-Instruct model card ([apache-2.0](../../licenses/ByteDance-Seed--Seed-OSS-36B-Instruct.txt)) · OpenAI ([CC BY 4.0](../../licenses/report-2508.10925v1.txt)) · Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks ([CC BY 4.0](../../licenses/paper-2605.01417v1-CC-BY-4.0.txt)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Roblox LLM Leaderboard ([MIT](../../licenses/roblox-llm-leaderboard-MIT.txt)) · ServiceNow BrowserGym leaderboard ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · BRIDGE Medical Leaderboard ([MIT](../../licenses/bridge-medical-leaderboard-MIT.txt)) · EQ-Bench 3 canonical leaderboard results ([MIT](../../licenses/eqbench3-MIT.txt)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Terminal-Bench: Benchmarking Agents on Hard ([CC BY 4.0](../../licenses/paper-2601.11868v1-CC-BY-4.0.txt)) · Multiverse Computing HyperNova 60B 2602 model card: evaluation table ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Agentic coding without the cloud: evaluating open-weight LLMs on longitudinal data preparation tasks ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · LongSocialBench: Do Long-Context LLMs Understand Online Discussion Threads? ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · NVIDIA ([CC BY 4.0](../../licenses/report-2512.20848v1.txt)) · SurgiQ: A Large-Scale Multi-Domain Benchmark for Evaluating Surgical Understanding in Large Language Models ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/))
## How to run it
### Through an API
api (Groq) · $0.13/M blended · $0.075 in / $0.3 out per 1M tokens · [console.groq.com](https://console.groq.com/docs/models)
[price source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/providers/groq/models/openai/gpt-oss-20b.toml): Standard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/models/openai/gpt-oss-20b.toml)
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run gpt-oss-20b (Q4_K_M, needs ~11.6 GB, from [huggingface.co/unsloth](https://huggingface.co/unsloth/gpt-oss-20b-GGUF/tree/d449b42d93e1c2c7bda5312f5c25c8fb91dfa9b4))
```sh
llama cli -hf unsloth/gpt-oss-20b-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf unsloth/gpt-oss-20b-GGUF:Q4_K_M
```
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run gpt-oss-20b (Q4_K_M, needs ~11.6 GB, from [huggingface.co/unsloth](https://huggingface.co/unsloth/gpt-oss-20b-GGUF/tree/d449b42d93e1c2c7bda5312f5c25c8fb91dfa9b4))
```sh
lms get https://huggingface.co/unsloth/gpt-oss-20b-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: 4bit (~11.8 GB, Mac only) `lms get https://huggingface.co/InferenceIllusionist/gpt-oss-20b-MLX-4bit -y` · 8bit (~22.2 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/gpt-oss-20b-MLX-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run gpt-oss-20b (mxfp4, needs ~13.1 GB, from [ollama.com](https://ollama.com/library/gpt-oss/tags))
```sh
ollama run gpt-oss:20b
```
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run gpt-oss-20b (4bit, needs ~11.8 GB, Mac only, from [huggingface.co/InferenceIllusionist](https://huggingface.co/InferenceIllusionist/gpt-oss-20b-MLX-4bit/tree/ab41846cff2f90082e25af75b1eb217ae1668005))
```sh
mlx_lm.chat --model InferenceIllusionist/gpt-oss-20b-MLX-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model InferenceIllusionist/gpt-oss-20b-MLX-4bit
```
Other versions: 8bit (~22.2 GB, Mac only) `mlx_lm.chat --model lmstudio-community/gpt-oss-20b-MLX-8bit`
Model repository and documentation: [huggingface.co/openai](https://huggingface.co/openai/gpt-oss-20b)
## Published speed measurements
Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine.
hardware precision speed engine and settings
NVIDIA DGX Spark (GB10) MXFP4 83.43 tok/s llama.cpp 11fb327bf (7941), CUDA …
NVIDIA DGX Spark (GB10) MXFP4 79.22 tok/s llama.cpp 11fb327bf (7941), CUDA …
NVIDIA DGX Spark (GB10) MXFP4 61.65 tok/s llama.cpp 11fb327bf (7941), CUDA …
Apple M2 Ultra (GPU core co… MXFP4 129.97 tok/s llama.cpp b828e18c7 (7948), Metal…
Apple M2 Ultra (GPU core co… MXFP4 123.38 tok/s llama.cpp b828e18c7 (7948), Metal…
Apple M2 Ultra (GPU core co… MXFP4 98.25 tok/s llama.cpp b828e18c7 (7948), Metal…
NVIDIA DGX Spark (GB10) MXFP4 75.22 tok/s llama.cpp 11fb327bf (7941), CUDA …
NVIDIA DGX Spark (GB10) MXFP4 70.36 tok/s llama.cpp 11fb327bf (7941), CUDA …
Apple M2 Ultra (GPU core co… MXFP4 117.39 tok/s llama.cpp b828e18c7 (7948), Metal…
Apple M2 Ultra (GPU core co… MXFP4 109.75 tok/s llama.cpp b828e18c7 (7948), Metal…
AMD Ryzen AI Max+ 395 (Stri… MXFP4 (summary-… 47.36 tok/s llama.cpp 6c7e9a544 (commit from …
Radeon RX 9070 XT mxfp4 121.43 tok/s ollama (version not recorded) · l…
Apple M2 Pro, 32 GB mxfp4 39.3 tok/s ollama (version not recorded) · l…
Apple M4, 16 GB mxfp4 6.92 tok/s ollama (version not recorded) · l…
Apple M4, 16 GB mxfp4 20.52 tok/s ollama (version not recorded) · l…
Apple M4, 16 GB mxfp4 24.54 tok/s ollama (version not recorded) · l…
Apple M5 Max (40-core GPU),… mxfp4 101.39 tok/s ollama (version not recorded) · l…
Radeon 840M / 860M Graphics… mxfp4 · layers … 8.73 tok/s ollama (version not recorded) · l…
Each speed links to its source, which has the full settings.
[interactive version](https://whatmodel.sh/model/gpt-oss-20b) · [JSON](https://whatmodel.sh/api?format=json&all=1)