[whatmodel.sh](../../README.md) / gpt-oss-120b
# gpt-oss-120b
OpenAI · released 2025-08 · 117B params (5.1B active) · apache-2.0
**score 61.5** ±0.5 · #187 of 732 · from 59 benchmark results across 25 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks gpt-oss-120b among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█●▅▅▄▄▂▁ #159 of 313 18% 13
coding ▁▁▁▃▅▇█●▅▄▃▁ #164 of 469 24% 4
chat & writing ▁▁▁▃▆▄▆█●▇▇▃ #164 of 402* 2.9% 1
math ▁▁▃▅█▆▄●▄▂▁▁ #112 of 431 13% 5
web search █··········● #1 of 2* 0.8% 1
agent tasks · #159 of 313 · 18% of its score
Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o4-mini](o4-mini.md), [o3-mini](o3-mini.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Jamba 1.5 Large](jamba-1-5-large.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 31.0% █·●▄·······█ beats 3/6 hard · avg 30% 2.9% sharp · at its level · small run · seve…
τ²-bench Retail 61.4% ●▄··▄··▄·▄·▄ beats 1/6 medium · avg 36% 1.9% at its level
BrowserGym WorkArena-L1 (AgentLab… 50.9% █···█▄▄·●█·█ beats 6/10 hard · avg 28% 1.8% at its level
Terminal-Bench 2.0 with Terminus … 18.7% ▃··▃·▃●██▅▃▅ beats 3/14 very hard · avg 6.1% 1.7% sharp
BrowserGym WorkArena-L2 (AgentLab… 11.5% ▃▅▃·▅●▃··█·▃ beats 6/11 very hard · avg 3.6% 1.6% sharp · noisy
τ²-bench Airline 45.3% ●▄······▄█▄▄ beats 0/6 hard · avg 28% 1.4% at its level
Terminal-Bench Hard 22.0% ▂▅▆▂▂▆▂▂●▅▅▂ beats 16/27 very hard · avg 4.7% 1.4% sharp · small run
τ²-bench Telecom 57.7% ▁·▅█▆●▂▃▃▂▃▁ beats 49/89 very hard · avg 15% 1.3% sharp · at its level · noisy · small run
τ-bench Retail (maker) 67.8% ▁····▃·▂▇●▂▆ beats 16/28 medium · avg 37% 1.1% at its level · small run
BrowserGym MiniWoB (AgentLab Gene… 66.4% ▅····▅▅●▃·▃█ beats 6/11 medium · avg 53% 1.0% blunt · at its level
τ-bench Airline (maker) 49.2% ▁▁▁·▅▄▁█●▁▁▇ beats 21/28 hard · avg 25% 0.8% at its level · small run
Humanity's Last Exam with tools (… 19.0% ▁·▁▂●▁▄▄▇█▄▅ beats 6/62 very hard · avg 11% 0.8% small run
T1-Bench multi-domain conversatio… 29.3% ▂▂··▂·▂·▂●█▄ beats 5/11 hard · avg 25% 0.0% blunt · at its level · noisy
coding · #164 of 469 · 24% of its score
Beats [Qwen3-Coder-Next (80B-A3B)](qwen3-coder-next.md), [Gemini 2.5 Pro](gemini-2-5-pro.md), [Gemini 3.1 Flash-Lite](gemini-3-1-flash-lite.md) here, though they rank higher overall. Trails [QwQ-32B-Preview](qwq-32b-preview.md), [Ministral 3 8B Reasoning 2512](ministral-3-8b-reasoning.md), [Phi-4-reasoning 14B](phi-4-reasoning.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Aider polyglot (225 exercises, tw… 41.8% ▂·▂▃▅▂●█▆▄▂▂ beats 21/49 very hard · avg 7.6% 19% sharp · at its level · several runs
SWE-bench Verified 31.0% █·●▄·······█ beats 3/6 hard · avg 30% 2.9% sharp · at its level · small run · seve…
WebApp1K-Duo-React pass@1 (onekq-… 75.0% ▁··▁·▁▁▁▆█▇● beats 39/45 hard · avg 29% 1.7% at its level
LiveCodeBench v6 (Aug 2024–May 20… 69.9% ▄▄·▄·····●·▄ beats 4/5 medium · avg 45% 1.0% at its level
chat & writing · #164 of 402 · 2.9% of its score
Beats [MiniMax-M2](minimax-m2.md), [MiniCPM5 2B](minicpm5-2b.md), [Granite 4.2 30B](granite-4-2-30b.md) here, though they rank higher overall. Trails [LFM2.5 1.2B Thinking](lfm2-5-1-2b-thinking.md), [LFM2.5 8B-A1B](lfm2-5-8b-a1b.md), [LFM2.5 2.6B](lfm2-5-2-6b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LMArena (text) 1351.7 ▁▁▁▂▃▃▃▆●▇█▃ beats 195/385 avg 1260 2.9% blunt
math · #112 of 431 · 13% of its score
Beats [Claude Sonnet 4.6](claude-sonnet-4-6.md), [MiniMax-M3](minimax-m3.md), [Nemotron 3 Ultra 550B-A55B](nemotron-3-ultra-550b-a55b.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
AIME 2024 75.8% ●···███·█··█ beats 0/5 medium · avg 50% 4.8% sharp · several runs
AIME 2025 77.7% █·●···███··█ beats 1/5 medium · avg 42% 4.5% sharp · several runs
OTIS mock AIME 2024–2025 88.9% ▃▃▄▃▃▃▇▇●▃▂▅ beats 139/197 very hard · avg 7.2% 2.3% sharp · at its level · noisy
MathConstraint full release, no t… 39.5% ▄··▄▄·▄●█▄·▄ beats 4/8 very hard · avg 11% 0.5% typical
MATH-500 96.0% ▄······●▄▄·█ beats 1/5 very easy · avg 86% 0.3% maxed out · noisy
web search · #1 of 2 · 0.8% of its score
test result vs the others in its run difficulty counts why
Humanity's Last Exam with tools (… 19.0% ▁·▁▂●▁▄▄▇█▄▅ beats 6/62 very hard · avg 11% 0.8% small run
other tests (29)
test result vs the others in its run difficulty counts why
GPQA Diamond 75.8% ▁▁▁▁▂▄▆▄●▇█▅ beats 115/211 hard · avg 41% 13% sharp · at its level · several runs
Medmarks-V mean win rate (19 veri… 0.5771 ▁▄▃▃▇▄█▃●▁▁▁ beats 27/30 avg 0.418 5.3% sharp
Humanity's Last Exam 10.6% ●····▄·█·▄·▄ beats 0/5 very hard · avg 4.1% 3.5% several runs
Medmarks-V weighted mean win rate… 58.7% ▂▃▅▂▆▃█▅█●▃▆ beats 28/34 medium · avg 41% 2.3% blunt · at its level
KCL-Essay vanilla 31.6% ▁···▁▃▂●▂▃▃▂ beats 10/23 hard · avg 17% 2.1% at its level
BRIDGE clinical text, zero-shot a… 37.2% ▁▁▁▁▁▂▂▃▄●█▃ beats 55/97 hard · avg 31% 1.8% blunt · at its level
EQ-Bench 3 (emotional intelligenc… 1167.5 ▁·▂▁▄▅●▃█▄▁▁ beats 24/62 avg 1034 1.7% typical
AraGen-v3 3C3H score (Inception, … 43.2% ▁▁·▁▂▂▄█●▅▄▄ beats 37/66 hard · avg 29% 1.7% at its level
MMLU-Pro 67.1% ▁▁·▁▁▁▃▂●▅█▁ beats 80/188 medium · avg 50% 1.7% at its level
CPTUB average (SpeakLeash, 0-5 ju… 76.4% ▁▂▂▂▃█▃▆●▄▂▃ beats 73/95 medium · avg 65% 1.5% typical
MMLU (maker) 90.0% ▁▁▁▁▁▂▂▅██▆● beats 209/214 medium · avg 73% 1.3% small run
KCL-MCQA vanilla 29.2% ▂···●▂▆▆██▆▂ beats 3/23 very hard · avg 29% 1.0% at its level
LongSocialBench (full-context, 4-… 39.5% ▆▂·●█·▂▂·▄▄▂ beats 4/16 very hard · avg 36% 0.9% blunt
SimpleQA (maker) 16.8% ▄▄▄▆█▆●██·▂█ beats 17/32 very hard · avg 6.0% 0.9% small run
MMMLU (maker) 81.3% ▂▁▁▂▃▂▄▄▅●█▃ beats 31/46 medium · avg 64% 0.8% at its level · small run
Multi-IF (aggregation unspecified) 83.0% ▄·····▄●▄▄·▄ beats 2/6 easy · avg 66% 0.7% typical
SurgiQ surgical MCQ (zero-shot, l… 64.6% ▂···▂▂·▃▂▃█● beats 14/17 medium · avg 53% 0.7% at its level
Chess Puzzles (Epoch) 20.0% █▁▁▁▁▂●▃▃▂▁▁ beats 81/137 very hard · avg 5.8% 0.6% sharp · too hard for it · noisy
WMT25 Terminology Translation (te… 83.8% ▂·▂··▂·●▂·█▆ beats 3/12 medium · avg 63% 0.6% noisy
Mystery Game Puzzles (Epoch) 0.0% ●··▂▃▇▄█▂▄▂▁ beats 7/77 very hard · avg 9.4% 0.5% sharp · too hard for it · noisy
Open PT-LLM Leaderboard average (… 43.0% ▃▁▁▁▁▁●▁▂█▅▁ beats 52/276 medium · avg 72% 0.5% noisy
SOOHAK-Mini (702 problems, Avg@3) 61.0% ▄··▄·▄·●▄█·▄ beats 3/7 medium · avg 36% 0.4% blunt · at its level
SOOHAK Challenge (340 problems, A… 11.3% █·███●█··█·█ beats 4/7 very hard · avg 4.3% 0.3% typical
HealthBench (maker) 57.6% ▂▂·▂·▅●▃▂·▃▃ beats 8/17 hard · avg 32% 0.3% blunt · at its level · small run
WMT24++ 76.6% ●·▃·▅▅▃···▃▃ beats 0/10 easy · avg 70% 0.3% blunt
HealthBench Hard (maker) 30.0% ▁····▁▁●█·▁▁ beats 3/13 very hard · avg 13% 0.2% blunt · small run
WMT26 General MT (GEMBA-ESA, LLM … 72.3% ▄·▄▄●·▄██▄█▄ beats 3/12 medium · avg 65% 0.2% blunt · at its level
Vectara hallucination leaderboard… 97.6% ▁▁▂▃▄▄█▆●▂▂▁ beats 117/140 very easy · avg 94% 0.1% blunt · maxed out · noisy
Vectara hallucination leaderboard… 85.8% ▁▂▁●▅█▂▃▄▃▁▁ beats 13/98 very easy · avg 90% 0.0% blunt
source results share of the score
OpenAI (submitted / maker-reported) 14 18% ▇▇
Prime Intellect INTELLECT-3 model card:… 7 18% ▇▇
ServiceNow-AI/Apriel-1.6-15b-Thinker mo… 5 13% ▇▇
Epoch AI 4 8% ▇
Medmarks: A Comprehensive Open-Source L… 2 8% ▇
Aider polyglot benchmark runs 1 8% ▇
ServiceNow BrowserGym leaderboard 3 4% ▇
Korean Canonical Legal Benchmark: Towar… 2 3% ▇
LMArena leaderboard dataset 1 3% ▇
BRIDGE Medical Leaderboard 1 2% ▇
EQ-Bench 3 canonical leaderboard results 1 2% ▇
Terminal-Bench: Benchmarking Agents on … 1 2% ▇
Inception Arabic/Hindi leaderboards 1 2% ▇
WebApp1K models leaderboard 1 2% ▇
SpeakLeash CPTUB 1 1% ▇
Multiverse Computing HyperNova 60B 2602… 1 1% ▇
Solar Open Technical Report 1 1% ▇
Cohere 3 1% ▇
LongSocialBench: Do Long-Context LLMs U… 1 1% ▇
Soohak: A Mathematician-Curated Benchma… 2 1% ▇
SurgiQ: A Large-Scale Multi-Domain Benc… 1 1% ▇
MathConstraint: Automated Generation of… 1 1% ▇
Open Portuguese LLM Leaderboard 1 0% ▇
Vectara hallucination leaderboard 2 0% ▇
T1-Bench: Benchmarking Multi-Scenario A… 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 61.5
Licences: OpenAI ([CC BY 4.0](../../licenses/report-2508.10925v1.txt)) · Prime Intellect INTELLECT-3 model card: Evaluation table ([MIT](https://huggingface.co/PrimeIntellect/INTELLECT-3/blob/ff39d4a4688989f3f28868923d030c28e1b7d81c/README.md)) · ServiceNow-AI/Apriel-1.6-15b-Thinker model card ([MIT](../../licenses/cards-final-NOTICE.txt)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks ([CC BY 4.0](../../licenses/paper-2605.01417v1-CC-BY-4.0.txt)) · Aider polyglot benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) · ServiceNow BrowserGym leaderboard ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities ([CC BY 4.0](../../licenses/cards-final-NOTICE.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · BRIDGE Medical Leaderboard ([MIT](../../licenses/bridge-medical-leaderboard-MIT.txt)) · EQ-Bench 3 canonical leaderboard results ([MIT](../../licenses/eqbench3-MIT.txt)) · Terminal-Bench: Benchmarking Agents on Hard ([CC BY 4.0](../../licenses/paper-2601.11868v1-CC-BY-4.0.txt)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · WebApp1K models leaderboard ([MIT](../../licenses/webapp1k-leaderboard-MIT.txt)) · SpeakLeash CPTUB ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · Multiverse Computing HyperNova 60B 2602 model card: evaluation table ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Solar Open Technical Report ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Cohere ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · LongSocialBench: Do Long-Context LLMs Understand Online Discussion Threads? ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs ([CC BY-SA 4.0](../../licenses/paper-2605.09063v3-CC-BY-SA-4.0.txt)) · SurgiQ: A Large-Scale Multi-Domain Benchmark for Evaluating Surgical Understanding in Large Language Models ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · MathConstraint: Automated Generation of Verified Combinatorial Reasoning Instances for LLMs ([CC BY 4.0](../../licenses/paper-2605.08498v1-CC-BY-4.0.txt)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/))
## How to run it
### Through an API
api (Groq) · $0.26/M blended · $0.15 in / $0.6 out per 1M tokens · [console.groq.com](https://console.groq.com/docs/models)
[price source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/providers/groq/models/openai/gpt-oss-120b.toml): Standard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/models/openai/gpt-oss-120b.toml)
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run gpt-oss-120b (Q4_K_M, needs ~62.8 GB, from [huggingface.co/unsloth](https://huggingface.co/unsloth/gpt-oss-120b-GGUF/tree/ff1a82da6ad466e32284fa3d2b86694db3204789))
```sh
llama cli -hf unsloth/gpt-oss-120b-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf unsloth/gpt-oss-120b-GGUF:Q4_K_M
```
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run gpt-oss-120b (Q4_K_M, needs ~62.8 GB, from [huggingface.co/unsloth](https://huggingface.co/unsloth/gpt-oss-120b-GGUF/tree/ff1a82da6ad466e32284fa3d2b86694db3204789))
```sh
lms get https://huggingface.co/unsloth/gpt-oss-120b-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: 4bit (~65.8 GB, Mac only) `lms get https://huggingface.co/egoriow/gpt-oss-120b-mlx -y` · 8bit (~124.2 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/gpt-oss-120b-MLX-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run gpt-oss-120b (mxfp4, needs ~70.7 GB, from [ollama.com](https://ollama.com/library/gpt-oss/tags))
```sh
ollama run gpt-oss:120b
```
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run gpt-oss-120b (4bit, needs ~65.8 GB, Mac only, from [huggingface.co/egoriow](https://huggingface.co/egoriow/gpt-oss-120b-mlx/tree/8f7099ef0dbb0a41be31d7386d35c46146023c11))
```sh
mlx_lm.chat --model egoriow/gpt-oss-120b-mlx
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model egoriow/gpt-oss-120b-mlx
```
Other versions: 8bit (~124.2 GB, Mac only) `mlx_lm.chat --model lmstudio-community/gpt-oss-120b-MLX-8bit`
Model repository and documentation: [huggingface.co/openai](https://huggingface.co/openai/gpt-oss-120b)
## Published speed measurements
Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine.
hardware precision speed engine and settings
NVIDIA DGX Spark (GB10) MXFP4 58.72 tok/s llama.cpp 11fb327bf (7941), CUDA …
NVIDIA DGX Spark (GB10) MXFP4 55.67 tok/s llama.cpp 11fb327bf (7941), CUDA …
NVIDIA DGX Spark (GB10) MXFP4 42.76 tok/s llama.cpp 11fb327bf (7941), CUDA …
Apple M2 Ultra (GPU core co… MXFP4 85.6 tok/s llama.cpp b828e18c7 (7948), Metal…
Apple M2 Ultra (GPU core co… MXFP4 82.03 tok/s llama.cpp b828e18c7 (7948), Metal…
Apple M2 Ultra (GPU core co… MXFP4 65.52 tok/s llama.cpp b828e18c7 (7948), Metal…
NVIDIA DGX Spark (GB10) MXFP4 52.87 tok/s llama.cpp 11fb327bf (7941), CUDA …
NVIDIA DGX Spark (GB10) MXFP4 49.45 tok/s llama.cpp 11fb327bf (7941), CUDA …
Apple M2 Ultra (GPU core co… MXFP4 78.23 tok/s llama.cpp b828e18c7 (7948), Metal…
Apple M2 Ultra (GPU core co… MXFP4 73.2 tok/s llama.cpp b828e18c7 (7948), Metal…
AMD Ryzen AI Max+ 395 (Stri… mxfp4 55.57 tok/s llama.cpp 2496f9c14 (build_number…
AMD Ryzen AI Max+ 395 (Stri… mxfp4 50.59 tok/s llama.cpp 2496f9c14 (build_number…
AMD Ryzen AI Max+ 395 (Stri… mxfp4 51.02 tok/s llama.cpp 2496f9c14 (build_number…
AMD Ryzen AI Max+ 395 (Stri… MXFP4 (summary-… 33.66 tok/s llama.cpp 6c7e9a544 (commit from …
DGX Spark (GB10), 122 GB mxfp4 43.93 tok/s ollama (version not recorded) · l…
Each speed links to its source, which has the full settings.
[interactive version](https://whatmodel.sh/model/gpt-oss-120b) · [JSON](https://whatmodel.sh/api?format=json&all=1)