[whatmodel.sh](../../README.md) / DeepSeek R1 0528 # DeepSeek R1 0528 DeepSeek · released 2025-05 · 685B params (37B active) · mit **score 64.8** ±0.6 · #145 of 732 · from 51 benchmark results across 12 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks DeepSeek R1 0528 among every model on that task or test, worst to best.}

task            vs the others         rank  counts  tests
agent tasks     ▁▁▃▆●▄▅▅▄▄▂▁   #206 of 313     12%     13
coding          ▁▁▁▃▅▇█▇●▄▃▁   #124 of 469     39%      7
chat & writing  ▁▁▁▃▆▄▆█▆●▇▃   #91 of 402*    3.2%      1
math            ▁▁▃▅█▆▄●▄▂▁▁   #103 of 431     18%      6
web search      ●········█·█       #3 of 3    2.6%      3

agent tasks · #206 of 313 · 12% of its score

Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Mellum2 12B-A2.5B Instruct](mellum2-12b-a2-5b-instruct.md) here, though they rank lower overall.

test                                result  vs the others in its run   difficulty            counts  why
SWE-bench Verified                   29.6%  █·●▄·······█ beats 2/6     hard · avg 18%          3.5%  sharp · at its level · small run · seve…
τ²-bench Retail                      59.9%  ●▄··▄··▄·▄·▄ beats 0/6     medium · avg 36%        1.9%  at its level
τ²-bench Airline                     47.3%  ▄●······▄█▄▄ beats 1/6     hard · avg 28%          1.5%  at its level
BrowseComp-ZH (maker)                35.7%  ▁▁▁▁▁▃·●▄▅▄█ beats 7/23    hard · avg 15%          1.1%  at its level · small run
SWE-bench Multilingual (maker)       30.5%  ▁··▁●▁▅██▂▂▁ beats 6/65    hard · avg 17%          1.0%  at its level · small run
BFCL V4 (2025-08-26 archive, mode…   49.0%  ▄▄····█●···▄ beats 4/5     hard · avg 26%          1.0%  at its level
Humanity's Last Exam with tools (…   24.8%  ▁·▁▂▂●▄▄▇█▄▅ beats 10/62   very hard · avg 12%     0.8%  at its level · small run
BFCL v3                              63.8%  ▁▁▁▁▁▃▅█▅●▁▁ beats 92/114  medium · avg 51%        0.6%  blunt · at its level
BrowseComp (maker)                    8.9%  ▂·●▁▂▂▆▅█▂▂▄ beats 4/46    very hard · avg 12%     0.6%  at its level · small run
Terminal-Bench (pre-2.0) with Ter…    5.7%  ●····█·····█ beats 0/2     very hard · avg 7.3%    0.3%  noisy · small run
τ-bench Airline (GPT-4.1 user) (m…   53.5%  ●········█·█ beats 0/2                                —  not counted yet: no other model in the …
τ-bench Retail (GPT-4.1 user) (ma…   63.9%  █●·········█ beats 1/2                                —  not counted yet: no other model in the …
BFCL v3 multi-turn (maker)           37.0%  no other model yet                                    —  not counted yet: no other model in the …

coding · #124 of 469 · 39% of its score

Beats [Qwen3.5 Plus](qwen3-5-plus.md), [Qwen3-Coder-Next (80B-A3B)](qwen3-coder-next.md), [Gemini 2.5 Pro](gemini-2-5-pro.md) here, though they rank higher overall. Trails [Llama 3.1 Nemotron Ultra 253B v1](llama-3-1-nemotron-ultra-253b.md), [Solar Pro 4](solar-pro-4.md), [Laguna XS 2.1 (33B-A3B)](laguna-xs-2-1.md) here, though they rank lower overall.

test                                result  vs the others in its run  difficulty            counts  why
Aider polyglot (225 exercises, tw…   71.4%  ▂·▂▃▅▂▃█▆●▂▂ beats 39/49  very hard · avg 7.6%     26%  sharp · at its level · several runs
SWE-bench Verified                   29.6%  █·●▄·······█ beats 2/6    hard · avg 18%          3.5%  sharp · at its level · small run · seve…
Luau Leetcode fully solved (Torpe…   59.6%  ▁▁▁▁·▁▂●▅▃█▇ beats 14/49  hard · avg 17%          3.0%  sharp · at its level
LiveCodeBench v6 (Aug 2024–May 20…   62.5%  ▄●·▄·····█·▄ beats 1/5    medium · avg 45%        1.8%  at its level · several runs
WebApp1K-Duo-React pass@1 (onekq-…   68.5%  ▁··▁·▁▁▁▆█●█ beats 27/45  hard · avg 29%          1.7%  at its level
LiveCodeBench 2024-08–2025-05 (ve…   73.3%  ▃···█··●··▃▃ beats 4/7    medium · avg 43%        1.3%  at its level · small run · several runs
SWE-bench Multilingual (maker)       30.5%  ▁··▁●▁▅██▂▂▁ beats 6/65   hard · avg 17%          1.0%  at its level · small run

chat & writing · #91 of 402 · 3.2% of its score

Beats [LongCat 2.0 (1.6T-A48B)](longcat-2-0.md), [Inkling Small (266B-A12B)](inkling-small.md), [Qwen3.5 122B-A10B](qwen3-5-122b-a10b.md) here, though they rank higher overall. Trails [Gemma 4 12B](gemma-4-12b.md), [Gemma 4 26B-A4B](gemma-4-26b-a4b.md), [Mistral Medium 3.5 (128B)](mistral-medium-3-5.md) here, though they rank lower overall.

test            result  vs the others in its run    difficulty  counts  why
LMArena (text)  1421.5  ▁▁▁▂▃▃▃▆▆●█▃ beats 273/385  avg 1260      3.2%  blunt

math · #103 of 431 · 18% of its score

Beats [Claude Sonnet 4.6](claude-sonnet-4-6.md), [MiniMax-M3](minimax-m3.md), [Nemotron 3 Ultra 550B-A55B](nemotron-3-ultra-550b-a55b.md) here, though they rank higher overall. Trails [Gemini 2.5 Flash](gemini-2-5-flash.md), [GLM-4.5-Air](glm-4-5-air.md), [Granite 4.2 30B](granite-4-2-30b.md) here, though they rank lower overall.

test                                result  vs the others in its run   difficulty            counts  why
AIME 2024                            83.2%  █···●██·█··█ beats 1/5     medium · avg 50%        6.3%  sharp · several runs
AIME 2025                            73.4%  ●·█···███··█ beats 0/5     medium · avg 42%        5.3%  sharp · several runs
MATH level 5                         96.6%  ▂▂▅▆▄█▆▃▅▂▃● beats 86/92   hard · avg 28%          2.6%  sharp
OTIS mock AIME 2024–2025             66.4%  ▃▃▄▃▃▃●▇█▃▂▅ beats 93/197  very hard · avg 7.2%    2.2%  sharp · at its level · noisy
HMMT 2025 (edition unspecified) (…   79.4%  ▃█▇▅●▅▁·▁··▁ beats 25/32   hard · avg 21%          1.9%  sharp · at its level · noisy · small ru…
MATH-500                             87.3%  ●······▄▄▄·█ beats 0/5     very easy · avg 86%     0.3%  maxed out · noisy

web search · #3 of 3 · 2.6% of its score

test                                result  vs the others in its run  difficulty           counts  why
BrowseComp-ZH (maker)                35.7%  ▁▁▁▁▁▃·●▄▅▄█ beats 7/23   hard · avg 15%         1.1%  at its level · small run
Humanity's Last Exam with tools (…   24.8%  ▁·▁▂▂●▄▄▇█▄▅ beats 10/62  very hard · avg 12%    0.8%  at its level · small run
BrowseComp (maker)                    8.9%  ▂·●▁▂▂▆▅█▂▂▄ beats 4/46   very hard · avg 12%    0.6%  at its level · small run

other tests (10)

test                                result  vs the others in its run    difficulty            counts  why
GPQA Diamond                         76.3%  ▁▁▁▁▂▄▆▄●▇█▅ beats 118/211  hard · avg 41%           17%  sharp · at its level · several runs
Humanity's Last Exam                 15.9%  ▄····▄·█·●·▄ beats 4/5      very hard · avg 4.1%    5.5%  several runs
MMLU-Pro                             75.3%  ▁▁·▁▁▁▃▂▄▅●▁ beats 171/188  medium · avg 50%        2.8%  several runs
MMLU-Redux (maker)                   93.4%  ▁▁▃▁▂▃▄▅▇▅●▄ beats 56/63    easy · avg 80%          2.3%  small run · several runs
RobloxQA v2.0 multiple choice (To…   89.2%  ▁▁▃▁●▄▃▃▆▂▂▂ beats 11/49    easy · avg 77%          1.3%  typical
SimpleQA (maker)                     27.8%  ▄▄▄▆█▆██●·▂█ beats 26/32    very hard · avg 9.5%    1.1%  at its level · small run
Multi-IF (aggregation unspecified)   73.8%  ●·····▄█▄▄·▄ beats 0/6      easy · avg 66%          0.7%  typical
CNMO 2024 (maker)                    86.9%  █··········● beats 1/1      medium · avg 59%        0.6%  small run
FRAMES (maker)                       83.0%  █··········● beats 1/1      medium · avg 64%        0.6%  small run
Vectara hallucination leaderboard…   92.3%  ▁▁▂▃●▄█▆▃▂▂▁ beats 38/140   very easy · avg 94%     0.1%  blunt · maxed out · noisy

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
deepseek-ai/DeepSeek-R1-0528 model card…       13   18% ▇▇
Prime Intellect INTELLECT-3 model card:…        7   18% ▇▇
deepseek-ai/DeepSeek-V3.1 model card (s…       11   15% ▇▇
ServiceNow-AI/Apriel-1.6-15b-Thinker mo…        5   14% ▇▇
Epoch AI                                        3   10% ▇
Aider polyglot benchmark runs                   1    8% ▇
deepseek-ai/DeepSeek-R1-0528-Qwen3-8B m…        4    6% ▇
Roblox LLM Leaderboard                          2    4% ▇
LMArena leaderboard dataset                     1    3% ▇
WebApp1K models leaderboard                     1    2% ▇
Berkeley Function Calling Leaderboard r…        2    2% ▇
Vectara hallucination leaderboard               1    0% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 64.8

_{Licences: deepseek-ai/DeepSeek-R1-0528 model card ([mit](../../licenses/deepseek-ai--DeepSeek-R1-0528.txt)) · Prime Intellect INTELLECT-3 model card: Evaluation table ([MIT](https://huggingface.co/PrimeIntellect/INTELLECT-3/blob/ff39d4a4688989f3f28868923d030c28e1b7d81c/README.md)) · deepseek-ai/DeepSeek-V3.1 model card ([MIT](../../licenses/deepseek-ai--DeepSeek-V3.1.txt)) · ServiceNow-AI/Apriel-1.6-15b-Thinker model card ([MIT](../../licenses/cards-final-NOTICE.txt)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Aider polyglot benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) · deepseek-ai/DeepSeek-R1-0528-Qwen3-8B model card ([MIT](../../licenses/card-deepseek-ai-deepseek-r1-0528-qwen3-8b.txt)) · Roblox LLM Leaderboard ([MIT](../../licenses/roblox-llm-leaderboard-MIT.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · WebApp1K models leaderboard ([MIT](../../licenses/webapp1k-leaderboard-MIT.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))}

Not counted (3)

source                  test                            score  why
deepseek-ai/DeepSeek-…  BFCL v3 multi-turn              37.0%  too few models have it yet
deepseek-ai/DeepSeek-…  τ-bench Airline (GPT-4.1 user)  53.5%  too few models have it yet
deepseek-ai/DeepSeek-…  τ-bench Retail (GPT-4.1 user)   63.9%  too few models have it yet

## How to run it ### Through an API api (Deep Infra) · $0.91/M blended · $0.5 in / $2.15 out per 1M tokens · [deepinfra.com](https://deepinfra.com/models) _{[price source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/providers/deepinfra/models/deepseek-ai/DeepSeek-R1-0528.toml): Standard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://deepinfra.com/deepseek-ai/DeepSeek-R1-0528): DeepInfra serving limit for deepseek-ai/DeepSeek-R1-0528 (the already-priced provider): maximum context size 163,840 tokens, input + output combined, fp4 endpoint. Not an intrinsic model claim. Observed 2026-09-25.} ### llama.cpp (recommended) 1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md) ```sh # macOS / Linux curl -LsSf https://llama.app/install.sh | sh # Windows irm https://llama.app/install.ps1 | iex ``` 2. Run DeepSeek R1 0528 (ud-q2_k_xl, needs ~206 GB, from [huggingface.co/unsloth](https://huggingface.co/unsloth/DeepSeek-R1-0528-GGUF/tree/main)) ```sh llama cli -hf unsloth/DeepSeek-R1-0528-GGUF:UD-Q2_K_XL ``` 3. Or start a local server and open the address it prints (http://localhost:9931) ```sh llama serve -hf unsloth/DeepSeek-R1-0528-GGUF:UD-Q2_K_XL ``` Other versions: ud-iq1_s (~103.25 GB) `llama cli -hf unsloth/DeepSeek-R1-0528-GGUF:UD-IQ1_S` ### LM Studio 1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download) 2. Run DeepSeek R1 0528 (4bit, needs ~377.6 GB, Mac only, from [huggingface.co/mlx-community](https://huggingface.co/mlx-community/DeepSeek-R1-0528-4bit/tree/5b83525ead6d2f731d4149aa844aa541fa59b2f3)) ```sh lms get https://huggingface.co/mlx-community/DeepSeek-R1-0528-4bit -y ``` 3. Then run lms ls to see its name, and lms chat followed by that name Other versions: ud-q2_k_xl (~206 GB) `lms get https://huggingface.co/unsloth/DeepSeek-R1-0528-GGUF@UD-Q2_K_XL -y` · ud-iq1_s (~103.25 GB) `lms get https://huggingface.co/unsloth/DeepSeek-R1-0528-GGUF@UD-IQ1_S -y` · 8bit (~713.1 GB, Mac only) `lms get https://huggingface.co/mlx-community/DeepSeek-R1-0528-8bit -y` ### Ollama 1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download) ```sh # Linux curl -fsSL https://ollama.com/install.sh | sh ``` 2. Run DeepSeek R1 0528 (Q4_K_M, needs ~411.5 GB, from [ollama.com](https://ollama.com/library/deepseek-r1/tags)) ```sh ollama run deepseek-r1:671b-0528-q4_K_M ``` ### MLX (Mac) 1. Install MLX (Mac). Apple silicon Macs only, text only. ```sh # with uv uv tool install mlx-lm # or pip install mlx-lm ``` 2. Run DeepSeek R1 0528 (4bit, needs ~377.6 GB, Mac only, from [huggingface.co/mlx-community](https://huggingface.co/mlx-community/DeepSeek-R1-0528-4bit/tree/5b83525ead6d2f731d4149aa844aa541fa59b2f3)) ```sh mlx_lm.chat --model mlx-community/DeepSeek-R1-0528-4bit ``` 3. Or start an OpenAI-compatible server at http://localhost:8080/v1 ```sh mlx_lm.server --model mlx-community/DeepSeek-R1-0528-4bit ``` Other versions: 8bit (~713.1 GB, Mac only) `mlx_lm.chat --model mlx-community/DeepSeek-R1-0528-8bit` Model repository and documentation: [huggingface.co/deepseek-ai](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528) ## Quantization evidence How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent. They don't describe every download of that precision.

precision                       test                                full → quantized    kept
INT4 W4A16 (weights only; GPT…  GPQA Diamond pass@1 (LightEval/op…     0.796 → 0.806  101.2%
INT4 W4A16 (weights only; GPT…  MATH-500 pass@1 (LightEval/open-r…     0.975 → 0.974   99.9%
INT4 W4A16 (weights only; GPT…  AIME 2024 pass@1 (LightEval/open-…     0.887 → 0.873   98.5%

_{Each kept value links to its source.} [interactive version](https://whatmodel.sh/model/deepseek-r1-0528) · [JSON](https://whatmodel.sh/api?format=json&all=1)