[whatmodel.sh](../../README.md) / DeepSeek V3.1
# DeepSeek V3.1
DeepSeek · released 2025-08 · 685B params (37B active) · mit
**score 65.3** ±0.6 · #138 of 732 · from 66 benchmark results across 9 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks DeepSeek V3.1 among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█●▅▅▄▄▂▁ #155 of 313 21% 11
coding ▁▁▁▃▅▇█▇●▄▃▁ #113 of 469 22% 7
chat & writing ▁▁▁▃▆▄▆█▆●▇▃ #99 of 402 5.4% 3
math ▁▁▃▅█▆▄●▄▂▁▁ #98 of 431 16% 4
web search █········●·█ #2 of 3 4.6% 3
agent tasks · #155 of 313 · 21% of its score
Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o4-mini](o4-mini.md), [Qwen3 235B-A22B Thinking 2507](qwen3-235b-a22b-thinking-2507.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Jamba 1.5 Large](jamba-1-5-large.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
τ²-bench Retail 64.9% ●···█·▃····▃ beats 1/6 hard · avg 35% 4.2% at its level · several runs
SWE-bench Verified (maker) 66.0% ▁▁▁▁▁▁▂▄●▆▁▁ beats 69/138 hard · avg 23% 3.5% sharp · at its level · small run · seve…
τ²-bench Telecom 38.5% ██··█●█···██ beats 3/6 very hard · avg 8.8% 3.0% sharp · at its level · noisy · several …
BrowseComp-ZH (maker) 49.2% ▁▁▁▁▁▃·▁▄●▄█ beats 12/23 hard · avg 17% 2.3% at its level · small run · several runs
τ²-bench Airline 46.0% ▄·▄··●···█▄▄ beats 2/6 hard · avg 28% 2.3% at its level · several runs
SWE-bench Multilingual (maker) 54.5% ▁··▁▃▁●██▂▂▁ beats 15/65 hard · avg 23% 2.2% at its level · small run · several runs
BrowseComp (maker) 30.0% ▂·▃▁▂●▆▅█▂▂▄ beats 10/46 hard · avg 15% 1.5% at its level · small run · several runs
Humanity's Last Exam with tools (… 29.8% ▁·▁▂▂▁●▄▇█▄▅ beats 12/62 very hard · avg 12% 0.8% at its level · small run
Terminal-Bench (pre-2.0 release) … 31.3% ▃·▃···▃▃●·▅█ beats 5/10 very hard · avg 14% 0.7% at its level · small run
BFCL v3 55.4% ▁▁▁▁▁▃▅●▆▅▄▁ beats 69/171 medium · avg 54% 0.3% blunt · at its level
Terminal-Bench (pre-2.0) with Ter… 31.3% █····█·····● beats 2/2 very hard · avg 7.3% 0.3% noisy · small run
coding · #113 of 469 · 22% of its score
Beats [Grok 4.3](grok-4-3.md), [Muse Glimmer 30B](muse-glimmer-30b.md), [Qwen3.5 Plus](qwen3-5-plus.md) here, though they rank higher overall. Trails [Llama 3.1 Nemotron Ultra 253B v1](llama-3-1-nemotron-ultra-253b.md), [Solar Pro 4](solar-pro-4.md), [Nemotron Cascade 2 30B-A3B](nemotron-cascade-2-30b-a3b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Aider polyglot (225 exercises, tw… 76.3% ▂▁▁▃▆▃▄▆█▃●▂ beats 64/71 very hard · avg 13% 10% sharp · small run · several runs
SWE-bench Verified (maker) 66.0% ▁▁▁▁▁▁▂▄●▆▁▁ beats 69/138 hard · avg 23% 3.5% sharp · at its level · small run · seve…
LiveCodeBench (version unspecifie… 73.5% ▄······●█▄·▄ beats 1/5 medium · avg 41% 2.7% at its level · several runs
SWE-bench Multilingual (maker) 54.5% ▁··▁▃▁●██▂▂▁ beats 15/65 hard · avg 23% 2.2% at its level · small run · several runs
MBPP+ 79.9% ▃·····▃●▃··▃ beats 4/6 medium · avg 60% 1.6% typical
HumanEval+ 92.7% ▄▄▄·····●▄·█ beats 3/6 easy · avg 66% 1.5% sharp · maxed out
LiveCodeBench 2024-08–2025-05 (ve… 74.8% ▃···█··●··▃▃ beats 5/7 medium · avg 41% 0.6% at its level · small run
chat & writing · #99 of 402 · 5.4% of its score
Beats [LongCat 2.0 (1.6T-A48B)](longcat-2-0.md), [Inkling Small (266B-A12B)](inkling-small.md), [GPT-5.4 nano](gpt-5-4-nano.md) here, though they rank higher overall. Trails [Gemma 4 12B](gemma-4-12b.md), [Kimi K2 Instruct (0905)](kimi-k2-0905.md), [Gemma 4 26B-A4B](gemma-4-26b-a4b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LMArena (text) 1415.8 ▁▁▁▂▃▃▃▆▆●█▃ beats 267/385 avg 1260 3.2% blunt
IFEval (aggregation unspecified) 86.7% █··█·●··████ beats 2/6 easy · avg 74% 1.2% typical
IFEval prompt-level strict 86.3% █●█··█·█···█ beats 1/5 easy · avg 73% 1.0% typical
math · #98 of 431 · 16% of its score
Beats [Claude Sonnet 4.6](claude-sonnet-4-6.md), [MiniMax-M3](minimax-m3.md), [Nemotron 3 Ultra 550B-A55B](nemotron-3-ultra-550b-a55b.md) here, though they rank higher overall. Trails [Gemini 2.5 Flash](gemini-2-5-flash.md), [Granite 4.2 30B](granite-4-2-30b.md), [Qwen3-Next 80B-A3B Instruct](qwen3-next-80b-a3b-instruct.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
AIME 2025 49.3% ▄▄···●▄··▄·█ beats 2/6 medium · avg 50% 6.5% sharp · several runs
AIME 2024 93.9% ▁▁▁▂▁▃▁▅███● beats 84/85 medium · avg 61% 4.1% sharp · several runs
HMMT 2025 (edition unspecified) 80.4% ▄··▄··▄·●··█ beats 3/5 hard · avg 31% 3.7% sharp · several runs
MATH-500 96.1% ▄·▄··●··▄·▄▄ beats 2/6 very easy · avg 92% 1.3% maxed out · noisy · several runs
web search · #2 of 3 · 4.6% of its score
test result vs the others in its run difficulty counts why
BrowseComp-ZH (maker) 49.2% ▁▁▁▁▁▃·▁▄●▄█ beats 12/23 hard · avg 17% 2.3% at its level · small run · several runs
BrowseComp (maker) 30.0% ▂·▃▁▂●▆▅█▂▂▄ beats 10/46 hard · avg 15% 1.5% at its level · small run · several runs
Humanity's Last Exam with tools (… 29.8% ▁·▁▂▂▁●▄▇█▄▅ beats 12/62 very hard · avg 12% 0.8% at its level · small run
other tests (19)
test result vs the others in its run difficulty counts why
GPQA Diamond 84.2% ▄··▄··▄▄···● beats 5/5 hard · avg 42% 10% sharp · at its level · several runs
MMLU-Pro 84.5% █▄·▄···▄··●▄ beats 5/6 medium · avg 58% 7.3% several runs
MMLU 91.0% ▄·····█▄▄●·▄ beats 5/6 medium · avg 68% 3.4% typical
MMLU-Redux 90.5% ███●··█····█ beats 3/5 medium · avg 73% 3.3% several runs
Humanity's Last Exam (maker) 15.9% ▁▁▁▄▆●▇██▆▁▃ beats 40/111 very hard · avg 4.4% 2.5% small run · several runs
VitaBench 20.3% █·······▄●·█ beats 4/6 very hard · avg 8.2% 2.4% several runs
ZebraLogic 85.3% █···█··●██·█ beats 2/5 easy · avg 75% 2.2% maxed out · several runs
CPTUB average (SpeakLeash, 0-5 ju… 80.5% ▁▂▂▂▃█▃▆▅●▂▃ beats 83/95 medium · avg 65% 1.4% typical
CMMLU 88.0% █····▄··▄●▄▄ beats 4/6 medium · avg 67% 1.4% typical
OJBench 33.6% ▄······▄●·▄█ beats 2/5 very hard · avg 12% 1.3% at its level
BeyondAIME 36.5% █····●·▄▄··▄ beats 2/6 medium · avg 42% 1.3% at its level · several runs
C-Eval 89.2% █····▄·●·▄▄▄ beats 3/6 medium · avg 68% 1.2% typical
COLLIE (constrained text generati… 43.8% ●···▄█▄····█ beats 0/6 hard · avg 30% 1.2% at its level
RPC-Bench overall, text input (Z.… 30.8% ▄··▄▄·▄●█·▄▄ beats 4/8 very hard · avg 12% 1.2% at its level
Open PT-LLM Leaderboard average (… 82.6% ▃▁▁▁▁▁▁▁▂█▅● beats 270/276 medium · avg 72% 0.4% noisy
DROP 84.2% ▄···▄·▄█·●·▄ beats 5/6 medium · avg 58% 0.1% blunt · at its level · noisy
miniF2F (pass@1) 49.6% ██······●··▄ beats 4/5 very hard · avg 14% 0.1% blunt · at its level · noisy
Vectara hallucination leaderboard… 94.5% ▁▁▂▃▄●█▆▃▂▂▁ beats 54/140 very easy · avg 94% 0.1% blunt · maxed out · noisy
Vectara hallucination leaderboard… 94.5% ▁▂▁▂▅█▂▃●▃▁▁ beats 82/98 very easy · avg 90% 0.0% blunt
source results share of the score
Meituan LongCat-Flash-Thinking model ca… 18 34% ▇▇▇▇
Meituan LongCat-Flash-Chat model card 17 23% ▇▇▇
deepseek-ai/DeepSeek-V3.1 model card (s… 15 21% ▇▇▇
DeepSeek-V3.1-Terminus model card (subm… 10 16% ▇▇
LMArena leaderboard dataset 1 3% ▇
SpeakLeash CPTUB 1 1% ▇
RPC-Bench leaderboard 1 1% ▇
Open Portuguese LLM Leaderboard 1 0% ▇
Vectara hallucination leaderboard 2 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 65.3
Licences: Meituan LongCat-Flash-Thinking model card ([MIT](https://huggingface.co/meituan-longcat/LongCat-Flash-Thinking/blob/7c9aad1e2daceaeab3a5a03deedb8e11e7a7819f/LICENSE)) · Meituan LongCat-Flash-Chat model card ([MIT](https://huggingface.co/meituan-longcat/LongCat-Flash-Chat/blob/f0b8e8d7380330f450ca1fde4ec56823a514584c/LICENSE)) · deepseek-ai/DeepSeek-V3.1 model card ([MIT](../../licenses/deepseek-ai--DeepSeek-V3.1.txt)) · DeepSeek-V3.1-Terminus model card ([MIT](https://huggingface.co/deepseek-ai/DeepSeek-V3.1-Terminus/blob/19510d6dc61f79dbd925bd51ee8a9081c509a4b6/LICENSE)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · SpeakLeash CPTUB ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · RPC-Bench leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
### Through an API
api (Deep Infra) · $0.42/M blended · $0.25 in / $0.95 out per 1M tokens · [deepinfra.com](https://deepinfra.com/models)
[price source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/providers/deepinfra/models/deepseek-ai/DeepSeek-V3.1.toml): Standard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/models/deepseek/deepseek-v3.1.toml)
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run DeepSeek V3.1 (ud-q4_k_xl, needs ~411.5 GB, from [huggingface.co/unsloth](https://huggingface.co/unsloth/DeepSeek-V3.1-GGUF/tree/main))
```sh
llama cli -hf unsloth/DeepSeek-V3.1-GGUF:UD-Q4_K_XL
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf unsloth/DeepSeek-V3.1-GGUF:UD-Q4_K_XL
```
Other versions: ud-q2_k_xl (~206 GB) `llama cli -hf unsloth/DeepSeek-V3.1-GGUF:UD-Q2_K_XL` · ud-iq1_s (~103.25 GB) `llama cli -hf unsloth/DeepSeek-V3.1-GGUF:UD-IQ1_S`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run DeepSeek V3.1 (ud-q4_k_xl, needs ~411.5 GB, from [huggingface.co/unsloth](https://huggingface.co/unsloth/DeepSeek-V3.1-GGUF/tree/main))
```sh
lms get https://huggingface.co/unsloth/DeepSeek-V3.1-GGUF@UD-Q4_K_XL -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: ud-q2_k_xl (~206 GB) `lms get https://huggingface.co/unsloth/DeepSeek-V3.1-GGUF@UD-Q2_K_XL -y` · ud-iq1_s (~103.25 GB) `lms get https://huggingface.co/unsloth/DeepSeek-V3.1-GGUF@UD-IQ1_S -y` · 4bit (~377.6 GB, Mac only) `lms get https://huggingface.co/mlx-community/DeepSeek-V3.1-4bit -y` · 8bit (~713.1 GB, Mac only) `lms get https://huggingface.co/mlx-community/DeepSeek-V3.1-8bit -y`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run DeepSeek V3.1 (4bit, needs ~377.6 GB, Mac only, from [huggingface.co/mlx-community](https://huggingface.co/mlx-community/DeepSeek-V3.1-4bit/tree/1634da2179770a14024405afa6d1e0ce70a71ff4))
```sh
mlx_lm.chat --model mlx-community/DeepSeek-V3.1-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model mlx-community/DeepSeek-V3.1-4bit
```
Other versions: 8bit (~713.1 GB, Mac only) `mlx_lm.chat --model mlx-community/DeepSeek-V3.1-8bit`
Model repository and documentation: [huggingface.co/deepseek-ai](https://huggingface.co/deepseek-ai/DeepSeek-V3.1)
[interactive version](https://whatmodel.sh/model/deepseek-v3-1) · [JSON](https://whatmodel.sh/api?format=json&all=1)