[whatmodel.sh](../../README.md) / DeepSeek V3 0324 # DeepSeek V3 0324 DeepSeek · released 2025-03 · 671B params (37B active) · mit **score 60.3** ±0.7 · #196 of 732 · from 30 benchmark results across 17 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks DeepSeek V3 0324 among every model on that task or test, worst to best.}

task            vs the others          rank  counts  tests
agent tasks     ▁▁▃▆●▄▅▅▄▄▂▁    #185 of 313     23%      7
coding          ▁▁▁▃▅▇█●▅▄▃▁    #155 of 469     32%      6
chat & writing  ▁▁▁▃▆▄▆█▆●▇▃   #122 of 402*    6.7%      2
math            ▁▁▃▅█●▄█▄▂▁▁    #195 of 431     16%      6

agent tasks · #185 of 313 · 23% of its score

Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md), [DeepSeek R1 0528](deepseek-r1-0528.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Jamba 1.5 Large](jamba-1-5-large.md) here, though they rank lower overall.

test                                result  vs the others in its run   difficulty            counts  why
τ²-bench Telecom                     32.5%  ██··●·██··██ beats 2/6     very hard · avg 9.9%     15%  sharp · at its level · several runs
SWE-bench Verified (maker)           45.4%  ▁▁▁▁▁▁▂●█▆▁▁ beats 34/138  hard · avg 21%          2.6%  sharp · at its level · small run
BFCL V4 (2025-08-26 archive, mode…   45.2%  ▄▄····●▄···▄ beats 3/5     hard · avg 26%          1.6%  at its level
SWE-bench Multilingual (maker)       29.3%  ▁··▁●▁▅██▂▂▁ beats 5/65    hard · avg 17%          1.5%  at its level · small run
BFCL v3                              64.7%  ▁▁▁▁▁▃▅█▅●▁▁ beats 96/114  medium · avg 51%        1.0%  blunt · at its level
Galileo Agent Leaderboard v1 (too…   90.5%  ▃▃▃▃▆▅▆▅●▂▅▂ beats 24/31   easy · avg 78%          0.6%  typical
Terminal-Bench (pre-2.0) with Ter…   13.3%  █····●·····█ beats 1/2     very hard · avg 7.3%    0.4%  noisy · small run

coding · #155 of 469 · 32% of its score

Beats [Qwen3-Coder-Next (80B-A3B)](qwen3-coder-next.md), [Gemini 2.5 Pro](gemini-2-5-pro.md), [Gemini 3.1 Flash-Lite](gemini-3-1-flash-lite.md) here, though they rank higher overall. Trails [Ministral 3 8B Reasoning 2512](ministral-3-8b-reasoning.md), [Phi-4-reasoning 14B](phi-4-reasoning.md), [DeepSeek V3 (Dec '24)](deepseek-v3.md) here, though they rank lower overall.

test                                result  vs the others in its run   difficulty            counts  why
Aider polyglot (225 exercises, tw…   55.1%  ▂·▂▃▅▂▃●▆▄▂▂ beats 31/49   very hard · avg 7.6%     23%  sharp · at its level · several runs
WebApp1K-Duo-React pass@1 (onekq-…   61.6%  ▁··▁·▁▁▁▆●▇█ beats 17/45   hard · avg 29%          2.9%  at its level
SWE-bench Verified (maker)           45.4%  ▁▁▁▁▁▁▂●█▆▁▁ beats 34/138  hard · avg 21%          2.6%  sharp · at its level · small run
SWE-bench Multilingual (maker)       29.3%  ▁··▁●▁▅██▂▂▁ beats 5/65    hard · avg 17%          1.5%  at its level · small run
LiveCodeBench (version unspecifie…   49.2%  ▁···▁▁▁●▇▃▄▁ beats 17/38   hard · avg 29%          1.3%  at its level · small run
LiveCodeBench 2024-08–2025-05 (ve…   43.0%  ●···█··▅··▃▃ beats 0/7     medium · avg 41%        0.9%  at its level · small run

chat & writing · #122 of 402 · 6.7% of its score

Beats [LongCat 2.0 (1.6T-A48B)](longcat-2-0.md), [Step 3.5 Flash (196B-A11B)](step-3-5-flash.md), [GPT-5 mini](gpt-5-mini.md) here, though they rank higher overall. Trails [LFM2.5 2.6B](lfm2-5-2-6b.md), [Nemotron-Cascade-8B](nemotron-cascade-8b.md) here, though they rank lower overall.

test                        result  vs the others in its run    difficulty      counts  why
LMArena (text)              1395.2  ▁▁▁▂▃▃▃▆▆●█▃ beats 241/385  avg 1260          4.8%  blunt
IFEval prompt-level strict   81.9%  ▄▄··●·▄··█·▄ beats 2/6      easy · avg 73%    1.9%  typical

math · #195 of 431 · 16% of its score

Beats [MiniMax-M2.1](minimax-m2-1.md) here, though they rank higher overall. Trails [Mixtral 8x7B v0.1 (base)](mixtral-8x7b-v0-1.md), [Phi-4-mini-reasoning 3.8B](phi-4-mini-reasoning.md), [DeepSeek R1 Distill Qwen 7B](deepseek-r1-distill-qwen-7b.md) here, though they rank lower overall.

test                                result  vs the others in its run   difficulty            counts  why
MATH level 5                         75.5%  ▂▂▅▆▄█▆●▅▂▃▄ beats 64/92   hard · avg 28%          7.4%  sharp
OTIS mock AIME 2024–2025             37.8%  ▃▃▄▃▃●▇▇█▃▂▅ beats 70/197  very hard · avg 7.2%    3.9%  sharp · at its level · noisy
AIME 2025 (maker)                    51.3%  ▁·▁▁▁▂▅●█▇▅▁ beats 38/133  hard · avg 34%          1.8%  sharp · at its level · noisy · small run
AIME 2024 (maker)                    59.4%  ▁▁▁▂▁▃▁●███▃ beats 28/85   medium · avg 47%        1.6%  sharp · noisy · small run
HMMT 2025 (edition unspecified) (…   29.2%  ●█▇▅▇▅▁·▁··▁ beats 0/32    hard · avg 24%          1.5%  sharp · at its level · noisy · small run
MATH-500 (maker)                     94.0%  ▁▃▂▃▄▇▅●█▄▂▁ beats 48/85   very easy · avg 89%        —  not counted yet: no other model in the …

other tests (9)

test                                result  vs the others in its run    difficulty           counts  why
GPQA Diamond                         67.6%  ▁▁▁▁▂▄▆▄●▇█▅ beats 102/211  hard · avg 39%          12%  sharp · at its level · several runs
EQ-Bench 3 (emotional intelligenc…  1172.2  ▁·▂▁▄▅●▃█▄▁▁ beats 27/62    avg 1034               2.9%  typical
CPTUB average (SpeakLeash, 0-5 ju…   80.6%  ▁▂▂▂▃█▃▆▅●▂▃ beats 84/95    medium · avg 65%       2.6%  typical
MMLU-Redux (maker)                   90.5%  ▁▁▃▁▂▃▄▅▇●█▄ beats 38/63    easy · avg 80%         2.3%  small run
Open Persian LLM Leaderboard v2 a…   72.4%  ▂▄▂▃█▄▃▆▃▅●▂ beats 76/81    medium · avg 57%       1.9%  blunt · at its level
Polish EQ-Bench v2 (SpeakLeash)      73.5%  ▁··▁▁▂▂▆▅▆█● beats 64/70    medium · avg 62%       1.8%  typical
MMLU-Pro (maker)                     81.2%  ▁▁·▁▁▁▃▂▄▅●▁ beats 133/188  medium · avg 60%       1.8%  small run
Open PT-LLM Leaderboard average (…   81.5%  ▃▁▁▁▁▁▁▁▂█●▁ beats 265/276  medium · avg 72%       0.8%  noisy
Vectara hallucination leaderboard…   92.0%  ▁▁▂▃●▄█▆▃▂▂▁ beats 36/140   very easy · avg 94%    0.2%  blunt · maxed out · noisy

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
deepseek-ai/DeepSeek-V3.1 model card (s…       11   29% ▇▇▇
Epoch AI                                        3   20% ▇▇
Aider polyglot benchmark runs                   1   12% ▇
MUA-RL: Multi-turn User-interacting Age…        1   11% ▇
LMArena leaderboard dataset                     1    5% ▇
Moonshot AI Kimi-K2-Instruct model card         1    4% ▇
EQ-Bench 3 canonical leaderboard results        1    3% ▇
WebApp1K models leaderboard                     1    3% ▇
SpeakLeash CPTUB                                1    3% ▇
Berkeley Function Calling Leaderboard r…        2    3% ▇
Open Persian LLM Leaderboard v2                 1    2% ▇
OpenCompass CompassAcademic leaderboard         1    2% ▇
SpeakLeash Polish EQ-Bench leaderboard          1    2% ▇
deepseek-ai/DeepSeek-V3-0324 model card…        1    1% ▇
Open Portuguese LLM Leaderboard                 1    1% ▇
Galileo Agent Leaderboard                       1    1% ▇
Vectara hallucination leaderboard               1    0% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 60.3

_{Licences: deepseek-ai/DeepSeek-V3.1 model card ([MIT](../../licenses/deepseek-ai--DeepSeek-V3.1.txt)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Aider polyglot benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) · MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Moonshot AI Kimi-K2-Instruct model card ([Modified MIT License (Kimi K2)](https://huggingface.co/moonshotai/Kimi-K2-Instruct/blob/fd1984e2b7a3350dbf7305fe73a4ede25c14de50/LICENSE)) · EQ-Bench 3 canonical leaderboard results ([MIT](../../licenses/eqbench3-MIT.txt)) · WebApp1K models leaderboard ([MIT](../../licenses/webapp1k-leaderboard-MIT.txt)) · SpeakLeash CPTUB ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · OpenCompass CompassAcademic leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · SpeakLeash Polish EQ-Bench leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · deepseek-ai/DeepSeek-V3-0324 model card ([MIT](../../licenses/deepseek-ai--DeepSeek-V3-0324.txt)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Galileo Agent Leaderboard ([Apache-2.0](../../licenses/galileo-agent-leaderboard-Apache-2.0.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))}

Not counted (1)

source                  test      score  why
deepseek-ai/DeepSeek-…  MATH-500  94.0%  too few models have it yet

## How to run it ### llama.cpp (recommended) 1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md) ```sh # macOS / Linux curl -LsSf https://llama.app/install.sh | sh # Windows irm https://llama.app/install.ps1 | iex ``` 2. Run DeepSeek V3 0324 (ud-q4_k_xl, needs ~403.1 GB, from [huggingface.co/unsloth](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main)) ```sh llama cli -hf unsloth/DeepSeek-V3-0324-GGUF:UD-Q4_K_XL ``` 3. Or start a local server and open the address it prints (http://localhost:9931) ```sh llama serve -hf unsloth/DeepSeek-V3-0324-GGUF:UD-Q4_K_XL ``` Other versions: ud-q2_k_xl (~201.8 GB) `llama cli -hf unsloth/DeepSeek-V3-0324-GGUF:UD-Q2_K_XL` · ud-iq1_s (~101.15 GB) `llama cli -hf unsloth/DeepSeek-V3-0324-GGUF:UD-IQ1_S` ### LM Studio 1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download) 2. Run DeepSeek V3 0324 (ud-q4_k_xl, needs ~403.1 GB, from [huggingface.co/unsloth](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main)) ```sh lms get https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF@UD-Q4_K_XL -y ``` 3. Then run lms ls to see its name, and lms chat followed by that name Other versions: ud-q2_k_xl (~201.8 GB) `lms get https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF@UD-Q2_K_XL -y` · ud-iq1_s (~101.15 GB) `lms get https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF@UD-IQ1_S -y` · 5bit (~461.5 GB, Mac only) `lms get https://huggingface.co/mlx-community/DeepSeek-V3-0324-5bit -y` ### MLX (Mac) 1. Install MLX (Mac). Apple silicon Macs only, text only. ```sh # with uv uv tool install mlx-lm # or pip install mlx-lm ``` 2. Run DeepSeek V3 0324 (5bit, needs ~461.5 GB, Mac only, from [huggingface.co/mlx-community](https://huggingface.co/mlx-community/DeepSeek-V3-0324-5bit/tree/35e9c4c166642a8ec12857286e67ec6a5f503be6)) ```sh mlx_lm.chat --model mlx-community/DeepSeek-V3-0324-5bit ``` 3. Or start an OpenAI-compatible server at http://localhost:8080/v1 ```sh mlx_lm.server --model mlx-community/DeepSeek-V3-0324-5bit ``` Model repository and documentation: [huggingface.co/deepseek-ai](https://huggingface.co/deepseek-ai/DeepSeek-V3-0324) [interactive version](https://whatmodel.sh/model/deepseek-v3-0324) · [JSON](https://whatmodel.sh/api?format=json&all=1)