[whatmodel.sh](../../README.md) / Qwen2.5 3B Instruct
# Qwen2.5 3B Instruct
Alibaba · released 2024-09 · 3.09B params · qwen-research
**score 44.3** ±0.6 · #490 of 732 · from 32 benchmark results across 11 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Qwen2.5 3B Instruct among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃●█▄▅▅▄▄▂▁ #266 of 313* 0.8% 1
coding ▁▁▁▃▅●█▇▅▄▃▁ #338 of 469 13% 5
chat & writing ▁▁▁▃▆▄●█▆▇▇▃ #281 of 402* 2.5% 1
math ▁▁▃●█▆▄█▄▂▁▁ #345 of 431 20% 4
agent tasks · #266 of 313 · 0.8% of its score
Beats [DeepSeek R1 0528 Qwen3 8B](deepseek-r1-0528-qwen3-8b.md), [Magistral Small 1.0 (2506)](magistral-small-2506.md), [Llama 4 Scout (109B-A17B)](llama-4-scout.md) here, though they rank higher overall. Trails [Granite 4.0 H 1B](granite-4-0-h-1b.md), [Granite 4.0 1B](granite-4-0-1b.md), [SmolLM3 3B](smollm3-3b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
BFCL v3 50.4% ▁▁▁▁▁▃●█▅▄▁▁ beats 35/114 medium · avg 51% 0.8% blunt · at its level
coding · #338 of 469 · 13% of its score
Beats [Qwen3-VL 8B Instruct](qwen3-vl-8b-instruct.md), [Qwen3-VL 4B Instruct](qwen3-vl-4b-instruct.md), [LFM2.5 2.6B](lfm2-5-2-6b.md) here, though they rank higher overall. Trails [Phi-3-mini-128k-instruct (Apr 2024)](phi-3-mini-128k-instruct.md), [DeepSeek LLM 67B Chat](deepseek-llm-67b-chat.md), [WizardLM 70B V1.0](wizardlm-70b-v1-0.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
CRUXEval (ZeroEval, 800 problems) 33.1% ▁●▆▃▆█▆▃▂·▁▁ beats 1/47 medium · avg 57% 10% sharp · at its level
CodeReviewQA: solution identifica… 45.2% ▃··▁▂▂▃▂●▃▄▂ beats 21/47 medium · avg 51% 1.0% sharp · at its level · noisy
CodeReviewQA: change localisation… 35.0% █▂▁▁▂▂▂▃▅●▅▇ beats 29/47 hard · avg 33% 0.8% sharp · noisy
CodeReviewQA: automated code refi… 21.3% ▁▁▁▁▁▁▂●▃█▅▄ beats 13/47 hard · avg 32% 0.8% typical
CodeReviewQA: change type recogni… 68.8% ▁▁▁······▁●█ beats 17/47 easy · avg 67% 0.0% blunt · at its level · noisy
chat & writing · #281 of 402 · 2.5% of its score
Beats [Ministral 3 14B Reasoning 2512](ministral-3-14b-reasoning.md), [Ministral 3 14B](ministral-3-14b.md), [Ministral 3 8B](ministral-3-8b.md) here, though they rank higher overall. Trails [EXAONE 3.5 2.4B Instruct](exaone-3-5-2-4b.md), [Flan-T5 XXL](flan-t5-xxl.md), [LFM2 1.2B](lfm2-1-2b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arabic IFEval prompt-level (Incep… 38.1% ▁·▁▁▂▁▄▄▆●█▄ beats 21/38 medium · avg 46% 2.5% sharp · at its level
math · #345 of 431 · 20% of its score
Beats [Athene-70B](athene-70b.md), [Gemma 2 9B IT](gemma-2-9b.md), [GPT-3.5 Turbo (0125)](gpt-3-5-turbo-0125.md) here, though they rank higher overall. Trails [Gemma 2B (base)](gemma-2b-base.md), [Stable Code 3B](stable-code-3b.md), [Zephyr 7B beta](zephyr-7b-beta.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
MGSM 53.5% ▅▃●█·▅···▃·▃ beats 3/10 easy · avg 72% 8.0% sharp · at its level
GSM8K 80.6% █·●▄▄█···█·▄ beats 3/10 very easy · avg 90% 7.1% sharp
MATH Level 5 (ZeroEval, 721 probl… 25.5% ▁▁▁▆●▂▇▇▄··▂ beats 14/38 hard · avg 30% 2.7% sharp
MATH 61.7% █▄██···▄●▄·▄ beats 8/10 medium · avg 61% 2.7% at its level
other tests (21)
test result vs the others in its run difficulty counts why
MMLU-Redux (ZeroEval, 2,778 quest… 64.3% ▁·▁▁●▄▄▅█▄▂▁ beats 6/53 easy · avg 75% 18% at its level
ARC Challenge 82.6% ▂▂▂●▂···▄··▂ beats 3/10 very easy · avg 89% 6.9% sharp
MMLU 65.0% █▄●▄▄▄·▄▄··▄ beats 4/10 easy · avg 75% 5.7% at its level
BIG-Bench Hard 56.2% ██●·█▄·▄▄··▄ beats 4/10 medium · avg 71% 4.6% sharp · at its level
OpenBookQA 79.3% ▂··●█▂·▂···▄ beats 2/10 easy · avg 85% 3.7% at its level
MMLU-Pro 44.7% █▄▄·●▄▄▄·▄·▄ beats 5/10 medium · avg 54% 3.2% at its level
CPTUB average (SpeakLeash, 0-5 ju… 50.1% ▁▂▂●▃█▃▆▅▄▂▃ beats 16/95 medium · avg 65% 3.0% at its level
Multilingual MMLU (Microsoft) 55.9% █▄▄█●··█···▄ beats 6/10 medium · avg 63% 2.9% at its level
ZebraLogic (ZeroEval, 1,000 puzzl… 4.8% ▁●▃▆██▁▁·▁▁▁ beats 2/57 hard · avg 15% 2.4% sharp · too hard for it
Polish EQ-Bench v2 (SpeakLeash) 36.1% ▁··▁▁▂●▆▅▆█▆ beats 9/70 medium · avg 62% 2.3% at its level
BRIDGE clinical text, zero-shot a… 26.6% ▁▁▁▁▁▂●▃▄▆█▃ beats 23/97 hard · avg 31% 1.9% blunt · at its level
Open Persian LLM Leaderboard v2 a… 43.5% ▂▄▂●█▄▃▆▃▅▂▂ beats 16/81 medium · avg 57% 1.8% blunt · at its level
HellaSwag 74.6% ▃▅●·▃▃▃···▃▃ beats 5/10 easy · avg 82% 1.4% at its level
WinoGrande 63.3% ▃····▃▃●▃·▅▅ beats 3/10 medium · avg 71% 1.2% at its level
Social IQA 75.3% ▃·▃▅▅●·▃···▃ beats 8/10 medium · avg 76% 1.1% blunt · at its level
BoolQ 65.4% ●··▃··██▃·▃▃ beats 0/10 easy · avg 83% 1.0% at its level
Open PT-LLM Leaderboard average (… 70.8% ▃▁▁▁▁▁▁▁▂●▅▁ beats 174/276 medium · avg 72% 0.9% at its level · noisy
GPQA (subset unspecified) 23.4% ●··▄▂··▂▂··▄ beats 1/10 very hard · avg 32% 0.7% sharp · too hard for it · noisy
PIQA 72.6% ▃·●·▃█·▃▅··▃ beats 1/10 medium · avg 82% 0.3% blunt · at its level
Vectara hallucination leaderboard… 93.0% ▁▁▂▃▄●█▆▃▂▂▁ beats 42/140 very easy · avg 94% 0.3% blunt · noisy
TruthfulQA MC2 64.3% ▂·●▂▂▄····▂▂ beats 4/10 medium · avg 64% 0.2% blunt · at its level
source results share of the score
microsoft/Phi-4-mini-instruct model card 16 51% ▇▇▇▇▇▇
WildEval ZeroEval published results 4 33% ▇▇▇▇
SpeakLeash CPTUB 1 3% ▇
CodeReviewQA: The Code Review Comprehen… 4 3% ▇
Inception Arabic/Hindi leaderboards 1 3% ▇
SpeakLeash Polish EQ-Bench leaderboard 1 2% ▇
BRIDGE Medical Leaderboard 1 2% ▇
Open Persian LLM Leaderboard v2 1 2% ▇
Open Portuguese LLM Leaderboard 1 1% ▇
Berkeley Function Calling Leaderboard r… 1 1% ▇
Vectara hallucination leaderboard 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 44.3
Licences: microsoft/Phi-4-mini-instruct model card ([MIT](../../licenses/card-microsoft-phi-4-mini-instruct.txt)) · WildEval ZeroEval published results ([Apache-2.0](../../licenses/zeroeval-published-results-Apache-2.0.txt)) · SpeakLeash CPTUB ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · SpeakLeash Polish EQ-Bench leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · BRIDGE Medical Leaderboard ([MIT](../../licenses/bridge-medical-leaderboard-MIT.txt)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Qwen2.5 3B Instruct (Q4_K_M, needs ~2.1 GB, from [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct-GGUF/tree/7dabda4d13d513e3e842b20f0d435c732f172cbe))
```sh
llama cli -hf Qwen/Qwen2.5-3B-Instruct-GGUF:q4_k_m
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf Qwen/Qwen2.5-3B-Instruct-GGUF:q4_k_m
```
Other versions: Q8_0 (~3.6 GB) `llama cli -hf Qwen/Qwen2.5-3B-Instruct-GGUF:q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Qwen2.5 3B Instruct (Q4_K_M, needs ~2.1 GB, from [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct-GGUF/tree/7dabda4d13d513e3e842b20f0d435c732f172cbe))
```sh
lms get https://huggingface.co/Qwen/Qwen2.5-3B-Instruct-GGUF@q4_k_m -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~3.6 GB) `lms get https://huggingface.co/Qwen/Qwen2.5-3B-Instruct-GGUF@q8_0 -y` · 4bit (~1.7 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/Qwen2.5-3B-Instruct-MLX-4bit -y` · 8bit (~3.3 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/Qwen2.5-3B-Instruct-MLX-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Qwen2.5 3B Instruct (Q4_K_M, needs ~2.35 GB, from [ollama.com](https://ollama.com/library/qwen2.5/tags))
```sh
ollama run qwen2.5:3b-instruct-q4_K_M
```
Other versions: Q8_0 (~4.21 GB) `ollama run qwen2.5:3b-instruct-q8_0`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run Qwen2.5 3B Instruct (4bit, needs ~1.7 GB, Mac only, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Qwen2.5-3B-Instruct-MLX-4bit/tree/50cc48c69ebb636f687d6767290dc86492eadbe4))
```sh
mlx_lm.chat --model lmstudio-community/Qwen2.5-3B-Instruct-MLX-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model lmstudio-community/Qwen2.5-3B-Instruct-MLX-4bit
```
Other versions: 8bit (~3.3 GB, Mac only) `mlx_lm.chat --model lmstudio-community/Qwen2.5-3B-Instruct-MLX-8bit`
Model repository and documentation: [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct)
## Published speed measurements
Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine.
hardware precision speed engine and settings
Apple M2 Pro (16-core GPU),… q4_k_m 60.74 tok/s ollama 0.30.8 · whatmodel bench.p…
Each speed links to its source, which has the full settings.
[interactive version](https://whatmodel.sh/model/qwen2-5-3b) · [JSON](https://whatmodel.sh/api?format=json&all=1)