[whatmodel.sh](../../README.md) / Qwen2.5 1.5B Instruct
# Qwen2.5 1.5B Instruct
Alibaba · released 2024-09 · 1.54B params · apache-2.0
**score 38.0** ±0.9 · #596 of 732 · from 29 benchmark results across 12 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Qwen2.5 1.5B Instruct among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃●█▄▅▅▄▄▂▁ #277 of 313* 1.8% 1
coding ▁▁▁▃●▇█▇▅▄▃▁ #365 of 469 9.2% 5
chat & writing ▁▁▁●▆▄▆█▆▇▇▃ #384 of 402 6.5% 3
math ▁▁▃●█▆▄█▄▂▁▁ #361 of 431 24% 2
agent tasks · #277 of 313 · 1.8% of its score
Beats [Magistral Small 1.0 (2506)](magistral-small-2506.md), [Llama 4 Scout (109B-A17B)](llama-4-scout.md), [Phi-4 14B](phi-4.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
BFCL v3 47.8% ▁▁▁▁▁▃●█▅▄▁▁ beats 30/114 medium · avg 51% 1.8% blunt · at its level
coding · #365 of 469 · 9.2% of its score
Beats [LFM2.5 8B-A1B](lfm2-5-8b-a1b.md), [Qwen3-VL 2B Thinking](qwen3-vl-2b-thinking.md), [DeepSeek R1 Distill Qwen 7B](deepseek-r1-distill-qwen-7b.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
HumanEval 30.5% ▁▁▁▁▂▂●▆█▅▂▁ beats 49/205 easy · avg 76% 5.5% sharp · at its level · small run
CodeReviewQA: solution identifica… 37.8% ▃··▁▂▂▃●█▃▄▂ beats 20/47 medium · avg 51% 1.4% sharp · noisy
CodeReviewQA: automated code refi… 20.0% ▁▁▁▁▁▁▂●▃█▅▄ beats 11/47 hard · avg 32% 1.3% typical
CodeReviewQA: change localisation… 1.2% █●▁▁▂▂▂▃▅▅▅▇ beats 10/47 hard · avg 33% 0.9% sharp · too hard for it · noisy
CodeReviewQA: change type recogni… 78.2% ▁▁▁······▁▃● beats 40/47 easy · avg 67% 0.1% blunt · at its level · noisy
chat & writing · #384 of 402 · 6.5% of its score
Beats [Phi-2 (2.7B)](phi-2.md) here, though they rank higher overall. Trails [Gemma 3 270M IT](gemma-3-270m.md), [SmolLM2 360M Instruct](smollm2-360m.md), [Granite 4.0 350M](granite-4-0-350m.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
AlpacaEval 2 length-controlled 7.4% ●▃·▁▁█▇▅·▁·▁ beats 1/22 medium · avg 39% 2.8% sharp
IFEval prompt-level loose 44.2% ●▃▁▃▁▄▁▁█▃▃▁ beats 0/22 easy · avg 78% 2.2% at its level
IFEval (aggregation unspecified) 47.4% ▁●▄▂▂▄▆██▅▂▁ beats 7/145 easy · avg 76% 1.5% at its level · small run
math · #361 of 431 · 24% of its score
Beats [GPT-3.5 Turbo (0125)](gpt-3-5-turbo-0125.md), [DBRX Instruct](dbrx-instruct.md), [Command R Plus (04-2024)](command-r-plus-04-2024.md) here, though they rank higher overall. Trails [OPT-2.7B](opt-2-7b.md), [Gemma 2B (base)](gemma-2b-base.md), [Stable Code 3B](stable-code-3b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
GSM8K 66.2% ▂▃··●▂▅█▆▃▂▃ beats 3/22 very easy · avg 88% 17% sharp · at its level · several runs
MATH (subset unspecified) 40.6% ▃▃●▂▂▆·····▁ beats 9/22 medium · avg 58% 6.9% at its level · small run · several runs
other tests (15)
test result vs the others in its run difficulty counts why
BIG-Bench Hard 45.8% ▁▄·●▁·▅█·▄▄▁ beats 4/22 medium · avg 72% 15% sharp · at its level · several runs
MMLU 59.7% ▁▂·▁·●▁█·▃▃▅ beats 4/22 easy · avg 75% 13% at its level
CPTUB average (SpeakLeash, 0-5 ju… 35.2% ▁●▂▂▃█▃▆▅▄▂▃ beats 4/95 medium · avg 65% 6.8% at its level
Polish EQ-Bench v2 (SpeakLeash) 27.8% ▁··▁▁●▂▆▅▆█▆ beats 6/70 medium · avg 62% 4.7% at its level
Open Persian LLM Leaderboard v2 a… 36.8% ▂●▂▃█▄▃▆▃▅▂▂ beats 9/81 medium · avg 57% 3.8% blunt · at its level
BRIDGE clinical text, zero-shot a… 22.2% ▁▁▁▁▁●▂▃▄▆█▃ beats 15/97 hard · avg 31% 3.6% blunt
AraGen-v3 3C3H score (Inception, … 13.2% ▁▁·▁▂●▄█▄▅▄▄ beats 9/66 hard · avg 29% 1.9% typical
HellaSwag 60.9% ▁▁▁▂▂▂▄●█▇▄▂ beats 43/135 easy · avg 82% 1.9% at its level · small run
Open PT-LLM Leaderboard average (… 65.6% ▃▁▁▁▁▁▁▁▂●▅▁ beats 117/276 medium · avg 72% 1.8% at its level · noisy
PopQA 15.5% ▁▁▁●▃▃▁█▁▃·▄ beats 3/21 hard · avg 31% 1.7% typical
EuroEval Life in the UK (English … 76.0% ▁▁▁▁▁▁▁▂●█▄▃ beats 52/146 easy · avg 82% 1.4% at its level · noisy
EuroEval HellaSwag (English), tes… 64.6% ▃·▁▁▂▁▂●█▇▄▁ beats 40/100 medium · avg 67% 1.2% sharp · noisy
Vectara hallucination leaderboard… 84.2% ▁▁●▃▄▄█▆▃▂▂▁ beats 12/140 very easy · avg 94% 0.9% blunt · noisy
PIQA 73.2% ▁▁▁▂▂●▅█▅▃▁▁ beats 34/123 easy · avg 84% 0.8% blunt · at its level · small run
TruthfulQA MC2 46.5% ▃●▂·▃▃·▅█▂▅▂ beats 3/21 medium · avg 60% 0.4% blunt · at its level
source results share of the score
2 OLMo 2 Furious 8 41% ▇▇▇▇▇
SmolLM2 paper 7 27% ▇▇▇
SpeakLeash CPTUB 1 7% ▇
SpeakLeash Polish EQ-Bench leaderboard 1 5% ▇
Open Persian LLM Leaderboard v2 1 4% ▇
CodeReviewQA: The Code Review Comprehen… 4 4% ▇
BRIDGE Medical Leaderboard 1 4% ▇
EuroEval leaderboards 2 3% ▇
Inception Arabic/Hindi leaderboards 1 2% ▇
Open Portuguese LLM Leaderboard 1 2% ▇
Berkeley Function Calling Leaderboard r… 1 2% ▇
Vectara hallucination leaderboard 1 1% ▇
prior: pull toward the middle 0% ▇
weighted average = 38.0
Licences: 2 OLMo 2 Furious ([CC BY 4.0](../../licenses/paper-2501.00656v3-CC-BY-4.0.txt)) · SmolLM2 paper ([CC BY 4.0](../../licenses/paper-2502.02737v1-CC-BY-4.0.txt)) · SpeakLeash CPTUB ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · SpeakLeash Polish EQ-Bench leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · BRIDGE Medical Leaderboard ([MIT](../../licenses/bridge-medical-leaderboard-MIT.txt)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Qwen2.5 1.5B Instruct (Q4_K_M, needs ~1.1 GB, from [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct-GGUF/tree/91cad51170dc346986eccefdc2dd33a9da36ead9))
```sh
llama cli -hf Qwen/Qwen2.5-1.5B-Instruct-GGUF:q4_k_m
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf Qwen/Qwen2.5-1.5B-Instruct-GGUF:q4_k_m
```
Other versions: Q8_0 (~1.9 GB) `llama cli -hf Qwen/Qwen2.5-1.5B-Instruct-GGUF:q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Qwen2.5 1.5B Instruct (Q4_K_M, needs ~1.1 GB, from [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct-GGUF/tree/91cad51170dc346986eccefdc2dd33a9da36ead9))
```sh
lms get https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct-GGUF@q4_k_m -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~1.9 GB) `lms get https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct-GGUF@q8_0 -y` · 4bit (~0.9 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/Qwen2.5-1.5B-Instruct-MLX-4bit -y` · 8bit (~1.6 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/Qwen2.5-1.5B-Instruct-MLX-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Qwen2.5 1.5B Instruct (Q4_K_M, needs ~1.42 GB, from [ollama.com](https://ollama.com/library/qwen2.5/tags))
```sh
ollama run qwen2.5:1.5b-instruct-q4_K_M
```
Other versions: Q8_0 (~2.35 GB) `ollama run qwen2.5:1.5b-instruct-q8_0`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run Qwen2.5 1.5B Instruct (4bit, needs ~0.9 GB, Mac only, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Qwen2.5-1.5B-Instruct-MLX-4bit/tree/ed6c7eaf905cbe537f8cac32ffee92a7414bf078))
```sh
mlx_lm.chat --model lmstudio-community/Qwen2.5-1.5B-Instruct-MLX-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model lmstudio-community/Qwen2.5-1.5B-Instruct-MLX-4bit
```
Other versions: 8bit (~1.6 GB, Mac only) `mlx_lm.chat --model lmstudio-community/Qwen2.5-1.5B-Instruct-MLX-8bit`
Model repository and documentation: [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct)
## Published speed measurements
Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine.
hardware precision speed engine and settings
Apple M2 Pro (16-core GPU),… q4_k_m 97.54 tok/s ollama 0.30.8 · whatmodel bench.p…
Each speed links to its source, which has the full settings.
[interactive version](https://whatmodel.sh/model/qwen2-5-1-5b) · [JSON](https://whatmodel.sh/api?format=json&all=1)