[whatmodel.sh](../../README.md) / Qwen2.5 7B Instruct
# Qwen2.5 7B Instruct
Alibaba · released 2024-09 · 7.62B params · apache-2.0
**score 49.6** ±0.5 · #378 of 732 · from 51 benchmark results across 18 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Qwen2.5 7B Instruct among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃●█▄▅▅▄▄▂▁ #242 of 313 4.7% 3
coding ▁▁▁▃▅▇●▇▅▄▃▁ #273 of 469 15% 6
chat & writing ▁▁▁▃▆▄●█▆▇▇▃ #268 of 402 4.5% 3
math ▁▁▃▅●▆▄█▄▂▁▁ #277 of 431 17% 7
agent tasks · #242 of 313 · 4.7% of its score
Beats [o3-mini](o3-mini.md), [Llama 4 Maverick (402B-A17B)](llama-4-maverick.md), [DeepSeek R1 0528 Qwen3 8B](deepseek-r1-0528-qwen3-8b.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Phi-3.5-mini-instruct](phi-3-5-mini-instruct.md), [Jamba 1.5 Mini](jamba-1-5-mini.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
τ²-bench Telecom 16.7% ▁·●█▆▅▂▃▃▂▃▁ beats 7/89 very hard · avg 12% 3.5% sharp · small run · several runs
BFCL v3 55.8% ▁▁▁▁▁▃▅●▅▄▁▁ beats 59/114 medium · avg 51% 0.6% blunt · at its level
BFCL (version not stated) 55.8% ●▆▄▂·█▆·▄··▂ beats 1/17 medium · avg 63% 0.6% blunt · at its level
coding · #273 of 469 · 15% of its score
Beats [Qwen3-VL 32B Instruct](qwen3-vl-32b-instruct.md), [Solar Open 100B (102B-A12B)](solar-open-100b.md), [GLM-4.7-Flash](glm-4-7-flash.md) here, though they rank higher overall. Trails [DeepSeek LLM 67B Chat](deepseek-llm-67b-chat.md), [QwQ-32B-Preview](qwq-32b-preview.md), [Phi-3 Medium 128K Instruct](phi-3-medium-128k-instruct.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
CRUXEval (ZeroEval, 800 problems) 52.8% ▁▂▆●▆█▆▃▂·▁▁ beats 14/47 medium · avg 57% 7.5% sharp · at its level
LiveCodeBench official (Jan 1–Nov… 25.3% ▁▁·▁▂▄●█▅▂·▂ beats 12/31 hard · avg 23% 4.9% sharp
CodeReviewQA: change localisation… 34.2% █▂▁▁▂▂▂▃▅●▅▇ beats 28/47 hard · avg 33% 0.9% sharp · at its level · noisy
CodeReviewQA: solution identifica… 51.6% ▃··▁▂▂▃▂●▃▄▂ beats 27/47 medium · avg 51% 0.9% sharp · at its level · noisy
CodeReviewQA: automated code refi… 35.7% ▁▁▁▁▁▁▂▂▃█●▄ beats 34/47 hard · avg 32% 0.7% at its level
CodeReviewQA: change type recogni… 75.9% ▁▁▁······▁▃● beats 33/47 easy · avg 67% 0.0% blunt · at its level · noisy
chat & writing · #268 of 402 · 4.5% of its score
Beats [Ministral 3 14B Reasoning 2512](ministral-3-14b-reasoning.md), [Ministral 3 14B](ministral-3-14b.md), [OpenThinker2 32B](openthinker2-32b.md) here, though they rank higher overall. Trails [Flan-T5 XXL](flan-t5-xxl.md), [North Micro Vision Instruct (2.4B)](north-micro-vision-instruct.md), [Granite 4.0 H 1B](granite-4-0-h-1b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arabic IFEval prompt-level (Incep… 50.2% ▁·▁▁▂▁▄▄▆▄●▄ beats 26/38 medium · avg 46% 2.2% sharp · at its level
AlpacaEval 2 length-controlled 29.7% ▃▃·▁▁●▇▅·▁·▁ beats 7/22 medium · avg 39% 1.7% sharp · at its level
IFEval prompt-level loose 74.7% ▁▃▁▃▁▄●▁█▃▃▁ beats 10/22 easy · avg 78% 0.5% typical
math · #277 of 431 · 17% of its score
Beats [GPT-4 Turbo (2024-04-09)](gpt-4-turbo-2024-04-09.md), [Molmo 2 8B](molmo-2-8b.md), [Llama 3.3 70B Instruct](llama-3-3-70b.md) here, though they rank higher overall. Trails [DeepSeek R1 Distill Qwen 1.5B](deepseek-r1-distill-qwen-1-5b.md), [Gemma 7B (base)](gemma-7b-base.md), [CodeGemma 7B (v1.0)](codegemma-7b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
MGSM 64.5% ▅▃▃█·●···▃·▃ beats 8/10 easy · avg 72% 4.9% sharp · at its level
GSM8K 83.8% ▂▃··▃▂▅●▆▃▂▃ beats 11/22 very easy · avg 90% 4.8% sharp · several runs
MATH Level 5 (ZeroEval, 721 probl… 51.5% ▁▁▁▆█▂▇●▄··▂ beats 26/38 hard · avg 30% 3.1% sharp · at its level
MATH 60.4% █▄██···●▄▄·▄ beats 7/10 medium · avg 61% 1.9% at its level
MATH (subset unspecified) 69.9% ▃▃█▂●▆·····▁ beats 16/22 medium · avg 58% 1.5% at its level · several runs
OMEGA (math generalisation) 13.7% █···▄·●·▄··█ beats 3/6 hard · avg 19% 0.7% typical
OTIS mock AIME 2024–2025 2.5% ▃●▄▃▃▃▇▇█▃▂▅ beats 21/197 very hard · avg 7.2% 0.5% sharp · too hard for it · noisy
other tests (26)
test result vs the others in its run difficulty counts why
MMLU-Redux (ZeroEval, 2,778 quest… 75.1% ▁·▁▁▄▄●▅█▄▂▁ beats 20/53 easy · avg 75% 12% at its level
MMLU 76.6% ▁▂·▁·▁▁█·●▃▅ beats 13/22 easy · avg 75% 7.8% at its level · several runs
BIG-Bench Hard 70.2% ▁▄·▁▁·▅●·▄▄▁ beats 11/22 medium · avg 72% 6.9% sharp · at its level · several runs
ARC Challenge 90.1% ▂▂▂█▂···●··▂ beats 9/10 very easy · avg 89% 3.6% sharp
ZebraLogic (ZeroEval, 1,000 puzzl… 12.0% ▁▁▃●██▁▁·▁▁▁ beats 15/57 hard · avg 15% 3.3% sharp
GPQA Diamond 35.5% ▁▁▁▁▂●▆▄▅▇█▅ beats 28/211 hard · avg 39% 2.7% sharp
MMLU-Pro 56.2% █▄▄·█▄▄▄·●·▄ beats 9/10 medium · avg 54% 2.5% at its level
OpenBookQA 82.6% ▂··▄█●·▂···▄ beats 7/10 easy · avg 85% 2.2% typical
CPTUB average (SpeakLeash, 0-5 ju… 61.3% ▁▂▂▂▃●▃▆▅▄▂▃ beats 36/95 medium · avg 65% 2.0% at its level
Multilingual MMLU (Microsoft) 64.4% █▄▄██··●···▄ beats 9/10 medium · avg 63% 2.0% at its level
EvalPerf correctness (118 tasks, … 80.0% ▂·▂▂▂▆▅▆●▆▃█ beats 17/29 easy · avg 74% 1.6% at its level
Polish EQ-Bench v2 (SpeakLeash) 58.6% ▁··▁▁▂▂▆▅●█▆ beats 38/70 medium · avg 62% 1.6% at its level
BRIDGE clinical text, zero-shot a… 31.3% ▁▁▁▁▁▂▂▃●▆█▃ beats 34/97 hard · avg 31% 1.5% blunt · at its level
AraGen-v3 3C3H score (Inception, … 26.3% ▁▁·▁▂▂▄●▄▅▄▄ beats 22/66 hard · avg 29% 1.3% at its level
Open Persian LLM Leaderboard v2 a… 52.5% ▂▄▂▃█●▃▆▃▅▂▂ beats 37/81 medium · avg 57% 1.3% blunt · at its level
WinoGrande 71.1% ▃····▃▃█▃·●▅ beats 7/10 medium · avg 71% 1.0% at its level
HellaSwag 80.0% ▃▅█·▃●▃···▃▃ beats 7/10 easy · avg 82% 0.9% typical
GPQA (subset unspecified) 30.6% █··▄▂··▂●··▄ beats 8/10 very hard · avg 32% 0.9% sharp · noisy
PopQA 18.1% ▁▁▁▃●▃▁█▁▃·▄ beats 5/21 hard · avg 31% 0.8% at its level
Social IQA 75.3% ▃·▃▅▅●·▃···▃ beats 7/10 medium · avg 76% 0.8% blunt · at its level
BoolQ 80.0% ▃··▃··█●▃·▃▃ beats 5/10 easy · avg 83% 0.7% at its level
Open PT-LLM Leaderboard average (… 73.8% ▃▁▁▁▁▁▁▁▂█●▁ beats 211/276 medium · avg 72% 0.6% at its level · noisy
PIQA 76.2% ▃·▅·●█·▃▅··▃ beats 3/10 medium · avg 82% 0.2% blunt · at its level
TruthfulQA MC2 63.1% ▃▃▂·▃▃·●█▂▅▂ beats 11/21 medium · avg 64% 0.2% blunt · at its level · several runs
Vectara hallucination leaderboard… 97.2% ▁▁▂▃▄▄█●▃▂▂▁ beats 109/140 very easy · avg 94% 0.2% blunt · maxed out · noisy
Chess Puzzles (Epoch) 0.0% ●▁▁▁▁▂▃▃▃▂▁▁ beats 41/137 very hard · avg 5.8% 0.1% sharp · too hard for it · noisy
source results share of the score
microsoft/Phi-4-mini-instruct model card 16 32% ▇▇▇▇
WildEval ZeroEval published results 4 26% ▇▇▇
2 OLMo 2 Furious 8 13% ▇▇
LiveCodeBench official website results 1 5% ▇
Inception Arabic/Hindi leaderboards 2 4% ▇
Epoch AI 3 3% ▇
CodeReviewQA: The Code Review Comprehen… 4 2% ▇
SpeakLeash CPTUB 1 2% ▇
Reinforcing Real-world Service Agents: … 1 2% ▇
allenai/Olmo-3-7B-Instruct model card 3 2% ▇
EvalPlus / EvalPerf published evaluatio… 1 2% ▇
SpeakLeash Polish EQ-Bench leaderboard 1 2% ▇
Synthesize and Reward: Reinforcement Le… 1 2% ▇
BRIDGE Medical Leaderboard 1 2% ▇
Open Persian LLM Leaderboard v2 1 1% ▇
Open Portuguese LLM Leaderboard 1 1% ▇
Berkeley Function Calling Leaderboard r… 1 1% ▇
Vectara hallucination leaderboard 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 49.6
Licences: microsoft/Phi-4-mini-instruct model card ([MIT](../../licenses/card-microsoft-phi-4-mini-instruct.txt)) · WildEval ZeroEval published results ([Apache-2.0](../../licenses/zeroeval-published-results-Apache-2.0.txt)) · 2 OLMo 2 Furious ([CC BY 4.0](../../licenses/paper-2501.00656v3-CC-BY-4.0.txt)) · LiveCodeBench official website results ([CC-BY-SA-4.0](../../licenses/lcb-sharealike-NOTICE.txt)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · SpeakLeash CPTUB ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · Reinforcing Real-world Service Agents: InteractCS-RL ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · allenai/Olmo-3-7B-Instruct model card ([Apache-2.0](../../licenses/allenai--Olmo-3-7B-Instruct.txt)) · EvalPlus / EvalPerf published evaluation runs ([Apache-2.0](../../licenses/evalplus-published-runs-Apache-2.0.txt)) · SpeakLeash Polish EQ-Bench leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Synthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · BRIDGE Medical Leaderboard ([MIT](../../licenses/bridge-medical-leaderboard-MIT.txt)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
### Through an API
api (Together AI (Turbo)) · $0.30/M blended · $0.3 in / $0.3 out per 1M tokens · [together.ai](https://www.together.ai/pricing)
[price source](https://www.together.ai/pricing): USD per 1M tokens, Together AI serverless inference, "Qwen2.5 7B Instruct Turbo" (Together optimized serving endpoint of Qwen2.5-7B-Instruct; its quantization may differ from the reference BF16 weights). Batch and dedicated endpoints excluded. Currency: USD. Together AI official pricing page, observed 2026-09-25. · [context source](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct/blob/a09a35458c702b33eeacc393d103063234e8bc28/README.md): Full supported model-card context requires the documented YaRN configuration beyond the default 32,768 tokens. This does not promise the same limit on every hosted API.
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Qwen2.5 7B Instruct (Q4_K_M, needs ~4.7 GB, from [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct-GGUF/tree/bb5d59e06d9551d752d08b292a50eb208b07ab1f))
```sh
llama cli -hf Qwen/Qwen2.5-7B-Instruct-GGUF:q4_k_m
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf Qwen/Qwen2.5-7B-Instruct-GGUF:q4_k_m
```
Other versions: Q8_0 (~8.1 GB) `llama cli -hf Qwen/Qwen2.5-7B-Instruct-GGUF:q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Qwen2.5 7B Instruct (Q4_K_M, needs ~4.7 GB, from [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct-GGUF/tree/bb5d59e06d9551d752d08b292a50eb208b07ab1f))
```sh
lms get https://huggingface.co/Qwen/Qwen2.5-7B-Instruct-GGUF@q4_k_m -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~8.1 GB) `lms get https://huggingface.co/Qwen/Qwen2.5-7B-Instruct-GGUF@q8_0 -y` · 4bit (~4.3 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/Qwen2.5-7B-Instruct-MLX-4bit -y` · 8bit (~8.1 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/Qwen2.5-7B-Instruct-MLX-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Qwen2.5 7B Instruct (Q4_K_M, needs ~5.07 GB, from [ollama.com](https://ollama.com/library/qwen2.5/tags))
```sh
ollama run qwen2.5:7b
```
Other versions: Q8_0 (~9.64 GB) `ollama run qwen2.5:7b-instruct-q8_0`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run Qwen2.5 7B Instruct (4bit, needs ~4.3 GB, Mac only, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Qwen2.5-7B-Instruct-MLX-4bit/tree/61ef4ce1bd8826cd334c614b985fad9d8a9c5382))
```sh
mlx_lm.chat --model lmstudio-community/Qwen2.5-7B-Instruct-MLX-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model lmstudio-community/Qwen2.5-7B-Instruct-MLX-4bit
```
Other versions: 8bit (~8.1 GB, Mac only) `mlx_lm.chat --model lmstudio-community/Qwen2.5-7B-Instruct-MLX-8bit`
Model repository and documentation: [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
## Published speed measurements
Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine.
hardware precision speed engine and settings
Apple M4 Pro (12-core CPU /… GGUF Q4_K_M (Ol… 59 tok/s Ollama v0.21.2 (llama.cpp, Metal)…
RTX 4060 Laptop q4_k_m 48.67 tok/s ollama (version not recorded) · l…
Each speed links to its source, which has the full settings.
## Quantization evidence
How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent. They don't describe every download of that precision.
precision test full → quantized kept
FP8 dynamic (weights FP8 per-… MMLU — published row 'MMLU (5-sho… 0.742 → 0.74 99.7%
FP8 dynamic (weights FP8 per-… OpenLLM v1 average of 6 tasks (MM… 0.732 → 0.729 99.7%
FP8 dynamic (weights FP8 per-… GSM8K — published row 'GSM-8K (5-… 0.804 → 0.801 99.6%
GGUF Q2_K_S (row label 'q2ks'… Average accuracy of the 10 listed… 0.657 → 0.639 97.2%
GGUF Q2_K_S (row label 'q2ks'… MMLU (lm-eval 0.4.9.1 default mml… 0.716 → 0.674 94.1%
GGUF Q2_K_S produced by AutoR… Average accuracy of 10 lm-eval ta… 0.657 → 0.627 95.4%
GGUF Q2_K_S produced by AutoR… Average accuracy of 10 lm-eval ta… 0.657 → 0.635 96.6%
GGUF Q3_K_M produced by AutoR… Average accuracy of 10 lm-eval ta… 0.657 → 0.642 97.7%
GGUF Q3_K_M produced by AutoR… Average accuracy of 10 lm-eval ta… 0.657 → 0.653 99.4%
GGUF Q3_K_S (row label 'q3ks-… Average accuracy of the 10 listed… 0.657 → 0.638 97.0%
GGUF Q3_K_S (row label 'q3ks-… MMLU (lm-eval 0.4.9.1 default mml… 0.716 → 0.691 96.5%
GGUF Q3_K_S (row label 'q3ks'… Average accuracy of the 10 listed… 0.657 → 0.648 98.6%
GGUF Q3_K_S (row label 'q3ks'… MMLU (lm-eval 0.4.9.1 default mml… 0.716 → 0.703 98.2%
GGUF Q3_K_S produced by AutoR… Average accuracy of 10 lm-eval ta… 0.657 → 0.638 97.1%
GGUF Q3_K_S produced by AutoR… Average accuracy of 10 lm-eval ta… 0.657 → 0.65 99.0%
GGUF Q4_K_M (row label 'q4km'… Average accuracy of the 10 listed… 0.657 → 0.658 100.2%
GGUF Q4_K_M (row label 'q4km'… MMLU (lm-eval 0.4.9.1 default mml… 0.716 → 0.714 99.7%
GGUF Q4_K_M produced by AutoR… Average accuracy of 10 lm-eval ta… 0.657 → 0.656 99.8%
GGUF Q4_K_M produced by AutoR… Average accuracy of 10 lm-eval ta… 0.657 → 0.657 100.1%
GGUF Q4_K_S produced by AutoR… Average accuracy of 10 lm-eval ta… 0.657 → 0.656 99.8%
GGUF Q4_K_S produced by AutoR… Average accuracy of 10 lm-eval ta… 0.657 → 0.66 100.5%
GPTQ INT4 (AutoGPTQ 0.7.1, gr… Clinical accuracy, unweighted mea… 0.435 → 0.431 98.9%
GPTQ INT8 (AutoGPTQ 0.7.1, gr… Clinical accuracy, unweighted mea… 0.435 → 0.434 99.8%
GPTQ-Int8 (Qwen2.5-7B-Instruc… Bangla zero-shot accuracy, unweig… 0.581 → 0.583 100.3%
INT4 W4A16 (weight-only INT4,… GSM8K — published row 'GSM-8K (5-… 0.804 → 0.806 100.3%
INT4 W4A16 (weight-only INT4,… OpenLLM v1 average of 6 tasks (MM… 0.732 → 0.727 99.4%
INT4 W4A16 (weight-only INT4,… MMLU — published row 'MMLU (5-sho… 0.742 → 0.732 98.6%
INT8 W8A8 (weights INT8 per-c… GSM8K — published row 'GSM-8K (5-… 0.804 → 0.807 100.5%
INT8 W8A8 (weights INT8 per-c… OpenLLM v1 average of 6 tasks (MM… 0.732 → 0.731 99.8%
INT8 W8A8 (weights INT8 per-c… MMLU — published row 'MMLU (5-sho… 0.742 → 0.739 99.5%
Each kept value links to its source.
[interactive version](https://whatmodel.sh/model/qwen2-5-7b) · [JSON](https://whatmodel.sh/api?format=json&all=1)