[whatmodel.sh](../../README.md) / Qwen3 4B Instruct 2507
# Qwen3 4B Instruct 2507
Alibaba · released 2025-08 · 4.02B params · apache-2.0
**score 55.3** ±0.6 · #272 of 732 · from 55 benchmark results across 10 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Qwen3 4B Instruct 2507 among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆●▄▅▅▄▄▂▁ #236 of 313 14% 7
coding ▁▁▁▃▅●█▇▅▄▃▁ #327 of 469 12% 5
chat & writing ▁▁▁▃▆▄▆●▆▇▇▃ #186 of 402 3.9% 2
math ▁▁▃▅█▆●█▄▂▁▁ #180 of 431 15% 8
agent tasks · #236 of 313 · 14% of its score
Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md), [Llama 3.1 Nemotron Ultra 253B v1](llama-3-1-nemotron-ultra-253b.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [LFM2.5 1.2B Thinking](lfm2-5-1-2b-thinking.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
τ²-bench Telecom 13.2% ▁·●█▆▅▂▃▃▂▃▁ beats 4/89 very hard · avg 11% 5.6% sharp · several runs
BFCL v3 (maker) 61.9% █▄··●█·█▄█▄█ beats 3/13 medium · avg 59% 1.7% blunt · at its level · several runs
BFCL V4 (December 2025 archive) 35.7% ▁▁▁▂██●▅▆▂▂▂ beats 39/77 hard · avg 31% 1.5% at its level
τ-bench Retail (maker) 48.7% ▁····▃·●▇█▂▆ beats 4/28 hard · avg 30% 1.5% at its level · small run
τ²-bench Retail (maker) 40.4% ▁····●▁▃▆█▃▁ beats 4/44 hard · avg 28% 1.4% at its level · small run
τ²-bench Airline (maker) 24.0% ●▁·▂▁▃▂▁█▅▂▃ beats 0/42 hard · avg 17% 1.1% small run
τ-bench Airline (maker) 32.0% ▁▁▁·●▄▁█▇▁▁▇ beats 5/28 hard · avg 18% 1.0% at its level · small run
coding · #327 of 469 · 12% of its score
Trails [DeepSeek LLM 67B Chat](deepseek-llm-67b-chat.md), [WizardLM 70B V1.0](wizardlm-70b-v1-0.md), [Llama 3.2 11B Vision Instruct](llama-3-2-11b-vision.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Aider polyglot (225 exercises, tw… 12.9% ▂▁▁▃●▃▄▆█▃▄▂ beats 10/71 very hard · avg 8.3% 4.7% sharp · small run
LiveCodeBench v6 (window unspecif… 35.1% ▃▅●▅▅▃▅···▃█ beats 3/16 hard · avg 35% 2.5% at its level
Luau Leetcode fully solved (Torpe… 15.4% ▁●▁▁·▁▂▃▅▃█▇ beats 2/49 hard · avg 17% 2.3% sharp · at its level
LiveCodeBench v6 (Feb–May 2025) (… 35.1% ▁●▃▁█▅▄▇▅▅▃█ beats 1/38 hard · avg 27% 1.3% at its level · small run
MultiPL-E (maker) 76.8% ▃··▄·▅▅▇▄●▃▄ beats 23/31 easy · avg 65% 1.0% at its level · small run
chat & writing · #186 of 402 · 3.9% of its score
Beats [Seed-OSS 36B Instruct](seed-oss-36b.md), [Gemma 2 9B IT SimPO](gemma-2-9b-simpo.md), [Nemotron-Cascade-14B-Thinking](nemotron-cascade-14b-thinking.md) here, though they rank higher overall. Trails [LFM2.5 1.2B Instruct](lfm2-5-1-2b-instruct.md), [LFM2.5 1.2B Thinking](lfm2-5-1-2b-thinking.md), [LFM2.5 8B-A1B](lfm2-5-8b-a1b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
IFEval (aggregation unspecified) … 83.4% ▃··▃···█●▅▃▃ beats 6/11 easy · avg 77% 2.5% several runs
Arena-Hard v2 aggregate (maker) 43.4% ▁▁··▄●▁▃█▁▁▇ beats 6/22 hard · avg 19% 1.3% sharp · at its level · small run
math · #180 of 431 · 15% of its score
Beats [MiniMax-M2.1](minimax-m2-1.md), [Mistral Medium 3.5 (128B)](mistral-medium-3-5.md), [GPT-4.1](gpt-4-1.md) here, though they rank higher overall. Trails [OpenThinker3 7B](openthinker3-7b.md), [Olmo 3 7B Think](olmo-3-7b-think.md), [OpenThinker2 32B](openthinker2-32b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
AIME 2025 (maker) 47.4% ▃·▃▃●··▃▅▃·█ beats 5/12 medium · avg 44% 4.6% sharp · at its level · several runs
GSM8K 88.6% █····█··▄·●█ beats 5/7 very easy · avg 86% 2.3% sharp · maxed out
HMMT 2025 (edition unspecified) (… 31.0% ●█▇▅▇▅▁·▁··▁ beats 1/32 very hard · avg 14% 1.8% sharp · small run
OTIS mock AIME 2024–2025 52.2% ▃▃▄▃▃▃●▇█▃▂▅ beats 79/197 very hard · avg 7.2% 1.7% sharp · noisy
MATH (subset unspecified) 93.4% █▄··▄█▄····● beats 7/7 easy · avg 80% 1.5% typical
PolyMath (Marco evaluation) 47.2% ▄··▄██··▄··● beats 7/7 hard · avg 29% 1.4% at its level
MGSM (Marco evaluation) 84.4% ▄·▄·▄▄▄·▄··● beats 7/7 easy · avg 69% 1.0% typical
PolyMATH (maker) 31.1% ▁···▃●·▇█▄▄▄ beats 3/23 hard · avg 22% 0.9% at its level · small run
other tests (23)
test result vs the others in its run difficulty counts why
GPQA Diamond 45.8% ▁▁▁▁▂▄●▄▅▇█▅ beats 50/211 hard · avg 37% 11% sharp · at its level · several runs
MMLU-Pro (maker) 69.6% ▄·▄·▄▄●▄█▄██ beats 4/12 medium · avg 63% 8.5% at its level · several runs
GPQA (subset unspecified) (maker) 62.0% ▂····▂·▂▂●▆█ beats 6/14 hard · avg 49% 7.4% sharp · at its level · several runs
MMLU-Redux (maker) 84.2% ▃··▃·▃·●▃▅▃▅ beats 3/11 easy · avg 79% 4.7% several runs
SuperGPQA (maker) 42.8% ▄··▄·▄●▄█▄██ beats 3/11 medium · avg 37% 3.4% at its level · several runs
MMLU 80.8% ▄···▄▄·▄█··● beats 7/7 easy · avg 74% 3.2% typical
RobloxQA v2.0 multiple choice (To… 82.1% ▁●▃▁█▄▃▃▆▂▂▂ beats 2/49 easy · avg 77% 2.1% typical
MMLU-ProX-Lite (Marco) 58.3% █··▄··█··█·● beats 7/7 medium · avg 42% 2.0% at its level
BRIDGE clinical text, zero-shot a… 34.1% ▁▁▁▁▁▂▂▃●▆█▃ beats 42/97 hard · avg 31% 1.9% blunt · at its level
GlobalMMLU (Marco evaluation) 70.2% ▃··▃▃··▅···● beats 7/7 medium · avg 56% 1.5% at its level
MMMLU (Marco evaluation) 71.3% ▄···▄·█·▄·▄● beats 7/7 medium · avg 59% 1.3% at its level
INCLUDE (maker) 60.1% ▂····▂▂▄●▄█▄ beats 12/36 medium · avg 55% 1.3% at its level · small run
AGIEval (Marco suite unspecified) 51.7% ▄··▄█▄··▄··● beats 6/7 medium · avg 46% 1.0% blunt · at its level
MMLU-ProX (maker) 61.6% ▁·▁▁▁●▁▃▃█▅▄ beats 5/36 medium · avg 54% 0.8% at its level · small run
LiveBench 2024-11-25 (maker) 63.0% ●··▂·▂·▂█▆▂▂ beats 0/13 medium · avg 53% 0.7% at its level · small run
Open PT-LLM Leaderboard average (… 74.7% ▃▁▁▁▁▁▁▁▂█●▁ beats 221/276 medium · avg 72% 0.7% at its level · noisy
Multi-IF (aggregation unspecified… 69.0% ▁▁▁▁▁▁●▄█▄▁▂ beats 11/41 medium · avg 62% 0.7% at its level · small run
WritingBench (maker) 83.4% ▁·▁▂▁··●█▂▂▁ beats 6/20 easy · avg 69% 0.6% small run
Creative Writing v3 (maker) 83.5% ▂····▂▆··●█▆ beats 5/14 medium · avg 64% 0.6% at its level · small run
EuroEval HellaSwag (English), tes… 77.3% ▃·▁▁▂▁▂▅█●▄▁ beats 65/100 medium · avg 67% 0.5% sharp · at its level · noisy
ZebraLogic (maker) 80.2% █·▁▁▁●·▄▄▁▄▁ beats 13/24 medium · avg 45% 0.5% at its level · noisy · small run
EuroEval Life in the UK (English … 81.7% ▁▁▁▁▁▁▁▂▄●▄▃ beats 74/146 easy · avg 82% 0.3% noisy
Chess Puzzles (Epoch) 4.0% ●▁▁▁▁▂▃▃▃▂▁▁ beats 35/137 very hard · avg 5.8% 0.3% sharp · too hard for it · noisy
source results share of the score
Qwen/Qwen3-4B-Instruct-2507 model card … 25 33% ▇▇▇▇
Marco-MoE: Open Multilingual Mixture-of… 11 24% ▇▇▇
Qwen (submitted / maker-reported) 8 24% ▇▇▇
Epoch AI 3 7% ▇
Roblox LLM Leaderboard 2 4% ▇
Synthesize and Reward: Reinforcement Le… 1 3% ▇
BRIDGE Medical Leaderboard 1 2% ▇
Berkeley Function Calling Leaderboard r… 1 1% ▇
EuroEval leaderboards 2 1% ▇
Open Portuguese LLM Leaderboard 1 1% ▇
prior: pull toward the middle 0% ▇
weighted average = 55.3
Licences: Qwen/Qwen3-4B-Instruct-2507 model card ([apache-2.0](../../licenses/Qwen--Qwen3-4B-Instruct-2507.txt)) · Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling ([CC BY 4.0](../../licenses/cards-final-NOTICE.txt)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Roblox LLM Leaderboard ([MIT](../../licenses/roblox-llm-leaderboard-MIT.txt)) · Synthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · BRIDGE Medical Leaderboard ([MIT](../../licenses/bridge-medical-leaderboard-MIT.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt))
## How to run it
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Qwen3 4B Instruct 2507 (Q4_K_M, needs ~2.5 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Qwen3-4B-Instruct-2507-GGUF/tree/4edb920b6f14e3b9284d4502a6485103d72cde05))
```sh
llama cli -hf lmstudio-community/Qwen3-4B-Instruct-2507-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf lmstudio-community/Qwen3-4B-Instruct-2507-GGUF:Q4_K_M
```
Other versions: Q8_0 (~4.3 GB) `llama cli -hf lmstudio-community/Qwen3-4B-Instruct-2507-GGUF:Q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Qwen3 4B Instruct 2507 (Q4_K_M, needs ~2.5 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Qwen3-4B-Instruct-2507-GGUF/tree/4edb920b6f14e3b9284d4502a6485103d72cde05))
```sh
lms get https://huggingface.co/lmstudio-community/Qwen3-4B-Instruct-2507-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~4.3 GB) `lms get https://huggingface.co/lmstudio-community/Qwen3-4B-Instruct-2507-GGUF@Q8_0 -y` · 4bit (~2.3 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/Qwen3-4B-Instruct-2507-MLX-4bit -y` · 8bit (~4.3 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/Qwen3-4B-Instruct-2507-MLX-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Qwen3 4B Instruct 2507 (Q4_K_M, needs ~2.91 GB, from [ollama.com](https://ollama.com/library/qwen3/tags))
```sh
ollama run qwen3:4b-instruct
```
Other versions: Q8_0 (~5.32 GB) `ollama run qwen3:4b-instruct-2507-q8_0`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run Qwen3 4B Instruct 2507 (4bit, needs ~2.3 GB, Mac only, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Qwen3-4B-Instruct-2507-MLX-4bit/tree/7479749d4b89a9c12057d32a0e5ed07b2829e721))
```sh
mlx_lm.chat --model lmstudio-community/Qwen3-4B-Instruct-2507-MLX-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model lmstudio-community/Qwen3-4B-Instruct-2507-MLX-4bit
```
Other versions: 8bit (~4.3 GB, Mac only) `mlx_lm.chat --model lmstudio-community/Qwen3-4B-Instruct-2507-MLX-8bit`
Model repository and documentation: [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507)
## Published speed measurements
Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine.
hardware precision speed engine and settings
Apple M4 Max (64GB; CPU/GPU… bf16 (unquantiz… 52.47 tok/s mlx-lm 0.28.2, mlx 0.29.2.dev2025…
Apple M4 Max (64GB; CPU/GPU… MLX 8-bit affin… 86.907 tok/s mlx-lm 0.28.2, mlx 0.29.2.dev2025…
Apple M4 Max (64GB; CPU/GPU… MLX 6-bit affin… 104.68 tok/s mlx-lm 0.28.2, mlx 0.29.2.dev2025…
Apple M4 Max (64GB; CPU/GPU… MLX 5-bit affin… 110.29 tok/s mlx-lm 0.28.2, mlx 0.29.2.dev2025…
Apple M4 Max (64GB; CPU/GPU… MLX 5-bit affin… 116.39 tok/s mlx-lm 0.28.2, mlx 0.29.2.dev2025…
Apple M4 Max (64GB; CPU/GPU… MLX 4-bit affin… 126 tok/s mlx-lm 0.28.2, mlx 0.29.2.dev2025…
Apple M4 Max (64GB; CPU/GPU… MLX 4-bit affin… 134.52 tok/s mlx-lm 0.28.2, mlx 0.29.2.dev2025…
RTX 2080 q8_0 60.34 tok/s ollama (version not recorded) · l…
Each speed links to its source, which has the full settings.
## Quantization evidence
How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent. They don't describe every download of that precision.
precision test full → quantized kept
INT4 W4A16 (weight-only INT4,… GSM8K-Platinum — published row 'G… 0.956 → 0.961 100.5%
INT4 W4A16 (weight-only INT4,… MATH-500 — published row 'Math 50… 0.843 → 0.842 99.8%
INT4 W4A16 (weight-only INT4,… IFEval — published row 'IfEval'; … 0.89 → 0.886 99.6%
INT4 W4A16 (weight-only INT4,… MMLU (CoT) — published row 'MMLU-… 0.775 → 0.768 99.1%
INT4 W4A16 (weight-only INT4,… MMLU-Pro — published row 'MMLU-Pr… 0.701 → 0.689 98.3%
INT8 W8A8 (weights INT8 per-c… GSM8K-Platinum — published row 'G… 0.956 → 0.957 100.1%
INT8 W8A8 (weights INT8 per-c… IFEval — published row 'IfEval'; … 0.89 → 0.89 100.0%
INT8 W8A8 (weights INT8 per-c… GPQA Diamond — published row 'GPQ… 0.452 → 0.452 100.0%
INT8 W8A8 (weights INT8 per-c… MMLU-Pro — published row 'MMLU-Pr… 0.701 → 0.7 99.8%
INT8 W8A8 (weights INT8 per-c… MMLU (CoT) — published row 'MMLU-… 0.775 → 0.77 99.3%
INT8 W8A8 (weights INT8 per-c… MATH-500 — published row 'Math 50… 0.843 → 0.836 99.2%
INT8 W8A8 (weights INT8 per-c… AIME 2025 — published row 'AIME 2… 0.476 → 0.467 98.0%
MLX 4-bit affine quantization… MMLU-Pro (mlx_lm.evaluate --task … 0.64 → 0.615 96.0%
MLX 4-bit affine quantization… MMLU-Pro (mlx_lm.evaluate --task … 0.64 → 0.607 94.8%
MLX 5-bit affine quantization… MMLU-Pro (mlx_lm.evaluate --task … 0.64 → 0.632 98.6%
MLX 5-bit affine quantization… MMLU-Pro (mlx_lm.evaluate --task … 0.64 → 0.624 97.4%
MLX 6-bit affine quantization… MMLU-Pro (mlx_lm.evaluate --task … 0.64 → 0.635 99.2%
MLX 8-bit affine quantization… MMLU-Pro (mlx_lm.evaluate --task … 0.64 → 0.638 99.7%
Each kept value links to its source.
[interactive version](https://whatmodel.sh/model/qwen3-4b-instruct-2507) · [JSON](https://whatmodel.sh/api?format=json&all=1)