[whatmodel.sh](../../README.md) / Qwen3 4B Thinking 2507
# Qwen3 4B Thinking 2507
Alibaba · released 2025-08 · 4.02B params · apache-2.0
**score 56.3** ±0.8 · #256 of 732 · from 34 benchmark results across 9 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Qwen3 4B Thinking 2507 among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█●▅▅▄▄▂▁ #176 of 313 18% 6
coding ▁▁▁▃▅▇●▇▅▄▃▁ #234 of 469* 2.6% 1
chat & writing ▁▁▁▃▆▄▆●▆▇▇▃ #191 of 402* 6.0% 2
math ▁▁▃▅█▆●█▄▂▁▁ #187 of 431 28% 7
agent tasks · #176 of 313 · 18% of its score
Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md), [DeepSeek R1 0528](deepseek-r1-0528.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Jamba 1.5 Large](jamba-1-5-large.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
τ²-bench Telecom (maker) 27.2% ▁·▅●▆▅▂▃▃▂▃▁ beats 20/89 very hard · avg 13% 8.2% sharp · at its level · small run
τ-bench Retail (maker) 66.1% ▁····▃·▂▇●▂▆ beats 15/28 medium · avg 42% 2.6% at its level · small run
τ²-bench Retail (maker) 53.5% ▁····▂▁●▆█▃▁ beats 6/44 hard · avg 34% 2.3% at its level · small run
τ-bench Airline (maker) 48.0% ▁▁▁·▅▄▁█●▁▁▇ beats 17/28 hard · avg 28% 2.3% at its level · small run
τ²-bench Airline (maker) 58.0% ▁▁·▂▁▃▂▁●▅▂▃ beats 25/42 hard · avg 31% 1.6% at its level · small run
BFCL v3 (maker) 71.2% ▁▁▁▁▁▃▅█▆▅●▁ beats 156/171 medium · avg 61% 0.9% blunt · at its level · small run
coding · #234 of 469 · 2.6% of its score
Beats [Qwen3.5 4B](qwen3-5-4b.md), [Qwen3-VL 32B Instruct](qwen3-vl-32b-instruct.md), [Solar Open 100B (102B-A12B)](solar-open-100b.md) here, though they rank higher overall. Trails [QwQ-32B-Preview](qwq-32b-preview.md), [Ministral 3 3B Reasoning 2512](ministral-3-3b-reasoning.md), [GPT-4 (0613)](gpt-4-0613.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LiveCodeBench v6 (Feb–May 2025) (… 55.2% ▁▁▃▁●▅▄▇▅▅▃█ beats 8/38 medium · avg 43% 2.6% at its level · small run
chat & writing · #191 of 402 · 6.0% of its score
Beats [Seed-OSS 36B Instruct](seed-oss-36b.md), [Gemma 2 9B IT SimPO](gemma-2-9b-simpo.md), [Nemotron-Cascade-14B-Thinking](nemotron-cascade-14b-thinking.md) here, though they rank higher overall. Trails [LFM2.5 1.2B Instruct](lfm2-5-1-2b-instruct.md), [LFM2.5 1.2B Thinking](lfm2-5-1-2b-thinking.md), [LFM2.5 8B-A1B](lfm2-5-8b-a1b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena-Hard v2 aggregate (maker) 34.9% ▁▁··●▃▁▃█▁▁▇ beats 2/22 hard · avg 20% 4.5% sharp · at its level · small run
IFEval (aggregation unspecified) … 87.4% ▁▂▄▂▂▄▆●█▅▂▁ beats 88/145 easy · avg 80% 1.5% small run
math · #187 of 431 · 28% of its score
Beats [MiniMax-M2.1](minimax-m2-1.md), [DeepSeek V3 0324](deepseek-v3-0324.md), [DeepSeek V3 (Dec '24)](deepseek-v3.md) here, though they rank higher overall. Trails [Phi-4-mini-reasoning 3.8B](phi-4-mini-reasoning.md), [OpenThinker3 7B](openthinker3-7b.md), [Olmo 3 7B Think](olmo-3-7b-think.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
HMMT 2025 (edition unspecified) (… 55.5% ▃█●▅▇▅▁·▁··▁ beats 13/32 hard · avg 34% 6.2% sharp · at its level · small run
AIME 2025 (maker) 81.3% ▁·▁▁▁▂▅▆█●▅▁ beats 87/133 medium · avg 59% 5.3% sharp · small run
MathArena ArXivMath January 2026 … 23.9% ●··▄▂▄█▆▆▄▄█ beats 0/23 hard · avg 23% 5.2% at its level
MathArena ArXivMath February 2026… 18.0% ▁●▁▁█▇▄▃▁▃·▃ beats 1/24 very hard · avg 9.4% 4.5% typical
MathArena ArXivMath December 2025… 32.4% ▃·●▃█▁·▄·▁▁▁ beats 2/18 hard · avg 19% 3.1% at its level
PolyMATH (maker) 46.2% ▁···▃▁·●█▄▄▄ beats 8/23 hard · avg 35% 2.2% at its level · small run
GSM8K 94.9% ▃··▃··▃▃·▃·● beats 7/7 very easy · avg 93% 1.4% sharp · maxed out · noisy
other tests (17)
test result vs the others in its run difficulty counts why
MMLU-Pro (maker) 74.0% ▁▁·▁▁▁▃▂▄●█▁ beats 98/188 medium · avg 66% 6.1% several runs
GPQA (subset unspecified) (maker) 65.8% ▂▁▂▂▄▃▆▃▅●▃▃ beats 69/106 hard · avg 51% 5.8% sharp · at its level · small run
Medmarks-V weighted mean win rate… 48.9% ▂▃▅▂▆▃●▅█▅▃▆ beats 15/34 medium · avg 41% 5.6% blunt · at its level
BRIDGE clinical text, zero-shot a… 40.0% ▁▁▁▁▁▂▂▃▄▆●▃ beats 76/97 hard · avg 31% 4.1% blunt · at its level
KINA (Knowledge Index of Noah's A… 24.8% ▁▁▂▁▃●█▆▄▅▂▃ beats 9/39 very hard · avg 21% 3.5% typical
INCLUDE (maker) 64.4% ▂····▂▂▄●▄█▄ beats 15/36 medium · avg 59% 2.6% at its level · small run
OJBench (maker) 17.9% ▁···▁▃·●▄▅██ beats 5/26 very hard · avg 13% 2.5% small run
MMLU-Redux (maker) 86.1% ▁▁▃▁▂▃▄●▇▅█▄ beats 23/63 easy · avg 81% 2.4% small run
SuperGPQA (maker) 47.8% ▂▁▂▂▅▆●▇▇▇█▃ beats 18/46 medium · avg 39% 1.9% at its level · small run
Creative Writing v3 (maker) 75.6% ▂····●▆··▆█▆ beats 1/14 easy · avg 66% 1.7% small run
MMLU-ProX (maker) 64.2% ▁·▁▁▁●▁▃▃█▅▄ beats 6/36 medium · avg 59% 1.6% at its level · small run
WritingBench (maker) 83.3% ▁·▁▂▁··●█▂▂▁ beats 5/20 easy · avg 71% 1.6% small run
LiveBench 2024-11-25 (maker) 71.8% ▄··▂·●·▂█▆▂▂ beats 3/13 medium · avg 62% 1.5% at its level · small run
Open PT-LLM Leaderboard average (… 59.1% ▃▁▁▁▁▁▁▁●█▅▁ beats 87/276 medium · avg 72% 1.4% at its level · noisy
Multi-IF (aggregation unspecified… 77.3% ▁▁▁▁▁▁▃▄●▄▁▂ beats 24/41 medium · avg 65% 1.3% at its level · small run
EuroEval HellaSwag (English), tes… 74.0% ▃·▁▁▂▁▂▅●▇▄▁ beats 58/100 medium · avg 67% 1.1% sharp · at its level · noisy
EuroEval Life in the UK (English … 89.1% ▁▁▁▁▁▁▁▂▄█●▃ beats 113/146 easy · avg 82% 0.6% noisy
source results share of the score
Qwen/Qwen3-4B-Thinking-2507 model card … 23 66% ▇▇▇▇▇▇▇▇
MathArena ArXivMath / BrokenArXiv / ArX… 3 13% ▇▇
Medmarks: A Comprehensive Open-Source L… 1 6% ▇
BRIDGE Medical Leaderboard 1 4% ▇
Knowledge Index of Noah's Ark 1 3% ▇
Qwen/Qwen3.5-0.8B model card (submitted… 1 3% ▇
EuroEval leaderboards 2 2% ▇
Unsupervised Confidence Calibration for… 1 1% ▇
Open Portuguese LLM Leaderboard 1 1% ▇
prior: pull toward the middle 0% ▇
weighted average = 56.3
Licences: Qwen/Qwen3-4B-Thinking-2507 model card ([apache-2.0](../../licenses/Qwen--Qwen3-4B-Thinking-2507.txt)) · MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets ([CC BY-SA 4.0](../../licenses/matharena-sharealike-NOTICE.txt)) · Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks ([CC BY 4.0](../../licenses/paper-2605.01417v1-CC-BY-4.0.txt)) · BRIDGE Medical Leaderboard ([MIT](../../licenses/bridge-medical-leaderboard-MIT.txt)) · Knowledge Index of Noah's Ark ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Qwen/Qwen3.5-0.8B model card ([apache-2.0](../../licenses/Qwen--Qwen3.5-0.8B.txt)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · Unsupervised Confidence Calibration for Reasoning LLMs from a Single Generation ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt))
## How to run it
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Qwen3 4B Thinking 2507 (Q4_K_M, needs ~2.5 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Qwen3-4B-Thinking-2507-GGUF/tree/0252537265b088d8c6f681066e1d4c2f415d090b))
```sh
llama cli -hf lmstudio-community/Qwen3-4B-Thinking-2507-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf lmstudio-community/Qwen3-4B-Thinking-2507-GGUF:Q4_K_M
```
Other versions: Q8_0 (~4.3 GB) `llama cli -hf lmstudio-community/Qwen3-4B-Thinking-2507-GGUF:Q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Qwen3 4B Thinking 2507 (Q4_K_M, needs ~2.5 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Qwen3-4B-Thinking-2507-GGUF/tree/0252537265b088d8c6f681066e1d4c2f415d090b))
```sh
lms get https://huggingface.co/lmstudio-community/Qwen3-4B-Thinking-2507-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~4.3 GB) `lms get https://huggingface.co/lmstudio-community/Qwen3-4B-Thinking-2507-GGUF@Q8_0 -y` · 4bit (~2.3 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/Qwen3-4B-Thinking-2507-MLX-4bit -y` · 8bit (~4.3 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/Qwen3-4B-Thinking-2507-MLX-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Qwen3 4B Thinking 2507 (Q4_K_M, needs ~2.91 GB, from [ollama.com](https://ollama.com/library/qwen3/tags))
```sh
ollama run qwen3:4b
```
Other versions: Q8_0 (~5.32 GB) `ollama run qwen3:4b-thinking-2507-q8_0`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run Qwen3 4B Thinking 2507 (4bit, needs ~2.3 GB, Mac only, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Qwen3-4B-Thinking-2507-MLX-4bit/tree/1fc4709626fbf06a76b7d0231293a3d23a539f31))
```sh
mlx_lm.chat --model lmstudio-community/Qwen3-4B-Thinking-2507-MLX-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model lmstudio-community/Qwen3-4B-Thinking-2507-MLX-4bit
```
Other versions: 8bit (~4.3 GB, Mac only) `mlx_lm.chat --model lmstudio-community/Qwen3-4B-Thinking-2507-MLX-8bit`
Model repository and documentation: [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507)
## Published speed measurements
Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine.
hardware precision speed engine and settings
Apple M4, 16 GB q4_k_m 35.05 tok/s ollama (version not recorded) · l…
Intel CoffeeLake-S GT2 [UHD… q4_k_m 5.9 tok/s ollama (version not recorded) · l…
Intel(R) Arc(TM) B390 GPU, … q4_k_m · layers… 35.53 tok/s ollama (version not recorded) · l…
RTX 2080 q4_k_m 99.02 tok/s ollama (version not recorded) · l…
RTX 2080 q4_k_m 99.47 tok/s ollama (version not recorded) · l…
Each speed links to its source, which has the full settings.
## Quantization evidence
How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent. They don't describe every download of that precision.
precision test full → quantized kept
INT4 W4A16 (weight-only INT4,… MATH-500 — published row 'Math 50… 0.894 → 0.906 101.3%
INT4 W4A16 (weight-only INT4,… IFEval — published row 'IfEval (0… 0.902 → 0.902 100.0%
INT4 W4A16 (weight-only INT4,… GSM8K-Platinum — published row 'G… 0.953 → 0.949 99.6%
INT4 W4A16 (weight-only INT4,… AIME 2025 — published row 'AIME 2… 0.738 → 0.721 97.7%
INT4 W4A16 (weight-only INT4,… MMLU-Pro — published row 'MMLU-Pr… 0.736 → 0.713 96.9%
INT4 W4A16 (weight-only INT4,… GPQA Diamond — published row 'GPQ… 0.689 → 0.633 91.8%
INT8 W8A8 (weights INT8 per-c… GSM8K-Platinum — published row 'G… 0.953 → 0.958 100.5%
INT8 W8A8 (weights INT8 per-c… MATH-500 — published row 'Math 50… 0.894 → 0.893 99.8%
INT8 W8A8 (weights INT8 per-c… IFEval — published row 'IfEval (0… 0.902 → 0.898 99.6%
INT8 W8A8 (weights INT8 per-c… MMLU-Pro — published row 'MMLU-Pr… 0.736 → 0.727 98.8%
INT8 W8A8 (weights INT8 per-c… AIME 2025 — published row 'AIME 2… 0.738 → 0.717 97.2%
INT8 W8A8 (weights INT8 per-c… GPQA Diamond — published row 'GPQ… 0.689 → 0.667 96.7%
Each kept value links to its source.
[interactive version](https://whatmodel.sh/model/qwen3-4b-thinking-2507) · [JSON](https://whatmodel.sh/api?format=json&all=1)