[whatmodel.sh](../../README.md) / Qwen3 8B
# Qwen3 8B
Alibaba · released 2025-04 · 8.19B params · apache-2.0
**score 55.6** ±0.6 · #267 of 732 · from 47 benchmark results across 21 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Qwen3 8B among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆●▄▅▅▄▄▂▁ #201 of 313 19% 6
coding ▁▁▁▃▅▇●▇▅▄▃▁ #257 of 469 5.6% 2
chat & writing ▁▁▁▃▆▄▆●▆▇▇▃ #184 of 402* 1.8% 1
math ▁▁▃▅█▆●█▄▂▁▁ #174 of 431 26% 8
agent tasks · #201 of 313 · 19% of its score
Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md), [DeepSeek R1 0528](deepseek-r1-0528.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Mellum2 12B-A2.5B Instruct](mellum2-12b-a2-5b-instruct.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
τ²-bench Telecom 19.1% ●··██··██··█ beats 0/5 very hard · avg 9.9% 12% sharp · several runs
Terminal-Bench Hard 2.1% ▂●▆▂▂▆▂▂█▅▅▂ beats 1/27 very hard · avg 1.3% 2.1% sharp · too hard for it
BFCL v3 66.3% ▁▁▁▁▁▃▅█▅●▁▁ beats 100/114 medium · avg 59% 2.1% blunt · at its level · several runs
BFCL V4 (December 2025 archive) 42.6% ▁▁▁▂██▅●▆▂▂▂ beats 49/77 hard · avg 31% 1.6% at its level
EEGAgentBench overall (tool-based… 33.2% ▂·▁●··▇▇█▂·▁ beats 3/25 hard · avg 28% 0.9% blunt · at its level
BFCL (version not stated) 60.2% ▄●▄▂·█▆·▄··▂ beats 4/17 medium · avg 63% 0.7% blunt · at its level
coding · #257 of 469 · 5.6% of its score
Beats [Qwen3-VL 32B Instruct](qwen3-vl-32b-instruct.md), [Solar Open 100B (102B-A12B)](solar-open-100b.md), [GLM-4.7-Flash](glm-4-7-flash.md) here, though they rank higher overall. Trails [DeepSeek LLM 67B Chat](deepseek-llm-67b-chat.md), [QwQ-32B-Preview](qwq-32b-preview.md), [InternLM2.5 20B Chat](internlm2-5-20b-chat.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LiveCodeBench v6 (window unspecif… 51.0% ▃▅█▅●▃▅···▃█ beats 8/16 hard · avg 35% 5.1% at its level · several runs
SciCode main problems 6.7% ▄··▄····●▄·▄ beats 3/5 very hard · avg 3.8% 0.5% too hard for it
chat & writing · #184 of 402 · 1.8% of its score
Beats [Seed-OSS 36B Instruct](seed-oss-36b.md), [Gemma 2 9B IT SimPO](gemma-2-9b-simpo.md), [Nemotron-Cascade-14B-Thinking](nemotron-cascade-14b-thinking.md) here, though they rank higher overall. Trails [LFM2.5 1.2B Instruct](lfm2-5-1-2b-instruct.md), [LFM2.5 1.2B Thinking](lfm2-5-1-2b-thinking.md), [LFM2.5 8B-A1B](lfm2-5-8b-a1b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
IFEval (aggregation unspecified) … 85.0% ▃··▃···██●▃▃ beats 9/11 easy · avg 77% 1.8% typical
math · #174 of 431 · 26% of its score
Beats [MiMo-V2.5 (310B-A15B)](mimo-v2-5.md), [MiniMax-M2.1](minimax-m2-1.md), [Mistral Medium 3.5 (128B)](mistral-medium-3-5.md) here, though they rank higher overall. Trails [OpenThinker3 7B](openthinker3-7b.md), [Olmo 3 7B Think](olmo-3-7b-think.md), [OpenThinker2 32B](openthinker2-32b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
AIME 2025 (maker) 67.3% ▃·▃▃█··▃●▃·█ beats 7/12 medium · avg 52% 7.1% sharp · at its level · several runs
AIME 2024 77.9% ●▄···█·····▄ beats 1/5 medium · avg 62% 4.4% sharp
HMMT 2025 (edition unspecified) 41.0% ●··▄▄·▄····█ beats 0/5 hard · avg 29% 3.8% sharp · at its level
OMEGA (math generalisation) 43.4% ▄······█▄▄·● beats 6/6 hard · avg 31% 3.3% at its level · several runs
MATH (subset unspecified) 95.1% ▃·····▃···▅● beats 4/6 very easy · avg 91% 2.8% maxed out · several runs
OTIS mock AIME 2024–2025 56.1% ▃▃▄▃▃▃●▇█▃▂▅ beats 85/197 very hard · avg 7.2% 1.9% sharp · noisy
MATH-500 97.4% ▄·▄·····█··● beats 4/5 very easy · avg 92% 1.7% maxed out
GSM8K 94.9% ▃··▃··▃▃·▃·● beats 6/7 very easy · avg 93% 0.8% sharp · maxed out · noisy
other tests (21)
test result vs the others in its run difficulty counts why
GPQA Diamond 56.8% ▁▁▁▁▂▄▆●▅▇█▅ beats 84/211 hard · avg 39% 10% sharp · at its level · several runs
MMLU-Pro (maker) 74.6% ▄·▄·▄▄▄▄●▄██ beats 7/12 medium · avg 63% 6.9% at its level · several runs
GPQA (subset unspecified) (maker) 62.0% ▂····▂·▂▂●▆█ beats 5/14 hard · avg 49% 5.5% sharp · at its level
MMLU-Redux (maker) 87.5% ▃··▃·▃·█●▅▃▅ beats 6/11 easy · avg 79% 3.5% typical
Medmarks-V weighted mean win rate… 50.2% ▂▃▅▂▆▃●▅█▅▃▆ beats 17/34 medium · avg 41% 2.8% blunt · at its level
SuperGPQA (maker) 47.6% ▄··▄·▄▄▄●▄██ beats 6/11 medium · avg 37% 2.6% at its level
CPTUB average (SpeakLeash, 0-5 ju… 61.2% ▁▂▂▂▃●▃▆▅▄▂▃ beats 35/95 medium · avg 65% 2.2% at its level
EQ-Bench 3 (emotional intelligenc… 1083.3 ▁·▂▁●▅▅▃█▄▁▁ beats 11/62 avg 1034 2.1% typical
BRIDGE clinical text, zero-shot a… 40.0% ▁▁▁▁▁▂▂▃▄▆●▃ beats 75/97 hard · avg 31% 2.0% blunt · at its level
AraGen-v3 3C3H score (Inception, … 26.6% ▁▁·▁▂▂▄●▄▅▄▄ beats 23/66 hard · avg 29% 2.0% at its level
Humanity's Last Exam 4.2% ●··██···█·██ beats 0/5 very hard · avg 3.7% 1.8% too hard for it
KINA (Knowledge Index of Noah's A… 22.3% ▁▁▂▁●▄█▆▄▅▂▃ beats 6/39 very hard · avg 21% 1.7% typical
Open Persian LLM Leaderboard v2 a… 55.0% ▂▄▂▃█●▃▆▃▅▂▂ beats 41/81 medium · avg 57% 1.5% blunt · at its level
DiagnosticIQ industrial maintenan… 46.2% ▁··●▁▁▁▅█▅▁▅ beats 1/24 hard · avg 51% 0.7% blunt · at its level
Open PT-LLM Leaderboard average (… 78.3% ▃▁▁▁▁▁▁▁▂█●▁ beats 243/276 medium · avg 72% 0.7% at its level · noisy
IFBench (aggregation unspecified) 35.3% █●█···██···█ beats 1/5 hard · avg 34% 0.6% at its level
EuroEval HellaSwag (English), tes… 75.5% ▃·▁▁▂▁▂▅●▇▄▁ beats 60/100 medium · avg 67% 0.6% sharp · at its level · noisy
Chess Puzzles (Epoch) 5.0% ●▁▁▁▁▂▃▃▃▂▁▁ beats 21/137 very hard · avg 5.8% 0.3% sharp · too hard for it · noisy
EuroEval Life in the UK (English … 89.8% ▁▁▁▁▁▁▁▂▄█●▃ beats 115/146 easy · avg 82% 0.3% noisy
Vectara hallucination leaderboard… 97.0% ▁▁▂▃▄▄█●▃▂▂▁ beats 102/140 very easy · avg 94% 0.2% blunt · maxed out · noisy
Vectara hallucination leaderboard… 95.2% ▁▂▁▂▅█▂▃▄●▁▁ beats 92/98 very easy · avg 90% 0.0% blunt
source results share of the score
Falcon-H1R: Pushing the Reasoning Front… 12 31% ▇▇▇▇
Qwen (submitted / maker-reported) 8 25% ▇▇▇
Epoch AI 3 7% ▇
MUA-RL: Multi-turn User-interacting Age… 1 6% ▇
allenai/Olmo-3-7B-Think model card 2 5% ▇
Synthesize and Reward: Reinforcement Le… 1 3% ▇
Medmarks: A Comprehensive Open-Source L… 1 3% ▇
Berkeley Function Calling Leaderboard r… 2 2% ▇
allenai/Olmo-3-7B-Instruct model card 3 2% ▇
SpeakLeash CPTUB 1 2% ▇
EQ-Bench 3 canonical leaderboard results 1 2% ▇
BRIDGE Medical Leaderboard 1 2% ▇
Inception Arabic/Hindi leaderboards 1 2% ▇
Knowledge Index of Noah's Ark 1 2% ▇
Open Persian LLM Leaderboard v2 1 1% ▇
EEGAgentBench: Benchmarking LLM Agents … 1 1% ▇
EuroEval leaderboards 2 1% ▇
Unsupervised Confidence Calibration for… 1 1% ▇
DiagnosticIQ: A Benchmark for LLM-Based… 1 1% ▇
Open Portuguese LLM Leaderboard 1 1% ▇
Vectara hallucination leaderboard 2 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 55.6
Licences: Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling ([CC BY 4.0](../../licenses/report-2601.02346v1.txt)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · allenai/Olmo-3-7B-Think model card ([Apache-2.0](../../licenses/allenai--Olmo-3-7B-Think.txt)) · Synthesize and Reward: Reinforcement Learning for Multi-Step Tool Use in Live Environments ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks ([CC BY 4.0](../../licenses/paper-2605.01417v1-CC-BY-4.0.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · allenai/Olmo-3-7B-Instruct model card ([Apache-2.0](../../licenses/allenai--Olmo-3-7B-Instruct.txt)) · SpeakLeash CPTUB ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · EQ-Bench 3 canonical leaderboard results ([MIT](../../licenses/eqbench3-MIT.txt)) · BRIDGE Medical Leaderboard ([MIT](../../licenses/bridge-medical-leaderboard-MIT.txt)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Knowledge Index of Noah's Ark ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · Unsupervised Confidence Calibration for Reasoning LLMs from a Single Generation ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
### Through an API
api (Alibaba) · $0.31/M blended · $0.18 in / $0.7 out per 1M tokens · [alibabacloud.com](https://www.alibabacloud.com/help/en/model-studio/models)
[price source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/providers/alibaba/models/qwen3-8b.toml): Standard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://huggingface.co/Qwen/Qwen3-8B/blob/b968826d9c46dd6066d109eabc6255188de91218/README.md): Native/default model-card context. Any YaRN extension must be configured separately; hosted provider limits may differ.
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Qwen3 8B (Q4_K_M, needs ~5 GB, from [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen3-8B-GGUF/tree/7c41481f57cb95916b40956ab2f0b139b296d974))
```sh
llama cli -hf Qwen/Qwen3-8B-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf Qwen/Qwen3-8B-GGUF:Q4_K_M
```
Other versions: Q8_0 (~8.7 GB) `llama cli -hf Qwen/Qwen3-8B-GGUF:Q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Qwen3 8B (Q4_K_M, needs ~5 GB, from [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen3-8B-GGUF/tree/7c41481f57cb95916b40956ab2f0b139b296d974))
```sh
lms get https://huggingface.co/Qwen/Qwen3-8B-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~8.7 GB) `lms get https://huggingface.co/Qwen/Qwen3-8B-GGUF@Q8_0 -y` · 4bit (~4.4 GB, Mac only) `lms get https://huggingface.co/Qwen/Qwen3-8B-MLX-4bit -y` · 8bit (~8.4 GB, Mac only) `lms get https://huggingface.co/Qwen/Qwen3-8B-MLX-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Qwen3 8B (Q4_K_M, needs ~5.41 GB, from [ollama.com](https://ollama.com/library/qwen3/tags))
```sh
ollama run qwen3:8b
```
Other versions: Q8_0 (~10.33 GB) `ollama run qwen3:8b-q8_0`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run Qwen3 8B (4bit, needs ~4.4 GB, Mac only, from [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen3-8B-MLX-4bit/tree/383413e909f3bc5303ce195ebbdf0339c5a1a2a3))
```sh
mlx_lm.chat --model Qwen/Qwen3-8B-MLX-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model Qwen/Qwen3-8B-MLX-4bit
```
Other versions: 8bit (~8.4 GB, Mac only) `mlx_lm.chat --model Qwen/Qwen3-8B-MLX-8bit`
Model repository and documentation: [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen3-8B)
## Published speed measurements
Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine.
hardware precision speed engine and settings
NVIDIA GeForce RTX 5090 NVFP4 weights, … 151.4 tok/s TensorRT-LLM v1.1.0, PyTorch back…
NVIDIA GeForce RTX 5090 NVFP4 weights, … 93.1 tok/s TensorRT-LLM v1.1.0, C++ engine (…
NVIDIA GeForce RTX 5090 GGUF BF16 92.2 tok/s llama.cpp (version not stated) · …
Apple M2 Pro (16-core GPU),… Q4_K_M 21.05 tok/s ollama 0.30.8 · whatmodel bench.s…
Apple M2 Pro (16-core GPU),… q4_k_m 27.94 tok/s ollama 0.30.8 · whatmodel bench.p…
Apple M3 Pro, 36 GB q4_k_m 24.56 tok/s ollama (version not recorded) · l…
Apple M4, 16 GB q4_k_m 18.77 tok/s ollama (version not recorded) · l…
Apple M4, 16 GB q4_k_m 17.67 tok/s ollama (version not recorded) · l…
Apple M4, 16 GB q4_k_m 19.74 tok/s ollama (version not recorded) · l…
Each speed links to its source, which has the full settings.
## Quantization evidence
How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent. They don't describe every download of that precision.
precision test full → quantized kept
AutoRound 2-bit (group size 3… WMT24++ X->English COMET (wmt22-c… 0.782 → 0.628 80.4%
AutoRound 2-bit (group size 3… WMT24++ English->X COMET (wmt22-c… 0.702 → 0.432 61.6%
AutoRound 4-bit (group size 1… WMT24++ X->English COMET (wmt22-c… 0.782 → 0.773 99.0%
AutoRound 4-bit (group size 1… WMT24++ English->X COMET (wmt22-c… 0.702 → 0.689 98.1%
AutoRound AutoScheme mixed-pr… Average accuracy of 10 lm-eval ta… 0.633 → 0.613 96.9%
AutoRound AutoScheme mixed-pr… MMLU (lm-eval default mmlu task; … 0.73 → 0.692 94.8%
AutoRound AutoScheme mixed-pr… MATH-500 (EvalScope via llama-ser… 0.798 → 0.7 87.7%
AutoRound AutoScheme mixed-pr… GPQA Diamond (EvalScope via llama… 0.46 → 0.399 86.8%
AutoRound AutoScheme mixed-pr… MATH-500 (EvalScope via llama-ser… 0.798 → 0.698 87.5%
AutoRound AutoScheme mixed-pr… GPQA Diamond (EvalScope via llama… 0.46 → 0.369 80.2%
AutoRound AutoScheme mixed-pr… Average accuracy of 10 lm-eval ta… 0.633 → 0.607 96.0%
AutoRound AutoScheme mixed-pr… MMLU (lm-eval default mmlu task; … 0.73 → 0.69 94.5%
AutoRound AutoScheme mixed-pr… MATH-500 (EvalScope via llama-ser… 0.798 → 0.714 89.5%
AutoRound AutoScheme mixed-pr… GPQA Diamond (EvalScope via llama… 0.46 → 0.348 75.8%
AutoRound AutoScheme mixed-pr… Average accuracy of 10 lm-eval ta… 0.633 → 0.609 96.3%
AutoRound AutoScheme mixed-pr… MMLU (lm-eval default mmlu task; … 0.73 → 0.693 94.9%
AutoRound AutoScheme mixed-pr… MATH-500 (EvalScope via llama-ser… 0.798 → 0.706 88.5%
AutoRound AutoScheme mixed-pr… GPQA Diamond (EvalScope via llama… 0.46 → 0.364 79.1%
AutoRound AutoScheme mixed-pr… Average accuracy of 10 lm-eval ta… 0.633 → 0.615 97.3%
AutoRound AutoScheme mixed-pr… MMLU (lm-eval default mmlu task; … 0.73 → 0.694 95.1%
AutoRound AutoScheme mixed-pr… GPQA Diamond (EvalScope via llama… 0.46 → 0.505 109.9%
AutoRound AutoScheme mixed-pr… Average accuracy of 10 lm-eval ta… 0.633 → 0.632 99.9%
AutoRound AutoScheme mixed-pr… MATH-500 (EvalScope via llama-ser… 0.798 → 0.788 98.7%
AutoRound AutoScheme mixed-pr… MMLU (lm-eval default mmlu task; … 0.73 → 0.718 98.4%
AutoRound AutoScheme mixed-pr… MATH-500 (EvalScope via llama-ser… 0.798 → 0.814 102.0%
AutoRound AutoScheme mixed-pr… GPQA Diamond (EvalScope via llama… 0.46 → 0.444 96.7%
AutoRound AutoScheme mixed-pr… Average accuracy of 10 lm-eval ta… 0.633 → 0.625 98.7%
AutoRound AutoScheme mixed-pr… MMLU (lm-eval default mmlu task; … 0.73 → 0.719 98.5%
AutoRound AutoScheme mixed-pr… MATH-500 (EvalScope via llama-ser… 0.798 → 0.788 98.7%
AutoRound AutoScheme mixed-pr… GPQA Diamond (EvalScope via llama… 0.46 → 0.45 97.8%
AutoRound AutoScheme mixed-pr… Average accuracy of 10 lm-eval ta… 0.633 → 0.625 98.8%
AutoRound AutoScheme mixed-pr… MMLU (lm-eval default mmlu task; … 0.73 → 0.718 98.3%
AutoRound AutoScheme mixed-pr… MATH-500 (EvalScope via llama-ser… 0.798 → 0.816 102.3%
AutoRound AutoScheme mixed-pr… GPQA Diamond (EvalScope via llama… 0.46 → 0.45 97.8%
AutoRound AutoScheme mixed-pr… Average accuracy of 10 lm-eval ta… 0.633 → 0.628 99.3%
AutoRound AutoScheme mixed-pr… MMLU (lm-eval default mmlu task; … 0.73 → 0.723 99.0%
AWQ 4-bit (AutoAWQ, zero poin… WMT24++ X->English COMET (wmt22-c… 0.782 → 0.774 99.0%
AWQ 4-bit (AutoAWQ, zero poin… WMT24++ English->X COMET (wmt22-c… 0.702 → 0.69 98.3%
AWQ INT4 (w4, group_size 128,… MMLU-Redux, non-thinking mode (Qw… 0.795 → 0.791 99.5%
AWQ INT4 (w4, group_size 128,… MMLU-Redux, thinking mode (Qwen-r… 0.875 → 0.864 98.7%
AWQ INT4 (w4, group_size 128,… LiveBench 2024-11-25, thinking mo… 0.671 → 0.655 97.6%
AWQ INT4 (w4, group_size 128,… GPQA (subset not stated in card),… 0.62 → 0.59 95.2%
AWQ INT4 (w4, group_size 128,… AIME 2024, thinking mode (Qwen-re… 0.76 → 0.713 93.8%
AWQ INT4 (w4, group_size 128,… LiveBench 2024-11-25, non-thinkin… 0.535 → 0.489 91.4%
AWQ INT4 (w4, group_size 128,… GPQA (subset not stated in card),… 0.393 → 0.359 91.3%
bitsandbytes NF4 (Transformer… WMT24++ X->English COMET (wmt22-c… 0.782 → 0.776 99.3%
bitsandbytes NF4 (Transformer… WMT24++ English->X COMET (wmt22-c… 0.702 → 0.692 98.6%
FP8 block (weights FP8 128x12… OpenLLM v2 average of 6 tasks (IF… 0.289 → 0.291 100.6%
FP8 block (weights FP8 128x12… OpenLLM v1 average of 6 tasks (AR… 0.721 → 0.721 100.0%
FP8 block (weights FP8 128x12… GSM8K — published row 'GSM8K (Str… 0.879 → 0.878 99.8%
FP8 block (weights FP8 128x12… MMLU — published row 'MMLU (Acc, … 0.749 → 0.747 99.8%
FP8 dynamic (weights FP8 per-… GSM8K — published row 'GSM-8K (5-… 0.76 → 0.805 106.0%
FP8 dynamic (weights FP8 per-… GPQA Diamond — published row 'GPQ… 0.586 → 0.611 104.3%
FP8 dynamic (weights FP8 per-… OpenLLM v2 average of 6 tasks (MM… 0.343 → 0.353 103.0%
FP8 dynamic (weights FP8 per-… AIME 2024 — published row 'AIME 2… 0.746 → 0.763 102.4%
FP8 dynamic (weights FP8 per-… OpenLLM v1 average of 6 tasks (MM… 0.642 → 0.648 101.0%
FP8 dynamic (weights FP8 per-… LiveCodeBench — published row 'Li… 0.563 → 0.566 100.6%
FP8 dynamic (weights FP8 per-… MMLU — published row 'MMLU (5-sho… 0.72 → 0.723 100.5%
FP8 dynamic (weights FP8 per-… AIME 2025 — published row 'AIME 2… 0.652 → 0.637 97.8%
GGUF Q2_K (bartowski/Qwen3-8B… Average accuracy of 10 lm-eval ta… 0.633 → 0.607 96.0%
Each kept value links to its source.
69 more in [quant.json](../../data/quant.json)
[interactive version](https://whatmodel.sh/model/qwen3-8b) · [JSON](https://whatmodel.sh/api?format=json&all=1)