[whatmodel.sh](../../README.md) / Qwen3.8 27B
# Qwen3.8 27B
Alibaba · released 2026-08 · 27.3B params · apache-2.0
**score 71.8** ±0.7 · #62 of 732 · from 33 benchmark results across 9 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Qwen3.8 27B among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█▄▅▅●▄▂▁ #52 of 313 30% 8
coding ▁▁▁▃▅▇█▇▅▄●▁ #36 of 469 26% 6
chat & writing ▁▁▁▃▆▄▆█▆▇●▃ #72 of 402* 5.1% 1
vision & documents ▁▁▂▂▁▃▄▂▆▇●▄ #43 of 153 15% 6
agent tasks · #52 of 313 · 30% of its score
Beats [Gemini 3.1 Pro (Preview)](gemini-3-1-pro.md), [Claude Opus 4.6](claude-opus-4-6.md), [Muse Spark 1.1](muse-spark-1-1.md) here, though they rank higher overall. Trails [Claude Sonnet 4.6](claude-sonnet-4-6.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena Agent (October 2026 aggrega… -0.0446 IPS ▁▁▁▃●▂█▁▂▂▁▁ beats 15/59 avg -0.379 IPS 13% sharp
SWE-bench Pro public set, Qwen-re… 61.7% ▂·····▂·▂█▃● beats 10/12 hard · avg 21% 4.4% at its level · several runs
Terminal-Bench 2.1 (maker) 73.0% ▁·▁·▁▁▄▃▆●▅█ beats 41/63 hard · avg 17% 3.1% sharp · at its level · small run
SWE-bench Multilingual (maker) 73.8% ▁··▁▃▁▅█●▂▂▁ beats 44/65 hard · avg 29% 3.1% typical
Thinkingbox-bench pass@1 (507 sta… 51.7% ▅··▃·▃·█▃●██ beats 9/15 very hard · avg 5.1% 2.2% sharp · at its level
OSWorld-Verified (maker) 84.3% █▃·▂▅▂▃▂▅▃▆● beats 25/27 hard · avg 27% 1.9% small run
WebArena-Verified (maker) 64.8% █···█······● beats 2/2 hard · avg 24% 1.1% at its level · small run
DeepSWE v1.1 (resolved) (maker) 42.2% ▂····●▁▁▆███ beats 2/31 hard · avg 16% 0.7% at its level · noisy
coding · #36 of 469 · 26% of its score
Beats [GPT-5.5](gpt-5-5.md), [GPT-5.6 Terra](gpt-5-6-terra.md), [Claude Opus 4.8](claude-opus-4-8.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
Arena WebDev (Code Arena) 1589.8 ▁▁▃▂▅█▃▆●▅▁▂ beats 99/129 avg 981 9.1% sharp
LiveCodeBench v6 (window unspecif… 90.3% ▃▅█▅▅▃▅···▃● beats 14/16 medium · avg 47% 5.0% small run · several runs
SWE-bench Pro public set, Qwen-re… 61.7% ▂·····▂·▂█▃● beats 10/12 hard · avg 21% 4.4% at its level · several runs
Luau Leetcode fully solved (Torpe… 64.9% ▁▁▁▁·▁▂▃●▃█▇ beats 16/49 hard · avg 17% 4.0% sharp · at its level
SWE-bench Multilingual (maker) 73.8% ▁··▁▃▁▅█●▂▂▁ beats 44/65 hard · avg 29% 3.1% typical
DeepSWE v1.1 (resolved) (maker) 42.2% ▂····●▁▁▆███ beats 2/31 hard · avg 16% 0.7% at its level · noisy
chat & writing · #72 of 402 · 5.1% of its score
Beats [Claude Haiku 5.5](claude-haiku-5-5.md), [DeepSeek V4 Flash (0731)](deepseek-v4-flash.md) here, though they rank higher overall. Trails [Gemma 4 26B-A4B](gemma-4-26b-a4b.md), [Nemotron Cascade 2 30B-A3B](nemotron-cascade-2-30b-a3b.md), [Gemma 4 31B](gemma-4-31b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LMArena (text) 1435.6 ▁▁▁▂▃▃▃▆▆▇●▃ beats 295/385 avg 1260 5.1% blunt
vision & documents · #43 of 153 · 15% of its score
Beats [MiMo-V2.6-Pro](mimo-v2-6-pro.md), [MiMo-V2.6-Flash (309B-A15B)](mimo-v2-6-flash.md), [GPT-6 Luna](gpt-6-luna.md) here, though they rank higher overall. Trails [Qwen3.6 Flash](qwen3-6-flash.md), [Gemini 3.5 Flash-Lite](gemini-3-5-flash-lite.md), [Qwen3.6 Plus](qwen3-6-plus.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
CharXiv (reasoning questions) (ma… 83.7% ▄▂▄▆▄▄▆··▄●▂ beats 18/21 medium · avg 38% 7.8% several runs
Arena vision (style controlled) 1242.4 ▁▁▂▂▂▂▃▄▅▅●▇ beats 98/152 avg 1061 3.4% blunt
SciDocBench scientific-document w… 52.9% ▃▃···█●▅▃·▃▅ beats 5/11 hard · avg 25% 1.7% blunt · at its level
OmniDocBench 1.5 overall (higher … 91.1% ▂··▂▂▂··▂▂█● beats 12/13 easy · avg 70% 1.3% small run
RealWorldQA (maker) 85.9% ▁▃▁▇▃▇▇▅█▁▅● beats 38/39 easy · avg 72% 0.5% blunt · small run
PureDocBench document parsing (Av… 77.6% ▂·▁·▂▂▅█·▁●▂ beats 20/22 medium · avg 62% 0.1% blunt · at its level
other tests (9)
test result vs the others in its run difficulty counts why
GPQA Diamond (maker) 89.2% ▁▁▁▁▂▄▅▅██●▄ beats 267/322 hard · avg 40% 12% sharp · several runs
Humanity's Last Exam (maker) 30.8% ▁▁▁▄▆▆▇█●▆▁▃ beats 80/111 very hard · avg 5.2% 6.1% at its level · several runs
IFBench (aggregation unspecified)… 79.5% ▂▆▂▄▄▄▂▄▄▆█● beats 78/86 medium · avg 43% 3.5% several runs
BRIDGE clinical text, zero-shot a… 46.6% ▁▁▁▁▁▂▂▃▄▆█● beats 95/97 hard · avg 31% 3.3% blunt · at its level
NL2Repo (maker) 42.3% ▂▂▂·▃▃▅●▃▂▅▃ beats 11/22 very hard · avg 11% 2.1% at its level · small run
Open Persian LLM Leaderboard v2 a… 76.0% ▂▄▂▃█▄▃▆▃▅▂● beats 79/81 medium · avg 57% 1.5% blunt
RobloxQA v2.0 multiple choice (To… 90.8% ▁▁▃▁●▄▃▃▆▂▂▂ beats 18/49 easy · avg 77% 1.4% typical
ERQA (maker) 65.5% ▃··▃··█·▃█●▅ beats 10/12 medium · avg 36% 1.1% at its level · small run
AndroidWorld (maker) 81.9% ▁·▁▁▂▃█▅▂▂·● beats 24/24 medium · avg 47% 0.9% small run
source results share of the score
LMArena leaderboard dataset 4 31% ▇▇▇▇
Qwen/Qwen3.8-27B model card (submitted … 14 28% ▇▇▇
Qwen (submitted / maker-reported) 8 27% ▇▇▇
Roblox LLM Leaderboard 2 5% ▇
BRIDGE Medical Leaderboard 1 3% ▇
One Success Isn't Reliability: Thinking… 1 2% ▇
SciDocBench: A Workflow-Centered Benchm… 1 2% ▇
Open Persian LLM Leaderboard v2 1 2% ▇
How Far Is Document Parsing from Solved… 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 71.8
Licences: LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Qwen/Qwen3.8-27B model card ([apache-2.0](../../licenses/Qwen--Qwen3.8-27B.txt)) · Roblox LLM Leaderboard ([MIT](../../licenses/roblox-llm-leaderboard-MIT.txt)) · BRIDGE Medical Leaderboard ([MIT](../../licenses/bridge-medical-leaderboard-MIT.txt)) · One Success Isn't Reliability: Thinkingbox ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/))
## How to run it
### Through an API
api (Groq) · $1.60/M blended · $0.8 in / $4 out per 1M tokens · [console.groq.com](https://console.groq.com/docs/models)
[price source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/providers/groq/models/qwen/qwen3.8-27b.toml): Standard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/models/alibaba/qwen3.8-27b.toml)
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Qwen3.8 27B (Q4_K_M, needs ~19 GB, from [huggingface.co/ggml-org](https://huggingface.co/ggml-org/Qwen3.8-27B-GGUF/tree/71bc7b627595dc8a91039addd9c791ae548d6747))
```sh
llama cli -hf ggml-org/Qwen3.8-27B-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf ggml-org/Qwen3.8-27B-GGUF:Q4_K_M
```
Other versions: Q8_0 (~28.6 GB) `llama cli -hf ggml-org/Qwen3.8-27B-GGUF:Q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Qwen3.8 27B (Q4_K_M, needs ~19 GB, from [huggingface.co/ggml-org](https://huggingface.co/ggml-org/Qwen3.8-27B-GGUF/tree/71bc7b627595dc8a91039addd9c791ae548d6747))
```sh
lms get https://huggingface.co/ggml-org/Qwen3.8-27B-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~28.6 GB) `lms get https://huggingface.co/ggml-org/Qwen3.8-27B-GGUF@Q8_0 -y` · 4bit (~16.1 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/Qwen3.8-27B-MLX-4bit -y` · 8bit (~29.5 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/Qwen3.8-27B-MLX-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Qwen3.8 27B (Q4_K_M, needs ~16.88 GB, from [ollama.com](https://ollama.com/library/qwen3.8/tags))
```sh
ollama run qwen3.8:27b
```
Other versions: Q8_0 (~33.26 GB) `ollama run qwen3.8:27b-q8_0`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run Qwen3.8 27B (4bit, needs ~16.1 GB, Mac only, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Qwen3.8-27B-MLX-4bit/tree/6067b15cf581666a4aecf6af3afaba4bb5efc20c))
```sh
mlx_lm.chat --model lmstudio-community/Qwen3.8-27B-MLX-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model lmstudio-community/Qwen3.8-27B-MLX-4bit
```
Other versions: 8bit (~29.5 GB, Mac only) `mlx_lm.chat --model lmstudio-community/Qwen3.8-27B-MLX-8bit`
Model repository and documentation: [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen3.8-27B)
## Published speed measurements
Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine.
hardware precision speed engine and settings
AMD Ryzen AI Max+ 395 (Stri… Q4_K_M 20.42 tok/s Ollama 0.32.13 system service, Ol…
Apple M5 Pro (64GB; CPU/GPU… GGUF Q8_0 3.8 tok/s llama.cpp server (Metal; --parall…
Apple M5 Pro (64GB; CPU/GPU… MLX 8-bit (mlx-… 3.9 tok/s vllm-metal (Metal; version not st…
Apple M5 Pro (64GB; CPU/GPU… MLX 8-bit (mlx-… 4.7 tok/s omlx (MLX; SSD prefix cache, in-m…
AMD Radeon RX 9070 XT UD-IQ4_XS 16.38 tok/s llama.cpp (build not recorded), V…
Apple M2 Pro, 32 GB q4_K_M 8.63 tok/s ollama (version not recorded) · l…
Each speed links to its source, which has the full settings.
## Quantization evidence
How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent. They don't describe every download of that precision.
precision test full → quantized kept
INT4 W4A16 (AWQ smoothing + G… GSM8K-Platinum strict-match (lm-e… 0.958 → 0.968 101.1%
INT4 W4A16 (AWQ smoothing + G… IFEval prompt-level strict 0-shot… 0.922 → 0.919 99.7%
INT4 W4A16 (AWQ smoothing + G… MATH-500 pass@1 (lighteval, vLLM;… 0.837 → 0.833 99.5%
INT4 W4A16 (AWQ smoothing + G… MMLU-Pro exact-match (lm-evaluati… 0.845 → 0.835 98.8%
INT4 W4A16 (AWQ smoothing + G… AIME 2025 pass@1 (lighteval, vLLM… 0.954 → 0.942 98.7%
INT4 W4A16 (AWQ smoothing + G… GPQA Diamond pass@1 (lighteval, v… 0.892 → 0.879 98.5%
Mixed NVFP4/FP8 (MLP projecti… IFEval 0-shot (lm-evaluation-harn… 0.912 → 0.92 100.9%
Mixed NVFP4/FP8 (MLP projecti… MATH-500 0-shot (lighteval, vLLM;… 0.837 → 0.843 100.7%
Mixed NVFP4/FP8 (MLP projecti… GSM8K-Platinum 0-shot (lm-evaluat… 0.963 → 0.967 100.5%
Mixed NVFP4/FP8 (MLP projecti… GPQA Diamond 0-shot (lighteval, v… 0.896 → 0.892 99.6%
Mixed NVFP4/FP8 (MLP projecti… AIME 2025 0-shot (lighteval, vLLM… 0.967 → 0.95 98.3%
MXFP4 W4A4 (FP4 weights and a… MATH-500 pass@1 (lighteval, vLLM;… 0.837 → 0.847 101.2%
MXFP4 W4A4 (FP4 weights and a… IFEval prompt-level strict 0-shot… 0.912 → 0.909 99.7%
MXFP4 W4A4 (FP4 weights and a… GSM8K-Platinum strict-match 0-sho… 0.963 → 0.952 98.9%
MXFP4 W4A4 (FP4 weights and a… GPQA Diamond pass@1 (lighteval, v… 0.896 → 0.877 97.9%
MXFP4 W4A4 (FP4 weights and a… AIME 2025 pass@1 (lighteval, vLLM… 0.967 → 0.925 95.7%
Each kept value links to its source.
[interactive version](https://whatmodel.sh/model/qwen3-8-27b) · [JSON](https://whatmodel.sh/api?format=json&all=1)