[whatmodel.sh](../../README.md) / Qwen3.5 27B
# Qwen3.5 27B
Alibaba · released 2026-02 · 27.8B params · apache-2.0
**score 67.6** ±0.4 · #109 of 732 · from 141 benchmark results across 15 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Qwen3.5 27B among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█▄▅●▄▄▂▁ #90 of 313 25% 18
coding ▁▁▁▃▅▇█▇●▄▃▁ #108 of 469 20% 7
chat & writing ▁▁▁▃▆▄▆█▆●▇▃ #106 of 402* 2.1% 2
math ▁▁▃▅█▆▄●▄▂▁▁ #101 of 431 17% 12
vision & documents ▁▁▂▂▁▃▄▂▆●█▄ #63 of 153 10% 21
web search ▁▁▁●▄█▃▄·▁·▁ #68 of 84 3.3% 5
agent tasks · #90 of 313 · 25% of its score
Beats [Grok 4.20 (0309 reasoning)](grok-4-20.md), [GPT-5.1](gpt-5-1.md), [o3-pro](o3-pro.md) here, though they rank higher overall. Trails [MiniMax-M2.7](minimax-m2-7.md), [Step 3.5 Flash (196B-A11B)](step-3-5-flash.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 68.6% ▄▄·····▄▄▄▄● beats 6/7 hard · avg 16% 7.5% sharp · at its level · several runs
SWE-bench Multilingual 57.7% ▄·▄·····█▄●▄ beats 6/7 hard · avg 18% 2.6% at its level · several runs
Terminal-Bench 2.1 41.6% ▄······█▄▄█● beats 7/7 very hard · avg 7.2% 1.9% sharp · at its level
SWE-bench Pro (split unspecified) 42.1% ▄···▄···▄▄█● beats 6/7 very hard · avg 6.2% 1.7% sharp · at its level
τ²-bench aggregate (maker) 79.0% ▁▁▁·▁▁▃▃▃●█▁ beats 20/39 medium · avg 41% 1.4% several runs
ContractBench success rate (33 ob… 64.6% ▆··▆▄▂▄▂▆▆●▄ beats 18/23 very hard · avg 13% 1.4% sharp · at its level
SWE-bench Pro public set, Qwen-re… 51.2% ▄······▄●·▄▄ beats 3/5 hard · avg 19% 1.2% at its level · several runs
Terminal-Bench 2.0 (maker) 41.6% ▂▂▅▅█●██▆▂▃▃ beats 15/36 very hard · avg 11% 1.0% at its level · small run
WideSearch (maker) 61.1% █·▂●··▂···▂▂ beats 5/10 hard · avg 33% 0.9% at its level · small run · several runs
OSWorld-Verified (maker) 56.2% █▃·▂●▂▃▂▅▃▆▆ beats 8/27 hard · avg 20% 0.9% at its level · small run
BrowseComp-ZH (maker) 62.1% ▁▁▁▁▁▃·▁▄▅●█ beats 16/23 hard · avg 25% 0.8% at its level · small run
MCP-Atlas (public set) (maker) 68.4% ▂·▁▂▂·▁▃█●▆▂ beats 23/39 hard · avg 30% 0.6% at its level · small run
BrowseComp (maker) 61.0% ▂·▃▁▂▂▆▅●▂▂▄ beats 27/46 hard · avg 27% 0.6% at its level · small run
Humanity's Last Exam with tools (… 48.5% ▁·▁▂▂▁▄▄●█▄▅ beats 34/62 hard · avg 18% 0.6% at its level · small run
Toolathlon (Tool Decathlon) (make… 31.5% ▂▂▄▂●▄▆▂█▆██ beats 5/27 very hard · avg 11% 0.6% at its level · small run
MCPMark (maker) 36.3% ▃·····█··▃●▃ beats 6/8 very hard · avg 11% 0.6% at its level · small run
BFCL V4 (December 2025 archive) (… 68.5% ▁▁▁▂▆▆▅▆█▂●▃ beats 105/114 medium · avg 41% 0.4% at its level · small run
τ³-bench (maker) 68.4% ▁▁▂▁·▁▁·▁▁▂● beats 16/25 medium · avg 40% 0.4% at its level · small run
coding · #108 of 469 · 20% of its score
Beats [Grok 4.3](grok-4-3.md), [Muse Glimmer 30B](muse-glimmer-30b.md), [Qwen3.5 Plus](qwen3-5-plus.md) here, though they rank higher overall. Trails [Llama 3.1 Nemotron Ultra 253B v1](llama-3-1-nemotron-ultra-253b.md), [Nemotron Cascade 2 30B-A3B](nemotron-cascade-2-30b-a3b.md), [MiniMax-M2.1](minimax-m2-1.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 68.6% ▄▄·····▄▄▄▄● beats 6/7 hard · avg 16% 7.5% sharp · at its level · several runs
Arena WebDev (Code Arena) 1358.3 ▁▁▃▂●█▃▆▅▅▁▂ beats 31/129 avg 981 3.0% sharp
LiveCodeBench v6 (Feb–May 2025) (… 80.7% ▁▁▃▁█▅▄▇▅▅●█ beats 32/38 medium · avg 48% 2.9% several runs
SWE-bench Multilingual 57.7% ▄·▄·····█▄●▄ beats 6/7 hard · avg 18% 2.6% at its level · several runs
SWE-bench Pro (split unspecified) 42.1% ▄···▄···▄▄█● beats 6/7 very hard · avg 6.2% 1.7% sharp · at its level
SWE-bench Pro public set, Qwen-re… 51.2% ▄······▄●·▄▄ beats 3/5 hard · avg 19% 1.2% at its level · several runs
SciCode (level unspecified) 25.6% ▄···●▄█▄·▄·▄ beats 1/7 very hard · avg 13% 0.8% at its level
chat & writing · #106 of 402 · 2.1% of its score
Beats [LongCat 2.0 (1.6T-A48B)](longcat-2-0.md), [Inkling Small (266B-A12B)](inkling-small.md), [GPT-5.4 nano](gpt-5-4-nano.md) here, though they rank higher overall. Trails [Mistral Large 3 (675B-A41B)](mistral-large-3.md), [Gemma 4 12B](gemma-4-12b.md), [GPT-4.1](gpt-4-1.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LMArena (text) 1408.8 ▁▁▁▂▃▃▃▆▆●█▃ beats 254/385 avg 1260 1.7% blunt
IFEval (aggregation unspecified) … 95.0% ▁▂▄▂▂▄▆██▅●▁ beats 141/145 easy · avg 81% 0.4% maxed out · small run
math · #101 of 431 · 17% of its score
Beats [Claude Sonnet 4.6](claude-sonnet-4-6.md), [MiniMax-M3](minimax-m3.md), [Nemotron 3 Ultra 550B-A55B](nemotron-3-ultra-550b-a55b.md) here, though they rank higher overall. Trails [Gemini 2.5 Flash](gemini-2-5-flash.md), [GLM-4.5-Air](glm-4-5-air.md), [Granite 4.2 30B](granite-4-2-30b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
HMMT February 2026 (maker) 84.3% ▃▃▂▅·███●▅▃▅ beats 27/35 medium · avg 36% 3.3% sharp · several runs
AIME 2026 (maker) 92.6% ▁▁▁▂▂▂▆●▆▂·▁ beats 34/52 medium · avg 59% 2.8% sharp · maxed out · small run · several…
IMO-AnswerBench (maker) 79.9% ▁▁·▁▁▁▄▆●▄▂▃ beats 24/44 medium · avg 36% 2.2% small run · several runs
MathArena ArXivMath February 2026… 31.3% ▁▁▁▁●▇▄▃▁▃·▃ beats 6/24 very hard · avg 9.4% 1.9% at its level
MathArena ArXivMath January 2026 … 53.3% ▂··▄▂▄●▆▆▄▄█ beats 7/23 hard · avg 23% 1.4% at its level
MATH-Vision (maker) 86.0% ▁▁·▁··▄▁▃▇▇● beats 21/24 medium · avg 55% 1.2% small run · several runs
MathArena ArXivMath December 2025… 41.2% ▃·▃▃●▁·▄·▁▁▁ beats 8/18 hard · avg 19% 1.0% at its level
HMMT February 2025 (maker) 92.0% ▁▁▁▃▁▅▅▃●█▁▄ beats 21/31 medium · avg 63% 0.9% small run
MathVista (testmini) (maker) 87.8% ▁··▁▄▄▆▇▄▅█● beats 47/48 medium · avg 64% 0.6% small run
HMMT November 2025 (maker) 89.8% ▁▁····▁▅▃●▇▁ beats 13/23 easy · avg 69% 0.6% small run
PolyMATH (maker) 71.2% ▁···▃▁·▇█▄▄● beats 22/23 medium · avg 35% 0.5% at its level · small run
DynaMath (maker) 87.7% ▂·····▂··█▄● beats 10/10 easy · avg 66% 0.5% small run
vision & documents · #63 of 153 · 10% of its score
Beats [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md), [GPT-5](gpt-5.md), [MiniMax-M3](minimax-m3.md) here, though they rank higher overall. Trails [Gemma 4 26B-A4B](gemma-4-26b-a4b.md), [Qwen3.6 Flash](qwen3-6-flash.md), [Qwen3.5 35B-A3B](qwen3-5-35b-a3b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
MATH-Vision (maker) 86.0% ▁▁·▁··▄▁▃▇▇● beats 21/24 medium · avg 55% 1.2% small run · several runs
Arena vision (style controlled) 1217.2 ▁▁▂▂▂▂▃▄▅●█▇ beats 84/152 avg 1061 1.1% blunt
OmniDocBench 1.5 overall (higher … 88.9% ▂··▂▂▂··▂▂●▅ beats 6/13 easy · avg 71% 0.9% small run · several runs
CharXiv (reasoning questions) (ma… 79.5% ▂▅▃▅▅▆▆▅▆●▆▂ beats 76/101 medium · avg 43% 0.7% small run
MathVista (testmini) (maker) 87.8% ▁··▁▄▄▆▇▄▅█● beats 47/48 medium · avg 64% 0.6% small run
MMMU (val) (maker) 82.3% ▁····▁▁▃█▆▆● beats 50/53 medium · avg 58% 0.6% small run
MMMU Pro (maker) 75.0% ▂▃▁▃▆▇▃▇▇●█▅ beats 38/52 medium · avg 47% 0.6% at its level · small run
BEAR-Bench (bilingual RU/EN enter… 60.8% ▃▃·▅·▃·█·●▅▅ beats 8/14 hard · avg 24% 0.6% at its level
MMStar (maker) 81.0% ▃▅▂▃▂▂▆██▆●▅ beats 31/35 medium · avg 63% 0.4% small run
HallusionBench (maker) 70.0% ▂▁·▂▂▂▃▅▃█▅● beats 32/33 medium · avg 54% 0.4% blunt · at its level · small run · seve…
CountBench (maker) 97.8% ▂▂▃▂·▂▂▃▃·▅● beats 18/18 very easy · avg 92% 0.4% maxed out · small run
PMC-VQA (maker) 62.4% ▃···▃··▃▃·▃● beats 5/7 medium · avg 39% 0.4% at its level · small run
SimpleVQA (maker) 56.0% ▃·▃·▃·▃·●▅▃▃ beats 4/10 hard · avg 35% 0.3% blunt · at its level · small run
MMBench-EN dev v1.1 (maker) 92.6% █▄···▂··▄▂●▆ beats 11/14 easy · avg 82% 0.3% blunt · maxed out · small run
ScreenSpot-Pro (maker) 70.3% ▂▃▅▆█●▅▂···▂ beats 18/24 medium · avg 48% 0.3% blunt · at its level · small run
AI2D test (maker) 92.9% ▁▁▅▇▃▇·▄·●·▃ beats 23/28 easy · avg 86% 0.2% blunt · small run · several runs
SpaceCast-Bench predictive spatia… 39.3% ▃·▅▃▅●··▅··█ beats 6/11 very hard · avg 33% 0.2% noisy
RealWorldQA (maker) 83.7% ▁▃▁▇▃▇▇▅█▁●▅ beats 32/39 easy · avg 72% 0.2% blunt · small run
CC-OCR (maker) 81.0% ▁·▂··▂▂▃▃▂▆● beats 20/26 easy · avg 71% 0.1% blunt · small run
OCRBench (maker) 89.4% ▁·▃▁▆▅▂█▅●▁▃ beats 29/37 easy · avg 83% 0.1% blunt · small run
PureDocBench document parsing (Av… 69.6% ▂·▁·▂▂▅●·▁▂▂ beats 13/22 medium · avg 62% 0.0% blunt · at its level
web search · #68 of 84 · 3.3% of its score
Beats [DeepSeek V4 Flash (Preview, Apr 2026)](deepseek-v4-flash-preview.md), [Nemotron 3 Ultra 550B-A55B](nemotron-3-ultra-550b-a55b.md), [GPT-5.4 nano](gpt-5-4-nano.md) here, though they rank higher overall. Trails [GPT-4.1 nano](gpt-4-1-nano-2025-04-14.md), [GPT-4o mini (2024-07-18)](gpt-4o-mini-2024-07-18.md), [Mistral Small 3.1 24B (2503)](mistral-small-3-1.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
WideSearch (maker) 61.1% █·▂●··▂···▂▂ beats 5/10 hard · avg 33% 0.9% at its level · small run · several runs
BrowseComp-ZH (maker) 62.1% ▁▁▁▁▁▃·▁▄▅●█ beats 16/23 hard · avg 25% 0.8% at its level · small run
BrowseComp (maker) 61.0% ▂·▃▁▂▂▆▅●▂▂▄ beats 27/46 hard · avg 27% 0.6% at its level · small run
Humanity's Last Exam with tools (… 48.5% ▁·▁▂▂▁▄▄●█▄▅ beats 34/62 hard · avg 18% 0.6% at its level · small run
SealQA Seal-0 (maker) 47.2% █····█·····● beats 2/2 hard · avg 20% 0.4% at its level · small run
other tests (48)
test result vs the others in its run difficulty counts why
GPQA Diamond (maker) 85.5% ▁▁▁▁▂▄▅▅█●█▄ beats 236/322 hard · avg 44% 6.7% sharp · several runs
Humanity's Last Exam (maker) 24.3% ▁▁▁▄▆▆▇●█▆▁▃ beats 61/111 very hard · avg 5.1% 4.2% several runs
MMLU-Pro (maker) 86.1% ▁▁·▁▁▁▃▂▄▅●▁ beats 178/188 medium · avg 56% 3.7% several runs
NL2Repo (maker) 27.3% ▄·····●·▄▄·▄ beats 1/5 very hard · avg 8.9% 1.4% at its level · several runs
KINA (Knowledge Index of Noah's A… 39.4% ▁▁▂▁▃▄█▆▄●▂▃ beats 32/39 very hard · avg 21% 1.2% at its level
VitaBench (maker) 41.9% ▄··▂▄▆▆█▄▄●▂ beats 20/21 very hard · avg 12% 1.2% at its level · small run · several runs
MedXPertQA MM (maker) 62.4% ▂···▂·▃··▆▂● beats 11/13 very hard · avg 31% 1.2% sharp · at its level · small run
QwenClawBench (Qwen internal) (ma… 52.2% ▄·▄····▄···● beats 3/4 hard · avg 23% 1.1% at its level · several runs
LongBench v2 (maker) 60.6% ▂·▁·▂▃▁▄▄▃●▄ beats 26/33 very hard · avg 35% 0.9% at its level · small run
KOPA-Bench (pass@1) 44.8% ▄·█··▄··█▄·● beats 7/8 very hard · avg 14% 0.9% at its level
LVBench (maker) 73.6% ▇·█▅▅▄●··▃▁▁ beats 22/28 medium · avg 53% 0.8% at its level · small run · several runs
Claw-Eval Pass^3 (maker) 46.2% ●▄▄·▄·▄▄·▄·▄ beats 0/8 hard · avg 25% 0.8% at its level · several runs
DeepPlanning (maker) 22.6% ▂·····▂··▄●▄ beats 4/9 very hard · avg 9.2% 0.8% small run · several runs
C-Eval (maker) 90.5% ▁···▁▁▂▁▂▂▃● beats 25/33 medium · avg 71% 0.7% small run
ZEROBench subquestions (maker) 36.2% ▄·▄···▄·▄█●▄ beats 6/8 hard · avg 16% 0.7% at its level · small run · several runs
INCLUDE (maker) 81.6% ▂····▂▂▄▄▄█● beats 32/36 medium · avg 55% 0.7% small run
MMLU-Redux (maker) 93.2% ▁▁▃▁▂▃▄▅▇▅●▄ beats 53/63 easy · avg 78% 0.7% small run
MMMLU (maker) 85.9% ▂▁▁▂▃▂▄▄▅█●▃ beats 37/46 medium · avg 63% 0.7% small run
SkillsBench 78-task subset (OpenC… 27.2% ▄·····▄···▄● beats 3/4 very hard · avg 11% 0.7% at its level · several runs
CPTUB average (SpeakLeash, 0-5 ju… 86.7% ▁▂▂▂▃█▃▆▅▄▂● beats 92/95 medium · avg 65% 0.6% typical
OJBench (maker) 40.1% ▁···▁▃·▅▄▅█● beats 24/26 very hard · avg 13% 0.6% at its level · small run
AndroidWorld (maker) 64.2% ▁·▁▁▂▃█●▂▂·▂ beats 18/24 medium · avg 42% 0.6% at its level · small run · several runs
Video-MME (without subtitles) (ma… 82.8% ▁▃▁▃▁·█▄▄▅·● beats 25/27 medium · avg 68% 0.6% blunt · small run · several runs
Claw-Eval average score (maker) 64.3% ▄·▄··█●▄▄▄▄▄ beats 4/9 medium · avg 48% 0.6% blunt · at its level · several runs
IFBench (aggregation unspecified)… 76.5% ▂▆▂▄▄▄▂▄▄▆●▇ beats 73/86 medium · avg 43% 0.5% at its level · small run
Global PIQA (maker) 87.5% ▃▃··▃··▃▃▅●▅ beats 9/11 medium · avg 73% 0.5% small run
MultiChallenge (maker) 60.8% ▁·▁·▁▃▆▅▃▂●▅ beats 21/30 hard · avg 31% 0.5% at its level · small run
SuperGPQA (maker) 65.6% ▂▁▂▂▅▆▃▇▇▇●▃ beats 42/46 medium · avg 37% 0.5% at its level · small run
MAXIFE (maker) 88.0% ▂·▂·▂···▆·▄● beats 10/11 easy · avg 67% 0.5% small run
MMLU-ProX (maker) 82.2% ▁·▁▁▁▂▁▃▃█●▄ beats 28/36 medium · avg 58% 0.5% small run
VideoMMMU (maker) 82.3% ▂···▂···▂▂●▆ beats 7/10 medium · avg 60% 0.5% small run
VLMs Are Blind (maker) 96.9% ▂·▂·····▃··● beats 6/8 very easy · avg 90% 0.4% maxed out · small run
BFCL multi-turn (version not stat… 65.4% ▄▄···▄·▄█▄▄● beats 8/8 hard · avg 26% 0.4% at its level
LingoQA (maker) 82.0% ▄······▄·█·● beats 5/5 medium · avg 58% 0.4% small run
SLAKE (maker) 80.0% ▄·····▄▄·█●▄ beats 6/7 medium · avg 60% 0.4% small run
Video-MME (with subtitles) (maker) 87.0% ▁▁▁▁▁·▁▃▁▁▁● beats 13/16 medium · avg 71% 0.4% small run
MMVU (maker) 73.3% ▃···▃···▃▃▅● beats 6/8 medium · avg 49% 0.4% at its level · small run
FullStackBench (Chinese) (maker) 57.4% █······●···█ beats 1/2 hard · avg 30% 0.4% at its level · small run
FullStackBench (English) (maker) 60.1% █····●·····█ beats 1/2 hard · avg 33% 0.4% at its level · small run
RefSpatial-Bench (maker) 67.7% ▃·▃····▃▃▅●█ beats 6/10 medium · avg 41% 0.4% at its level · small run
MLVU (maker) 85.9% ▂···▂···▂▄●█ beats 6/10 easy · avg 69% 0.4% small run
MMLongBench-Doc (maker) 60.2% ▃···▃···▃▅●▅ beats 7/9 medium · avg 36% 0.4% at its level · small run
ERQA (maker) 60.5% ▃··▃··█·●█▅▅ beats 5/12 medium · avg 37% 0.4% at its level · small run
EmbSpatial-Bench (maker) 84.5% ▁·····▁·▁·▃● beats 9/10 easy · avg 67% 0.3% small run
NOVA-63 (maker) 58.1% ▃▃·▃···▃█▃▃● beats 9/11 medium · avg 36% 0.3% blunt · at its level · small run
MVBench (maker) 74.6% ▁··▃▄▁▃▄▄▄●▄ beats 20/26 medium · avg 65% 0.1% blunt · at its level · small run
WMT24++ (maker) 77.6% ▁·▁▁·▁·▁▂●█▃ beats 13/27 easy · avg 68% 0.1% blunt · small run
Vectara hallucination leaderboard… 87.9% ▁▂▁▂●█▂▃▄▃▁▁ beats 23/98 very easy · avg 90% 0.0% blunt
source results share of the score
Qwen/Qwen3.5-122B-A10B model card (subm… 71 37% ▇▇▇▇
Qwen/Qwen3.6-35B-A3B model card (submit… 27 16% ▇▇
Qwen/Qwen3.6-27B model card (submitted … 16 16% ▇▇
Qwen/Qwen3.5-27B model card (submitted … 7 8% ▇
Kwaipilot/KAT-Coder-V2.5-Dev model card 5 7% ▇
LMArena leaderboard dataset 3 6% ▇
MathArena ArXivMath / BrokenArXiv / ArX… 3 4% ▇
ContractBench: Can LLM Agents Preserve … 1 1% ▇
Multi-Step Tool-Calling over Korean Ope… 2 1% ▇
Knowledge Index of Noah's Ark 1 1% ▇
SpeakLeash CPTUB 1 1% ▇
BEAR-Bench: A Bilingual Enterprise and … 1 1% ▇
SpaceCast-Bench: Evaluating Predictive … 1 0% ▇
How Far Is Document Parsing from Solved… 1 0% ▇
Vectara hallucination leaderboard 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 67.6
Licences: Qwen/Qwen3.5-122B-A10B model card ([apache-2.0](../../licenses/Qwen--Qwen3.5-122B-A10B.txt)) · Qwen/Qwen3.6-35B-A3B model card ([apache-2.0](../../licenses/Qwen--Qwen3.6-35B-A3B.txt)) · Qwen/Qwen3.6-27B model card ([apache-2.0](../../licenses/Qwen--Qwen3.6-27B.txt)) · Qwen/Qwen3.5-27B model card ([apache-2.0](../../licenses/Qwen--Qwen3.5-27B.txt)) · Kwaipilot/KAT-Coder-V2.5-Dev model card ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets ([CC BY-SA 4.0](../../licenses/matharena-sharealike-NOTICE.txt)) · ContractBench: Can LLM Agents Preserve Observation Contracts? ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Multi-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Knowledge Index of Noah's Ark ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · SpeakLeash CPTUB ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · SpaceCast-Bench: Evaluating Predictive Spatial Reasoning in Vision-Language Models ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
### Through an API
api (Deep Infra) · $0.84/M blended · $0.26 in / $2.6 out per 1M tokens · [deepinfra.com](https://deepinfra.com/models)
[price source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/providers/deepinfra/models/Qwen/Qwen3.5-27B.toml): Standard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/models/alibaba/qwen3.5-27b.toml)
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Qwen3.5 27B (Q4_K_M, needs ~16.5 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Qwen3.5-27B-GGUF/tree/3b2745e699b00222112360a39f8854bf62774c1a))
```sh
llama cli -hf lmstudio-community/Qwen3.5-27B-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf lmstudio-community/Qwen3.5-27B-GGUF:Q4_K_M
```
Other versions: Q8_0 (~28.6 GB) `llama cli -hf lmstudio-community/Qwen3.5-27B-GGUF:Q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Qwen3.5 27B (Q4_K_M, needs ~16.5 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Qwen3.5-27B-GGUF/tree/3b2745e699b00222112360a39f8854bf62774c1a))
```sh
lms get https://huggingface.co/lmstudio-community/Qwen3.5-27B-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~28.6 GB) `lms get https://huggingface.co/lmstudio-community/Qwen3.5-27B-GGUF@Q8_0 -y` · 4bit (~16.1 GB, Mac only) `lms get https://huggingface.co/mlx-community/Qwen3.5-27B-4bit -y` · 8bit (~29.5 GB, Mac only) `lms get https://huggingface.co/mlx-community/Qwen3.5-27B-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Qwen3.5 27B (Q4_K_M, needs ~17.18 GB, from [ollama.com](https://ollama.com/library/qwen3.5/tags))
```sh
ollama run qwen3.5:27b
```
Other versions: Q8_0 (~33.86 GB) `ollama run qwen3.5:27b-q8_0` · nvfp4 (~17.18 GB, Mac only) `ollama run qwen3.5:27b-mlx`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run Qwen3.5 27B (4bit, needs ~16.1 GB, Mac only, from [huggingface.co/mlx-community](https://huggingface.co/mlx-community/Qwen3.5-27B-4bit/tree/45797d2985a12c55e6473686e9ea91b95e959553))
```sh
mlx_lm.chat --model mlx-community/Qwen3.5-27B-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model mlx-community/Qwen3.5-27B-4bit
```
Other versions: 8bit (~29.5 GB, Mac only) `mlx_lm.chat --model mlx-community/Qwen3.5-27B-8bit`
Model repository and documentation: [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen3.5-27B)
## Published speed measurements
Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine.
hardware precision speed engine and settings
Apple M5 Max (40-core GPU),… mlx-nvfp4 29.29 tok/s ollama (version not recorded) · l…
Each speed links to its source, which has the full settings.
[interactive version](https://whatmodel.sh/model/qwen3-5-27b) · [JSON](https://whatmodel.sh/api?format=json&all=1)