[whatmodel.sh](../../README.md) / Qwen3 235B-A22B Instruct 2507
# Qwen3 235B-A22B Instruct 2507
Alibaba · released 2025-07 · 235B params (22B active) · apache-2.0
**score 64.3** ±0.5 · #152 of 732 · from 81 benchmark results across 9 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Qwen3 235B-A22B Instruct 2507 among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█●▅▅▄▄▂▁ #171 of 313 25% 10
coding ▁▁▁▃▅▇█●▅▄▃▁ #168 of 469 17% 7
chat & writing ▁▁▁▃▆▄▆█▆●▇▃ #92 of 402 8.2% 3
math ▁▁▃▅█▆▄●▄▂▁▁ #80 of 431 11% 5
agent tasks · #171 of 313 · 25% of its score
Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md), [DeepSeek R1 0528](deepseek-r1-0528.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Jamba 1.5 Large](jamba-1-5-large.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
τ²-bench Telecom 22.5% █●··███···██ beats 1/6 very hard · avg 8.3% 6.9% sharp · at its level · small run · seve…
τ²-bench Retail 70.5% ▅···●·▃····▃ beats 2/6 medium · avg 40% 4.1% at its level · several runs
τ²-bench Airline 36.0% ●·▄··▄···█▄▄ beats 0/6 hard · avg 22% 3.2% at its level · several runs
τ-bench Retail (maker) 71.3% ▁····▃·▂▇█●▆ beats 22/28 hard · avg 32% 3.1% at its level · several runs
SWE-bench Verified 42.0% ▁▁▁▁▁▁▂●█▆▁▁ beats 30/138 hard · avg 17% 2.5% sharp · at its level
τ-bench Airline (maker) 44.0% ▁▁▁·▅▄▁●▇▁▁▇ beats 13/28 hard · avg 19% 2.0% at its level · several runs
BFCL V4 (December 2025 archive) 48.0% ▁▁▁▂██▅●▆▂▂▂ beats 54/77 hard · avg 31% 1.4% at its level
BFCL v3 (maker) 70.9% ▁▁▁▁▁▃▅█▆▅●▁ beats 155/171 medium · avg 56% 0.8% blunt · at its level · several runs
Galileo Agent Leaderboard v2 (act… 53.0% ▅▃·▃▃▅▅█▃●▅▅ beats 14/19 hard · avg 26% 0.6% at its level
Terminal-Bench (pre-2.0 release) 17.3% ▄·●···▄▄···█ beats 1/5 very hard · avg 12% 0.6% at its level
coding · #168 of 469 · 17% of its score
Beats [Qwen3-Coder-Next (80B-A3B)](qwen3-coder-next.md), [Gemini 2.5 Pro](gemini-2-5-pro.md), [Grok 4.1 Fast (Reasoning)](grok-4-1-fast-reasoning.md) here, though they rank higher overall. Trails [QwQ-32B-Preview](qwq-32b-preview.md), [Ministral 3 8B Reasoning 2512](ministral-3-8b-reasoning.md), [Phi-4-reasoning 14B](phi-4-reasoning.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Aider polyglot (225 exercises, tw… 57.3% ▂▁▁▃▆▃▄▆●▃▄▂ beats 49/71 very hard · avg 8.2% 5.4% sharp · at its level
LiveCodeBench v6 (Feb–May 2025) (… 51.8% ▁▁▃▁●▅▄▇▅▅▃█ beats 5/38 hard · avg 27% 2.8% at its level · several runs
SWE-bench Verified 42.0% ▁▁▁▁▁▁▂●█▆▁▁ beats 30/138 hard · avg 17% 2.5% sharp · at its level
LiveCodeBench (version unspecifie… 46.5% █·······▄●▄▄ beats 3/5 hard · avg 21% 2.0% at its level
MultiPL-E (maker) 87.9% ▃··▄·▅▅▇▄█▃● beats 31/31 easy · avg 68% 1.6% several runs
MBPP+ 79.9% ▃·····▃●▃··▃ beats 3/6 medium · avg 60% 1.5% typical
HumanEval+ 94.5% ▄▄▄·····▄▄·● beats 6/6 easy · avg 66% 1.5% sharp
chat & writing · #92 of 402 · 8.2% of its score
Beats [LongCat 2.0 (1.6T-A48B)](longcat-2-0.md), [Inkling Small (266B-A12B)](inkling-small.md), [Qwen3.5 122B-A10B](qwen3-5-122b-a10b.md) here, though they rank higher overall. Trails [Gemma 4 12B](gemma-4-12b.md), [Gemma 4 26B-A4B](gemma-4-26b-a4b.md), [Mistral Medium 3.5 (128B)](mistral-medium-3-5.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LMArena (text) 1422.2 ▁▁▁▂▃▃▃▆▆●█▃ beats 274/385 avg 1260 3.0% blunt
Arena-Hard v2 aggregate (maker) 79.2% ▁▁··▄▃▁▃█▁▁● beats 18/22 medium · avg 37% 2.7% sharp · small run · several runs
IFEval (aggregation unspecified) 88.5% █··█·█··█●██ beats 4/6 easy · avg 74% 2.5% several runs
math · #80 of 431 · 11% of its score
Beats [Claude Sonnet 4.6](claude-sonnet-4-6.md), [Claude Opus 4.5](claude-opus-4-5.md), [Qwen3.5 397B-A17B](qwen3-5-397b-a17b.md) here, though they rank higher overall. Trails [Granite 4.2 30B](granite-4-2-30b.md), [Qwen3-Next 80B-A3B Instruct](qwen3-next-80b-a3b-instruct.md), [INTELLECT-3](intellect-3.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
AIME 2025 68.3% ▄▄···▄▄··▄·● beats 6/6 hard · avg 32% 5.5% sharp · small run · several runs
HMMT 2025 (edition unspecified) (… 55.4% ▃█●▅▇▅▁·▁··▁ beats 12/32 hard · avg 19% 3.5% sharp · at its level · small run · seve…
PolyMATH (maker) 50.2% ▁···▃▁·▇●▄▄▄ beats 12/23 hard · avg 24% 0.9% at its level
AIME 2024 81.7% ▁▁▁▂▁▃▁▅█●█▃ beats 57/85 hard · avg 30% 0.8% sharp · at its level · noisy
MATH-500 98.8% ▄·▄··█··▄·▄● beats 6/6 very easy · avg 87% 0.6% maxed out · noisy
other tests (23)
test result vs the others in its run difficulty counts why
MMLU-Pro 84.8% █▄·▄···▄··▄● beats 6/6 medium · avg 58% 6.2% several runs
MMLU 90.2% ▄·····█▄●▄·▄ beats 4/6 medium · avg 68% 3.4% typical
GPQA Diamond 77.4% █·█·█··█●··█ beats 4/5 hard · avg 40% 3.2% sharp · at its level
GPQA (subset unspecified) (maker) 77.5% ▂▁▂▂▄▃▆▃▅█●▃ beats 93/106 hard · avg 44% 2.6% sharp · at its level
INCLUDE (maker) 79.5% ▂····▂▂▄▄▄●▄ beats 26/36 medium · avg 57% 2.4% at its level · several runs
ARC-AGI-1 public evaluation set (… 17.0% ▁·▇●▁·█▇▁·▇▃ beats 7/27 very hard · avg 4.0% 2.2% sharp · at its level · noisy
MMLU-Redux (maker) 93.1% ▁▁▃▁▂▃▄▅▇▅●▄ beats 52/63 easy · avg 78% 2.2% several runs
Creative Writing v3 (maker) 87.5% ▂····▂▆··▆█● beats 13/14 easy · avg 69% 1.7% several runs
SuperGPQA (maker) 62.6% ▂▁▂▂▅▆▃▇▇▇●▃ beats 37/46 medium · avg 36% 1.6% at its level · several runs
MMLU-ProX (maker) 79.4% ▁·▁▁▁▂▁▃▃●▅▄ beats 23/36 medium · avg 57% 1.5% several runs
SimpleQA (maker) 54.3% ▄▄▄▆█▆███·▂● beats 31/32 hard · avg 18% 1.5% at its level · small run · several runs
CMMLU 88.1% █····▄··▄▄●▄ beats 5/6 medium · avg 67% 1.3% typical
WritingBench (maker) 85.2% ▁·▁▂▁··▅●▂▂▁ beats 12/20 easy · avg 70% 1.3% several runs
ChineseSimpleQA (maker) 84.3% ▄··▄▄▄··▄··● beats 6/6 medium · avg 59% 1.3% small run · several runs
LiveBench 2024-11-25 (maker) 75.4% ▄··▂·▂·▂●▆▂▂ beats 7/13 medium · avg 55% 1.3% several runs
C-Eval 92.7% █····▄·▄·▄▄● beats 6/6 medium · avg 68% 1.2% typical
Multi-IF (aggregation unspecified… 77.5% ▁▁▁▁▁▁▃▄●▄▁▂ beats 26/41 medium · avg 59% 1.2% several runs
COLLIE (constrained text generati… 49.7% ▄···▄●▄····█ beats 2/6 hard · avg 30% 1.1% at its level
VitaBench 8.5% ●·······▄█·█ beats 1/6 very hard · avg 6.5% 1.0% typical
ARC-AGI-2 public evaluation set (… 0.8% ●··▂▅▁▃▁▃·▁▃ beats 5/24 very hard · avg 0.3% 1.0% sharp · too hard for it · noisy
ZebraLogic 94.2% █···█··███·● beats 5/5 medium · avg 51% 0.8% noisy · several runs
BeyondAIME 57.6% █····█·▄▄··● beats 6/6 hard · avg 24% 0.4% at its level
DROP 78.6% ▄···▄·▄●·▄·▄ beats 3/6 medium · avg 58% 0.1% blunt · at its level · noisy
source results share of the score
Qwen/Qwen3-Next-80B-A3B-Instruct model … 24 33% ▇▇▇▇
Meituan LongCat-Flash-Chat model card 22 31% ▇▇▇▇
Qwen3-VL-235B-A22B-Instruct model card … 18 16% ▇▇
Qwen/Qwen3-235B-A22B-Instruct-2507 mode… 11 11% ▇
ARC Prize Foundation model_baseline out… 2 3% ▇
LMArena leaderboard dataset 1 3% ▇
Berkeley Function Calling Leaderboard r… 1 1% ▇
SEATauBench: Progressively Adapting Too… 1 1% ▇
Galileo Agent Leaderboard 1 1% ▇
prior: pull toward the middle 0% ▇
weighted average = 64.3
Licences: Qwen/Qwen3-Next-80B-A3B-Instruct model card ([apache-2.0](../../licenses/Qwen--Qwen3-Next-80B-A3B-Instruct.txt)) · Meituan LongCat-Flash-Chat model card ([MIT](https://huggingface.co/meituan-longcat/LongCat-Flash-Chat/blob/f0b8e8d7380330f450ca1fde4ec56823a514584c/LICENSE)) · Qwen3-VL-235B-A22B-Instruct model card ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Qwen/Qwen3-235B-A22B-Instruct-2507 model card ([apache-2.0](../../licenses/Qwen--Qwen3-235B-A22B-Instruct-2507.txt)) · ARC Prize Foundation model\_baseline outputs on ARC-AGI-1/2 public evaluation sets ([MIT](../../licenses/arcprize-model-baseline-outputs-MIT.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · SEATauBench: Progressively Adapting Tool-Agent-User Evaluation Into Low-Resource Southeast Asian Languages ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Galileo Agent Leaderboard ([Apache-2.0](../../licenses/galileo-agent-leaderboard-Apache-2.0.txt))
## How to run it
### Through an API
api (Deep Infra) · $0.21/M blended · $0.09 in / $0.55 out per 1M tokens · [deepinfra.com](https://deepinfra.com/models)
[price source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/providers/deepinfra/models/Qwen/Qwen3-235B-A22B-Instruct-2507.toml): Standard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/models/alibaba/qwen3-235b-a22b-instruct-2507.toml)
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Qwen3 235B-A22B Instruct 2507 (ud-q2_k_xl, needs ~71 GB, from [huggingface.co/unsloth](https://huggingface.co/unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF/tree/main))
```sh
llama cli -hf unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF:UD-Q2_K_XL
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF:UD-Q2_K_XL
```
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Qwen3 235B-A22B Instruct 2507 (4bit, needs ~132.2 GB, Mac only, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Qwen3-235B-A22B-Instruct-2507-MLX-4bit/tree/10761c281c9dd6836b41fd625da2475d18ade4c8))
```sh
lms get https://huggingface.co/lmstudio-community/Qwen3-235B-A22B-Instruct-2507-MLX-4bit -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: ud-q2_k_xl (~71 GB) `lms get https://huggingface.co/unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF@UD-Q2_K_XL -y` · 8bit (~249.8 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/Qwen3-235B-A22B-Instruct-2507-MLX-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Qwen3 235B-A22B Instruct 2507 (Q4_K_M, needs ~141.5 GB, from [ollama.com](https://ollama.com/library/qwen3/tags))
```sh
ollama run qwen3:235b-a22b-instruct-2507-q4_K_M
```
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run Qwen3 235B-A22B Instruct 2507 (4bit, needs ~132.2 GB, Mac only, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Qwen3-235B-A22B-Instruct-2507-MLX-4bit/tree/10761c281c9dd6836b41fd625da2475d18ade4c8))
```sh
mlx_lm.chat --model lmstudio-community/Qwen3-235B-A22B-Instruct-2507-MLX-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model lmstudio-community/Qwen3-235B-A22B-Instruct-2507-MLX-4bit
```
Other versions: 8bit (~249.8 GB, Mac only) `mlx_lm.chat --model lmstudio-community/Qwen3-235B-A22B-Instruct-2507-MLX-8bit`
Model repository and documentation: [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507)
## Published speed measurements
Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine.
hardware precision speed engine and settings
AMD Ryzen AI Max+ 395 (Stri… UD-Q3_K_XL 15.2 tok/s llama.cpp bbd0f9177 (commit from …
Each speed links to its source, which has the full settings.
## Quantization evidence
How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent. They don't describe every download of that precision.
precision test full → quantized kept
INT8 W8A8 (weights static per… MATH-500 0-shot (lighteval math_5… 0.89 → 0.897 100.8%
INT8 W8A8 (weights static per… MMLU-Pro 5-shot (lm-evaluation-ha… 0.831 → 0.834 100.4%
INT8 W8A8 (weights static per… IFEval (lm-evaluation-harness ife… 0.917 → 0.917 100.0%
INT8 W8A8 (weights static per… MMLU 5-shot CoT (lm-evaluation-ha… 0.893 → 0.893 99.9%
INT8 W8A8 (weights static per… GSM8K-Platinum 5-shot CoT (lm-eva… 0.975 → 0.97 99.5%
NVFP4 W4A4 (FP4 E2M1 weights … OpenLLM v2 average as published (… 0.603 → 0.609 101.0%
NVFP4 W4A4 (FP4 E2M1 weights … IFEval (OpenLLM v2 leaderboard co… 0.902 → 0.91 100.9%
NVFP4 W4A4 (FP4 E2M1 weights … GSM8K (OpenLLM v1 configuration, … 0.899 → 0.904 100.5%
NVFP4 W4A4 (FP4 E2M1 weights … MMLU-Pro (OpenLLM v2 leaderboard … 0.635 → 0.633 99.7%
NVFP4 W4A4 (FP4 E2M1 weights … MMLU (OpenLLM v1 configuration, 5… 0.875 → 0.871 99.5%
NVFP4 W4A4 (FP4 E2M1 weights … OpenLLM v1 average (6 tasks: arc_… 0.805 → 0.8 99.3%
NVFP4 W4A4 (FP4 E2M1 weights … MATH-500 0-shot (lighteval, vLLM) 0.942 → 0.892 94.7%
NVFP4 W4A4 (FP4 E2M1 weights … Reasoning average (2 tasks: GPQA … 0.817 → 0.769 94.2%
NVFP4 W4A4 (FP4 E2M1 weights … GPQA Diamond 0-shot (lighteval, v… 0.692 → 0.646 93.4%
Each kept value links to its source.
[interactive version](https://whatmodel.sh/model/qwen3-235b-a22b-instruct-2507) · [JSON](https://whatmodel.sh/api?format=json&all=1)