[whatmodel.sh](../../README.md) / Qwen3 235B-A22B Thinking 2507
# Qwen3 235B-A22B Thinking 2507
Alibaba · released 2025-07 · 235B params (22B active) · apache-2.0
**score 65.7** ±0.5 · #133 of 732 · from 92 benchmark results across 12 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Qwen3 235B-A22B Thinking 2507 among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█●▅▅▄▄▂▁ #157 of 313 23% 8
coding ▁▁▁▃▅▇█▇●▄▃▁ #92 of 469 3.6% 2
chat & writing ▁▁▁▃▆▄▆█▆●▇▃ #115 of 402 7.3% 4
math ▁▁▃▅█▆▄●▄▂▁▁ #73 of 431 17% 6
agent tasks · #157 of 313 · 23% of its score
Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o4-mini](o4-mini.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Jamba 1.5 Large](jamba-1-5-large.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
τ²-bench Telecom (maker) 45.6% ▃·█········● beats 5/6 very hard · avg 9.8% 11% sharp · at its level · several runs
τ²-bench Retail 68.2% █···●·█·██·█ beats 1/5 hard · avg 35% 3.5% at its level · several runs
τ²-bench Airline 58.0% ▄·····●··▄·█ beats 2/5 hard · avg 32% 2.3% at its level · several runs
τ-bench Retail (maker) 67.8% ▁····▃·▂▇●▂▆ beats 17/28 medium · avg 37% 2.0% at its level · several runs
τ-bench Airline (maker) 46.0% ▁▁▁·▅▄▁●▇▁▁▇ beats 15/28 hard · avg 22% 1.5% at its level · several runs
τ²-bench aggregate (maker) 58.5% ▁▁▁·▁▁●▃▃▄█▁ beats 9/39 medium · avg 37% 0.9% at its level
BFCL v3 75.7% ▁▁▁▁▁▃▅█▆▅▄● beats 167/171 medium · avg 59% 0.8% blunt · several runs
Galileo Agent Leaderboard v2 (act… 34.0% ▅▃·▃▃●▅█▃█▅▅ beats 6/19 hard · avg 26% 0.5% at its level
coding · #92 of 469 · 3.6% of its score
Beats [Grok 4.20 (0309 reasoning)](grok-4-20.md), [GPT-5.4 mini](gpt-5-4-mini.md), [GPT-5.1](gpt-5-1.md) here, though they rank higher overall. Trails [Llama 3.1 Nemotron Ultra 253B v1](llama-3-1-nemotron-ultra-253b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LiveCodeBench v6 (Feb–May 2025) (… 74.1% ▁▁▃▁█▅▄▇●▅▃█ beats 26/38 medium · avg 44% 2.4% several runs
LiveCodeBench (version unspecifie… 75.4% ▄······▄●▄·▄ beats 3/5 medium · avg 41% 1.2% at its level
chat & writing · #115 of 402 · 7.3% of its score
Beats [LongCat 2.0 (1.6T-A48B)](longcat-2-0.md), [Step 3.5 Flash (196B-A11B)](step-3-5-flash.md), [GPT-5 mini](gpt-5-mini.md) here, though they rank higher overall. Trails [Nemotron-Cascade-8B](nemotron-cascade-8b.md), [Mistral Large 3 (675B-A41B)](mistral-large-3.md), [Gemini 2.5 Flash](gemini-2-5-flash.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena-Hard v2 aggregate (maker) 79.7% ▁▁··▄▃▁▃█▁▁● beats 19/22 hard · avg 28% 3.0% sharp · at its level · several runs
LMArena (text) 1400.1 ▁▁▁▂▃▃▃▆▆●█▃ beats 245/385 avg 1260 2.2% blunt
IFEval (aggregation unspecified) … 87.8% ▁▂▄▂▂▄▆●█▅▂▁ beats 93/145 easy · avg 77% 1.4% several runs
IFEval prompt-level strict 89.3% ███··●·█···█ beats 3/5 easy · avg 73% 0.7% typical
math · #73 of 431 · 17% of its score
Beats [Claude Sonnet 4.6](claude-sonnet-4-6.md), [Claude Opus 4.5](claude-opus-4-5.md), [Qwen3.7 Plus](qwen3-7-plus.md) here, though they rank higher overall. Trails [Granite 4.2 30B](granite-4-2-30b.md), [INTELLECT-3](intellect-3.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
HMMT 2025 (edition unspecified) 83.8% ▄··▄··▄·▄··● beats 5/5 hard · avg 31% 6.7% sharp · several runs
AIME 2025 92.5% ▁·▁▁▁▂▅▆█▇●▁ beats 124/133 medium · avg 50% 5.3% sharp · several runs
AIME 2024 93.9% ▁▁▁▂▁▃▁▅███● beats 83/85 medium · avg 61% 2.2% sharp
PolyMATH (maker) 60.1% ▁···▃▁·▇█●▄▄ beats 17/23 hard · avg 32% 1.4% at its level · several runs
OTIS mock AIME 2024–2025 86.7% ▃▃▄▃▃▃▇▇●▃▂▅ beats 131/197 very hard · avg 7.2% 1.4% sharp · noisy
MATH-500 99.6% █···██·█·█·● beats 5/5 very easy · avg 92% 0.5% maxed out · noisy
other tests (24)
test result vs the others in its run difficulty counts why
GPQA Diamond 80.1% ▁▁▁▁▂▄▆▄▅●█▅ beats 128/211 hard · avg 42% 10% sharp · at its level · several runs
MMLU-Pro 84.4% ▄·▄···●·▄··▄ beats 2/5 medium · avg 55% 4.9% several runs
MMLU-Redux 91.4% ████··●····█ beats 4/5 medium · avg 73% 3.9% several runs
GPQA (subset unspecified) (maker) 81.1% ▂▁▂▂▄▃▆▃▅█●▃ beats 98/106 hard · avg 49% 3.3% sharp · several runs
Humanity's Last Exam (maker) 18.2% ▁▁▁▄▆▆●██▆▁▃ beats 47/111 very hard · avg 5.1% 3.3% several runs
OJBench 32.1% ▄······●▄·▄█ beats 1/5 very hard · avg 13% 2.8% at its level · several runs
INCLUDE (maker) 81.0% ▂····▂▂▄▄▄●▄ beats 31/36 medium · avg 59% 2.8% several runs
Medmarks-V weighted mean win rate… 60.3% ▂▃▅▂▆▃█▅█▅●▆ beats 30/34 medium · avg 41% 1.8% blunt · at its level
MMLU-ProX (maker) 81.0% ▁·▁▁▁▂▁▃▃█●▄ beats 27/36 medium · avg 61% 1.7% several runs
SuperGPQA (maker) 64.9% ▂▁▂▂▅▆▃▇▇▇●▃ beats 41/46 medium · avg 39% 1.7% at its level · several runs
KINA (Knowledge Index of Noah's A… 32.1% ▁▁▂▁▃▄█●▄▅▂▃ beats 25/39 very hard · avg 21% 1.6% at its level
WritingBench (maker) 88.3% ▁·▁▂▁··▅█▂▂● beats 20/20 easy · avg 71% 1.5% several runs
Multi-IF (aggregation unspecified… 80.6% ▁▁▁▁▁▁▃▄█●▁▂ beats 33/41 medium · avg 60% 1.4% several runs
LiveBench 2024-11-25 (maker) 78.4% ▄··▂·▂·▂█▆●▂ beats 12/13 medium · avg 61% 1.3% several runs
ZebraLogic 97.5% ██··█·██···● beats 5/5 easy · avg 75% 1.1% maxed out
Creative Writing v3 (maker) 86.1% ▂····▂▆··▆●▆ beats 11/14 medium · avg 65% 1.1% several runs
Mystery Game Puzzles (Epoch) 9.0% ●··▂▃▇▄█▂▄▂▁ beats 5/77 very hard · avg 9.4% 0.9% sharp · too hard for it · noisy
VitaBench 21.5% █····●█·██·█ beats 1/5 very hard · avg 8.2% 0.9% typical
Chess Puzzles (Epoch) 12.0% █▁▁▁▁●▃▃▃▂▁▁ beats 57/137 very hard · avg 5.8% 0.9% sharp · noisy
SimpleQA Verified 40.4% ▁▁▃▂▃▃▆●█▄▂▅ beats 34/81 very hard · avg 8.0% 0.7% at its level
BeyondAIME 71.5% █··▄····▄··● beats 4/5 medium · avg 42% 0.6% at its level
SOOHAK Challenge (340 problems, A… 8.0% █·●████··█·█ beats 1/7 very hard · avg 4.3% 0.3% typical
SOOHAK-Mini (702 problems, Avg@3) 56.2% ▄··▄·●·▄▄█·▄ beats 2/7 medium · avg 36% 0.3% blunt · at its level
miniF2F (pass@1) 11.9% ●█······▄··▄ beats 1/5 very hard · avg 14% 0.1% blunt · at its level · noisy
source results share of the score
Meituan LongCat-Flash-Thinking model ca… 18 23% ▇▇▇
Qwen/Qwen3-235B-A22B-Thinking-2507 mode… 24 21% ▇▇▇
Qwen/Qwen3-Next-80B-A3B-Thinking model … 19 20% ▇▇
Qwen3-VL-235B-A22B-Thinking model card … 14 9% ▇
Qwen/Qwen3.5-27B model card (submitted … 5 8% ▇
Epoch AI 5 7% ▇
Qwen (submitted / maker-reported) 1 4% ▇
LMArena leaderboard dataset 1 2% ▇
Medmarks: A Comprehensive Open-Source L… 1 2% ▇
Knowledge Index of Noah's Ark 1 2% ▇
Soohak: A Mathematician-Curated Benchma… 2 1% ▇
Galileo Agent Leaderboard 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 65.7
Licences: Meituan LongCat-Flash-Thinking model card ([MIT](https://huggingface.co/meituan-longcat/LongCat-Flash-Thinking/blob/7c9aad1e2daceaeab3a5a03deedb8e11e7a7819f/LICENSE)) · Qwen/Qwen3-235B-A22B-Thinking-2507 model card ([apache-2.0](../../licenses/Qwen--Qwen3-235B-A22B-Thinking-2507.txt)) · Qwen/Qwen3-Next-80B-A3B-Thinking model card ([apache-2.0](../../licenses/Qwen--Qwen3-Next-80B-A3B-Thinking.txt)) · Qwen3-VL-235B-A22B-Thinking model card ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Qwen/Qwen3.5-27B model card ([apache-2.0](../../licenses/Qwen--Qwen3.5-27B.txt)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks ([CC BY 4.0](../../licenses/paper-2605.01417v1-CC-BY-4.0.txt)) · Knowledge Index of Noah's Ark ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs ([CC BY-SA 4.0](../../licenses/paper-2605.09063v3-CC-BY-SA-4.0.txt)) · Galileo Agent Leaderboard ([Apache-2.0](../../licenses/galileo-agent-leaderboard-Apache-2.0.txt))
## How to run it
### Through an API
api (Novita AI) · $0.97/M blended · $0.3 in / $3 out per 1M tokens · [novita.ai](https://novita.ai/models)
[price source](https://github.com/BerriAI/litellm/blob/c19ce71bcdad234ca87a7af10cee45375ce811a8/model_prices_and_context_window.json): USD per 1M text tokens, standard uncached input and output. Model endpoint qwen/qwen3-235b-a22b-thinking-2507. Product: Serverless chat completions. API model: qwen/qwen3-235b-a22b-thinking-2507. Settings: service tier=standard, batch=no, cache=uncached input. Additional/excluded charges: Tools, cache storage, priority service and other modalities excluded.. Currency: USD. · [context source](https://github.com/BerriAI/litellm/blob/c19ce71bcdad234ca87a7af10cee45375ce811a8/model_prices_and_context_window.json): provider maximum input tokens; not an intrinsic model maximum or legacy max\_tokens
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Qwen3 235B-A22B Thinking 2507 (ud-q2_k_xl, needs ~71 GB, from [huggingface.co/unsloth](https://huggingface.co/unsloth/Qwen3-235B-A22B-Thinking-2507-GGUF/tree/main))
```sh
llama cli -hf unsloth/Qwen3-235B-A22B-Thinking-2507-GGUF:UD-Q2_K_XL
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf unsloth/Qwen3-235B-A22B-Thinking-2507-GGUF:UD-Q2_K_XL
```
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Qwen3 235B-A22B Thinking 2507 (4bit, needs ~132.2 GB, Mac only, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Qwen3-235B-A22B-Thinking-2507-MLX-4bit/tree/7dbfe08304bd287abf110e86c24660e9c9f28d89))
```sh
lms get https://huggingface.co/lmstudio-community/Qwen3-235B-A22B-Thinking-2507-MLX-4bit -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: ud-q2_k_xl (~71 GB) `lms get https://huggingface.co/unsloth/Qwen3-235B-A22B-Thinking-2507-GGUF@UD-Q2_K_XL -y` · 8bit (~249.8 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/Qwen3-235B-A22B-Thinking-2507-MLX-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Qwen3 235B-A22B Thinking 2507 (Q4_K_M, needs ~141.5 GB, from [ollama.com](https://ollama.com/library/qwen3/tags))
```sh
ollama run qwen3:235b-a22b-thinking-2507-q4_K_M
```
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run Qwen3 235B-A22B Thinking 2507 (4bit, needs ~132.2 GB, Mac only, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Qwen3-235B-A22B-Thinking-2507-MLX-4bit/tree/7dbfe08304bd287abf110e86c24660e9c9f28d89))
```sh
mlx_lm.chat --model lmstudio-community/Qwen3-235B-A22B-Thinking-2507-MLX-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model lmstudio-community/Qwen3-235B-A22B-Thinking-2507-MLX-4bit
```
Other versions: 8bit (~249.8 GB, Mac only) `mlx_lm.chat --model lmstudio-community/Qwen3-235B-A22B-Thinking-2507-MLX-8bit`
Model repository and documentation: [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen3-235B-A22B-Thinking-2507)
[interactive version](https://whatmodel.sh/model/qwen3-235b-a22b-thinking-2507) · [JSON](https://whatmodel.sh/api?format=json&all=1)