[whatmodel.sh](../../README.md) / GLM-5.1
# GLM-5.1
Z.ai · released 2026-04 · 744B params (40B active) · mit
**score 69.5** ±0.4 · #82 of 732 · from 81 benchmark results across 17 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks GLM-5.1 among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█▄▅▅●▄▂▁ #63 of 313 26% 19
coding ▁▁▁▃▅▇█▇▅●▃▁ #58 of 469 18% 9
chat & writing ▁▁▁▃▆▄▆█▆▇●▃ #41 of 402* 2.5% 2
math ▁▁▃▅█▆▄●▄▂▁▁ #76 of 431 32% 16
web search ▁▁▁▅●█▃▄·▁·▁ #52 of 84 5.0% 5
agent tasks · #63 of 313 · 26% of its score
Beats [GPT-5.2](gpt-5-2.md), [Gemini 3.6 Flash](gemini-3-6-flash.md), [LongCat 2.0 (1.6T-A48B)](longcat-2-0.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 74.2% ▁··▁·▁▂▁▄●▄▁ beats 16/29 hard · avg 19% 7.8% sharp · at its level · several runs
Terminal-Bench 2.1 59.3% █··▄█·●····▄ beats 5/6 very hard · avg 13% 2.5% sharp · at its level · several runs
SWE-bench Multilingual 74.8% ▄··▄▄···●▄·█ beats 3/6 hard · avg 30% 1.8% at its level
MCP-Atlas (public set) (maker) 71.8% █··●·██·████ beats 1/7 hard · avg 30% 1.7% at its level · several runs
Humanity's Last Exam with tools 50.4% ▄····▄·█·●·▄ beats 4/5 hard · avg 15% 1.5% at its level · several runs
BrowseComp 59.4% ▄·▄▄···▄··●▄ beats 5/6 hard · avg 18% 1.4% at its level · several runs
SWE-bench Pro (split unspecified)… 58.4% ▂·▁▁▁▁▂▃█●▂▁ beats 41/58 very hard · avg 9.6% 1.2% sharp · at its level · small run
τ³-bench 69.7% ▄····●·█·▄·█ beats 1/6 medium · avg 42% 1.1% at its level · several runs
K-BrowseComp-Verified (Korean web… 30.7% ▃····█··▃●·▃ beats 6/8 very hard · avg 3.6% 1.1% typical
Toolathlon (Tool Decathlon) (make… 40.7% ▂▂▄▂▄▄●▂█▆██ beats 11/27 very hard · avg 13% 1.0% at its level · small run · several runs
Terminal-Bench 2.0 (maker) 63.5% ▂▂▅▅████●▂▃▃ beats 28/36 hard · avg 15% 0.9% at its level · small run
Thinkingbox-bench pass@1 (507 sta… 33.2% ▅··▃·▃·●▃▅██ beats 6/15 very hard · avg 5.1% 0.8% sharp · at its level
EEGAgentBench overall (tool-based… 46.7% ▂·▁▁··▇●█▂·▁ beats 14/25 hard · avg 28% 0.6% blunt · at its level
Legal Research Bench (agentic, al… 27.6% ▄█··▄▄▄●█·▄█ beats 6/12 very hard · avg 6.3% 0.6% typical
τ³-bench telecom (tau2-bench ≥ v1… 96.9% ▄·▄···▄·●·█▄ beats 3/6 easy · avg 79% 0.4% maxed out
BrowseComp with context managemen… 79.3% ▄·▄▂·█●·▆▄·▂ beats 9/15 medium · avg 46% 0.4% small run
KlinikeBench strict pass@1 (333 i… 4.8% ▁▁·▁▁▁●▅▃█▅█ beats 5/27 very hard · avg 2.4% 0.4% typical
TerminalWorld-Verified (200 tasks… 57.0% ▄··▄·▄·●···▄ beats 3/5 hard · avg 28% 0.4% blunt · at its level
SWE-bench Pro (maker) 58.4% ▄▂▂▂▄▂▆●·▂·▂ beats 11/16 hard · avg 32% 0.3% blunt · at its level · small run
coding · #58 of 469 · 18% of its score
Beats [GPT-5.4](gpt-5-4.md), [Gemini 3.1 Pro (Preview)](gemini-3-1-pro.md), [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 74.2% ▁··▁·▁▂▁▄●▄▁ beats 16/29 hard · avg 19% 7.8% sharp · at its level · several runs
Arena WebDev (Code Arena) 1507.9 ▁▁▃▂▅█▃●▅▅▁▂ beats 73/129 avg 981 3.5% sharp
SWE-bench Multilingual 74.8% ▄··▄▄···●▄·█ beats 3/6 hard · avg 30% 1.8% at its level
SWE-bench Pro (split unspecified)… 58.4% ▂·▁▁▁▁▂▃█●▂▁ beats 41/58 very hard · avg 9.6% 1.2% sharp · at its level · small run
LiveCodeBench v6 (window unspecif… 85.7% ██···●··████ beats 2/6 medium · avg 46% 1.1% typical
SciCode subproblems 47.7% ▃····▃··●·▃▃ beats 2/6 hard · avg 19% 1.0% at its level
SciCode (level unspecified) 43.8% ▄·····▄█●▄·▄ beats 4/6 hard · avg 16% 1.0% at its level
LiveCodeBench v6 (Feb–May 2025) 86.4% ▄··█····▄▄·● beats 5/6 medium · avg 47% 0.8% typical
SWE-bench Pro (maker) 58.4% ▄▂▂▂▄▂▆●·▂·▂ beats 11/16 hard · avg 32% 0.3% blunt · at its level · small run
chat & writing · #41 of 402 · 2.5% of its score
Beats [GPT-6 Sol](gpt-6-sol.md), [MiMo-V2.6-Flash (309B-A15B)](mimo-v2-6-flash.md), [Claude Haiku 5.5](claude-haiku-5-5.md) here, though they rank higher overall. Trails [MiMo-V2.5-Pro (1T-A42B)](mimo-v2-5-pro.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LMArena (text) 1465.4 ▁▁▁▂▃▃▃▆▆▇●▃ beats 340/385 avg 1260 2.0% blunt
IFEval (aggregation unspecified) 93.9% ▁▂▄▂▂▄▆██▅●▁ beats 138/145 easy · avg 80% 0.5% maxed out
math · #76 of 431 · 32% of its score
Beats [Claude Sonnet 4.6](claude-sonnet-4-6.md), [Claude Opus 4.5](claude-opus-4-5.md), [Qwen3.7 Plus](qwen3-7-plus.md) here, though they rank higher overall. Trails [Granite 4.2 30B](granite-4-2-30b.md), [INTELLECT-3](intellect-3.md), [Qwen3 235B-A22B Thinking 2507](qwen3-235b-a22b-thinking-2507.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
FrontierMath tiers 1–3 v2 36.8% ▂··▁▁▃▄●█▆▄▃ beats 27/86 very hard · avg 1.2% 9.3% sharp · at its level
MathArena ArXivMath April 2026 (4… 36.3% ▁·····●██▄·▁ beats 4/20 very hard · avg 7.6% 4.1% sharp · at its level
AIME 2026 95.4% ▄·▄·▄▄··●··▄ beats 5/6 medium · avg 63% 2.7% sharp · maxed out · small run · several…
MathArena ArXivMath February 2026… 39.1% ▁▁▁▁█●▄▃▁▃·▃ beats 11/24 very hard · avg 9.4% 2.3% at its level
HMMT February 2026 (maker) 82.6% ▃▃▂▅·███●▅▃▅ beats 23/35 hard · avg 30% 2.2% sharp · small run · several runs
MathArena ArXivMath March 2026 (3… 49.2% █●▅·█▃▃·▅·▃▃ beats 3/14 hard · avg 16% 1.9% at its level
IMO-AnswerBench 86.8% ▄····▄·●▄·█▄ beats 2/6 medium · avg 40% 1.8% small run · several runs
MathArena ArXivMath January 2026 … 65.2% ▂··▄▂▄█▆▆●▄█ beats 16/23 hard · avg 23% 1.6% at its level
MathArena ArXivMath May 2026 (40 … 52.5% ▁··▁▄●▄▁·▃·▁ beats 6/17 very hard · avg 11% 1.5% at its level
MathArena BrokenArXiv April 2026 … 8.2% ▂●▆█▃▃▂▂▂··▂ beats 1/18 very hard · avg 0.8% 1.0% sharp · noisy
MathArena BrokenArXiv May 2026 (5… 10.0% ●█·▄▄▄····▄▄ beats 1/8 very hard · avg 1.9% 0.7% sharp
OTIS mock AIME 2024–2025 93.3% ▃▃▄▃▃▃▇▇●▃▂▅ beats 156/197 very hard · avg 7.2% 0.7% sharp · noisy
MathArena BrokenArXiv March 2026 … 6.5% ●·▆█▂·▂▂···▂ beats 2/14 very hard · avg 1.0% 0.6% sharp · noisy
MathArena BrokenArXiv February 20… 9.3% ▃▂▂●█▃▂·▃··▂ beats 4/16 very hard · avg 0.9% 0.6% sharp · noisy
HMMT November 2025 (maker) 94.0% ▁▁····▁▅▃█●▁ beats 20/23 easy · avg 73% 0.5% maxed out · small run
MathArena ArXivLean March 2026 (4… 2.4% ▂····●·····█ beats 1/5 very hard · avg 1.2% 0.5% too hard for it
web search · #52 of 84 · 5.0% of its score
Beats [Gemini 3.1 Pro (Preview)](gemini-3-1-pro.md), [GPT-5.2](gpt-5-2.md), [Kimi K2.6](kimi-k2-6.md) here, though they rank higher overall. Trails [GPT-4o mini (2024-07-18)](gpt-4o-mini-2024-07-18.md), [Mistral Small 3.1 24B (2503)](mistral-small-3-1.md), [Llama 4 Scout (109B-A17B)](llama-4-scout.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Humanity's Last Exam with tools 50.4% ▄····▄·█·●·▄ beats 4/5 hard · avg 15% 1.5% at its level · several runs
BrowseComp 59.4% ▄·▄▄···▄··●▄ beats 5/6 hard · avg 18% 1.4% at its level · several runs
K-BrowseComp-Verified (Korean web… 30.7% ▃····█··▃●·▃ beats 6/8 very hard · avg 3.6% 1.1% typical
Legal Research Bench (agentic, al… 27.6% ▄█··▄▄▄●█·▄█ beats 6/12 very hard · avg 6.3% 0.6% typical
BrowseComp with context managemen… 79.3% ▄·▄▂·█●·▆▄·▂ beats 9/15 medium · avg 46% 0.4% small run
other tests (18)
test result vs the others in its run difficulty counts why
GPQA Diamond 89.9% ▁▁▁▁▂▄▆▄▅▇●▅ beats 174/211 hard · avg 44% 8.6% sharp · several runs
Humanity's Last Exam 27.2% ▄··●▄··▄·▄·▄ beats 2/6 very hard · avg 5.7% 5.2% at its level · several runs
GPQA (subset unspecified) 86.1% ●█▄▄·▄·····▄ beats 0/6 hard · avg 47% 3.0% sharp
MMLU-Pro 85.9% ▄·····●▄▄·▄█ beats 1/6 medium · avg 53% 2.6% several runs
Global-MMLU-Lite 90.3% ▃······▅··●▃ beats 3/6 medium · avg 61% 1.5% typical
Chess Puzzles (Epoch) 19.0% █▁▁▁▁▂●▃▃▂▁▁ beats 75/137 very hard · avg 5.8% 1.2% sharp · noisy
EQ-Bench 3 (emotional intelligenc… 1309.6 ▁·▂▁▄▅▅▃●▄▁▁ beats 51/62 avg 1034 1.2% typical
NL2Repo (maker) 42.7% ▂▂▂·▃▃▅●▃▂▅▃ beats 12/22 very hard · avg 12% 1.2% at its level · small run · several runs
AgentWorldBench (open-ended, over… 51.3% ▄▄··▄●·▄█▄██ beats 3/11 hard · avg 26% 1.0% blunt · at its level
WildClawBench 48.2% ▃·▅▅█·▅█●▃▃▃ beats 13/17 very hard · avg 11% 1.0% at its level
IFBench (aggregation unspecified) 76.3% ▃······█●▃·▃ beats 4/6 medium · avg 45% 0.9% typical
MMLU-ProX 85.8% ▄·····▄·▄▄●▄ beats 5/6 medium · avg 58% 0.9% typical
IFBench prompt-level (loose) 76.6% ▄▄·●··▄▄···█ beats 2/6 medium · avg 47% 0.8% typical
MultiChallenge 63.0% ▁··········● beats 1/6 hard · avg 28% 0.7% at its level
SimpleQA Verified 34.0% ▁▁▃▂▃▃●▆█▄▂▅ beats 27/81 very hard · avg 8.0% 0.7% at its level
CritPt 3.7% ▄····▄●···█▄ beats 3/6 very hard · avg 1.1% 0.7% too hard for it · noisy · several runs
Legal Research Bench (single-shot… 3.1% ▃▃●▃·▅···█▃▃ beats 4/12 very hard · avg 0.8% 0.6% too hard for it
CyberGym (maker) 68.7% ▄···▄··●█▄█▄ beats 2/8 hard · avg 28% 0.3% at its level · small run
source results share of the score
NVIDIA 17 22% ▇▇▇
Epoch AI 6 18% ▇▇
MathArena ArXivMath / BrokenArXiv / ArX… 10 15% ▇▇
zai-org/GLM-5.1 model card (submitted /… 13 10% ▇
A.X K2 Technical Report 7 10% ▇
zai-org/GLM-5.2 model card (submitted /… 13 10% ▇
LMArena leaderboard dataset 2 5% ▇
LGAI-EXAONE/K-EXAONE-2.0-750B-A37B mode… 3 3% ▇
EQ-Bench 3 canonical leaderboard results 1 1% ▇
Legal Research Bench 2 1% ▇
K-BrowseComp: A Web Browsing Agent Benc… 1 1% ▇
Qwen/Qwen-AgentWorld-35B-A3B model card 1 1% ▇
WildClawBench: A Benchmark for Real-Wor… 1 1% ▇
One Success Isn't Reliability: Thinking… 1 1% ▇
EEGAgentBench: Benchmarking LLM Agents … 1 1% ▇
KlinikeBench: Evaluating Language Model… 1 0% ▇
TerminalWorld: Benchmarking Agents on R… 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 69.5
Licences: NVIDIA ([CC BY 4.0](../../licenses/report-2606.15007v1.txt)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets ([CC BY-SA 4.0](../../licenses/matharena-sharealike-NOTICE.txt)) · zai-org/GLM-5.1 model card ([mit](../../licenses/zai-org--GLM-5.1.txt)) · A.X K2 Technical Report ([CC BY 4.0](http://creativecommons.org/licenses/by/4.0/)) · zai-org/GLM-5.2 model card ([mit](../../licenses/zai-org--GLM-5.2.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · LGAI-EXAONE/K-EXAONE-2.0-750B-A37B model card ([Apache-2.0](https://huggingface.co/LGAI-EXAONE/K-EXAONE-2.0-750B-A37B/blob/fec0c8dd25fdd20ffd09869d5af5277966d5523d/LICENSE)) · EQ-Bench 3 canonical leaderboard results ([MIT](../../licenses/eqbench3-MIT.txt)) · Legal Research Bench ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Qwen/Qwen-AgentWorld-35B-A3B model card ([Apache-2.0](https://huggingface.co/Qwen/Qwen-AgentWorld-35B-A3B/blob/60d2b0434a53d2e62a7c00a489586815d94ebffb/README.md)) · WildClawBench: A Benchmark for Real-World ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · One Success Isn't Reliability: Thinkingbox ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks ([CC BY 4.0](../../licenses/paper-2605.22535v1-CC-BY-4.0.txt))
## How to run it
### Through an API
api (Z.AI) · $2.15/M blended · $1.4 in / $4.4 out per 1M tokens · [docs.z.ai](https://docs.z.ai/guides/overview/pricing)
[price source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/providers/zai/models/glm-5.1.toml): Standard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/models/zhipuai/glm-5.1.toml)
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run GLM-5.1 (ud-q4_k_xl, needs ~446.9 GB, from [huggingface.co/unsloth](https://huggingface.co/unsloth/GLM-5.1-GGUF/tree/main))
```sh
llama cli -hf unsloth/GLM-5.1-GGUF:UD-Q4_K_XL
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf unsloth/GLM-5.1-GGUF:UD-Q4_K_XL
```
Other versions: ud-q2_k_xl (~223.7 GB) `llama cli -hf unsloth/GLM-5.1-GGUF:UD-Q2_K_XL` · ud-iq1_m (~112.1 GB) `llama cli -hf unsloth/GLM-5.1-GGUF:UD-IQ1_M`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run GLM-5.1 (ud-q4_k_xl, needs ~446.9 GB, from [huggingface.co/unsloth](https://huggingface.co/unsloth/GLM-5.1-GGUF/tree/main))
```sh
lms get https://huggingface.co/unsloth/GLM-5.1-GGUF@UD-Q4_K_XL -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: ud-q2_k_xl (~223.7 GB) `lms get https://huggingface.co/unsloth/GLM-5.1-GGUF@UD-Q2_K_XL -y` · ud-iq1_m (~112.1 GB) `lms get https://huggingface.co/unsloth/GLM-5.1-GGUF@UD-IQ1_M -y` · 4bit (~418.6 GB, Mac only) `lms get https://huggingface.co/catalystsec/GLM-5.1-4bit -y`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run GLM-5.1 (4bit, needs ~418.6 GB, Mac only, from [huggingface.co/catalystsec](https://huggingface.co/catalystsec/GLM-5.1-4bit/tree/486f24ddc83059f6cac2fe0a6ceabca5af76ad20))
```sh
mlx_lm.chat --model catalystsec/GLM-5.1-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model catalystsec/GLM-5.1-4bit
```
Model repository and documentation: [huggingface.co/zai-org](https://huggingface.co/zai-org/GLM-5.1)
[interactive version](https://whatmodel.sh/model/glm-5-1) · [JSON](https://whatmodel.sh/api?format=json&all=1)