[whatmodel.sh](../../README.md) / GLM-4 (0520) # GLM-4 (0520) Z.ai · released 2024-05 · closed weights **score 53.2** ±1.1 · comparison only: no current run option verified · from 10 benchmark results across 4 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks GLM-4 (0520) among every model on that task or test, worst to best.}

task            vs the others          rank  counts  tests
agent tasks     ▁▁▂▆█▄●▄▄▃▁▁   #161 of 354*    2.2%      1
coding          █··········●       #1 of 2*    7.7%      1
chat & writing  ▁▁▁▃▅▄●█▇▇▇▃    #306 of 474     56%      3
math            █··········●       #1 of 2*     13%      2

agent tasks · #161 of 354 · 2.2% of its score

Beats [Gemini 2.5 Pro](gemini-2-5-pro.md), [DeepSeek V3.2 Exp](deepseek-v3-2-exp.md), [o1 (2024-12-17)](o1-2024-12-17.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Large](jamba-1-5-large.md) here, though they rank lower overall.

test                               result  vs the others in its run  difficulty      counts  why
BFCL (version not stated) (maker)   81.8%  █·········█● beats 4/5    easy · avg 77%    2.2%  blunt

coding · #1 of 2 · 7.7% of its score

test               result  vs the others in its run    difficulty      counts  why
HumanEval (maker)   78.5%  ▁▁▁▁▂▂▆▆█●▂▁ beats 172/205  easy · avg 71%    7.7%  sharp · small run

chat & writing · #306 of 474 · 56% of its score

Beats [Mistral Small 4 (119B-A6.5B)](mistral-small-4.md), [Qwen2.5 Coder 32B Instruct](qwen2-5-coder-32b.md), [Ministral 3 14B Reasoning 2512](ministral-3-14b-reasoning.md) here, though they rank higher overall. Trails [Flan-T5 XXL](flan-t5-xxl.md), [Granite 4.0 H 1B](granite-4-0-h-1b.md), [Granite 4.0 1B](granite-4-0-1b.md) here, though they rank lower overall.

test                                result  vs the others in its run    difficulty        counts  why
Arena-Hard (v0.1, GPT-4-1106 judg…   63.8%  ▂▁▁▅█▅▄▃●▅▃▁ beats 44/60    medium · avg 47%     31%  sharp · at its level
LMArena (text)                      1273.4  ▁▁▁▂▃▃●▆▆▇█▃ beats 104/385  avg 1260             13%  blunt
Polyrating rating (bias-aware re-…  1202.0  ▂▁▂▁▃▇█▆▇●▂▂ beats 81/95    avg 1187             12%  typical

math · #1 of 2 · 13% of its score

test                               result  vs the others in its run    difficulty           counts  why
GSM8K (maker)                       93.3%  ▂▂▂▂▃▄▅▇██●▄ beats 188/206  very easy · avg 86%    7.8%  sharp · small run
MATH (subset unspecified) (maker)   61.3%  ▂▄▄▃█●▅▆▄▃▄▁ beats 35/80    medium · avg 55%       5.0%  at its level · small run

other tests (3)

test                               result  vs the others in its run    difficulty      counts  why
GPQA (subset unspecified) (maker)   39.9%  ▂▁▂▂▄▃●▃▅█▃▃ beats 37/106   hard · avg 36%    7.9%  sharp · small run
BIG-Bench Hard (maker)              84.7%  ▁▁▁▁▂▄▅▆█▃●▁ beats 143/155  easy · avg 78%    7.3%  sharp · small run
MMLU (maker)                        83.3%  ▁▁▁▁▁▂▂▅██●▄ beats 179/214  easy · avg 78%    5.9%  small run

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
ChatGLM: A Family of Large Language Mod…        7   44% ▇▇▇▇▇
Arena-Hard-Auto v0.1 leaderboard                1   31% ▇▇▇▇
LMArena leaderboard dataset                     1   13% ▇▇
Polyrating: A Cost-Effective and Bias-A…        1   12% ▇
prior: pull toward the middle                        1% ▇
weighted average                                   = 53.2

_{Licences: ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools ([CC BY 4.0](http://creativecommons.org/licenses/by/4.0/)) · Arena-Hard-Auto v0.1 leaderboard ([Apache-2.0](../../licenses/arena-hard-auto-v0-1-Apache-2.0.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/))} ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/glm-4-0520) · [JSON](https://whatmodel.sh/api?format=json&all=1)