[whatmodel.sh](../../README.md) / GLM-4 (0116) # GLM-4 (0116) Z.ai · released 2024-01 · proprietary **score 51.1** ±1.3 · comparison only: no current run option verified · from 7 benchmark results across 2 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks GLM-4 (0116) among every model on that task or test, worst to best.}

task            vs the others          rank  counts  tests
coding          ●··········█       #2 of 2*     11%      1
chat & writing  ▁▁▁▃▅▄●█▇▇▇▃   #300 of 474*     43%      1
math            ●··········█       #2 of 2*     18%      2

coding · #2 of 2 · 11% of its score

test               result  vs the others in its run    difficulty      counts  why
HumanEval (maker)   72.0%  ▁▁▁▁▂▂▆▆●▅▂▁ beats 142/205  easy · avg 71%     11%  sharp · at its level · small run

chat & writing · #300 of 474 · 43% of its score

Beats [Mistral Small 4 (119B-A6.5B)](mistral-small-4.md), [Qwen2.5 Coder 32B Instruct](qwen2-5-coder-32b.md), [Ministral 3 14B Reasoning 2512](ministral-3-14b-reasoning.md) here, though they rank higher overall. Trails [Flan-T5 XXL](flan-t5-xxl.md), [Granite 4.0 H 1B](granite-4-0-h-1b.md), [Granite 4.0 1B](granite-4-0-1b.md) here, though they rank lower overall.

test                                result  vs the others in its run  difficulty        counts  why
Arena-Hard (v0.1, GPT-4-1106 judg…   55.7%  ▂▁▁▅█▅▄●▃▅▃▁ beats 38/60  medium · avg 47%     43%  sharp · at its level

math · #2 of 2 · 18% of its score

test                               result  vs the others in its run    difficulty           counts  why
GSM8K (maker)                       87.6%  ▂▂▂▂▃▄▅▇██●▄ beats 167/206  very easy · avg 86%     11%  sharp · small run
MATH (subset unspecified) (maker)   47.9%  ▂▄▄▃●▇▅▆▄▃▄▁ beats 22/80    medium · avg 55%       6.7%  at its level · small run

other tests (3)

test                               result  vs the others in its run    difficulty      counts  why
BIG-Bench Hard (maker)              82.3%  ▁▁▁▁▂▄▅▆█●▃▁ beats 138/155  easy · avg 78%     10%  sharp · at its level · small run
GPQA (subset unspecified) (maker)   35.7%  ▂▁▂▂▄●▆▃▅█▃▃ beats 31/106   hard · avg 36%     10%  sharp · small run
MMLU (maker)                        81.5%  ▁▁▁▁▁▂▂▅██●▄ beats 174/214  easy · avg 78%    8.1%  at its level · small run

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
ChatGLM: A Family of Large Language Mod…        6   56% ▇▇▇▇▇▇▇
Arena-Hard-Auto v0.1 leaderboard                1   43% ▇▇▇▇▇
prior: pull toward the middle                        1% ▇
weighted average                                   = 51.1

_{Licences: ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools ([CC BY 4.0](http://creativecommons.org/licenses/by/4.0/)) · Arena-Hard-Auto v0.1 leaderboard ([Apache-2.0](../../licenses/arena-hard-auto-v0-1-Apache-2.0.txt))} ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/glm-4-0116) · [JSON](https://whatmodel.sh/api?format=json&all=1)