[whatmodel.sh](../../README.md) / ERNIE 4.5 21B-A3B # ERNIE 4.5 21B-A3B Baidu · released 2025-06 · 21B params (3B active) · apache-2.0 **score 52.6** ±1.2 · comparison only: no current run option verified · from 21 benchmark results across 3 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks ERNIE 4.5 21B-A3B among every model on that task or test, worst to best.}

task            vs the others          rank  counts  tests
agent tasks     ▁▁▂●█▄▅▄▄▃▁▁   #323 of 354*    3.7%      1
coding          ●··········█        #2 of 2     14%      3
chat & writing  ●··········█       #2 of 2*    2.9%      1
math            ●··········█        #2 of 2     30%      4

agent tasks · #323 of 354 · 3.7% of its score

Beats [Magistral Small 1.0 (2506)](magistral-small-2506.md), [Llama 4 Scout (109B-A17B)](llama-4-scout.md) here, though they rank higher overall. Trails [Granite 4.0 H 350M](granite-4-0-h-350m.md), [Qwen3 0.6B](qwen3-0-6b.md), [Qwen2.5 1.5B Instruct](qwen2-5-1-5b.md) here, though they rank lower overall.

test                                result  vs the others in its run  difficulty      counts  why
EEGAgentBench overall (tool-based…   22.8%  ●·▁▁··▇▇█▂·▁ beats 0/25   hard · avg 28%    3.7%  blunt · at its level

coding · #2 of 2 · 14% of its score

test                                result  vs the others in its run    difficulty      counts  why
HumanEval+ (maker)                   89.6%  ▁·▁▁▂▃▄▄█▅●▃ beats 131/145  easy · avg 82%    7.0%  sharp · small run
LiveCodeBench (version unspecifie…   26.4%  ▁···▁●▁█▇▃▄▁ beats 4/38     hard · avg 27%    4.1%  at its level · small run
MBPP+ (maker)                        77.5%  ▁··▁▁▃▄█▆●▁▁ beats 104/120  easy · avg 70%    3.1%  at its level · small run

chat & writing · #2 of 2 · 2.9% of its score

test                                result  vs the others in its run   difficulty      counts  why
IFEval (aggregation unspecified) …   80.0%  ▁▂▄▂▂▄●██▅▂▁ beats 53/145  easy · avg 78%    2.9%  small run

math · #2 of 2 · 30% of its score

test               result  vs the others in its run    difficulty           counts  why
GSM8K (maker)       93.3%  ▂▂▂▂▃▄▅▇██●▄ beats 189/206  very easy · avg 88%    9.2%  sharp · small run
AIME 2025 (maker)   19.7%  ▁·▁▁▁●▅▆█▇▅▁ beats 10/133   hard · avg 18%         8.5%  sharp · small run
AIME 2024 (maker)   27.9%  ▁▁▁▂▁●▁▅███▃ beats 15/85    hard · avg 29%         8.2%  sharp · at its level · small run
MATH-500 (maker)    87.2%  ▁▃▂▃▄●▅▇█▄▂▁ beats 31/85    very easy · avg 87%    4.4%  small run

other tests (12)

test                                result  vs the others in its run    difficulty        counts  why
BIG-Bench Hard (maker)               83.0%  ▁▁▁▁▂▄▅▆█▃●▁ beats 141/155  easy · avg 78%      8.2%  sharp · small run
MMLU (maker)                         76.0%  ▁▁▁▁▁▂▂▅█●▆▄ beats 155/214  medium · avg 72%    7.0%  at its level · small run
MMLU-Pro (maker)                     66.0%  ▁▁·▁▁▁▃▂●▅█▁ beats 76/188   medium · avg 59%    5.4%  at its level · small run
C-Eval (maker)                       85.4%  ▁···▁▁▂▁▂▂●█ beats 15/33    easy · avg 79%      5.1%  small run
SimpleQA (maker)                     24.2%  ▄▄▄▆█▆█●█·▂█ beats 23/32    hard · avg 17%      4.3%  small run
CMMLU (maker)                        84.8%  ▅·▅····█▃●█▅ beats 9/14     easy · avg 79%      4.1%  small run
ChineseSimpleQA (maker)              48.9%  ●··▄▄▄··▄··█ beats 0/6      medium · avg 52%    3.1%  at its level · small run
SysBench (aggregation unspecified…   48.0%  ●··█····█··█ beats 0/3      medium · avg 52%    2.9%  at its level · small run
EuroEval HellaSwag (English), tes…   83.0%  ▃·▁▁▂▁▂▅█●▄▁ beats 79/100   medium · avg 67%    2.6%  sharp · at its level · noisy
Multi-IF (aggregation unspecified…   61.0%  ▁▁▁▁▁●▃▄█▄▁▂ beats 6/41     medium · avg 61%    2.4%  at its level · small run
MuSR (maker)                         67.7%  █·██······●█ beats 3/4      medium · avg 57%    1.7%  blunt · at its level · small run
EuroEval Life in the UK (English …   83.7%  ▁▁▁▁▁▁▁▂▄●▄▃ beats 90/146   easy · avg 82%      1.5%  noisy

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
ERNIE 4.5 post-trained 300B-A47B benchm…       18   92% ▇▇▇▇▇▇▇▇▇▇▇
EuroEval leaderboards                           2    4% ▇
EEGAgentBench: Benchmarking LLM Agents …        1    4% ▇
prior: pull toward the middle                        1% ▇
weighted average                                   = 52.6

_{Licences: ERNIE 4.5 post-trained 300B-A47B benchmark chart ([Apache-2.0](../../licenses/cards-final-NOTICE.txt)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/))} ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/ernie-4-5-21b-a3b) · [JSON](https://whatmodel.sh/api?format=json&all=1)