[whatmodel.sh](../../README.md) / UL2 20B
# UL2 20B
Google · released 2022-05 · 20B params · apache-2.0
**score 28.0** ±2.5 · comparison only: no current run option verified · from 4 benchmark results across 2 sources · † no independently run results verified
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks UL2 20B among every model on that task or test, worst to best.
task rank counts tests
math not ranked yet — 1
math · 0.0% of its score
test result vs the others in its run difficulty counts why
GSM8K (maker) 5.4% ▂▂▂●▃▄▅▇██▆▄ beats 27/206 easy · avg 72% — not counted yet: no other model in the …
other tests (5)
test result vs the others in its run difficulty counts why
ARC Challenge (maker) 42.9% ▄▁▁▂▁▄█●▃▂▃▂ beats 86/140 easy · avg 86% 53% sharp · at its level
TriviaQA (maker) 48.7% ▁▁▂▂▄▄▃▆●█▅▂ beats 45/80 easy · avg 82% 24% at its level · small run
Natural Questions (closed book) (… 10.2% ▂▁▁▁▂▄▄●▅█▅▁ beats 27/68 medium · avg 39% 14% small run
ARC Easy (maker) 61.6% ▁▃▂▄●█▅▃▁··▁ beats 35/100 easy · avg 87% 6.2% blunt · at its level
OpenBookQA (maker) 53.6% ▅▁▁▁▂▂▃█●▁▃▁ beats 83/107 hard · avg 49% — not counted yet: no other model in the …
source results share of the score
Self-Consistency Improves Chain of Thou… 2 59% ▇▇▇▇▇▇▇
Recitation-Augmented Language Models (s… 2 38% ▇▇▇▇▇
prior: pull toward the middle 3% ▇
weighted average = 28.0
Licences: Self-Consistency Improves Chain of Thought Reasoning in Language Models ([CC BY 4.0](../../licenses/paper-2203.11171v4-CC-BY-4-0.txt)) · Recitation-Augmented Language Models ([CC BY 4.0](../../licenses/paper-2210.01296v2-recitation-CC-BY-4-0.txt))
Not counted (3)
source test score why
Large Language Models… GSM8K 5.4% too few models have it yet
Large Language Models… ARC Challenge 49.9% only model in this source's run
Large Language Models… OpenBookQA 53.6% too few models have it yet