[whatmodel.sh](../../README.md) / T0 11B
# T0 11B
BigScience · released 2021-10 · 11B params · apache-2.0
**score 34.5** ±2.5 · comparison only: no current run option verified · from 9 benchmark results across 6 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks T0 11B among every model on that task or test, worst to best.
other tests (11)
test result vs the others in its run difficulty counts why
WebQSP closed-book zero-shot (ans… 40.8% ██···██····● beats 4/4 easy · avg 66% 25% at its level
WinoGrande (maker) 60.5% █··▁▃·▁··▁·● beats 12/12 medium · avg 68% 19% noisy
Super-NaturalInstructions English… 32.3% ●····█··█··█ beats 0/3 easy · avg 71% 13% at its level
BIG-Bench Hard, 27 subsets, zero-… 33.6% █····●··█··█ beats 1/3 medium · avg 56% 13% blunt · at its level
Mintaka (English) closed-book zer… 23.7% ▄····▄···●·█ beats 2/4 medium · avg 48% 8.7% blunt · at its level
Super-NaturalInstructions English… 33.1% ▄·▄······●▄█ beats 2/5 medium · avg 63% 8.5% at its level · noisy · small run
COPA (maker) 93.0% ▃··█▅▃▃▅·▃▃● beats 12/12 easy · avg 91% 5.2% at its level · noisy
LMentry accuracy (zero-shot) 40.9% ▂▆·▄●█▂·▂··▂ beats 6/14 medium · avg 65% 4.3% blunt · at its level
Story Cloze Test (maker) 94.7% █·▂··▃··▂▂▂● beats 11/12 easy · avg 88% 1.2% blunt · at its level · noisy
HellaSwag 63.7% ▁▁▁▂▂▂▄●█▇▄▂ beats 51/135 very easy · avg 91% — not counted yet: no other model in the …
PIQA 84.9% ▁▁▁▂▂▃▅█▅▃●▁ beats 120/123 easy · avg 87% — not counted yet: no other model in the …
source results share of the score
Knowledge-Augmented Language Model Prom… 2 34% ▇▇▇▇
Crosslingual Generalization through Mul… 3 25% ▇▇▇
Super-NaturalInstructions: Generalizati… 1 13% ▇▇
The CoT Collection: Improving Zero-shot… 1 13% ▇▇
Self-Instruct: Aligning Language Models… 1 8% ▇
LMentry: A Language Model Benchmark of … 1 4% ▇
prior: pull toward the middle 3% ▇
weighted average = 34.5
Licences: Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering ([CC BY 4.0](../../licenses/acl-2023-nlrse-1-7-CC-BY-4-0.txt)) · Crosslingual Generalization through Multitask Finetuning ([CC BY 4.0](../../licenses/acl-2023-acl-long-891-CC-BY-4-0.txt)) · Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks ([CC BY 4.0](../../licenses/acl-2022-emnlp-main-340-CC-BY-4-0.txt)) · The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-782-CC-BY-4-0.txt)) · Self-Instruct: Aligning Language Models with Self-Generated Instructions ([CC BY 4.0](../../licenses/acl-2023-acl-long-754-CC-BY-4-0.txt)) · LMentry: A Language Model Benchmark of Elementary Language Tasks ([CC BY 4.0](../../licenses/acl-2023-findings-acl-666-CC-BY-4-0.txt))
Not counted (3)
source test score why
NovaCOMET: Open Commo… HellaSwag 63.7% too few models have it yet
NovaCOMET: Open Commo… WinoGrande 58.9% only model in this source's run
NovaCOMET: Open Commo… PIQA 84.9% too few models have it yet