[whatmodel.sh](../../README.md) / T5 Small # T5 Small Google · released 2019-10 · 0.061B params · apache-2.0 **score 11.8** ±2.9 · #716 of 732 · from 8 benchmark results across 2 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks T5 Small among every model on that task or test, worst to best. 
task  vs the others          rank  counts  tests
math  ▁●▃▅█▆▄█▄▂▁▁   #419 of 431*     13%      1
math · #419 of 431 · 13% of its score Beats [Flan-T5 Large](flan-t5-large.md), [OPT-66B](opt-66b.md), [Flan-T5 Base](flan-t5-base.md) here, though they rank higher overall. Trails [GPT-Neo 125M](gpt-neo-125m.md), [OPT-125M](opt-125m.md), [GPT-2 XL (1.5B)](gpt-2-xl.md) here, though they rank lower overall. 
test   result  vs the others in its run  difficulty        counts  why
GSM8K    2.0%  ▅·▂●·······▂ beats 6/9    medium · avg 62%     13%  sharp · too hard for it · noisy
other tests (8) 
test                                result  vs the others in its run  difficulty           counts  why
COPEN concept identification in c…   40.1%  ▃···▃▃·▃·▃·● beats 5/7    easy · avg 81%          41%  typical
CommonsenseQA                        19.3%  ●····▁·····▁ beats 5/9    easy · avg 75%          18%  sharp · too hard for it
SVAMP                                 1.3%  █·●▂▄▂▂····▂ beats 4/12   easy · avg 70%          15%  sharp · too hard for it
StrategyQA                           46.0%  ●········▁▃▁ beats 3/9    hard · avg 62%         5.0%  too hard for it · noisy
COPEN conceptual similarity judgm…    6.5%  ●···█···██·▄ beats 0/7    hard · avg 22%         3.6%  blunt · too hard for it
COPEN conceptual property judgmen…   53.2%  █·······▁··● beats 7/7    very hard · avg 52%    0.2%  blunt · too hard for it · noisy
AQuA-RAT                             24.8%  █·····▁·●··▁ beats 8/9    medium · avg 53%       0.1%  sharp · too hard for it · noisy
Open PT-LLM Leaderboard average (…   12.6%  ▃▁▁▁▁▁▁▁▂█▅▁ beats 0/276  medium · avg 72%          —  failed run: Failed run: average 12.63 v…
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
Large Language Models Are Reasoning Tea…        5   52% ▇▇▇▇▇▇
COPEN                                           3   44% ▇▇▇▇▇
prior: pull toward the middle                        4% ▇
weighted average                                   = 11.8
Licences: Large Language Models Are Reasoning Teachers ([CC BY 4.0](../../licenses/acl-2023-acl-long-830-CC-BY-4-0.txt)) · COPEN ([CC BY 4.0](../../licenses/copen-emnlp2022-zero-shot-CC-BY-4.0.txt))
Not counted (1) 
source                  test                            score  why
Open Portuguese LLM L…  Open PT-LLM Leaderboard avera…  12.6%  failed run: Failed run: average 12.63 vs the leaderboard's random baseline 32.64; the original T5 encoder-decoder checkpoint does not follow the few-shot generative answer format (every task near or below chance), so this is not a measurement of ability.
## How to run it ### Other ways to run it 1. Run it as published (fp32, needs ~0.24 GB, from [huggingface.co/google-t5](https://huggingface.co/google-t5/t5-small/tree/df1b051c49625cf57a3d0d8d3863ed4d13564fe4)) ```sh uv run --with torch --with 'transformers<5' python -c "from transformers import pipeline; print(pipeline('text2text-generation', model='google-t5/t5-small')('translate English to German: The house is wonderful.', max_new_tokens=40)[0]['generated_text'])" ``` Model repository and documentation: [huggingface.co/google-t5](https://huggingface.co/google-t5/t5-small) [interactive version](https://whatmodel.sh/model/t5-small) · [JSON](https://whatmodel.sh/api?format=json&all=1)