[whatmodel.sh](../../README.md) / T5 Base # T5 Base Google · released 2019-10 · 0.223B params · apache-2.0 **score 14.0** ±2.7 · #712 of 732 · from 8 benchmark results across 2 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks T5 Base among every model on that task or test, worst to best.}

task  vs the others          rank  counts  tests
math  ▁●▃▅█▆▄█▄▂▁▁   #420 of 431*     15%      1

math · #420 of 431 · 15% of its score

Beats [Flan-T5 Large](flan-t5-large.md), [OPT-66B](opt-66b.md), [Flan-T5 Base](flan-t5-base.md) here, though they rank higher overall. Trails [GPT-Neo 125M](gpt-neo-125m.md), [OPT-125M](opt-125m.md), [GPT-2 XL (1.5B)](gpt-2-xl.md) here, though they rank lower overall.

test   result  vs the others in its run  difficulty        counts  why
GSM8K    1.7%  ▅·▂●·······▂ beats 4/9    medium · avg 62%     15%  sharp · too hard for it · noisy

other tests (8)

test                                result  vs the others in its run  difficulty           counts  why
COPEN concept identification in c…   42.3%  ▃···▃▃·▃·▃·● beats 6/7    easy · avg 81%          36%  typical
CommonsenseQA                        13.4%  ●····▁·····▁ beats 4/9    easy · avg 75%          21%  sharp · too hard for it
SVAMP                                 0.3%  ●·▆▂▄▂▂····▂ beats 3/12   easy · avg 70%          16%  sharp · too hard for it
StrategyQA                           52.5%  █········●▃▁ beats 6/9    hard · avg 62%         4.7%  too hard for it · noisy
COPEN conceptual similarity judgm…   10.3%  ▄···█···●█·▄ beats 3/7    hard · avg 22%         3.1%  blunt · too hard for it
COPEN conceptual property judgmen…   49.5%  ●·······▁··▁ beats 1/7    very hard · avg 52%    0.1%  blunt · too hard for it · noisy
AQuA-RAT                             23.2%  █·····●·▁··▁ beats 7/9    medium · avg 53%       0.1%  sharp · too hard for it · noisy
Open PT-LLM Leaderboard average (…   13.7%  ▃▁▁▁▁▁▁▁▂█▅▁ beats 0/276  medium · avg 72%          —  failed run: Failed run: average 13.68 v…

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
Large Language Models Are Reasoning Tea…        5   57% ▇▇▇▇▇▇▇
COPEN                                           3   40% ▇▇▇▇▇
prior: pull toward the middle                        3% ▇
weighted average                                   = 14.0

_{Licences: Large Language Models Are Reasoning Teachers ([CC BY 4.0](../../licenses/acl-2023-acl-long-830-CC-BY-4-0.txt)) · COPEN ([CC BY 4.0](../../licenses/copen-emnlp2022-zero-shot-CC-BY-4.0.txt))}

Not counted (1)

source                  test                            score  why
Open Portuguese LLM L…  Open PT-LLM Leaderboard avera…  13.7%  failed run: Failed run: average 13.68 vs the leaderboard's random baseline 32.64; the original T5 encoder-decoder checkpoint does not follow the few-shot generative answer format (every task near or below chance), so this is not a measurement of ability.

## How to run it ### Other ways to run it 1. Run it as published (fp32, needs ~0.89 GB, from [huggingface.co/google-t5](https://huggingface.co/google-t5/t5-base/tree/a9723ea7f1b39c1eae772870f3b547bf6ef7e6c1)) ```sh uv run --with torch --with 'transformers<5' python -c "from transformers import pipeline; print(pipeline('text2text-generation', model='google-t5/t5-base')('translate English to German: The house is wonderful.', max_new_tokens=40)[0]['generated_text'])" ``` Model repository and documentation: [huggingface.co/google-t5](https://huggingface.co/google-t5/t5-base) [interactive version](https://whatmodel.sh/model/t5-base) · [JSON](https://whatmodel.sh/api?format=json&all=1)