[whatmodel.sh](../../README.md) / Tk-Instruct 11B (def) # Tk-Instruct 11B (def) AllenAI / UW · released 2022-05 · 11B params · apache-2.0 **score 26.4** ±4.1 · comparison only: no current run option verified · from 3 benchmark results across 2 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks Tk-Instruct 11B (def) among every model on that task or test, worst to best.
other tests (3) 
test                                result  vs the others in its run  difficulty        counts  why
BIG-Bench Hard                       11.8%  ●▁▁▁▂▄▅▆█▃▃▁ beats 0/155  medium · avg 58%     45%  sharp · too hard for it · small run
Super-NaturalInstructions English…   45.6%  ▄·▄······▄▄● beats 5/5    easy · avg 77%       35%  at its level · small run
LMentry accuracy (zero-shot)         48.0%  ▂▆·▄▄●▂·▂··▂ beats 9/14   medium · avg 65%     11%  blunt · at its level
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
Unnatural Instructions: Tuning Language…        2   81% ▇▇▇▇▇▇▇▇▇▇
LMentry: A Language Model Benchmark of …        1   11% ▇
prior: pull toward the middle                        8% ▇
weighted average                                   = 26.4
Licences: Unnatural Instructions: Tuning Language Models with ([CC BY 4.0](../../licenses/acl-2023-acl-long-806-CC-BY-4-0.txt)) · LMentry: A Language Model Benchmark of Elementary Language Tasks ([CC BY 4.0](../../licenses/acl-2023-findings-acl-666-CC-BY-4-0.txt)) ## How to run it No verified API or local download yet. Model repository and documentation: [huggingface.co/allenai](https://huggingface.co/allenai/tk-instruct-11b-def) [interactive version](https://whatmodel.sh/model/tk-instruct-11b-def) · [JSON](https://whatmodel.sh/api?format=json&all=1)