[whatmodel.sh](../../README.md) / Flan-UL2 20B # Flan-UL2 20B Google · released 2023-03 · 20B params · apache-2.0 **score 47.7** ±3.5 · comparison only: no current run option verified · from 5 benchmark results across 2 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks Flan-UL2 20B among every model on that task or test, worst to best.
other tests (5) 
test                                result  vs the others in its run  difficulty           counts  why
ZeroSCROLLS Qasper (zero-shot F1)    56.9%  ▂▂··▂·▂··▂█● beats 10/10  medium · avg 56%        33%  blunt · at its level
ZeroSCROLLS QuALITY (zero-shot ac…   75.6%  ▃····▃·▃▃█●▅ beats 8/10   easy · avg 81%          30%  at its level
ZeroSCROLLS NarrativeQA (zero-sho…   25.5%  ▃··▃·█▃·●▃·▃ beats 8/10   hard · avg 29%          18%  blunt · at its level
FANToM BeliefQ [Choice], short co…   23.0%  ●·····▂····▁ beats 6/11   very hard · avg 55%    9.4%  sharp · too hard for it · noisy
ZeroSCROLLS MuSiQue (zero-shot F1)   51.3%  ▄·█··██·▄▄▄● beats 10/10  medium · avg 43%       4.9%  blunt · at its level
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
ZeroSCROLLS: A Zero-Shot Benchmark for …        4   85% ▇▇▇▇▇▇▇▇▇▇
FANToM: A Benchmark for Stress-testing …        1    9% ▇
prior: pull toward the middle                        5% ▇
weighted average                                   = 47.7
Licences: ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-536-CC-BY-4-0.txt)) · FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-890-CC-BY-4-0.txt)) ## How to run it No verified API or local download yet. Model repository and documentation: [huggingface.co/google](https://huggingface.co/google/flan-ul2) [interactive version](https://whatmodel.sh/model/flan-ul2) · [JSON](https://whatmodel.sh/api?format=json&all=1)