[whatmodel.sh](../../README.md) / Flan-U-PaLM 540B # Flan-U-PaLM 540B Google · released 2022-10 · 540B params · proprietary **score 51.4** ±1.4 · comparison only: no current run option verified · from 4 benchmark results across 1 source · † no independently run results verified Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks Flan-U-PaLM 540B among every model on that task or test, worst to best.}

task  vs the others          rank  counts  tests
math  ▁▁▃▆██●█▃▁▁▁   #193 of 530*     31%      1

math · #193 of 530 · 31% of its score

Beats [MiMo-V2.5 (310B-A15B)](mimo-v2-5.md), [MiniMax-M2.1](minimax-m2-1.md), [Mistral Medium 3.5 (128B)](mistral-medium-3-5.md) here, though they rank higher overall. Trails [OpenThinker3 7B](openthinker3-7b.md) here, though they rank lower overall.

test          result  vs the others in its run  difficulty        counts  why
MGSM (maker)   60.4%  ▆·▂·▄▂▂▂▂█▄● beats 18/18  medium · avg 48%     31%  sharp · at its level

other tests (3)

test                                result  vs the others in its run  difficulty        counts  why
BIG-Bench Hard (maker)               59.3%  ▂▂▂▂·▅·▃▂█▃● beats 18/18  medium · avg 60%     30%  sharp · at its level
MMLU (maker)                         74.1%  ▁···▁··▃▁▅▅● beats 18/18  easy · avg 74%       28%  at its level
TyDiQA-GoldP (1-shot, EM, 8-langu…   68.3%  ▂··▂·▂▄▄·██● beats 18/18  medium · avg 61%     10%  at its level

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
Scaling Instruction-Finetuned Language …        4   99% ▇▇▇▇▇▇▇▇▇▇▇▇
prior: pull toward the middle                        1% ▇
weighted average                                   = 51.4

_{Licences: Scaling Instruction-Finetuned Language Models ([CC BY 4.0](../../licenses/paper-2210.11416v5-CC-BY-4.0.txt))} ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/flan-u-palm-540b) · [JSON](https://whatmodel.sh/api?format=json&all=1)