[whatmodel.sh](../../README.md) / Flan-PaLM 62B # Flan-PaLM 62B Google · released 2022-10 · 62.5B params · proprietary **score 44.4** ±1.2 · comparison only: no current run option verified · from 7 benchmark results across 2 sources · † no independently run results verified Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks Flan-PaLM 62B among every model on that task or test, worst to best.}

task  vs the others          rank  counts  tests
math  ▁▁▃▆●█▅█▃▁▁▁   #320 of 530*     21%      1

math · #320 of 530 · 21% of its score

Beats [GPT-4o (2024-11-20)](gpt-4o-2024-11-20.md), [GPT-4 Turbo (2024-04-09)](gpt-4-turbo-2024-04-09.md), [Llama 3.1 405B Instruct](llama-3-1-405b.md) here, though they rank higher overall. Trails [DeepSeek R1 Distill Qwen 1.5B](deepseek-r1-distill-qwen-1-5b.md), [Gemma 7B (base)](gemma-7b-base.md), [CodeGemma 7B (v1.0)](codegemma-7b.md) here, though they rank lower overall.

test          result  vs the others in its run  difficulty        counts  why
MGSM (maker)   28.5%  ▆·▂·▄▂▂▂▂●▄▆ beats 10/18  medium · avg 48%     21%  sharp · at its level

other tests (6)

test                                result  vs the others in its run  difficulty        counts  why
MMLU (maker)                         59.6%  ▁···▁··▃▁▅●█ beats 9/18   easy · avg 74%       25%  at its level
BIG-Bench Hard (maker)               47.5%  ▂▂▂▂·▅·▃▂●▃▃ beats 10/18  medium · avg 60%     20%  sharp · at its level
MedQA USMLE 4-option (few-shot pr…   46.1%  ▄······▄▄●·█ beats 3/5    medium · avg 58%     11%  at its level
MedMCQA (few-shot prompted) (make…   46.2%  ▄·····▄··●·█ beats 3/5    medium · avg 56%     10%  at its level
TyDiQA-GoldP (1-shot, EM, 8-langu…   58.7%  ▂··▂·▂▄▄·██● beats 15/18  medium · avg 61%    8.7%  at its level
PubMedQA (few-shot prompted) (mak…   77.2%  ▄·······▄▄▄● beats 4/5    medium · avg 75%    3.2%  at its level · noisy

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
Scaling Instruction-Finetuned Language …        4   75% ▇▇▇▇▇▇▇▇▇
Large Language Models Encode Clinical K…        3   24% ▇▇▇
prior: pull toward the middle                        1% ▇
weighted average                                   = 44.4

_{Licences: Scaling Instruction-Finetuned Language Models ([CC BY 4.0](../../licenses/paper-2210.11416v5-CC-BY-4.0.txt)) · Large Language Models Encode Clinical Knowledge ([CC BY 4.0](../../licenses/paper-2212.13138v1-CC-BY-4-0.txt))} ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/flan-palm-62b) · [JSON](https://whatmodel.sh/api?format=json&all=1)