[whatmodel.sh](../../README.md) / PaLM 62B
# PaLM 62B
Google · released 2022-04 · 62.5B params · proprietary
**score 39.3** ±0.5 · comparison only: no current run option verified · from 74 benchmark results across 10 sources · † no independently run results verified
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks PaLM 62B among every model on that task or test, worst to best.
task vs the others rank counts tests
coding █·●······█·█ #3 of 4* 2.5% 2
math ▁▁▃▆●█▅█▃▁▁▁ #378 of 530 18% 3
coding · #3 of 4 · 2.5% of its score
test result vs the others in its run difficulty counts why
HumanEval (maker) 15.9% ▁▁▁▁●▂▆▆█▅▂▁ beats 27/205 medium · avg 46% 1.7% sharp
MBPP (maker) 21.4% ▁●▁▄▅█▆▅▄▁▂▁ beats 3/86 medium · avg 50% 0.9% at its level · small run
math · #378 of 530 · 18% of its score
Beats [Molmo 2 8B](molmo-2-8b.md), [GPT-4 (0613)](gpt-4-0613.md), [Athene-70B](athene-70b.md) here, though they rank higher overall. Trails [DeepSeek R1 Distill Qwen 1.5B](deepseek-r1-distill-qwen-1-5b.md), [Tulu 2 DPO 7B](tulu-2-dpo-7b.md), [Gemma 7B (base)](gemma-7b-base.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
GSM8K (maker) 29.9% █·▅▃▃▃·▃·●▃▅ beats 9/12 easy · avg 67% 15% sharp · at its level · several runs
MGSM (maker) 18.2% ▆·▂·▄▂▂●▂█▄▆ beats 8/18 medium · avg 48% 2.1% sharp
Minerva MATH (maker) 4.4% ▄··●··▄·▄··█ beats 1/5 hard · avg 33% 0.6% typical
other tests (46)
test result vs the others in its run difficulty counts why
MAWPS (CoT-paper union) (maker) 80.3% ▅▃█···▅·▃●▃▅ beats 9/12 very easy · avg 94% 11% sharp
SVAMP (maker) 46.7% ▃▅█···▅·▃●▃▅ beats 9/12 easy · avg 82% 9.3% sharp · at its level
ASDiv (maker) 61.9% ▅▃▃▅··▃▃·▃●█ beats 9/12 easy · avg 83% 7.2% sharp · at its level
CommonsenseQA (maker) 65.4% █······▁·▁●▃ beats 9/12 easy · avg 88% 5.9% sharp · at its level · several runs
AQuA-RAT (maker) 22.4% █▁·▁●·····▂▁ beats 9/12 medium · avg 48% 4.5% sharp · too hard for it
MMLU (maker) 55.1% ▁···▁··▃▁●▅█ beats 7/18 easy · avg 74% 4.4% at its level · several runs
WinoGrande (maker) 76.6% ▄····▄··▄·█● beats 5/6 easy · avg 85% 3.3% at its level · several runs
BIG-bench Sports Understanding (m… 72.1% █·····▃·▁·●▁ beats 11/12 medium · avg 81% 2.8% sharp · at its level
TriviaQA (maker) 67.3% ▁▁▂▂▄▄▃▆▅●▅▂ beats 64/80 easy · avg 80% 2.6% at its level · several runs
BIG-Bench Hard (maker) 37.4% ▂▂▂▂·●·▃▂█▃▃ beats 5/18 medium · avg 60% 2.1% sharp
BoolQ (maker) 84.8% ▁····▁▂█▆▅●▄ beats 90/103 easy · avg 90% 2.0% at its level · several runs
BIG-bench Date Understanding (mak… 29.8% ▃·▅·▅▃▅·▃●·█ beats 9/12 medium · avg 52% 2.0% at its level
Natural Questions (closed book) (… 18.1% ▂▁▁▁▂▄▄▃●█▅▁ beats 40/68 hard · avg 32% 2.0% several runs
ARC Challenge (maker) 52.5% ▄▁▁▂▁▄█●▃▂▃▂ beats 91/140 medium · avg 66% 1.6% sharp · at its level · small run · seve…
HellaSwag (maker) 79.7% ▁▁▁▂▂▂▄▆█●▄▂ beats 109/135 easy · avg 87% 1.5% at its level · several runs
MMLU-STEM (Minerva subset) (maker) 39.1% ▄······▄●·▄█ beats 2/5 medium · avg 62% 1.4% sharp
MedQA USMLE 4-option (few-shot pr… 40.9% ▄······▄●▄·█ beats 2/5 medium · avg 58% 1.3% typical
ANLI Round 1 (prompted) (maker) 36.4% █▁▃·▃·●·▃··▁ beats 13/16 hard · avg 51% 1.2% sharp · small run
MedMCQA (few-shot prompted) (make… 43.4% ▄·····▄··●·█ beats 2/5 medium · avg 56% 1.1% typical
TyDiQA-GoldP (1-shot, EM, 8-langu… 40.5% ▂··▂·▂▄▄·●██ beats 7/18 medium · avg 61% 1.1% at its level
StrategyQA (maker) 58.4% █·····▅▃▃●▃█ beats 7/12 medium · avg 70% 1.0% several runs
COPA (maker) 93.0% ▁·▁▁▁▅▅█▆●▁▁ beats 61/67 very easy · avg 96% 0.9% small run · several runs
PIQA (maker) 80.5% ▁▁▁▂▂▃▅█●▃▁▁ beats 97/123 easy · avg 85% 0.7% blunt · at its level · several runs
WiC (prompted) (maker) 49.4% ●·▂▁······▁▁ beats 5/13 hard · avg 60% 0.7% sharp · too hard for it · noisy · small…
ANLI Round 2 (prompted) (maker) 37.2% █▅▃▅▃▃▅▃●··▃ beats 13/16 hard · avg 45% 0.7% small run
LAMBADA (OpenAI version), last-wo… 75.4% ▁·····▁▁▃▃█● beats 77/86 easy · avg 84% 0.7% blunt · several runs
MultiRC (SuperGLUE, prompted, F1a… 72.1% ▃·▃██···●··▃ beats 8/10 very easy · avg 85% 0.6% at its level · small run
ARC Easy (maker) 75.2% ▁▃▂▄▆█●▃▁··▁ beats 84/100 easy · avg 85% 0.6% blunt · at its level · small run · seve…
RTE (prompted) (maker) 67.9% ▅▃·▃··▅·▅▅▃● beats 12/13 medium · avg 75% 0.6% at its level · small run
Story Cloze Test (maker) 83.5% ▃▃▃▂▅█●▂▁··▂ beats 42/50 easy · avg 89% 0.6% blunt · at its level · small run · seve…
OCWCourses (Minerva) (maker) 5.9% ▄·····●█··▄▄ beats 1/5 hard · avg 15% 0.5% too hard for it
ARCADE New Tasks pass@1 (notebook… 7.2% ●·█········█ beats 0/2 hard · avg 23% 0.5% small run
SQuAD 2.0 (prompted, F1) (maker) 75.6% ▃▃···▃▃█▃▃●▃ beats 9/10 very easy · avg 85% 0.4% at its level · small run
PubMedQA (few-shot prompted) (mak… 57.8% ▄·······▄●▄█ beats 2/5 medium · avg 75% 0.4% at its level · noisy
CoQA (maker) 75.7% ▂···▃▂▂·▆▅▂● beats 14/17 very easy · avg 88% 0.3% small run
Winograd Schema Challenge (WSC273… 87.5% █▃·▅▃▃█▅██●▃ beats 20/21 easy · avg 92% 0.3% blunt · small run
SuperGLUE WSC (prompted) (maker) 86.3% █··▁·▃·▄▂▁▅● beats 25/26 medium · avg 82% 0.3% blunt · at its level · small run
ANLI Round 3 (prompted) (maker) 36.7% ▂·▆▂██·●··▂▂ beats 13/16 hard · avg 44% 0.3% small run
OpenBookQA (maker) 50.4% ▅▁▁▁▂▂▃●▃▁▃▁ beats 74/107 medium · avg 59% 0.3% noisy · small run
RACE-middle (maker) 64.3% ▄▄····▄█·█●█ beats 8/10 easy · avg 74% 0.2% blunt · at its level · small run
ReCoRD (SuperGLUE, prompted) (mak… 91.4% ▄·▄···▄███●▄ beats 9/10 very easy · avg 95% 0.2% blunt · maxed out · small run
QuAC (prompted, F1) (maker) 40.0% ▅··▃▃█··▃●·▃ beats 8/10 medium · avg 53% 0.2% blunt · at its level · small run
DROP (maker) 57.9% ▃▅█▅·▃·●▄▂▁▃ beats 29/44 medium · avg 62% 0.2% blunt · at its level · small run
RACE-high (maker) 47.5% ▂▄·▂▂▂▂▂▆▄▂● beats 16/17 medium · avg 53% 0.2% blunt · at its level · small run
CommitmentBank (prompted) (maker) 57.1% ▂▃·▂█▂▂▂·●·▂ beats 12/13 medium · avg 60% 0.1% blunt · at its level · small run
WebQuestions (closed book) (maker) 11.5% ▅▅▃▅▃·▃▃▅█●▃ beats 15/17 very hard · avg 4.1% 0.0% blunt · too hard for it · small run
source results share of the score
Chain-of-Thought Prompting Elicits Reas… 9 49% ▇▇▇▇▇▇
PaLM: Scaling Language Modeling with Pa… 33 19% ▇▇
Scaling Instruction-Finetuned Language … 4 9% ▇
Solving Quantitative Reasoning Problems… 4 6% ▇
Transcending Scaling Laws with 0.1% Ext… 7 5% ▇
Universal Self-Adaptive Prompting (subm… 9 4% ▇
Better Zero-Shot Reasoning with Self-Ad… 3 3% ▇
Large Language Models Encode Clinical K… 3 3% ▇
WinoDict: Probing language models for i… 1 1% ▇
Natural Language to Code Generation in … 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 39.3
Licences: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models ([CC BY 4.0](../../licenses/paper-2201.11903v6-CC-BY-4-0.txt)) · PaLM: Scaling Language Modeling with Pathways ([CC BY 4.0](../../licenses/paper-2204.02311v5-CC-BY-4-0.txt)) · Scaling Instruction-Finetuned Language Models ([CC BY 4.0](../../licenses/paper-2210.11416v5-CC-BY-4.0.txt)) · Solving Quantitative Reasoning Problems with Language Models ([CC BY 4.0](../../licenses/paper-2206.14858v2-CC-BY-4-0.txt)) · Universal Self-Adaptive Prompting ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-461-CC-BY-4-0.txt)) · Better Zero-Shot Reasoning with Self-Adaptive Prompting ([CC BY 4.0](../../licenses/acl-2023-findings-acl-216-CC-BY-4-0.txt)) · Large Language Models Encode Clinical Knowledge ([CC BY 4.0](../../licenses/paper-2212.13138v1-CC-BY-4-0.txt)) · WinoDict: Probing language models for in-context word acquisition ([CC BY 4.0](../../licenses/acl-2023-eacl-main-7-CC-BY-4-0.txt)) · Natural Language to Code Generation in Interactive Data Science Notebooks ([CC BY 4.0](../../licenses/acl-2023-acl-long-9-CC-BY-4-0.txt))
## How to run it
No verified API or local download yet.
[interactive version](https://whatmodel.sh/model/palm-62b) · [JSON](https://whatmodel.sh/api?format=json&all=1)