[whatmodel.sh](../../README.md) / LaMDA 137B (pre-trained, LaMDA-PT) # LaMDA 137B (pre-trained, LaMDA-PT) Google · released 2022-01 · 137B params · proprietary **score 33.8** ±0.6 · comparison only: no current run option verified · from 20 benchmark results across 3 sources · † no independently run results verified Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks LaMDA 137B (pre-trained, LaMDA-PT) among every model on that task or test, worst to best. 
task    vs the others          rank  counts  tests
coding  ●·█······█·█       #4 of 4*    1.9%      1
math    ▁▁▃●██▅█▃▁▁▁   #449 of 530*     13%      1
coding · #4 of 4 · 1.9% of its score 
test               result  vs the others in its run   difficulty        counts  why
HumanEval (maker)   14.0%  ▁▁▁▁●▂▆▆█▅▂▁ beats 24/205  medium · avg 46%    1.9%  sharp
math · #449 of 530 · 13% of its score Beats [GPT-3.5 Turbo (0125)](gpt-3-5-turbo-0125.md), [DBRX Instruct](dbrx-instruct.md), [Command R Plus (04-2024)](command-r-plus-04-2024.md) here, though they rank higher overall. Trails [OPT-1.3B](opt-1-3b.md), [OPT-2.7B](opt-2-7b.md), [Gemma 2B (base)](gemma-2b-base.md) here, though they rank lower overall. 
test           result  vs the others in its run  difficulty      counts  why
GSM8K (maker)   14.3%  █·▅▃▃▃·●·▃▃▅ beats 8/12   easy · avg 67%     13%  sharp · several runs
other tests (12) 
test                                result  vs the others in its run   difficulty           counts  why
MAWPS (CoT-paper union) (maker)      57.9%  ▅▃█···▅·●▃▃▅ beats 8/12    very easy · avg 94%     23%  sharp · at its level
SVAMP (maker)                        37.5%  ▃▅█···▅·●▃▃▅ beats 8/12    easy · avg 82%          18%  sharp · at its level · several runs
ASDiv (maker)                        46.6%  ▅▃▃▅··▃▃·●▃█ beats 8/12    easy · avg 83%          15%  sharp · at its level · several runs
CommonsenseQA (maker)                53.6%  █······▁·●▁▃ beats 8/12    easy · avg 88%         9.3%  sharp · at its level · several runs
AQuA-RAT (maker)                     20.6%  █●·▁▁·····▂▁ beats 7/12    medium · avg 48%       4.8%  sharp · too hard for it · several runs
ARC Challenge (maker)                55.1%  ▄▁▁▂▁▄█●▃▂▃▂ beats 98/140  easy · avg 86%         3.6%  sharp · at its level
BIG-bench Sports Understanding (m…   59.5%  █·····▃·●·▄▁ beats 8/12    medium · avg 81%       3.5%  sharp
BIG-bench Date Understanding (mak…   21.5%  ▃·▅·▅▃▅·●▃·█ beats 8/12    medium · avg 52%       3.0%  at its level
StrategyQA (maker)                   62.4%  █·····▅▃▃▅●█ beats 9/12    medium · avg 70%       2.0%  several runs
AddSub (maker)                       52.9%  ●·········██ beats 0/2     very easy · avg 89%    1.4%  at its level · small run
MultiArith (maker)                   51.8%  ●·········██ beats 0/2     very easy · avg 93%    1.2%  sharp · at its level · noisy · small run
ARC Easy (maker)                     75.3%  ▁▃▂▄▆█●▃▁··▁ beats 78/100  easy · avg 87%         0.3%  blunt · at its level
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
Chain-of-Thought Prompting Elicits Reas…        9   77% ▇▇▇▇▇▇▇▇▇
Self-Consistency Improves Chain of Thou…       10   21% ▇▇▇
PaLM: Scaling Language Modeling with Pa…        1    2% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 33.8
Licences: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models ([CC BY 4.0](../../licenses/paper-2201.11903v6-CC-BY-4-0.txt)) · Self-Consistency Improves Chain of Thought Reasoning in Language Models ([CC BY 4.0](../../licenses/paper-2203.11171v4-CC-BY-4-0.txt)) · PaLM: Scaling Language Modeling with Pathways ([CC BY 4.0](../../licenses/paper-2204.02311v5-CC-BY-4-0.txt)) ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/lamda-137b) · [JSON](https://whatmodel.sh/api?format=json&all=1)