[whatmodel.sh](../../README.md) / Minerva 540B # Minerva 540B Google · released 2022-06 · 540.35B params · proprietary **score 47.6** ±1.3 · comparison only: no current run option verified · from 7 benchmark results across 2 sources · † no independently run results verified Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks Minerva 540B among every model on that task or test, worst to best. 
task  vs the others         rank  counts  tests
math  ▁▁▃▆█●▅█▃▁▁▁   #256 of 530     53%      2
math · #256 of 530 · 53% of its score Beats [MiniMax-M2.1](minimax-m2-1.md), [DeepSeek V3 (Dec '24)](deepseek-v3.md), [GPT-4o (2024-08-06)](gpt-4o-2024-08-06.md) here, though they rank higher overall. Trails [Phi-2 (2.7B)](phi-2.md), [Granite 8B Code Base (4K)](granite-8b-code-base-4k.md), [Mixtral 8x7B v0.1 (base)](mixtral-8x7b-v0-1.md) here, though they rank lower overall. 
test                  result  vs the others in its run  difficulty      counts  why
GSM8K (maker)          58.8%  ▃····▃··▃··● beats 5/5    easy · avg 66%     46%  sharp · at its level · several runs
Minerva MATH (maker)   33.6%  ▄··▄··▄·▄··● beats 5/5    hard · avg 33%    6.3%  at its level
other tests (4) 
test                                result  vs the others in its run   difficulty        counts  why
CommonsenseQA (maker)                72.2%  █·····▂▂▃▂●▄ beats 32/38   easy · avg 83%       17%  sharp · at its level · small run
MMLU-STEM (Minerva subset) (maker)   63.9%  ▄······▄▄·▄● beats 5/5     medium · avg 62%     14%  sharp · at its level
BIG-Bench Hard (maker)               37.2%  ▁▁▁▁▂●▅▆█▃▃▁ beats 28/155  medium · avg 52%     10%  sharp · at its level · small run
OCWCourses (Minerva) (maker)         17.6%  ▄·····▄█··▄● beats 5/5     hard · avg 15%      5.9%  typical
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
Solving Quantitative Reasoning Problems…        4   56% ▇▇▇▇▇▇▇
Transcending Scaling Laws with 0.1% Ext…        3   43% ▇▇▇▇▇
prior: pull toward the middle                        1% ▇
weighted average                                   = 47.6
Licences: Solving Quantitative Reasoning Problems with Language Models ([CC BY 4.0](../../licenses/paper-2206.14858v2-CC-BY-4-0.txt)) ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/minerva-540b) · [JSON](https://whatmodel.sh/api?format=json&all=1)