[whatmodel.sh](../../README.md) / MPT-7B (base) # MPT-7B (base) MosaicML · released 2023-05 · 6.7B params · apache-2.0 **score 30.2** ±0.7 · comparison only: no current run option verified · from 34 benchmark results across 6 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks MPT-7B (base) among every model on that task or test, worst to best.}

other tests (15)

test                                result  vs the others in its run  difficulty           counts  why
ARC Challenge                        41.9%  ▁·▁··▁▁▇▂●▆▃ beats 16/26  easy · avg 82%          39%  sharp · several runs
WinoGrande                           68.0%  ▁····▁··▁▂●▄ beats 19/26  easy · avg 86%          12%  at its level · several runs
HellaSwag                            76.2%  ▃···▁▇▅█●▃▁▁ beats 20/26  very easy · avg 91%     10%  at its level · several runs
BoolQ                                73.9%  ▂▃▂▅▆▂▅▆●▂▂▂ beats 19/26  easy · avg 88%         7.8%  at its level · several runs
OpenBookQA                           31.4%  █·▁▁▃·▁▃▅●▃▄ beats 18/26  easy · avg 81%         6.3%  at its level · several runs
ARC Easy                             70.0%  ▁▁·▁▁▄▅▄●▅▁▃ beats 19/26  easy · avg 85%         5.8%  blunt · at its level · several runs
MMLU                                 38.3%  ▄···●▄·█···▄ beats 1/5    easy · avg 79%         4.7%  typical
PIQA                                 78.9%  ▁·▁··▁▁▃█▅●▂ beats 21/26  easy · avg 87%         3.9%  blunt · at its level · several runs
SciQ                                 93.7%  ▁·▁·▁·▃▅▃●▄▁ beats 18/26  very easy · avg 97%    3.7%  blunt · several runs
Social IQA                           45.1%  ▁···▁▃█▁●▁·▁ beats 23/26  medium · avg 60%       2.3%  blunt · several runs
Factual knowledge recall, Premium…    3.5%  ●····▃▅·██·█ beats 1/13   medium · avg 52%       2.0%  typical
Factual knowledge recall, Premium…   30.4%  ▃▅▃··●▃·█▃▃▅ beats 4/13   medium · avg 61%       1.4%  at its level
LAMBADA (OpenAI version), last-wo…   68.6%  ▁··▂▁▁▁▂●█▅▃ beats 12/26  very easy · avg 86%    1.2%  blunt · at its level
TruthfulQA MC2                       33.5%  ●·█·█▄···▄·█ beats 1/9    medium · avg 56%       0.3%  blunt · at its level · several runs
COPA                                 65.6%  ▅·▃·●··█▃··▃ beats 3/8    easy · avg 92%         0.3%  at its level · noisy

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
Stability AI                                   11   46% ▇▇▇▇▇▇
OLMo: Accelerating the Science of Langu…        8   21% ▇▇▇
Textbooks Are All You Need II: phi-1.5 …        8   17% ▇▇
Enhancing Chat Language Models by Scali…        4   12% ▇
Towards a Holistic Evaluation of LLMs o…        2    3% ▇
From Test-Taking to Test-Making: Examin…        1    0% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 30.2

_{Licences: Stability AI ([Apache-2.0](../../licenses/stablelm-original-eval-json-Apache-2.0.txt)) · OLMo: Accelerating the Science of Language Models ([CC BY 4.0](../../licenses/paper-2402.00838v4-CC-BY-4.0.txt)) · Textbooks Are All You Need II: phi-1.5 technical report ([CC BY 4.0](../../licenses/paper-2309.05463v1-CC-BY-4.0.txt)) · Enhancing Chat Language Models by Scaling High-quality Instructional Conversations ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-183-CC-BY-4-0.txt)) · Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · From Test-Taking to Test-Making: Examining LLM Authoring of Commonsense Assessment Items ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/))} ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/mpt-7b) · [JSON](https://whatmodel.sh/api?format=json&all=1)