[whatmodel.sh](../../README.md) / Alpaca 7B # Alpaca 7B Stanford CRFM · released 2023-03 · 7B params · cc-by-nc-4.0 **score 30.5** ±1.4 · comparison only: no current run option verified · from 10 benchmark results across 5 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks Alpaca 7B among every model on that task or test, worst to best. 
task  vs the others          rank  counts  tests
math  ▁▁●▆██▅█▃▁▁▁   #478 of 530*     27%      1
math · #478 of 530 · 27% of its score Beats [Yi 34B Chat](yi-34b-chat.md), [Command R (03-2024)](command-r-03-2024.md), [DeepSeek LLM 67B Chat](deepseek-llm-67b-chat.md) here, though they rank higher overall. Trails [GPT-Neo 125M](gpt-neo-125m.md), [GPT-2 XL (1.5B)](gpt-2-xl.md), [GPT-Neo 1.3B](gpt-neo-1-3b.md) here, though they rank lower overall. 
test   result  vs the others in its run  difficulty      counts  why
GSM8K    8.6%  █··●······▄▄ beats 2/5    easy · avg 67%     27%  sharp
other tests (8) 
test                                result  vs the others in its run  difficulty           counts  why
MMLU                                 42.9%  ▄···▄●·█···▄ beats 2/5    easy · avg 79%          27%  several runs
ARC Challenge                        52.6%  ▄·▄···●·▄··▄ beats 2/5    easy · avg 86%          18%  sharp · at its level
HellaSwag                            76.9%  ▄·▄●··▄····▄ beats 2/5    very easy · avg 92%    9.2%  at its level
Static HHH alignment (BIG-bench h…   62.9%  █·█········● beats 2/2    medium · avg 77%       5.7%  small run
SciTab 3-class claim verification…   27.6%  ●···▄▄·▄▄▄·▄ beats 1/7    medium · avg 51%       4.2%  blunt · at its level
API-Bank evaluation system, zero-…   15.2%  ▄······●··▄▄ beats 1/4    medium · avg 50%       3.8%  typical
TruthfulQA MC1                       19.5%  █·····●····█ beats 1/2    medium · avg 50%       3.5%  small run
TruthfulQA MC2                       39.6%  ▃···●·▃····▃ beats 1/5    medium · avg 56%       0.7%  blunt · at its level
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
Enhancing Chat Language Models by Scali…        4   48% ▇▇▇▇▇▇
INFORM : Information eNtropy based mult…        1   27% ▇▇▇
Aligning Large Language Models through …        3   16% ▇▇
SCITAB: A Challenging Benchmark for Com…        1    4% ▇
API-Bank: A Comprehensive Benchmark for…        1    4% ▇
prior: pull toward the middle                        1% ▇
weighted average                                   = 30.5
Licences: Enhancing Chat Language Models by Scaling High-quality Instructional Conversations ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-183-CC-BY-4-0.txt)) · INFORM : Information eNtropy based multi-step reasoning FOR large language Models ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-216-CC-BY-4-0.txt)) · Aligning Large Language Models through Synthetic Feedback ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-844-CC-BY-4-0.txt)) · SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-483-CC-BY-4-0.txt)) · API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-187-CC-BY-4-0.txt)) ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/alpaca-7b) · [JSON](https://whatmodel.sh/api?format=json&all=1)