[whatmodel.sh](../../README.md) / LLaMA 65B
# LLaMA 65B
Meta · released 2023-02 · 65.2B params · llama-1-noncommercial-research
**score 40.0** ±0.7 · comparison only: no current run option verified · from 38 benchmark results across 13 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks LLaMA 65B among every model on that task or test, worst to best.
task vs the others rank counts tests
coding ▁▁▁●▄▆█▇▅▄▃▁ #497 of 539 15% 2
math ▁▁▃▆█●▅█▃▁▁▁ #286 of 530 15% 2
coding · #497 of 539 · 15% of its score
Beats [DeepSeek R1 Distill Qwen 7B](deepseek-r1-distill-qwen-7b.md), [Qwen3.5 2B](qwen3-5-2b.md), [DeepSeek R1 Distill Llama 8B](deepseek-r1-distill-llama-8b.md) here, though they rank higher overall. Trails [Llama 3.2 1B Instruct](llama-3-2-1b.md), [SmolLM2 1.7B Instruct](smollm2-1-7b.md), [Mistral 7B Instruct v0.1](mistral-7b-instruct-v0-1.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
HumanEval 23.5% ▄······▄·▄▄● beats 5/5 medium · avg 59% 12% sharp · at its level · several runs
MBPP (maker) 37.7% ▁▁▁▄●█▆▅▄▁▂▁ beats 16/86 medium · avg 59% 2.7% at its level
math · #286 of 530 · 15% of its score
Beats [GPT-4o (2024-08-06)](gpt-4o-2024-08-06.md), [Command A (03-2025)](command-a.md), [GPT-4o (2024-05-13)](gpt-4o-2024-05-13.md) here, though they rank higher overall. Trails [Gemma 7B (base)](gemma-7b-base.md), [CodeGemma 7B (v1.0)](codegemma-7b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
GSM8K 50.0% █···▄▄····▄● beats 5/5 easy · avg 72% 11% sharp · at its level · several runs
MATH (maker) 10.6% ▃▅▂●▅▃▅▅█▃▃▂ beats 8/30 hard · avg 24% 4.0% typical
other tests (22)
test result vs the others in its run difficulty counts why
MMLU 63.3% ▄·▄····▄·▄·● beats 5/5 easy · avg 79% 14% at its level · several runs
ARC Challenge 46.3% ▄····▄·▄▄·▄● beats 6/6 easy · avg 74% 14% sharp · at its level · several runs
BIG-Bench Hard 58.1% █···▄··▄··▄● beats 5/5 easy · avg 74% 5.9% sharp · at its level
WinoGrande 77.1% ▃·▃▃▃▅·█···● beats 9/9 easy · avg 87% 5.8% at its level · several runs
F2WTQ free-form table QA, 2-shot … 36.2% ▄···█·▄●··▄▄ beats 4/7 medium · avg 56% 4.2% at its level
HellaSwag 80.7% ▅█······▃▃·● beats 8/8 very easy · avg 92% 4.1% several runs
ImageNetVC visual commonsense, te… 65.1% ▂·▄·▂█▄▄█▄▂● beats 21/21 easy · avg 74% 3.3% blunt · at its level
Factual knowledge recall, Premium… 35.3% ▅····▃▅·██·● beats 12/13 medium · avg 52% 2.8% at its level
Natural Questions (closed book) (… 35.0% ▂▁▁▁▂▄▄▃▅█●▁ beats 67/68 medium · avg 53% 2.1% at its level
OpenBookQA (maker) 60.2% ▅▁▁▁▂▂▃█●▁▃▁ beats 92/107 easy · avg 76% 2.0% at its level
TriviaQA (maker) 72.6% ▁▁▂▂▄▄▃▆▅█●▂ beats 72/80 easy · avg 84% 1.8% at its level
BoolQ (maker) 85.3% ▁····▁▂█▆▅▆● beats 93/103 easy · avg 90% 1.5% at its level
Factual knowledge recall, Premium… 52.4% ▃▅▃··▅▃·█▃▃● beats 13/13 medium · avg 61% 1.5% at its level
Spider dev execution accuracy, ze… 12.2% █·█●··▄····▄ beats 4/6 medium · avg 54% 1.3% sharp · at its level · noisy
PIQA 80.8% ▄··▄··▄█··▄● beats 6/6 easy · avg 86% 1.3% blunt · at its level · several runs
This is not a Dataset (negation),… 50.3% ●▁····▃▁·▁·▁ beats 7/14 hard · avg 66% 1.3% sharp · noisy
ARC Easy 58.8% ▃··●···▃·▃·▃ beats 1/6 easy · avg 87% 1.2% blunt · at its level · several runs
DiBiMT lexical-ambiguity translat… 57.9% ██···██···█● beats 5/5 easy · avg 74% 1.1% blunt · at its level
Social IQA (maker) 52.3% ▁·▁██▄●▁▂▃▇▁ beats 39/59 medium · avg 63% 0.8% blunt · at its level
StoryAnalogy multiple-choice stor… 28.6% ●·······█··█ beats 0/2 hard · avg 42% 0.2% blunt · noisy · small run
TruthfulQA MC2 42.5% ▃···█·●····▃ beats 4/5 medium · avg 56% 0.2% blunt · at its level
IndoMMLU (zero-shot) 35.8% ▅▃▅··▃···▃▃● beats 8/10 hard · avg 37% 0.1% blunt · noisy
source results share of the score
LLaMA: Open and Efficient Foundation La… 15 37% ▇▇▇▇
How Far Can Camels Go? Exploring the St… 4 20% ▇▇
Enhancing Chat Language Models by Scali… 4 12% ▇
Outlier Suppression+: Accurate quantiza… 5 11% ▇
Towards a Holistic Evaluation of LLMs o… 2 4% ▇
Investigating Table-to-Text Generation … 1 4% ▇
ImageNetVC: Zero- and Few-Shot Visual C… 1 3% ▇
Enhancing Computation Efficiency in Lar… 1 3% ▇
Battle of the Large Language Models: Do… 1 1% ▇
This is not a Dataset: A Large Negation… 1 1% ▇
Towards Effective Disambiguation for Ma… 1 1% ▇
StoryAnalogy: Deriving Story-level Anal… 1 0% ▇
Large Language Models Only Pass Primary… 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 40.0
Licences: LLaMA: Open and Efficient Foundation Language Models ([CC BY 4.0](../../licenses/paper-2302.13971v1-CC-BY-4.0.txt)) · How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources ([CC BY 4.0](../../licenses/paper-2306.04751v2-tulu1-CC-BY-4-0.txt)) · Enhancing Chat Language Models by Scaling High-quality Instructional Conversations ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-183-CC-BY-4-0.txt)) · Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-102-CC-BY-4-0.txt)) · Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Investigating Table-to-Text Generation Capabilities of Large Language Models in Real-World Information Seeking Scenarios ([CC BY 4.0](../../licenses/acl-2023-emnlp-industry-17-CC-BY-4-0.txt)) · ImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-133-CC-BY-4-0.txt)) · Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantization ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-910-CC-BY-4-0.txt)) · Battle of the Large Language Models: Dolly vs LLaMA vs Vicuna vs Guanaco vs Bard vs ChatGPT - A Text-to-SQL Parsing Comparison ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-750-CC-BY-4-0.txt)) · This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-531-CC-BY-4-0.txt)) · Towards Effective Disambiguation for Machine Translation with Large Language Models ([CC BY 4.0](../../licenses/acl-2023-wmt-1-44-CC-BY-4-0.txt)) · StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-706-CC-BY-4-0.txt)) · Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-760-CC-BY-4-0.txt))
## How to run it
No verified API or local download yet.
[interactive version](https://whatmodel.sh/model/llama-65b) · [JSON](https://whatmodel.sh/api?format=json&all=1)