[whatmodel.sh](../../README.md) / Llama 3.1 8B Instruct
# Llama 3.1 8B Instruct
Meta · released 2024-07 · 8.03B params · llama3.1
**score 46.1** ±0.3 · #452 of 732 · from 87 benchmark results across 31 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Llama 3.1 8B Instruct among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃●█▄▅▅▄▄▂▁ #274 of 313 1.6% 4
coding ▁▁▁▃▅●█▇▅▄▃▁ #323 of 469 28% 13
chat & writing ▁▁▁▃▆●▆█▆▇▇▃ #304 of 402 6.0% 6
math ▁▁▃●█▆▄█▄▂▁▁ #349 of 431 14% 8
agent tasks · #274 of 313 · 1.6% of its score
Beats [Magistral Small 1.0 (2506)](magistral-small-2506.md), [Llama 4 Scout (109B-A17B)](llama-4-scout.md), [Phi-4 14B](phi-4.md) here, though they rank higher overall. Trails [Granite 4.0 H 1B](granite-4-0-h-1b.md), [Granite 4.0 1B](granite-4-0-1b.md), [SmolLM3 3B](smollm3-3b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
MASSIVE-Agents AST (10k, zero-sho… 9.1% █····██●·███ beats 3/6 very hard · avg 15% 0.6% sharp
BFCL V4 (December 2025 archive) 25.8% ▁▁▁▂●█▅▅▆▂▂▂ beats 16/77 hard · avg 31% 0.4% at its level
Galileo Agent Leaderboard v1 (too… 67.8% ▃●▃▃▆▅▆▅█▂▅▂ beats 2/31 easy · avg 78% 0.3% at its level
BFCL v3 49.6% ▁▁▁▁▁▃●█▅▄▁▁ beats 33/114 medium · avg 51% 0.3% blunt · at its level
coding · #323 of 469 · 28% of its score
Beats [Solar Open 100B (102B-A12B)](solar-open-100b.md), [Qwen3 4B Instruct 2507](qwen3-4b-instruct-2507.md), [Qwen3-VL 8B Instruct](qwen3-vl-8b-instruct.md) here, though they rank higher overall. Trails [DeepSeek LLM 67B Chat](deepseek-llm-67b-chat.md), [WizardLM 70B V1.0](wizardlm-70b-v1-0.md), [Code Tulu 2 34B](code-tulu-2-34b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
HumanEval 69.5% ▁·▁▁▃▄▆●█▃▁▁ beats 57/99 easy · avg 76% 10% sharp · at its level · several runs
HumanEval+ 62.8% ▁·▁▁▁▃▄▄●█▃▁ beats 57/99 easy · avg 70% 6.9% sharp · at its level
CRUXEval (ZeroEval, 800 problems) 39.9% ▁▂●▃▆█▆▃▂·▁▁ beats 5/47 medium · avg 57% 3.3% sharp · at its level
Aider code editing (133 Exercism … 37.6% ▁·▁▂●▁▆▄█▆▂▂ beats 5/62 medium · avg 51% 2.2% sharp · at its level
LiveCodeBench official (Jan 1–Nov… 16.1% ▁▁·▁●▄▆█▅▂·▂ beats 3/31 hard · avg 23% 1.7% sharp
MBPP+ 55.6% ▁···▁▁▂▅●█▆▂ beats 29/75 easy · avg 66% 1.3% at its level
MBPP 52.2% ▄···▄·▄▄█··● beats 7/7 medium · avg 61% 0.7% at its level
Luau Leetcode fully solved (Torpe… 8.2% ●▁▁▁·▁▂▃▅▃█▇ beats 0/49 hard · avg 17% 0.5% sharp
WebApp1K-React pass@1 (onekq-ai) 25.1% ▃▁▁●▁▄▅▃▄▇▁▄ beats 4/31 medium · avg 40% 0.4% sharp · at its level · noisy
CodeReviewQA: solution identifica… 54.3% ▃··▁▂▂▃▂●▃▄▂ beats 29/47 medium · avg 51% 0.4% sharp · at its level · noisy
CodeReviewQA: change localisation… 29.8% █▂▁▁▂▂▂▃●▅▅▇ beats 25/47 hard · avg 33% 0.3% sharp · noisy
CodeReviewQA: automated code refi… 26.2% ▁▁▁▁▁▁▂▂●█▅▄ beats 17/47 hard · avg 32% 0.3% at its level
CodeReviewQA: change type recogni… 71.8% ▁▁▁······▁▃● beats 21/47 easy · avg 67% 0.0% blunt · at its level · noisy
chat & writing · #304 of 402 · 6.0% of its score
Beats [Ministral 3 14B Reasoning 2512](ministral-3-14b-reasoning.md), [Gemma 4 E4B](gemma-4-e4b.md), [OpenThinker3 7B](openthinker3-7b.md) here, though they rank higher overall. Trails [LFM2 350M](lfm2-350m.md), [EXAONE 3.5 2.4B Instruct](exaone-3-5-2-4b.md), [LFM2 700M](lfm2-700m.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena-Hard (v0.1, GPT-4-1106 judg… 21.3% ▂▁▁▅●▅▄▃▃▅▃▁ beats 16/60 medium · avg 47% 2.5% sharp · at its level
LMArena (text) 1211.2 ▁▁▁▂▃●▃▆▆▇█▃ beats 73/385 avg 1260 1.2% blunt
Arabic IFEval prompt-level (Incep… 41.6% ▁·▁▁▂▁▄▄▆●█▄ beats 23/38 medium · avg 46% 0.9% sharp · at its level
AlpacaEval 2 length-controlled 25.8% ▃▃·▁●█▇▅·▁·▁ beats 5/22 medium · avg 39% 0.7% sharp · at its level
IFEval (aggregation unspecified) 50.4% ▄····█·▄··●█ beats 4/7 easy · avg 65% 0.4% at its level
IFEval prompt-level loose 80.6% ▁▃▁▃▁▄▁●█▃▃▁ beats 11/22 easy · avg 78% 0.3% at its level
math · #349 of 431 · 14% of its score
Beats [Athene-70B](athene-70b.md), [Gemma 2 9B IT](gemma-2-9b.md), [GPT-3.5 Turbo (0125)](gpt-3-5-turbo-0125.md) here, though they rank higher overall. Trails [Gemma 2B (base)](gemma-2b-base.md), [Stable Code 3B](stable-code-3b.md), [Zephyr 7B beta](zephyr-7b-beta.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
GSM8K 83.4% ▂▃··▃▂▅●▆▃▂▃ beats 10/22 very easy · avg 90% 4.0% sharp · several runs
MGSM 56.7% ▅▃▃●·▅···▃·▃ beats 4/10 easy · avg 72% 2.5% sharp · at its level
GSM8K (ZeroEval, zero-shot) 84.0% ▁▄·▁▄█●▅▅███ beats 16/43 very easy · avg 87% 2.1% sharp
MATH (subset unspecified) 42.5% ▃▃●▂▂▆·····▁ beats 11/22 medium · avg 49% 1.9% at its level · several runs
MATH level 5 22.9% ▂▂▅●▄█▆▃▅▂▃▄ beats 20/92 hard · avg 28% 1.3% sharp
MATH Level 5 (ZeroEval, 721 probl… 22.2% ▁▁▁▆●▂▇▇▄··▂ beats 11/38 hard · avg 30% 1.0% sharp
MATH 47.6% █▄●█···▄▄▄·▄ beats 4/10 medium · avg 61% 0.9% at its level
OTIS mock AIME 2024–2025 1.7% ▃●▄▃▃▃▇▇█▃▂▅ beats 12/197 very hard · avg 7.2% 0.1% sharp · too hard for it · noisy
other tests (38)
test result vs the others in its run difficulty counts why
BIG-Bench Hard 71.9% ▁▄·▁▁·▅●·▄▄▁ beats 14/22 medium · avg 63% 5.8% sharp · at its level · several runs
MMLU-Redux (ZeroEval, 2,778 quest… 67.2% ▁·▁▁▄●▄▅█▄▂▁ beats 11/53 easy · avg 75% 5.7% at its level
MMLU 71.3% ▁▂·▁·▁▁●·▃▃▅ beats 12/22 easy · avg 75% 5.1% at its level · several runs
LiveBench 2024-07-26 28.0% ▁▁▁▂▂▃▂●▆▃▅▃ beats 22/61 medium · avg 38% 4.3% at its level
MMLU-Pro 44.2% ▃▂▅●▃·█▃▅··▂ beats 10/23 medium · avg 54% 4.1% at its level · several runs
ARC Challenge 83.1% ▂▂▂●▂···▄··▂ beats 5/10 very easy · avg 89% 2.5% sharp · several runs
GPQA (subset unspecified) 26.3% █··●▂··▂▂··▄ beats 4/10 hard · avg 37% 2.3% sharp · several runs
Medmarks-V mean win rate (19 veri… 0.3967 ▁▄●▃▇▄█▃▅▁▁▁ beats 5/30 avg 0.418 2.2% sharp
CLIcK Korean (Daekeun Kim) 47.8% ●▄▄▄█▄▂·▄█·▄ beats 2/23 medium · avg 56% 2.0% at its level
HAE-RAE 1.0 (Daekeun Kim) 53.9% ▄●▄▂▂█▂▄▂█▂▄ beats 3/22 medium · avg 61% 1.8% at its level
OpenBookQA 84.8% ▂··▄█▂·●···▄ beats 8/10 easy · avg 85% 1.7% at its level · several runs
WinoGrande 64.7% ▃····▃▃●▃·▅▅ beats 4/10 medium · avg 81% 1.3% at its level · several runs
RobloxQA v2.0 multiple choice (To… 78.7% ●▁▃▁█▄▃▃▆▂▂▂ beats 1/49 easy · avg 77% 1.0% at its level
ZebraLogic (ZeroEval, 1,000 puzzl… 12.8% ▁▁▃●██▁▁·▁▁▁ beats 17/57 hard · avg 15% 1.0% sharp
CPTUB average (SpeakLeash, 0-5 ju… 60.2% ▁▂▂▂▃●▃▆▅▄▂▃ beats 33/95 medium · avg 65% 0.9% at its level
Medmarks-V weighted mean win rate… 39.7% ▂▃●▂▆▃█▅█▅▃▆ beats 5/34 medium · avg 41% 0.9% blunt · at its level
HellaSwag 72.8% ▃●█·▃▃▃···▃▃ beats 2/10 easy · avg 85% 0.9% several runs
Multilingual MMLU (Microsoft) 56.2% █▄▄█●··█···▄ beats 7/10 medium · avg 63% 0.9% at its level
EvalPerf correctness (118 tasks, … 64.3% ▂·▂▂●▆▅▆▆▆▃█ beats 3/29 easy · avg 74% 0.8% at its level
GPQA Diamond 27.0% ▁▁●▁▂▄▆▄▅▇█▅ beats 8/211 hard · avg 39% 0.8% sharp
BRIDGE clinical text, zero-shot a… 29.0% ▁▁▁▁▁▂▂●▄▆█▃ beats 27/97 hard · avg 31% 0.6% blunt · at its level
BoolQ 82.8% ▃··▃··██●·▃▃ beats 8/10 easy · avg 88% 0.6% at its level · several runs
Open Persian LLM Leaderboard v2 a… 50.3% ▂▄▂▃●▄▃▆▃▅▂▂ beats 29/81 medium · avg 57% 0.6% blunt · at its level
AraGen-v3 3C3H score (Inception, … 24.9% ▁▁·▁▂▂▄●▄▅▄▄ beats 20/66 hard · avg 29% 0.5% typical
Social IQA 71.8% ▃·●▅▅█·▃···▃ beats 1/10 medium · avg 76% 0.4% blunt · at its level · several runs
Judgemark v2 (LLM-as-judge for cr… 35.7% ●▃··▃▅▅···█▃ beats 0/10 medium · avg 53% 0.4% at its level
PopQA 28.4% ▁▁▁▃▃▃▁●▁▃·▄ beats 11/21 hard · avg 31% 0.3% at its level
RULER 64k 84.7% ▁▁·▁·▂▃▄●▂·▁ beats 14/22 easy · avg 82% 0.3% typical
Open PT-LLM Leaderboard average (… 70.2% ▃▁▁▁▁▁▁▁▂●▅▁ beats 166/276 medium · avg 72% 0.3% at its level · noisy
DiagnosticIQ industrial maintenan… 38.7% ●··▃▁▁▁▅█▅▁▅ beats 0/24 hard · avg 51% 0.2% blunt · at its level
KMMLU-HARD (Daekeun Kim, zero-sho… 24.0% ●···▁▁▁▂▃·▆▂ beats 7/23 very hard · avg 29% 0.2% sharp · too hard for it
EuroEval Life in the UK (English … 84.4% ▁▁▁▁▁▁▁▂▄●▄▃ beats 95/146 easy · avg 82% 0.2% at its level · noisy
PIQA 83.2% ▃·▅·▃█·▃●··▃ beats 8/10 medium · avg 82% 0.1% blunt · at its level
TruthfulQA MC2 55.1% ▃▃▂·●▃·▅█▂▅▂ beats 6/21 medium · avg 64% 0.1% blunt · at its level · several runs
RULER 128k 77.0% ▄··▂▄▂▆█●▄▄▂ beats 15/21 easy · avg 67% 0.1% at its level · noisy
Vectara hallucination leaderboard… 94.6% ▁▁▂▃▄●█▆▃▂▂▁ beats 57/140 very easy · avg 94% 0.1% blunt · maxed out · noisy
LIBRA Mini (Russian long context,… 43.9% ▄···▄█·█●··▄ beats 7/8 medium · avg 38% 0.1% blunt · at its level
Chess Puzzles (Epoch) 0.0% ●▁▁▁▁▂▃▃▃▂▁▁ beats 23/137 very hard · avg 5.8% 0.0% sharp · too hard for it · noisy
source results share of the score
EvalPlus / EvalPerf published evaluatio… 4 17% ▇▇
microsoft/Phi-4-mini-instruct model card 16 16% ▇▇
IBM Granite 3.0 Language Models technic… 15 15% ▇▇
WildEval ZeroEval published results 5 13% ▇▇
2 OLMo 2 Furious 8 6% ▇
LiveBench model judgments 1 4% ▇
Daekeun Kim Korean LLM benchmark runs 3 4% ▇
Medmarks: A Comprehensive Open-Source L… 2 3% ▇
Arena-Hard-Auto v0.1 leaderboard 1 3% ▇
Epoch AI 4 2% ▇
Aider code editing benchmark runs 1 2% ▇
MMLU-Pro author-released evaluation runs 1 2% ▇
LiveCodeBench official website results 1 2% ▇
Roblox LLM Leaderboard 2 1% ▇
Inception Arabic/Hindi leaderboards 2 1% ▇
LMArena leaderboard dataset 1 1% ▇
CodeReviewQA: The Code Review Comprehen… 4 1% ▇
SpeakLeash CPTUB 1 1% ▇
Berkeley Function Calling Leaderboard r… 2 1% ▇
BRIDGE Medical Leaderboard 1 1% ▇
MASSIVE-Agents 1 1% ▇
Open Persian LLM Leaderboard v2 1 1% ▇
WebApp1K models leaderboard 1 0% ▇
Judgemark v2 results 1 0% ▇
NVIDIA RULER repository README results … 2 0% ▇
Open Portuguese LLM Leaderboard 1 0% ▇
Galileo Agent Leaderboard 1 0% ▇
DiagnosticIQ: A Benchmark for LLM-Based… 1 0% ▇
EuroEval leaderboards 1 0% ▇
Vectara hallucination leaderboard 1 0% ▇
ai-forever LIBRA leaderboard 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 46.1
Licences: EvalPlus / EvalPerf published evaluation runs ([Apache-2.0](../../licenses/evalplus-published-runs-Apache-2.0.txt)) · microsoft/Phi-4-mini-instruct model card ([MIT](../../licenses/card-microsoft-phi-4-mini-instruct.txt)) · IBM Granite 3.0 Language Models technical report ([Apache-2.0](https://github.com/ibm-granite/granite-3.0-language-models/blob/5960b7fb3308588bf5f86348278ec9d6015ab647/LICENSE)) · WildEval ZeroEval published results ([Apache-2.0](../../licenses/zeroeval-published-results-Apache-2.0.txt)) · 2 OLMo 2 Furious ([CC BY 4.0](../../licenses/paper-2501.00656v3-CC-BY-4.0.txt)) · LiveBench model judgments ([Apache-2.0 (stated in the LiveBench datasheet)](https://www.apache.org/licenses/LICENSE-2.0)) · Daekeun Kim Korean LLM benchmark runs ([MIT](../../licenses/korean-personal-benchmark-runs-MIT.txt)) · Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks ([CC BY 4.0](../../licenses/paper-2605.01417v1-CC-BY-4.0.txt)) · Arena-Hard-Auto v0.1 leaderboard ([Apache-2.0](../../licenses/arena-hard-auto-v0-1-Apache-2.0.txt)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Aider code editing benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) · MMLU-Pro author-released evaluation runs ([Apache-2.0](../../licenses/mmlu-pro-author-runs-Apache-2.0.txt)) · LiveCodeBench official website results ([CC-BY-SA-4.0](../../licenses/lcb-sharealike-NOTICE.txt)) · Roblox LLM Leaderboard ([MIT](../../licenses/roblox-llm-leaderboard-MIT.txt)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · SpeakLeash CPTUB ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · BRIDGE Medical Leaderboard ([MIT](../../licenses/bridge-medical-leaderboard-MIT.txt)) · MASSIVE-Agents ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · WebApp1K models leaderboard ([MIT](../../licenses/webapp1k-leaderboard-MIT.txt)) · Judgemark v2 results ([MIT](../../licenses/judgemark-v2-MIT.txt)) · NVIDIA RULER repository README results table ([Apache-2.0](https://github.com/NVIDIA/RULER/blob/c3f5e3b4f87f97e048793bb510a3a6b19a46bf3a/LICENSE)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Galileo Agent Leaderboard ([Apache-2.0](../../licenses/galileo-agent-leaderboard-Apache-2.0.txt)) · DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · ai-forever LIBRA leaderboard ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt))
## How to run it
### Through an API
api (Amazon Bedrock) · $0.22/M blended · $0.22 in / $0.22 out per 1M tokens · [docs.aws.amazon.com](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html)
[price source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/providers/amazon-bedrock/models/meta.llama3-1-8b-instruct-v1:0.toml): Standard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/models/meta/llama-3.1-8b-instruct.toml): model catalogue context ceiling (not a promise by a hosted API)
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Llama 3.1 8B Instruct (Q4_K_M, needs ~4.9 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF/tree/8601e6db71269a2b12255ebdf09ab75becf22cc8))
```sh
llama cli -hf lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF:Q4_K_M
```
Other versions: Q8_0 (~8.5 GB) `llama cli -hf lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF:Q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Llama 3.1 8B Instruct (Q4_K_M, needs ~4.9 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF/tree/8601e6db71269a2b12255ebdf09ab75becf22cc8))
```sh
lms get https://huggingface.co/lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~8.5 GB) `lms get https://huggingface.co/lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF@Q8_0 -y` · 4bit (~4.5 GB, Mac only) `lms get https://huggingface.co/mlx-community/Llama-3.1-8B-Instruct-4bit -y` · 8bit (~8.5 GB, Mac only) `lms get https://huggingface.co/mlx-community/Meta-Llama-3.1-8B-Instruct-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Llama 3.1 8B Instruct (Q4_K_M, needs ~5.32 GB, from [ollama.com](https://ollama.com/library/llama3.1/tags))
```sh
ollama run llama3.1:8b
```
Other versions: Q8_0 (~10.14 GB) `ollama run llama3.1:8b-instruct-q8_0`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run Llama 3.1 8B Instruct (4bit, needs ~4.5 GB, Mac only, from [huggingface.co/mlx-community](https://huggingface.co/mlx-community/Llama-3.1-8B-Instruct-4bit/tree/90215b22ec18e72f623dde2ea7af4097025160e2))
```sh
mlx_lm.chat --model mlx-community/Llama-3.1-8B-Instruct-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model mlx-community/Llama-3.1-8B-Instruct-4bit
```
Other versions: 8bit (~8.5 GB, Mac only) `mlx_lm.chat --model mlx-community/Meta-Llama-3.1-8B-Instruct-8bit`
Model repository and documentation: [huggingface.co/meta-llama](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct)
## Published speed measurements
Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine.
hardware precision speed engine and settings
Apple M2 Pro, 32 GB q8_0 22.29 tok/s ollama (version not recorded) · l…
Apple M5 Max (40-core GPU),… q4_k_m 89.26 tok/s ollama (version not recorded) · l…
RTX 4060 Laptop q4_k_m 47.17 tok/s ollama (version not recorded) · l…
Each speed links to its source, which has the full settings.
## Quantization evidence
How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent. They don't describe every download of that precision.
precision test full → quantized kept
AutoRound 2-bit (group size 3… WMT24++ X->English COMET (wmt22-c… 0.737 → 0.466 63.2%
AutoRound 2-bit (group size 3… WMT24++ English->X COMET (wmt22-c… 0.694 → 0.357 51.5%
AutoRound 4-bit (group size 1… WMT24++ English->X COMET (wmt22-c… 0.694 → 0.68 98.1%
AutoRound 4-bit (group size 1… WMT24++ X->English COMET (wmt22-c… 0.737 → 0.704 95.5%
AWQ 4-bit (AutoAWQ, zero poin… WMT24++ English->X COMET (wmt22-c… 0.694 → 0.676 97.5%
AWQ 4-bit (AutoAWQ, zero poin… WMT24++ X->English COMET (wmt22-c… 0.737 → 0.709 96.1%
bitsandbytes NF4 (Transformer… WMT24++ English->X COMET (wmt22-c… 0.694 → 0.681 98.1%
bitsandbytes NF4 (Transformer… WMT24++ X->English COMET (wmt22-c… 0.737 → 0.7 94.9%
GGUF Q2_K with imatrix (llama… WMT24++ X->English COMET (wmt22-c… 0.737 → 0.697 94.5%
GGUF Q2_K with imatrix (llama… WMT24++ English->X COMET (wmt22-c… 0.694 → 0.556 80.2%
GGUF Q4_K_M with imatrix (lla… WMT24++ English->X COMET (wmt22-c… 0.694 → 0.683 98.4%
GGUF Q4_K_M with imatrix (lla… WMT24++ X->English COMET (wmt22-c… 0.737 → 0.723 98.0%
MXFP4 via AutoRound 'OPT-RNT(… Average of 11 lm-eval tasks (ARC-… 0.652 → 0.591 90.7%
MXFP4 via AutoRound 'OPT-RTN(… Average of 11 lm-eval tasks (ARC-… 0.652 → 0.585 89.7%
MXFP4 via AutoRound 'orig-RTN… Average of 11 lm-eval tasks (ARC-… 0.652 → 0.572 87.8%
NVFP4 via AutoRound 'opt-rtn'… Average of 11 lm-eval tasks (ARC-… 0.652 → 0.639 98.1%
NVFP4 via AutoRound 'RTN' (it… Average of 11 lm-eval tasks (ARC-… 0.652 → 0.626 96.1%
ONNX INT4 AWQ (NVIDIA ModelOp… MMLU 5-shot (ModelOpt-Windows acc… 0.684 → 0.661 96.6%
Each kept value links to its source.
[interactive version](https://whatmodel.sh/model/llama-3-1-8b) · [JSON](https://whatmodel.sh/api?format=json&all=1)