[whatmodel.sh](../../README.md) / Llama 3.2 1B Instruct # Llama 3.2 1B Instruct Meta · released 2024-09 · 1.24B params · llama3.2 **score 32.5** ±0.6 · #648 of 732 · from 65 benchmark results across 19 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks Llama 3.2 1B Instruct among every model on that task or test, worst to best. 
task            vs the others         rank  counts  tests
agent tasks     ●▁▃▆█▄▅▅▄▄▂▁   #309 of 313    2.3%      3
coding          ▁▁▁▃●▇█▇▅▄▃▁   #416 of 469    8.6%      6
chat & writing  ▁▁▁●▆▄▆█▆▇▇▃   #376 of 402     12%      6
math            ▁▁●▅█▆▄█▄▂▁▁   #393 of 431     28%      6
agent tasks · #309 of 313 · 2.3% of its score Beats [QwQ-32B-Preview](qwq-32b-preview.md), [DeepSeek-Coder-V2 Lite Instruct (16B-A2.4B)](deepseek-coder-v2-lite-instruct.md), [Gemma 3 1B](gemma-3-1b.md) here, though they rank higher overall. Trails [Granite 4.0 350M](granite-4-0-350m.md), [Granite 4.0 H 350M](granite-4-0-h-350m.md), [Qwen2.5 0.5B Instruct](qwen2-5-0-5b.md) here, though they rank lower overall. 
test                                result  vs the others in its run  difficulty           counts  why
BFCL v3                              21.9%  ▁●▁▁▁▃▅█▅▄▁▁ beats 5/114  medium · avg 51%       1.1%  blunt · at its level · several runs
BFCL V4 (December 2025 archive)      10.8%  ▁●▁▂██▅▅▆▂▂▂ beats 1/77   hard · avg 31%         0.8%  typical
MASSIVE-Agents AST (10k, zero-sho…    0.2%  ●····███·███ beats 0/6    very hard · avg 15%    0.5%  sharp · too hard for it
coding · #416 of 469 · 8.6% of its score Beats [DeepSeek R1 Distill Qwen 7B](deepseek-r1-distill-qwen-7b.md), [Qwen3.5 2B](qwen3-5-2b.md), [DeepSeek R1 Distill Llama 8B](deepseek-r1-distill-llama-8b.md) here, though they rank higher overall. 
test                                result  vs the others in its run  difficulty        counts  why
HumanEval                            36.0%  ▄·●█·▄·▄···█ beats 1/7    easy · avg 78%      5.7%  sharp · at its level · several runs
MBPP                                 37.0%  ▄···▄·▄●█··█ beats 3/7    medium · avg 61%    1.8%  at its level
CodeReviewQA: automated code refi…    5.7%  ▁▁▁●▁▁▂▂▃█▅▄ beats 4/47   hard · avg 32%      0.4%  typical
CodeReviewQA: solution identifica…    0.8%  ●··▁▂▂▃▂█▃▄▂ beats 4/47   medium · avg 51%    0.3%  sharp · noisy
CodeReviewQA: change localisation…    0.0%  ●▂▁▁▂▂▂▃▅▅▅▇ beats 5/47   hard · avg 33%      0.2%  sharp · too hard for it · noisy
CodeReviewQA: change type recogni…   76.1%  ▁▁▁······▁▃● beats 34/47  easy · avg 67%      0.0%  blunt · at its level · noisy
chat & writing · #376 of 402 · 12% of its score Beats [Code Llama 70B Instruct](codellama-70b-instruct.md), [Phi-2 (2.7B)](phi-2.md), [Qwen2.5 1.5B Instruct](qwen2-5-1-5b.md) here, though they rank higher overall. Trails [Gemma 3 270M IT](gemma-3-270m.md), [SmolLM2 360M Instruct](smollm2-360m.md), [Granite 4.0 350M](granite-4-0-350m.md) here, though they rank lower overall. 
test                                result  vs the others in its run   difficulty           counts  why
LMArena (text)                      1111.1  ▁▁▁●▃▃▃▆▆▇█▃ beats 18/385  avg 1260               3.7%  blunt
IFEval (aggregation unspecified)     41.7%  ▄····●·▄··██ beats 1/7     easy · avg 82%         3.3%  at its level · several runs
IFEval mean of four measures         52.4%  ●·█··█·██·██ beats 0/6     very easy · avg 86%    1.9%  at its level
IFEval prompt-level loose            54.0%  ▁●▁▃▁▄▁▁█▃▃▁ beats 2/22    easy · avg 78%         1.0%  at its level
Arabic IFEval prompt-level (Incep…   11.1%  ▁·▁▁▂●▄▄▆▄█▄ beats 5/38    medium · avg 46%       0.9%  sharp
AlpacaEval 2 length-controlled       10.1%  ▃●·▁▁█▇▅·▁·▁ beats 3/22    medium · avg 39%       0.8%  sharp · too hard for it
math · #393 of 431 · 28% of its score Beats [Yi 34B Chat](yi-34b-chat.md), [Command R (03-2024)](command-r-03-2024.md), [DeepSeek LLM 67B Chat](deepseek-llm-67b-chat.md) here, though they rank higher overall. Trails [GPT-Neo 125M](gpt-neo-125m.md), [GPT-2 XL (1.5B)](gpt-2-xl.md), [GPT-Neo 1.3B](gpt-neo-1-3b.md) here, though they rank lower overall. 
test                       result  vs the others in its run  difficulty            counts  why
GSM8K                       45.4%  ▂●··▃▂▅█▆▃▂▃ beats 2/22   very easy · avg 87%      17%  sharp · at its level · several runs
MGSM                        29.1%  ●···█▄··▄··▄ beats 0/6    easy · avg 83%          5.2%  sharp · at its level
MATH (subset unspecified)   21.6%  ●▃█▂▂▆·····▁ beats 2/22   medium · avg 49%        4.8%  several runs
MATH-500                    23.4%  ●··█▄·▄···▄▄ beats 0/6    easy · avg 77%          0.9%  at its level · noisy
AIME 2025                    0.3%  ●█··▄···▄··█ beats 0/6    hard · avg 30%          0.4%  sharp · too hard for it · noisy
OTIS mock AIME 2024–2025     0.6%  ●▃▄▃▃▃▇▇█▃▂▅ beats 5/197  very hard · avg 7.2%    0.0%  sharp · too hard for it · noisy
other tests (26) 
test                                result  vs the others in its run   difficulty            counts  why
MMLU                                 46.7%  ▁▂·●·▁▁█·▃▃▅ beats 3/22    easy · avg 75%           11%  several runs
BIG-Bench Hard                       40.2%  ▁●·▁▁·▅█·▄▄▁ beats 3/22    medium · avg 63%        8.4%  sharp · several runs
HellaSwag                            59.7%  ●·▄··█▄·▄·▄▄ beats 0/7     easy · avg 85%          2.7%  at its level · several runs
CPTUB average (SpeakLeash, 0-5 ju…   38.4%  ▁●▂▂▃█▃▆▅▄▂▃ beats 6/95    medium · avg 65%        2.7%  at its level
MMLU-Pro                             19.5%  ▄·····●··█▄█ beats 1/7     medium · avg 44%        2.6%  several runs
EQ-Bench 3 (emotional intelligenc…   856.9  ●·▂▁▄▅▅▃█▄▁▁ beats 0/62    avg 1034                2.2%  typical
WinoGrande                           61.0%  ●···▄▄▄▄···█ beats 0/7     medium · avg 81%        2.2%  at its level
GPQA (subset unspecified)            25.7%  ▄·●··▄··▄█·█ beats 1/7     hard · avg 37%          2.2%  sharp · too hard for it · several runs
IFBench (aggregation unspecified)    15.9%  ●·█·█······█ beats 0/6     medium · avg 38%        2.1%  several runs
Polish EQ-Bench v2 (SpeakLeash)      20.6%  ▁··▁●▂▂▆▅▆█▆ beats 3/70    medium · avg 62%        1.7%  typical
Open Persian LLM Leaderboard v2 a…   35.0%  ▂●▂▃█▄▃▆▃▅▂▂ beats 6/81    medium · avg 57%        1.6%  blunt · at its level
BRIDGE clinical text, zero-shot a…   12.7%  ▁●▁▁▁▂▂▃▄▆█▃ beats 1/97    hard · avg 31%          1.4%  blunt
MMMLU                                38.1%  ▄···▄·▄●··▄█ beats 3/6     medium · avg 57%        1.3%  typical
BoolQ                                66.7%  ●▄····▄▄·█·█ beats 0/7     easy · avg 88%          1.1%  at its level
ARC Challenge                        41.4%  ▄····●··▄▄▄█ beats 2/7     medium · avg 72%        1.1%  sharp
OpenBookQA                           34.6%  ●▄··▄··▄·█▄▄ beats 0/7     medium · avg 55%        1.0%  typical
Open PT-LLM Leaderboard average (…   35.9%  ▃▁▁▁●▁▁▁▂█▅▁ beats 44/276  medium · avg 72%        0.6%  noisy
EuroEval Life in the UK (English …   32.4%  ▁▁▁▁●▁▁▂▄█▄▃ beats 7/146   easy · avg 82%          0.6%  at its level · noisy
PopQA                                13.8%  ▁▁●▃▃▃▁█▁▃·▄ beats 2/21    hard · avg 31%          0.6%  typical
AraGen-v3 3C3H score (Inception, …   10.5%  ▁▁·▁▂●▄█▄▅▄▄ beats 7/66    hard · avg 29%          0.5%  too hard for it
Vectara hallucination leaderboard…   79.3%  ▁●▂▃▄▄█▆▃▂▂▁ beats 4/140   very easy · avg 94%     0.5%  blunt · noisy
GPQA Diamond                         23.9%  ●▁▁▁▂▄▆▄▅▇█▅ beats 5/211   hard · avg 39%          0.4%  sharp · too hard for it
PIQA                                 72.3%  ▁▁▁▂▂●▅█▅▃▁▁ beats 28/123  easy · avg 84%          0.4%  blunt · at its level · small run
Social IQA                           35.5%  ●·····▃··█▃▅ beats 0/7     medium · avg 66%        0.2%  blunt · at its level
TruthfulQA MC2                       41.5%  ●▃▂·▃▃·▅█▂▅▂ beats 0/21    medium · avg 60%        0.2%  blunt · at its level
Chess Puzzles (Epoch)                 0.0%  ●▁▁▁▁▂▃▃▃▂▁▁ beats 37/137  very hard · avg 5.8%    0.0%  sharp · too hard for it · noisy
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
IBM Granite 3.0 Language Models technic…       15   31% ▇▇▇▇
LiquidAI/LFM2-1.2B model card                   7   15% ▇▇
2 OLMo 2 Furious                                8   15% ▇▇
SmolLM2 paper                                   7   11% ▇
LiquidAI/LFM2.5-1.2B-Thinking model card        7   10% ▇
LMArena leaderboard dataset                     1    4% ▇
SpeakLeash CPTUB                                1    3% ▇
EQ-Bench 3 canonical leaderboard results        1    2% ▇
SpeakLeash Polish EQ-Bench leaderboard          1    2% ▇
Open Persian LLM Leaderboard v2                 1    2% ▇
Berkeley Function Calling Leaderboard r…        2    1% ▇
Inception Arabic/Hindi leaderboards             2    1% ▇
BRIDGE Medical Leaderboard                      1    1% ▇
CodeReviewQA: The Code Review Comprehen…        4    1% ▇
Open Portuguese LLM Leaderboard                 1    1% ▇
EuroEval leaderboards                           1    1% ▇
MASSIVE-Agents                                  1    0% ▇
Vectara hallucination leaderboard               1    0% ▇
Epoch AI                                        3    0% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 32.5
Licences: IBM Granite 3.0 Language Models technical report ([Apache-2.0](https://github.com/ibm-granite/granite-3.0-language-models/blob/5960b7fb3308588bf5f86348278ec9d6015ab647/LICENSE)) · LiquidAI/LFM2-1.2B model card ([LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM2-1.2B/blob/40f3da0d0164913923aee9462c23077868b816a3/LICENSE)) · 2 OLMo 2 Furious ([CC BY 4.0](../../licenses/paper-2501.00656v3-CC-BY-4.0.txt)) · SmolLM2 paper ([CC BY 4.0](../../licenses/paper-2502.02737v1-CC-BY-4.0.txt)) · LiquidAI/LFM2.5-1.2B-Thinking model card ([LFM Open License v1.0](../../licenses/cards-gap-NOTICE.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · SpeakLeash CPTUB ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · EQ-Bench 3 canonical leaderboard results ([MIT](../../licenses/eqbench3-MIT.txt)) · SpeakLeash Polish EQ-Bench leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · BRIDGE Medical Leaderboard ([MIT](../../licenses/bridge-medical-leaderboard-MIT.txt)) · CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · MASSIVE-Agents ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/))
Not counted (1) 
source                  test          score  why
LiquidAI/LFM2.5-1.2B-…  GPQA Diamond  16.6%  failed run: Below guessing in Liquid AI's greedy-decoding comparison run; Epoch measures the same checkpoint at 0.239.
## How to run it ### Through an API api (Novita AI) · $0.020/M blended · $0.02 in / $0.02 out per 1M tokens · [novita.ai](https://novita.ai/models) [price source](https://github.com/BerriAI/litellm/blob/c19ce71bcdad234ca87a7af10cee45375ce811a8/model_prices_and_context_window.json): USD per 1M text tokens, standard uncached input and output. Model endpoint meta-llama/llama-3.2-1b-instruct. Product: Serverless chat completions. API model: meta-llama/llama-3.2-1b-instruct. Settings: service tier=standard, batch=no, cache=uncached input. Additional/excluded charges: Tools, cache storage, priority service and other modalities excluded.. Currency: USD. · [context source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/models/meta/llama-3.2-1b.toml) ### llama.cpp (recommended) 1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md) ```sh # macOS / Linux curl -LsSf https://llama.app/install.sh | sh # Windows irm https://llama.app/install.ps1 | iex ``` 2. Run Llama 3.2 1B Instruct (Q4_K_M, needs ~0.8 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Llama-3.2-1B-Instruct-GGUF/tree/199151125cf15a129ab3b548b26afeed976df066)) ```sh llama cli -hf lmstudio-community/Llama-3.2-1B-Instruct-GGUF:Q4_K_M ``` 3. Or start a local server and open the address it prints (http://localhost:9931) ```sh llama serve -hf lmstudio-community/Llama-3.2-1B-Instruct-GGUF:Q4_K_M ``` Other versions: Q8_0 (~1.3 GB) `llama cli -hf lmstudio-community/Llama-3.2-1B-Instruct-GGUF:Q8_0` ### LM Studio 1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download) 2. Run Llama 3.2 1B Instruct (Q4_K_M, needs ~0.8 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Llama-3.2-1B-Instruct-GGUF/tree/199151125cf15a129ab3b548b26afeed976df066)) ```sh lms get https://huggingface.co/lmstudio-community/Llama-3.2-1B-Instruct-GGUF@Q4_K_M -y ``` 3. Then run lms ls to see its name, and lms chat followed by that name Other versions: Q8_0 (~1.3 GB) `lms get https://huggingface.co/lmstudio-community/Llama-3.2-1B-Instruct-GGUF@Q8_0 -y` · 4bit (~0.7 GB, Mac only) `lms get https://huggingface.co/rajib-sarwar/Llama-3.2-1B-Instruct-4bit-MLX -y` · 8bit (~1.3 GB, Mac only) `lms get https://huggingface.co/mlx-community/Llama-3.2-1B-Instruct-8bit -y` ### Ollama 1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download) ```sh # Linux curl -fsSL https://ollama.com/install.sh | sh ``` 2. Run Llama 3.2 1B Instruct (Q8_0, needs ~1.99 GB, from [ollama.com](https://ollama.com/library/llama3.2/tags)) ```sh ollama run llama3.2:1b ``` Other versions: fp16 (~3.48 GB) `ollama run llama3.2:1b-instruct-fp16` ### MLX (Mac) 1. Install MLX (Mac). Apple silicon Macs only, text only. ```sh # with uv uv tool install mlx-lm # or pip install mlx-lm ``` 2. Run Llama 3.2 1B Instruct (4bit, needs ~0.7 GB, Mac only, from [huggingface.co/rajib-sarwar](https://huggingface.co/rajib-sarwar/Llama-3.2-1B-Instruct-4bit-MLX/tree/4a9cf1c9b1f5af9122259c8bfe0e1de21e9af25d)) ```sh mlx_lm.chat --model rajib-sarwar/Llama-3.2-1B-Instruct-4bit-MLX ``` 3. Or start an OpenAI-compatible server at http://localhost:8080/v1 ```sh mlx_lm.server --model rajib-sarwar/Llama-3.2-1B-Instruct-4bit-MLX ``` Other versions: 8bit (~1.3 GB, Mac only) `mlx_lm.chat --model mlx-community/Llama-3.2-1B-Instruct-8bit` Model repository and documentation: [huggingface.co/meta-llama](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct) ## Published speed measurements Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine. 
hardware                      precision        speed  engine and settings
Apple M5 Max (40-core GPU),…  q8_0       250.8 tok/s  ollama (version not recorded) · l…
Intel Arc Graphics 130V/140…  q8_0       27.86 tok/s  ollama (version not recorded) · l…
Intel CoffeeLake-S GT2 [UHD…  q8_0       13.36 tok/s  ollama (version not recorded) · l…
Each speed links to its source, which has the full settings. [interactive version](https://whatmodel.sh/model/llama-3-2-1b) · [JSON](https://whatmodel.sh/api?format=json&all=1)