[whatmodel.sh](../../README.md) / SmolLM3 3B
# SmolLM3 3B
Hugging Face · released 2025-07 · 3.08B params · apache-2.0
**score 43.8** ±0.9 · #505 of 732 · from 25 benchmark results across 6 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks SmolLM3 3B among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█▄▅●▄▄▂▁ #106 of 313* 1.7% 1
coding ▁▁▁▃●▇█▇▅▄▃▁ #391 of 469* 2.2% 1
chat & writing ▁▁▁▃▆▄●█▆▇▇▃ #290 of 402* 7.5% 2
math ▁▁▃▅●▆▄█▄▂▁▁ #264 of 431 11% 3
agent tasks · #106 of 313 · 1.7% of its score
Beats [Grok 4.20 (0309 reasoning)](grok-4-20.md), [o3-pro](o3-pro.md), [Grok 4.3](grok-4-3.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
BFCL (version not stated) (maker) 88.8% ▄▆▄▂·█▆·●··▂ beats 16/17 very easy · avg 88% 1.7% blunt · small run
coding · #391 of 469 · 2.2% of its score
Beats [LFM2.5 8B-A1B](lfm2-5-8b-a1b.md), [DeepSeek R1 Distill Qwen 7B](deepseek-r1-distill-qwen-7b.md), [Qwen3.5 2B](qwen3-5-2b.md) here, though they rank higher overall. Trails [Qwen2.5 1.5B Instruct](qwen2-5-1-5b.md), [WizardLM 13B V1.2](wizardlm-13b-v1-2.md), [Falcon3 3B Instruct](falcon3-3b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LiveCodeBench v4 (maker) 30.0% ●·█········█ beats 0/2 medium · avg 45% 2.2% at its level · small run
chat & writing · #290 of 402 · 7.5% of its score
Beats [Ministral 3 14B Reasoning 2512](ministral-3-14b-reasoning.md), [Ministral 3 8B](ministral-3-8b.md), [Gemma 4 E4B](gemma-4-e4b.md) here, though they rank higher overall. Trails [EXAONE 3.5 2.4B Instruct](exaone-3-5-2-4b.md), [LFM2 700M](lfm2-700m.md), [Flan-T5 XXL](flan-t5-xxl.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
IFEval mean of four measures 72.4% ●··▄▄·▄····▄ beats 1/5 easy · avg 84% 5.4% typical
IFEval (aggregation unspecified) … 71.2% ▁▂▄▂●▄▆██▅▂▁ beats 33/145 easy · avg 80% 2.1% at its level · small run
math · #264 of 431 · 11% of its score
Beats [GPT-4o (2024-11-20)](gpt-4o-2024-11-20.md), [GPT-4 Turbo (2024-04-09)](gpt-4-turbo-2024-04-09.md), [Llama 3.1 405B Instruct](llama-3-1-405b.md) here, though they rank higher overall. Trails [DeepSeek R1 Distill Qwen 1.5B](deepseek-r1-distill-qwen-1-5b.md), [Gemma 7B (base)](gemma-7b-base.md), [CodeGemma 7B (v1.0)](codegemma-7b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
AIME 2025 (maker) 36.7% ▁·▁▁▁▂●▆█▇▅▁ beats 23/133 medium · avg 52% 7.5% sharp · at its level · small run
MATH-500 73.6% ▃····▃·●···▃ beats 3/5 easy · avg 79% 3.2% at its level
GSM8K loose substring accuracy 81.1% █·█··●██···█ beats 2/5 easy · avg 83% 0.5% blunt
other tests (16)
test result vs the others in its run difficulty counts why
Medmarks-V mean win rate (19 veri… 0.3548 ▁●▃▃▇▄█▃▅▁▁▁ beats 2/30 avg 0.418 17% sharp
GPQA Diamond 26.3% ●█····███··█ beats 0/5 medium · avg 52% 13% sharp · small run · several runs
MMLU 59.8% ▄●···█·····▄ beats 1/5 medium · avg 73% 7.8% at its level
Medmarks-V weighted mean win rate… 35.5% ▂●▅▂▆▃█▅█▅▃▆ beats 2/34 medium · avg 41% 6.6% blunt · at its level
MMLU-Pro 23.9% █●█···██···█ beats 1/5 medium · avg 42% 4.3% typical
IFBench (aggregation unspecified) 17.9% ●··███··█··█ beats 0/5 hard · avg 31% 4.1% typical
Multi-IF three-turn average 58.9% ▄··●▄··▄···▄ beats 2/5 easy · avg 70% 3.8% at its level
MMMLU seven languages (Liquid) 50.0% ▄·●····█···▄ beats 1/5 medium · avg 59% 3.4% at its level
AraGen-v3 3C3H score (Inception, … 16.2% ▁▁·▁▂▂●█▄▅▄▄ beats 14/66 hard · avg 29% 3.2% typical
MGSM six languages, loose (Liquid) 68.7% ▄·●·█···▄··▄ beats 1/5 easy · avg 81% 2.6% at its level
Global-MMLU (maker) 64.1% ▃▅●·▃···▅·▅█ beats 3/11 medium · avg 70% 2.4% at its level · small run
Open PT-LLM Leaderboard average (… 67.5% ▃▁▁▁▁▁▁▁▂●▅▁ beats 135/276 medium · avg 72% 2.1% at its level · noisy
GSM-Plus 58.9% █·····█●█·██ beats 2/5 easy · avg 83% 2.0% blunt · small run · several runs
MixEval-Hard (maker) 30.8% ●····█·····█ beats 0/2 medium · avg 39% 1.7% at its level · small run
EuroEval HellaSwag (English), tes… 59.0% ▃·▁▁▂▁▂●█▇▄▁ beats 32/100 medium · avg 67% 1.7% sharp · at its level · noisy
EuroEval Life in the UK (English … 75.7% ▁▁▁▁▁▁▁▂●█▄▃ beats 51/146 easy · avg 82% 1.4% at its level · noisy
source results share of the score
LFM2 Technical Report 11 41% ▇▇▇▇▇
HuggingFaceTB/SmolLM3-3B model card (su… 8 27% ▇▇▇
Medmarks: A Comprehensive Open-Source L… 2 23% ▇▇▇
Inception Arabic/Hindi leaderboards 1 3% ▇
EuroEval leaderboards 2 3% ▇
Open Portuguese LLM Leaderboard 1 2% ▇
prior: pull toward the middle 0% ▇
weighted average = 43.8
Licences: LFM2 Technical Report ([CC BY 4.0](../../licenses/cards-gap-NOTICE.txt)) · HuggingFaceTB/SmolLM3-3B model card ([Apache-2.0](../../licenses/HuggingFaceTB--SmolLM3-3B.txt)) · Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks ([CC BY 4.0](../../licenses/paper-2605.01417v1-CC-BY-4.0.txt)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt))
## How to run it
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run SmolLM3 3B (gguf-q4_k_m, needs ~2.35 GB, from [huggingface.co/ggml-org](https://huggingface.co/ggml-org/SmolLM3-3B-GGUF/tree/main))
```sh
llama cli -hf ggml-org/SmolLM3-3B-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf ggml-org/SmolLM3-3B-GGUF:Q4_K_M
```
Other versions: gguf-q8_0 (~4.2 GB) `llama cli -hf ggml-org/SmolLM3-3B-GGUF:Q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run SmolLM3 3B (gguf-q4_k_m, needs ~2.35 GB, from [huggingface.co/ggml-org](https://huggingface.co/ggml-org/SmolLM3-3B-GGUF/tree/main))
```sh
lms get https://huggingface.co/ggml-org/SmolLM3-3B-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: gguf-q8_0 (~4.2 GB) `lms get https://huggingface.co/ggml-org/SmolLM3-3B-GGUF@Q8_0 -y` · 4bit (~1.7 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/SmolLM3-3B-MLX-4bit -y` · 8bit (~3.3 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/SmolLM3-3B-MLX-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run SmolLM3 3B (gguf-q4_k_m, needs ~2.35 GB, from [huggingface.co/ggml-org](https://huggingface.co/ggml-org/SmolLM3-3B-GGUF/tree/main))
```sh
ollama run hf.co/ggml-org/SmolLM3-3B-GGUF:Q4_K_M
```
Other versions: gguf-q8_0 (~4.2 GB) `ollama run hf.co/ggml-org/SmolLM3-3B-GGUF:Q8_0`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run SmolLM3 3B (4bit, needs ~1.7 GB, Mac only, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/SmolLM3-3B-MLX-4bit/tree/2fd55434f0dcd0f22dada2a1fbf15b3fb8ef0842))
```sh
mlx_lm.chat --model lmstudio-community/SmolLM3-3B-MLX-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model lmstudio-community/SmolLM3-3B-MLX-4bit
```
Other versions: 8bit (~3.3 GB, Mac only) `mlx_lm.chat --model lmstudio-community/SmolLM3-3B-MLX-8bit`
Model repository and documentation: [huggingface.co/HuggingFaceTB](https://huggingface.co/HuggingFaceTB/SmolLM3-3B)
## Quantization evidence
How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent. They don't describe every download of that precision.
precision test full → quantized kept
FP8 W8A8 dynamic (weights sym… AIME 2024 pass@1 (64 samples; Lig… 0.453 → 0.475 104.8%
FP8 W8A8 dynamic (weights sym… Reasoning average of 3 tasks (AIM… 0.586 → 0.589 100.5%
FP8 W8A8 dynamic (weights sym… GPQA Diamond pass@1 (8 samples; L… 0.412 → 0.409 99.2%
FP8 W8A8 dynamic (weights sym… MATH-500 pass@1 (4 samples; Light… 0.893 → 0.883 98.9%
INT4 W4A16 (weights only, gro… GPQA Diamond pass@1 (8 samples; L… 0.412 → 0.419 101.6%
INT4 W4A16 (weights only, gro… MATH-500 pass@1 (4 samples; Light… 0.893 → 0.875 98.0%
INT4 W4A16 (weights only, gro… Reasoning average of 3 tasks (AIM… 0.586 → 0.562 95.9%
INT4 W4A16 (weights only, gro… AIME 2024 pass@1 (64 samples; Lig… 0.453 → 0.393 86.7%
Each kept value links to its source.
[interactive version](https://whatmodel.sh/model/smollm3-3b) · [JSON](https://whatmodel.sh/api?format=json&all=1)