[whatmodel.sh](../../README.md) / Gemma 3 1B
# Gemma 3 1B
Google · released 2025-03 · 1B params · gemma
**score 33.2** ±0.7 · #642 of 732 · from 51 benchmark results across 13 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Gemma 3 1B among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ●▁▃▆█▄▅▅▄▄▂▁ #313 of 313 2.3% 2
coding ▁▁▁●▅▇█▇▅▄▃▁ #446 of 469 10% 4
chat & writing ▁▁▁▃▆●▆█▆▇▇▃ #307 of 402 7.5% 4
math ▁▁▃●█▆▄█▄▂▁▁ #368 of 431 34% 6
agent tasks · #313 of 313 · 2.3% of its score
Trails [Falcon3 1B Instruct](falcon3-1b.md), [Granite 4.0 350M](granite-4-0-350m.md), [Granite 4.0 H 350M](granite-4-0-h-350m.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
BFCL v3 16.3% ●▁▁▁▁▃▅█▅▄▁▁ beats 0/114 medium · avg 51% 1.3% blunt · at its level · several runs
BFCL V4 (December 2025 archive) 7.2% ●▁▁▂██▅▅▆▂▂▂ beats 0/77 hard · avg 31% 1.0% typical
coding · #446 of 469 · 10% of its score
Beats [Qwen3.5 2B](qwen3-5-2b.md), [Molmo 7B-D (0924)](molmo-7b-d.md), [Tulu 2 7B](tulu-2-7b.md) here, though they rank higher overall. Trails [Falcon3 1B Instruct](falcon3-1b.md), [Granite 3.0 1B-A400M Instruct](granite-3-0-1b-a400m.md), [Llama 3.2 1B Instruct](llama-3-2-1b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
HumanEval (maker) 41.5% ▁▁▁▁▂▂●▆█▅▂▁ beats 70/205 easy · avg 83% 5.7% sharp · at its level
MBPP (maker) 35.2% ▁▁▁●▅█▆▅▄▁▂▁ beats 10/86 easy · avg 70% 2.4% at its level
LiveCodeBench (version unspecifie… 1.9% ●···▁▁▁█▇▃▄▁ beats 0/38 hard · avg 23% 0.8% too hard for it
BIRD-SQL (dev) (maker) 6.4% ●·······▂▂█▆ beats 0/11 medium · avg 43% 0.6% typical
chat & writing · #307 of 402 · 7.5% of its score
Beats [Ministral 3 14B Reasoning 2512](ministral-3-14b-reasoning.md), [Gemma 4 E4B](gemma-4-e4b.md), [OpenThinker3 7B](openthinker3-7b.md) here, though they rank higher overall. Trails [LFM2 350M](lfm2-350m.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
IFEval (aggregation unspecified) 62.9% ▄····●▄···▄█ beats 1/6 easy · avg 82% 3.0% at its level · several runs
IFEval mean of four measures 63.2% █·●··█·██·██ beats 1/6 very easy · avg 86% 2.3% at its level
IFEval prompt-level loose 60.6% ▁▃▁●▁▄▁▁█▃▃▁ beats 4/22 easy · avg 78% 1.3% at its level
AlpacaEval 2 length-controlled 20.4% ▃▃·●▁█▇▅·▁·▁ beats 4/22 medium · avg 39% 1.0% sharp
math · #368 of 431 · 34% of its score
Beats [GPT-3.5 Turbo (0125)](gpt-3-5-turbo-0125.md), [DBRX Instruct](dbrx-instruct.md), [Command R Plus (04-2024)](command-r-plus-04-2024.md) here, though they rank higher overall. Trails [OPT-1.3B](opt-1-3b.md), [OPT-2.7B](opt-2-7b.md), [Gemma 2B (base)](gemma-2b-base.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
GSM8K 35.0% ●▃··▃▂▅█▆▃▂▃ beats 0/22 very easy · avg 93% 21% sharp · at its level · several runs
MGSM 43.6% █···●▄··▄··▄ beats 3/6 easy · avg 83% 6.6% sharp · at its level
MATH (subset unspecified) 40.3% ▃▃●▂▂▆·····▁ beats 8/22 easy · avg 82% 5.1% at its level · several runs
MATH-500 45.2% ▄··●▄·▄···▄▄ beats 2/6 easy · avg 77% 1.1% at its level · noisy
AIME 2025 1.0% ▄●··▄···▄··█ beats 2/6 hard · avg 30% 0.5% sharp · too hard for it · noisy
OTIS mock AIME 2024–2025 1.1% ●▃▄▃▃▃▇▇█▃▂▅ beats 10/197 very hard · avg 7.2% 0.0% sharp · too hard for it · noisy
other tests (22)
test result vs the others in its run difficulty counts why
MMLU 38.9% ▁●·▁·▁▁█·▃▃▅ beats 1/22 easy · avg 75% 12% several runs
BIG-Bench Hard 39.4% ▁●·▁▁·▅█·▄▄▁ beats 1/22 easy · avg 79% 8.0% sharp · several runs
IFBench (aggregation unspecified) 20.5% ▄·●·█······█ beats 1/6 medium · avg 38% 2.7% several runs
MIMIC-CDM full information (4 pat… 21.9% ●·······▄█·▄ beats 0/4 easy · avg 71% 2.4% at its level
MMLU-Pro 14.0% ●···█·██·███ beats 0/6 medium · avg 52% 2.1% small run · several runs
GPQA Diamond 19.9% ●▁▁▁▂▄▆▄▅▇█▅ beats 3/211 hard · avg 37% 2.0% sharp · too hard for it · small run · s…
Open Persian LLM Leaderboard v2 a… 37.5% ▂▄●▃█▄▃▆▃▅▂▂ beats 12/81 medium · avg 57% 2.0% blunt · at its level
MIMIC-CDM clinical decision makin… 21.4% ●······█··██ beats 0/3 medium · avg 60% 1.9% at its level
Global-MMLU-Lite (maker) 34.2% ●···▂·▂▂▂·██ beats 0/12 medium · avg 64% 1.8% typical
BRIDGE clinical text, zero-shot a… 15.7% ▁▁▁●▁▂▂▃▄▆█▃ beats 5/97 hard · avg 31% 1.7% blunt
FACTS Grounding (maker) 36.4% ●········▄·█ beats 0/3 easy · avg 72% 1.7% at its level
MMMLU 34.4% ▄···●·▄▄··▄█ beats 1/6 medium · avg 57% 1.6% typical
Open PT-LLM Leaderboard average (… 54.2% ▃▁▁▁▁▁▁●▂█▅▁ beats 74/276 medium · avg 72% 0.8% noisy
GPQA (subset unspecified) 21.1% ●······▄▄█·▄ beats 0/6 hard · avg 37% 0.8% sharp · too hard for it · noisy
Big Bench Extra Hard micro avg (m… 7.2% ●▄·█▄·▄·▄·▄▄ beats 0/8 hard · avg 18% 0.8% too hard for it · small run
SimpleQA (maker) 2.2% ●▄▄▆█▆███·▂█ beats 0/32 very hard · avg 6.8% 0.7% too hard for it
PopQA 9.6% ●▁▁▃▃▃▁█▁▃·▄ beats 0/21 hard · avg 31% 0.7% typical
EuroEval Life in the UK (English … 43.7% ▁▁▁▁▁▁●▂▄█▄▃ beats 15/146 easy · avg 82% 0.7% at its level · noisy
WMT24++ (chrF) (maker) 35.9% ●······█··██ beats 0/3 medium · avg 52% 0.7% blunt · at its level
Vectara hallucination leaderboard… 94.7% ▁▁▂▃▄●█▆▃▂▂▁ beats 58/140 very easy · avg 94% 0.6% blunt · noisy
TruthfulQA MC2 43.8% ●▃▂·▃▃·▅█▂▅▂ beats 1/21 medium · avg 60% 0.2% blunt · at its level
Chess Puzzles (Epoch) 0.0% ●▁▁▁▁▂▃▃▃▂▁▁ beats 29/137 very hard · avg 5.8% 0.0% sharp · too hard for it · noisy
source results share of the score
Google DeepMind Gemma 3 270M IT model c… 13 33% ▇▇▇▇
LiquidAI/LFM2-1.2B model card 7 19% ▇▇
2 OLMo 2 Furious 8 19% ▇▇
LiquidAI/LFM2.5-1.2B-Thinking model card 8 13% ▇▇
Gemma Team (submitted / maker-reported) 3 4% ▇
MIMIC Clinical Decision Making leaderbo… 2 4% ▇
Open Persian LLM Leaderboard v2 1 2% ▇
Berkeley Function Calling Leaderboard r… 2 2% ▇
BRIDGE Medical Leaderboard 1 2% ▇
Open Portuguese LLM Leaderboard 1 1% ▇
EuroEval leaderboards 1 1% ▇
Vectara hallucination leaderboard 1 1% ▇
Epoch AI 3 1% ▇
prior: pull toward the middle 0% ▇
weighted average = 33.2
Licences: Google DeepMind Gemma 3 270M IT model card ([CC-BY-4.0](../../licenses/cards-final-NOTICE.txt)) · LiquidAI/LFM2-1.2B model card ([LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM2-1.2B/blob/40f3da0d0164913923aee9462c23077868b816a3/LICENSE)) · 2 OLMo 2 Furious ([CC BY 4.0](../../licenses/paper-2501.00656v3-CC-BY-4.0.txt)) · LiquidAI/LFM2.5-1.2B-Thinking model card ([LFM Open License v1.0](../../licenses/cards-gap-NOTICE.txt)) · Gemma Team ([CC BY 4.0](../../licenses/report-2503.19786v1.txt)) · MIMIC Clinical Decision Making leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · BRIDGE Medical Leaderboard ([MIT](../../licenses/bridge-medical-leaderboard-MIT.txt)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/))
## How to run it
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Gemma 3 1B (Q4_K_M, needs ~0.8 GB, from [huggingface.co/ggml-org](https://huggingface.co/ggml-org/gemma-3-1b-it-GGUF/tree/f9c28bcd85737ffc5aef028638d3341d49869c27))
```sh
llama cli -hf ggml-org/gemma-3-1b-it-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf ggml-org/gemma-3-1b-it-GGUF:Q4_K_M
```
Other versions: Q8_0 (~1.1 GB) `llama cli -hf ggml-org/gemma-3-1b-it-GGUF:Q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Gemma 3 1B (Q4_K_M, needs ~0.8 GB, from [huggingface.co/ggml-org](https://huggingface.co/ggml-org/gemma-3-1b-it-GGUF/tree/f9c28bcd85737ffc5aef028638d3341d49869c27))
```sh
lms get https://huggingface.co/ggml-org/gemma-3-1b-it-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~1.1 GB) `lms get https://huggingface.co/ggml-org/gemma-3-1b-it-GGUF@Q8_0 -y` · 4bit (~0.7 GB, Mac only) `lms get https://huggingface.co/mlx-community/gemma-3-1b-it-4bit -y` · 8bit (~1.4 GB, Mac only) `lms get https://huggingface.co/mlx-community/gemma-3-1b-it-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Gemma 3 1B (Q4_K_M, needs ~1.1 GB, from [ollama.com](https://ollama.com/library/gemma3/tags))
```sh
ollama run gemma3:1b
```
Other versions: qat (~1.1 GB) `ollama run gemma3:1b-it-qat`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run Gemma 3 1B (4bit, needs ~0.7 GB, Mac only, from [huggingface.co/mlx-community](https://huggingface.co/mlx-community/gemma-3-1b-it-4bit/tree/2d44e83dc9e80843d22fb941d3d699a0b1351aa6))
```sh
mlx_lm.chat --model mlx-community/gemma-3-1b-it-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model mlx-community/gemma-3-1b-it-4bit
```
Other versions: 8bit (~1.4 GB, Mac only) `mlx_lm.chat --model mlx-community/gemma-3-1b-it-8bit`
Model repository and documentation: [huggingface.co/google](https://huggingface.co/google/gemma-3-1b-it)
## Published speed measurements
Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine.
hardware precision speed engine and settings
Intel CoffeeLake-S GT2 [UHD… q4_k_m 14.84 tok/s ollama (version not recorded) · l…
DGX Spark (GB10), 122 GB qat 180.37 tok/s ollama (version not recorded) · l…
Each speed links to its source, which has the full settings.
[interactive version](https://whatmodel.sh/model/gemma-3-1b) · [JSON](https://whatmodel.sh/api?format=json&all=1)