[whatmodel.sh](../../README.md) / Gemma 4 12B
# Gemma 4 12B
Google · released 2026-06 · 11.9B params · apache-2.0
**score 62.0** ±0.8 · #183 of 732 · from 30 benchmark results across 3 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Gemma 4 12B among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█●▅▅▄▄▂▁ #164 of 313 21% 8
coding ▁▁▁▃▅▇█▇●▄▃▁ #135 of 469 6.8% 3
chat & writing ▁▁▁▃▆▄▆█▆▇●▃ #78 of 402* 0.9% 2
math ▁▁▃▅█▆●█▄▂▁▁ #166 of 431 19% 4
vision & documents ▁▁▂▂▁▃▄▂●▇█▄ #76 of 153 10% 4
web search ▅·▅▅●·▃▃▃·▅▃ #8 of 15* 1.2% 1
agent tasks · #164 of 313 · 21% of its score
Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md), [DeepSeek R1 0528](deepseek-r1-0528.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Jamba 1.5 Large](jamba-1-5-large.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Terminal-Bench Hard (maker) 18.0% ██··█·█·●··█ beats 4/5 very hard · avg 4.7% 6.0% sharp
τ²-bench Telecom (maker) 54.4% ▄····█··▄·●▄ beats 4/5 very hard · avg 13% 4.7% sharp · at its level · noisy
Tau2 – airline (maker) 75.0% ▃·▃··▃·····● beats 3/5 medium · avg 42% 3.4% at its level
τ²-bench aggregate (maker) 69.0% ▄·▄··▄····●▄ beats 4/5 hard · avg 34% 2.3% at its level
BFCL V4 (December 2025 archive) 37.4% ▁▁▁▂▆▆●▆█▂▄▃ beats 48/114 hard · avg 34% 1.5% at its level
τ³-bench 13.3% ▃·▃·█·●··▃·▃ beats 5/7 very hard · avg 13% 1.3% typical
BrowseComp 10.0% ▅···●·▃···▃▃ beats 4/7 very hard · avg 11% 1.2% at its level · noisy
Tau2 – retail (maker) 77.6% ▄····▄··▄·●█ beats 3/5 medium · avg 40% 1.1% at its level · noisy
coding · #135 of 469 · 6.8% of its score
Beats [Qwen3.5 Plus](qwen3-5-plus.md), [Qwen3-Coder-Next (80B-A3B)](qwen3-coder-next.md), [Gemini 2.5 Pro](gemini-2-5-pro.md) here, though they rank higher overall. Trails [Llama 3.1 Nemotron Ultra 253B v1](llama-3-1-nemotron-ultra-253b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SciCode (maker) 38.0% █▄·······●▄▄ beats 3/5 hard · avg 21% 2.7% at its level
SciCode (level unspecified) 39.8% █·····█████● beats 6/6 hard · avg 17% 2.2% at its level
LiveCodeBench v6 (Feb–May 2025) (… 72.0% ▄··▄·▄···●·█ beats 3/5 medium · avg 43% 1.9% at its level
chat & writing · #78 of 402 · 0.9% of its score
Beats [Claude Haiku 5.5](claude-haiku-5-5.md), [LongCat 2.0 (1.6T-A48B)](longcat-2-0.md), [Muse Glimmer 30B](muse-glimmer-30b.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
IFEval (maker) 97.2% ▄··▄··●···▄▄ beats 3/5 very easy · avg 94% 0.7% maxed out
IFEval (aggregation unspecified) 94.8% ▁▂▄▂▂▄▆██▅●▁ beats 140/145 easy · avg 73% 0.2% noisy
math · #166 of 431 · 19% of its score
Beats [MiMo-V2.5 (310B-A15B)](mimo-v2-5.md), [MiniMax-M2.7](minimax-m2-7.md), [MiniMax-M2.1](minimax-m2-1.md) here, though they rank higher overall. Trails [Olmo 3 7B Think](olmo-3-7b-think.md), [Qwen3-VL 4B Thinking](qwen3-vl-4b-thinking.md), [Nemotron 3 Nano 30B-A3B](nemotron-3-nano-30b-a3b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
HMMT February 2026 65.6% █·▄··▄··●█·▄ beats 4/7 hard · avg 32% 9.3% sharp · at its level
IMO-AnswerBench 57.2% ▄▄···▄·●··█▄ beats 3/6 hard · avg 32% 4.9% at its level
MATH-Vision (maker) 79.7% ██·█·····●██ beats 3/5 medium · avg 53% 2.6% at its level
AIME 2026 (maker) 77.5% ▄·▄▄····●··█ beats 3/5 medium · avg 40% 2.5% sharp · noisy
vision & documents · #76 of 153 · 10% of its score
Beats [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md), [GPT-5.4 nano](gpt-5-4-nano.md), [Nemotron 3.5 Super VL 120B A12B](nemotron-3-5-super-vl-120b-a12b.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
MMMU Pro (maker) 69.1% ██·█····●·██ beats 3/5 medium · avg 48% 3.7% at its level
MATH-Vision (maker) 79.7% ██·█·····●██ beats 3/5 medium · avg 53% 2.6% at its level
InfographicVQA (maker) 88.4% ▄█·······●·▄ beats 3/5 easy · avg 70% 2.5% typical
OmniDocBench 1.5 ↓ (maker) 16.4% █·█·····█●██ beats 3/5 avg 31% 1.4% blunt
web search · #8 of 15 · 1.2% of its score
Beats [Qwen3.5 9B](qwen3-5-9b.md) here, though they rank higher overall. Trails [Spark-X2.5 1.7B](spark-x2-5-1-7b.md), [Granite 4.2 3B](granite-4-2-3b.md), [Spark-X2.6 1.7B](spark-x2-6-1-7b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
BrowseComp 10.0% ▅···●·▃···▃▃ beats 4/7 very hard · avg 11% 1.2% at its level · noisy
other tests (10)
test result vs the others in its run difficulty counts why
Humanity's Last Exam 13.1% ▄▄·▄··▄·▄·▄● beats 6/7 very hard · avg 3.8% 8.1% typical
GPQA Diamond (maker) 78.8% █···▄·····●█ beats 3/5 hard · avg 46% 7.5% sharp · at its level
GPQA (subset unspecified) 72.8% █···▃···▅·●▃ beats 6/7 hard · avg 44% 5.7% sharp · at its level
MedXPertQA MM (maker) 48.7% ▂···▂·▃··●▂█ beats 5/13 very hard · avg 30% 5.5% sharp · at its level
MMLU-Pro (maker) 77.2% ▄··█···●··▄▄ beats 3/5 medium · avg 60% 3.9% typical
MMMLU (maker) 83.4% ██··█···●·██ beats 3/5 medium · avg 69% 3.5% typical
MRCR v2 8-needle, 128k (maker) 43.4% ▄▄·▄···●···▄ beats 3/5 hard · avg 22% 3.2% at its level
IFBench (maker) 74.0% ▄▄·▄······▄● beats 4/5 medium · avg 40% 2.7% at its level
Big Bench Extra Hard micro avg (m… 53.0% █··▄···●·▄·▄ beats 3/5 hard · avg 28% 2.5% at its level
HLE (maker) 5.2% ●···█······█ beats 0/2 very hard · avg 4.0% 1.0% small run
source results share of the score
Gemma Team (submitted / maker-reported) 20 63% ▇▇▇▇▇▇▇▇
XHToken/Spark-X2.5-4B model card 9 34% ▇▇▇▇
google/gemma-4-31B-it model card (submi… 1 2% ▇
prior: pull toward the middle 0% ▇
weighted average = 62.0
Licences: Gemma Team ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · XHToken/Spark-X2.5-4B model card ([Apache-2.0](https://huggingface.co/XHToken/Spark-X2.5-4B/blob/0bcb35678590218655dff3765b9e61c83b35e9c4/README.md)) · google/gemma-4-31B-it model card ([Apache-2.0](../../licenses/google--gemma-4-31B-it.txt))
## How to run it
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Gemma 4 12B (Q4_K_M, needs ~7.4 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/gemma-4-12B-it-GGUF/tree/40b870babe39398ce917bbf4b4ff9b5a1f12710e))
```sh
llama cli -hf lmstudio-community/gemma-4-12B-it-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf lmstudio-community/gemma-4-12B-it-GGUF:Q4_K_M
```
Other versions: Q8_0 (~12.7 GB) `llama cli -hf lmstudio-community/gemma-4-12B-it-GGUF:Q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Gemma 4 12B (Q4_K_M, needs ~7.4 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/gemma-4-12B-it-GGUF/tree/40b870babe39398ce917bbf4b4ff9b5a1f12710e))
```sh
lms get https://huggingface.co/lmstudio-community/gemma-4-12B-it-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~12.7 GB) `lms get https://huggingface.co/lmstudio-community/gemma-4-12B-it-GGUF@Q8_0 -y` · 4bit (~6.7 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/gemma-4-12B-it-MLX-4bit -y` · 8bit (~12.7 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/gemma-4-12B-it-MLX-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Gemma 4 12B (Q4_K_M, needs ~7.64 GB, from [ollama.com](https://ollama.com/library/gemma4/tags))
```sh
ollama run gemma4:12b
```
Other versions: qat (~7.64 GB) `ollama run gemma4:12b-it-qat`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run Gemma 4 12B (4bit, needs ~6.7 GB, Mac only, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/gemma-4-12B-it-MLX-4bit/tree/f45bda5639cf187cd5f127a5484813f5f1e3a6ad))
```sh
mlx_lm.chat --model lmstudio-community/gemma-4-12B-it-MLX-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model lmstudio-community/gemma-4-12B-it-MLX-4bit
```
Other versions: 8bit (~12.7 GB, Mac only) `mlx_lm.chat --model lmstudio-community/gemma-4-12B-it-MLX-8bit`
Model repository and documentation: [huggingface.co/google](https://huggingface.co/google/gemma-4-12B-it)
## Published speed measurements
Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine.
hardware precision speed engine and settings
AMD Ryzen AI Max+ 395 (Stri… Gemma QAT GGUF … 29.34 tok/s llama.cpp ac4cddeb0 (build_number…
AMD Radeon 780M Graphics, 6… q4_k_m · layers… 6.49 tok/s ollama (version not recorded) · l…
Each speed links to its source, which has the full settings.
[interactive version](https://whatmodel.sh/model/gemma-4-12b) · [JSON](https://whatmodel.sh/api?format=json&all=1)