[whatmodel.sh](../../README.md) / Gemma 4 E4B
# Gemma 4 E4B
Google · released 2026-04 · 8B params (4.5B active) · apache-2.0
**score 52.8** ±0.5 · #320 of 732 · from 77 benchmark results across 14 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Gemma 4 E4B among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃●█▄▅▅▄▄▂▁ #244 of 313 18% 15
coding ▁▁▁▃▅▇●▇▅▄▃▁ #270 of 469 16% 10
chat & writing ▁▁▁▃●▄▆█▆▇▇▃ #330 of 402 1.3% 2
math ▁▁▃▅●▆▄█▄▂▁▁ #259 of 431 23% 6
vision & documents ▁▁▂▂▁●▄▂▆▇█▄ #117 of 153 5.4% 4
web search ▅·●▅█·▃▃▃·▅▃ #13 of 15* 1.1% 2
agent tasks · #244 of 313 · 18% of its score
Beats [o3-mini](o3-mini.md), [Llama 4 Maverick (402B-A17B)](llama-4-maverick.md), [DeepSeek R1 0528 Qwen3 8B](deepseek-r1-0528-qwen3-8b.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Phi-3.5-mini-instruct](phi-3-5-mini-instruct.md), [Jamba 1.5 Mini](jamba-1-5-mini.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
BFCL V4 (December 2025 archive) 47.0% ▄·█·●▄▄▄·▄·▄ beats 3/8 medium · avg 43% 3.6% at its level · several runs
τ²-bench Telecom (maker) 18.4% ▄····●··▄·▄▄ beats 1/5 very hard · avg 13% 1.9% sharp · noisy · several runs
SWE-bench Verified 15.0% ▄··▄▄█··●·▄█ beats 5/8 hard · avg 16% 1.8% sharp · noisy · several runs
Tau2 – airline (maker) 52.0% ▃·▃··●·····█ beats 2/5 medium · avg 42% 1.7% at its level
ContractBench success rate (33 ob… 17.2% ▆··●▄▂▄▂▆▆█▄ beats 3/23 very hard · avg 13% 1.5% sharp
SWE-bench Pro (split unspecified) 3.3% █▂···●···▄·▂ beats 5/8 very hard · avg 7.2% 1.3% sharp · noisy · several runs
Terminal-Bench Hard (maker) 8.0% ██··●·█·█··█ beats 2/5 very hard · avg 4.7% 1.3% sharp · too hard for it
τ²-bench aggregate (maker) 42.2% ▄·▄··●····█▄ beats 2/5 hard · avg 34% 1.0% at its level
τ³-bench Banking 4.1% ▂▂·●█▂▃▃▃▆▂▅ beats 4/24 very hard · avg 4.0% 1.0% blunt · too hard for it · several runs
Terminal-Bench 2.1 3.4% ▁·●·▁▁▄▃▆█▅█ beats 1/63 very hard · avg 6.2% 0.9% sharp
BrowseComp-ZH 7.0% ▄▄·●▄▄·█···█ beats 2/8 very hard · avg 13% 0.8% noisy
Tau2 – retail (maker) 67.1% ▄····▄··●·▄█ beats 2/5 medium · avg 40% 0.6% at its level · noisy
τ³-bench 10.1% ▃·▃·●·▃··▃·▃ beats 4/7 very hard · avg 13% 0.4% typical
BrowseComp 8.3% ▅···●·▃···▃▃ beats 2/7 very hard · avg 11% 0.3% noisy
T1-Bench multi-domain conversatio… 35.6% ▂▂··▂·▂·▂▂●▄ beats 9/11 hard · avg 25% 0.0% blunt · at its level · noisy
coding · #270 of 469 · 16% of its score
Beats [Qwen3-VL 32B Instruct](qwen3-vl-32b-instruct.md), [Solar Open 100B (102B-A12B)](solar-open-100b.md), [GLM-4.7-Flash](glm-4-7-flash.md) here, though they rank higher overall. Trails [DeepSeek LLM 67B Chat](deepseek-llm-67b-chat.md), [QwQ-32B-Preview](qwq-32b-preview.md), [Phi-3 Medium 128K Instruct](phi-3-medium-128k-instruct.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LiveCodeBench v6 (window unspecif… 53.9% ▃····█·▃●▃·▃ beats 5/8 medium · avg 38% 5.5% at its level · several runs
HumanEval+ 89.1% ▁·▁▁▂▃▄▄█▅●▃ beats 129/145 easy · avg 80% 1.8% sharp
SWE-bench Verified 15.0% ▄··▄▄█··●·▄█ beats 5/8 hard · avg 16% 1.8% sharp · noisy · several runs
LiveCodeBench Pro 2025Q2 (Easy) 45.8% ▄···█▄··●█▄▄ beats 4/8 hard · avg 33% 1.6% at its level
SWE-bench Pro (split unspecified) 3.3% █▂···●···▄·▂ beats 5/8 very hard · avg 7.2% 1.3% sharp · noisy · several runs
SciCode (maker) 24.0% █●·······▄▄▄ beats 2/5 hard · avg 21% 1.0% typical
LiveCodeBench Pro 2025Q2 (Medium) 1.8% █···●··▄▂··▂ beats 4/8 very hard · avg 2.0% 0.9% sharp · too hard for it · noisy
LiveCodeBench v6 (Feb–May 2025) (… 52.0% ▄··▄·●···▄·█ beats 2/5 medium · avg 43% 0.9% at its level
SciCode (level unspecified) 27.5% █·····███●██ beats 4/6 hard · avg 17% 0.7% typical
MBPP+ 70.9% ▁··▁▁▃▄█●▄▁▁ beats 86/120 medium · avg 65% 0.7% at its level
chat & writing · #330 of 402 · 1.3% of its score
Trails [LFM2 350M](lfm2-350m.md), [Gemma 3 1B](gemma-3-1b.md), [Qwen3 0.6B](qwen3-0-6b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
IFEval (aggregation unspecified) 44.4% ▄●····▄··███ beats 1/8 easy · avg 81% 0.8% several runs
IFEval (maker) 96.7% ▄··▄··●···▄▄ beats 2/5 very easy · avg 94% 0.5% maxed out
math · #259 of 431 · 23% of its score
Beats [GPT-4o (2024-11-20)](gpt-4o-2024-11-20.md), [GPT-4 Turbo (2024-04-09)](gpt-4-turbo-2024-04-09.md), [Llama 3.1 405B Instruct](llama-3-1-405b.md) here, though they rank higher overall. Trails [DeepSeek R1 Distill Qwen 1.5B](deepseek-r1-distill-qwen-1-5b.md), [Gemma 7B (base)](gemma-7b-base.md), [CodeGemma 7B (v1.0)](codegemma-7b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
HMMT February 2026 30.1% ▅··●▃·▃··▃·█ beats 2/8 hard · avg 32% 8.3% sharp · at its level · several runs
AIME 2025 37.1% █●▄▄▄····█·▄ beats 2/8 medium · avg 42% 5.1% sharp · at its level · several runs
AIME 2026 45.0% ▄·▄●·▄▄···█▄ beats 2/8 medium · avg 48% 4.2% sharp · at its level · several runs
IMO-AnswerBench 26.9% ▄●···▄·▄··█▄ beats 1/6 hard · avg 32% 2.3% at its level
MATH-500 88.2% █●▄▄▄··▄···▄ beats 2/8 very easy · avg 89% 1.5% typical
MATH-Vision (maker) 59.5% ██·●·····███ beats 2/5 medium · avg 53% 1.4% at its level
vision & documents · #117 of 153 · 5.4% of its score
Beats [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md), [GPT-4 Turbo (2024-04-09)](gpt-4-turbo-2024-04-09.md), [Pixtral Large (2411)](pixtral-large.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
MMMU Pro (maker) 52.6% ██·●····█·██ beats 2/5 medium · avg 48% 1.8% at its level
InfographicVQA (maker) 70.0% ▄●·······█·▄ beats 1/5 easy · avg 70% 1.6% at its level
MATH-Vision (maker) 59.5% ██·●·····███ beats 2/5 medium · avg 53% 1.4% at its level
OmniDocBench 1.5 ↓ (maker) 18.1% █·█·····●███ beats 2/5 avg 31% 0.6% blunt
web search · #13 of 15 · 1.1% of its score
Beats [Nemotron 3 Nano 4B](nemotron-3-nano-4b.md) here, though they rank higher overall. Trails [Qwen3.5 2B](qwen3-5-2b.md), [LFM2.5 8B-A1B](lfm2-5-8b-a1b.md), [LFM2.5 2.6B](lfm2-5-2-6b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
BrowseComp-ZH 7.0% ▄▄·●▄▄·█···█ beats 2/8 very hard · avg 13% 0.8% noisy
BrowseComp 8.3% ▅···●·▃···▃▃ beats 2/7 very hard · avg 11% 0.3% noisy
other tests (24)
test result vs the others in its run difficulty counts why
GPQA Diamond (maker) 58.6% █···●·····▄█ beats 2/5 hard · avg 43% 7.1% sharp · at its level · several runs
MMLU-Pro 68.3% ▂··▂█·●▂···▂ beats 6/8 medium · avg 60% 4.1% at its level · several runs
MMLU-Redux 83.7% ▂···█··●▂··▂ beats 6/8 easy · avg 79% 3.2% several runs
GPQA (subset unspecified) 54.5% █···●···▅·▃▃ beats 3/7 hard · avg 44% 2.4% sharp · at its level
MMMLU (maker) 76.6% ██··●···█·██ beats 2/5 medium · avg 69% 2.1% at its level
EuroEval Life in the UK (English … 81.5% ▂·▂▂·●·▃▃·█▃ beats 3/14 easy · avg 80% 2.0% typical
Humanity's Last Exam 3.9% ▄▄·●··▄·▄·▄█ beats 2/7 very hard · avg 3.8% 1.8% too hard for it
BRIDGE clinical text, zero-shot a… 39.2% ▁▁▁▁▁▂▂▃▄▆●▃ beats 65/97 hard · avg 31% 1.7% blunt · at its level
MedXPertQA MM (maker) 28.7% ▂···▂·●··▆▂█ beats 3/13 very hard · avg 30% 1.6% sharp
IFBench (aggregation unspecified) 28.3% ●····▃▃·█·▃▃ beats 1/8 medium · avg 39% 1.5% at its level · several runs
Open Persian LLM Leaderboard v2 a… 59.0% ▂▄▂▃█▄●▆▃▅▂▂ beats 48/81 medium · avg 57% 1.4% blunt · at its level
IFBench (maker) 44.0% ▄▄·●······▄█ beats 2/5 medium · avg 40% 1.3% at its level
EuroEval HellaSwag (English), val… 70.0% ▂··▂▂·▂●▆█·▄ beats 4/15 medium · avg 63% 1.2% at its level
MRCR v2 8-needle, 128k (maker) 25.4% ▄▄·●···█···▄ beats 2/5 hard · avg 22% 1.2% at its level
OJBench 19.0% ▃····▃▅··●▃▃ beats 4/8 very hard · avg 12% 1.1% typical
KOPA-Bench (pass@1) 18.6% ▄·●··▄··█▄·█ beats 2/8 very hard · avg 14% 1.0% typical
Big Bench Extra Hard micro avg (m… 33.1% █··●···▄·▄·▄ beats 2/5 hard · avg 28% 1.0% at its level
SuperGPQA 38.7% ▃·▃·▃▅●····▃ beats 5/8 hard · avg 32% 1.0% at its level
LongBench v2 42.7% ▃······▅▅▃·● beats 6/8 very hard · avg 32% 0.8% typical
Multi-IF (aggregation unspecified) 45.9% ▄●···▄·▄·▄██ beats 1/8 medium · avg 60% 0.7% at its level · several runs
BFCL multi-turn (version not stat… 20.8% ●▄···▄·▄█▄▄▄ beats 0/8 hard · avg 26% 0.6% at its level
Claw-Eval average score 58.0% ▄·●··█▄▄▄▄▄▄ beats 1/9 medium · avg 53% 0.6% blunt · at its level
NTCIR SciClaimEval Subtask 1 pair… 52.6% ▃●···▃·▃▃█·▃ beats 1/8 medium · avg 45% 0.5% blunt · at its level
GSM-Plus 87.4% ▁··▁▁▁·▃▁▄●▁ beats 11/16 easy · avg 83% 0.5% blunt
source results share of the score
openbmb/MiniCPM5-2B model card 20 27% ▇▇▇
Gemma Team (submitted / maker-reported) 19 27% ▇▇▇
JetBrains/Mellum2.1-12B-A2.5B-Thinking … 11 14% ▇▇
XHToken/Spark-X2.5-4B model card 9 13% ▇▇
LiquidAI/LFM2.5-2.6B model card 7 6% ▇
EuroEval leaderboards 2 3% ▇
BRIDGE Medical Leaderboard 1 2% ▇
Multi-Step Tool-Calling over Korean Ope… 2 2% ▇
ContractBench: Can LLM Agents Preserve … 1 2% ▇
Open Persian LLM Leaderboard v2 1 1% ▇
google/gemma-4-31B-it model card (submi… 1 1% ▇
SciTrue: Reliable Scientific Claim Vali… 1 1% ▇
LiquidAI/LFM2.5-8B-A1B model card 1 0% ▇
T1-Bench: Benchmarking Multi-Scenario A… 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 52.8
Licences: openbmb/MiniCPM5-2B model card ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Gemma Team ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · JetBrains/Mellum2.1-12B-A2.5B-Thinking model card ([Apache-2.0](https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking/blob/92ddae9fc7665e9f801d141d2e5a6b2caf2460c4/README.md)) · XHToken/Spark-X2.5-4B model card ([Apache-2.0](https://huggingface.co/XHToken/Spark-X2.5-4B/blob/0bcb35678590218655dff3765b9e61c83b35e9c4/README.md)) · LiquidAI/LFM2.5-2.6B model card ([LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM2.5-2.6B/blob/654f9463ce32b05d0429d76fe1f580b27d4c1ac0/LICENSE)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · BRIDGE Medical Leaderboard ([MIT](../../licenses/bridge-medical-leaderboard-MIT.txt)) · Multi-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · ContractBench: Can LLM Agents Preserve Observation Contracts? ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · google/gemma-4-31B-it model card ([Apache-2.0](../../licenses/google--gemma-4-31B-it.txt)) · SciTrue: Reliable Scientific Claim Validation with Frontier and Open Language Models at the NTCIR SciClaimEval Task ([CC BY 4.0](../../licenses/paper-2609.00654v1-CC-BY-4.0.txt)) · LiquidAI/LFM2.5-8B-A1B model card ([LFM Open License v1.0](../../licenses/cards-gap-NOTICE.txt)) · T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/))
## How to run it
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Gemma 4 E4B (Q4_K_M, needs ~5.3 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/gemma-4-E4B-it-GGUF/tree/99210a71094c6aa8559f764e0cadedd7d89de94d))
```sh
llama cli -hf lmstudio-community/gemma-4-E4B-it-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf lmstudio-community/gemma-4-E4B-it-GGUF:Q4_K_M
```
Other versions: Q8_0 (~8 GB) `llama cli -hf lmstudio-community/gemma-4-E4B-it-GGUF:Q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Gemma 4 E4B (Q4_K_M, needs ~5.3 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/gemma-4-E4B-it-GGUF/tree/99210a71094c6aa8559f764e0cadedd7d89de94d))
```sh
lms get https://huggingface.co/lmstudio-community/gemma-4-E4B-it-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~8 GB) `lms get https://huggingface.co/lmstudio-community/gemma-4-E4B-it-GGUF@Q8_0 -y` · 4bit (~5.1 GB, Mac only) `lms get https://huggingface.co/AtomicChat/gemma-4-E4B-it-MLX-4bit -y` · 8bit (~8.9 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/gemma-4-E4B-it-MLX-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Gemma 4 E4B (Q4_K_M, needs ~5.3 GB, from [ollama.com](https://ollama.com/library/gemma4/tags))
```sh
ollama run gemma4:e4b
```
Other versions: qat (~5.3 GB) `ollama run gemma4:e4b-it-qat`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run Gemma 4 E4B (4bit, needs ~5.1 GB, Mac only, from [huggingface.co/AtomicChat](https://huggingface.co/AtomicChat/gemma-4-E4B-it-MLX-4bit/tree/0e86fdca66729ef34be8445e7b52e4c6cf2d8d7c))
```sh
mlx_lm.chat --model AtomicChat/gemma-4-E4B-it-MLX-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model AtomicChat/gemma-4-E4B-it-MLX-4bit
```
Other versions: 8bit (~8.9 GB, Mac only) `mlx_lm.chat --model lmstudio-community/gemma-4-E4B-it-MLX-8bit`
Model repository and documentation: [huggingface.co/google](https://huggingface.co/google/gemma-4-E4B-it)
## Published speed measurements
Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine.
hardware precision speed engine and settings
Apple M5 Pro (64GB; CPU/GPU… BF16 (paper's m… 24.3 tok/s vllm-metal (Metal; version not st…
Apple M5 Pro (64GB; CPU/GPU… BF16 (paper's m… 24.1 tok/s vllm-mlx (MLX; version not stated…
Apple M5 Pro (64GB; CPU/GPU… BF16 (paper's m… 22.6 tok/s mlx_lm server (MLX; version not s…
Apple M5 Pro (64GB; CPU/GPU… BF16 (paper's m… 22.4 tok/s llama.cpp server (Metal; --parall…
Apple M5 Pro (64GB; CPU/GPU… BF16 (paper's m… 14.1 tok/s Hugging Face Transformers (PyTorc…
Apple M5 Pro (64GB; CPU/GPU… BF16 (paper's m… 21.6 tok/s omlx (MLX; SSD prefix cache, in-m…
Apple M5 Pro (64GB; CPU/GPU… BF16 (paper's m… 21.1 tok/s vllm-metal (Metal; version not st…
Apple M5 Pro (64GB; CPU/GPU… BF16 (paper's m… 19.4 tok/s vllm-mlx (MLX; version not stated…
Apple M5 Pro (64GB; CPU/GPU… BF16 (paper's m… 18.3 tok/s llama.cpp server (Metal; --parall…
NVIDIA DGX Spark (GB10) BF16 20.1 tok/s llama.cpp (CUDA build, GGUF BF16 …
NVIDIA DGX Spark (GB10) BF16 18.1 tok/s llama.cpp (CUDA build, GGUF BF16 …
NVIDIA DGX Spark (GB10) BF16 17.2 tok/s vLLM (CUDA, Safetensors BF16; ver…
NVIDIA DGX Spark (GB10) BF16 16.1 tok/s vLLM (CUDA, Safetensors BF16; ver…
NVIDIA DGX Spark (GB10) BF16 16 tok/s SGLang (CUDA, Safetensors BF16; v…
DGX Spark (GB10), 122 GB q4_k_m 59.1 tok/s ollama (version not recorded) · l…
NVIDIA GeForce GTX 1070, 16… qat 39.06 tok/s ollama (version not recorded) · l…
Each speed links to its source, which has the full settings.
[interactive version](https://whatmodel.sh/model/gemma-4-e4b) · [JSON](https://whatmodel.sh/api?format=json&all=1)