[whatmodel.sh](../../README.md) / Gemma 4 26B-A4B
# Gemma 4 26B-A4B
Google · released 2026-04 · 25.2B params (3.8B active) · apache-2.0
**score 63.5** ±0.5 · #162 of 732 · from 64 benchmark results across 19 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Gemma 4 26B-A4B among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█●▅▅▄▄▂▁ #167 of 313 25% 13
coding ▁▁▁▃▅▇█▇●▄▃▁ #128 of 469 22% 10
chat & writing ▁▁▁▃▆▄▆█▆▇●▃ #70 of 402* 3.4% 2
math ▁▁▃▅█▆▄●▄▂▁▁ #97 of 431 7.0% 3
vision & documents ▁▁▂▂▁▃▄▂▆●█▄ #59 of 153 8.7% 5
web search ●▁▁▅▄█▃▄·▁·▁ #81 of 84* 0.8% 1
agent tasks · #167 of 313 · 25% of its score
Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md), [DeepSeek R1 0528](deepseek-r1-0528.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Jamba 1.5 Large](jamba-1-5-large.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 35.8% ▄●·····▄▄▄▄█ beats 1/7 hard · avg 16% 3.6% sharp · at its level
Terminal-Bench 2.1 27.3% ▄······●▄▄█▄ beats 2/7 very hard · avg 7.2% 3.1% sharp · at its level
Terminal-Bench Hard (maker) 14.0% ██··█·●·█··█ beats 3/5 very hard · avg 4.7% 3.0% sharp
SWE-bench Pro (split unspecified) 9.6% ●···▄···▄▄██ beats 0/7 very hard · avg 6.2% 2.7% sharp · at its level
ContractBench success rate (33 ob… 38.4% ▆··▆▄▂●▂▆▆█▄ beats 10/23 very hard · avg 13% 2.6% sharp · at its level
τ²-bench Telecom (maker) 43.0% ▄····█··●·▄▄ beats 3/5 very hard · avg 13% 2.2% sharp · at its level · noisy
τ²-bench aggregate (maker) 68.2% ▄·▄··▄····●▄ beats 3/5 medium · avg 39% 2.2% at its level · small run · several runs
SWE-bench Multilingual 27.3% ▄·●·····█▄█▄ beats 1/7 hard · avg 18% 1.8% at its level
Tau2 – airline (maker) 76.0% ▃·▃··▃·····● beats 5/5 medium · avg 42% 1.5% at its level
Humanity's Last Exam with tools (… 17.2% ▁·▁●▂▁▄▄▇█▄▅ beats 3/62 very hard · avg 9.9% 0.8% small run
SWE-bench Pro public set, Qwen-re… 13.8% ●·····▂·▂█▃▅ beats 0/12 hard · avg 16% 0.7% at its level
Tau2 – retail (maker) 85.5% ▄····▄··▄·▄● beats 4/5 medium · avg 40% 0.5% noisy
T1-Bench multi-domain conversatio… 37.7% ▂▂··▂·▂·▂▂█● beats 10/11 hard · avg 25% 0.0% blunt · at its level · noisy
coding · #128 of 469 · 22% of its score
Beats [Qwen3.5 Plus](qwen3-5-plus.md), [Qwen3-Coder-Next (80B-A3B)](qwen3-coder-next.md), [Gemini 2.5 Pro](gemini-2-5-pro.md) here, though they rank higher overall. Trails [Llama 3.1 Nemotron Ultra 253B v1](llama-3-1-nemotron-ultra-253b.md), [Solar Pro 4](solar-pro-4.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena WebDev (Code Arena) 1358.8 ▁▁▃▂●█▃▆▅▅▁▂ beats 32/129 avg 981 5.6% sharp
SWE-bench Verified 35.8% ▄●·····▄▄▄▄█ beats 1/7 hard · avg 16% 3.6% sharp · at its level
Luau Leetcode fully solved (Torpe… 38.9% ▁▁▁▁·●▂▃▅▃█▇ beats 7/49 hard · avg 17% 2.9% sharp · at its level
SWE-bench Pro (split unspecified) 9.6% ●···▄···▄▄██ beats 0/7 very hard · avg 6.2% 2.7% sharp · at its level
SWE-bench Multilingual 27.3% ▄·●·····█▄█▄ beats 1/7 hard · avg 18% 1.8% at its level
SciCode (level unspecified) 30.8% ▄···▄▄●▄·▄·▄ beats 4/7 very hard · avg 13% 1.4% at its level
SciCode (maker) 40.0% █▄·······▄●▄ beats 4/5 hard · avg 21% 1.3% at its level
LiveCodeBench v6 (window unspecif… 77.1% ▂▃▄▄▅██●▄▇▄▆ beats 43/67 medium · avg 47% 1.3% small run
LiveCodeBench v6 (Feb–May 2025) (… 77.1% ▄··▄·▄···▄·● beats 4/5 medium · avg 43% 0.9% at its level
SWE-bench Pro public set, Qwen-re… 13.8% ●·····▂·▂█▃▅ beats 0/12 hard · avg 16% 0.7% at its level
chat & writing · #70 of 402 · 3.4% of its score
Beats [Claude Haiku 5.5](claude-haiku-5-5.md), [DeepSeek V4 Flash (0731)](deepseek-v4-flash.md), [GPT-5.2](gpt-5-2.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
LMArena (text) 1437.6 ▁▁▁▂▃▃▃▆▆▇●▃ beats 300/385 avg 1260 3.1% blunt
IFEval (maker) 98.5% ▄··▄··█···●▄ beats 4/5 very easy · avg 94% 0.3% maxed out
math · #97 of 431 · 7.0% of its score
Beats [Claude Sonnet 4.6](claude-sonnet-4-6.md), [MiniMax-M3](minimax-m3.md), [Nemotron 3 Ultra 550B-A55B](nemotron-3-ultra-550b-a55b.md) here, though they rank higher overall. Trails [Gemini 2.5 Flash](gemini-2-5-flash.md), [Granite 4.2 30B](granite-4-2-30b.md), [Qwen3-Next 80B-A3B Instruct](qwen3-next-80b-a3b-instruct.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
AIME 2026 (maker) 88.3% ▄·▄▄····▄··● beats 4/5 medium · avg 51% 2.6% sharp · small run · several runs
OTIS mock AIME 2024–2025 82.2% ▃▃▄▃▃▃▇●█▃▂▅ beats 118/197 very hard · avg 7.2% 2.4% sharp · at its level · noisy
MATH-Vision (maker) 82.4% ██·█·····█●█ beats 4/5 medium · avg 53% 2.1% several runs
vision & documents · #59 of 153 · 8.7% of its score
Beats [GPT-6 Luna](gpt-6-luna.md), [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md), [GPT-5](gpt-5.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
MMMU Pro (maker) 73.8% ██·█····█·●█ beats 4/5 medium · avg 48% 2.4% at its level · several runs
Arena vision (style controlled) 1241.7 ▁▁▂▂▂▂▃▄▅▅●▇ beats 96/152 avg 1061 2.1% blunt
MATH-Vision (maker) 82.4% ██·█·····█●█ beats 4/5 medium · avg 53% 2.1% several runs
InfographicVQA (maker) 89.3% ▄█·······●·▄ beats 4/5 easy · avg 70% 1.1% typical
OmniDocBench 1.5 ↓ (maker) 14.9% █·█·····██●█ beats 4/5 avg 31% 1.0% blunt · several runs
web search · #81 of 84 · 0.8% of its score
Beats [GLM-4.5](glm-4-5.md), [Claude Haiku 4.5](claude-haiku-4-5.md) here, though they rank higher overall. Trails [GPT-4.1 nano](gpt-4-1-nano-2025-04-14.md), [GPT-4o mini (2024-07-18)](gpt-4o-mini-2024-07-18.md), [Mistral Small 3.1 24B (2503)](mistral-small-3-1.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Humanity's Last Exam with tools (… 17.2% ▁·▁●▂▁▄▄▇█▄▅ beats 3/62 very hard · avg 9.9% 0.8% small run
other tests (25)
test result vs the others in its run difficulty counts why
GPQA Diamond 73.2% ▁▁▁▁▂▄▆▄●▇█▅ beats 111/211 hard · avg 39% 11% sharp · at its level · several runs
MedXPertQA MM (maker) 58.1% ▂···▂·▃··▆●█ beats 8/13 very hard · avg 30% 4.2% sharp · at its level · several runs
MMLU-Pro (maker) 82.6% ▄··█···▄··●▄ beats 4/5 medium · avg 60% 2.7% several runs
MRCR v2 8-needle, 128k (maker) 44.1% ▄▄·▄···●···▄ beats 4/5 hard · avg 22% 2.6% at its level · several runs
MMMLU (maker) 86.3% ██··█···█·●█ beats 4/5 medium · avg 69% 2.4% several runs
BRIDGE clinical text, zero-shot a… 44.5% ▁▁▁▁▁▂▂▃▄▆█● beats 93/97 hard · avg 31% 2.0% blunt · at its level
EQ-Bench 3 (emotional intelligenc… 1272.3 ▁·▂▁▄▅▅▃●▄▁▁ beats 40/62 avg 1034 1.9% typical
AraGen-v3 3C3H score (Inception, … 67.3% ▁▁·▁▂▂▄█▄▅●▄ beats 53/66 hard · avg 29% 1.8% at its level
KOPA-Bench (pass@1) 35.3% ▄·█··▄··●▄·█ beats 5/8 very hard · avg 14% 1.6% at its level
CPTUB average (SpeakLeash, 0-5 ju… 84.3% ▁▂▂▂▃█▃▆▅▄●▃ beats 88/95 medium · avg 65% 1.5% typical
RobloxQA v2.0 multiple choice (To… 90.9% ▁▁▃▁●▄▃▃▆▂▂▂ beats 19/49 easy · avg 77% 1.4% typical
IFBench (maker) 72.0% ▄▄·▄······●█ beats 3/5 medium · avg 40% 1.2% at its level
Open Persian LLM Leaderboard v2 a… 70.7% ▂▄▂▃█▄▃▆▃●▂▂ beats 75/81 medium · avg 57% 1.2% blunt · at its level
Big Bench Extra Hard micro avg (m… 64.8% █··▄···▄·●·▄ beats 4/5 hard · avg 28% 1.2% at its level
NL2Repo 11.6% ●▂▂·▃▃▅█▃▂▅▃ beats 0/22 very hard · avg 7.0% 1.1% typical
QwenClawBench (Qwen internal) 38.7% ●·▄····▄···█ beats 0/4 hard · avg 23% 1.0% at its level
Humanity's Last Exam (maker) 8.7% ▁▁▁▄●▆▇██▆▁▃ beats 17/111 very hard · avg 3.9% 1.0% small run
EuroEval Life in the UK (English … 93.4% ▂·▂▂·▂·▃▃·●▃ beats 12/14 easy · avg 80% 1.0% typical
EuroEval HellaSwag (English), val… 82.3% ▂··▂▂·▂▆▆●·▄ beats 13/15 medium · avg 63% 0.9% typical
Chess Puzzles (Epoch) 6.0% █●▁▁▁▂▃▃▃▂▁▁ beats 48/137 very hard · avg 5.8% 0.9% sharp · too hard for it · noisy
BFCL multi-turn (version not stat… 55.4% ▄▄···▄·▄█●▄▄ beats 6/8 hard · avg 26% 0.8% at its level
DiagnosticIQ industrial maintenan… 63.2% ▁··▃▁▁▁▅●▅▁▅ beats 14/24 hard · avg 51% 0.7% blunt · at its level
HLE (maker) 8.7% █···●······█ beats 1/2 very hard · avg 4.0% 0.5% small run
SkillsBench 78-task subset (OpenC… 12.3% ▄·····●···▄█ beats 1/4 very hard · avg 7.4% 0.5% typical
Vectara hallucination leaderboard… 94.8% ▁▂▁▂▅█▂▃▄●▁▁ beats 87/98 very easy · avg 90% 0.0% blunt
source results share of the score
Gemma Team (submitted / maker-reported) 20 29% ▇▇▇
google/diffusiongemma-26B-A4B-it model … 12 14% ▇▇
Kwaipilot/KAT-Coder-V2.5-Dev model card 5 13% ▇▇
LMArena leaderboard dataset 3 11% ▇
Epoch AI 3 8% ▇
Roblox LLM Leaderboard 2 4% ▇
Qwen/Qwen3.6-35B-A3B model card 4 3% ▇
ContractBench: Can LLM Agents Preserve … 1 3% ▇
Multi-Step Tool-Calling over Korean Ope… 2 2% ▇
BRIDGE Medical Leaderboard 1 2% ▇
EuroEval leaderboards 2 2% ▇
EQ-Bench 3 canonical leaderboard results 1 2% ▇
google/gemma-4-31B-it model card (submi… 2 2% ▇
Inception Arabic/Hindi leaderboards 1 2% ▇
SpeakLeash CPTUB 1 1% ▇
Open Persian LLM Leaderboard v2 1 1% ▇
DiagnosticIQ: A Benchmark for LLM-Based… 1 1% ▇
T1-Bench: Benchmarking Multi-Scenario A… 1 0% ▇
Vectara hallucination leaderboard 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 63.5
Licences: Gemma Team ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · google/diffusiongemma-26B-A4B-it model card ([Apache-2.0](https://huggingface.co/google/diffusiongemma-26B-A4B-it/blob/f7f5b7f5fa82ffc52addd066915886d497f5517b/README.md)) · Kwaipilot/KAT-Coder-V2.5-Dev model card ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Roblox LLM Leaderboard ([MIT](../../licenses/roblox-llm-leaderboard-MIT.txt)) · Qwen/Qwen3.6-35B-A3B model card ([apache-2.0](../../licenses/Qwen--Qwen3.6-35B-A3B.txt)) · ContractBench: Can LLM Agents Preserve Observation Contracts? ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Multi-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · BRIDGE Medical Leaderboard ([MIT](../../licenses/bridge-medical-leaderboard-MIT.txt)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · EQ-Bench 3 canonical leaderboard results ([MIT](../../licenses/eqbench3-MIT.txt)) · google/gemma-4-31B-it model card ([Apache-2.0](../../licenses/google--gemma-4-31B-it.txt)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · SpeakLeash CPTUB ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
### Through an API
api (DeepInfra) · $0.14/M blended · $0.07 in / $0.34 out per 1M tokens · [deepinfra.com](https://deepinfra.com/google/gemma-4-26B-A4B-it)
[price source](https://github.com/BerriAI/litellm/blob/c19ce71bcdad234ca87a7af10cee45375ce811a8/model_prices_and_context_window.json): USD per 1M text tokens, standard uncached input and output. Product: Serverless text generation. API model: google/gemma-4-26B-A4B-it. Settings: service tier=standard, batch=no, cache=uncached input. Additional/excluded charges: Tools, cache storage, priority service and other modalities excluded.. Currency: USD. · [context source](https://github.com/BerriAI/litellm/blob/c19ce71bcdad234ca87a7af10cee45375ce811a8/model_prices_and_context_window.json): provider maximum input tokens; not an intrinsic model maximum or legacy max\_tokens
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Gemma 4 26B-A4B (Q4_K_M, needs ~16.8 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/gemma-4-26B-A4B-it-GGUF/tree/f6e6747823b2912661935db7e0009287c4838073))
```sh
llama cli -hf lmstudio-community/gemma-4-26B-A4B-it-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf lmstudio-community/gemma-4-26B-A4B-it-GGUF:Q4_K_M
```
Other versions: Q8_0 (~26.9 GB) `llama cli -hf lmstudio-community/gemma-4-26B-A4B-it-GGUF:Q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Gemma 4 26B-A4B (Q4_K_M, needs ~16.8 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/gemma-4-26B-A4B-it-GGUF/tree/f6e6747823b2912661935db7e0009287c4838073))
```sh
lms get https://huggingface.co/lmstudio-community/gemma-4-26B-A4B-it-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~26.9 GB) `lms get https://huggingface.co/lmstudio-community/gemma-4-26B-A4B-it-GGUF@Q8_0 -y` · 4bit (~15.6 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/gemma-4-26B-A4B-it-MLX-4bit -y` · 8bit (~28 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/gemma-4-26B-A4B-it-MLX-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Gemma 4 26B-A4B (Q4_K_M, needs ~15.62 GB, from [ollama.com](https://ollama.com/library/gemma4/tags))
```sh
ollama run gemma4:26b
```
Other versions: qat (~15.62 GB) `ollama run gemma4:26b-a4b-it-qat`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run Gemma 4 26B-A4B (4bit, needs ~15.6 GB, Mac only, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/gemma-4-26B-A4B-it-MLX-4bit/tree/79290e197d238113e50abe4cbd4731b65288f88e))
```sh
mlx_lm.chat --model lmstudio-community/gemma-4-26B-A4B-it-MLX-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model lmstudio-community/gemma-4-26B-A4B-it-MLX-4bit
```
Other versions: 8bit (~28 GB, Mac only) `mlx_lm.chat --model lmstudio-community/gemma-4-26B-A4B-it-MLX-8bit`
Model repository and documentation: [huggingface.co/google](https://huggingface.co/google/gemma-4-26B-A4B-it)
## Published speed measurements
Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine.
hardware precision speed engine and settings
AMD Ryzen AI Max+ 395 (Stri… UD-Q4_K_M 55.45 tok/s llama.cpp 0eca4d490 (build_number…
AMD Ryzen AI Max+ 395 (Stri… UD-Q4_K_M 54.18 tok/s llama.cpp 4fc4ec554 (build_number…
AMD Ryzen AI Max+ 395 (Stri… Gemma QAT GGUF … 74.8 tok/s llama.cpp ac4cddeb0 (build_number…
Radeon RX 9070 XT q4_k_m · layers… 34.85 tok/s ollama (version not recorded) · l…
NVIDIA GeForce GTX 1650 wit… q4_k_m · layers… 5.15 tok/s ollama (version not recorded) · l…
NVIDIA GeForce GTX 1650 wit… q4_k_m · layers… 5.18 tok/s ollama (version not recorded) · l…
Each speed links to its source, which has the full settings.
## Quantization evidence
How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent. They don't describe every download of that precision.
precision test full → quantized kept
FP8 dynamic (data-free; FP8 w… AIME 2025 0-shot pass@1, mean of … 0.887 → 0.917 103.3%
FP8 dynamic (data-free; FP8 w… GPQA Diamond 0-shot pass@1 (witho… 0.732 → 0.748 102.1%
FP8 dynamic (data-free; FP8 w… MATH-500 0-shot pass@1 (without t… 0.848 → 0.859 101.3%
FP8 dynamic (data-free; FP8 w… IFEval 0-shot prompt-level strict… 0.944 → 0.951 100.7%
FP8 dynamic (data-free; FP8 w… MATH-500 0-shot pass@1 (with thin… 0.859 → 0.859 100.1%
FP8 dynamic (data-free; FP8 w… AIME 2025 0-shot pass@1, mean of … 0.8 → 0.8 100.0%
FP8 dynamic (data-free; FP8 w… MMLU-Pro 0-shot custom-extract (w… 0.852 → 0.851 99.9%
FP8 dynamic (data-free; FP8 w… MMLU-Pro 0-shot custom-extract (w… 0.835 → 0.833 99.7%
FP8 dynamic (data-free; FP8 w… IFEval 0-shot prompt-level strict… 0.9 → 0.893 99.3%
FP8 dynamic (data-free; FP8 w… LiveCodeBench v6 0-shot pass@1 (w… 0.745 → 0.739 99.2%
FP8 dynamic (data-free; FP8 w… GPQA Diamond 0-shot pass@1 (with … 0.808 → 0.801 99.2%
FP8 dynamic (data-free; FP8 w… LiveCodeBench v6 0-shot pass@1 (w… 0.779 → 0.76 97.6%
NVFP4 (FP4 weights/activation… MATH-500 0-shot pass@1 (without t… 0.848 → 0.876 103.3%
NVFP4 (FP4 weights/activation… IFEval 0-shot prompt-level strict… 0.944 → 0.942 99.7%
NVFP4 (FP4 weights/activation… MATH-500 0-shot pass@1 (with thin… 0.859 → 0.851 99.1%
NVFP4 (FP4 weights/activation… GPQA Diamond 0-shot pass@1 (witho… 0.732 → 0.724 98.9%
NVFP4 (FP4 weights/activation… MMLU-Pro 0-shot custom-extract (w… 0.835 → 0.817 97.8%
NVFP4 (FP4 weights/activation… MMLU-Pro 0-shot custom-extract (w… 0.852 → 0.833 97.8%
NVFP4 (FP4 weights/activation… IFEval 0-shot prompt-level strict… 0.9 → 0.872 97.0%
NVFP4 (FP4 weights/activation… GPQA Diamond 0-shot pass@1 (with … 0.808 → 0.776 96.0%
NVFP4 (FP4 weights/activation… LiveCodeBench v6 0-shot pass@1 (w… 0.779 → 0.728 93.4%
NVFP4 (FP4 weights/activation… LiveCodeBench v6 0-shot pass@1 (w… 0.745 → 0.682 91.6%
NVFP4 (FP4 weights/activation… AIME 2025 0-shot pass@1, mean of … 0.887 → 0.813 91.5%
NVFP4 (FP4 weights/activation… AIME 2025 0-shot pass@1, mean of … 0.8 → 0.662 82.8%
Each kept value links to its source.
[interactive version](https://whatmodel.sh/model/gemma-4-26b-a4b) · [JSON](https://whatmodel.sh/api?format=json&all=1)