[whatmodel.sh](../../README.md) / Gemma 4 31B
# Gemma 4 31B
Google · released 2026-04 · 30.7B params · apache-2.0
**score 66.2** ±0.5 · #121 of 732 · from 75 benchmark results across 27 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Gemma 4 31B among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█▄●▅▄▄▂▁ #119 of 313 34% 17
coding ▁▁▁▃▅▇█▇●▄▃▁ #115 of 469 25% 9
chat & writing ▁▁▁▃▆▄▆█▆▇●▃ #55 of 402* 3.1% 2
math ▁▁▃▅█▆▄●▄▂▁▁ #119 of 431 3.4% 3
vision & documents ▁▁▂▂▁▃▄▂▆●█▄ #50 of 153 12% 11
web search ▁▁▁●▄█▃▄·▁·▁ #70 of 84 2.9% 3
agent tasks · #119 of 313 · 34% of its score
Beats [o3-pro](o3-pro.md), [Grok 4.3](grok-4-3.md), [Qwen3.6 Plus](qwen3-6-plus.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Large](jamba-1-5-large.md), [Devstral 2 (2512)](devstral-2.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Pro (split unspecified) 33.0% ▄···▄···●▄██ beats 2/7 very hard · avg 6.2% 5.2% sharp · at its level · several runs
SWE-bench Verified 60.6% ▄▄·····▄▄●▄█ beats 4/7 hard · avg 16% 4.9% sharp · at its level · several runs
Terminal-Bench Hard (maker) 36.0% ██··█·█·█··● beats 5/5 very hard · avg 4.7% 4.7% sharp · at its level · several runs
Terminal-Bench 2.1 30.3% ▄······█●▄█▄ beats 3/7 very hard · avg 7.2% 3.2% sharp · at its level
ContractBench success rate (33 ob… 37.4% ▆··▆▄▂●▂▆▆█▄ beats 9/23 very hard · avg 13% 2.4% sharp · at its level
SWE-bench Pro public set, Qwen-re… 35.7% ●······▄█·▄▄ beats 0/5 hard · avg 19% 2.0% at its level · several runs
τ²-bench Telecom (maker) 69.3% ▄····█··▄·▄● beats 5/5 very hard · avg 13% 2.0% sharp · at its level · noisy
SWE-bench Multilingual 49.3% ▄·▄·····●▄█▄ beats 3/7 hard · avg 18% 1.7% at its level
K-BrowseComp-Verified (Korean web… 23.3% ▃····█··●█·▃ beats 4/8 very hard · avg 3.6% 1.3% typical
Tau2 – airline (maker) 75.0% ▃·▃··▃·····● beats 4/5 medium · avg 42% 1.3% typical
OSWorld-Verified 58.5% █▃·▂●▂▃▂▅▃▆▆ beats 10/27 hard · avg 26% 1.3% at its level · small run
τ²-bench aggregate (maker) 76.9% ▄·▄··▄····█● beats 5/5 hard · avg 34% 0.9% at its level
MCP-Atlas (public set) 54.2% ▂·▁▂▂·▁●█▆▆▂ beats 8/39 hard · avg 29% 0.9% at its level · small run
DeepSearchQA 61.7% ●·▄▄···▄▄··█ beats 0/6 medium · avg 42% 0.8% at its level · small run
Humanity's Last Exam with tools (… 26.5% ▁·▁▂▂●▄▄▇█▄▅ beats 11/62 very hard · avg 9.9% 0.8% at its level · small run
Tau2 – retail (maker) 86.4% ▄····▄··▄·▄● beats 5/5 medium · avg 40% 0.4% noisy
T1-Bench multi-domain conversatio… 41.1% ▂▂··▂·▂·▂▂█● beats 11/11 hard · avg 25% 0.0% blunt · at its level · noisy
coding · #115 of 469 · 25% of its score
Beats [Grok 4.3](grok-4-3.md), [Qwen3.5 Plus](qwen3-5-plus.md), [DeepSeek V3.2](deepseek-v3-2.md) here, though they rank higher overall. Trails [Llama 3.1 Nemotron Ultra 253B v1](llama-3-1-nemotron-ultra-253b.md), [Solar Pro 4](solar-pro-4.md), [Nemotron Cascade 2 30B-A3B](nemotron-cascade-2-30b-a3b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Pro (split unspecified) 33.0% ▄···▄···●▄██ beats 2/7 very hard · avg 6.2% 5.2% sharp · at its level · several runs
Arena WebDev (Code Arena) 1368.7 ▁▁▃▂●█▃▆▅▅▁▂ beats 35/129 avg 981 5.2% sharp
SWE-bench Verified 60.6% ▄▄·····▄▄●▄█ beats 4/7 hard · avg 16% 4.9% sharp · at its level · several runs
Luau Leetcode fully solved (Torpe… 55.3% ▁▁▁▁·▁▂●▅▃█▇ beats 12/49 hard · avg 17% 2.7% sharp · at its level
SWE-bench Pro public set, Qwen-re… 35.7% ●······▄█·▄▄ beats 0/5 hard · avg 19% 2.0% at its level · several runs
SWE-bench Multilingual 49.3% ▄·▄·····●▄█▄ beats 3/7 hard · avg 18% 1.7% at its level
SciCode (level unspecified) 33.2% ▄···▄▄█●·▄·▄ beats 5/7 very hard · avg 13% 1.4% at its level
SciCode (maker) 43.0% █▄·······▄▄● beats 5/5 hard · avg 21% 1.2% at its level
LiveCodeBench v6 (Feb–May 2025) (… 80.0% ▄··▄·▄···▄·● beats 5/5 medium · avg 43% 0.7% at its level
chat & writing · #55 of 402 · 3.1% of its score
Beats [MiMo-V2.6-Flash (309B-A15B)](mimo-v2-6-flash.md), [Claude Haiku 5.5](claude-haiku-5-5.md), [GPT-6 Luna](gpt-6-luna.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
LMArena (text) 1452.3 ▁▁▁▂▃▃▃▆▆▇●▃ beats 322/385 avg 1260 2.9% blunt
IFEval (maker) 98.9% ▄··▄··█···▄● beats 5/5 very easy · avg 94% 0.2% maxed out
math · #119 of 431 · 3.4% of its score
Beats [Claude Sonnet 4.6](claude-sonnet-4-6.md), [MiniMax-M3](minimax-m3.md), [MiMo-V2.5 (310B-A15B)](mimo-v2-5.md) here, though they rank higher overall. Trails [Olmo 3.1 32B Think](olmo-3-1-32b-think.md), [gpt-oss-120b](gpt-oss-120b.md), [Qwen3-VL 32B Thinking](qwen3-vl-32b-thinking.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
OTIS mock AIME 2024–2025 73.3% ▃▃▄▃▃▃▇●█▃▂▅ beats 109/197 very hard · avg 7.2% 1.7% sharp · noisy
MATH-Vision (maker) 85.6% ██·█·····██● beats 5/5 medium · avg 53% 0.9% typical
AIME 2026 (maker) 89.2% ▄·▄▄····▄··● beats 5/5 medium · avg 40% 0.8% sharp · noisy
vision & documents · #50 of 153 · 12% of its score
Beats [MiMo-V2.6-Pro](mimo-v2-6-pro.md), [MiMo-V2.6-Flash (309B-A15B)](mimo-v2-6-flash.md), [GPT-6 Luna](gpt-6-luna.md) here, though they rank higher overall. Trails [Qwen3.6 Flash](qwen3-6-flash.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena vision (style controlled) 1261.8 ▁▁▂▂▂▂▃▄▅▅●▇ beats 115/152 avg 1061 1.9% blunt
ChartArena chart parsing (English… 46.8% ▁·▁·▃▃▁▇▃●·▄ beats 18/22 hard · avg 27% 1.6% blunt · at its level
MMMU Pro (maker) 76.9% ██·█····█·█● beats 5/5 medium · avg 48% 1.5% at its level
Arena document (style controlled) 1444.5 ▁▁▃▃▄●▇█▅▇▄▅ beats 9/37 avg 1331 1.0% blunt
BEAR-Bench (bilingual RU/EN enter… 23.5% ▃▃·●·▃·█·█▅▅ beats 2/14 hard · avg 24% 1.0% at its level
CharXiv (reasoning questions) 77.7% ▂▅▃▅▅▆▆▅●█▆▂ beats 71/101 medium · avg 42% 1.0% small run
MATH-Vision (maker) 85.6% ██·█·····██● beats 5/5 medium · avg 53% 0.9% typical
InfographicVQA (maker) 92.0% ▄█·······█·● beats 5/5 easy · avg 70% 0.9% typical
OmniDocBench 1.5 overall (higher … 72.5% ▂··●▂▂··▂▂█▅ beats 1/13 medium · avg 54% 0.8% small run
OmniDocBench 1.5 ↓ (maker) 13.1% █·█·····███● beats 5/5 avg 31% 0.6% blunt
ScreenSpot-Pro 75.9% ▂▃▅▆██●▂···▂ beats 21/24 medium · avg 52% 0.5% blunt · at its level · small run
web search · #70 of 84 · 2.9% of its score
Beats [DeepSeek V4 Flash (Preview, Apr 2026)](deepseek-v4-flash-preview.md), [Nemotron 3 Ultra 550B-A55B](nemotron-3-ultra-550b-a55b.md), [GPT-5.4 nano](gpt-5-4-nano.md) here, though they rank higher overall. Trails [GPT-4.1 nano](gpt-4-1-nano-2025-04-14.md), [GPT-4o mini (2024-07-18)](gpt-4o-mini-2024-07-18.md), [Mistral Small 3.1 24B (2503)](mistral-small-3-1.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
K-BrowseComp-Verified (Korean web… 23.3% ▃····█··●█·▃ beats 4/8 very hard · avg 3.6% 1.3% typical
DeepSearchQA 61.7% ●·▄▄···▄▄··█ beats 0/6 medium · avg 42% 0.8% at its level · small run
Humanity's Last Exam with tools (… 26.5% ▁·▁▂▂●▄▄▇█▄▅ beats 11/62 very hard · avg 9.9% 0.8% at its level · small run
other tests (32)
test result vs the others in its run difficulty counts why
GPQA Diamond 75.8% ▁▁▁▁▂▄▆▄●▇█▅ beats 116/211 hard · avg 39% 6.6% sharp · at its level · several runs
MedXPertQA MM (maker) 61.3% ▂···▂·▃··▆▂● beats 9/13 very hard · avg 30% 2.5% sharp · at its level
NL2Repo 15.5% ●·····█·▄▄·▄ beats 0/5 very hard · avg 8.9% 2.3% at its level · several runs
BRIDGE clinical text, zero-shot a… 46.7% ▁▁▁▁▁▂▂▃▄▆█● beats 96/97 hard · avg 31% 1.9% blunt · at its level
EQ-Bench 3 (emotional intelligenc… 1272.9 ▁·▂▁▄▅▅▃●▄▁▁ beats 41/62 avg 1034 1.8% typical
AraGen-v3 3C3H score (Inception, … 72.7% ▁▁·▁▂▂▄█▄▅●▄ beats 57/66 hard · avg 29% 1.5% at its level
MRCR v2 8-needle, 128k (maker) 66.4% ▄▄·▄···█···● beats 5/5 hard · avg 22% 1.4% at its level
MMLU-Pro (maker) 85.2% ▄··█···▄··▄● beats 5/5 medium · avg 60% 1.3% typical
MMMLU (maker) 88.4% ██··█···█·█● beats 5/5 medium · avg 69% 1.2% typical
Chess Puzzles (Epoch) 5.0% ●▁▁▁▁▂▃▃▃▂▁▁ beats 18/137 very hard · avg 5.8% 1.2% sharp · noisy
CPTUB average (SpeakLeash, 0-5 ju… 86.2% ▁▂▂▂▃█▃▆▅▄▂● beats 91/95 medium · avg 65% 1.2% typical
RobloxQA v2.0 multiple choice (To… 92.6% ▁▁▃▁█●▃▃▆▂▂▂ beats 22/49 easy · avg 77% 1.1% typical
IFBench (maker) 76.0% ▄▄·▄······▄● beats 5/5 medium · avg 40% 1.0% at its level
Big Bench Extra Hard micro avg (m… 74.4% █··▄···▄·▄·● beats 5/5 hard · avg 28% 1.0% at its level
Open Persian LLM Leaderboard v2 a… 78.5% ▂▄▂▃█▄▃▆▃▅▂● beats 81/81 medium · avg 57% 1.0% blunt
WildClawBench 37.6% ▂·▄▄▆●▄█▆▂▂▂ beats 8/20 very hard · avg 14% 1.0% at its level · small run
Icelandic LLM Leaderboard average… 20.7% ▁▁▃▂●▄▁▃▄▄▄█ beats 8/38 very hard · avg 14% 1.0% at its level
QwenClawBench (Qwen internal) 41.7% ▄·●····▄···█ beats 1/4 hard · avg 23% 1.0% at its level
SimpleQA Verified 10.4% ▁●▃▂▃▃▆▆█▄▂▅ beats 2/81 very hard · avg 8.0% 1.0% at its level
EuroEval HellaSwag (English), val… 89.1% ▂··▂▂·▂▆▆█·● beats 15/15 medium · avg 63% 0.8% typical
EuroEval Life in the UK (English … 95.2% ▂·▂▂·▂·▃▃·█● beats 14/14 easy · avg 80% 0.8% maxed out
Next Steps data preparation (bala… 85.9% ▄·▄···█·▄▄·● beats 6/6 medium · avg 52% 0.7% typical
Gaia2 36.4% ●·····█····█ beats 0/2 hard · avg 17% 0.7% at its level · small run
DiagnosticIQ industrial maintenan… 65.1% ▁··▃▁▁▁▅█●▁▅ beats 17/24 hard · avg 51% 0.6% blunt · at its level
SkillsBench (with skills) 32.4% ●·········██ beats 0/2 hard · avg 17% 0.6% at its level · small run
HLE (maker) 19.5% █···█······● beats 2/2 very hard · avg 4.0% 0.5% small run
SkillsBench 78-task subset (OpenC… 23.6% ▄·····▄···●█ beats 2/4 very hard · avg 7.4% 0.5% typical
WMT25 Terminology Translation (te… 92.1% ▂·▂··▂·▄▂·█● beats 12/12 medium · avg 63% 0.5% noisy
NTCIR SciClaimEval Subtask 1 pair… 69.9% ▃▃···▃·▃▃●·▃ beats 5/8 medium · avg 45% 0.5% blunt · at its level
WMT24++ 79.2% █·▃·▅▅●···▃▃ beats 8/10 easy · avg 70% 0.3% blunt
WMT26 General MT (GEMBA-ESA, LLM … 79.5% ▄·▄▄▄·▄██●█▄ beats 9/12 medium · avg 65% 0.2% blunt
Vectara hallucination leaderboard… 92.6% ▁▂▁▂▅█▂●▄▃▁▁ beats 66/98 very easy · avg 90% 0.0% blunt
source results share of the score
Gemma Team (submitted / maker-reported) 20 25% ▇▇▇
Kwaipilot/KAT-Coder-V2.5-Dev model card 5 12% ▇
meta-models/Muse-Glimmer-30B model card 11 12% ▇
LMArena leaderboard dataset 4 11% ▇
Epoch AI 4 8% ▇
Roblox LLM Leaderboard 2 4% ▇
Qwen/Qwen3.6-35B-A3B model card 4 3% ▇
Qwen/Qwen3.6-27B model card 2 3% ▇
ContractBench: Can LLM Agents Preserve … 1 2% ▇
BRIDGE Medical Leaderboard 1 2% ▇
EQ-Bench 3 canonical leaderboard results 1 2% ▇
google/gemma-4-31B-it model card (submi… 2 2% ▇
ChartArena: Benchmarking Chart Parsing … 1 2% ▇
NVIDIA ModelOpt quantized-checkpoint ca… 1 2% ▇
EuroEval leaderboards 2 2% ▇
Inception Arabic/Hindi leaderboards 1 1% ▇
K-BrowseComp: A Web Browsing Agent Benc… 1 1% ▇
SpeakLeash CPTUB 1 1% ▇
Open Persian LLM Leaderboard v2 1 1% ▇
Miðeind Icelandic LLM Leaderboard 1 1% ▇
BEAR-Bench: A Bilingual Enterprise and … 1 1% ▇
Cohere 3 1% ▇
Agentic coding without the cloud: evalu… 1 1% ▇
DiagnosticIQ: A Benchmark for LLM-Based… 1 1% ▇
SciTrue: Reliable Scientific Claim Vali… 1 0% ▇
T1-Bench: Benchmarking Multi-Scenario A… 1 0% ▇
Vectara hallucination leaderboard 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 66.2
Licences: Gemma Team ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Kwaipilot/KAT-Coder-V2.5-Dev model card ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · meta-models/Muse-Glimmer-30B model card ([Apache-2.0](../../licenses/speed-meta-muse-glimmer-30b-Apache-2.0.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Roblox LLM Leaderboard ([MIT](../../licenses/roblox-llm-leaderboard-MIT.txt)) · Qwen/Qwen3.6-35B-A3B model card ([apache-2.0](../../licenses/Qwen--Qwen3.6-35B-A3B.txt)) · Qwen/Qwen3.6-27B model card ([apache-2.0](../../licenses/Qwen--Qwen3.6-27B.txt)) · ContractBench: Can LLM Agents Preserve Observation Contracts? ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · BRIDGE Medical Leaderboard ([MIT](../../licenses/bridge-medical-leaderboard-MIT.txt)) · EQ-Bench 3 canonical leaderboard results ([MIT](../../licenses/eqbench3-MIT.txt)) · google/gemma-4-31B-it model card ([Apache-2.0](../../licenses/google--gemma-4-31B-it.txt)) · ChartArena: Benchmarking Chart Parsing across Languages ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · NVIDIA ModelOpt quantized-checkpoint cards: baseline ([Apache-2.0 (Gemma-4-31B-IT-NVFP4 repo) / MIT (DeepSeek-V4-Pro-0813-NVFP4 repo)](https://www.apache.org/licenses/LICENSE-2.0)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · SpeakLeash CPTUB ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Miðeind Icelandic LLM Leaderboard ([CC BY 4.0](../../licenses/mideind-icelandic-llm-leaderboard-CC-BY-4.0.txt)) · BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Cohere ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Agentic coding without the cloud: evaluating open-weight LLMs on longitudinal data preparation tasks ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · SciTrue: Reliable Scientific Claim Validation with Frontier and Open Language Models at the NTCIR SciClaimEval Task ([CC BY 4.0](../../licenses/paper-2609.00654v1-CC-BY-4.0.txt)) · T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
### Through an API
api (DeepInfra) · $0.19/M blended · $0.13 in / $0.38 out per 1M tokens · [deepinfra.com](https://deepinfra.com/google/gemma-4-31B-it)
[price source](https://github.com/BerriAI/litellm/blob/c19ce71bcdad234ca87a7af10cee45375ce811a8/model_prices_and_context_window.json): USD per 1M text tokens, standard uncached input and output. Product: Serverless text generation. API model: google/gemma-4-31B-it. Settings: service tier=standard, batch=no, cache=uncached input. Additional/excluded charges: Tools, cache storage, priority service and other modalities excluded.. Currency: USD. · [context source](https://github.com/BerriAI/litellm/blob/c19ce71bcdad234ca87a7af10cee45375ce811a8/model_prices_and_context_window.json): provider maximum input tokens; not an intrinsic model maximum or legacy max\_tokens
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Gemma 4 31B (Q4_K_M, needs ~18.7 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/gemma-4-31B-it-GGUF/tree/67a72ce462184ca84e9531dfe657ee73b4ecc89d))
```sh
llama cli -hf lmstudio-community/gemma-4-31B-it-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf lmstudio-community/gemma-4-31B-it-GGUF:Q4_K_M
```
Other versions: Q8_0 (~32.6 GB) `llama cli -hf lmstudio-community/gemma-4-31B-it-GGUF:Q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Gemma 4 31B (Q4_K_M, needs ~18.7 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/gemma-4-31B-it-GGUF/tree/67a72ce462184ca84e9531dfe657ee73b4ecc89d))
```sh
lms get https://huggingface.co/lmstudio-community/gemma-4-31B-it-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~32.6 GB) `lms get https://huggingface.co/lmstudio-community/gemma-4-31B-it-GGUF@Q8_0 -y` · 4bit (~18.4 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/gemma-4-31B-it-MLX-4bit -y` · 8bit (~33.8 GB, Mac only) `lms get https://huggingface.co/lmstudio-community/gemma-4-31B-it-MLX-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Gemma 4 31B (Q4_K_M, needs ~18.92 GB, from [ollama.com](https://ollama.com/library/gemma4/tags))
```sh
ollama run gemma4:31b
```
Other versions: qat (~18.92 GB) `ollama run gemma4:31b-it-qat`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run Gemma 4 31B (4bit, needs ~18.4 GB, Mac only, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/gemma-4-31B-it-MLX-4bit/tree/6dafa401f876e1a1cfe3b846c9d26312c560d282))
```sh
mlx_lm.chat --model lmstudio-community/gemma-4-31B-it-MLX-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model lmstudio-community/gemma-4-31B-it-MLX-4bit
```
Other versions: 8bit (~33.8 GB, Mac only) `mlx_lm.chat --model lmstudio-community/gemma-4-31B-it-MLX-8bit`
Model repository and documentation: [huggingface.co/google](https://huggingface.co/google/gemma-4-31B-it)
## Published speed measurements
Reference results for the hardware, engine and settings below. These are not verified speeds of the downloads above or predictions for your machine.
hardware precision speed engine and settings
AMD Ryzen AI Max+ 395 (Stri… Q4_0 (Google QA… 11.38 tok/s llama.cpp 86a9c79f8 (build_number…
Each speed links to its source, which has the full settings.
## Quantization evidence
How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent. They don't describe every download of that precision.
precision test full → quantized kept
FP8 block-wise (data-free; ll… AIME 2025 0-shot pass@1, mean of … 0.658 → 0.683 103.8%
FP8 block-wise (data-free; ll… LiveCodeBench v6 0-shot pass@1 (w… 0.714 → 0.735 102.9%
FP8 block-wise (data-free; ll… GPQA Diamond 0-shot pass@1 (witho… 0.774 → 0.779 100.7%
FP8 block-wise (data-free; ll… IFEval 0-shot prompt-level strict… 0.907 → 0.912 100.6%
FP8 block-wise (data-free; ll… GPQA Diamond 0-shot pass@1 (with … 0.864 → 0.869 100.6%
FP8 block-wise (data-free; ll… AIME 2025 0-shot pass@1, mean of … 0.871 → 0.875 100.5%
FP8 block-wise (data-free; ll… MATH-500 0-shot pass@1 (with thin… 0.879 → 0.881 100.2%
FP8 block-wise (data-free; ll… MMLU-Pro 0-shot custom-extract (w… 0.854 → 0.854 100.0%
FP8 block-wise (data-free; ll… MMLU-Pro 0-shot custom-extract (w… 0.871 → 0.87 100.0%
FP8 block-wise (data-free; ll… IFEval 0-shot prompt-level strict… 0.946 → 0.942 99.6%
FP8 block-wise (data-free; ll… MATH-500 0-shot pass@1 (without t… 0.894 → 0.887 99.2%
FP8 dynamic (data-free; FP8 w… LiveCodeBench v6 0-shot pass@1 (w… 0.743 → 0.792 106.7%
FP8 dynamic (data-free; FP8 w… AIME 2025 0-shot pass@1, mean of … 0.871 → 0.9 103.4%
FP8 dynamic (data-free; FP8 w… MATH-500 0-shot pass@1 (with thin… 0.879 → 0.895 101.8%
FP8 dynamic (data-free; FP8 w… MATH-500 0-shot pass@1 (without t… 0.894 → 0.903 101.0%
FP8 dynamic (data-free; FP8 w… GPQA Diamond 0-shot pass@1 (witho… 0.774 → 0.781 100.9%
FP8 dynamic (data-free; FP8 w… AIME 2025 0-shot pass@1, mean of … 0.658 → 0.662 100.6%
FP8 dynamic (data-free; FP8 w… IFEval 0-shot prompt-level strict… 0.907 → 0.911 100.4%
FP8 dynamic (data-free; FP8 w… IFEval 0-shot prompt-level strict… 0.946 → 0.949 100.3%
FP8 dynamic (data-free; FP8 w… MMLU-Pro 0-shot custom-extract (w… 0.871 → 0.871 100.0%
FP8 dynamic (data-free; FP8 w… MMLU-Pro 0-shot custom-extract (w… 0.854 → 0.853 99.9%
FP8 dynamic (data-free; FP8 w… GPQA Diamond 0-shot pass@1 (with … 0.864 → 0.86 99.6%
FP8 dynamic (data-free; FP8 w… LiveCodeBench v6 0-shot pass@1 (w… 0.714 → 0.707 98.9%
NVFP4 (FP4 weights/activation… LiveCodeBench v6 0-shot pass@1 (w… 0.743 → 0.779 104.9%
NVFP4 (FP4 weights/activation… GPQA Diamond 0-shot pass@1 (with … 0.864 → 0.87 100.8%
NVFP4 (FP4 weights/activation… IFEval 0-shot prompt-level strict… 0.946 → 0.94 99.3%
NVFP4 (FP4 weights/activation… MMLU-Pro 0-shot custom-extract (w… 0.871 → 0.864 99.2%
NVFP4 (FP4 weights/activation… AIME 2025 0-shot pass@1, mean of … 0.871 → 0.863 99.0%
NVFP4 (FP4 weights/activation… IFEval 0-shot prompt-level strict… 0.907 → 0.898 99.0%
NVFP4 (FP4 weights/activation… MATH-500 0-shot pass@1 (with thin… 0.879 → 0.87 98.9%
NVFP4 (FP4 weights/activation… LiveCodeBench v6 0-shot pass@1 (w… 0.714 → 0.707 98.9%
NVFP4 (FP4 weights/activation… MMLU-Pro 0-shot custom-extract (w… 0.854 → 0.845 98.9%
NVFP4 (FP4 weights/activation… GPQA Diamond 0-shot pass@1 (witho… 0.774 → 0.766 98.9%
NVFP4 (FP4 weights/activation… AIME 2025 0-shot pass@1, mean of … 0.658 → 0.65 98.7%
NVFP4 (FP4 weights/activation… MATH-500 0-shot pass@1 (without t… 0.894 → 0.851 95.2%
Each kept value links to its source.
[interactive version](https://whatmodel.sh/model/gemma-4-31b) · [JSON](https://whatmodel.sh/api?format=json&all=1)