[whatmodel.sh](../../README.md) / Gemini 3 Pro (Preview)
# Gemini 3 Pro (Preview)
Google · released 2025-11 · closed weights
**score 71.4** ±0.7 · comparison only: no current run option verified · from 36 benchmark results across 21 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Gemini 3 Pro (Preview) among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▂▆█▄▅●▄▃▁▁ #86 of 354 31% 10
coding ▁▁▁▃▄▆█▇▅●▃▁ #80 of 539 19% 3
chat & writing ▁▁▁▃▅▄▇█▇▇▇● #19 of 474* 4.8% 1
math ▁▁▃▆██▅█●▁▁▁ #57 of 530 10% 5
vision & documents ▁▁▂▃▂▃▅▄▆█●▄ #24 of 189 15% 9
web search ▂▁▂▆▅●▅▄▁▂▁▁ #38 of 96* 1.7% 1
agent tasks · #86 of 354 · 31% of its score
Trails [MiMo-V2.5 (310B-A15B)](mimo-v2-5.md), [Qwen3.6 27B](qwen3-6-27b.md), [Muse Glimmer 30B](muse-glimmer-30b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 72.9% ▁··▁·▁▂▁●█▄▁ beats 11/29 hard · avg 19% 8.9% sharp
MCP-Atlas (public set) 66.6% ▄······▄▄▄●█ beats 4/6 hard · avg 26% 5.6% at its level · several runs
τ²-bench Telecom (maker) 98.0% ▁·▅█▆▅▂▃▃▂●▁ beats 81/89 medium · avg 61% 3.6% sharp · maxed out · small run
τ³-bench banking_knowledge (v1.0.… 18.0% ▃▃▂●·▆▃▂█▅▅▂ beats 6/25 very hard · avg 4.3% 2.9% at its level
τ²-bench aggregate 85.4% ▄····●▄▄···▄ beats 2/5 medium · avg 45% 2.4% typical
BFCL V4 (December 2025 archive) 68.1% ▁▁▁▂██▅▅▆▂●▂ beats 72/77 hard · avg 31% 2.1% at its level
τ³-bench airline (tau2-bench ≥ v1… 80.5% ▃······●·▃█▃ beats 1/6 medium · avg 51% 1.8% typical
τ³-bench retail (tau2-bench ≥ v1.… 75.9% ▄▄·●··▄·▄··▄ beats 2/6 medium · avg 48% 1.7% typical
τ³-bench telecom (tau2-bench ≥ v1… 91.0% █·▄●▄······▄ beats 3/6 medium · avg 64% 1.4% typical
τ²-Bench-Verified (GPT-5.1 user) 79.4% ▂··▄···▂▄▄●▄ beats 9/13 medium · avg 45% 0.7% typical
coding · #80 of 539 · 19% of its score
Beats [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md), [GPT-5.2](gpt-5-2.md) here, though they rank higher overall. Trails [Llama 3.1 Nemotron Ultra 253B v1](llama-3-1-nemotron-ultra-253b.md), [MiMo-V2.5 (310B-A15B)](mimo-v2-5.md), [MiMo-V2.5-Pro (1T-A42B)](mimo-v2-5-pro.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 72.9% ▁··▁·▁▂▁●█▄▁ beats 11/29 hard · avg 19% 8.9% sharp
Arena WebDev (Code Arena) 1440.0 ▁▁▃▂▅●▃▆▅▅▁▂ beats 60/129 avg 981 8.5% sharp
WebApp1K-Duo-React pass@1 (onekq-… 75.0% ▁··▁·▁▁▁▆█▇● beats 37/45 hard · avg 29% 1.9% typical
chat & writing · #19 of 474 · 4.8% of its score
Beats [GPT-6 Astra](gpt-6-astra.md), [GPT-6.1 Sol](gpt-6-1-sol.md), [Claude Sonnet 5.5](claude-sonnet-5-5.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
LMArena (text) 1485.4 ▁▁▁▂▃▃▃▆▆▇█● beats 368/385 avg 1260 4.8% blunt
math · #57 of 530 · 10% of its score
Trails [A.X K2 (688B-A33B)](ax-k2.md), [DeepSeek V4 Flash (Preview, Apr 2026)](deepseek-v4-flash-preview.md), [Gemini 3 Flash (Preview)](gemini-3-flash-preview.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
MathArena ArXivMath January 2026 … 62.0% ▂··▄▂▄█▆●▄▄█ beats 14/23 hard · avg 23% 3.8% at its level
MathArena ArXivMath December 2025… 51.1% ▃·▃▃█▁·●·▁▁▁ beats 14/18 hard · avg 19% 2.8% at its level
MATH-Vision 86.6% █·····▄▄·●·▄ beats 4/5 medium · avg 47% 1.6% typical
OTIS mock AIME 2024–2025 91.4% ▃▃▄▃▃▃▇▇●▃▂▅ beats 149/197 very hard · avg 7.2% 1.2% sharp · noisy
FormalProofBench (200 graduate-le… 18.5% ▃▅·▃··▅▃●··▃ beats 9/10 very hard · avg 3.1% 0.9% noisy
vision & documents · #24 of 189 · 15% of its score
Beats [Claude Sonnet 5.5](claude-sonnet-5-5.md), [GPT-6 Sol](gpt-6-sol.md), [MiMo-V2.6-Pro](mimo-v2-6-pro.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
CharXiv (reasoning questions) (ma… 81.4% ▃·▃·▃▃▃·▃●█▅ beats 6/11 medium · avg 38% 5.5% typical
Arena vision (style controlled) 1289.3 ▁▁▂▂▂▂▃▄▅▅█● beats 141/152 avg 1061 3.2% blunt
Arena document (style controlled) 1451.1 ▁▁▃▃▄▃●█▅▇▄▅ beats 12/37 avg 1331 1.6% blunt
MATH-Vision 86.6% █·····▄▄·●·▄ beats 4/5 medium · avg 47% 1.6% typical
CharXiv descriptive questions 96.0% ▂▄▄▅▇▄██▇▂▁● beats 58/59 easy · avg 69% 1.4% maxed out
Diagram-MMU diagram question answ… 86.5% ▄·····▄▄·█·● beats 6/6 medium · avg 56% 0.7% blunt
Molmo2-VideoCount accuracy (val) 37.1% ▃··▃·▃····█● beats 6/6 hard · avg 22% 0.5% blunt · at its level
Ai2 Molmo2 human preference Elo (… 1082.0 ▄··▄··▄··▄●▄ beats 4/6 avg 968 0.4% blunt
Molmo2-CapTest dense video captio… 36.0% █···●·████·█ beats 1/6 hard · avg 31% 0.2% blunt · at its level
web search · #38 of 96 · 1.7% of its score
Beats [Gemini 3.1 Pro (Preview)](gemini-3-1-pro.md), [Claude Opus 4.7](claude-opus-4-7.md), [Claude Sonnet 5](claude-sonnet-5.md) here, though they rank higher overall. Trails [GPT-4o mini (2024-07-18)](gpt-4o-mini-2024-07-18.md), [Mistral Small 3.1 24B (2503)](mistral-small-3-1.md), [GPT-4o (2024-08-06)](gpt-4o-2024-08-06.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena search systems (style contr… 1200.8 ▁·▁▁▂▂▂█●▃▂▂ beats 21/36 avg 1069 1.7% blunt
other tests (8)
test result vs the others in its run difficulty counts why
Medmarks-V mean win rate (19 veri… 0.6628 ▁▄▃▃▇▄█▃▅▁▁● beats 30/30 avg 0.418 8.8% sharp
WebCompass (overall) 66.7% ▅·▃▃▃···▃·▃● beats 8/9 hard · avg 18% 4.6% at its level
GPQA Diamond 92.6% ▁▁▁▁▂▄▆▄▅▇█● beats 190/211 hard · avg 39% 4.4% sharp
Chess Puzzles (Epoch) 31.0% █▁▁▁▁▂▃▃●▂▁▁ beats 106/137 very hard · avg 5.8% 3.5% sharp · noisy
EQ-Bench 3 (emotional intelligenc… 1307.1 ▁·▂▁▄▅▅▃●▄▁▁ beats 48/62 avg 1034 2.9% typical
RPC-Bench overall, text input (Z.… 32.5% ▄··▄▄·▄▄●·▄▄ beats 6/8 very hard · avg 12% 2.0% at its level
AraGen-v3 3C3H score (Inception, … 64.1% ▁▁·▁▂▂▄█▄▅●▄ beats 51/66 hard · avg 29% 1.9% typical
Vectara hallucination leaderboard… 86.4% ▁▂▁●▅█▂▃▄▃▁▁ beats 15/98 very easy · avg 90% 0.0% blunt
source results share of the score
LMArena leaderboard dataset 5 20% ▇▇
Epoch AI 4 18% ▇▇
Google DeepMind Gemini model page and m… 2 9% ▇
Medmarks: A Comprehensive Open-Source L… 1 9% ▇
τ²/τ³-bench leaderboard: Sierra's own e… 4 8% ▇
MathArena ArXivMath / BrokenArXiv / ArX… 2 7% ▇
WebCompass: Towards Multimodal Web Codi… 1 5% ▇
Qwen/Qwen3.5-397B-A17B model card 2 4% ▇
EQ-Bench 3 canonical leaderboard results 1 3% ▇
zai-org/GLM-5 model card 1 3% ▇
GLM-5 Team 1 3% ▇
Berkeley Function Calling Leaderboard r… 1 2% ▇
RPC-Bench leaderboard 1 2% ▇
Inception Arabic/Hindi leaderboards 1 2% ▇
WebApp1K models leaderboard 1 2% ▇
Baidu - ERNIE 5.0 Technical Report 1 1% ▇
Molmo2: Open Weights and Data for Visio… 3 1% ▇
FormalProofBench 1 1% ▇
Amazon AGI 1 1% ▇
Diagram-MMU: A Multi-Modal Benchmark fo… 1 1% ▇
Vectara hallucination leaderboard 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 71.4
Licences: LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks ([CC BY 4.0](../../licenses/paper-2605.01417v1-CC-BY-4.0.txt)) · τ²/τ³-bench leaderboard: Sierra's own evaluation runs ([MIT](https://opensource.org/license/mit)) · MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets ([CC BY-SA 4.0](../../licenses/matharena-sharealike-NOTICE.txt)) · WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models ([CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/)) · Qwen/Qwen3.5-397B-A17B model card ([apache-2.0](../../licenses/Qwen--Qwen3.5-397B-A17B.txt)) · EQ-Bench 3 canonical leaderboard results ([MIT](../../licenses/eqbench3-MIT.txt)) · zai-org/GLM-5 model card ([MIT](../../licenses/zai-org--GLM-5.txt)) · GLM-5 Team ([CC BY 4.0](../../licenses/report-2602.15763v2.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · RPC-Bench leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · WebApp1K models leaderboard ([MIT](../../licenses/webapp1k-leaderboard-MIT.txt)) · Baidu - ERNIE 5.0 Technical Report ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding ([CC BY 4.0](http://creativecommons.org/licenses/by/4.0/)) · FormalProofBench ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Amazon AGI ([MIT](../../licenses/tau2-verified-MIT.txt)) · Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
No verified API or local download yet.
[interactive version](https://whatmodel.sh/model/gemini-3-pro-preview) · [JSON](https://whatmodel.sh/api?format=json&all=1)