[whatmodel.sh](../../README.md) / Gemini 2.5 Pro
# Gemini 2.5 Pro
Google · released 2025-06 · closed weights
**score 66.4** ±0.5 · #119 of 732 · from 52 benchmark results across 22 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Gemini 2.5 Pro among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█▄●▅▄▄▂▁ #147 of 313 21% 12
coding ▁▁▁▃▅▇█●▅▄▃▁ #176 of 469 17% 4
chat & writing ▁▁▁▃▆▄▆█▆▇●▃ #59 of 402* 3.9% 2
math ▁▁▃▅█▆▄●▄▂▁▁ #122 of 431 26% 7
vision & documents ▁▁▂▂▁▃▄▂▆●█▄ #55 of 153 12% 9
web search ▁▁▁●▄█▃▄·▁·▁ #66 of 84* 1.1% 1
agent tasks · #147 of 313 · 21% of its score
Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Large](jamba-1-5-large.md), [GPT-4o (2024-05-13)](gpt-4o-2024-05-13.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 57.6% ▁··▁·●▂▁▄█▄▁ beats 2/29 hard · avg 19% 7.0% sharp · at its level
ContractBench success rate (33 ob… 51.5% ▆··▆▄▂▄▂●▆█▄ beats 13/23 very hard · avg 13% 2.5% sharp · at its level
Terminal-Bench 2.0 with Terminus … 32.6% ▃··▃·▃▃█●▅▃▅ beats 9/14 very hard · avg 6.1% 2.3% sharp · at its level
τ²-bench Retail 70.9% █···█·█·●█·█ beats 3/5 hard · avg 35% 2.2% at its level
τ³-bench banking_knowledge (v1.0.… 13.7% ▃●▂▃·▆▃▂█▅▅▂ beats 3/25 very hard · avg 4.3% 1.4% typical
τ²-bench Telecom 38.3% █··●█·█·█··█ beats 1/5 very hard · avg 9.6% 1.4% sharp · at its level · noisy
τ²-bench Airline 58.0% ▄·····●··▄·█ beats 1/5 hard · avg 28% 1.1% at its level
EEGAgentBench overall (tool-based… 44.2% ▂·▁▁··●▇█▂·▁ beats 8/25 hard · avg 28% 0.9% blunt · at its level
BFCL V4 (2025-11-03 archive, mode… 52.3% ▃··▅▃▃·●·▃·█ beats 6/10 hard · avg 33% 0.7% at its level
Galileo Agent Leaderboard v2 (act… 43.0% ▅▃·▃▃▅▅●▃█▅▅ beats 10/19 hard · avg 26% 0.6% at its level
τ²-Bench-Verified (GPT-5.1 user) 56.4% ▂··●···▂▄▄█▄ beats 2/13 medium · avg 45% 0.5% at its level
BFCL v3 63.2% ▁▁▁▁▁▃▅█●▅▄▁ beats 111/171 medium · avg 54% 0.3% blunt · at its level
coding · #176 of 469 · 17% of its score
Beats [Qwen3-Coder-Next (80B-A3B)](qwen3-coder-next.md) here, though they rank higher overall. Trails [QwQ-32B-Preview](qwq-32b-preview.md), [Ministral 3 8B Reasoning 2512](ministral-3-8b-reasoning.md), [Phi-4-reasoning 14B](phi-4-reasoning.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 57.6% ▁··▁·●▂▁▄█▄▁ beats 2/29 hard · avg 19% 7.0% sharp · at its level
Arena WebDev (Code Arena) 1226.1 ▁▁●▂▅█▃▆▅▅▁▂ beats 7/129 avg 981 5.3% sharp
Luau Leetcode fully solved (Torpe… 65.4% ▁▁▁▁·▁▂▃●▃█▇ beats 18/49 hard · avg 17% 2.7% sharp · at its level
LiveCodeBench (version unspecifie… 74.2% ▄······▄●▄·▄ beats 2/5 medium · avg 41% 1.6% at its level
chat & writing · #59 of 402 · 3.9% of its score
Beats [Claude Haiku 5.5](claude-haiku-5-5.md), [GPT-6 Luna](gpt-6-luna.md), [Grok 4.7](grok-4-7.md) here, though they rank higher overall. Trails [Gemma 4 31B](gemma-4-31b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LMArena (text) 1445.4 ▁▁▁▂▃▃▃▆▆▇●▃ beats 312/385 avg 1260 3.0% blunt
IFEval prompt-level strict 92.4% ███··█·█···● beats 5/5 easy · avg 73% 0.9% typical
math · #122 of 431 · 26% of its score
Beats [Claude Sonnet 4.6](claude-sonnet-4-6.md), [MiniMax-M3](minimax-m3.md), [MiMo-V2.5 (310B-A15B)](mimo-v2-5.md) here, though they rank higher overall. Trails [Olmo 3.1 32B Think](olmo-3-1-32b-think.md), [Granite 4.2 8B](granite-4-2-8b.md), [gpt-oss-120b](gpt-oss-120b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
FrontierMath tiers 1–3 v2 24.6% ▂··▁▁▃●▄█▆▄▃ beats 16/86 very hard · avg 1.2% 11% sharp · at its level
AIME 2025 89.2% ▁·▁▁▁▂▅▆█▇●▁ beats 113/133 medium · avg 50% 3.4% sharp
FrontierMath tier 4 v2 0.0% ●▂▁▂█▅▂▃▂▁▁▁ beats 1/67 very hard · avg 0.1% 3.0% sharp · too hard for it
AIME 2024 90.7% ▁▁▁▂▁▃▁▅██●▃ beats 74/85 medium · avg 61% 2.8% sharp · maxed out
HMMT 2025 (edition unspecified) 79.3% ▄··▄··●·▄··█ beats 2/5 hard · avg 31% 2.5% sharp
OTIS mock AIME 2024–2025 84.7% ▃▃▄▃▃▃▇●█▃▂▅ beats 126/197 very hard · avg 7.2% 1.7% sharp · noisy
MATH-500 98.0% █···●█·█·█·█ beats 1/5 very easy · avg 92% 0.6% maxed out · noisy
vision & documents · #55 of 153 · 12% of its score
Beats [GPT-6 Luna](gpt-6-luna.md), [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md), [GPT-5](gpt-5.md) here, though they rank higher overall. Trails [Qwen3.6 Flash](qwen3-6-flash.md), [Gemma 4 31B](gemma-4-31b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
CharXiv (reasoning questions) (ma… 69.6% ▃·▃·●▃▃·▃▃█▅ beats 2/11 medium · avg 38% 4.3% at its level
Arena vision (style controlled) 1247.6 ▁▁▂▂▂▂▃▄▅▅●▇ beats 104/152 avg 1061 2.0% blunt
ChartArena chart parsing (English… 47.7% ▁·▁·▃▃▁▇▃●·▄ beats 19/22 hard · avg 27% 1.6% blunt · at its level
CharXiv descriptive questions 93.4% ▂▄▄▅▇▄██▇●▁▂ beats 56/59 easy · avg 69% 1.2% typical
Arena document (style controlled) 1430.6 ▁▁▃●▄▃▇█▅▇▄▅ beats 4/37 avg 1331 1.0% blunt
BEAR-Bench (bilingual RU/EN enter… 67.4% ▃▃·▅·▃·█·█●▅ beats 12/14 hard · avg 24% 1.0% at its level
Molmo2-VideoCount accuracy (val) 35.8% ▃··▃·▃····●▃ beats 4/6 hard · avg 22% 0.3% blunt · at its level
Ai2 Molmo2 human preference Elo (… 1096.0 ▄··▄··▄··▄█● beats 6/6 avg 968 0.2% blunt
Molmo2-CapTest dense video captio… 42.1% █···█·●███·█ beats 2/6 hard · avg 31% 0.1% blunt · at its level
web search · #66 of 84 · 1.1% of its score
Beats [DeepSeek V4 Flash (Preview, Apr 2026)](deepseek-v4-flash-preview.md), [Grok 4.3](grok-4-3.md), [Nemotron 3 Ultra 550B-A55B](nemotron-3-ultra-550b-a55b.md) here, though they rank higher overall. Trails [GPT-4.1 nano](gpt-4-1-nano-2025-04-14.md), [GPT-4o mini (2024-07-18)](gpt-4o-mini-2024-07-18.md), [Mistral Small 3.1 24B (2503)](mistral-small-3-1.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena search systems (style contr… 1144.2 ▁·▁▁●▂▂█▆▃▂▂ beats 4/36 avg 1069 1.1% blunt
other tests (17)
test result vs the others in its run difficulty counts why
GPQA Diamond 85.3% ▁▁▁▁▂▄▆▄▅●█▅ beats 146/211 hard · avg 42% 8.8% sharp · several runs
KCL-Essay vanilla 60.5% ▁···▁▃▂█▂▃▃● beats 23/23 hard · avg 17% 2.3% at its level
MMLU-Pro 86.7% ▄·▄···█·▄··● beats 5/5 medium · avg 55% 2.2% typical
MMLU-Redux 90.1% ██●█··█····█ beats 2/5 medium · avg 73% 1.8% typical
ZebraLogic 92.4% █●··█·██···█ beats 1/5 easy · avg 75% 1.5% maxed out
Chess Puzzles (Epoch) 20.0% █▁▁▁▁▂●▃▃▂▁▁ beats 80/137 very hard · avg 5.8% 1.3% sharp · noisy
KCL-MCQA vanilla 66.1% ▂···█▂▆▆██▆● beats 23/23 very hard · avg 29% 1.3% at its level
OJBench 41.6% ▄······▄▄·▄● beats 5/5 very hard · avg 12% 1.3% at its level
VitaBench 24.3% █····█●·██·█ beats 2/5 very hard · avg 8.2% 1.2% at its level
RobloxQA v2.0 multiple choice (To… 92.8% ▁▁▃▁█●▃▃▆▂▂▂ beats 24/49 easy · avg 77% 1.1% typical
LongSocialBench (full-context, 4-… 51.5% ▆▂·▄█·▂▂·●▄▂ beats 13/16 very hard · avg 36% 1.1% blunt · at its level
Icelandic LLM Leaderboard average… 52.2% ▁▁▃▂▂▄▁▃▄●▄█ beats 21/38 very hard · avg 14% 1.0% at its level
BeyondAIME 63.0% ●··▄····▄··█ beats 0/5 medium · avg 42% 0.8% at its level
FinSheet-Bench accuracy (501 ques… 78.8% ●···██·····█ beats 0/3 medium · avg 53% 0.7% typical
DiagnosticIQ industrial maintenan… 57.6% ▁··▃▁▁▁●█▅▁▅ beats 6/24 hard · avg 51% 0.6% blunt · at its level
miniF2F (pass@1) 13.9% █●······▄··▄ beats 2/5 very hard · avg 14% 0.1% blunt · at its level · noisy
Vectara hallucination leaderboard… 93.0% ▁▂▁▂▅█▂●▄▃▁▁ beats 69/98 very easy · avg 90% 0.0% blunt
source results share of the score
Meituan LongCat-Flash-Thinking model ca… 18 30% ▇▇▇▇
Epoch AI 6 29% ▇▇▇
LMArena leaderboard dataset 5 12% ▇
Google DeepMind Gemini model page and m… 1 4% ▇
Roblox LLM Leaderboard 2 4% ▇
Korean Canonical Legal Benchmark: Towar… 2 4% ▇
ContractBench: Can LLM Agents Preserve … 1 2% ▇
Terminal-Bench: Benchmarking Agents on … 1 2% ▇
ChartArena: Benchmarking Chart Parsing … 1 2% ▇
τ²/τ³-bench leaderboard: Sierra's own e… 1 1% ▇
Baidu - ERNIE 5.0 Technical Report 1 1% ▇
LongSocialBench: Do Long-Context LLMs U… 1 1% ▇
Miðeind Icelandic LLM Leaderboard 1 1% ▇
BEAR-Bench: A Bilingual Enterprise and … 1 1% ▇
EEGAgentBench: Benchmarking LLM Agents … 1 1% ▇
FinSheet-Bench: From Simple Lookups to … 1 1% ▇
Molmo2: Open Weights and Data for Visio… 3 1% ▇
Berkeley Function Calling Leaderboard r… 1 1% ▇
DiagnosticIQ: A Benchmark for LLM-Based… 1 1% ▇
Galileo Agent Leaderboard 1 1% ▇
Amazon AGI 1 1% ▇
Vectara hallucination leaderboard 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 66.4
Licences: Meituan LongCat-Flash-Thinking model card ([MIT](https://huggingface.co/meituan-longcat/LongCat-Flash-Thinking/blob/7c9aad1e2daceaeab3a5a03deedb8e11e7a7819f/LICENSE)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Roblox LLM Leaderboard ([MIT](../../licenses/roblox-llm-leaderboard-MIT.txt)) · Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities ([CC BY 4.0](../../licenses/cards-final-NOTICE.txt)) · ContractBench: Can LLM Agents Preserve Observation Contracts? ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Terminal-Bench: Benchmarking Agents on Hard ([CC BY 4.0](../../licenses/paper-2601.11868v1-CC-BY-4.0.txt)) · ChartArena: Benchmarking Chart Parsing across Languages ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · τ²/τ³-bench leaderboard: Sierra's own evaluation runs ([MIT](https://opensource.org/license/mit)) · Baidu - ERNIE 5.0 Technical Report ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · LongSocialBench: Do Long-Context LLMs Understand Online Discussion Threads? ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Miðeind Icelandic LLM Leaderboard ([CC BY 4.0](../../licenses/mideind-icelandic-llm-leaderboard-CC-BY-4.0.txt)) · BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · FinSheet-Bench: From Simple Lookups to Complex Reasoning ([CC BY 4.0](../../licenses/paper-2603.07316v1-CC-BY-4.0.txt)) · Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding ([CC BY 4.0](http://creativecommons.org/licenses/by/4.0/)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Galileo Agent Leaderboard ([Apache-2.0](../../licenses/galileo-agent-leaderboard-Apache-2.0.txt)) · Amazon AGI ([MIT](../../licenses/tau2-verified-MIT.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
### Through an API
api (Google) · $3.44/M blended · $1.25 in / $10 out per 1M tokens · [ai.google.dev](https://ai.google.dev/gemini-api/docs/models)
[price source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/providers/google/models/gemini-2.5-pro.toml): Standard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/models/google/gemini-2.5-pro.toml)
[interactive version](https://whatmodel.sh/model/gemini-2-5-pro) · [JSON](https://whatmodel.sh/api?format=json&all=1)