[whatmodel.sh](../../README.md) / Gemini 2.5 Flash
# Gemini 2.5 Flash
Google · released 2025-06 · closed weights
**score 61.8** ±0.7 · #185 of 732 · from 42 benchmark results across 16 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Gemini 2.5 Flash among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆●▄▅▅▄▄▂▁ #192 of 313 24% 10
coding ▁▁▁▃▅▇█●▅▄▃▁ #207 of 469 13% 4
chat & writing ▁▁▁▃▆▄▆█▆●▇▃ #109 of 402* 7.2% 2
math ▁▁▃▅█▆▄●▄▂▁▁ #83 of 431 6.5% 4
vision & documents ▁▁▂▂▁▃▄▂●▇█▄ #82 of 153 17% 7
agent tasks · #192 of 313 · 24% of its score
Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md), [DeepSeek R1 0528](deepseek-r1-0528.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Mellum2 12B-A2.5B Instruct](mellum2-12b-a2-5b-instruct.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 40.6% ▁▁▁▁▁▁●▄█▆▁▁ beats 27/138 hard · avg 17% 4.2% sharp · at its level
ContractBench success rate (33 ob… 19.2% ▆··●▄▂▄▂▆▆█▄ beats 5/23 very hard · avg 13% 4.0% sharp · at its level
Terminal-Bench 2.0 with Terminus … 16.9% ▃··▃·●▃██▅▃▅ beats 2/14 very hard · avg 6.1% 3.0% sharp
τ²-bench Retail 64.8% ●···█·▃····▃ beats 0/6 medium · avg 40% 3.0% at its level
BFCL V4 (December 2025 archive) 56.2% ▁▁▁▂██▅▅●▂▂▂ beats 67/77 hard · avg 31% 2.4% at its level
τ²-bench Telecom 16.5% ●█··███···██ beats 0/6 very hard · avg 8.8% 2.3% sharp · at its level · noisy
τ²-bench Airline 41.5% ▄·●··▄···█▄▄ beats 1/6 hard · avg 25% 1.8% at its level
Galileo Agent Leaderboard v2 (act… 38.0% ▅▃·▃▃▅●█▃█▅▅ beats 8/19 hard · avg 26% 1.0% at its level
Terminal-Bench (pre-2.0 release) 12.3% ●·▄···▄▄···█ beats 0/5 very hard · avg 12% 1.0% at its level
τ²-Bench-Verified (GPT-5.1 user) 41.6% ●··▄···▂▄▄█▄ beats 0/13 medium · avg 45% 0.9% at its level
coding · #207 of 469 · 13% of its score
Trails [QwQ-32B-Preview](qwq-32b-preview.md), [Ministral 3 3B Reasoning 2512](ministral-3-3b-reasoning.md), [DeepSeek-Coder-V2 Instruct (0724)](deepseek-coder-v2-0724.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 40.6% ▁▁▁▁▁▁●▄█▆▁▁ beats 27/138 hard · avg 17% 4.2% sharp · at its level
LiveCodeBench (version unspecifie… 39.6% ●·······▄▄▄▄ beats 1/5 hard · avg 21% 3.3% at its level
HumanEval+ 87.8% ▄●▄·····▄▄·█ beats 1/6 easy · avg 66% 3.0% sharp
MBPP+ 76.2% ●·····▃█▃··▃ beats 0/6 medium · avg 60% 2.7% typical
chat & writing · #109 of 402 · 7.2% of its score
Beats [LongCat 2.0 (1.6T-A48B)](longcat-2-0.md), [Inkling Small (266B-A12B)](inkling-small.md), [GPT-5.4 nano](gpt-5-4-nano.md) here, though they rank higher overall. Trails [Mistral Large 3 (675B-A41B)](mistral-large-3.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LMArena (text) 1409.2 ▁▁▁▂▃▃▃▆▆●█▃ beats 255/385 avg 1260 4.9% blunt
IFEval (aggregation unspecified) 83.9% ●··█·█··████ beats 0/6 easy · avg 74% 2.2% typical
math · #83 of 431 · 6.5% of its score
Beats [Claude Sonnet 4.6](claude-sonnet-4-6.md), [Claude Opus 4.5](claude-opus-4-5.md), [Qwen3.5 397B-A17B](qwen3-5-397b-a17b.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
AIME 2025 67.3% ▄▄···▄▄··▄·● beats 5/6 hard · avg 16% 2.4% sharp · at its level · noisy
MathNet-Solve-Test micro-average … 41.1% ▃▃···●▃·▃·▃▃ beats 3/8 very hard · avg 15% 1.5% at its level
AIME 2024 79.7% ▁▁▁▂▁▃▁▅█●█▃ beats 52/85 hard · avg 30% 1.4% sharp · at its level · noisy
MATH-500 98.4% ▄·▄··█··▄·●▄ beats 5/6 very easy · avg 87% 1.1% maxed out · noisy
vision & documents · #82 of 153 · 17% of its score
Beats [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md), [GPT-5.4 nano](gpt-5-4-nano.md), [GPT-5 mini](gpt-5-mini.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
CharXiv (reasoning questions) (ma… 63.7% ▃·●·▃▃▃·▃▃█▅ beats 1/11 medium · avg 38% 10% at its level · several runs
Arena vision (style controlled) 1215.5 ▁▁▂▂▂▂▃▄▅●█▇ beats 81/152 avg 1061 3.3% blunt
BEAR-Bench (bilingual RU/EN enter… 49.0% ▃▃·▅·▃·●·█▅▅ beats 6/14 hard · avg 24% 1.7% at its level
CharXiv descriptive questions 83.6% ▄····●···█▄▄ beats 1/5 easy · avg 67% 1.1% noisy
Molmo2-VideoCount accuracy (val) 31.9% ▃··▃·●····█▃ beats 2/6 hard · avg 22% 0.5% blunt · at its level
Ai2 Molmo2 human preference Elo (… 1084.0 ▄··▄··▄··▄●▄ beats 5/6 avg 968 0.4% blunt
Molmo2-CapTest dense video captio… 46.0% █···█·██●█·█ beats 4/6 hard · avg 31% 0.2% blunt · at its level
other tests (15)
test result vs the others in its run difficulty counts why
MMLU 86.3% ●·····█▄▄▄·▄ beats 0/6 medium · avg 68% 6.6% typical
GPQA Diamond 80.3% █·█·█··██··● beats 5/5 hard · avg 40% 5.7% sharp · at its level
MMLU-Pro 82.0% ●▄·▄···▄··▄▄ beats 1/6 medium · avg 58% 4.7% typical
KCL-Essay vanilla 53.7% ▁···▁▃▂█▂▃●▂ beats 21/23 hard · avg 17% 3.6% at its level
BRIDGE clinical text, zero-shot a… 44.8% ▁▁▁▁▁▂▂▃▄▆█● beats 94/97 hard · avg 31% 3.1% blunt · at its level
CMMLU 78.3% ●····▄··▄▄▄▄ beats 1/6 medium · avg 67% 2.4% typical
C-Eval 78.8% ●····▄·▄·▄▄▄ beats 0/6 medium · avg 68% 2.3% typical
COLLIE (constrained text generati… 48.6% ▄···●█▄····█ beats 1/6 hard · avg 30% 1.9% at its level
KCL-MCQA vanilla 51.8% ▂···█▂▆▆█●▆▂ beats 19/23 very hard · avg 29% 1.8% at its level
VitaBench 8.0% ●·······▄█·█ beats 0/6 very hard · avg 6.5% 1.6% typical
ZebraLogic 51.8% ●···█··███·█ beats 0/5 medium · avg 51% 1.0% noisy
Open PT-LLM Leaderboard average (… 85.6% ▃▁▁▁▁▁▁▁▂█▅● beats 275/276 medium · avg 72% 0.8% noisy
BeyondAIME 44.2% █····█·▄●··▄ beats 5/6 hard · avg 24% 0.6% at its level
DROP 45.0% ●···▄·▄█·▄·▄ beats 0/6 medium · avg 58% 0.2% blunt · at its level · noisy
Vectara hallucination leaderboard… 92.2% ▁▂▁▂▅█●▃▄▃▁▁ beats 63/98 very easy · avg 90% 0.0% blunt
source results share of the score
Meituan LongCat-Flash-Chat model card 22 55% ▇▇▇▇▇▇▇
LMArena leaderboard dataset 2 8% ▇
Google DeepMind Gemini model page and m… 1 8% ▇
Korean Canonical Legal Benchmark: Towar… 2 5% ▇
ContractBench: Can LLM Agents Preserve … 1 4% ▇
BRIDGE Medical Leaderboard 1 3% ▇
Terminal-Bench: Benchmarking Agents on … 1 3% ▇
ServiceNow-AI/Apriel-1.6-15b-Thinker mo… 2 3% ▇
Berkeley Function Calling Leaderboard r… 1 2% ▇
BEAR-Bench: A Bilingual Enterprise and … 1 2% ▇
MathNet: a Global Multimodal Benchmark … 1 2% ▇
Molmo2: Open Weights and Data for Visio… 3 1% ▇
Galileo Agent Leaderboard 1 1% ▇
Amazon AGI 1 1% ▇
Open Portuguese LLM Leaderboard 1 1% ▇
Vectara hallucination leaderboard 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 61.8
Licences: Meituan LongCat-Flash-Chat model card ([MIT](https://huggingface.co/meituan-longcat/LongCat-Flash-Chat/blob/f0b8e8d7380330f450ca1fde4ec56823a514584c/LICENSE)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities ([CC BY 4.0](../../licenses/cards-final-NOTICE.txt)) · ContractBench: Can LLM Agents Preserve Observation Contracts? ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · BRIDGE Medical Leaderboard ([MIT](../../licenses/bridge-medical-leaderboard-MIT.txt)) · Terminal-Bench: Benchmarking Agents on Hard ([CC BY 4.0](../../licenses/paper-2601.11868v1-CC-BY-4.0.txt)) · ServiceNow-AI/Apriel-1.6-15b-Thinker model card ([MIT](../../licenses/cards-final-NOTICE.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval ([CC BY 4.0](../../licenses/paper-2604.18584v2-CC-BY-4.0.txt)) · Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding ([CC BY 4.0](http://creativecommons.org/licenses/by/4.0/)) · Galileo Agent Leaderboard ([Apache-2.0](../../licenses/galileo-agent-leaderboard-Apache-2.0.txt)) · Amazon AGI ([MIT](../../licenses/tau2-verified-MIT.txt)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
### Through an API
api (Google) · $0.85/M blended · $0.3 in / $2.5 out per 1M tokens · [ai.google.dev](https://ai.google.dev/gemini-api/docs/models)
[price source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/providers/google/models/gemini-2.5-flash.toml): Standard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/models/google/gemini-2.5-flash.toml)
[interactive version](https://whatmodel.sh/model/gemini-2-5-flash) · [JSON](https://whatmodel.sh/api?format=json&all=1)