[whatmodel.sh](../../README.md) / Gemini 2.5 Flash Preview (05-20) # Gemini 2.5 Flash Preview (05-20) Google · released 2025-05 · proprietary **score 61.8** ±1.1 · comparison only: no current run option verified · from 16 benchmark results across 7 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks Gemini 2.5 Flash Preview (05-20) among every model on that task or test, worst to best. 
task         vs the others          rank  counts  tests
agent tasks  ▁▁▂▆●▄▅▄▄▃▁▁   #242 of 354*     12%      1
coding       ▁▁▁▃▄▆█▇●▄▃▁    #153 of 539     50%      3
math         ▁▁▃▆██▅●▃▁▁▁   #154 of 530*     10%      1
agent tasks · #242 of 354 · 12% of its score Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md), [DeepSeek R1 (Jan '25)](deepseek-r1.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Mellum2 12B-A2.5B Instruct](mellum2-12b-a2-5b-instruct.md) here, though they rank lower overall. 
test              result  vs the others in its run  difficulty            counts  why
τ²-bench Telecom   16.9%  ●█··█·██··██ beats 0/6    very hard · avg 8.9%     12%  sharp · at its level
coding · #153 of 539 · 50% of its score Beats [Qwen3.5 Plus](qwen3-5-plus.md), [Qwen3-Coder-Next (80B-A3B)](qwen3-coder-next.md), [Gemini 2.5 Pro](gemini-2-5-pro.md) here, though they rank higher overall. Trails [Llama 3.1 Nemotron Ultra 253B v1](llama-3-1-nemotron-ultra-253b.md) here, though they rank lower overall. 
test                                result  vs the others in its run  difficulty            counts  why
Aider polyglot (225 exercises, tw…   55.1%  ▂·▂▃▅▂▃●▆▄▂▂ beats 30/49  very hard · avg 7.6%     34%  sharp · at its level
LiveCodeBench official (Aug 1, 20…   61.9%  ▃▃▅▃··▃▅●▅▅▃ beats 8/15   hard · avg 24%          8.6%  sharp · at its level
WebApp1K-Duo-React pass@1 (onekq-…   63.4%  ▁··▁·▁▁▁▆●▇█ beats 18/45  hard · avg 29%          7.5%  at its level
math · #154 of 530 · 10% of its score Beats [Claude Sonnet 4.6](claude-sonnet-4-6.md), [MiMo-V2.5 (310B-A15B)](mimo-v2-5.md), [MiniMax-M2.7](minimax-m2-7.md) here, though they rank higher overall. Trails [Olmo 3 32B Think](olmo-3-32b-think.md), [Nemotron-Cascade-8B](nemotron-cascade-8b.md), [Olmo 3.1 32B Think](olmo-3-1-32b-think.md) here, though they rank lower overall. 
test                      result  vs the others in its run    difficulty            counts  why
OTIS mock AIME 2024–2025   70.8%  ▃▃▄▃▃▃▇●█▃▂▅ beats 102/197  very hard · avg 7.2%     10%  sharp · at its level · noisy
other tests (11) 
test                                result  vs the others in its run  difficulty           counts  why
AHELM Speech Robust Bench (LibriS…    7.7%  ▂·█··▆▄·▂●·▄ beats 11/13  avg 34%                9.0%  typical
HindiGen-v1 3C3H score (Inception…   73.7%  ▁▁▁▁·▂·█▆●▇▁ beats 24/31  medium · avg 46%       5.7%  at its level
AHELM CoRe-Bench conversational a…   64.4%  ▂·▄··▂▂·●▂█▄ beats 5/13   medium · avg 52%       3.5%  typical
AHELM VoxCeleb2 same-speaker dete…   75.9%  ▂·▂·▂█▂▄▄··● beats 13/13  very hard · avg 54%    3.1%  sharp · noisy
AHELM LibriSpeech transcription b…    7.7%  ▃···▃·█▃●▃▅█ beats 6/13   avg 11%                2.3%  typical
AHELM MUStARD audio sarcasm detec…   57.4%  ▂·······▆●▆▆ beats 4/13   very hard · avg 57%    2.1%  blunt
AHELM AIR-Bench Foundation music …   57.9%  ▃█▃▅●·▃·▃▅▃▃ beats 7/13   medium · avg 57%       0.8%  blunt · at its level
AHELM MELD audio emotion classifi…   34.0%  ▃··▃●▃·▃▅▅█▅ beats 2/13   medium · avg 39%       0.7%  blunt · at its level
AHELM FLEURS 7-language transcrip…  273.2%  ●▂···▂▆▂█▂·▂ beats 0/13   avg 83%                0.4%  blunt · noisy
AHELM Multilingual LibriSpeech 7-…   60.3%  ●▂▂··▂▂·▂▂█▆ beats 0/13   avg 14%                0.1%  blunt · noisy
AHELM VocalSound human-sound reco…   62.6%  ██▄●·█·█·█▄▄ beats 5/13   easy · avg 67%         0.0%  blunt · at its level · noisy
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
Aider polyglot benchmark runs                   1   34% ▇▇▇▇
AHELM: A Holistic Evaluation of Audio-L…       10   22% ▇▇▇
Moonshot AI Kimi-K2-Instruct model card         1   12% ▇
Epoch AI                                        1   10% ▇
LiveCodeBench official website results          1    9% ▇
WebApp1K models leaderboard                     1    7% ▇
Inception Arabic/Hindi leaderboards             1    6% ▇
prior: pull toward the middle                        1% ▇
weighted average                                   = 61.8
Licences: Aider polyglot benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) · AHELM: A Holistic Evaluation of Audio-Language Models ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Moonshot AI Kimi-K2-Instruct model card ([Modified MIT License (Kimi K2)](https://huggingface.co/moonshotai/Kimi-K2-Instruct/blob/fd1984e2b7a3350dbf7305fe73a4ede25c14de50/LICENSE)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · LiveCodeBench official website results ([CC-BY-SA-4.0](../../licenses/lcb-sharealike-NOTICE.txt)) · WebApp1K models leaderboard ([MIT](../../licenses/webapp1k-leaderboard-MIT.txt)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/gemini-2-5-flash-preview-05-20) · [JSON](https://whatmodel.sh/api?format=json&all=1)