[whatmodel.sh](../../README.md) / Gemini 1.5 Flash (002)
# Gemini 1.5 Flash (002)
Google · released 2024-09 · proprietary
**score 53.8** ±0.6 · comparison only: no current run option verified · from 26 benchmark results across 11 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Gemini 1.5 Flash (002) among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▂▆█●▅▄▄▃▁▁ #190 of 354* 0.7% 1
coding ▁▁▁▃▄▆●▇▅▄▃▁ #299 of 539 56% 5
chat & writing ▁▁▁▃▅▄▇●▇▇▇▃ #257 of 474* 4.2% 1
math ▁▁▃▆█●▅█▃▁▁▁ #280 of 530* 11% 2
vision & documents ▁▁▂▃▂▃●▄▆██▄ #125 of 189* 2.8% 1
agent tasks · #190 of 354 · 0.7% of its score
Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md), [DeepSeek R1 0528](deepseek-r1-0528.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Jamba 1.5 Large](jamba-1-5-large.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Galileo Agent Leaderboard v1 (too… 89.5% ▃▃▃▃▆▅▆●█▂▅▂ beats 21/31 easy · avg 78% 0.7% typical
coding · #299 of 539 · 56% of its score
Beats [Qwen3-VL 32B Instruct](qwen3-vl-32b-instruct.md), [Solar Open 100B (102B-A12B)](solar-open-100b.md), [GLM-4.7-Flash](glm-4-7-flash.md) here, though they rank higher overall. Trails [QwQ-32B-Preview](qwq-32b-preview.md), [InternLM2.5 20B Chat](internlm2-5-20b-chat.md), [Ministral 3 3B Reasoning 2512](ministral-3-3b-reasoning.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
HumanEval 82.3% ▁·▁▁▃▄▆▅█●▁▁ beats 87/99 easy · avg 76% 18% sharp
HumanEval+ 75.6% ▁·▁▁▁▃▄▄▄█●▁ beats 86/99 easy · avg 70% 17% sharp
LiveCodeBench official (Jan 1–Nov… 34.4% ▁▁·▁▂▄▆●▅▂·▂ beats 18/31 hard · avg 23% 9.4% sharp · at its level
Aider code editing (133 Exercism … 51.1% ▁·▁▂▄▁●▄█▆▂▂ beats 16/62 medium · avg 51% 7.8% sharp · at its level
MBPP+ 67.5% ▁···▁▁▂▅▇●▆▂ beats 56/75 easy · avg 66% 4.0% at its level
chat & writing · #257 of 474 · 4.2% of its score
Beats [Gemma 2 9B IT SimPO](gemma-2-9b-simpo.md), [Granite 4.2 8B](granite-4-2-8b.md), [Mistral Small 4 (119B-A6.5B)](mistral-small-4.md) here, though they rank higher overall. Trails [LFM2.5 1.2B Instruct](lfm2-5-1-2b-instruct.md), [Granite 4.0 H Micro 3B](granite-4-0-h-micro.md), [Gemma 3n E4B](gemma-3n-e4b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
LMArena (text) 1309.3 ▁▁▁▂▃▃▃●▆▇█▃ beats 139/385 avg 1260 4.2% blunt
math · #280 of 530 · 11% of its score
Beats [GPT-4o (2024-08-06)](gpt-4o-2024-08-06.md), [Command A (03-2025)](command-a.md), [GPT-4o (2024-05-13)](gpt-4o-2024-05-13.md) here, though they rank higher overall. Trails [Gemma 7B (base)](gemma-7b-base.md), [CodeGemma 7B (v1.0)](codegemma-7b.md), [Phi-2 (2.7B)](phi-2.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
MATH level 5 61.9% ▂▂▅▆▄█●▃▅▂▃▄ beats 51/92 hard · avg 28% 9.4% sharp · at its level
OTIS mock AIME 2024–2025 16.3% ▃▃▄▃●▃▇▇█▃▂▅ beats 55/197 very hard · avg 7.2% 1.8% sharp · noisy
vision & documents · #125 of 189 · 2.8% of its score
Beats [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md), [GPT-4o (2024-08-06)](gpt-4o-2024-08-06.md), [GPT-4o (2024-05-13)](gpt-4o-2024-05-13.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
Arena vision (style controlled) 1141.7 ▁▁▂▂▂▂▃●▅▅█▇ beats 43/152 avg 1061 2.8% blunt
other tests (16)
test result vs the others in its run difficulty counts why
MMLU-Pro 64.0% ▃▂▅█▃·●▃▅··▂ beats 16/23 medium · avg 54% 6.8% at its level
GPQA Diamond 47.3% ▁▁▁▁▂▄●▄▅▇█▅ beats 58/211 hard · avg 39% 5.8% sharp · at its level
AHELM Speech Robust Bench (LibriS… 21.4% ▂·█··●▄·▂▂·▄ beats 5/13 avg 34% 3.0% typical
LiveBench 2024-08-31 49.6% ▄█·●·▄▄·▄··▄ beats 3/7 medium · avg 40% 2.3% at its level
EvalPerf correctness (118 tasks, … 77.6% ▂·▂▂▂▆▅●▆▆▃█ beats 14/29 easy · avg 74% 2.1% typical
AHELM CoRe-Bench conversational a… 77.6% ▂·▄··▂▂·▄▂●▄ beats 11/13 medium · avg 52% 1.5% at its level
Open PT-LLM Leaderboard average (… 79.9% ▃▁▁▁▁▁▁▁▂█●▁ beats 259/276 medium · avg 72% 0.9% at its level · noisy
AHELM LibriSpeech transcription b… 6.2% ▃···▃·█▃▅●▅█ beats 8/13 avg 11% 0.7% typical
AHELM VoxCeleb2 same-speaker dete… 54.2% ▂·▂·▂█●▄▄··▄ beats 7/13 very hard · avg 54% 0.6% sharp · noisy
AHELM MUStARD audio sarcasm detec… 55.8% ▂·······●█▆▆ beats 2/13 very hard · avg 57% 0.5% blunt
AHELM AIR-Bench Foundation music … 50.8% ▃●▃▅▃·▃·▃▅▃▃ beats 1/13 medium · avg 57% 0.3% blunt · at its level
Vectara hallucination leaderboard… 96.6% ▁▁▂▃▄▄█●▃▂▂▁ beats 97/140 very easy · avg 94% 0.2% blunt · maxed out · noisy
AHELM MELD audio emotion classifi… 42.5% ▃··▃▃▃·▃●▅█▅ beats 5/13 medium · avg 39% 0.2% blunt · at its level
AHELM FLEURS 7-language transcrip… 46.1% ▄▂···▂▆▂●▂·▂ beats 9/13 avg 83% 0.1% blunt · noisy
AHELM Multilingual LibriSpeech 7-… 7.1% ▂▂▂··▂▂·▂▂●▆ beats 9/13 avg 14% 0.0% blunt · noisy
AHELM VocalSound human-sound reco… 41.8% ●█▄▄·█·█·█▄▄ beats 0/13 easy · avg 67% 0.0% blunt · at its level · noisy
source results share of the score
EvalPlus / EvalPerf published evaluatio… 4 41% ▇▇▇▇▇
Epoch AI 3 17% ▇▇
LiveCodeBench official website results 1 9% ▇
Aider code editing benchmark runs 1 8% ▇
AHELM: A Holistic Evaluation of Audio-L… 10 7% ▇
LMArena leaderboard dataset 2 7% ▇
MMLU-Pro author-released evaluation runs 1 7% ▇
LiveBench model judgments 1 2% ▇
Open Portuguese LLM Leaderboard 1 1% ▇
Galileo Agent Leaderboard 1 1% ▇
Vectara hallucination leaderboard 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 53.8
Licences: EvalPlus / EvalPerf published evaluation runs ([Apache-2.0](../../licenses/evalplus-published-runs-Apache-2.0.txt)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · LiveCodeBench official website results ([CC-BY-SA-4.0](../../licenses/lcb-sharealike-NOTICE.txt)) · Aider code editing benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) · AHELM: A Holistic Evaluation of Audio-Language Models ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · MMLU-Pro author-released evaluation runs ([Apache-2.0](../../licenses/mmlu-pro-author-runs-Apache-2.0.txt)) · LiveBench model judgments ([Apache-2.0 (stated in the LiveBench datasheet)](https://www.apache.org/licenses/LICENSE-2.0)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Galileo Agent Leaderboard ([Apache-2.0](../../licenses/galileo-agent-leaderboard-Apache-2.0.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
No verified API or local download yet.
[interactive version](https://whatmodel.sh/model/gemini-1-5-flash-002) · [JSON](https://whatmodel.sh/api?format=json&all=1)