[whatmodel.sh](../../README.md) / Gemini 1.5 Flash (001) # Gemini 1.5 Flash (001) Google · released 2024-05 · proprietary **score 51.3** ±0.7 · comparison only: no current run option verified · from 26 benchmark results across 9 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks Gemini 1.5 Flash (001) among every model on that task or test, worst to best.}

task                vs the others          rank  counts  tests
coding              ▁▁▁▃▄▆●▇▅▄▃▁   #295 of 539*     16%      2
chat & writing      ▁▁▁▃▅▄●█▇▇▇▃   #287 of 474*    5.1%      1
math                ▁▁▃▆●█▅█▃▁▁▁    #395 of 530     23%      4
vision & documents  ▁▁▂▃●▃▅▄▆██▄    #156 of 189     12%      4

coding · #295 of 539 · 16% of its score

Beats [Qwen3-VL 32B Instruct](qwen3-vl-32b-instruct.md), [Solar Open 100B (102B-A12B)](solar-open-100b.md), [GLM-4.7-Flash](glm-4-7-flash.md) here, though they rank higher overall. Trails [QwQ-32B-Preview](qwq-32b-preview.md), [InternLM2.5 20B Chat](internlm2-5-20b-chat.md), [Ministral 3 3B Reasoning 2512](ministral-3-3b-reasoning.md) here, though they rank lower overall.

test                               result  vs the others in its run  difficulty        counts  why
CRUXEval (ZeroEval, 800 problems)   63.7%  ▁▂▆▃▆●▆▃▂·▁▁ beats 22/47  medium · avg 57%     13%  sharp · at its level
WebApp1K-React pass@1 (onekq-ai)    57.0%  ▃▁▁█▁●▅▃▄▇▁▄ beats 13/31  medium · avg 40%    2.4%  sharp · at its level · noisy

chat & writing · #287 of 474 · 5.1% of its score

Beats [Mistral Small 4 (119B-A6.5B)](mistral-small-4.md), [Qwen2.5 Coder 32B Instruct](qwen2-5-coder-32b.md), [Ministral 3 14B Reasoning 2512](ministral-3-14b-reasoning.md) here, though they rank higher overall. Trails [Flan-T5 XXL](flan-t5-xxl.md), [Granite 4.0 H 1B](granite-4-0-h-1b.md), [LFM2.5 1.2B Instruct](lfm2-5-1-2b-instruct.md) here, though they rank lower overall.

test            result  vs the others in its run    difficulty  counts  why
LMArena (text)  1286.5  ▁▁▁▂▃▃●▆▆▇█▃ beats 115/385  avg 1260      5.1%  blunt

math · #395 of 530 · 23% of its score

Beats [GPT-4 (0613)](gpt-4-0613.md), [Athene-70B](athene-70b.md) here, though they rank higher overall. Trails [Stable Code 3B](stable-code-3b.md), [Qwen3 0.6B](qwen3-0-6b.md), [Falcon 40B (base)](falcon-40b.md) here, though they rank lower overall.

test                                result  vs the others in its run   difficulty            counts  why
MATH level 5                         25.1%  ▂▂▅●▄█▆▃▅▂▃▄ beats 23/92   hard · avg 28%           10%  sharp · at its level
MATH Level 5 (ZeroEval, 721 probl…   34.8%  ▁▁▁▆█●▇▇▄··▂ beats 17/38   hard · avg 30%          6.2%  sharp · at its level
GSM8K (ZeroEval, zero-shot)          91.4%  ▁▄·▁▄█▅▅●███ beats 23/43   very easy · avg 87%     5.4%  sharp
OTIS mock AIME 2024–2025              3.9%  ▃▃●▃▃▃▇▇█▃▂▅ beats 26/197  very hard · avg 7.2%    1.4%  sharp · noisy

vision & documents · #156 of 189 · 12% of its score

Beats [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md), [Gemma 3 12B](gemma-3-12b.md) here, though they rank higher overall. Trails [Qwen3.5 2B](qwen3-5-2b.md), [Qwen2.5-VL 7B Instruct](qwen2-5-vl-7b.md) here, though they rank lower overall.

test                                result  vs the others in its run   difficulty        counts  why
Arena vision (style controlled)     1059.1  ▁▁▂▂▂●▃▄▅▅█▇ beats 22/152  avg 1061            3.4%  blunt
CharXiv (reasoning questions)        33.9%  ▂▂▂█●▂·▂▂··▂ beats 8/13    hard · avg 34%      3.2%  at its level
MMMU val (VHELM, zero-shot exact …   56.6%  ▄▄··▄▄●··▄▄█ beats 5/9     medium · avg 50%    2.9%  at its level
MathVista testmini (VHELM, zero-s…   51.2%  ▃·▅···▃▃●·▃█ beats 5/9     medium · avg 42%    2.5%  at its level

other tests (15)

test                                result  vs the others in its run    difficulty           counts  why
MMLU-Redux (ZeroEval, 2,778 quest…   77.4%  ▁·▁▁▄▄▄●█▄▂▁ beats 24/53    easy · avg 75%          21%  at its level
ZebraLogic (ZeroEval, 1,000 puzzl…   19.4%  ▁▁▃▆●█▁▁·▁▁▁ beats 29/57    hard · avg 15%         7.2%  sharp
GPQA Diamond                         40.4%  ▁▁▁▁▂●▆▄▅▇█▅ beats 38/211   hard · avg 39%         5.8%  sharp
AHELM Speech Robust Bench (LibriS…   49.8%  ▂·●··▆▄·▂▂·▄ beats 1/13     avg 34%                3.6%  typical
AHELM CoRe-Bench conversational a…   72.2%  ▂·▄··▂▂·▄●█▄ beats 7/13     medium · avg 52%       1.9%  at its level
Open PT-LLM Leaderboard average (…   79.3%  ▃▁▁▁▁▁▁▁▂█●▁ beats 254/276  medium · avg 72%       1.2%  at its level · noisy
AHELM LibriSpeech transcription b…   34.2%  ●···▃·█▃▅▃▅█ beats 0/13     avg 11%                0.9%  typical
AHELM MUStARD audio sarcasm detec…   55.5%  ▂·······●█▆▆ beats 1/13     very hard · avg 57%    0.6%  blunt
AHELM VoxCeleb2 same-speaker dete…   52.2%  ▂·▂·●█▂▄▄··▄ beats 2/13     very hard · avg 54%    0.6%  sharp · noisy
AHELM AIR-Bench Foundation music …   48.3%  ●█▃▅▃·▃·▃▅▃▃ beats 0/13     medium · avg 57%       0.3%  blunt · at its level
Vectara hallucination leaderboard…   93.4%  ▁▁▂▃▄●█▆▃▂▂▁ beats 46/140   very easy · avg 94%    0.3%  blunt · maxed out · noisy
AHELM MELD audio emotion classifi…   47.1%  ▃··▃▃▃·▃▅▅●▅ beats 9/13     medium · avg 39%       0.3%  blunt · at its level
AHELM FLEURS 7-language transcrip…   72.3%  ▄▂···▂●▂█▂·▂ beats 4/13     avg 83%                0.2%  blunt · noisy
AHELM Multilingual LibriSpeech 7-…    8.8%  ▂▂▂··▂▂·▂●█▆ beats 6/13     avg 14%                0.0%  blunt · noisy
AHELM VocalSound human-sound reco…   46.3%  ●█▄▄·█·█·█▄▄ beats 1/13     easy · avg 67%         0.0%  blunt · at its level · noisy

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
WildEval ZeroEval published results             5   53% ▇▇▇▇▇▇
Epoch AI                                        3   17% ▇▇
LMArena leaderboard dataset                     2    8% ▇
AHELM: A Holistic Evaluation of Audio-L…       10    8% ▇
VHELM: A Holistic Evaluation of Vision …        2    5% ▇
CharXiv leaderboard                             1    3% ▇
WebApp1K models leaderboard                     1    2% ▇
Open Portuguese LLM Leaderboard                 1    1% ▇
Vectara hallucination leaderboard               1    0% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 51.3

_{Licences: WildEval ZeroEval published results ([Apache-2.0](../../licenses/zeroeval-published-results-Apache-2.0.txt)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · AHELM: A Holistic Evaluation of Audio-Language Models ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · VHELM: A Holistic Evaluation of Vision Language Models ([CC BY 4.0](http://creativecommons.org/licenses/by/4.0/)) · CharXiv leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · WebApp1K models leaderboard ([MIT](../../licenses/webapp1k-leaderboard-MIT.txt)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))} ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/gemini-1-5-flash-001) · [JSON](https://whatmodel.sh/api?format=json&all=1)