[whatmodel.sh](../../README.md) / Gemini 1.5 Flash-8B (001) # Gemini 1.5 Flash-8B (001) Google · released 2024-10 · proprietary **score 49.2** ±1.8 · comparison only: no current run option verified · from 5 benchmark results across 3 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks Gemini 1.5 Flash-8B (001) among every model on that task or test, worst to best.}

task                vs the others          rank  counts  tests
chat & writing      ▁▁▁▃▅▄●█▇▇▇▃   #326 of 474*     33%      1
math                ▁▁▃▆●█▅█▃▁▁▁   #358 of 530*    6.0%      1
vision & documents  ▁▁▂▃●▃▅▄▆██▄   #154 of 189*     22%      1

chat & writing · #326 of 474 · 33% of its score

Beats [Ministral 3 14B Reasoning 2512](ministral-3-14b-reasoning.md), [Ministral 3 14B](ministral-3-14b.md), [Phi-4 14B](phi-4.md) here, though they rank higher overall. Trails [Flan-T5 XXL](flan-t5-xxl.md), [LFM2 1.2B](lfm2-1-2b.md), [North Micro Vision Instruct (2.4B)](north-micro-vision-instruct.md) here, though they rank lower overall.

test            result  vs the others in its run   difficulty  counts  why
LMArena (text)  1258.8  ▁▁▁▂▃▃●▆▆▇█▃ beats 93/385  avg 1260       33%  blunt

math · #358 of 530 · 6.0% of its score

Beats [Molmo 2 8B](molmo-2-8b.md), [DeepSeek-Coder-V2 Instruct (0724)](deepseek-coder-v2-0724.md), [Llama 3.1 Nemotron 70B Instruct](llama-3-1-nemotron-70b.md) here, though they rank higher overall. Trails [DeepSeek R1 Distill Qwen 1.5B](deepseek-r1-distill-qwen-1-5b.md), [Gemma 7B (base)](gemma-7b-base.md), [Mistral 7B v0.1 (base)](mistral-7b-v0-1.md) here, though they rank lower overall.

test                      result  vs the others in its run   difficulty            counts  why
OTIS mock AIME 2024–2025    4.6%  ▃▃●▃▃▃▇▇█▃▂▅ beats 30/197  very hard · avg 7.2%    6.0%  sharp · too hard for it · noisy

vision & documents · #154 of 189 · 22% of its score

Beats [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md), [Gemma 3 12B](gemma-3-12b.md), [Llama 3.2 90B Vision Instruct](llama-3-2-90b-vision.md) here, though they rank higher overall. Trails [Qwen3.5 2B](qwen3-5-2b.md) here, though they rank lower overall.

test                             result  vs the others in its run   difficulty  counts  why
Arena vision (style controlled)  1071.3  ▁▁▂▂▂●▃▄▅▅█▇ beats 25/152  avg 1061       22%  blunt

other tests (2)

test                                result  vs the others in its run    difficulty        counts  why
GPQA Diamond                         33.0%  ▁▁▁▁●▄▆▄▅▇█▅ beats 22/211   hard · avg 39%       31%  sharp
Open PT-LLM Leaderboard average (…   75.4%  ▃▁▁▁▁▁▁▁▂█●▁ beats 228/276  medium · avg 72%    7.7%  at its level · noisy

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                           results  share of the score
LMArena leaderboard dataset            2   54% ▇▇▇▇▇▇▇
Epoch AI                               2   37% ▇▇▇▇
Open Portuguese LLM Leaderboard        1    8% ▇
prior: pull toward the middle               1% ▇
weighted average                          = 49.2

_{Licences: LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt))} ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/gemini-1-5-flash-8b-001) · [JSON](https://whatmodel.sh/api?format=json&all=1)