[whatmodel.sh](../../README.md) / Gemini 1.5 Flash Experimental (0827) # Gemini 1.5 Flash Experimental (0827) Google · released 2024-08 · proprietary **score 54.2** ±0.8 · comparison only: no current run option verified · from 6 benchmark results across 3 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks Gemini 1.5 Flash Experimental (0827) among every model on that task or test, worst to best. 
task    vs the others          rank  counts  tests
coding  ▁▁▁▃▄▆●▇▅▄▃▁   #302 of 539*     27%      2
math    ▁▁▃▆█●▅█▃▁▁▁   #297 of 530*    8.8%      1
coding · #302 of 539 · 27% of its score Beats [Qwen3-VL 32B Instruct](qwen3-vl-32b-instruct.md), [Solar Open 100B (102B-A12B)](solar-open-100b.md), [GLM-4.7-Flash](glm-4-7-flash.md) here, though they rank higher overall. Trails [QwQ-32B-Preview](qwq-32b-preview.md), [InternLM2.5 20B Chat](internlm2-5-20b-chat.md), [Ministral 3 3B Reasoning 2512](ministral-3-3b-reasoning.md) here, though they rank lower overall. 
test                                result  vs the others in its run  difficulty        counts  why
CRUXEval (ZeroEval, 800 problems)    74.5%  ▁▂▆▃▆█●▃▂·▁▁ beats 34/47  medium · avg 57%     15%  sharp · at its level
Aider code editing (133 Exercism …   52.6%  ▁·▁▂▄▁●▄█▆▂▂ beats 18/62  medium · avg 51%     12%  sharp · at its level
math · #297 of 530 · 8.8% of its score Beats [GPT-4o (2024-08-06)](gpt-4o-2024-08-06.md), [Command A (03-2025)](command-a.md), [GPT-4o (2024-05-13)](gpt-4o-2024-05-13.md) here, though they rank higher overall. Trails [Gemma 7B (base)](gemma-7b-base.md), [CodeGemma 7B (v1.0)](codegemma-7b.md), [Phi-2 (2.7B)](phi-2.md) here, though they rank lower overall. 
test                                result  vs the others in its run  difficulty      counts  why
MATH Level 5 (ZeroEval, 721 probl…   54.5%  ▁▁▁▆█▂▇●▄··▂ beats 29/38  hard · avg 30%    8.8%  sharp · at its level
other tests (3) 
test                                result  vs the others in its run  difficulty        counts  why
LiveBench 2024-07-26                 46.9%  ▁▁▁▂▂▃▂█▆▃●▃ beats 49/61  medium · avg 38%     28%  at its level
MMLU-Redux (ZeroEval, 2,778 quest…   82.1%  ▁·▁▁▄▄▄▅●▄▂▁ beats 35/53  easy · avg 75%       24%  typical
ZebraLogic (ZeroEval, 1,000 puzzl…   25.0%  ▁▁▃▆█●▁▁·▁▁▁ beats 37/57  hard · avg 15%       12%  sharp · at its level
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                               results  share of the score
WildEval ZeroEval published results        4   60% ▇▇▇▇▇▇▇
LiveBench model judgments                  1   28% ▇▇▇
Aider code editing benchmark runs          1   12% ▇
prior: pull toward the middle                   0% ▇
weighted average                              = 54.2
Licences: WildEval ZeroEval published results ([Apache-2.0](../../licenses/zeroeval-published-results-Apache-2.0.txt)) · LiveBench model judgments ([Apache-2.0 (stated in the LiveBench datasheet)](https://www.apache.org/licenses/LICENSE-2.0)) · Aider code editing benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/gemini-1-5-flash-exp-0827) · [JSON](https://whatmodel.sh/api?format=json&all=1)