[whatmodel.sh](../../README.md) / Grok 3 Beta # Grok 3 Beta SpaceXAI (xAI) · released 2025-04 · proprietary **score 63.2** ±1.1 · comparison only: no current run option verified · from 8 benchmark results across 6 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks Grok 3 Beta among every model on that task or test, worst to best.}

task         vs the others          rank  counts  tests
agent tasks  ▁▁▂▆●▄▅▄▄▃▁▁   #230 of 354*     11%      2
coding       ▁▁▁▃▄▆█●▅▄▃▁   #164 of 539*     39%      2
math         ▁▁▃▆██●█▃▁▁▁   #185 of 530*     22%      2
web search   ▂▁▂▆▅█▅▄▁●▁▁      #6 of 96*    8.3%      1

agent tasks · #230 of 354 · 11% of its score

Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md), [DeepSeek R1 (Jan '25)](deepseek-r1.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Mellum2 12B-A2.5B Instruct](mellum2-12b-a2-5b-instruct.md) here, though they rank lower overall.

test                                result  vs the others in its run   difficulty            counts  why
Finance Agent Benchmark v1 (537 q…   25.8%  █·▃▃·█▅▅●▅▃▅ beats 13/18   very hard · avg 5.5%    8.3%  sharp
BFCL v3                              63.3%  ▁▁▁▁▁▃▅█●▄▁▁ beats 89/114  medium · avg 51%        2.5%  blunt · at its level

coding · #164 of 539 · 39% of its score

Beats [Qwen3.5 Plus](qwen3-5-plus.md), [Qwen3-Coder-Next (80B-A3B)](qwen3-coder-next.md), [Gemini 2.5 Pro](gemini-2-5-pro.md) here, though they rank higher overall. Trails [Llama 3.1 Nemotron Ultra 253B v1](llama-3-1-nemotron-ultra-253b.md), [Devstral Small 2 24B](devstral-small-2.md), [Gemma 4 12B](gemma-4-12b.md) here, though they rank lower overall.

test                                result  vs the others in its run  difficulty            counts  why
Aider polyglot (225 exercises, tw…   53.3%  ▂·▂▃▅▂▃●▆▄▂▂ beats 29/49  very hard · avg 7.6%     32%  sharp · at its level
WebApp1K-Duo-React pass@1 (onekq-…   60.2%  ▁··▁·▁▁▁●█▇█ beats 14/45  hard · avg 29%          6.9%  at its level

math · #185 of 530 · 22% of its score

Beats [MiMo-V2.5 (310B-A15B)](mimo-v2-5.md), [MiniMax-M2.7](minimax-m2-7.md), [MiniMax-M2.1](minimax-m2-1.md) here, though they rank higher overall. Trails [Olmo 3 7B Think](olmo-3-7b-think.md), [Nemotron Nano 9B v2](nemotron-nano-9b-v2.md), [Qwen3-VL 4B Thinking](qwen3-vl-4b-thinking.md) here, though they rank lower overall.

test                      result  vs the others in its run   difficulty            counts  why
MATH level 5               88.7%  ▂▂▅▆▄█▆▃●▂▃▄ beats 76/92   hard · avg 28%           13%  sharp
OTIS mock AIME 2024–2025   55.6%  ▃▃▄▃▃▃●▇█▃▂▅ beats 82/197  very hard · avg 7.2%    9.4%  sharp · at its level · noisy

web search · #6 of 96 · 8.3% of its score

Beats [GPT-6 Astra](gpt-6-astra.md), [Claude Opus 5.5](claude-opus-5-5.md), [GPT-6.1 Sol](gpt-6-1-sol.md) here, though they rank higher overall. Trails [GPT-4.1](gpt-4-1.md) here, though they rank lower overall.

test                                result  vs the others in its run  difficulty            counts  why
Finance Agent Benchmark v1 (537 q…   25.8%  █·▃▃·█▅▅●▅▃▅ beats 13/18  very hard · avg 5.5%    8.3%  sharp

other tests (2)

test                                result  vs the others in its run    difficulty      counts  why
GPQA Diamond                         75.8%  ▁▁▁▁▂▄▆▄●▇█▅ beats 114/211  hard · avg 39%     20%  sharp · at its level
EQ-Bench 3 (emotional intelligenc…  1176.6  ▁·▂▁▄▅●▃█▄▁▁ beats 28/62    avg 1034          7.5%  typical

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
Epoch AI                                        3   42% ▇▇▇▇▇
Aider polyglot benchmark runs                   1   32% ▇▇▇▇
Finance Agent Benchmark                         1    8% ▇
EQ-Bench 3 canonical leaderboard results        1    7% ▇
WebApp1K models leaderboard                     1    7% ▇
Berkeley Function Calling Leaderboard r…        1    2% ▇
prior: pull toward the middle                        1% ▇
weighted average                                   = 63.2

_{Licences: Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Aider polyglot benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) · Finance Agent Benchmark ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · EQ-Bench 3 canonical leaderboard results ([MIT](../../licenses/eqbench3-MIT.txt)) · WebApp1K models leaderboard ([MIT](../../licenses/webapp1k-leaderboard-MIT.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt))} ## How to run it No verified API or local download yet. [interactive version](https://whatmodel.sh/model/grok-3-beta) · [JSON](https://whatmodel.sh/api?format=json&all=1)