[whatmodel.sh](../../README.md) / GPT-4.1
# GPT-4.1
OpenAI · released 2025-04 · closed weights
**score 62.2** ±0.5 · #176 of 732 · from 69 benchmark results across 23 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks GPT-4.1 among every model on that task or test, worst to best.
task vs the others rank counts tests
agent tasks ▁▁▃▆█●▅▅▄▄▂▁ #165 of 313 26% 11
coding ▁▁▁▃▅▇█●▅▄▃▁ #151 of 469 19% 7
chat & writing ▁▁▁▃▆▄▆█▆●▇▃ #105 of 402 6.3% 4
math ▁▁▃▅█▆●█▄▂▁▁ #186 of 431 14% 9
vision & documents ▁▁▂▂▁▃▄▂●▇█▄ #77 of 153 4.2% 3
web search ▁▁▁▅▄█▃▄·●·▁ #3 of 84* 1.4% 1
agent tasks · #165 of 313 · 26% of its score
Beats [o1 (2024-12-17)](o1-2024-12-17.md), [o3-mini](o3-mini.md), [DeepSeek R1 0528](deepseek-r1-0528.md) here, though they rank higher overall. Trails [SmolLM3 3B](smollm3-3b.md), [Jamba 1.5 Mini](jamba-1-5-mini.md), [Jamba 1.5 Large](jamba-1-5-large.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
τ²-bench Telecom (maker) 34.0% ▃·●▅▃▃··█▃▃▅ beats 1/13 very hard · avg 9.9% 11% sharp · at its level · several runs
SWE-bench Verified 48.5% ▁··●·▁▂▁▄█▄▁ beats 1/29 hard · avg 19% 7.3% sharp · at its level · several runs
τ²-bench Retail 74.1% ▅···█·●····▃ beats 5/6 medium · avg 40% 2.6% at its level · several runs
τ²-bench Airline 56.0% ▄·▄··▄···●▄▄ beats 4/6 hard · avg 28% 1.7% at its level · several runs
Finance Agent Benchmark v1 (537 q… 26.7% █·▃▃·█▅▅▅●▃▅ beats 14/18 very hard · avg 5.5% 1.4% sharp
BFCL V4 (December 2025 archive) 54.0% ▁▁▁▂██▅▅●▂▂▂ beats 62/77 hard · avg 31% 1.1% at its level
Galileo Agent Leaderboard v2 (act… 62.0% ▅▃·▃▃▅▅█▃█▅● beats 19/19 hard · avg 26% 0.5% at its level
BFCL v3 68.9% ▁▁▁▁▁▃▅█▅●▁▁ beats 107/114 medium · avg 51% 0.4% blunt · at its level
Terminal-Bench (pre-2.0 release) 28.4% ▄·▄···▄●···█ beats 3/5 very hard · avg 12% 0.4% at its level
KlinikeBench strict pass@1 (333 i… 7.5% ▁▁·▁▁▁▁●▃█▅█ beats 6/27 very hard · avg 2.4% 0.3% too hard for it
Galileo Agent Leaderboard v1 (too… 91.8% ▃▃▃▃▆▅▆▅█●▅▂ beats 27/31 easy · avg 78% 0.3% typical
coding · #151 of 469 · 19% of its score
Beats [Qwen3-Coder-Next (80B-A3B)](qwen3-coder-next.md), [Gemini 2.5 Pro](gemini-2-5-pro.md), [Gemini 3.1 Flash-Lite](gemini-3-1-flash-lite.md) here, though they rank higher overall. Trails [Ministral 3 8B Reasoning 2512](ministral-3-8b-reasoning.md), [Phi-4-reasoning 14B](phi-4-reasoning.md), [Llama 3.1 Nemotron Ultra 253B v1](llama-3-1-nemotron-ultra-253b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
SWE-bench Verified 48.5% ▁··●·▁▂▁▄█▄▁ beats 1/29 hard · avg 19% 7.3% sharp · at its level · several runs
Aider polyglot (225 exercises, tw… 52.4% ▂·▂▃▅▂▃●▆▄▂▂ beats 28/49 very hard · avg 7.6% 5.8% sharp · at its level
LiveCodeBench (version unspecifie… 39.2% ●·······▄▄▄▄ beats 0/5 hard · avg 21% 1.5% at its level
HumanEval+ 93.3% ▄▄▄·····▄●·█ beats 4/6 easy · avg 66% 1.3% sharp
WebApp1K-Duo-React pass@1 (onekq-… 68.8% ▁··▁·▁▁▁▆█●█ beats 29/45 hard · avg 29% 1.3% at its level
MBPP+ 79.4% ▃·····●█▃··▃ beats 1/6 medium · avg 60% 1.2% typical
HumanEval (simple-evals) (maker) 94.5% ▆█▆▆●··▂···▂ beats 13/15 easy · avg 83% 0.5% maxed out
chat & writing · #105 of 402 · 6.3% of its score
Beats [LongCat 2.0 (1.6T-A48B)](longcat-2-0.md), [Inkling Small (266B-A12B)](inkling-small.md), [GPT-5.4 nano](gpt-5-4-nano.md) here, though they rank higher overall. Trails [Mistral Large 3 (675B-A41B)](mistral-large-3.md), [Gemma 4 12B](gemma-4-12b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena-Hard v2.0 hard prompts (sty… 50.0% ▂·▃▅▅·▃●▆▂▂▂ beats 15/22 very hard · avg 12% 2.6% sharp · at its level
LMArena (text) 1414.6 ▁▁▁▂▃▃▃▆▆●█▃ beats 265/385 avg 1260 2.2% blunt
IFEval (aggregation unspecified) 85.6% █··●·█··████ beats 1/6 easy · avg 74% 1.0% typical
Arena-Hard v2.0 creative writing … 61.5% ▆▂▂▆▂▂▃●▂▃·▂ beats 17/22 hard · avg 19% 0.6% sharp · at its level · noisy
math · #186 of 431 · 14% of its score
Beats [MiniMax-M2.1](minimax-m2-1.md) here, though they rank higher overall. Trails [Phi-4-mini-reasoning 3.8B](phi-4-mini-reasoning.md), [OpenThinker3 7B](openthinker3-7b.md), [Olmo 3 7B Think](olmo-3-7b-think.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
FrontierMath tiers 1–3 v2 6.0% ▂··●▁▃▄▄█▆▄▃ beats 4/86 very hard · avg 1.2% 4.9% sharp
MATH level 5 83.0% ▂▂▅▆▄█▆▃●▂▃▄ beats 69/92 hard · avg 28% 2.6% sharp
OTIS mock AIME 2024–2025 38.3% ▃▃▄▃▃●▇▇█▃▂▅ beats 71/197 very hard · avg 7.2% 1.7% sharp · at its level · noisy
MATH-500 (simple-evals) (maker) 82.1% ▃··●▅▃···▃·█ beats 2/9 medium · avg 61% 1.3% sharp
AIME 2025 32.0% ●▄···▄▄··▄·█ beats 0/6 hard · avg 16% 1.1% sharp · at its level · noisy
MathNet-Solve-Test micro-average … 21.4% ▃●···█▃·▃·▃▃ beats 1/8 very hard · avg 15% 0.7% at its level
AIME 2024 47.0% ▁▁▁▂▁▃▁●███▃ beats 21/85 hard · avg 30% 0.6% sharp · at its level · noisy
MGSM (simple-evals) (maker) 86.9% ▁····▁●▁█▁▃▁ beats 3/16 easy · avg 80% 0.6% typical
MATH-500 90.6% ●·▄··█··▄·▄▄ beats 0/6 very easy · avg 87% 0.5% maxed out · noisy
vision & documents · #77 of 153 · 4.2% of its score
Beats [DeepSeek V4 Flash Vision (Exp)](deepseek-v4-flash-vision-exp.md), [GPT-5.4 nano](gpt-5-4-nano.md), [Nemotron 3.5 Super VL 120B A12B](nemotron-3-5-super-vl-120b-a12b.md) here, though they rank higher overall. Trails [Gemma 4 12B](gemma-4-12b.md), [Qwen3.5 9B](qwen3-5-9b.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Arena vision (style controlled) 1213.9 ▁▁▂▂▂▂▃▄▅●█▇ beats 79/152 avg 1061 1.5% blunt
CharXiv (reasoning questions) (ma… 56.7% ▄··▂●··▂···▄ beats 5/9 hard · avg 32% 1.4% at its level
CharXiv descriptive questions (ma… 87.9% ▄▄·····▄·●█▄ beats 3/6 easy · avg 71% 1.3% typical
web search · #3 of 84 · 1.4% of its score
Beats [GPT-6 Astra](gpt-6-astra.md), [Claude Opus 5.5](claude-opus-5-5.md), [GPT-6.1 Sol](gpt-6-1-sol.md) here, though they rank higher overall.
test result vs the others in its run difficulty counts why
Finance Agent Benchmark v1 (537 q… 26.7% █·▃▃·█▅▅▅●▃▅ beats 14/18 very hard · avg 5.5% 1.4% sharp
other tests (28)
test result vs the others in its run difficulty counts why
GPQA Diamond 66.9% ▁▁▁▁▂▄▆▄●▇█▅ beats 100/211 hard · avg 39% 6.1% sharp · at its level · several runs
GPQA Diamond (simple-evals) (make… 66.3% █▃▃█·▃▃●▃▃▃▅ beats 10/16 hard · avg 39% 4.0% sharp · at its level
MMLU 89.6% ▄·····●▄▄▄·▄ beats 1/6 medium · avg 68% 2.8% typical
CLIcK Korean (Daekeun Kim) 80.5% ▆▄▄▄█●▂·▄█·▄ beats 14/23 medium · avg 56% 2.5% typical
KMMLU-HARD (Daekeun Kim, zero-sho… 42.8% █···▁▁▁●▃·▆▂ beats 11/23 very hard · avg 29% 2.4% sharp · at its level
MMLU-Pro 81.7% ●▄·▄···▄··▄▄ beats 0/6 medium · avg 58% 2.0% typical
HAE-RAE 1.0 (Daekeun Kim) 85.8% ▄▄▄▂▂█▂●▂█▂▄ beats 14/22 medium · avg 61% 2.0% typical
KCL-Essay vanilla 42.4% ▁···▁▃▂█▂●▃▂ beats 16/23 hard · avg 17% 1.6% at its level
ARC-AGI-1 public evaluation set (… 11.8% ▁·●▃▁·█▇▁·▇▃ beats 5/27 very hard · avg 4.0% 1.5% sharp · at its level · noisy
EQ-Bench 3 (emotional intelligenc… 1206.0 ▁·▂▁▄▅●▃█▄▁▁ beats 31/62 avg 1034 1.3% typical
AraGen-v3 3C3H score (Inception, … 74.5% ▁▁·▁▂▂▄█▄▅●▄ beats 58/66 hard · avg 29% 1.3% at its level
MMLU (simple-evals, zero-shot CoT… 90.2% ▂▂·█▄▄▂·●▂·▂ beats 11/16 easy · avg 79% 1.2% typical
CMMLU 77.6% ●····▄··▄▄▄▄ beats 0/6 medium · avg 67% 1.1% typical
C-Eval 79.5% ●····▄·▄·▄▄▄ beats 1/6 medium · avg 68% 1.0% typical
HindiGen-v1 3C3H score (Inception… 73.4% ▁▁▁▁·▂·█▆●▇▁ beats 23/31 medium · avg 46% 1.0% at its level
Open Persian LLM Leaderboard v2 a… 74.4% ▂▄▂▃█▄▃▆▃▅●▂ beats 78/81 medium · avg 57% 0.8% blunt · at its level
COLLIE (constrained text generati… 50.0% ▄···▄●▄····█ beats 3/6 hard · avg 30% 0.8% at its level
KCL-MCQA vanilla 47.9% ▂···█▂▆▆█●▆▂ beats 16/23 very hard · avg 29% 0.8% at its level
VitaBench 19.0% █·······▄●·█ beats 3/6 very hard · avg 6.5% 0.7% typical
SimpleQA Verified 31.1% ▁▁▃▂▃▃●▆█▄▂▅ beats 21/81 very hard · avg 8.0% 0.6% typical
Chess Puzzles (Epoch) 6.0% █●▁▁▁▂▃▃▃▂▁▁ beats 49/137 very hard · avg 5.8% 0.5% sharp · too hard for it · noisy
ARC-AGI-2 public evaluation set (… 0.0% ●··▂▅▁▃▁▃·▁▃ beats 2/24 very hard · avg 0.3% 0.5% sharp · too hard for it · noisy
BeyondAIME 22.1% ●····█·▄▄··▄ beats 1/6 hard · avg 24% 0.3% at its level
SimpleQA (simple-evals) (maker) 41.6% ▄·▁▁▁▁··●▁·▁ beats 10/14 hard · avg 17% 0.2% at its level · noisy
DROP F1, 3-shot (simple-evals) (m… 79.4% ▂▂●▆▆█·▂···▄ beats 2/16 easy · avg 78% 0.2% blunt
DROP 66.9% ▄···●·▄█·▄·▄ beats 1/6 medium · avg 58% 0.1% blunt · at its level · noisy
Vectara hallucination leaderboard… 98.0% ▁▁▂▃▄▄█▆●▂▂▁ beats 121/140 very easy · avg 94% 0.1% blunt · maxed out · noisy
Vectara hallucination leaderboard… 94.4% ▁▂▁▂▅█▂▃●▃▁▁ beats 80/98 very easy · avg 90% 0.0% blunt
source results share of the score
Meituan LongCat-Flash-Chat model card 21 24% ▇▇▇
Epoch AI 7 19% ▇▇
OpenAI simple-evals README benchmark re… 7 8% ▇
Daekeun Kim Korean LLM benchmark runs 3 7% ▇
Aider polyglot benchmark runs 1 6% ▇
MUA-RL: Multi-turn User-interacting Age… 1 5% ▇
OpenAI vendor-reported benchmark scores… 3 4% ▇
LMArena leaderboard dataset 2 4% ▇
τ²/τ³-bench leaderboard: Sierra's own e… 3 4% ▇
Arena-Hard-Auto v2.0-Preview leaderboard 2 3% ▇
Korean Canonical Legal Benchmark: Towar… 2 2% ▇
Inception Arabic/Hindi leaderboards 2 2% ▇
Moonshot AI Kimi-K2-Instruct model card 1 2% ▇
ARC Prize Foundation model_baseline out… 2 2% ▇
Berkeley Function Calling Leaderboard r… 2 1% ▇
Finance Agent Benchmark 1 1% ▇
EQ-Bench 3 canonical leaderboard results 1 1% ▇
WebApp1K models leaderboard 1 1% ▇
Open Persian LLM Leaderboard v2 1 1% ▇
Galileo Agent Leaderboard 2 1% ▇
MathNet: a Global Multimodal Benchmark … 1 1% ▇
KlinikeBench: Evaluating Language Model… 1 0% ▇
Vectara hallucination leaderboard 2 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 62.2
Licences: Meituan LongCat-Flash-Chat model card ([MIT](https://huggingface.co/meituan-longcat/LongCat-Flash-Chat/blob/f0b8e8d7380330f450ca1fde4ec56823a514584c/LICENSE)) · Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · OpenAI simple-evals README benchmark results ([MIT](../../licenses/openai-simple-evals-MIT.txt)) · Daekeun Kim Korean LLM benchmark runs ([MIT](../../licenses/korean-personal-benchmark-runs-MIT.txt)) · Aider polyglot benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) · MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · τ²/τ³-bench leaderboard: Sierra's own evaluation runs ([MIT](https://opensource.org/license/mit)) · Arena-Hard-Auto v2.0-Preview leaderboard ([Apache-2.0](../../licenses/arena-hard-auto-Apache-2.0.txt)) · Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities ([CC BY 4.0](../../licenses/cards-final-NOTICE.txt)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Moonshot AI Kimi-K2-Instruct model card ([Modified MIT License (Kimi K2)](https://huggingface.co/moonshotai/Kimi-K2-Instruct/blob/fd1984e2b7a3350dbf7305fe73a4ede25c14de50/LICENSE)) · ARC Prize Foundation model\_baseline outputs on ARC-AGI-1/2 public evaluation sets ([MIT](../../licenses/arcprize-model-baseline-outputs-MIT.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · Finance Agent Benchmark ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · EQ-Bench 3 canonical leaderboard results ([MIT](../../licenses/eqbench3-MIT.txt)) · WebApp1K models leaderboard ([MIT](../../licenses/webapp1k-leaderboard-MIT.txt)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Galileo Agent Leaderboard ([Apache-2.0](../../licenses/galileo-agent-leaderboard-Apache-2.0.txt)) · MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval ([CC BY 4.0](../../licenses/paper-2604.18584v2-CC-BY-4.0.txt)) · KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
## How to run it
### Through an API
api (OpenAI) · $3.50/M blended · $2 in / $8 out per 1M tokens · [platform.openai.com](https://platform.openai.com/docs/models)
[price source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/providers/openai/models/gpt-4.1.toml): Standard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ. · [context source](https://github.com/anomalyco/models.dev/blob/a668908cfe7073faa0363699486fc41a3ff52412/models/openai/gpt-4.1.toml)
[interactive version](https://whatmodel.sh/model/gpt-4-1) · [JSON](https://whatmodel.sh/api?format=json&all=1)