[whatmodel.sh](../../README.md) / Phi-4 14B # Phi-4 14B Microsoft · released 2024-12 · 14.7B params · mit **score 53.2** ±0.6 · #313 of 732 · from 49 benchmark results across 17 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks Phi-4 14B among every model on that task or test, worst to best. 
task            vs the others          rank  counts  tests
agent tasks     ▁▁▃●█▄▅▅▄▄▂▁   #278 of 313*    2.7%      2
coding          ▁▁▁▃▅▇●▇▅▄▃▁    #232 of 469     18%      7
chat & writing  ▁▁▁▃▆▄●█▆▇▇▃    #276 of 402     12%      4
math            ▁▁▃▅█●▄█▄▂▁▁    #230 of 431     16%      5
agent tasks · #278 of 313 · 2.7% of its score Beats [Magistral Small 1.0 (2506)](magistral-small-2506.md), [Llama 4 Scout (109B-A17B)](llama-4-scout.md) here, though they rank higher overall. Trails [Qwen2.5 1.5B Instruct](qwen2-5-1-5b.md), [Granite 4.0 H 1B](granite-4-0-h-1b.md), [Granite 4.0 1B](granite-4-0-1b.md) here, though they rank lower overall. 
test                             result  vs the others in its run   difficulty        counts  why
BFCL V4 (December 2025 archive)   28.8%  ▁▁▁▂█●▅▅▆▂▂▂ beats 28/77   hard · avg 31%      1.8%  at its level
BFCL v3                           47.0%  ▁▁▁▁▁▃●█▅▄▁▁ beats 28/114  medium · avg 51%    0.9%  blunt · at its level
coding · #232 of 469 · 18% of its score Beats [Qwen3.5 4B](qwen3-5-4b.md), [Qwen3-VL 32B Instruct](qwen3-vl-32b-instruct.md), [Solar Open 100B (102B-A12B)](solar-open-100b.md) here, though they rank higher overall. Trails [QwQ-32B-Preview](qwq-32b-preview.md), [Ministral 3 3B Reasoning 2512](ministral-3-3b-reasoning.md), [GPT-4 (0613)](gpt-4-0613.md) here, though they rank lower overall. 
test                                result  vs the others in its run    difficulty        counts  why
HumanEval+ (maker)                   82.8%  ▁·▁▁▂▃▄▄█●▄▃ beats 116/145  easy · avg 77%       11%  sharp · several runs
HumanEval (maker)                    82.6%  ▁▁▁▁▂▂▆▆█●▂▁ beats 183/205  easy · avg 71%      2.1%  sharp · small run
CodeReviewQA: change localisation…   44.8%  █▂▁▁▂▂▂▃▅▅●▇ beats 34/47    hard · avg 33%      1.5%  sharp · at its level · noisy
HumanEval (simple-evals) (maker)     82.6%  ▂●▆█▅▃··▂··▂ beats 3/19     easy · avg 76%      1.2%  typical
CodeReviewQA: solution identifica…   77.5%  ▃··▁▂▂▃▂█▃●▂ beats 42/47    medium · avg 51%    1.2%  sharp · at its level · noisy
CodeReviewQA: automated code refi…   37.1%  ▁▁▁▁▁▁▂▂▃█●▄ beats 37/47    hard · avg 32%      1.1%  at its level
CodeReviewQA: change type recogni…   76.6%  ▁▁▁······▁▃● beats 35/47    easy · avg 67%      0.0%  blunt · at its level · noisy
chat & writing · #276 of 402 · 12% of its score Beats [Ministral 3 14B Reasoning 2512](ministral-3-14b-reasoning.md), [Ministral 3 14B](ministral-3-14b.md) here, though they rank higher overall. Trails [Flan-T5 XXL](flan-t5-xxl.md), [LFM2 1.2B](lfm2-1-2b.md), [North Micro Vision Instruct (2.4B)](north-micro-vision-instruct.md) here, though they rank lower overall. 
test                                result  vs the others in its run   difficulty        counts  why
Arena-Hard (version/judge unspeci…   75.4%  ▃▃▂▅▃▆▅█●▅▂▃ beats 23/30   medium · avg 59%    6.0%  at its level · several runs
LMArena (text)                      1256.3  ▁▁▁▂▃▃●▆▆▇█▃ beats 92/385  avg 1260            4.0%  blunt
IFEval (aggregation unspecified) …   63.0%  ▁▂▄●▂▄▆██▅▂▁ beats 24/145  easy · avg 74%      1.3%  several runs
IFEval strict (prompt/instruction…   62.3%  ●·········██ beats 0/2     easy · avg 67%      0.8%  at its level · small run
math · #230 of 431 · 16% of its score Beats [MiniMax-M2.1](minimax-m2-1.md), [GPT-4o (2024-08-06)](gpt-4o-2024-08-06.md), [Command A (03-2025)](command-a.md) here, though they rank higher overall. Trails [Gemma 7B (base)](gemma-7b-base.md), [Phi-2 (2.7B)](phi-2.md), [Granite 8B Code Base (4K)](granite-8b-code-base-4k.md) here, though they rank lower overall. 
test                               result  vs the others in its run   difficulty            counts  why
MATH level 5                        64.9%  ▂▂▅▆▄█●▃▅▂▃▄ beats 56/92   hard · avg 28%          8.8%  sharp · at its level
MGSM (maker)                        80.6%  ▂·▁▁▁▁▁▂▅█▃● beats 41/42   easy · avg 72%          2.7%  sharp · small run
MATH (subset unspecified) (maker)   80.4%  ▂▄▄▃█▇▅●▄▃▄▁ beats 54/80   easy · avg 72%          1.6%  small run
OTIS mock AIME 2024–2025            13.8%  ▃▃▄●▃▃▇▇█▃▂▅ beats 52/197  very hard · avg 7.2%    1.6%  sharp · noisy
MGSM (simple-evals) (maker)         80.6%  ▁·▁●·▁▇▃█▁▃▁ beats 2/20    easy · avg 81%          1.1%  typical
other tests (24) 
test                                result  vs the others in its run    difficulty            counts  why
MMLU-Pro (maker)                     70.4%  ▁▁·▁▁▁▃▂▄●█▁ beats 92/188   medium · avg 59%        7.3%  at its level · several runs
CLIcK Korean (Daekeun Kim)           59.2%  ▆▄●▄█▄▂·▄█·▄ beats 5/23     medium · avg 56%        6.3%  at its level
HAE-RAE 1.0 (Daekeun Kim)            59.2%  ▄▄●▂▂█▂▄▂█▂▄ beats 4/22     medium · avg 61%        5.4%  at its level
GPQA Diamond                         56.1%  ▁▁▁▁▂▄▆●▅▇█▅ beats 81/211   hard · avg 39%          5.3%  sharp · at its level
GPQA Diamond (simple-evals) (make…   56.1%  ▁·····▄█●▂▃▃ beats 13/22    hard · avg 39%          3.2%  sharp · at its level
CPTUB average (SpeakLeash, 0-5 ju…   66.1%  ▁▂▂▂▃█●▆▅▄▂▃ beats 46/95    medium · avg 65%        2.8%  at its level
FlenQA (3K-token subset) (maker)     82.0%  ●···▄····▄·█ beats 0/4      easy · avg 86%          2.4%  sharp · noisy · several runs
BRIDGE clinical text, zero-shot a…   36.1%  ▁▁▁▁▁▂▂▃▄●█▃ beats 51/97    hard · avg 31%          2.3%  blunt · at its level
AraGen-v3 3C3H score (Inception, …   30.0%  ▁▁·▁▂▂▄●▄▅▄▄ beats 26/66    hard · avg 29%          2.3%  at its level
Polish EQ-Bench v2 (SpeakLeash)      64.4%  ▁··▁▁▂▂▆▅▆●▆ beats 44/70    medium · avg 62%        2.2%  at its level
GPQA (subset unspecified) (maker)    56.1%  ▂▁▂▂▄▃▆▃●█▃▃ beats 60/106   hard · avg 44%          1.9%  sharp · at its level · small run
MMLU (maker)                         84.8%  ▁▁▁▁▁▂▂▅██●▄ beats 185/214  easy · avg 77%          1.8%  small run
KMMLU-HARD (Daekeun Kim, zero-sho…   24.2%  ●···▁▁▁▂▃·▆▂ beats 6/23     very hard · avg 29%     1.7%  sharp · too hard for it
MMLU (simple-evals, zero-shot CoT…   84.8%  ▁···▂▁●█▂▂▃▁ beats 5/22     easy · avg 80%          1.4%  typical
SimpleQA (maker)                      3.0%  ●▄▄▆█▆███·▂█ beats 1/32     very hard · avg 4.6%    1.2%  too hard for it · small run
Open PT-LLM Leaderboard average (…   76.1%  ▃▁▁▁▁▁▁▁▂█●▁ beats 230/276  medium · avg 72%        0.9%  at its level · noisy
DiagnosticIQ industrial maintenan…   50.5%  ▁··▃●▁▁▅█▅▁▅ beats 3/24     hard · avg 51%          0.8%  blunt · at its level
EuroEval HellaSwag (English), tes…   84.4%  ▃·▁▁▂▁▂▅█●▄▁ beats 82/100   medium · avg 67%        0.7%  sharp · at its level · noisy
EuroEval Life in the UK (English …   89.2%  ▁▁▁▁▁▁▁▂▄█●▃ beats 114/146  easy · avg 82%          0.4%  noisy
Chess Puzzles (Epoch)                 1.0%  ●▁▁▁▁▂▃▃▃▂▁▁ beats 32/137   very hard · avg 5.8%    0.2%  sharp · too hard for it · noisy
Vectara hallucination leaderboard…   95.3%  ▁▁▂▃▄▄●▆▃▂▂▁ beats 67/140   very easy · avg 94%     0.2%  blunt · maxed out · noisy
DROP F1, 3-shot (simple-evals) (m…   75.5%  ●▂▆█▆·▃·▂▅·▂ beats 1/23     easy · avg 79%          0.2%  blunt
Vectara hallucination leaderboard…   96.3%  ▁▂▁▂▅█▂▃▄▃▁● beats 96/98    very easy · avg 90%     0.0%  blunt
LiveBench (release not stated) (m…   47.6%  no other model yet                                     —  not counted yet: no other model in the …
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
Epoch AI                                        4   16% ▇▇
Microsoft Research (submitted / maker-r…        9   16% ▇▇
Daekeun Kim Korean LLM benchmark runs           3   13% ▇▇
microsoft/phi-4 model card (submitted /…        6   11% ▇
Microsoft (submitted / maker-reported)          5   11% ▇
microsoft/Phi-4-reasoning model card (s…        5    9% ▇
LMArena leaderboard dataset                     1    4% ▇
CodeReviewQA: The Code Review Comprehen…        4    4% ▇
SpeakLeash CPTUB                                1    3% ▇
Berkeley Function Calling Leaderboard r…        2    3% ▇
BRIDGE Medical Leaderboard                      1    2% ▇
Inception Arabic/Hindi leaderboards             1    2% ▇
SpeakLeash Polish EQ-Bench leaderboard          1    2% ▇
EuroEval leaderboards                           2    1% ▇
Open Portuguese LLM Leaderboard                 1    1% ▇
DiagnosticIQ: A Benchmark for LLM-Based…        1    1% ▇
Vectara hallucination leaderboard               2    0% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 53.2
Licences: Epoch AI ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Microsoft Research ([CC BY 4.0](../../licenses/report-2412.08905v1.txt)) · Daekeun Kim Korean LLM benchmark runs ([MIT](../../licenses/korean-personal-benchmark-runs-MIT.txt)) · microsoft/phi-4 model card ([MIT](../../licenses/card-microsoft-phi-4.txt)) · Microsoft ([CC BY 4.0](../../licenses/report-2504.21318v1.txt)) · microsoft/Phi-4-reasoning model card ([mit](../../licenses/microsoft--Phi-4-reasoning.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · SpeakLeash CPTUB ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · Berkeley Function Calling Leaderboard result archive ([Apache-2.0](../../licenses/bfcl-result-archive-v4-Apache-2.0.txt)) · BRIDGE Medical Leaderboard ([MIT](../../licenses/bridge-medical-leaderboard-MIT.txt)) · Inception Arabic/Hindi leaderboards ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · SpeakLeash Polish EQ-Bench leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))
Not counted (1) 
source              test                            score  why
Microsoft Research  LiveBench (release not stated)  47.6%  too few models have it yet
## How to run it ### Through an API api (DeepInfra) · $0.088/M blended · $0.07 in / $0.14 out per 1M tokens · [deepinfra.com](https://deepinfra.com/microsoft/phi-4) [price source](https://github.com/BerriAI/litellm/blob/c19ce71bcdad234ca87a7af10cee45375ce811a8/model_prices_and_context_window.json): USD per 1M text tokens, standard uncached input and output. Product: Serverless text generation. API model: microsoft/phi-4. Settings: service tier=standard, batch=no, cache=uncached input. Additional/excluded charges: Tools, cache storage, priority service and other modalities excluded.. Currency: USD. · [context source](https://github.com/BerriAI/litellm/blob/c19ce71bcdad234ca87a7af10cee45375ce811a8/model_prices_and_context_window.json): provider maximum input tokens; not an intrinsic model maximum or legacy max\_tokens ### llama.cpp (recommended) 1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md) ```sh # macOS / Linux curl -LsSf https://llama.app/install.sh | sh # Windows irm https://llama.app/install.ps1 | iex ``` 2. Run Phi-4 14B (Q4_K_M, needs ~9.1 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/phi-4-GGUF/tree/0d33cfcf28e6de39a11cce88b451aec3ad6ba490)) ```sh llama cli -hf lmstudio-community/phi-4-GGUF:Q4_K_M ``` 3. Or start a local server and open the address it prints (http://localhost:9931) ```sh llama serve -hf lmstudio-community/phi-4-GGUF:Q4_K_M ``` Other versions: Q8_0 (~15.6 GB) `llama cli -hf lmstudio-community/phi-4-GGUF:Q8_0` ### LM Studio 1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download) 2. Run Phi-4 14B (Q4_K_M, needs ~9.1 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/phi-4-GGUF/tree/0d33cfcf28e6de39a11cce88b451aec3ad6ba490)) ```sh lms get https://huggingface.co/lmstudio-community/phi-4-GGUF@Q4_K_M -y ``` 3. Then run lms ls to see its name, and lms chat followed by that name Other versions: Q8_0 (~15.6 GB) `lms get https://huggingface.co/lmstudio-community/phi-4-GGUF@Q8_0 -y` · 4bit (~8.2 GB, Mac only) `lms get https://huggingface.co/mlx-community/phi-4-4bit -y` · 8bit (~15.6 GB, Mac only) `lms get https://huggingface.co/mlx-community/phi-4-8bit -y` ### Ollama 1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download) ```sh # Linux curl -fsSL https://ollama.com/install.sh | sh ``` 2. Run Phi-4 14B (Q4_K_M, needs ~9.32 GB, from [ollama.com](https://ollama.com/library/phi4/tags)) ```sh ollama run phi4:14b ``` Other versions: Q8_0 (~18.14 GB) `ollama run phi4:14b-q8_0` ### MLX (Mac) 1. Install MLX (Mac). Apple silicon Macs only, text only. ```sh # with uv uv tool install mlx-lm # or pip install mlx-lm ``` 2. Run Phi-4 14B (4bit, needs ~8.2 GB, Mac only, from [huggingface.co/mlx-community](https://huggingface.co/mlx-community/phi-4-4bit/tree/fc0f8f23d369dc29b55cad1d65cb5bf0dcbee910)) ```sh mlx_lm.chat --model mlx-community/phi-4-4bit ``` 3. Or start an OpenAI-compatible server at http://localhost:8080/v1 ```sh mlx_lm.server --model mlx-community/phi-4-4bit ``` Other versions: 8bit (~15.6 GB, Mac only) `mlx_lm.chat --model mlx-community/phi-4-8bit` Model repository and documentation: [huggingface.co/microsoft](https://huggingface.co/microsoft/phi-4) ## Quantization evidence How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent. They don't describe every download of that precision. 
precision                       test                                full → quantized    kept
FP8 W8A8 dynamic (weights sym…  GSM8K 5-shot strict-match (lm-eva…     0.901 → 0.907  100.7%
FP8 W8A8 dynamic (weights sym…  MMLU 5-shot (lm-evaluation-harnes…     0.803 → 0.803  100.0%
FP8 W8A8 dynamic (weights sym…  OpenLLM v1 average of 6 tasks (AR…     0.765 → 0.765   99.9%
INT4 W4A16 (weights only, sym…  GSM8K 5-shot strict-match (lm-eva…     0.901 → 0.897   99.6%
INT4 W4A16 (weights only, sym…  MMLU 5-shot (lm-evaluation-harnes…     0.803 → 0.799   99.5%
INT4 W4A16 (weights only, sym…  OpenLLM v1 average of 6 tasks (AR…      0.765 → 0.76   99.3%
INT8 W8A8 (weights static per…  GSM8K 5-shot strict-match (lm-eva…     0.901 → 0.903  100.3%
INT8 W8A8 (weights static per…  MMLU 5-shot (lm-evaluation-harnes…     0.803 → 0.804  100.1%
INT8 W8A8 (weights static per…  OpenLLM v1 average of 6 tasks (AR…     0.765 → 0.763   99.8%
Each kept value links to its source. [interactive version](https://whatmodel.sh/model/phi-4) · [JSON](https://whatmodel.sh/api?format=json&all=1)