[whatmodel.sh](../../README.md) / Phi-3-mini-128k-instruct (Apr 2024) # Phi-3-mini-128k-instruct (Apr 2024) Microsoft · released 2024-04 · 3.82B params · mit **score 41.2** ±1.5 · #543 of 732 · from 10 benchmark results across 7 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks Phi-3-mini-128k-instruct (Apr 2024) among every model on that task or test, worst to best.}

task            vs the others         rank  counts  tests
coding          ▁▁▁▃▅●█▇▅▄▃▁   #335 of 469     13%      4
chat & writing  ▁▁▁●▆▄▆█▆▇▇▃   #371 of 402     74%      3

coding · #335 of 469 · 13% of its score

Beats [Qwen3-VL 8B Instruct](qwen3-vl-8b-instruct.md), [Qwen3-VL 4B Instruct](qwen3-vl-4b-instruct.md), [LFM2.5 2.6B](lfm2-5-2-6b.md) here, though they rank higher overall.

test                                result  vs the others in its run  difficulty        counts  why
CodeReviewQA: solution identifica…   45.9%  ▃··▁▂▂▃▂●▃▄▂ beats 22/47  medium · avg 51%    5.2%  sharp · noisy
CodeReviewQA: automated code refi…    7.7%  ▁▁▁▁●▁▂▂▃█▅▄ beats 6/47   hard · avg 32%      4.3%  typical
CodeReviewQA: change localisation…   30.7%  █▂▁▁▂▂▂▃●▅▅▇ beats 26/47  hard · avg 33%      3.8%  sharp · noisy
CodeReviewQA: change type recogni…   76.9%  ▁▁▁······▁▃● beats 39/47  easy · avg 67%      0.2%  blunt · at its level · noisy

chat & writing · #371 of 402 · 74% of its score

Beats [Code Llama 70B Instruct](codellama-70b-instruct.md), [Gemma 4 E2B](gemma-4-e2b.md) here, though they rank higher overall. Trails [Gemma 3 270M IT](gemma-3-270m.md), [SmolLM2 360M Instruct](smollm2-360m.md), [Granite 4.0 350M](granite-4-0-350m.md) here, though they rank lower overall.

test                                result  vs the others in its run   difficulty        counts  why
Arena-Hard (v0.1, GPT-4-1106 judg…   15.4%  ▂▁▁●█▅▄▃▃▅▃▁ beats 9/60    medium · avg 47%     30%  sharp
LMArena (text)                      1129.9  ▁▁▁●▃▃▃▆▆▇█▃ beats 27/385  avg 1260             23%  blunt
Polyrating rating (bias-aware re-…  1072.0  ▂▁▂▁▃▇●▆▇▄▂▂ beats 34/95   avg 1187             21%  typical

other tests (3)

test                                result  vs the others in its run    difficulty           counts  why
Open PT-LLM Leaderboard average (…   65.7%  ▃▁▁▁▁▁▁▁▂●▅▁ beats 119/276  medium · avg 72%       5.3%  at its level · noisy
EuroEval Life in the UK (English …   80.2%  ▁▁▁▁▁▁▁▂▄●▄▃ beats 65/146   easy · avg 82%         3.8%  at its level · noisy
Vectara hallucination leaderboard…   96.9%  ▁▁▂▃▄▄█●▃▂▂▁ beats 101/140  very easy · avg 94%    2.2%  blunt · noisy

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
Arena-Hard-Auto v0.1 leaderboard                1   30% ▇▇▇▇
LMArena leaderboard dataset                     1   23% ▇▇▇
Polyrating: A Cost-Effective and Bias-A…        1   21% ▇▇▇
CodeReviewQA: The Code Review Comprehen…        4   13% ▇▇
Open Portuguese LLM Leaderboard                 1    5% ▇
EuroEval leaderboards                           1    4% ▇
Vectara hallucination leaderboard               1    2% ▇
prior: pull toward the middle                        1% ▇
weighted average                                   = 41.2

_{Licences: Arena-Hard-Auto v0.1 leaderboard ([Apache-2.0](../../licenses/arena-hard-auto-v0-1-Apache-2.0.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0))} ## How to run it ### Other ways to run it 1. Run it as published (BF16, needs ~7.64 GB, from [huggingface.co/microsoft](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct/tree/bb5bf1e4001277a606e11debca0ef80323e5f824)) ```sh uv run --with torch --with 'transformers<5' --with accelerate python -c "from transformers import pipeline; print(pipeline('text-generation', model='microsoft/Phi-3-mini-128k-instruct', revision='bb5bf1e4001277a606e11debca0ef80323e5f824', torch_dtype='auto', device_map='auto')('Once upon a time', max_new_tokens=40)[0]['generated_text'])" ``` Model repository and documentation: [huggingface.co/microsoft](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct) [interactive version](https://whatmodel.sh/model/phi-3-mini-128k-instruct) · [JSON](https://whatmodel.sh/api?format=json&all=1)