[whatmodel.sh](../../README.md) / Phi-3-vision-128k-instruct # Phi-3-vision-128k-instruct Microsoft · released 2024-05 · 4.15B params · mit **score 43.0** ±1.7 · #518 of 732 · from 9 benchmark results across 4 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks Phi-3-vision-128k-instruct among every model on that task or test, worst to best.}

task                vs the others          rank  counts  tests
math                ▁▁▃▅●▆▄█▄▂▁▁   #310 of 431*    5.9%      1
vision & documents  ●▁▂▂▁▃▄▂▆▇█▄    #153 of 153     99%      9

math · #310 of 431 · 5.9% of its score

Beats [Molmo 2 8B](molmo-2-8b.md), [Llama 3.1 Nemotron 70B Instruct](llama-3-1-nemotron-70b.md), [GPT-4 (0613)](gpt-4-0613.md) here, though they rank higher overall. Trails [DeepSeek R1 Distill Qwen 1.5B](deepseek-r1-distill-qwen-1-5b.md), [Gemma 7B (base)](gemma-7b-base.md), [Mistral 7B v0.1 (base)](mistral-7b-v0-1.md) here, though they rank lower overall.

test                          result  vs the others in its run  difficulty        counts  why
MathVista (testmini) (maker)   44.5%  ▁··▁●▄▆▇▄▅█▄ beats 4/48   medium · avg 49%    5.9%  at its level · small run

vision & documents · #153 of 153 · 99% of its score

Trails [Qwen3.5 0.8B](qwen3-5-0-8b.md), [Qwen2-VL 2B Instruct](qwen2-vl-2b.md), [Phi-3.5-vision-instruct](phi-3-5-vision-instruct.md) here, though they rank lower overall.

test                                result  vs the others in its run  difficulty        counts  why
CharXiv descriptive questions        60.5%  ▄█●▄▄·██·▄·█ beats 3/12   easy · avg 69%       42%  at its level
Arena vision (style controlled)      880.5  ●▁▂▂▂▂▃▄▅▅█▇ beats 0/152  avg 1061             21%  blunt
CharXiv (reasoning questions)        31.6%  ▂▂▂●▃▂·▂▂··▂ beats 4/13   hard · avg 34%       15%  typical
MMMU (val) (maker)                   40.4%  ▁····▁●▃█▆▆▅ beats 2/53   hard · avg 48%      9.1%  small run
MathVista (testmini) (maker)         44.5%  ▁··▁●▄▆▇▄▅█▄ beats 4/48   medium · avg 49%    5.9%  at its level · small run
PhysBench physical-world understa…   38.4%  ▅●█··▅▃····▃ beats 2/9    hard · avg 42%      3.2%  blunt
ChartQA (maker)                      81.4%  ▁··▁▁▁▁▁▂●▅█ beats 16/44  easy · avg 78%      1.3%  blunt · at its level · noisy · small run
TextVQA (maker)                      70.9%  ▁▁▁▄●▁▃·▃▃▃█ beats 9/24   easy · avg 75%      1.2%  blunt · at its level · small run
AI2D test (maker)                    76.7%  ▁▁▅●▃▇·▄·█·▃ beats 9/28   easy · avg 82%      0.8%  blunt · at its level · small run

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
CharXiv leaderboard                             2   56% ▇▇▇▇▇▇▇
LMArena leaderboard dataset                     1   21% ▇▇
Phi-3 Technical Report (submitted / mak…        5   18% ▇▇
PhysBench: Benchmarking and Enhancing V…        1    3% ▇
prior: pull toward the middle                        1% ▇
weighted average                                   = 43.0

_{Licences: CharXiv leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Phi-3 Technical Report ([CC BY 4.0](http://creativecommons.org/licenses/by/4.0/)) · PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/))} ## How to run it ### Other ways to run it 1. Run it as published (BF16, needs ~8.29 GB, NVIDIA only, from [github.com/vllm-project](https://github.com/vllm-project/vllm/blob/64cb68384241f046d07ba3797e1b14dbc82a0b2d/docs/models/supported_models.md)) ```sh vllm serve microsoft/Phi-3-vision-128k-instruct --trust-remote-code ``` Model repository and documentation: [huggingface.co/microsoft](https://huggingface.co/microsoft/Phi-3-vision-128k-instruct) [interactive version](https://whatmodel.sh/model/phi-3-vision-128k-instruct) · [JSON](https://whatmodel.sh/api?format=json&all=1)