[whatmodel.sh](../../README.md) / Phi-3-small-8k-instruct # Phi-3-small-8k-instruct Microsoft · released 2024-05 · 7.39B params · mit **score 44.1** ±1.5 · #495 of 732 · from 4 benchmark results across 4 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks Phi-3-small-8k-instruct among every model on that task or test, worst to best.}

task            vs the others         rank  counts  tests
chat & writing  ▁▁▁▃●▄▆█▆▇▇▃   #333 of 402     93%      3

chat & writing · #333 of 402 · 93% of its score

Beats [OpenThinker3 7B](openthinker3-7b.md), [Code Llama 70B Instruct](codellama-70b-instruct.md), [Gemma 4 E2B](gemma-4-e2b.md) here, though they rank higher overall. Trails [LFM2 350M](lfm2-350m.md), [Gemma 3 1B](gemma-3-1b.md), [Qwen3 0.6B](qwen3-0-6b.md) here, though they rank lower overall.

test                                result  vs the others in its run   difficulty        counts  why
Arena-Hard (v0.1, GPT-4-1106 judg…   29.8%  ▂▁▁▅█●▄▃▃▅▃▁ beats 25/60   medium · avg 47%     45%  sharp
LMArena (text)                      1171.1  ▁▁▁▂●▃▃▆▆▇█▃ beats 51/385  avg 1260             25%  blunt
Polyrating rating (bias-aware re-…  1103.0  ▂▁▂▁▃▇●▆▇▄▂▂ beats 48/95   avg 1187             23%  typical

other tests (1)

test                                result  vs the others in its run    difficulty        counts  why
Open PT-LLM Leaderboard average (…   70.8%  ▃▁▁▁▁▁▁▁▂●▅▁ beats 172/276  medium · avg 72%    6.0%  at its level · noisy

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                    results  share of the score
Arena-Hard-Auto v0.1 leaderboard                1   45% ▇▇▇▇▇
LMArena leaderboard dataset                     1   25% ▇▇▇
Polyrating: A Cost-Effective and Bias-A…        1   23% ▇▇▇
Open Portuguese LLM Leaderboard                 1    6% ▇
prior: pull toward the middle                        1% ▇
weighted average                                   = 44.1

_{Licences: Arena-Hard-Auto v0.1 leaderboard ([Apache-2.0](../../licenses/arena-hard-auto-v0-1-Apache-2.0.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt))} ## How to run it ### Other ways to run it 1. Run it as published (BF16, needs ~14.78 GB, NVIDIA only, from [huggingface.co/microsoft](https://huggingface.co/microsoft/Phi-3-small-8k-instruct/blob/188b876e997ee4055df01afe46274c14c400fe40/README.md)) ```sh pip install tiktoken==0.6.0 triton==2.3.0 transformers==4.40.2 torch && python -c "import torch; from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline; r='microsoft/Phi-3-small-8k-instruct'; d=torch.cuda.current_device(); m=AutoModelForCausalLM.from_pretrained(r, torch_dtype='auto', trust_remote_code=True).to(d); t=AutoTokenizer.from_pretrained(r, trust_remote_code=True); print(pipeline('text-generation', model=m, tokenizer=t, device=d)([{'role': 'user', 'content': 'What about solving an 2x + 3 = 7 equation?'}], max_new_tokens=500, return_full_text=False, do_sample=False)[0]['generated_text'])" ``` Model repository and documentation: [huggingface.co/microsoft](https://huggingface.co/microsoft/Phi-3-small-8k-instruct) [interactive version](https://whatmodel.sh/model/phi-3-small-8k-instruct) · [JSON](https://whatmodel.sh/api?format=json&all=1)