[whatmodel.sh](../../README.md) / Yi 1.5 9B Chat # Yi 1.5 9B Chat 01.AI · released 2024-05 · 8.83B params · apache-2.0 **score 45.7** ±0.8 · #459 of 732 · from 15 benchmark results across 7 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks Yi 1.5 9B Chat among every model on that task or test, worst to best. 
task    vs the others          rank  counts  tests
coding  ▁▁▁▃▅▇●▇▅▄▃▁    #285 of 469     22%      5
math    ▁▁▃●█▆▄█▄▂▁▁   #371 of 431*     16%      2
coding · #285 of 469 · 22% of its score Beats [Qwen3-VL 32B Instruct](qwen3-vl-32b-instruct.md), [Solar Open 100B (102B-A12B)](solar-open-100b.md), [Qwen3-VL 30B-A3B Instruct](qwen3-vl-30b-a3b-instruct.md) here, though they rank higher overall. Trails [DeepSeek LLM 67B Chat](deepseek-llm-67b-chat.md) here, though they rank lower overall. 
test                                result  vs the others in its run  difficulty        counts  why
CRUXEval (ZeroEval, 800 problems)    44.8%  ▁▂●▃▆█▆▃▂·▁▁ beats 9/47   medium · avg 57%     17%  sharp · at its level
CodeReviewQA: solution identifica…   58.9%  ▃··▁▂▂▃▂●▃▄▂ beats 33/47  medium · avg 51%    1.8%  sharp · at its level · noisy
CodeReviewQA: change localisation…   43.2%  █▂▁▁▂▂▂▃▅●▅▇ beats 33/47  hard · avg 33%      1.6%  sharp · noisy
CodeReviewQA: automated code refi…   30.7%  ▁▁▁▁▁▁▂▂▃●▅▄ beats 22/47  hard · avg 32%      1.4%  at its level
CodeReviewQA: change type recogni…   78.6%  ▁▁▁······▁▃● beats 42/47  easy · avg 67%      0.1%  blunt · at its level · noisy
math · #371 of 431 · 16% of its score Beats [GPT-3.5 Turbo (0125)](gpt-3-5-turbo-0125.md), [DBRX Instruct](dbrx-instruct.md), [Command R Plus (04-2024)](command-r-plus-04-2024.md) here, though they rank higher overall. Trails [OPT-1.3B](opt-1-3b.md), [OPT-2.7B](opt-2-7b.md), [Gemma 2B (base)](gemma-2b-base.md) here, though they rank lower overall. 
test                                result  vs the others in its run  difficulty           counts  why
GSM8K (ZeroEval, zero-shot)          76.4%  ▁▄·▁●█▅▅▅███ beats 7/43   very easy · avg 87%     11%  sharp
MATH Level 5 (ZeroEval, 721 probl…   20.0%  ▁▁▁●█▂▇▇▄··▂ beats 8/38   hard · avg 30%         5.3%  sharp
other tests (8) 
test                                          result  vs the others in its run    difficulty           counts  why
MMLU-Redux (ZeroEval, 2,778 quest…             65.0%  ▁·▁▁●▄▄▅█▄▂▁ beats 8/53     easy · avg 75%          30%  at its level
BigCodeBench (Complete)                        42.2%  ▂▁▁▄▂▅▇●▆▂▂▂ beats 33/58    medium · avg 48%       6.9%  at its level
BigCodeBench (Instruct)                        34.3%  ▃▂·▆▆●█▆▅▃▃▃ beats 15/33    medium · avg 39%       6.8%  at its level
WildBench v2 WB-Score (task macro)  38.6654 WB-Score  ▁·▁▁·▃█▄●▆▅▆ beats 25/52    avg 43.019 WB-Score    6.6%  sharp
ZebraLogic (ZeroEval, 1,000 puzzl…              2.3%  ●▁▃▆██▁▁·▁▁▁ beats 0/57     hard · avg 15%         4.8%  sharp
Polish EQ-Bench v2 (SpeakLeash)                48.8%  ▁··▁▁▂▂▆●▆█▆ beats 23/70    medium · avg 62%       3.9%  at its level
Open PT-LLM Leaderboard average (…             67.9%  ▃▁▁▁▁▁▁▁▂●▅▁ beats 143/276  medium · avg 72%       1.5%  at its level · noisy
Vectara hallucination leaderboard…             95.1%  ▁▁▂▃▄▄●▆▃▂▂▁ beats 61/140   very easy · avg 94%    0.5%  blunt · maxed out · noisy
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
WildEval ZeroEval published results             5   69% ▇▇▇▇▇▇▇▇
BigCodeBench paper                              2   14% ▇▇
WildBench v2 leaderboard data                   1    7% ▇
CodeReviewQA: The Code Review Comprehen…        4    5% ▇
SpeakLeash Polish EQ-Bench leaderboard          1    4% ▇
Open Portuguese LLM Leaderboard                 1    2% ▇
Vectara hallucination leaderboard               1    0% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 45.7
Licences: WildEval ZeroEval published results ([Apache-2.0](../../licenses/zeroeval-published-results-Apache-2.0.txt)) · BigCodeBench paper ([CC BY 4.0](http://creativecommons.org/licenses/by/4.0/)) · WildBench v2 leaderboard data ([Apache-2.0](https://github.com/allenai/WildBench/blob/d6b8dcaf377d173d031980f97c16e1a82618c03d/LICENSE)) · CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · SpeakLeash Polish EQ-Bench leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) ## How to run it ### llama.cpp (recommended) 1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md) ```sh # macOS / Linux curl -LsSf https://llama.app/install.sh | sh # Windows irm https://llama.app/install.ps1 | iex ``` 2. Run Yi 1.5 9B Chat (Q4_K_M, needs ~5.3 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Yi-1.5-9B-Chat-GGUF/tree/1d48fb0b470dc424efc7555727eaaeffd0e323d2)) ```sh llama cli -hf lmstudio-community/Yi-1.5-9B-Chat-GGUF:Q4_K_M ``` 3. Or start a local server and open the address it prints (http://localhost:9931) ```sh llama serve -hf lmstudio-community/Yi-1.5-9B-Chat-GGUF:Q4_K_M ``` Other versions: Q8_0 (~9.4 GB) `llama cli -hf lmstudio-community/Yi-1.5-9B-Chat-GGUF:Q8_0` ### LM Studio 1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download) 2. Run Yi 1.5 9B Chat (Q4_K_M, needs ~5.3 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/Yi-1.5-9B-Chat-GGUF/tree/1d48fb0b470dc424efc7555727eaaeffd0e323d2)) ```sh lms get https://huggingface.co/lmstudio-community/Yi-1.5-9B-Chat-GGUF@Q4_K_M -y ``` 3. Then run lms ls to see its name, and lms chat followed by that name Other versions: Q8_0 (~9.4 GB) `lms get https://huggingface.co/lmstudio-community/Yi-1.5-9B-Chat-GGUF@Q8_0 -y` · 4bit (~5 GB, Mac only) `lms get https://huggingface.co/mlx-community/Yi-1.5-9B-Chat-4bit -y` ### Ollama 1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download) ```sh # Linux curl -fsSL https://ollama.com/install.sh | sh ``` 2. Run Yi 1.5 9B Chat (Q4_K_M, needs ~5.8 GB, from [ollama.com](https://ollama.com/library/yi/tags)) ```sh ollama run yi:9b-chat-v1.5-q4_K_M ``` Other versions: Q8_0 (~11.1 GB) `ollama run yi:9b-chat-v1.5-q8_0` ### MLX (Mac) 1. Install MLX (Mac). Apple silicon Macs only, text only. ```sh # with uv uv tool install mlx-lm # or pip install mlx-lm ``` 2. Run Yi 1.5 9B Chat (4bit, needs ~5 GB, Mac only, from [huggingface.co/mlx-community](https://huggingface.co/mlx-community/Yi-1.5-9B-Chat-4bit/tree/e15f886479c44e7d90f0ac13ace69b2319b71c2f)) ```sh mlx_lm.chat --model mlx-community/Yi-1.5-9B-Chat-4bit ``` 3. Or start an OpenAI-compatible server at http://localhost:8080/v1 ```sh mlx_lm.server --model mlx-community/Yi-1.5-9B-Chat-4bit ``` Model repository and documentation: [huggingface.co/01-ai](https://huggingface.co/01-ai/Yi-1.5-9B-Chat) [interactive version](https://whatmodel.sh/model/yi-1-5-9b-chat) · [JSON](https://whatmodel.sh/api?format=json&all=1)