[whatmodel.sh](../../README.md) / Qwen1.5 7B Chat # Qwen1.5 7B Chat Alibaba · released 2024-02 · 7.72B params · tongyi-qianwen **score 38.5** ±0.7 · #586 of 732 · from 13 benchmark results across 10 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks Qwen1.5 7B Chat among every model on that task or test, worst to best. 
task            vs the others          rank  counts  tests
coding          ▁▁▁▃●▇█▇▅▄▃▁   #403 of 469*     25%      1
chat & writing  ▁▁▁▃●▄▆█▆▇▇▃    #359 of 402     22%      3
coding · #403 of 469 · 25% of its score Beats [DeepSeek R1 Distill Qwen 7B](deepseek-r1-distill-qwen-7b.md), [Qwen3.5 2B](qwen3-5-2b.md), [DeepSeek R1 Distill Llama 8B](deepseek-r1-distill-llama-8b.md) here, though they rank higher overall. Trails [Qwen2.5 1.5B Instruct](qwen2-5-1-5b.md), [WizardLM 13B V1.2](wizardlm-13b-v1-2.md), [Falcon3 3B Instruct](falcon3-3b.md) here, though they rank lower overall. 
test       result  vs the others in its run  difficulty      counts  why
HumanEval   42.1%  ▁█▄●▇█·▄▇▅▄▄ beats 11/42  easy · avg 75%     25%  sharp · at its level
chat & writing · #359 of 402 · 22% of its score Beats [Code Llama 70B Instruct](codellama-70b-instruct.md), [Gemma 4 E2B](gemma-4-e2b.md), [Phi-3-mini-4k-instruct (June 2024 update)](phi-3-mini-4k-instruct-june-2024.md) here, though they rank higher overall. Trails [Gemma 3 270M IT](gemma-3-270m.md), [SmolLM2 360M Instruct](smollm2-360m.md), [Granite 4.0 H 350M](granite-4-0-h-350m.md) here, though they rank lower overall. 
test                                result  vs the others in its run   difficulty        counts  why
AlpacaEval 2 length-controlled       14.8%  ▁▂▂▃▆●█▆▄▄▁▁ beats 39/122  medium · avg 35%     11%  sharp
LMArena (text)                      1143.9  ▁▁▁●▃▃▃▆▆▇█▃ beats 36/385  avg 1260            5.8%  blunt
Polyrating rating (bias-aware re-…  1086.0  ▂▁▂▁▃▇●▆▇▄▂▂ beats 40/95   avg 1187            5.3%  typical
other tests (9) 
test                                          result  vs the others in its run    difficulty           counts  why
LiveBench 2024-07-26                           18.0%  ▁▁▁▂▂●▂█▆▃▅▃ beats 12/61    medium · avg 38%        15%  typical
EvoEval (overall)                              25.5%  ▂▂▃●▄█▁▄▅▃▁▃ beats 10/42    medium · avg 52%        13%  at its level
MMLU-Pro                                       26.4%  ●▂▅█▃·█▃▅··▂ beats 1/23     medium · avg 54%       7.2%  typical
WildBench v2 WB-Score (task macro)  23.4232 WB-Score  ▁·▁▁·●█▄▇▆▅▆ beats 6/52     avg 43.019 WB-Score    6.2%  sharp
MedQA (USMLE, 4 options)                       45.4%  ▁▁··▄▁▁··●▃█ beats 8/17     medium · avg 71%       4.4%  sharp · at its level
Polish EQ-Bench v2 (SpeakLeash)                25.0%  ▁··▁▁●▂▆▅▆█▆ beats 4/70     medium · avg 62%       3.3%  at its level
MedMCQA                                        45.3%  ▂▂··▄▄▂▂·●██ beats 9/17     medium · avg 63%       2.0%  at its level
Open PT-LLM Leaderboard average (…             67.7%  ▃▁▁▁▁▁▁▁▂●▅▁ beats 138/276  medium · avg 72%       1.3%  at its level · noisy
PubMedQA                                       58.6%  ▂·▂●·▄▂▄▄▂█▂ beats 2/17     medium · avg 75%       0.3%  blunt · at its level
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
Top Leaderboard Ranking = Top Coding Pr…        2   38% ▇▇▇▇▇
LiveBench model judgments                       1   15% ▇▇
AlpacaEval 1.0 and 2.0 leaderboards             1   11% ▇
MMLU-Pro author-released evaluation runs        1    7% ▇
Open Medical-LLM Leaderboard results            3    7% ▇
WildBench v2 leaderboard data                   1    6% ▇
LMArena leaderboard dataset                     1    6% ▇
Polyrating: A Cost-Effective and Bias-A…        1    5% ▇
SpeakLeash Polish EQ-Bench leaderboard          1    3% ▇
Open Portuguese LLM Leaderboard                 1    1% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 38.5
Licences: Top Leaderboard Ranking = Top Coding Proficiency ([CC BY 4.0](../../licenses/paper-2403.19114v1-CC-BY-4.0.txt)) · LiveBench model judgments ([Apache-2.0 (stated in the LiveBench datasheet)](https://www.apache.org/licenses/LICENSE-2.0)) · AlpacaEval 1.0 and 2.0 leaderboards ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · MMLU-Pro author-released evaluation runs ([Apache-2.0](../../licenses/mmlu-pro-author-runs-Apache-2.0.txt)) · Open Medical-LLM Leaderboard results ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · WildBench v2 leaderboard data ([Apache-2.0](https://github.com/allenai/WildBench/blob/d6b8dcaf377d173d031980f97c16e1a82618c03d/LICENSE)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · SpeakLeash Polish EQ-Bench leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) ## How to run it ### llama.cpp (recommended) 1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md) ```sh # macOS / Linux curl -LsSf https://llama.app/install.sh | sh # Windows irm https://llama.app/install.ps1 | iex ``` 2. Run Qwen1.5 7B Chat (Q4_K_M, needs ~4.8 GB, from [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen1.5-7B-Chat-GGUF/tree/79b781af68a192ee2ec43cded6c3b5448a70df66)) ```sh llama cli -hf Qwen/Qwen1.5-7B-Chat-GGUF:q4_k_m ``` 3. Or start a local server and open the address it prints (http://localhost:9931) ```sh llama serve -hf Qwen/Qwen1.5-7B-Chat-GGUF:q4_k_m ``` Other versions: Q8_0 (~8.2 GB) `llama cli -hf Qwen/Qwen1.5-7B-Chat-GGUF:q8_0` ### LM Studio 1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download) 2. Run Qwen1.5 7B Chat (Q4_K_M, needs ~4.8 GB, from [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen1.5-7B-Chat-GGUF/tree/79b781af68a192ee2ec43cded6c3b5448a70df66)) ```sh lms get https://huggingface.co/Qwen/Qwen1.5-7B-Chat-GGUF@q4_k_m -y ``` 3. Then run lms ls to see its name, and lms chat followed by that name Other versions: Q8_0 (~8.2 GB) `lms get https://huggingface.co/Qwen/Qwen1.5-7B-Chat-GGUF@q8_0 -y` · 4bit (~5.2 GB, Mac only) `lms get https://huggingface.co/mlx-community/Qwen1.5-7B-Chat-4bit -y` ### Ollama 1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download) ```sh # Linux curl -fsSL https://ollama.com/install.sh | sh ``` 2. Run Qwen1.5 7B Chat (Q4_K_M, needs ~5.13 GB, from [ollama.com](https://ollama.com/library/qwen/tags)) ```sh ollama run qwen:7b-chat-v1.5-q4_K_M ``` Other versions: Q8_0 (~9.76 GB) `ollama run qwen:7b-chat-v1.5-q8_0` ### MLX (Mac) 1. Install MLX (Mac). Apple silicon Macs only, text only. ```sh # with uv uv tool install mlx-lm # or pip install mlx-lm ``` 2. Run Qwen1.5 7B Chat (4bit, needs ~5.2 GB, Mac only, from [huggingface.co/mlx-community](https://huggingface.co/mlx-community/Qwen1.5-7B-Chat-4bit/tree/efdba7c29d03d9a3c28b061534be628c49b3738e)) ```sh mlx_lm.chat --model mlx-community/Qwen1.5-7B-Chat-4bit ``` 3. Or start an OpenAI-compatible server at http://localhost:8080/v1 ```sh mlx_lm.server --model mlx-community/Qwen1.5-7B-Chat-4bit ``` Model repository and documentation: [huggingface.co/Qwen](https://huggingface.co/Qwen/Qwen1.5-7B-Chat) [interactive version](https://whatmodel.sh/model/qwen1-5-7b-chat) · [JSON](https://whatmodel.sh/api?format=json&all=1)