[whatmodel.sh](../../README.md) / SmolLM2 1.7B Instruct # SmolLM2 1.7B Instruct Hugging Face · released 2024-10 · 1.71B params · apache-2.0 **score 33.0** ±1.0 · #643 of 732 · from 18 benchmark results across 5 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks SmolLM2 1.7B Instruct among every model on that task or test, worst to best. 
task            vs the others          rank  counts  tests
coding          ▁▁▁▃●▇█▇▅▄▃▁   #415 of 469*    6.4%      1
chat & writing  ▁▁▁●▆▄▆█▆▇▇▃    #381 of 402     19%      4
math            ▁▁●▅█▆▄█▄▂▁▁    #389 of 431     30%      2
coding · #415 of 469 · 6.4% of its score Beats [DeepSeek R1 Distill Qwen 7B](deepseek-r1-distill-qwen-7b.md), [Qwen3.5 2B](qwen3-5-2b.md), [DeepSeek R1 Distill Llama 8B](deepseek-r1-distill-llama-8b.md) here, though they rank higher overall. 
test               result  vs the others in its run   difficulty      counts  why
HumanEval (maker)   28.1%  ▁▁▁▁▂●▆▆█▅▂▁ beats 40/205  easy · avg 76%    6.4%  sharp · at its level · small run
chat & writing · #381 of 402 · 19% of its score Beats [Phi-2 (2.7B)](phi-2.md), [Qwen2.5 1.5B Instruct](qwen2-5-1-5b.md), [Nous-Hermes-13b](nous-hermes-13b.md) here, though they rank higher overall. Trails [Gemma 3 270M IT](gemma-3-270m.md), [SmolLM2 360M Instruct](smollm2-360m.md), [Granite 4.0 350M](granite-4-0-350m.md) here, though they rank lower overall. 
test                                result  vs the others in its run   difficulty        counts  why
LMArena (text)                      1114.8  ▁▁▁●▃▃▃▆▆▇█▃ beats 19/385  avg 1260             11%  blunt
IFEval prompt-level loose            51.6%  ▁●▁▃▁▄▁▁█▃▃▁ beats 1/22    easy · avg 78%      3.1%  at its level
AlpacaEval 2 length-controlled        5.8%  ●▃·▁▁█▇▅·▁·▁ beats 0/22    medium · avg 39%    2.4%  sharp
IFEval (aggregation unspecified) …   56.7%  ▁▂▄●▂▄▆██▅▂▁ beats 20/145  easy · avg 76%      2.0%  at its level · small run
math · #389 of 431 · 30% of its score Beats [Yi 34B Chat](yi-34b-chat.md), [Llama 3 8B Instruct](llama-3-8b.md), [Command R (03-2024)](command-r-03-2024.md) here, though they rank higher overall. Trails [GPT-Neo 125M](gpt-neo-125m.md), [GPT-2 XL (1.5B)](gpt-2-xl.md), [GPT-Neo 1.3B](gpt-neo-1-3b.md) here, though they rank lower overall. 
test                       result  vs the others in its run  difficulty           counts  why
GSM8K                       45.3%  ▂●··▃▂▅█▆▃▂▃ beats 1/22   very easy · avg 88%     22%  sharp · at its level · several runs
MATH (subset unspecified)   20.3%  ●▃█▂▂▆·····▁ beats 0/22   medium · avg 58%       7.6%  small run · several runs
other tests (8) 
test                                result  vs the others in its run   difficulty        counts  why
BIG-Bench Hard                       39.8%  ▁●·▁▁·▅█·▄▄▁ beats 2/22    medium · avg 72%     15%  sharp · several runs
MMLU                                 34.3%  ●▂·▁·▁▁█·▃▃▅ beats 0/22    easy · avg 75%       14%  typical
CPTUB average (SpeakLeash, 0-5 ju…   29.9%  ●▂▂▂▃█▃▆▅▄▂▃ beats 1/95    medium · avg 65%    8.4%  at its level
HellaSwag (maker)                    66.1%  ▁▁▁▂▂▂▄●█▇▄▂ beats 58/135  easy · avg 82%      2.6%  at its level · small run
PopQA                                16.4%  ▁▁▁●▃▃▁█▁▃·▄ beats 4/21    hard · avg 31%      1.8%  typical
EuroEval Life in the UK (English …   50.2%  ▁▁▁▁▁▁●▂▄█▄▃ beats 19/146  easy · avg 82%      1.8%  at its level · noisy
PIQA (maker)                         74.4%  ▁▁▁▂▂▃●█▅▃▁▁ beats 42/123  easy · avg 84%      1.1%  blunt · at its level · small run
TruthfulQA MC2                       45.3%  ▃●▂·▃▃·▅█▂▅▂ beats 2/21    medium · avg 60%    0.5%  blunt · at its level
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
2 OLMo 2 Furious                                8   46% ▇▇▇▇▇
SmolLM2 paper (submitted / maker-report…        7   33% ▇▇▇▇
LMArena leaderboard dataset                     1   11% ▇
SpeakLeash CPTUB                                1    8% ▇
EuroEval leaderboards                           1    2% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 33.0
Licences: 2 OLMo 2 Furious ([CC BY 4.0](../../licenses/paper-2501.00656v3-CC-BY-4.0.txt)) · SmolLM2 paper ([CC BY 4.0](../../licenses/paper-2502.02737v1-CC-BY-4.0.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · SpeakLeash CPTUB ([MIT](../../licenses/hf-leaderboard-spaces-MIT.txt)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) ## How to run it ### llama.cpp (recommended) 1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md) ```sh # macOS / Linux curl -LsSf https://llama.app/install.sh | sh # Windows irm https://llama.app/install.ps1 | iex ``` 2. Run SmolLM2 1.7B Instruct (Q4_K_M, needs ~1.1 GB, from [huggingface.co/HuggingFaceTB](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct-GGUF/tree/2d4a76a30b4af41ecd395c35725ac11688d4cfe4)) ```sh llama cli -hf HuggingFaceTB/SmolLM2-1.7B-Instruct-GGUF:q4_k_m ``` 3. Or start a local server and open the address it prints (http://localhost:9931) ```sh llama serve -hf HuggingFaceTB/SmolLM2-1.7B-Instruct-GGUF:q4_k_m ``` ### LM Studio 1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download) 2. Run SmolLM2 1.7B Instruct (Q4_K_M, needs ~1.1 GB, from [huggingface.co/HuggingFaceTB](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct-GGUF/tree/2d4a76a30b4af41ecd395c35725ac11688d4cfe4)) ```sh lms get https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct-GGUF@q4_k_m -y ``` 3. Then run lms ls to see its name, and lms chat followed by that name Other versions: 4bit (~1 GB, Mac only) `lms get https://huggingface.co/Irfanuruchi/SmolLM2-1.7B-Instruct-MLX-4bit -y` · 8bit (~1.8 GB, Mac only) `lms get https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct-Q8-mlx -y` ### Ollama 1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download) ```sh # Linux curl -fsSL https://ollama.com/install.sh | sh ``` 2. Run SmolLM2 1.7B Instruct (Q4_K_M, needs ~1.53 GB, from [ollama.com](https://ollama.com/library/smollm2/tags)) ```sh ollama run smollm2:1.7b-instruct-q4_K_M ``` Other versions: Q8_0 (~2.55 GB) `ollama run smollm2:1.7b-instruct-q8_0` ### MLX (Mac) 1. Install MLX (Mac). Apple silicon Macs only, text only. ```sh # with uv uv tool install mlx-lm # or pip install mlx-lm ``` 2. Run SmolLM2 1.7B Instruct (4bit, needs ~1 GB, Mac only, from [huggingface.co/Irfanuruchi](https://huggingface.co/Irfanuruchi/SmolLM2-1.7B-Instruct-MLX-4bit/tree/8804903a4d9dc29fd7edf893168988760f4e73a6)) ```sh mlx_lm.chat --model Irfanuruchi/SmolLM2-1.7B-Instruct-MLX-4bit ``` 3. Or start an OpenAI-compatible server at http://localhost:8080/v1 ```sh mlx_lm.server --model Irfanuruchi/SmolLM2-1.7B-Instruct-MLX-4bit ``` Other versions: 8bit (~1.8 GB, Mac only) `mlx_lm.chat --model HuggingFaceTB/SmolLM2-1.7B-Instruct-Q8-mlx` Model repository and documentation: [huggingface.co/HuggingFaceTB](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct) [interactive version](https://whatmodel.sh/model/smollm2-1-7b) · [JSON](https://whatmodel.sh/api?format=json&all=1)