[whatmodel.sh](../../README.md) / BLOOM 560M
# BLOOM 560M
BigScience · released 2022-07 · 0.559B params · bigscience-bloom-rail-1.0
**score -0.1** ±1.2 · #725 of 732 · from 38 benchmark results across 7 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks BLOOM 560M among every model on that task or test, worst to best.
task vs the others rank counts tests
coding ●··█·█·██··█ #6 of 6* 1.6% 1
math ●▁▃▅█▆▄█▄▂▁▁ #426 of 431* 0.7% 1
coding · #6 of 6 · 1.6% of its score
test result vs the others in its run difficulty counts why
HumanEval (maker) 0.8% ●···▄·▄·█··▄ beats 0/5 easy · avg 79% 1.6% sharp · too hard for it
math · #426 of 431 · 0.7% of its score
Beats [Flan-T5 Base](flan-t5-base.md), [Flan-T5 Small](flan-t5-small.md) here, though they rank higher overall. Trails [GPT-Neo 125M](gpt-neo-125m.md), [OPT-125M](opt-125m.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
GSM8K 0.5% ●▄█▄···▄·▄·▄ beats 0/7 easy · avg 65% 0.7% sharp · too hard for it
other tests (22)
test result vs the others in its run difficulty counts why
ARC Easy 47.6% ▃·▅▃●▅·██▃██ beats 4/19 easy · avg 87% 20% blunt · at its level · several runs
PIQA 63.7% ▂·▄●·▄▂▆▂▄█▄ beats 3/19 easy · avg 87% 13% blunt · at its level · several runs
SciQ 80.4% ▂··▂▃●▂▆▃▃█▂ beats 4/19 very easy · avg 98% 10% blunt · at its level
LAMBADA (OpenAI version), last-wo… 34.1% ▃··●▃▃▅▅▅███ beats 2/19 very easy · avg 87% 10% blunt · at its level · several runs
WinoGrande 50.4% ●····▄▆▂▄▆▆█ beats 1/19 easy · avg 88% 6.9% too hard for it · several runs
Story Cloze Test 61.9% ▂●▄▂·▂·▆·▂█▂ beats 1/14 easy · avg 89% 5.4% blunt · several runs
COPA (maker) 55.0% ▃··●▅▃▃▅·▃▃▃ beats 3/12 very easy · avg 95% 5.1% several runs
ARC Challenge 22.1% ●··▁·▁▁·▁▂▁▂ beats 5/19 easy · avg 81% 4.3% sharp · too hard for it · several runs
CoQA 22.7% ●····▄▄·▄▄·█ beats 0/6 very easy · avg 92% 4.3% at its level
HellaSwag 31.6% ●·▄▄·▄·▄··▄█ beats 0/7 easy · avg 81% 3.8% typical
TriviaQA 5.7% ▁··●█▃▃▁▄▅▄█ beats 1/28 medium · avg 45% 3.6% too hard for it · several runs
Winograd Schema Challenge (WSC273) 66.7% ●▄·▄·▄··█··█ beats 0/7 very easy · avg 93% 3.5% blunt · at its level
Natural Questions (closed book) 1.6% ▁·●·▁▂▁▂·▅▂█ beats 1/28 medium · avg 48% 1.5% too hard for it
RACE-high 30.2% ●···█··▄▄··█ beats 0/6 medium · avg 51% 1.4% blunt · too hard for it
BoolQ 55.1% ●··▄·█▄··█·▄ beats 0/7 easy · avg 87% 1.4% too hard for it
DROP 6.2% ●▄···█·▄·▄·▄ beats 0/6 hard · avg 34% 1.2% blunt · too hard for it
TruthfulQA MC2 42.4% ▅▃●▆▁▂▅···▁▁ beats 14/31 medium · avg 57% 0.7% blunt · at its level · several runs
OpenBookQA 17.2% ●········▂▂▂ beats 4/7 hard · avg 40% 0.5% too hard for it
Open PT-LLM Leaderboard average (… 25.4% ●▁▁▁▁▁▁▁▂█▅▁ beats 28/276 medium · avg 72% 0.3% too hard for it · noisy
WebQuestions (closed book) 0.8% █···●··▄▄▄·▄ beats 2/6 very hard · avg 3.5% 0.1% blunt · too hard for it
HaluEval QA hallucination detecti… 50.0% ●▁▁·▁▁·▁▁▁▁▁ beats 9/27 hard · avg 66% 0.0% too hard for it · noisy
LogiQA (harness) 22.6% ●··········▁ beats 13/14 very hard · avg 25% 0.0% too hard for it · noisy
source results share of the score
EleutherAI lm-evaluation-harness result… 18 40% ▇▇▇▇▇
Pythia: A Suite for Analyzing Large Lan… 6 33% ▇▇▇▇
GPTQ: Accurate Post-Training Quantizati… 5 21% ▇▇▇
Hallucinations Leaderboard results 4 2% ▇
BLOOM: A 176B-Parameter Open-Access Mul… 1 2% ▇
Crosslingual Generalization through Mul… 3 1% ▇
Open Portuguese LLM Leaderboard 1 0% ▇
prior: pull toward the middle 1% ▇
weighted average = -0.1
Licences: EleutherAI lm-evaluation-harness results folder ([MIT](../../licenses/lm-eval-harness-results-2023-05-MIT.txt)) · Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling ([CC BY-SA 4.0](../../licenses/paper-2304.01373v2-CC-BY-SA-4.0.txt)) · GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers ([CC BY 4.0](../../licenses/paper-2210.17323v2-gptq-CC-BY-4-0.txt)) · Hallucinations Leaderboard results ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · BLOOM: A 176B-Parameter Open-Access Multilingual Language Model ([CC BY 4.0](../../licenses/paper-2211.05100v4-CC-BY-4.0.txt)) · Crosslingual Generalization through Multitask Finetuning ([CC BY 4.0](../../licenses/acl-2023-acl-long-891-CC-BY-4-0.txt)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt))
## How to run it
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run BLOOM 560M (Q4_K_M, needs ~0.6 GB, from [huggingface.co/afrideva](https://huggingface.co/afrideva/bloom-560m-GGUF/tree/fe8ad7e97957783ea24f6960f73e28e709aa6e96))
```sh
llama cli -hf afrideva/bloom-560m-GGUF:q4_k_m
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf afrideva/bloom-560m-GGUF:q4_k_m
```
Other versions: Q8_0 (~0.9 GB) `llama cli -hf afrideva/bloom-560m-GGUF:q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run BLOOM 560M (Q4_K_M, needs ~0.6 GB, from [huggingface.co/afrideva](https://huggingface.co/afrideva/bloom-560m-GGUF/tree/fe8ad7e97957783ea24f6960f73e28e709aa6e96))
```sh
lms get https://huggingface.co/afrideva/bloom-560m-GGUF@q4_k_m -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~0.9 GB) `lms get https://huggingface.co/afrideva/bloom-560m-GGUF@q8_0 -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run BLOOM 560M (Q4_K_M, needs ~0.6 GB, from [huggingface.co/afrideva](https://huggingface.co/afrideva/bloom-560m-GGUF/tree/fe8ad7e97957783ea24f6960f73e28e709aa6e96))
```sh
ollama run hf.co/afrideva/bloom-560m-GGUF:q4_k_m
```
Other versions: Q8_0 (~0.9 GB) `ollama run hf.co/afrideva/bloom-560m-GGUF:q8_0`
### Other ways to run it
1. Run it as published (fp16, needs ~1.12 GB, from [huggingface.co/bigscience](https://huggingface.co/bigscience/bloom-560m/tree/ac2ae5fab2ce3f9f40dc79b5ca9f637430d24971))
```sh
uv run --with torch --with 'transformers<5' python -c "from transformers import pipeline; print(pipeline('text-generation', model='bigscience/bloom-560m')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"
```
Model repository and documentation: [huggingface.co/bigscience](https://huggingface.co/bigscience/bloom-560m)
[interactive version](https://whatmodel.sh/model/bloom-560m) · [JSON](https://whatmodel.sh/api?format=json&all=1)