[whatmodel.sh](../../README.md) / Pythia 1B
# Pythia 1B
EleutherAI · released 2023-02 · 1.01B params · apache-2.0
**score 12.2** ±1.3 · #714 of 732 · from 18 benchmark results across 4 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Pythia 1B among every model on that task or test, worst to best.
other tests (13)
test result vs the others in its run difficulty counts why
SciQ (maker) 84.0% ▂··▂▃▂▂●▃▃█▂ beats 7/19 very easy · avg 98% 19% blunt · several runs
ARC Easy (maker) 56.9% ▃·▅▃▃▅·●█▃██ beats 8/19 easy · avg 87% 17% blunt · at its level · several runs
WinoGrande (maker) 53.7% ▄····▄●▂▄▆▆█ beats 5/19 easy · avg 87% 14% several runs
PIQA (maker) 70.7% ▂·▄▄·▄▂●▂▄█▄ beats 8/19 easy · avg 87% 11% blunt · at its level · several runs
ARC Challenge (maker) 24.4% ●··▁·▁▁·▁▂▁▂ beats 6/19 medium · avg 71% 10% sharp · too hard for it · several runs
HellaSwag 44.7% ●·····▂▂·▂·█ beats 0/8 very easy · avg 91% 8.9% at its level
LAMBADA (OpenAI version), last-wo… 56.2% ▃··▅▃▃▅▅●███ beats 9/19 very easy · avg 87% 6.2% blunt · at its level
OpenBookQA 37.8% ●·····▄▄▄██▄ beats 0/8 easy · avg 81% 4.2% typical
BoolQ 61.8% ●····▃··█▃·▃ beats 2/8 easy · avg 88% 3.9% typical
MedQA (USMLE, 4 options) 25.6% ▁●··▄▁▁··▄▃█ beats 1/17 medium · avg 71% 1.5% sharp · too hard for it
MedMCQA 31.6% ▂▂··▄●▂▂·▄██ beats 5/17 medium · avg 63% 1.5% too hard for it
Open PT-LLM Leaderboard average (… 25.2% ●▁▁▁▁▁▁▁▂█▅▁ beats 9/276 medium · avg 72% 0.9% too hard for it · noisy
PubMedQA 58.2% ▂·●▆·▄▂▄▄▂█▂ beats 1/17 medium · avg 75% 0.8% blunt · at its level
source results share of the score
Pythia: A Suite for Analyzing Large Lan… 6 48% ▇▇▇▇▇▇
OLMo: Accelerating the Science of Langu… 8 47% ▇▇▇▇▇▇
Open Medical-LLM Leaderboard results 3 4% ▇
Open Portuguese LLM Leaderboard 1 1% ▇
prior: pull toward the middle 1% ▇
weighted average = 12.2
Licences: Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling ([CC BY-SA 4.0](../../licenses/paper-2304.01373v2-CC-BY-SA-4.0.txt)) · OLMo: Accelerating the Science of Language Models ([CC BY 4.0](../../licenses/paper-2402.00838v4-CC-BY-4.0.txt)) · Open Medical-LLM Leaderboard results ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt))
## How to run it
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Pythia 1B (Q4_K_M, needs ~0.7 GB, from [huggingface.co/mradermacher](https://huggingface.co/mradermacher/pythia-1b-GGUF/tree/56d6c599008603650e8ee607d49b885bfd766b72))
```sh
llama cli -hf mradermacher/pythia-1b-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf mradermacher/pythia-1b-GGUF:Q4_K_M
```
Other versions: Q8_0 (~1.1 GB) `llama cli -hf mradermacher/pythia-1b-GGUF:Q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Pythia 1B (Q4_K_M, needs ~0.7 GB, from [huggingface.co/mradermacher](https://huggingface.co/mradermacher/pythia-1b-GGUF/tree/56d6c599008603650e8ee607d49b885bfd766b72))
```sh
lms get https://huggingface.co/mradermacher/pythia-1b-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~1.1 GB) `lms get https://huggingface.co/mradermacher/pythia-1b-GGUF@Q8_0 -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Pythia 1B (Q4_K_M, needs ~0.7 GB, from [huggingface.co/mradermacher](https://huggingface.co/mradermacher/pythia-1b-GGUF/tree/56d6c599008603650e8ee607d49b885bfd766b72))
```sh
ollama run hf.co/mradermacher/pythia-1b-GGUF:Q4_K_M
```
Other versions: Q8_0 (~1.1 GB) `ollama run hf.co/mradermacher/pythia-1b-GGUF:Q8_0`
### Other ways to run it
1. Run it as published (fp16, needs ~2.09 GB, from [huggingface.co/EleutherAI](https://huggingface.co/EleutherAI/pythia-1b/tree/f73d7dcc545c8bd326d8559c8ef84ffe92fea6b2))
```sh
uv run --with torch --with 'transformers<5' python -c "from transformers import pipeline; print(pipeline('text-generation', model='EleutherAI/pythia-1b')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"
```
Model repository and documentation: [huggingface.co/EleutherAI](https://huggingface.co/EleutherAI/pythia-1b)
[interactive version](https://whatmodel.sh/model/pythia-1b) · [JSON](https://whatmodel.sh/api?format=json&all=1)