[whatmodel.sh](../../README.md) / Pythia 2.8B (deduped)
# Pythia 2.8B (deduped)
EleutherAI · released 2023-02 · 2.91B params · apache-2.0
**score 19.8** ±1.2 · #702 of 732 · from 15 benchmark results across 3 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Pythia 2.8B (deduped) among every model on that task or test, worst to best.
other tests (15)
test result vs the others in its run difficulty counts why
ARC Challenge 32.9% ▁·▁··▁▁●▂█▆▃ beats 4/26 easy · avg 82% 30% sharp
WinoGrande 58.2% ▁····▁··●▂█▄ beats 2/26 easy · avg 86% 11% typical
BoolQ 64.1% ▂▃▂●▆▂▅▆█▂▂▂ beats 6/26 easy · avg 88% 10% at its level
SciQ 88.2% ▁·▁·▁·●▅▃█▄▁ beats 3/26 very easy · avg 97% 9.5% blunt
HellaSwag 59.4% ▃···▁●▅█▇▃▁▁ beats 3/26 very easy · avg 91% 9.1% at its level
ARC Easy 59.1% ▁▁·▁▁●▅▄█▅▁▃ beats 5/26 easy · avg 85% 8.0% blunt · at its level
PIQA 74.1% ▁·▁··▁▁●█▅█▂ beats 6/26 easy · avg 87% 5.8% blunt · at its level
LAMBADA (OpenAI version), last-wo… 65.1% ▁··▂▁▁▁●▆█▅▃ beats 6/26 very easy · avg 86% 4.0% blunt · at its level
MedQA (USMLE, 4 options) 27.7% ▁▁··●▁▁··▄▃█ beats 3/17 medium · avg 71% 2.8% sharp · too hard for it
OpenBookQA 23.8% ●·▁▁▃·▁▃▅▇▃▄ beats 0/26 hard · avg 49% 2.4% too hard for it
Social IQA 40.9% ▁···▁●█▁▁▁·▁ beats 5/26 hard · avg 56% 2.4% blunt
MedMCQA 26.3% ●▂··▄▄▂▂·▄██ beats 0/17 medium · avg 63% 2.1% typical
Open PT-LLM Leaderboard average (… 24.7% ●▁▁▁▁▁▁▁▂█▅▁ beats 34/276 medium · avg 72% 1.3% noisy
PubMedQA 64.8% ▂·▂▆·●▂▄▄▂█▂ beats 6/17 medium · avg 75% 0.7% blunt · at its level
TruthfulQA MC2 35.6% █·█·●▄···▄·█ beats 4/9 medium · avg 55% 0.5% blunt · at its level
source results share of the score
Stability AI 11 92% ▇▇▇▇▇▇▇▇▇▇▇
Open Medical-LLM Leaderboard results 3 6% ▇
Open Portuguese LLM Leaderboard 1 1% ▇
prior: pull toward the middle 1% ▇
weighted average = 19.8
Licences: Stability AI ([Apache-2.0](../../licenses/stablelm-original-eval-json-Apache-2.0.txt)) · Open Medical-LLM Leaderboard results ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt))
## How to run it
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Pythia 2.8B (deduped) (Q4_K_M, needs ~1.8 GB, from [huggingface.co/mradermacher](https://huggingface.co/mradermacher/pythia-2.8b-deduped-GGUF/tree/c77004d5ea7949bf08cd59f9f81610fb030333a6))
```sh
llama cli -hf mradermacher/pythia-2.8b-deduped-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf mradermacher/pythia-2.8b-deduped-GGUF:Q4_K_M
```
Other versions: Q8_0 (~3 GB) `llama cli -hf mradermacher/pythia-2.8b-deduped-GGUF:Q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Pythia 2.8B (deduped) (Q4_K_M, needs ~1.8 GB, from [huggingface.co/mradermacher](https://huggingface.co/mradermacher/pythia-2.8b-deduped-GGUF/tree/c77004d5ea7949bf08cd59f9f81610fb030333a6))
```sh
lms get https://huggingface.co/mradermacher/pythia-2.8b-deduped-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~3 GB) `lms get https://huggingface.co/mradermacher/pythia-2.8b-deduped-GGUF@Q8_0 -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Pythia 2.8B (deduped) (Q4_K_M, needs ~1.8 GB, from [huggingface.co/mradermacher](https://huggingface.co/mradermacher/pythia-2.8b-deduped-GGUF/tree/c77004d5ea7949bf08cd59f9f81610fb030333a6))
```sh
ollama run hf.co/mradermacher/pythia-2.8b-deduped-GGUF:Q4_K_M
```
Other versions: Q8_0 (~3 GB) `ollama run hf.co/mradermacher/pythia-2.8b-deduped-GGUF:Q8_0`
### Other ways to run it
1. Run it as published (fp16, needs ~5.68 GB, from [huggingface.co/EleutherAI](https://huggingface.co/EleutherAI/pythia-2.8b-deduped/tree/7d977fed8c4ce9649816af8cd5fe36a639cbe5b2))
```sh
uv run --with torch --with 'transformers<5' --with accelerate python -c "from transformers import pipeline; print(pipeline('text-generation', model='EleutherAI/pythia-2.8b-deduped', torch_dtype='auto', device_map='auto')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"
```
Model repository and documentation: [huggingface.co/EleutherAI](https://huggingface.co/EleutherAI/pythia-2.8b-deduped)
[interactive version](https://whatmodel.sh/model/pythia-2-8b-deduped) · [JSON](https://whatmodel.sh/api?format=json&all=1)