[whatmodel.sh](../../README.md) / OpenLLaMA 3B v2
# OpenLLaMA 3B v2
OpenLM Research · released 2023-07 · apache-2.0
**score 24.6** ±1.1 · #690 of 732 · from 12 benchmark results across 2 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks OpenLLaMA 3B v2 among every model on that task or test, worst to best.
other tests (12)
test result vs the others in its run difficulty counts why
ARC Challenge 36.1% ▁·▁··▁▁▇●█▆▃ beats 10/26 easy · avg 82% 40% sharp
WinoGrande 62.9% ▁····▁··▁●█▄ beats 7/26 easy · avg 86% 12% at its level
BoolQ 65.7% ▂▃▂▅●▂▅▆█▂▂▂ beats 9/26 easy · avg 88% 9.5% at its level
HellaSwag 70.0% ▃···▁▇▅●▇▃▁▁ beats 12/26 very easy · avg 91% 7.9% at its level
ARC Easy 63.5% ▁▁·▁▁▄●▄█▅▁▃ beats 10/26 easy · avg 85% 7.1% blunt · at its level
SciQ 92.4% ▁·▁·▁·▃▅●█▄▁ beats 12/26 very easy · avg 97% 6.9% blunt
PIQA 76.7% ▁·▁··▁▁▃█●█▂ beats 14/26 easy · avg 87% 5.1% blunt · at its level
LAMBADA (OpenAI version), last-wo… 66.7% ▁··▂▁▁▁▂●█▅▃ beats 8/26 very easy · avg 86% 3.3% blunt · at its level
OpenBookQA 26.0% █·▁●▃·▁▃▅▇▃▄ beats 7/26 hard · avg 49% 3.1% too hard for it
Social IQA 41.2% ▁···▁▃●▁▁▁·▁ beats 7/26 hard · avg 56% 2.5% blunt
Open PT-LLM Leaderboard average (… 30.6% ●▁▁▁▁▁▁▁▂█▅▁ beats 33/276 medium · avg 72% 1.6% noisy
TruthfulQA MC2 34.6% █·●·█▄···▄·█ beats 3/9 medium · avg 55% 0.4% blunt · at its level
source results share of the score
Stability AI 11 98% ▇▇▇▇▇▇▇▇▇▇▇▇
Open Portuguese LLM Leaderboard 1 2% ▇
prior: pull toward the middle 1% ▇
weighted average = 24.6
Licences: Stability AI ([Apache-2.0](../../licenses/stablelm-original-eval-json-Apache-2.0.txt)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt))
## How to run it
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run OpenLLaMA 3B v2 (Q4_K_M, needs ~2.6 GB, from [huggingface.co/lstgst](https://huggingface.co/lstgst/open_llama_3b_v2-Q4_K_M-GGUF/tree/8728b2275ba163d674202dc1080f05208edc0100))
```sh
llama cli -hf lstgst/open_llama_3b_v2-Q4_K_M-GGUF:q4_k_m
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf lstgst/open_llama_3b_v2-Q4_K_M-GGUF:q4_k_m
```
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run OpenLLaMA 3B v2 (Q4_K_M, needs ~2.6 GB, from [huggingface.co/lstgst](https://huggingface.co/lstgst/open_llama_3b_v2-Q4_K_M-GGUF/tree/8728b2275ba163d674202dc1080f05208edc0100))
```sh
lms get https://huggingface.co/lstgst/open_llama_3b_v2-Q4_K_M-GGUF@q4_k_m -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run OpenLLaMA 3B v2 (Q4_K_M, needs ~2.6 GB, from [huggingface.co/lstgst](https://huggingface.co/lstgst/open_llama_3b_v2-Q4_K_M-GGUF/tree/8728b2275ba163d674202dc1080f05208edc0100))
```sh
ollama run hf.co/lstgst/open_llama_3b_v2-Q4_K_M-GGUF:q4_k_m
```
### Other ways to run it
1. Run it as published (fp16, needs ~6.85 GB, from [huggingface.co/openlm-research](https://huggingface.co/openlm-research/open_llama_3b_v2/tree/bce5d60d3b0c68318862270ec4e794d83308d80a))
```sh
uv run --with torch --with 'transformers<5' --with accelerate python -c "from transformers import pipeline; print(pipeline('text-generation', model='openlm-research/open_llama_3b_v2', torch_dtype='auto', device_map='auto')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"
```
Model repository and documentation: [huggingface.co/openlm-research](https://huggingface.co/openlm-research/open_llama_3b_v2)
[interactive version](https://whatmodel.sh/model/open-llama-3b-v2) · [JSON](https://whatmodel.sh/api?format=json&all=1)