[whatmodel.sh](../../README.md) / Hermes 3 Llama 3.1 405B
# Hermes 3 Llama 3.1 405B
Nous Research · released 2024-08 · 405.85B params · llama3
**score 53.5** ±1.2 · #309 of 732 · from 14 benchmark results across 2 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Hermes 3 Llama 3.1 405B among every model on that task or test, worst to best.
task vs the others rank counts tests
coding ▁▁▁▃▅▇●▇▅▄▃▁ #229 of 469* 27% 1
chat & writing █········█·● #1 of 3* 2.4% 1
coding · #229 of 469 · 27% of its score
Beats [Qwen3.5 4B](qwen3-5-4b.md), [Qwen3-VL 32B Instruct](qwen3-vl-32b-instruct.md), [Solar Open 100B (102B-A12B)](solar-open-100b.md) here, though they rank higher overall. Trails [QwQ-32B-Preview](qwq-32b-preview.md), [Ministral 3 3B Reasoning 2512](ministral-3-3b-reasoning.md), [GPT-4 (0613)](gpt-4-0613.md) here, though they rank lower overall.
test result vs the others in its run difficulty counts why
Aider code editing (133 Exercism … 63.9% ▁·▁▂▄▁▆▄●▆▂▂ beats 32/62 medium · avg 51% 27% sharp · at its level
chat & writing · #1 of 3 · 2.4% of its score
test result vs the others in its run difficulty counts why
IFEval (aggregation unspecified) … 84.9% ▁▂▄▂▂▄▆●█▅▂▁ beats 72/145 easy · avg 77% 2.4% small run
other tests (12)
test result vs the others in its run difficulty counts why
GPQA (subset unspecified) (maker) 44.8% ▂▁▂▂▄▃▆●▅█▃▃ beats 48/106 hard · avg 38% 13% sharp · small run
BIG-Bench Hard (maker) 75.4% ▁▁▁▁▂▄▅▆█●▃▁ beats 129/155 medium · avg 68% 12% sharp · at its level · small run
ARC Challenge (maker) 69.5% ▄▁▁▂▁▄█▇●▂▃▂ beats 117/140 medium · avg 67% 8.6% sharp · at its level · small run
MMLU (maker) 85.0% ▁▁▁▁▁▂▂▅██●▄ beats 186/214 easy · avg 78% 8.3% small run
MMLU-Pro (maker) 54.1% ▁▁·▁▁▁▃●▄▅█▁ beats 53/188 medium · avg 48% 7.4% at its level · small run
OpenBookQA (maker) 48.8% ▅▁▁▁▂▂▃●▃▁▃▁ beats 66/107 hard · avg 49% 5.0% at its level · small run
WinoGrande (maker) 86.3% ▁▁▁▁▂▁▁▅▇█▃● beats 131/131 easy · avg 83% 4.9% at its level · small run
HellaSwag (maker) 90.2% ▁▁▁▂▂▂▄▆█▇▄● beats 135/135 easy · avg 87% 3.6% small run
BoolQ (maker) 88.9% ▁····▁▂█▆▅▆● beats 102/103 easy · avg 88% 3.5% small run
MuSR (maker) 48.3% █······●···█ beats 1/2 medium · avg 51% 2.2% blunt · at its level · small run
PIQA (maker) 86.0% ▁▁▁▂▂▃▅█▅▃●▁ beats 121/123 easy · avg 85% 1.4% blunt · at its level · small run
TruthfulQA MC2 (maker) 65.6% █▇▇▄▂▄▁▆●▄▁▁ beats 64/79 medium · avg 59% 0.4% blunt · at its level · small run
source results share of the score
Nous Research vendor-reported scores fo… 13 72% ▇▇▇▇▇▇▇▇▇
Aider code editing benchmark runs 1 27% ▇▇▇
prior: pull toward the middle 1% ▇
weighted average = 53.5
Licences: Aider code editing benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt))
## How to run it
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Hermes 3 Llama 3.1 405B (Q4_K_M, needs ~243.1 GB, from [huggingface.co/bartowski](https://huggingface.co/bartowski/Hermes-3-Llama-3.1-405B-GGUF/tree/69eba52b31f63bfd9b750b04778b847e22bc50bf))
```sh
llama cli -hf bartowski/Hermes-3-Llama-3.1-405B-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf bartowski/Hermes-3-Llama-3.1-405B-GGUF:Q4_K_M
```
Other versions: Q8_0 (~431.2 GB) `llama cli -hf bartowski/Hermes-3-Llama-3.1-405B-GGUF:Q8_0` · IQ2_XXS (~107.3 GB) `llama cli -hf bartowski/Hermes-3-Llama-3.1-405B-GGUF:IQ2_XXS`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Hermes 3 Llama 3.1 405B (Q4_K_M, needs ~243.1 GB, from [huggingface.co/bartowski](https://huggingface.co/bartowski/Hermes-3-Llama-3.1-405B-GGUF/tree/69eba52b31f63bfd9b750b04778b847e22bc50bf))
```sh
lms get https://huggingface.co/bartowski/Hermes-3-Llama-3.1-405B-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~431.2 GB) `lms get https://huggingface.co/bartowski/Hermes-3-Llama-3.1-405B-GGUF@Q8_0 -y` · IQ2_XXS (~107.3 GB) `lms get https://huggingface.co/bartowski/Hermes-3-Llama-3.1-405B-GGUF@IQ2_XXS -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Hermes 3 Llama 3.1 405B (Q4_0, needs ~229 GB, from [ollama.com](https://ollama.com/library/hermes3/tags))
```sh
ollama run hermes3:405b
```
Model repository and documentation: [huggingface.co/NousResearch](https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-405B)
[interactive version](https://whatmodel.sh/model/hermes-3-llama-3-1-405b) · [JSON](https://whatmodel.sh/api?format=json&all=1)