[whatmodel.sh](../../README.md) / Llama 3 70B (base) # Llama 3 70B (base) Meta · released 2024-04 · 70.55B params · llama3 **score 47.4** ±2.1 · #421 of 732 · from 4 benchmark results across 4 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at _{● marks Llama 3 70B (base) among every model on that task or test, worst to best.}

other tests (4)

test                                result  vs the others in its run    difficulty        counts  why
BigCodeBench (Complete)              43.3%  ▂▁▁▄▂▅▇●▆▂▂▂ beats 36/58    medium · avg 48%     52%  at its level
Polish EQ-Bench v2 (SpeakLeash)      46.1%  ▁··▁▁▂▂●▅▆█▆ beats 18/70    medium · avg 62%     29%  at its level
Open PT-LLM Leaderboard average (…   77.9%  ▃▁▁▁▁▁▁▁▂█●▁ beats 242/276  medium · avg 72%     11%  at its level · noisy
EuroEval Life in the UK (English …   91.8%  ▁▁▁▁▁▁▁▂▄█●▃ beats 123/146  easy · avg 82%      6.4%  at its level · noisy

## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md)

source                                  results  share of the score
BigCodeBench paper                            1   52% ▇▇▇▇▇▇
SpeakLeash Polish EQ-Bench leaderboard        1   29% ▇▇▇
Open Portuguese LLM Leaderboard               1   11% ▇
EuroEval leaderboards                         1    6% ▇
prior: pull toward the middle                      2% ▇
weighted average                                 = 47.4

_{Licences: BigCodeBench paper ([CC BY 4.0](http://creativecommons.org/licenses/by/4.0/)) · SpeakLeash Polish EQ-Bench leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE))} ## How to run it ### LM Studio (recommended) 1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download) 2. Run Llama 3 70B (base) (4bit, needs ~39.7 GB, Mac only, from [huggingface.co/mlx-community](https://huggingface.co/mlx-community/Meta-Llama-3-70B-4bit/tree/fb4021fd52b19efd694e09c7c4aff3b5cbe5081b)) ```sh lms get https://huggingface.co/mlx-community/Meta-Llama-3-70B-4bit -y ``` 3. Then run lms ls to see its name, and lms chat followed by that name ### MLX (Mac) 1. Install MLX (Mac). Apple silicon Macs only, text only. ```sh # with uv uv tool install mlx-lm # or pip install mlx-lm ``` 2. Run Llama 3 70B (base) (4bit, needs ~39.7 GB, Mac only, from [huggingface.co/mlx-community](https://huggingface.co/mlx-community/Meta-Llama-3-70B-4bit/tree/fb4021fd52b19efd694e09c7c4aff3b5cbe5081b)) ```sh mlx_lm.chat --model mlx-community/Meta-Llama-3-70B-4bit ``` 3. Or start an OpenAI-compatible server at http://localhost:8080/v1 ```sh mlx_lm.server --model mlx-community/Meta-Llama-3-70B-4bit ``` ### Other ways to run it 1. Run it as published (BF16, needs ~141.11 GB, from [huggingface.co/meta-llama](https://huggingface.co/meta-llama/Meta-Llama-3-70B/tree/c82494877ce7f6d7d317c56ec081328e382c72fe)) ```sh uv run --with torch --with 'transformers<5' --with accelerate python -c "from transformers import pipeline; print(pipeline('text-generation', model='meta-llama/Meta-Llama-3-70B', torch_dtype='auto', device_map='auto')('Once upon a time', max_new_tokens=40)[0]['generated_text'])" ``` Model repository and documentation: [huggingface.co/meta-llama](https://huggingface.co/meta-llama/Meta-Llama-3-70B) [interactive version](https://whatmodel.sh/model/llama-3-70b-base) · [JSON](https://whatmodel.sh/api?format=json&all=1)