[whatmodel.sh](../../README.md) / Falcon 7B (base)
# Falcon 7B (base)
TII · released 2023-04 · 7.22B params · apache-2.0
**score 30.6** ±0.7 · #658 of 732 · from 37 benchmark results across 10 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Falcon 7B (base) among every model on that task or test, worst to best.
other tests (21)
test result vs the others in its run difficulty counts why
ARC Challenge 43.7% ▁·▁··▁▁▇▂█●▃ beats 19/26 easy · avg 82% 35% sharp · at its level · several runs
WinoGrande 67.2% ▁····▁··▁▂●▄ beats 17/26 easy · avg 86% 12% at its level · several runs
BoolQ 73.5% ▂▃▂▅▆▂▅●█▂▂▂ beats 18/26 easy · avg 88% 7.9% at its level · several runs
HellaSwag 76.3% ▃···▁▇▅█●▃▁▁ beats 22/26 very easy · avg 91% 7.4% at its level · several runs
OpenBookQA 30.6% █·▁▁▃·▁▃▅●▃▄ beats 17/26 easy · avg 81% 6.5% at its level · several runs
ARC Easy 70.8% ▁▁·▁▁▄▅▄█●▁▃ beats 20/26 easy · avg 85% 5.8% blunt · at its level · several runs
PIQA 79.5% ▁·▁··▁▁▃█▅█● beats 25/26 easy · avg 87% 4.0% blunt · at its level · several runs
Natural Questions (closed book) 22.1% ▁·▁·▁▂▁▂·▅●█ beats 17/28 medium · avg 48% 3.8% typical
SciQ 94.0% ▁·▁·▁·▃▅▃●▄▁ beats 21/26 very easy · avg 97% 3.7% blunt · several runs
ImageNetVC visual commonsense, te… 55.7% ▂·▄·▂█▄▄█●▂▆ beats 17/21 easy · avg 74% 3.7% blunt · at its level
Social IQA 42.1% ▁···▁▃●▁▁▁·▁ beats 13/26 medium · avg 60% 2.3% blunt · several runs
Factual knowledge recall, Premium… 23.6% ▅····▃▅·●█·█ beats 7/13 medium · avg 52% 2.1% typical
Factual knowledge recall, Premium… 30.4% ▃▅▃··●▃·█▃▃▅ beats 5/13 medium · avg 61% 1.4% at its level
LAMBADA (OpenAI version), last-wo… 74.6% ▁··▂▁▁▁▂▆█▅● beats 24/26 very easy · avg 86% 1.2% blunt · at its level
Open PT-LLM Leaderboard average (… 36.9% ▃▁▁▁●▁▁▁▂█▅▁ beats 46/276 medium · avg 72% 0.9% noisy
TriviaQA 58.7% ▁··▁█▃▃▁▄▅●█ beats 22/28 easy · avg 73% 0.6% at its level · noisy
This is not a Dataset (negation),… 50.3% ●▁····▃▁·▁·▁ beats 6/14 hard · avg 66% 0.5% sharp · too hard for it · noisy
COPA 53.8% ●·▃·▃··█▃··▃ beats 1/8 easy · avg 92% 0.3% at its level · noisy
HaluEval QA hallucination detecti… 46.7% ●▁▁·▁▁·▁▁▁▁▁ beats 2/27 hard · avg 66% 0.2% too hard for it · noisy
TruthfulQA MC2 34.3% ●▃█▆▁▂▅···▁▁ beats 2/31 medium · avg 57% 0.1% blunt · at its level
IndoMMLU (zero-shot) 25.1% ▅●▅··▃···▃▃█ beats 2/10 hard · avg 37% 0.1% blunt · too hard for it · noisy
source results share of the score
Stability AI 10 47% ▇▇▇▇▇▇
OLMo: Accelerating the Science of Langu… 8 21% ▇▇▇
Textbooks Are All You Need II: phi-1.5 … 8 18% ▇▇
Hallucinations Leaderboard results 4 5% ▇
ImageNetVC: Zero- and Few-Shot Visual C… 1 4% ▇
Towards a Holistic Evaluation of LLMs o… 2 4% ▇
Open Portuguese LLM Leaderboard 1 1% ▇
This is not a Dataset: A Large Negation… 1 0% ▇
From Test-Taking to Test-Making: Examin… 1 0% ▇
Large Language Models Only Pass Primary… 1 0% ▇
prior: pull toward the middle 0% ▇
weighted average = 30.6
Licences: Stability AI ([Apache-2.0](../../licenses/stablelm-original-eval-json-Apache-2.0.txt)) · OLMo: Accelerating the Science of Language Models ([CC BY 4.0](../../licenses/paper-2402.00838v4-CC-BY-4.0.txt)) · Textbooks Are All You Need II: phi-1.5 technical report ([CC BY 4.0](../../licenses/paper-2309.05463v1-CC-BY-4.0.txt)) · Hallucinations Leaderboard results ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) · ImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories ([CC BY 4.0](../../licenses/acl-2023-findings-emnlp-133-CC-BY-4-0.txt)) · Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-531-CC-BY-4-0.txt)) · From Test-Taking to Test-Making: Examining LLM Authoring of Commonsense Assessment Items ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)) · Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU ([CC BY 4.0](../../licenses/acl-2023-emnlp-main-760-CC-BY-4-0.txt))
## How to run it
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Falcon 7B (base) (Q4_K_M, needs ~5 GB, from [huggingface.co/mradermacher](https://huggingface.co/mradermacher/falcon-7b-GGUF/tree/8fd1679ed38f3b54c80dad18a97708c559ec7517))
```sh
llama cli -hf mradermacher/falcon-7b-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf mradermacher/falcon-7b-GGUF:Q4_K_M
```
Other versions: Q8_0 (~7.7 GB) `llama cli -hf mradermacher/falcon-7b-GGUF:Q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Falcon 7B (base) (Q4_K_M, needs ~5 GB, from [huggingface.co/mradermacher](https://huggingface.co/mradermacher/falcon-7b-GGUF/tree/8fd1679ed38f3b54c80dad18a97708c559ec7517))
```sh
lms get https://huggingface.co/mradermacher/falcon-7b-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~7.7 GB) `lms get https://huggingface.co/mradermacher/falcon-7b-GGUF@Q8_0 -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Falcon 7B (base) (Q4_0, needs ~4.2 GB, from [ollama.com](https://ollama.com/library/falcon/tags))
```sh
ollama run falcon:7b-text
```
### Other ways to run it
1. Run it as published (BF16, needs ~14.43 GB, from [huggingface.co/tiiuae](https://huggingface.co/tiiuae/falcon-7b/tree/ec89142b67d748a1865ea4451372db8313ada0d8))
```sh
uv run --with torch --with 'transformers<5' --with accelerate python -c "from transformers import pipeline; print(pipeline('text-generation', model='tiiuae/falcon-7b', torch_dtype='auto', device_map='auto')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"
```
Model repository and documentation: [huggingface.co/tiiuae](https://huggingface.co/tiiuae/falcon-7b)
[interactive version](https://whatmodel.sh/model/falcon-7b) · [JSON](https://whatmodel.sh/api?format=json&all=1)