[whatmodel.sh](../../README.md) / InternLM2.5 7B Chat 1M
# InternLM2.5 7B Chat 1M
Shanghai AI Laboratory (InternLM) · released 2024-06 · 7.74B params · other
**score 47.5** ±4.0 · #419 of 732 · from 3 benchmark results across 2 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks InternLM2.5 7B Chat 1M among every model on that task or test, worst to best.
other tests (3)
test result vs the others in its run difficulty counts why
Open PT-LLM Leaderboard average (… 69.3% ▃▁▁▁▁▁▁▁▂●▅▁ beats 160/276 medium · avg 72% 40% at its level · noisy
RULER 64k 75.5% ▁▁·▁·●▃▄█▂·▁ beats 3/22 easy · avg 82% 39% typical
RULER 128k 68.9% ▄··▂▄▂▆●█▄▄▂ beats 10/21 easy · avg 67% 13% at its level · noisy
source results share of the score
NVIDIA RULER repository README results … 2 52% ▇▇▇▇▇▇
Open Portuguese LLM Leaderboard 1 40% ▇▇▇▇▇
prior: pull toward the middle 7% ▇
weighted average = 47.5
Licences: NVIDIA RULER repository README results table ([Apache-2.0](https://github.com/NVIDIA/RULER/blob/c3f5e3b4f87f97e048793bb510a3a6b19a46bf3a/LICENSE)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt))
## How to run it
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run InternLM2.5 7B Chat 1M (Q4_K_M, needs ~4.7 GB, from [huggingface.co/bartowski](https://huggingface.co/bartowski/internlm2_5-7b-chat-1m-GGUF/tree/204e800e2cf0c3d81b10b634eddf8786a0bb9f94))
```sh
llama cli -hf bartowski/internlm2_5-7b-chat-1m-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf bartowski/internlm2_5-7b-chat-1m-GGUF:Q4_K_M
```
Other versions: Q8_0 (~8.2 GB) `llama cli -hf bartowski/internlm2_5-7b-chat-1m-GGUF:Q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run InternLM2.5 7B Chat 1M (Q4_K_M, needs ~4.7 GB, from [huggingface.co/bartowski](https://huggingface.co/bartowski/internlm2_5-7b-chat-1m-GGUF/tree/204e800e2cf0c3d81b10b634eddf8786a0bb9f94))
```sh
lms get https://huggingface.co/bartowski/internlm2_5-7b-chat-1m-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~8.2 GB) `lms get https://huggingface.co/bartowski/internlm2_5-7b-chat-1m-GGUF@Q8_0 -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run InternLM2.5 7B Chat 1M (Q4_K_M, needs ~4.7 GB, from [huggingface.co/bartowski](https://huggingface.co/bartowski/internlm2_5-7b-chat-1m-GGUF/tree/204e800e2cf0c3d81b10b634eddf8786a0bb9f94))
```sh
ollama run hf.co/bartowski/internlm2_5-7b-chat-1m-GGUF:Q4_K_M
```
Other versions: Q8_0 (~8.2 GB) `ollama run hf.co/bartowski/internlm2_5-7b-chat-1m-GGUF:Q8_0`
### Other ways to run it
1. Run it as published (BF16, needs ~15.48 GB, from [huggingface.co/internlm](https://huggingface.co/internlm/internlm2_5-7b-chat-1m/tree/dfe8688420775b14e4400221c369c8585752aac2))
```sh
uv run --with torch --with 'transformers<5' --with accelerate python -c "from transformers import pipeline; print(pipeline('text-generation', model='internlm/internlm2_5-7b-chat-1m', torch_dtype='auto', device_map='auto', trust_remote_code=True)('Once upon a time', max_new_tokens=40)[0]['generated_text'])"
```
Model repository and documentation: [huggingface.co/internlm](https://huggingface.co/internlm/internlm2_5-7b-chat-1m)
[interactive version](https://whatmodel.sh/model/internlm2-5-7b-chat-1m) · [JSON](https://whatmodel.sh/api?format=json&all=1)