[whatmodel.sh](../../README.md) / Command R Plus (08-2024) # Command R Plus (08-2024) Cohere · released 2024-08 · 104B params · cc-by-nc-4.0 **score 48.4** ±0.9 · #403 of 732 · from 17 benchmark results across 7 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks Command R Plus (08-2024) among every model on that task or test, worst to best. 
task            vs the others          rank  counts  tests
agent tasks     ▁▁▃●█▄▅▅▄▄▂▁   #276 of 313*    6.2%      1
coding          ▁▁▁▃▅●█▇▅▄▃▁    #336 of 469     37%      6
chat & writing  ▁▁▁▃▆▄●█▆▇▇▃   #257 of 402*    8.5%      1
agent tasks · #276 of 313 · 6.2% of its score Beats [Magistral Small 1.0 (2506)](magistral-small-2506.md), [Llama 4 Scout (109B-A17B)](llama-4-scout.md), [Phi-4 14B](phi-4.md) here, though they rank higher overall. Trails [Granite 4.0 H 1B](granite-4-0-h-1b.md), [Granite 4.0 1B](granite-4-0-1b.md), [SmolLM3 3B](smollm3-3b.md) here, though they rank lower overall. 
test                        result  vs the others in its run  difficulty      counts  why
SWE-bench Verified (maker)   17.0%  ▃···▃·●·▃█·▃ beats 2/7    hard · avg 17%    6.2%  sharp
coding · #336 of 469 · 37% of its score Beats [Qwen3-VL 8B Instruct](qwen3-vl-8b-instruct.md), [Qwen3-VL 4B Instruct](qwen3-vl-4b-instruct.md), [LFM2.5 2.6B](lfm2-5-2-6b.md) here, though they rank higher overall. Trails [Phi-3-mini-128k-instruct (Apr 2024)](phi-3-mini-128k-instruct.md), [DeepSeek LLM 67B Chat](deepseek-llm-67b-chat.md), [WizardLM 70B V1.0](wizardlm-70b-v1-0.md) here, though they rank lower overall. 
test                                result  vs the others in its run  difficulty            counts  why
Aider code editing (133 Exercism …   38.3%  ▁·▁▂●▁▆▄█▆▂▂ beats 10/62  medium · avg 51%         16%  sharp · at its level
SWE-bench Verified (maker)           17.0%  ▃···▃·●·▃█·▃ beats 2/7    hard · avg 17%          6.2%  sharp
MBPP+ (maker)                        78.8%  ██·●█···████ beats 2/7    easy · avg 78%          4.6%  typical
LiveCodeBench v5 (Oct 2024–Feb 20…   14.4%  ▄▄·●·····█▄█ beats 2/7    hard · avg 18%          4.0%  typical
BIRD-SQL (dev) (maker)               52.7%  ▄··▄·▄·●··▄█ beats 3/7    medium · avg 47%        3.0%  at its level
Aider polyglot (225 exercises, tw…    2.2%  ●▄··█·█····▄ beats 1/7    very hard · avg 5.5%    2.9%  sharp · too hard for it · noisy
chat & writing · #257 of 402 · 8.5% of its score Beats [Mistral Small 4 (119B-A6.5B)](mistral-small-4.md), [Qwen2.5 Coder 32B Instruct](qwen2-5-coder-32b.md), [Ministral 3 14B Reasoning 2512](ministral-3-14b-reasoning.md) here, though they rank higher overall. Trails [Flan-T5 XXL](flan-t5-xxl.md), [Granite 4.0 H 1B](granite-4-0-h-1b.md), [Granite 4.0 1B](granite-4-0-1b.md) here, though they rank lower overall. 
test            result  vs the others in its run    difficulty  counts  why
LMArena (text)  1276.3  ▁▁▁▂▃▃●▆▆▇█▃ beats 107/385  avg 1260      8.5%  blunt
other tests (9) 
test                                result  vs the others in its run   difficulty           counts  why
LiveBench 2024-07-26                 34.5%  ▁▁▁▂▂▃▂█●▃▅▃ beats 36/61   medium · avg 38%        33%  at its level
RepoQA (32K, threshold 0.8) (make…   77.0%  ▄▄●··▄▄··▄·█ beats 2/7     easy · avg 80%         6.8%  typical
Open Persian LLM Leaderboard v2 a…   63.0%  ▂▄▂▃█▄▃●▃▅▂▂ beats 60/81   medium · avg 57%       4.0%  blunt · at its level
BigCodeBench (Instruct) (maker)      25.8%  ●··▄▄···▄▄█▄ beats 0/7     hard · avg 35%         3.5%  at its level
RULER 128k                           64.6%  ▄··▂▄▂●██▄▄▂ beats 7/21    easy · avg 70%         2.4%  at its level · small run · several runs
LongBench v2 (maker)                 27.8%  ▂·●·▂▃▁▄▄▃█▄ beats 2/33    very hard · avg 33%    2.0%  small run
RULER 64k                            85.4%  ▁▁·▁·▂▃▄●▂·▁ beats 16/22   easy · avg 82%         1.9%  typical
Vectara hallucination leaderboard…   94.6%  ▁▁▂▃▄●█▆▃▂▂▁ beats 56/140  very easy · avg 94%    0.6%  blunt · maxed out · noisy
Vectara hallucination leaderboard…   93.1%  ▁▂▁▂▅█▂●▄▃▁▁ beats 72/98   very easy · avg 90%    0.0%  blunt
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
Cohere (submitted / maker-reported)             9   35% ▇▇▇▇
LiveBench model judgments                       1   33% ▇▇▇▇
Aider code editing benchmark runs               1   16% ▇▇
LMArena leaderboard dataset                     1    8% ▇
Open Persian LLM Leaderboard v2                 1    4% ▇
NVIDIA RULER repository README results …        2    3% ▇
Vectara hallucination leaderboard               2    1% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 48.4
Licences: Cohere ([CC BY 4.0](../../licenses/report-2504.00698v2.txt)) · LiveBench model judgments ([Apache-2.0 (stated in the LiveBench datasheet)](https://www.apache.org/licenses/LICENSE-2.0)) · Aider code editing benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · NVIDIA RULER repository README results table ([Apache-2.0](https://github.com/NVIDIA/RULER/blob/c3f5e3b4f87f97e048793bb510a3a6b19a46bf3a/LICENSE)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) ## How to run it ### Through an API api (Cohere) · $4.38/M blended · $2.5 in / $10 out per 1M tokens · [docs.cohere.com](https://docs.cohere.com/docs/models) [price source](https://github.com/BerriAI/litellm/blob/c19ce71bcdad234ca87a7af10cee45375ce811a8/model_prices_and_context_window.json): USD per 1M text tokens, standard input and output (LiteLLM @c19ce71 row 'command-r-plus-08-2024', litellm\_provider cohere\_chat). Product: Cohere Chat API. API model: command-r-plus-08-2024 (Cohere models page: Live). Currency: USD. · [context source](https://huggingface.co/CohereLabs/c4ai-command-r-plus-08-2024/blob/e808c1a2249354ca211c9f08d1338e5039f633f8/README.md): Context length stated on the official model card. ### llama.cpp (recommended) 1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md) ```sh # macOS / Linux curl -LsSf https://llama.app/install.sh | sh # Windows irm https://llama.app/install.ps1 | iex ``` 2. Run Command R Plus (08-2024) (Q4_K_M, needs ~62.8 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/c4ai-command-r-plus-08-2024-GGUF/tree/059dd2d209151aafb863c7d3bdf07d1055b362e1)) ```sh llama cli -hf lmstudio-community/c4ai-command-r-plus-08-2024-GGUF:Q4_K_M ``` 3. Or start a local server and open the address it prints (http://localhost:9931) ```sh llama serve -hf lmstudio-community/c4ai-command-r-plus-08-2024-GGUF:Q4_K_M ``` ### LM Studio 1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download) 2. Run Command R Plus (08-2024) (Q4_K_M, needs ~62.8 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/c4ai-command-r-plus-08-2024-GGUF/tree/059dd2d209151aafb863c7d3bdf07d1055b362e1)) ```sh lms get https://huggingface.co/lmstudio-community/c4ai-command-r-plus-08-2024-GGUF@Q4_K_M -y ``` 3. Then run lms ls to see its name, and lms chat followed by that name Other versions: 4bit (~58.4 GB, Mac only) `lms get https://huggingface.co/mlx-community/c4ai-command-r-plus-08-2024-4bit -y` · 8bit (~110.3 GB, Mac only) `lms get https://huggingface.co/mlx-community/c4ai-command-r-plus-08-2024-8bit -y` ### Ollama 1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download) ```sh # Linux curl -fsSL https://ollama.com/install.sh | sh ``` 2. Run Command R Plus (08-2024) (Q4_K_M, needs ~62.9 GB, from [ollama.com](https://ollama.com/library/command-r-plus/tags)) ```sh ollama run command-r-plus:104b-08-2024-q4_K_M ``` Other versions: Q8_0 (~125.3 GB) `ollama run command-r-plus:104b-08-2024-q8_0` ### MLX (Mac) 1. Install MLX (Mac). Apple silicon Macs only, text only. ```sh # with uv uv tool install mlx-lm # or pip install mlx-lm ``` 2. Run Command R Plus (08-2024) (4bit, needs ~58.4 GB, Mac only, from [huggingface.co/mlx-community](https://huggingface.co/mlx-community/c4ai-command-r-plus-08-2024-4bit/tree/c1793cf166ecc2e82bca7979d72cc91a95dd4d76)) ```sh mlx_lm.chat --model mlx-community/c4ai-command-r-plus-08-2024-4bit ``` 3. Or start an OpenAI-compatible server at http://localhost:8080/v1 ```sh mlx_lm.server --model mlx-community/c4ai-command-r-plus-08-2024-4bit ``` Other versions: 8bit (~110.3 GB, Mac only) `mlx_lm.chat --model mlx-community/c4ai-command-r-plus-08-2024-8bit` Model repository and documentation: [huggingface.co/CohereLabs](https://huggingface.co/CohereLabs/c4ai-command-r-plus-08-2024) [interactive version](https://whatmodel.sh/model/command-r-plus-08-2024) · [JSON](https://whatmodel.sh/api?format=json&all=1)