[whatmodel.sh](../../README.md) / Command R (08-2024) # Command R (08-2024) Cohere · released 2024-08 · 32.3B params · cc-by-nc-4.0 **score 47.5** ±0.9 · #417 of 732 · from 16 benchmark results across 9 sources Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only. ## What it's good at ● marks Command R (08-2024) among every model on that task or test, worst to best. 
task            vs the others          rank  counts  tests
agent tasks     ▁▁●▆█▄▅▅▄▄▂▁   #293 of 313*    5.9%      1
coding          ▁▁▁▃▅●█▇▅▄▃▁    #347 of 469     37%      6
chat & writing  ▁▁▁▃▆▄●█▆▇▇▃   #256 of 402*    8.7%      1
agent tasks · #293 of 313 · 5.9% of its score Beats [Bielik 11B v2.3 Instruct](bielik-11b-v2-3-instruct.md), [GLM-4 9B Chat](glm-4-9b-chat.md) here, though they rank higher overall. Trails [Granite 4.0 H 350M](granite-4-0-h-350m.md), [Qwen3 0.6B](qwen3-0-6b.md), [Qwen2.5 1.5B Instruct](qwen2-5-1-5b.md) here, though they rank lower overall. 
test                        result  vs the others in its run  difficulty      counts  why
SWE-bench Verified (maker)   11.6%  ▃···●·▃·▃█·▃ beats 1/7    hard · avg 17%    5.9%  sharp
coding · #347 of 469 · 37% of its score Beats [LFM2.5 2.6B](lfm2-5-2-6b.md), [LFM2.5 8B-A1B](lfm2-5-8b-a1b.md), [Qwen3-VL 2B Thinking](qwen3-vl-2b-thinking.md) here, though they rank higher overall. Trails [Phi-3-mini-128k-instruct (Apr 2024)](phi-3-mini-128k-instruct.md), [DeepSeek LLM 67B Chat](deepseek-llm-67b-chat.md), [WizardLM 70B V1.0](wizardlm-70b-v1-0.md) here, though they rank lower overall. 
test                                result  vs the others in its run  difficulty            counts  why
Aider code editing (133 Exercism …   38.3%  ▁·▁▂●▁▆▄█▆▂▂ beats 9/62   medium · avg 51%         17%  sharp · at its level
SWE-bench Verified (maker)           11.6%  ▃···●·▃·▃█·▃ beats 1/7    hard · avg 17%          5.9%  sharp
MBPP+ (maker)                        74.3%  █●·██···████ beats 1/7    easy · avg 78%          4.9%  typical
LiveCodeBench v5 (Oct 2024–Feb 20…   11.0%  ▄●·▄·····█▄█ beats 1/7    hard · avg 18%          3.9%  typical
BIRD-SQL (dev) (maker)               47.3%  ▄··●·▄·█··▄█ beats 1/7    medium · avg 47%        3.0%  at its level
Aider polyglot (225 exercises, tw…    1.8%  ●▄··█·█····▄ beats 0/7    very hard · avg 5.5%    2.5%  sharp · too hard for it · noisy
chat & writing · #256 of 402 · 8.7% of its score Beats [Mistral Small 4 (119B-A6.5B)](mistral-small-4.md), [Qwen2.5 Coder 32B Instruct](qwen2-5-coder-32b.md), [Ministral 3 14B Reasoning 2512](ministral-3-14b-reasoning.md) here, though they rank higher overall. Trails [Flan-T5 XXL](flan-t5-xxl.md), [Granite 4.0 H 1B](granite-4-0-h-1b.md), [Granite 4.0 1B](granite-4-0-1b.md) here, though they rank lower overall. 
test            result  vs the others in its run    difficulty  counts  why
LMArena (text)  1276.3  ▁▁▁▂▃▃●▆▆▇█▃ beats 108/385  avg 1260      8.7%  blunt
other tests (9) 
test                                result  vs the others in its run    difficulty           counts  why
LiveBench 2024-07-26                 30.8%  ▁▁▁▂▂▃▂●▆▃▅▃ beats 29/61    medium · avg 38%        33%  at its level
RepoQA (32K, threshold 0.8) (make…   73.2%  ▄●▄··▄▄··▄·█ beats 1/7      easy · avg 80%         7.1%  typical
Open Persian LLM Leaderboard v2 a…   59.8%  ▂▄▂▃█▄▃●▃▅▂▂ beats 49/81    medium · avg 57%       4.1%  blunt · at its level
BigCodeBench (Instruct) (maker)      34.3%  ▄··▄●···▄▄█▄ beats 2/7      hard · avg 35%         3.5%  at its level
Open PT-LLM Leaderboard average (…   74.4%  ▃▁▁▁▁▁▁▁▂█●▁ beats 218/276  medium · avg 72%       2.1%  at its level · noisy
RULER 64k                            86.6%  ▁▁·▁·▂▃▄●▂·▁ beats 18/22    easy · avg 82%         2.0%  typical
EuroEval Life in the UK (English …   88.2%  ▁▁▁▁▁▁▁▂▄█●▃ beats 109/146  easy · avg 82%         1.2%  at its level · noisy
RULER 128k                           74.7%  ▄··▂▄▂▆█●▄▄▂ beats 13/21    easy · avg 67%         0.7%  at its level · noisy
Vectara hallucination leaderboard…   95.1%  ▁▁▂▃▄▄●▆▃▂▂▁ beats 60/140   very easy · avg 94%    0.6%  blunt · maxed out · noisy
## How the score was worked out Each result implies a score on its own, after adjusting for how hard that test is and who ran it. The model's score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its *counts* column above; here it is added up by who ran the tests. [Method](../method.md) 
source                                    results  share of the score
LiveBench model judgments                       1   33% ▇▇▇▇
Cohere (submitted / maker-reported)             7   31% ▇▇▇▇
Aider code editing benchmark runs               1   17% ▇▇
LMArena leaderboard dataset                     1    9% ▇
Open Persian LLM Leaderboard v2                 1    4% ▇
NVIDIA RULER repository README results …        2    3% ▇
Open Portuguese LLM Leaderboard                 1    2% ▇
EuroEval leaderboards                           1    1% ▇
Vectara hallucination leaderboard               1    1% ▇
prior: pull toward the middle                        0% ▇
weighted average                                   = 47.5
Licences: LiveBench model judgments ([Apache-2.0 (stated in the LiveBench datasheet)](https://www.apache.org/licenses/LICENSE-2.0)) · Cohere ([CC BY 4.0](../../licenses/report-2504.00698v2.txt)) · Aider code editing benchmark runs ([Apache-2.0](../../licenses/aider-polyglot-runs-Apache-2.0.txt)) · LMArena leaderboard dataset ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · NVIDIA RULER repository README results table ([Apache-2.0](https://github.com/NVIDIA/RULER/blob/c3f5e3b4f87f97e048793bb510a3a6b19a46bf3a/LICENSE)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · Vectara hallucination leaderboard ([Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)) ## How to run it ### Through an API api (Cohere) · $0.26/M blended · $0.15 in / $0.6 out per 1M tokens · [docs.cohere.com](https://docs.cohere.com/docs/models) [price source](https://github.com/BerriAI/litellm/blob/c19ce71bcdad234ca87a7af10cee45375ce811a8/model_prices_and_context_window.json): USD per 1M text tokens, standard input and output (LiteLLM @c19ce71 row 'command-r-08-2024', litellm\_provider cohere\_chat). Product: Cohere Chat API. API model: command-r-08-2024 (Cohere models page: Live). Currency: USD. · [context source](https://huggingface.co/CohereLabs/c4ai-command-r-08-2024/blob/dc835b893cd3fb8f14b24970dbc2a0a6d3c22ee3/README.md): Context length stated on the official model card. ### llama.cpp (recommended) 1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md) ```sh # macOS / Linux curl -LsSf https://llama.app/install.sh | sh # Windows irm https://llama.app/install.ps1 | iex ``` 2. Run Command R (08-2024) (Q4_K_M, needs ~19.8 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/c4ai-command-r-08-2024-GGUF/tree/284cc00405b04336bc3964abe08af2b792918b43)) ```sh llama cli -hf lmstudio-community/c4ai-command-r-08-2024-GGUF:Q4_K_M ``` 3. Or start a local server and open the address it prints (http://localhost:9931) ```sh llama serve -hf lmstudio-community/c4ai-command-r-08-2024-GGUF:Q4_K_M ``` Other versions: Q8_0 (~34.3 GB) `llama cli -hf lmstudio-community/c4ai-command-r-08-2024-GGUF:Q8_0` ### LM Studio 1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download) 2. Run Command R (08-2024) (Q4_K_M, needs ~19.8 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/c4ai-command-r-08-2024-GGUF/tree/284cc00405b04336bc3964abe08af2b792918b43)) ```sh lms get https://huggingface.co/lmstudio-community/c4ai-command-r-08-2024-GGUF@Q4_K_M -y ``` 3. Then run lms ls to see its name, and lms chat followed by that name Other versions: Q8_0 (~34.3 GB) `lms get https://huggingface.co/lmstudio-community/c4ai-command-r-08-2024-GGUF@Q8_0 -y` · 4bit (~18.2 GB, Mac only) `lms get https://huggingface.co/mlx-community/c4ai-command-r-08-2024-4bit -y` · 8bit (~34.3 GB, Mac only) `lms get https://huggingface.co/mlx-community/c4ai-command-r-08-2024-8bit -y` ### Ollama 1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download) ```sh # Linux curl -fsSL https://ollama.com/install.sh | sh ``` 2. Run Command R (08-2024) (Q4_K_M, needs ~19.88 GB, from [ollama.com](https://ollama.com/library/command-r/tags)) ```sh ollama run command-r:35b-08-2024-q4_K_M ``` Other versions: Q8_0 (~39.26 GB) `ollama run command-r:35b-08-2024-q8_0` ### MLX (Mac) 1. Install MLX (Mac). Apple silicon Macs only, text only. ```sh # with uv uv tool install mlx-lm # or pip install mlx-lm ``` 2. Run Command R (08-2024) (4bit, needs ~18.2 GB, Mac only, from [huggingface.co/mlx-community](https://huggingface.co/mlx-community/c4ai-command-r-08-2024-4bit/tree/563c818b2a8358765f78d30156f6e36f93b57110)) ```sh mlx_lm.chat --model mlx-community/c4ai-command-r-08-2024-4bit ``` 3. Or start an OpenAI-compatible server at http://localhost:8080/v1 ```sh mlx_lm.server --model mlx-community/c4ai-command-r-08-2024-4bit ``` Other versions: 8bit (~34.3 GB, Mac only) `mlx_lm.chat --model mlx-community/c4ai-command-r-08-2024-8bit` Model repository and documentation: [huggingface.co/CohereLabs](https://huggingface.co/CohereLabs/c4ai-command-r-08-2024) [interactive version](https://whatmodel.sh/model/command-r-08-2024) · [JSON](https://whatmodel.sh/api?format=json&all=1)