[whatmodel.sh](../../README.md) / Aya 23 35B
# Aya 23 35B
Cohere For AI · released 2024-05 · 34.98B params · cc-by-nc-4.0
**score 50.1** ±2.4 · #369 of 732 · from 5 benchmark results across 4 sources
Score within the [LMArena (text)](https://whatmodel.sh/?set=lmarena-text&score=set) comparison set only.
## What it's good at
● marks Aya 23 35B among every model on that task or test, worst to best.
other tests (5)
test result vs the others in its run difficulty counts why
Polish EQ-Bench v2 (SpeakLeash) 58.4% ▁··▁▁▂▂▆▅●█▆ beats 37/70 medium · avg 62% 36% at its level
Open Persian LLM Leaderboard v2 a… 55.2% ▂▄▂▃█●▃▆▃▅▂▂ beats 43/81 medium · avg 57% 28% blunt · at its level
Open PT-LLM Leaderboard average (… 71.4% ▃▁▁▁▁▁▁▁▂●▅▁ beats 183/276 medium · avg 72% 14% at its level · noisy
EuroEval HellaSwag (English), tes… 83.5% ▃·▁▁▂▁▂▅█●▄▁ beats 80/100 medium · avg 67% 12% sharp · at its level · noisy
EuroEval Life in the UK (English … 81.8% ▁▁▁▁▁▁▁▂▄●▄▃ beats 75/146 easy · avg 82% 7.4% noisy
source results share of the score
SpeakLeash Polish EQ-Bench leaderboard 1 36% ▇▇▇▇
Open Persian LLM Leaderboard v2 1 28% ▇▇▇
EuroEval leaderboards 2 19% ▇▇
Open Portuguese LLM Leaderboard 1 14% ▇▇
prior: pull toward the middle 3% ▇
weighted average = 50.1
Licences: SpeakLeash Polish EQ-Bench leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · Open Persian LLM Leaderboard v2 ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt)) · EuroEval leaderboards ([MIT](https://github.com/EuroEval/leaderboards/blob/1a77f1d2f8ba1ba05ab73287fb090d76bf41b2e0/LICENSE)) · Open Portuguese LLM Leaderboard ([Apache-2.0](../../licenses/hf-leaderboard-spaces-Apache-2.0.txt))
## How to run it
### llama.cpp (recommended)
1. Install llama.cpp. Homebrew and winget installs name the commands llama-cli and llama-server instead. [Other ways to install](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md)
```sh
# macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
# Windows
irm https://llama.app/install.ps1 | iex
```
2. Run Aya 23 35B (Q4_K_M, needs ~21.5 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/aya-23-35B-GGUF/tree/1dfd3c7340b838de4af1c70f22a89133b36bc315))
```sh
llama cli -hf lmstudio-community/aya-23-35B-GGUF:Q4_K_M
```
3. Or start a local server and open the address it prints (http://localhost:9931)
```sh
llama serve -hf lmstudio-community/aya-23-35B-GGUF:Q4_K_M
```
Other versions: Q8_0 (~37.2 GB) `llama cli -hf lmstudio-community/aya-23-35B-GGUF:Q8_0`
### LM Studio
1. Install LM Studio. Open the app once before using the lms command. [Download it (macOS / Windows / Linux)](https://lmstudio.ai/download)
2. Run Aya 23 35B (Q4_K_M, needs ~21.5 GB, from [huggingface.co/lmstudio-community](https://huggingface.co/lmstudio-community/aya-23-35B-GGUF/tree/1dfd3c7340b838de4af1c70f22a89133b36bc315))
```sh
lms get https://huggingface.co/lmstudio-community/aya-23-35B-GGUF@Q4_K_M -y
```
3. Then run lms ls to see its name, and lms chat followed by that name
Other versions: Q8_0 (~37.2 GB) `lms get https://huggingface.co/lmstudio-community/aya-23-35B-GGUF@Q8_0 -y` · 4bit (~19.7 GB, Mac only) `lms get https://huggingface.co/mlx-community/aya-23-35B-4bit -y` · 8bit (~37.2 GB, Mac only) `lms get https://huggingface.co/mlx-community/aya-23-35B-8bit -y`
### Ollama
1. Install Ollama. [Download it (macOS / Windows)](https://ollama.com/download)
```sh
# Linux
curl -fsSL https://ollama.com/install.sh | sh
```
2. Run Aya 23 35B (Q4_K_M, needs ~21.49 GB, from [ollama.com](https://ollama.com/library/aya/tags))
```sh
ollama run aya:35b-23-q4_K_M
```
Other versions: Q8_0 (~42.48 GB) `ollama run aya:35b-23-q8_0`
### MLX (Mac)
1. Install MLX (Mac). Apple silicon Macs only, text only.
```sh
# with uv
uv tool install mlx-lm
# or
pip install mlx-lm
```
2. Run Aya 23 35B (4bit, needs ~19.7 GB, Mac only, from [huggingface.co/mlx-community](https://huggingface.co/mlx-community/aya-23-35B-4bit/tree/7159bf2a1f973bc43878205b939d711ab6d59d75))
```sh
mlx_lm.chat --model mlx-community/aya-23-35B-4bit
```
3. Or start an OpenAI-compatible server at http://localhost:8080/v1
```sh
mlx_lm.server --model mlx-community/aya-23-35B-4bit
```
Other versions: 8bit (~37.2 GB, Mac only) `mlx_lm.chat --model mlx-community/aya-23-35B-8bit`
Model repository and documentation: [huggingface.co/CohereLabs](https://huggingface.co/CohereLabs/aya-23-35B)
[interactive version](https://whatmodel.sh/model/aya-23-35b) · [JSON](https://whatmodel.sh/api?format=json&all=1)