**Run the biggest language model you can on the smallest hardware you own.**
TV boxes, old tablets, single-board computers and routers all have a CPU, some RAM and flash storage that sit idle most of the day. smolmaxxing finds the biggest model each of them can really run, then pushes it further with custom kernels, careful memory use and weights streamed from flash. Every number in this repo was measured on the device itself.
## Three devices so far
| | ๐บ Xiaomi TV Box S (3rd Gen) | ๐ฑ iPad Air (2013) | ๐ OnePlus Nord CE 2 (2022) |
|---|---|---|---|
| **Chip** | Amlogic S7D, 4ร Cortex-A55 | Apple A7, 2 cores | MediaTek Dimensity 900, 2ร Cortex-A78 + 6ร A55 |
| **RAM** | 1.9 GB (about 1 GB usable) | 1 GB (about 700 MB usable) | 7.4 GB (about 3 GB free) |
| **OS** | Android 14, no root | iOS 12.5.7, jailbroken | Android 13, no root |
| **Everyday model** | Qwen3.5-2B at **7.7 tok/s** | Bonsai-1.7B (1-bit) at **24-26 tok/s** | Ling-mini-2.0, 16B, at **11-15 tok/s** |
| **Biggest model** | Ling-mini-2.0, 16B, at **5.1 tok/s** | Ling-mini-2.0, 16B, at **2.6-4.5 tok/s** | Qwen3.6-35B-A3B at **4.9-5.7 tok/s** |
| **Full results** | [docs/tv-box.md](docs/tv-box.md) | [ipad-air/README.md](ipad-air/README.md) | [EXPERIMENT.md ยง22](EXPERIMENT.md#22-device-3-a-oneplus-nord-ce-2-phone-16b-to-35b-2026-11-13) |
The TV box and the iPad run 8B and 16B mixture-of-experts models on about 1 GB of RAM, and the phone runs a 35B one with about 3 GB free. They all keep only part of each model in memory and read the rest from flash as it's needed.
More devices (single-board computers, routers, more phones) are coming. The techniques aren't tied to one box: the 1-bit kernel needs only basic NEON, and the flash streaming works on any Linux or Android device with more model than RAM.
**Got a small device?** Run a model on it and [submit your numbers](CONTRIBUTING.md#submit-your-device): we'll add them to the results.
## How it works
All of it is a patch on [llama.cpp](https://github.com/ggml-org/llama.cpp). The main ideas:
- **Kernels built for the chip.** A 1-bit lookup-table kernel, repacked 2-bit and ternary formats for the Cortex-A55, and versions of them for the A7, which has no dot-product instructions.
- **Less RAM per model.** Most of a small model's vocabulary is rarely used, so we drop those output rows. On the TV, a "headless" mode also stops the launcher and Google services while a model runs.
- **Experts paged from flash.** A mixture-of-experts model only uses a few experts per token. We keep the popular ones in RAM, read the others from flash on demand and prefetch the next layer's. This is how 8B and 16B models run on 1 GB, and a 35B model on a phone.
- **Streaming for dense models** bigger than RAM, at the flash's full speed.
[docs/how-it-works.md](docs/how-it-works.md) has every technique with its measured gain.
## Try it
The quickest start is a ready-made model file from Hugging Face. They need the patched llama.cpp: stock llama.cpp can't load them.
- **TV box or another ARMv8.2 Android device:** [docs/tv-box-setup.md](docs/tv-box-setup.md)
- **Jailbroken iPad on iOS 12:** [ipad-air/README.md](ipad-air/README.md#run-it)
- **All model files and how to rebuild them:** [docs/models.md](docs/models.md)
## Read more
| Page | What's in it |
|---|---|
| [docs/tv-box.md](docs/tv-box.md) | device #1: every model run on the TV box, speed and quality |
| [ipad-air/README.md](ipad-air/README.md) | device #2: the iPad Air results, setup and scripts |
| [EXPERIMENT.md ยง22](EXPERIMENT.md#22-device-3-a-oneplus-nord-ce-2-phone-16b-to-35b-2026-11-13) | device #3: the OnePlus phone, from 16B to a 35B model |
| [docs/how-it-works.md](docs/how-it-works.md) | the kernels, memory tricks, expert pager and streaming engine |
| [docs/models.md](docs/models.md) | downloads for both devices, rebuild targets and checksums |
| [docs/lessons.md](docs/lessons.md) | what we learned, including what didn't work |
| [EXPERIMENT.md](EXPERIMENT.md) | the full lab notebook: 160 experiments, failures included |
## What's in the repo
| Path | What |
|---|---|
| `patches/` | every change to llama.cpp for the TV box (the iPad's patch is in `ipad-air/patches/`) |
| `tools/` | model rebuilding, GGUF rewriting, benchmarks and kernel checks |
| `tv-llm-ui/` | the live token-stream chat page and its launcher |
| `ipad-air/` | device #2: iOS 12 build, `smolchat`, deploy and benchmark scripts |
| `bench/` | every chat benchmark (`results.jsonl`) and the raw logs |
| `data/` | vocabulary lists and evaluation texts |
| `tiny-llm-web/` | the first experiment: a 260K-parameter story model |
| `zigcc/` | zig wrappers for cross-compiling static aarch64 binaries |
## Star History
## License
[MIT](LICENSE). The llama.cpp patches are derivatives of [llama.cpp](https://github.com/ggml-org/llama.cpp) (MIT). Model weights keep their own licenses, and the evaluation texts are Wikipedia excerpts (CC BY-SA 4.0).
## Credits
Built on [llama.cpp](https://github.com/ggml-org/llama.cpp) and [llama2.c](https://github.com/karpathy/llama2.c). Models from [Qwen](https://huggingface.co/Qwen), [unsloth](https://huggingface.co/unsloth) (GGUFs and importance matrices), [PrismML](https://huggingface.co/prism-ml) (Bonsai), [inclusionAI](https://huggingface.co/inclusionAI) (Ling-mini-2.0), [tablinum](https://huggingface.co/tablinum) (Moon-Beam), [bartowski](https://huggingface.co/bartowski) and [mradermacher](https://huggingface.co/mradermacher) (GGUFs), Google (Gemma) and [stories260K](https://huggingface.co/karpathy/tinyllamas). The iPad is jailbroken with [Amethyst](https://github.com/staturnzz/amethyst). The banner uses [Archivo](https://fonts.google.com/specimen/Archivo) (SIL OFL) and is rendered by `tools/banner/render.sh`. The ๐ค logo is the pinching-hand emoji from Google's [Noto Emoji](https://github.com/googlefonts/noto-emoji) (Apache 2.0).