Kimi K3 inference in portable C99.
No BLAS. No framework. No GPU.
| 2.78T parameters |
1.56 TB checkpoint on disk |
8.24 GB peak RSS, measured |
176 KB the whole engine |
0 GPUs |
The same 2.78-trillion-parameter model, the same answer, on whatever machine you own.
More memory only buys speed:
| the machine you have | RAM | time per token | what is going on |
|---|---|---|---|
| an ordinary laptop | 8 GB | 26.5 s | the whole model streams off the disk on every step |
| a high-end laptop | 32 GB | 24.2 s | some of the model now sits in memory |
| a desktop | 64 GB | 19.8 s | more of it sits in memory |
| a heavy workstation | 128 GB+ | 5.6 s | the model fits entirely in memory, the disk wait is gone |
I am open to AI research roles and PhD positions. CV.
ultra preset, the faster MXFP4 kernels, cgroup-aware memory admission and a long list of fixes, most of 2.0.0 among them, came from the people below. Who did what.