ð English | TÃŒrkçe | Ø§ÙØ¹Ø±ØšÙØ© | æ¥æ¬èª | äžæ
Soup
ã³ãã³ãäžã€ã§ãLLM ã®ãã¡ã€ã³ãã¥ãŒãã³ã°ãšãã¹ããã¬ãŒãã³ã°ããSSH ããèšå®å°çãäžèŠã§ãã
ãŠã§ããµã€ã ·
ã¯ã€ãã¯ã¹ã¿ãŒã ·
Web UI ·
èšå® ·
ããã¥ã¡ã³ã ·
ã³ãã³ã ·
ã¢ãã« ·
Discord ·
Telegram ·
Product Hunt
---
Soup ã¯ãLLM ã®ãã¡ã€ã³ãã¥ãŒãã³ã°ã®é¢åãã·ã³ãã«ãªã¯ãŒã¯ãããŒã«å€ããŸããèšå®ãã¡ã€ã«ã¯äžã€ãã³ãã³ããäžã€ãããã ãã§ãã
```bash
pip install "soup-cli[train]" # ãã¡ã€ã³ãã¥ãŒãã³ã°ã«ã¯ [train] ã远å ããŸããçŽ ã® soup-cli ã¯è»œé㪠CLI ã§ã
soup init --template chat
soup train
```
**4 GB ã®ããŒã PC çš GPU ã§ã8B ã¢ãã«ããã¡ã€ã³ãã¥ãŒãã³ã°ã§ããŸãã** ã¬ã€ã€ãŒã¹ããªãŒãã³ã°ã¯ãåçµããããŒã¹ã¢ãã«ã
VRAM ã®å€ã«çœ®ãããã³ãŒããŒå±€ãäžå±€ã〠GPU ã«éã蟌ã¿ãŸããRTX 3050 Laptop 4 GB ã§ã®å®æž¬å€:
Llama-3.1-8B-Instruct + NF4 ã§ **952 MHz æã« 119.6 tok/sãããŒã¯ 3.32 GB**ïŒä¿®æ£åïŒâ å
šäœãã¡ã¢ãªã«åžžé§ãããéåžžã®å®è¡ãšãããåäœã§
å®å
šã«äžèŽããH100 ã§ãåã 3.32 GB ã§ 113.00 tok/s ãšç¬ç«ã«åçŸãããŸããã
ïŒã©ã¡ãã®æ°å€ã v0.72.2 ã§ãv0.73.0 ã®æ£ç¢ºæ§ä¿®æ£ïŒ32B ã§ â4.8% ã®ã³ã¹ãïŒããåã«æž¬å®ãããã®ã§ãã
ä¿®æ£åŸã®å®è¡ã¯ãå¥ã® RTX 3050 Laptop 4 GBïŒ@umran666ã#1103ïŒã§
**1935â1957 MHz æã« 208.6 tok/sãããŒã¯ 2.40 GB** ã§ãããããã¯åäžã»ãã·ã§ã³ã§ã® GEMM äžéã® 59% ã«ãããã
ä¿®æ£åã® 68% ã«å¯ŸããŠãã¯ããã¯ãããã§ã¯çŽ 14% é
ãããšã«ãªããŸããã€ãŸããèŠããã® 1.74x ã®åäžã¯é«éåã§ã¯ãªã
ã¯ããã¯ã®å·®ã§ãã倧ããªã¬ã€ã€ãŒã®ã¹ããªãŒãã³ã°ã«ãã£ãŠäž¡è¡ã®ããã ã§ã¡ã¢ãªã®äœ¿ããæ¹ãå€ãã£ãŠããããããã®å²åã§ãã
åãæ¡ä»¶ã§ã®æ¯èŒã§ã¯ãããŸãã â [èšé²](benchmarks/gate-v0.72.2-nf4.md)ãïŒãªããã€ã³æ¹åŒïŒ`stream_layers: true`ïŒã§ã
ãŸã BETA ã§ã â
[ä»çµã¿](docs/performance-and-quantization.md#layer-streaming-beta-v0720-nf4-v0722-disk--wider-archs-v0723-preference-losses-v0724) ·
[ãã¹ãŠã®æž¬å®çµæ](benchmarks/) · [è«æ](https://doi.org/10.5281/zenodo.21771064) ·
**[ç¡æã® Colab T4 ã§èªåã§ç¢ºããã](notebooks/proof-4gb.ipynb)**ïŒããã»ã¹ã 4 GB ã«å¶éããäžã§ã
ã¹ããªãŒãã³ã°ããã¢ãã«ãéåžžã®ã¢ãã«ãšãããåäœã§åäžã§ããããšãæ€èšŒããŸãïŒ

Llama-3.1-8B-Instruct + NF4ãLoRAãããããµã€ãº 1ãã·ãŒã±ã³ã¹é· 512ãRTX 3050 Laptop 4 GB â ããŒã¯ 3.32 GBã119.6 tok/sã952 MHz æïŒä¿®æ£åãv0.72.2ã#331 ä¿®æ£åŸ: 1935 MHz æã« 208.6 tok/sã2.40 GBãçŽæ¥ã®æ¯èŒã¯ã§ããŸããã@umran666ã#1103ïŒã åç»ïŒ90 ç§ïŒ
## ãªãSoupãªã®ã
LLM ã®åŠç¿ã¯ãä»ã§ãéªšã®æããäœæ¥ã§ããçµéšè±å¯ãªããŒã ã§ãããæéã® 30ã50% ããã¢ãã«ã®æ¹åã§ã¯ãªã
ã€ã³ãã©ãšã®æ Œéã«è²»ãããŠããŸããSoup ã¯ããã解決ããŸãã
- **SSH äžèŠã** æ
éãã GPU ãã·ã³ã«äºåºŠãš SSH ã§å
¥ãå¿
èŠã¯ãããŸããã
- **èšå®ã¯äžã€ã** å¿
èŠãªã®ã¯ã·ã³ãã«ãª YAML ãã¡ã€ã«ã ãã§ãã
- **ãã¹ãŠèªåã** ããããµã€ãºãGPU ã®æ€åºãéåå â ãã¹ãŠãä»»ãã§ãã
- **ããŒã«ã«ã§åäœã** QLoRA ã䜿ã£ãŠæå
ã® GPU ã§åŠç¿ã§ããŸããã¯ã©ãŠãã¯äžèŠã§ãã
## æ°çæ
å ±
**v0.77.2 â v0.77.1 以éã® `main` ããã®ãŸãŸåãåºãããªãªãŒã¹ïŒãã«ãªã¯ãšã¹ã 159 ä»¶ããã¡ 153 ä»¶ã¯
ã¡ã³ãããŒä»¥å€ã® 30 人ã«ãããã®ã§ãã** 倧åã¯ä¿®æ£ã§ãããæ°æ©èœãšãv0.77.1 ãåãå
¥ããŠãããã®ã
æåŠããããšããã 2 ã€ã®èŠåãå«ã¿ãŸãããµãŒããŒã `task: sft` ã®èšå®ãã¢ããã°ã¬ãŒãããåã«
[v0.77.2 ã®ãªãªãŒã¹ããŒã](https://github.com/MakazhanAlpamys/Soup/releases/tag/v0.77.2)ãèªãã§ãã ããã0.76.x ããç§»è¡ããå Žåã¯ããŸã
`soup doctor --config soup.yaml` ãå®è¡ãã[v0.77.0 ã®ãªãªãŒã¹ããŒã](https://github.com/MakazhanAlpamys/Soup/releases/tag/v0.77.0)ãèªãã§ãã ããã
- **`soup profile --model ` 㯠`soup.yaml` ãªãã§ãåãŸãããã«çããŸãã** ã¢ãã«ã®
`config.json` ããå®è¡ã®èŠç©ãããåºãã`--seq`ã`--batch`ã`--method`ã`--lora-r`ã`--optimizer`ã
`--gradient-checkpointing` ã§åœ¢ã決ãŸããŸããApple Silicon ã§ã¯ Metal ã®ã¯ãŒãã³ã°ã»ãããšæ¯èŒããŸãã
`soup plan` ãš `soup apply` 㯠`--minillm-on-policy` ãåãä»ããŸãã
- **Apple Silicon ã§ã®åŠç¿ãå¢ããŸããã** `preference_loss: dpo`ã`orpo`ã`simpo` ãæå®ãã
`task: preference` ã¯ãã€ãã£ã MLX ãã¬ãŒããŒã§åããMLX GRPO 㯠4 ãããã® `mlx-community`
ããŒã¹ã§åŠç¿ã§ããMLX Adafactor ã¯èšå®ãããåŠç¿çãšã¹ã±ãžã¥ãŒã«ã«åŸããŸãã
- **æ°ãã DPO ã¬ã·ãã 2 ã€ïŒ** `qwen3.5-0.8b-dpo` ãš `qwen3.5-27b-dpo`ã
- **VRAM ã®èŠç©ããã宿ž¬ã«è¿ã¥ããŸããã** optimizer ã¹ããããç¬ç«ãããã§ãŒãºãšããŠæ±ãã
PRM ã¹ãããã«ã¯ 1 äœçœ®åã® logits ãèšäžããåå²ããã FSDP / ZeRO ã®å®è¡ã¯ã«ãŒãããšã«èŠç©ããã
`stream_loss_chunk` ã®ã¹ãããã«ã¯æçµå±€ã®åèšç®ãèšäžããUnsloth ã® LoRA ã¿ãŒã²ããã¯ãã®ããŒããŒã
éžã¶ãšããã«æ±ããŸãã
- **é»ã£ãäžæžããšç©ºã®çµæãæžããŸããã** ããå€ãã®ã³ãã³ãããå
¥åãæã `--output` ãæåŠããŸã
ïŒ`soup data ingest`ã`persona-mix`ã`generate` ãªã©ïŒã`soup ship --adapter` ã¯äœãå€ããªãã¢ããã¿ãŒã
æåŠãã`soup distill-prompt`ã`soup data evolve`ã`gen-magpie`ã`filter --perplexity` ã¯ç©ºãŸãã¯
æªãã£ã«ã¿ãŒã®ãã¡ã€ã«ãæžã代ããã«å€±æããŸãã
- **éžå¥œåŠç¿ãš RL ã®ä¿®æ£ã** `data.max_length` ã SimPO ãš KTO ã§åã³å¹ãããã«ãªããPPO ã¯èŠæ±ããããšæ¹çã«
gradient checkpointing ãé©çšãïŒ4 ãããã»8 ãããã§ãã¯ã©ãã·ã¥ããªããªãïŒã`soup merge` ã¯
ã¢ããã¿ãŒãåŠç¿ãããšãã®ããŒã¹ã®ã³ããããèªã¿èŸŒã¿ãŸãã
- **ããã¹ããå
¥åã©ããã«è¡šç€ºã** æ°åã®ã³ãã³ããããã¹ã»ååã»ã¡ãã»ãŒãžããè§æ¬åŒ§ãš `:name:` ã®
ããã¹ããä¿ã£ããŸãŸã端æ«å¶åŸ¡æåãåãé€ããŠè¡šç€ºããŸãã
**ç¥ã£ãŠããã¹ãæåã®å€æŽã** `soup serve --tool-auth-token` 㯠`A-Z`ã`a-z`ã`0-9`ã`_`ã`-` ãããªã
16ã128 æåã§ãªããã°ãªããŸããïŒããã§ãªããã°çµäºã³ãŒã 2ïŒã`task: sft` ã¯ã`reasoning_effort`ã
`train_on_eot`ã`use_flash_attn`ã`use_liger` ããããããäžåºŠãé©çšããŠããªãã£ãç»åã»é³å£°ã»MLXã»
Unslothã»RAFTã»äºåããŒã¯ãã€ãºæžã¿ã®çµè·¯ã§æåŠããŸãã`soup completions` ã®ã¹ã¯ãªããã¯åçæãå¿
èŠã§ãã
`soup data from-traces --signal thumbs_up`ãMLX Adafactorã`soup merge`ãVRAM ã®äºåãã§ãã¯ã¯ã
v0.77.1 ãšç°ãªãçµæã«ãªãããšããããŸãã
éå»ã®ãã€ã©ã€ã㯠[GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases) ããŒãžã«ãããŸãã
## ã¯ã€ãã¯ã¹ã¿ãŒã
### 1. ã€ã³ã¹ããŒã«
Soup ã¯ã³ãã³ãã©ã€ã³ã¢ããªã±ãŒã·ã§ã³ãªã®ã§ãæãããããªã€ã³ã¹ããŒã«æ¹æ³ã¯ãå°çšã®ç°å¢ãäžããŠ
`soup` ã `PATH` ã«çœ®ãããšã§ãã
```bash
# 軜éã³ã¢: CLI + èšå® + ããŒã¿ããŒã«ãPyTorch ã¯å«ã¿ãŸãã
pipx install soup-cli
uv tool install soup-cli # åãèãæ¹ã§ãããã§ã« uv ã䜿ã£ãŠããå Žå
# åŠç¿ã¹ã¿ãã¯ã远å ïŒtorchãtransformersãpeftãtrlãdatasetsãâŠïŒ
pipx install "soup-cli[train]"
# ãã¹ãŠïŒtrain + serve + ui + dataïŒãäžåºŠã«
pipx install "soup-cli[all]"
# ãŸã㯠GitHub ããïŒææ°ã®éçºçïŒ
pipx install "git+https://github.com/MakazhanAlpamys/Soup.git"
```
ãã§ã« virtualenvãColab ããŒãããã¯ãDocker ã€ã¡ãŒãžã®äžã«ããå Žåã¯ãåãååãšãšã¯ã¹ãã©ã§ `pip` ã
çŽæ¥äœ¿ã£ãŠãã ããã
```bash
pip install soup-cli
pip install "soup-cli[train]"
pip install "soup-cli[all]"
pip install git+https://github.com/MakazhanAlpamys/Soup.git
```
èªåã®ã³ãŒããã `import soup_cli` ããããå Žåã¯ã`pipx` ã§ã¯ãªã `pip` ã䜿ã£ãŠãã ããã
pipx ã¯æå³çã«ã¢ããªã±ãŒã·ã§ã³ãä»ã®ãã¹ãŠããåé¢ããããã§ãã
ãšã¯ã¹ãã©ã®å®å
šãªäžèЧïŒ`fast`ã`mlx`ã`serve`ã`eval`ã`ui`ã`vision`ã`audio`ãâŠïŒã¯
[`docs/models.md`](docs/models.md#optional-extras) ã«ãããŸãã
> **`error: externally-managed-environment` ãšåºãŸããã?** ããã¯
> [PEP 668](https://peps.python.org/pep-0668/) ã«ãããã®ã§ãSoup ã®åé¡ã§ã¯ãããŸãããDebian 12ã
> Ubuntu 23.04 以éã§ã¯ã`apt` ããããã®ãã¡ã€ã«ã管çããŠããããã`pip` ãã·ã¹ãã ã® Python ã«æžã蟌ã
> ãªãããã«ãªã£ãŠããŸãã`pipx` ãš `uv tool` ã¯ãSoup ã«å°çšã®ç°å¢ãäžããããšã§ãããåé¿ããŸããäžã§æåã«
> æããŠããã®ã¯ãã®ããã§ãã`python3 -m venv .venv && source .venv/bin/activate` ãå®è¡ããŠããéåžžã®
> `pip` ãäœ¿ãæ¹æ³ã§ããåãããã«åäœããŸãã
> **ããã«ã¯ã©ãŒãã䜿ããã·ã³ã°ã«ã¯ã©ãŒãã¯äœ¿ããªãã§ãã ããã** `"soup-cli[train]"` ã¯ã`cmd.exe`ã
> PowerShellãbashãzsh ã®ã©ã®ã·ã§ã«ã§ãåäœããå¯äžã®æžãæ¹ã§ããå€ããã¥ãŒããªã¢ã«ãã
> `'soup-cli[train]'` ãã³ããŒã㊠pip ã«æåŠãããå Žåã¯ããããçç±ã§ã:
> [çç±ãšæ£ç¢ºãªãšã©ãŒ](docs/models.md#quoting-the-extra)ã
`soup init`ã`soup data âŠ` ãªã©ã®ããŒã¿ïŒæ€æ»ç³»ã³ãã³ãã¯ã軜éã€ã³ã¹ããŒã«ã§åäœããŸãã
ãã¡ã€ã³ãã¥ãŒãã³ã°ïŒ`soup train`ïŒã«ã¯ `[train]` ãšã¯ã¹ãã©ãå¿
èŠã§ãã
### 2. èšå®ãã¡ã€ã«ãäœæãã
```bash
soup init # 察話åãŠã£ã¶ãŒã
soup init --template chat # ãŸãã¯ãã³ãã¬ãŒãããå§ãã
```
ãã³ãã¬ãŒã: `chat`ã`code`ã`tool-calling`ã`medical`ã`reasoning`ã`vision`ã`kto`ã`orpo`ã
`simpo`ã`ipo`ã`bco`ã`rlhf`ã`pretrain`ã`moe`ã`longcontext`ã`embedding`ã`audio`ã
### 3. åŠç¿ã»ãã¹ãã»å
Ž
```bash
soup train --config soup.yaml # LoRAãéååããããåŠç â ãã¹ãŠãä»»ã
soup chat --model ./output # ã¢ãã«ãšå¯Ÿè©±ãã
soup push --model ./output --repo you/my-model
soup merge --adapter ./output # LoRA ãããŒã¹ã«ããŒãž
soup export --model ./output --format gguf --quant q4_k_m # Ollama / llama.cpp åãã® GGUF
```
ãã®ä»ã®ãšã¯ã¹ããŒãå
ïŒONNXãTensorRTãBitNetïŒãšãããã€ã®ãªãã·ã§ã³ã¯ã
[`docs/serving-and-export.md`](docs/serving-and-export.md) ã«ãããŸãã
## Web UI
ãã©ãŠã¶ã®ã»ããã奜ã¿ã§ãã? `soup ui` ã¯ãå®éšãåŠç¿ã®ã»ããã¢ãããã©ã€ãã¡ããªã¯ã¹ãããŒã¿ã»ããã®æ¢çŽ¢ã
ã¢ãã«ãšã®ãã£ããã®ããã®ããŒã«ã«ããã·ã¥ããŒããæäŸããŸãã
```bash
pip install "soup-cli[ui]"
soup ui
# http://127.0.0.1:7860/?token= ãéããŸã
```

[Web UI ã®ããã¥ã¡ã³ã](docs/serving-and-export.md#web-ui)
## èšå®
å®å
šãª `soup.yaml` ã®äŸ:
```yaml
base: meta-llama/Llama-3.1-8B-Instruct
task: sft
# backend: unsloth # 2ã5 åé«éãpip install "soup-cli[fast]"
data:
train: ./data/train.jsonl
format: alpaca
val_split: 0.1
training:
epochs: 3
lr: 2e-5
batch_size: auto
lora:
r: 64
alpha: 16
quantization: 4bit
output: ./output
```
`config/schema.py` ã¯ããã¹ãŠã®ãã£ãŒã«ãã®å¯äžã®ä¿¡é Œã§ããæ
å ±æºã§ããé«åºŠãªããŒã¿ãåŠç¿ãPEFT ã®
ãªãã·ã§ã³ã¯ [ããã¥ã¡ã³ã](#ããã¥ã¡ã³ã) ã«ãŸãšããããŠããŸãã
> **æªç¥ã®èšå®ããŒã¯ v0.75 ããæåŠãããŸãã** ã©ã®ã¢ãã«ã宣èšããŠããªãã㌠â `quantizaton` ã®ãããª
> ã¿ã€ããã¹ããããæ°ãã Soup ã«ããååšããªããã£ãŒã«ã â ã¯ã以åã¯åé¡ãªãæ€èšŒãééããŠç Žæ£ããã
> ãã®èšå®ãé©çšãããªããŸãŸå®è¡ãç¶ããŠããŸãããv0.74 ã¯èªã¿èŸŒã¿æã«ãããããæå³ãããã£ãŒã«ããšãšãã«
> ãããå ±åããŠããŸããã**v0.75** ããã¯åãèšå®ã®èªã¿èŸŒã¿ã倱æããŸãã®ã§ãç¡èŠãããããšã«é Œããã
> ããŒãä¿®æ£ãããåé€ããŠãã ããã[æªç¥ã®èšå®ããŒ](docs/backends-and-ops.md#unknown-config-keys)
> ãåç
§ããŠãã ããã
## ããã¥ã¡ã³ã
æ©èœã®å®å
šãªãªãã¡ã¬ã³ã¹ã¯ [`docs/`](docs/) ã«ãããŸãããŸãã¯ãããã:
| ã¬ã€ã | å
容 |
|---|---|
| [åŠç¿ã¿ã¹ã¯ãšææ³](docs/training.md) | SFTãDPO/GRPO/PPO/KTO/ORPO/SimPO/IPO/BCOãããŒã«åŒã³åºããPRMãäºååŠç¿ãèžçãåé¡ãããžã§ã³/é³å£°/TTSãã¢ã³ã©ãŒãã³ã°ãRAFT/RA-DITãã«ãŒãåŒ·åæ€åºåš |
| [PEFTã»é·æèã»å¹çå](docs/peft-and-efficiency.md) | DoRAãLoRA+ãrsLoRAãVeRAãOLoRAãNEFTuneãPiSSAãReLoRAããªããã£ãã€ã¶ãŒãš PEFT ã®ã©ã€ã³ããããLLaMA ProãGaLoreãYaRN/LongLoRAããããã³ã°ãã«ãªãã¥ã©ã ãèªåãã¥ãŒãã³ã° |
| [ããã©ãŒãã³ã¹ãšéåå](docs/performance-and-quantization.md) | QATãFP8ãQuant MenuïŒI + IIïŒãKV ãã£ãã·ã¥ãNVFP4ãä¿å圢åŒãCut Cross-EntropyãåŸé
ãã§ãã¯ãã€ã³ããã«ãŒãã«ãã¢ã¯ãã£ããŒã·ã§ã³ã®ãªãããŒããã¬ã€ã€ãŒã¹ããªãŒãã³ã°ããã«ã GPU / DeepSpeed / FSDP |
| [ããŒã¿ãšã³ãžãã¢ãªã³ã°](docs/data.md) | å皮圢åŒãAxolotl/LF äºæãã€ãã©ã€ã³ãããŒã¿ããŒã«ãåæããŒã¿çæãš forgeãå質ã¹ã³ã¢ã«ãŒãããã¬ãŒã¹ããŒã«ããªã¢ãŒãããŒã¿ã»ãããããã·ã³ã°ãã¬ã·ã DAG |
| [è©äŸ¡ãšãããŒã](docs/evaluation.md) | è©äŸ¡ã®èšèšïŒã²ãŒããè©äŸ¡ã²ãŒãä»ãåŠç¿ããã³ãããŒã¯ãNLG ææšããã£ãªãã¬ãŒã·ã§ã³ãElo ã¢ãªãŒãã蚺æãåŠç¿åŸ X ç·ãããŒããA/Bãããªããããã¥ãŒãã³ã°å¯èœæ§ã`soup advise` |
| [ãµãŒãã³ã°ãšãšã¯ã¹ããŒã](docs/serving-and-export.md) | OpenAI äºæãµãŒããŒããããæšè«ããã³ãããŒã¯ãããŒãžïŒãšã¯ã¹ããŒããAnthropic Messages ãšã³ããã€ã³ããææ©çãã³ãŒãã£ã³ã°ïŒç¬èªã®ãã©ãããåŠç¿ããŠèšæž¬ïŒããããã€èªåæçžŠãWeb UIãAgent Forge |
| [ã¢ããã¿ãŒãã¬ãžã¹ããªãã¬ããã³ã¹](docs/adapters-and-governance.md) | ã¢ããã¿ãŒã®ã©ã€ããµã€ã¯ã«ïŒç®¡çãã¢ãã«ã¬ãžã¹ããªãSoup CansãããŒã¿ãã©ã€ãã€ãŒã«ïŒ`soup loop`ïŒãç¥èç·šéãã¹ãã¢ãªã³ã°ããµãã©ã€ãã§ãŒã³ç®¡çïŒscan/sign/BOM/attest/audit/airgapïŒ |
| [ã³ã³ãã©ã€ã¢ã³ã¹ãšã¬ããã³ã¹ã®ã¯ã€ãã¯ã¹ã¿ãŒã](docs/compliance.md) | HIPAA/SOC2/EU-AI-Act/SR-11-7 çšã® `init` ãã³ãã¬ãŒããæ¥æŽïŒBOM/attest/repro-receiptïŒãç£æ»ãã°ããšã¢ã®ã£ãããã¢ãã«ã«ãŒãã®èªåçæïŒ`soup card`ïŒãCI ã²ãŒãïŒ`soup ci init`ïŒ |
| [ããã¯ãšã³ãããã©ãããã©ãŒã ãéçš](docs/backends-and-ops.md) | MLX/Unsloth ããã¯ãšã³ãã代æ¿ãããHF Hub 飿ºããªãŒããã€ããããå®éšãã©ããã³ã°ãplan/applyãç°å¢ããã¯ãã¡ã€ã«ãããŒããŠã§ã¢é©åæ§ãã·ã§ã«è£å®ããã©ã°ã€ã³ããŠãŒãã£ãªãã£ã³ãã³ã |
| [ã³ãã³ããªãã¡ã¬ã³ã¹](docs/commands.md) | `soup` ã³ãã³ãã®å®å
šãªäžèЧ |
| [察å¿ã¢ãã«ãšãšã¯ã¹ãã©](docs/models.md) | æšå¥šã¢ãã«ãã¡ããªãŒãVRAM ãµã€ãºã®ç®å®ãpip ãšã¯ã¹ãã©ã®äžèŠ§è¡š |
## ããŒã¿åœ¢åŒ
AlpacaãShareGPTãChatMLãéžå¥œãã¢ïŒDPO / ORPO / SimPO / IPO / KTOïŒãããžã§ã³ãé³å£°ãASRããã¬ãŒã³ããã¹ãã
åã蟌ã¿ãRAFT ãªã© â ãã¹ãŠ JSONLãJSONãCSVãParquetãTXT ããèªåæ€åºããããããã»ãšãã©ã®å Žåã¯
`data.train` ããã¡ã€ã«ã«åããã ãã§ãä»ã¯äœãå€ãããŸãããå圢åŒã®å®äŸã€ãã¹ããŒããšãããŒã¿ãã€ãã©ã€ã³
ïŒãªã¢ãŒã URIãã¹ããªãŒãã³ã°ãã·ã£ãŒãã£ã³ã°ãã€ã³ã¿ãŒãªãŒããèªåœæ¡åŒµãããã¥ã¡ã³ãåã蟌ã¿ïŒã¯ã
[`docs/data.md`](docs/data.md#data-formats) ã«ããã[Sesame CSM ã®ãã€ãã£ãé³å£°åŠç¿](docs/training.md#native-sesame-csm)ã¯å¥é説æãããŠããŸãã
## ãã䜿ãã³ãã³ã
```bash
soup train --config soup.yaml # åŠç¿ïŒSFT/DPO/GRPO/PPO/KTO/ORPO/SimPO/IPO/...ïŒ
soup infer --model ./output --input prompts.jsonl --output results.jsonl # ãããæšè«
soup chat --model ./output # 察話åãã£ãã
soup serve --model ./output # OpenAI äºæ API ãµãŒããŒ
soup ui # ããŒã«ã«ã®ãã©ãŠã¶ããã·ã¥ããŒã
soup merge --adapter ./output # LoRA ãããŒã¹ã¢ãã«ã«ããŒãž
soup export --model ./output --format gguf # ãããã€çšã«ãšã¯ã¹ããŒã
soup eval benchmark --model ./output # è©äŸ¡
soup data inspect ./data/train.jsonl # ããŒã¿ã»ããã®çµ±èš
soup recipes list # 100 以äžã®æ¢æã¢ãã«ã¬ã·ã
soup autopilot --model --data d.jsonl --goal chat # èšå®äžèŠ
soup doctor # GPU / äŸåé¢ä¿ / ç°å¢ããã§ãã¯
```
ã³ãã³ãã®å®å
šãªäžèŠ§ã¯ [`docs/commands.md`](docs/commands.md) ã«ãããŸãã
## 察å¿ã¢ãã«
Soup ã¯ã[HuggingFace Hub](https://huggingface.co/models?pipeline_tag=text-generation) äžã®**ãããã**
ããã¹ãçæã¢ãã«ã§åäœããŸã â `AutoModelForCausalLM` ã§èªã¿èŸŒãããªããèšå®ãäžåå€ããã«åããŸãã
Llama 3.x/4ãQwen 2.5/3ãGemma 3ãMistralãMixtralãDeepSeek R1/V3ãPhi-4 ãã¯ãã 100 以äžã®ã¢ãã«ãã
æ¢æã®ã¬ã·ããšããŠçšæãããŠããŸãïŒ`soup recipes list`ïŒã
| VRAM | æå€§ã¢ãã«ïŒQLoRA 4-bitïŒ | äŸ |
|---|---|---|
| 8 GB | ~7B | Llama-3.1-8B, Mistral-7B |
| 16 GB | ~14B | Phi-4-14B, Qwen2.5-14B |
| 24 GB | ~34B | CodeLlama-34B, Yi-1.5-34B |
| 48 GB | ~70B | Llama-3.3-70B |
| 80 GB+ | 70B+ïŒãã«ïŒãŸã㯠MoE | Mixtral-8x22B, DeepSeek-V3 |
ã¢ãã«ãšããžã§ã³ã®å®å
šãªäžèŠ§è¡šãšããªãã·ã§ã³ã®ãšã¯ã¹ãã©ã®äžèŠ§è¡šã¯ [`docs/models.md`](docs/models.md) ã«ãããŸãã
## Docker
CUDA ã PyTorch ãããŒã«ã«ã«ã€ã³ã¹ããŒã«ããã« Soup ãå®è¡ã§ããŸãïŒã€ã¡ãŒãžã¯ãªãªãŒã¹ããšã« GHCR ã«å
¬éãããŸãïŒ:
```bash
docker pull ghcr.io/makazhanalpamys/soup:latest
docker run --gpus all -v $(pwd):/workspace ghcr.io/makazhanalpamys/soup train --config soup.yaml --yes
docker compose up # ãŸãã¯ããŒã«ã«ã§ãã«ã
```
## åäœèŠä»¶
- Python 3.11ã3.12ããŸã㯠3.13ïŒCI ããã¹ãããŠããããŒãžã§ã³ã§ããPyTorch ã¹ã¿ãã¯ããŸã æ€èšŒãããŠ
ããªãããã3.14 以éã¯æªãµããŒãã§ãïŒ
- CUDA å¯Ÿå¿ GPUïŒæšå¥šïŒãApple SiliconïŒMPSïŒããŸã㯠CPUïŒå®éšç â éåžžã«é
ãïŒ
- QLoRA ã§ 7B ã¢ãã«ãæ±ãã«ã¯ 8 GB 以äžã® VRAM
ãã¹ãŠã®åŠç¿ã¿ã¹ã¯ã¯ããã¹ãçšã« CPU äžã§ãåäœããŸãïŒéååã¯èªåçã«ç¡å¹ã«ãªããŸãïŒããªãã·ã§ã³ã®
ãšã¯ã¹ãã©ïŒ`train`ã`all`ã`fast`ã`vision`ã`qat`ã`serve`ã`serve-fast`ã`ui`ã`eval`ã`deepspeed`ã
`liger`ã`mlx`ã`onnx`ã`tensorrt`ãâŠïŒã¯ã[`docs/models.md`](docs/models.md#optional-extras) ã«äžèЧããããŸãã
## ãã©ãã«ã·ã¥ãŒãã£ã³ã°
```bash
soup doctor # GPUãã·ã¹ãã ãªãœãŒã¹ãäŸåé¢ä¿ãããŒãžã§ã³ãäžãæã§ç¢ºèª
```
CUDA wheel ãããŒãžã§ã³äžäžèŽã«ã€ããŠã¯ã[`docs/backends-and-ops.md`](docs/backends-and-ops.md#troubleshooting) ãåç
§ããŠãã ããã
## éçº
```bash
git clone https://github.com/MakazhanAlpamys/Soup.git
cd Soup
pip install -e ".[dev]"
ruff check src/soup_cli/ tests/ # ãªã³ã
pytest tests/ -v # ãŠããããã¹ãïŒé«éãGPU äžèŠïŒ
pytest tests/ -m smoke -v # ã¹ã¢ãŒã¯ãã¹ãïŒå°ããªã¢ãã«ãããŠã³ããŒãããŠåŠç¿ïŒ
pytest tests/ -m gpu --no-cov -v # GPU ãã¹ãïŒCUDA ã«ãŒããå¿
èŠãçµæãå ±åããŠãã ãããCONTRIBUTING.md ãåç
§ïŒ
pre-commit install # ä»»æ: ã³ãããæã« ruff ã® lint+format
```
å®å
šãªã¯ãŒã¯ãããŒã¯ [CONTRIBUTING.md](CONTRIBUTING.md) ããè匱æ§ã®å ±å㯠[SECURITY.md](SECURITY.md) ã
åç
§ããŠãã ããããã¬ã¡ããªã¯å®å
šã«ãªããã€ã³ã§ãïŒ`SOUP_TELEMETRY=1`ãããã©ã«ãã¯ãªãã
[ãã©ã€ãã·ãŒããªã·ãŒ](docs/backends-and-ops.md#privacy-policy) ãåç
§ïŒã
## Soupãæ¯æŽãã
Soup 㯠Apache-2.0 ã§ãç¡æã§ã â ãããŠããããããããããç¶ããŸãã4 GB ã®ããŒã PC äžå°ã§ã
ãªãŒãã³ã«éçºã»ä¿å®ãããŠããŸãããããã®ããã¥ã¡ã³ãã«èŒã£ãŠãããã¹ãŠã®æ§èœå€ãã䞻匵ã§ã¯ãªã宿ž¬å€ã§
ããã®ã¯ãã®ããã§ãã
Soup ã®ãããã§åŠç¿ãäžååç¯çŽã§ãããªãã[ãªããžããªã«ã¹ã¿ãŒãä»ããŠ](https://github.com/MakazhanAlpamys/Soup)
ããã ãã®ãæãå©ãã«ãªããè²»çšãããããŸãããéçºã«çŽæ¥è³éãæäŸãããå Žåã¯ã次ã®ãšããã§ãã
**[â€ïž å¯ä»ãã](https://buy.stripe.com/4gMcN441k3pha3T19ye7m04)** â åçºã§ãéé¡ã¯èªç±ã§ãïŒæ±ºæžããŒãžã®
*Change amount*ïŒéé¡ã倿ŽïŒã䜿ã£ãŠãã ããïŒãæ¯æãã¯ãã¡ã³ãããŒã®ç»èšäºæ¥è
ã§ãã **MePlay, Inc.** ã®
å矩㧠Stripe ãåŠçããŸããæ±ºæžããŒãžãã«ãŒãã®å©çšæçްã«è¡šç€ºãããã®ã¯ãSoupãã§ã¯ãªãããã®ååã§ãã
å¯ä»ã¯ãããŒããŠã§ã¢ãå¿
èŠãªäœæ¥ â ãã«ã GPUã8B 以äžã®ã¢ãã«ã®æ€èšŒãApple Silicon â ã®ããã® GPU æéã«
å
ãŠãããŸãã4 GB ã®ããŒã PC äžå°ã§ã¯æãå±ããªãäœæ¥ã§ãã
ãŸãã«ãããã®é
ç®ãåã«é²ããããäžã€ã®æ¹æ³ã¯ã**ããŒããŠã§ã¢ãã®ãã®**ã§ãããããã¯ãæ€èšŒãããŠããªã
䞻匵ã§ã¯ãªããæ£çŽãªã\<ããŒããŠã§ã¢\>ãå¿
èŠããšããã²ãŒãã®èåŸã§æäŸãããŠããŸãããã®ããããã倧ããª
ãã·ã³ãã䜿ãããŠããªã GPU ã¯ã¬ãžããã«ã¢ã¯ã»ã¹ã§ãããªãã
[`help wanted`](https://github.com/MakazhanAlpamys/Soup/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22)
ã® issue ã®ãããããå®è¡ããŠæ°å€ãæçš¿ããããšã¯ãGPU æéã«è³éãæäŸããã®ãšåããããå©ãã«ãªããŸãã
ãããã® issue ã«ã¯ãçŸæç¹ã§ããŒããŠã§ã¢ãåå ã§æ¢ãŸã£ãŠãããã®ãæ£ç¢ºã«æžãããŠããŸãã
## ã³ã³ããªãã¥ãŒã¿ãŒ
ã³ãã¥ããã£ã«ãã£ãŠäœãããŸãã â€ïž â ã³ã³ããªãã¥ãŒã·ã§ã³ããŠãã ãã£ããã¹ãŠã®æ¹ã«æè¬ããŸãã
[CONTRIBUTORS.md](CONTRIBUTORS.md) ãåç
§ããŠãã ããã
[](https://github.com/MakazhanAlpamys/Soup/graphs/contributors)
## ãåãåãã
ãã°ãæ©èœãªã¯ãšã¹ã㯠[issue ãã©ãã«ãŒ](https://github.com/MakazhanAlpamys/Soup/issues) ãžã質åã¯
[Discussions](https://github.com/MakazhanAlpamys/Soup/discussions) ãžãé¡ãããŸããã©ã¡ããåçãæ©ãã
åãåé¡ã«ééããæ¬¡ã®äººã®å©ãã«ããªããŸãã
ã©ã€ããã£ãããã»ããã¢ããã®çžè«ããããŠäŒè©±ã®ã»ããåããŠãããã¹ãŠã®ããšã«ã€ããŠã¯ã
[Discord](https://discord.gg/dgd2pJcjwP) ãŸã㯠[Telegram ã³ãã¥ããã£](https://t.me/souptasters) ã«åå ããŠ
ãã ãããå幎åŸã«ãèŠã€ããããã¹ãããšã¯ Issues ã Discussions ã«æžããŠãã ãã â Discord ã®åçã
å©ããã®ã¯äžäººã§ãããissue ã¯åãããšã«ééãããã¹ãŠã®äººãå©ããŸãã
[è¡åèŠç¯](CODE_OF_CONDUCT.md) ã¯ããã§ãé©çšãããŸãã
å
¬éã®å Žã«é©ããªãããš â ã»ãã¥ãªãã£å ±åïŒ[SECURITY.md](SECURITY.md) ãåç
§ïŒãè¡åèŠç¯ã«é¢ããäºé
ã
å ±éé¢ä¿ â ã«ã€ããŠã¯ã**team@trysoup.dev** ã«ã¡ãŒã«ããŠãã ãããããã¯ãããžã§ã¯ãã®ã¢ãã¬ã¹ã§ã
Soup ã«é¢ããããããä»¶ã®æ£ããé£çµ¡å
ã§ãã**makazanalpamys@gmail.com** ã¯ã¡ã³ãããŒã®å人ã¢ãã¬ã¹ã§ãã
åã人ã«å±ããŸãã®ã§ãäºåã®é£çµ¡å
ãšããŠãå©çšããã ããŸãã
## Soupã®åŒçš
ã¬ã€ã€ãŒã¹ããªãŒãã³ã° â åçµããããŒã¹ã¢ãã«ããã¹ã RAM ãããã³ãŒããŒå±€ãäžå±€ãã€ã¹ããªãŒãã³ã°ããããšã§ã
4 GB ã®ããŒã PC çš GPU ã§ 8B ã¢ãã«ãåŠç¿ãããææ³ â ã¯ããã¬ããªã³ãã§èª¬æãããŠããŸããã¹ããªãŒãã³ã°å®è¡ã
åžžé§å®è¡ãšç
§åããŠæ€èšŒããæ£ç¢ºæ§ãããã³ã«ãå«ãŸããŠããŸãïŒé äŒæãšéäŒæã¯ãäžã€ã®äž»åŒµã§ã¯ãªãäºã€ã®äž»åŒµã§
ãããããå¥ã
ã«è¿°ã¹ãããŠããŸãïŒã
> Makazhan, A. (2026). *Exact Layer Streaming: LoRA Fine-Tuning of an 8B Model on a 4 GB Laptop
> GPU* (v3). Zenodo. https://doi.org/10.5281/zenodo.21918325
**ããŒãžã§ã³ 3ïŒ2026 幎 8 æ 13 æ¥ïŒãææ°çã§ãã** ã¿ã€ãã«ãšäž»åŒµ â 4 GB ã§ 8B â ã¯å€ããããv1 以éã«
枬å®å€ãå€ãã£ããã®ã¯ãããŸãããv3 ãè¡ã£ãŠããã®ã¯ã**ç§ãã¡ãå
¬éããŠãã説æã®æ€å**ã§ãããããã¯
ãã®è«æãäœã®ããã«ããã®ããæãçãèšãè¡šãæ¹æ³ã§ããããŸãã
- **v3 ã§æ€å: ãã¬ã€ã€ãŒã¹ããªãŒãã³ã°ã®åŸé㯠GPU ã§ã¯ãªãããã¹ãããããã€ã¹ãžã®è»¢éã§ããããšãã説æã**
ããã¯ãäžèšã® H100 ã§ã®åçŸããã®*æšè«*ã§ãããå®éã«æž¬å®ãããããšã¯ãããŸããã§ããã8 æ 11 æ¥ã«æž¬å®
ãããšãããå
¬éãããæ§æã§ã¯ããã¯èª€ãã§ããããã¹ãããããã€ã¹ãžã®ãã€ã転éããã¹ãŠåãé€ããŠã
åŸãããã®ã¯ **1.4%** ã«ããããèšç®ã¹ããªãŒã ãã³ããŒãåŸ
ã€æéã¯ã¹ãããã® **0.20%**ãã¹ãããã¯ãã®
ã«ãŒãã®åäžã»ãã·ã§ã³ã§ã® GEMM äžéã® **71.3%** ã§åäœããŠããŸããã¹ããªãŒãã³ã°åºæã®æå€§ã®ã³ã¹ãã¯ã
ã¬ã€ã€ãŒããšã® NF4 ééååã§ã9.8% ã§ã
ïŒ[èšé²](benchmarks/probe-v0.73.0-what-bounds-streaming.md)ïŒããã¹ãŠã®æž¬å®å€ã¯æå¹ãªãŸãŸã§ããåçŸã¯åŒ±ã圢ã§
æ®ããŸã â ãã®å¶çŽã¯äž¡æ¹ã®ãã·ã³ã«å
±éã§ãããGPU ã®èšç®èœåã§ã¯ãããŸããã
- **å
ãšã¯ãŸã£ããç°ãªãããŒããŠã§ã¢ã§ã®åçŸ**ïŒv2 ã§è¿œå ïŒ: RTX 3050 ã§ 119.6 tok/sïŒ952 MHz æïŒãH100 ã§äžå€®å€
113.00 tok/sãããŒã¯ã¯ã©ã¡ãã 3.32 GB ã§ããã©ã¡ãã #331 ã®ä¿®æ£ããåã®ãã®ã§ãè«æãå ±åããŠããã®ã¯ãã®æ°å€ã§ãã
#331 ä¿®æ£åŸã®è¡ïŒ@umran666ã#1103ïŒã¯ã1935â1957 MHz æã« 208.6 tok/sãããŒã¯ 2.40 GB ã§ãåäžã»ãã·ã§ã³ã§ã®
GEMM äžéã«å¯Ÿããå²å㯠59%ïŒä¿®æ£å㯠68%ïŒã§ããèŠããã® 1.74x ã®åäžã¯é«éåã§ã¯ãªãã¯ããã¯ã®å·®ã§ããã
äž¡è¡ã®ããã ã§ã¡ã¢ãªã®äœ¿ããæ¹ãå€ãã£ãŠããããããã®å²åã§ããåãæ¡ä»¶ã§ã®æ¯èŒã§ã¯ãããŸãã
ïŒ[èšé²](benchmarks/gate-v0.72.2-nf4.md)ïŒã
- **ãµã€ã¬ã³ããªåŸé
誀ãã®æ¬ é¥ãçºèŠããä¿®æ£ããŸããã** ã¬ã€ã€ãŒãããçŽ 165 MiB ãè¶
ãã NF4 ã§ã¯ã
é äŒæã¯ãããåäœã§äžèŽãããŸãŸã§ãæå€±æ²ç·ãå¥å
šã«èŠããŸããããåŸé
ã誀ã£ãŠããŸãããåå ã¯äžæµ
ã©ã€ãã©ãªã§ç¹å®ãããããã«å ±åãããŠããŸããä¿®æ£ã¯ãå®éã® 32B ãš 72B ã§ã®ã³ã³ãããŒã«ã«å¯ŸããŠ
ã²ãŒããããŠããŸãã
- **äžå±€ã®ããã¡ãã§ã¯ãªããå®éã®ã¢ãã«ãµã€ãºã§ã®ãããåäœã®äžèŽ**: é äŒæã¯ 0.5B ãã 72B ãŸã§ã
éäŒæã¯ 8B ãš 14B ã§ç¢ºèªããŠããŸãã
- **åŠç¿æžã¿ã¢ãã«ã®å質ãåããŠæž¬å®**ããåžžé§å®è¡ãšåºå¥ãã€ããŸããã§ããã
- **DeepSpeed ãšã®æ¯èŒ** â ç§ãã¡ã«ãšã£ãŠéœåã®ãããªãçµæãå«ããŠ: ZeRO-3 ã®å
«æã®ã«ãŒãã¯ãåžžé§ã§åŠç¿ãã
äžæã®ã«ãŒãããé
ãã
- **å¶éäºé
ã®ã»ã¯ã·ã§ã³ãæžãçŽããŸãã**: v1 ã®åé
ç®ã®ãã¡ãäžã€ãè§£æ¶ãããããã«åã€ãçµã蟌ãŸãã
äžã€ã®æ°ããé
ç®ã远å ãããŸããã
䜿çšããããŒãžã§ã³ãåŒçšããŠãã ããã`10.5281/zenodo.21771064` ã¯ã³ã³ã»ãã DOI ã§ãåžžã«ææ°ããŒãžã§ã³
ïŒçŸæç¹ã§ã¯ v3ïŒã«è§£æ±ºãããŸããv1 ãš v2 ã¯ãããããã®ããŒãžã§ã³ DOI ã§åŒãç¶ãåŒçšã§ããç·šéãããŸãã â
äžèšã®æ€åãæ°ããããŒãžã§ã³ã«ãªã£ãŠããã®ã¯ãç§ãã¡ãäœãããã€äž»åŒµãããã®èšé²ããã®ãŸãŸæ®ãããã§ãã
ãã®æ°å€ã®è£ä»ããšãªã枬å®èšé²ã¯ãã¹ãŠ [`benchmarks/`](benchmarks/) ã«ãããæžããããšããã«å
¬éãããŠ
ããŸã â 倱æã誀ãã ãšå€æããä»®å®ã枬å®ããã®ã¡ã«ç Žæ£ããæ°å€ãå«ããŠã
```bibtex
@misc{makazhan2026exact,
title = {Exact Layer Streaming: LoRA Fine-Tuning of an 8B Model on a 4 GB Laptop GPU},
author = {Makazhan, Alpamys},
year = {2026},
publisher = {Zenodo},
version = {v3},
doi = {10.5281/zenodo.21918325},
url = {https://doi.org/10.5281/zenodo.21918325}
}
```
## ã©ã€ã»ã³ã¹
[Apache-2.0](LICENSE)ãCopyright 2026 Makazhan Alpamys and the Soup contributorsãSoup ãŸãã¯ãã®æ¹å€ã³ããŒãåé
åžããå Žåã¯ãã©ã€ã»ã³ã¹ã®ç¬¬4æ¡ãšç¬¬6æ¡ãé©çšãããŸããã³ããŒããšã« `LICENSE` ãš `NOTICE` ãä¿æãã倿Žãããã¡ã€ã«ã«ãã®æšãæèšããäœåã®åºæãè¿°ã¹ãç¯å²ãè¶
ã㊠Soup ã®åç§°ã䜿çšããªãã§ãã ããã