# Fine-tune MiniCPM5-1B and MiniCPM5-2B with unsloth [unsloth](https://github.com/unslothai/unsloth) is the "single-GPU LoRA on a budget" framework β€” heavy custom kernels for memory savings, and a wrapper around TRL `SFTTrainer` for the actual training loop. MiniCPM5-1B and MiniCPM5-2B can use the same recipe; the **2Γ— memory reduction** is useful for long-context fine-tuning on a consumer GPU. > πŸ”‘ If you're using unsloth alongside vLLM in the same env, pin `transformers==4.57.3` β€” unsloth's vLLM coexistence patch only handles transformers ≀ 4.57.x. Recipe baked in below. ## Install ```bash pip install "unsloth==2026.5.2" # or latest: pip install unsloth ``` > ⚠️ unsloth pulls in `transformers>=5.0`, but their `import_fixes.py` then **explicitly refuses to load** if vLLM is in the same env and `transformers>=5.0` (because vLLM ≀ 0.10.x doesn't yet support transformers 5.x). The fix is to pin `transformers==4.57.3`: > ```bash > pip install --force-reinstall "transformers==4.57.3" > ``` > > If you don't have vLLM in the same env, you can keep `transformers>=5.0` and unsloth will skip the vLLM patch. ## Recipe ```python import json import torch from datasets import Dataset from unsloth import FastLanguageModel from trl import SFTTrainer, SFTConfig BASE = "openbmb/MiniCPM5-2B" # or openbmb/MiniCPM5-1B DATA = "/path/to/my_chat_data.jsonl" OUT = "./runs/minicpm5_unsloth" # 1. Load with unsloth's fast wrapper (this auto-patches the model) model, tok = FastLanguageModel.from_pretrained( model_name=BASE, max_seq_length=2048, dtype=torch.bfloat16, load_in_4bit=False, # set True for QLoRA on consumer GPUs full_finetuning=False, ) # 2. Add LoRA model = FastLanguageModel.get_peft_model( model, r=16, target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"], lora_alpha=32, lora_dropout=0.05, bias="none", use_gradient_checkpointing="unsloth", # 30 % less VRAM than vanilla GC random_state=42, ) # 3. Data β€” unsloth's SFTTrainer expects either `text` column or formatting_func rows = [json.loads(l) for l in open(DATA, encoding="utf-8") if l.strip()] ds = Dataset.from_list([ {"text": tok.apply_chat_template(r["messages"], tokenize=False)} for r in rows ]) if tok.pad_token is None: tok.pad_token = tok.eos_token # 4. Train (TRL SFTTrainer under the hood, with unsloth patches) trainer = SFTTrainer( model=model, args=SFTConfig( output_dir=OUT, dataset_text_field="text", num_train_epochs=2, per_device_train_batch_size=4, gradient_accumulation_steps=4, learning_rate=2e-4, warmup_ratio=0.03, lr_scheduler_type="cosine", bf16=True, max_length=2048, packing=False, logging_steps=10, save_steps=200, seed=42, report_to="none", dataloader_num_workers=2, ), train_dataset=ds, processing_class=tok, ) trainer.train() trainer.model.save_pretrained(f"{OUT}/adapter_final") ``` ## Sample output ``` ==((====))== Unsloth 2026.5.2: Fast Llama patching. Transformers: 4.57.3. \\ /| NVIDIA GPU. Num GPUs = 1. O^O/ \_/ \ Torch: 2.7.1+cu126. CUDA: 9.0. CUDA Toolkit: 12.6. \ / Bfloat16 = TRUE. Trainable parameters = model-size dependent (about 1% with the recipe below) πŸ¦₯ Unsloth: Padding-free auto-enabled, enabling faster training. {'loss': 4.6738, 'epoch': 0.2} {'loss': 3.9919, 'epoch': 0.4} {'loss': 3.5176, 'epoch': 0.6} {'loss': 3.4641, 'epoch': 0.8} {'loss': 3.5212, 'epoch': 1.0} {'train_runtime': 26.6, 'train_loss': 3.83} ``` Loss 4.67 β†’ 3.52 over 25 optimizer steps (1.03 % trainable). The first iteration is slow (~13 s) because unsloth JIT-compiles patched kernels; subsequent iters are < 600 ms each. ## QLoRA (Int4) for consumer GPUs ```python model, tok = FastLanguageModel.from_pretrained( model_name=BASE, max_seq_length=4096, dtype=torch.bfloat16, load_in_4bit=True, # ← 4-bit base for QLoRA full_finetuning=False, ) ``` With `load_in_4bit=True`, MiniCPM5-1B or MiniCPM5-2B + LoRA can run on a low-VRAM consumer GPU for QLoRA at 4K context. ## Inference with the LoRA adapter ```python from unsloth import FastLanguageModel model, tok = FastLanguageModel.from_pretrained( model_name="./runs/minicpm5_unsloth", # path to adapter max_seq_length=4096, dtype=torch.bfloat16, load_in_4bit=False, ) FastLanguageModel.for_inference(model) # 2Γ— faster generation inputs = tok.apply_chat_template( [{"role": "user", "content": "用一ε₯θ―θ§£ι‡Š GQA。"}], add_generation_prompt=True, enable_thinking=True, return_tensors="pt", ).to("cuda") out = model.generate(inputs, max_new_tokens=512, temperature=1.0, top_p=0.95) print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True)) ``` To merge for general serving: ```python model.save_pretrained_merged("./minicpm5-unsloth-merged", tok) ``` ## Q&A ### `RuntimeError: vLLM with version 0.10.x does not yet support transformers>=5.0.0` unsloth pulls in `transformers>=5.0` but checks for vLLM compatibility on import. Pin `transformers==4.57.3`: ```bash pip install --force-reinstall "transformers==4.57.3" ``` ### `Unsloth: You must specify a formatting_func` unsloth's TRL wrapper expects a flat `text` column or an explicit `formatting_func`. The recipe above uses `tok.apply_chat_template(...)` to convert messages β†’ text up-front and sets `dataset_text_field="text"` in `SFTConfig`. ### Compared to LLaMA-Factory / ms-swift / TRL-direct | Scenario | Pick | | --- | --- | | Single 8-24 GB consumer GPU + QLoRA | **unsloth** (2Γ— memory savings, JIT kernels) | | Multi-GPU + DeepSpeed | LLaMA-Factory or ms-swift | | WebUI / YAML pipeline | LLaMA-Factory | | Bare-metal Python + assistant-only loss | TRL direct (see [`trl.md`](./trl.md)) | For these model sizes, unsloth's headline 2Γ— speedup depends on the GPU and sequence length. The memory savings are still useful for long-context training (16K-128K). ## See also - [`llamafactory.md`](./llamafactory.md), [`ms_swift.md`](./ms_swift.md), [`trl.md`](./trl.md), [`xtuner.md`](./xtuner.md) β€” alternative frameworks