# ltx-video-mlx

Accelerated [LTX-2.3](https://huggingface.co/Lightricks/LTX-Video) (22B) text-to-video+audio generation and LoRA fine-tuning on Apple Silicon using [MLX](https://github.com/ml-explore/mlx) — no PyTorch required.

Generate 5-10 second videos with synchronized audio from text prompts or input images, running entirely on-device. Adapt the model to new styles or subjects with on-device LoRA training.

## Setup

```bash
git clone https://github.com/appautomaton/ltx-video-mlx.git
cd ltx-video-mlx
```

Install [uv](https://docs.astral.sh/uv/) if you haven't, then download the model weights (see [Model Weights](#model-weights) below).

Dependencies are installed automatically on first `uv run`.

## Quick Start

```bash
# Text-to-video
uv run python generate.py "A serene mountain lake at sunrise, golden light reflecting off calm water as thin mist drifts across the surface. Tripod-locked camera, live action, 4K."

# Image-to-video
uv run python generate.py -i input/photo.jpg "The person slowly turns and smiles at the camera"

# With options
uv run python generate.py -f 121 -b 8 --upscale "your prompt here"
```

Output is saved to `output/<timestamp>/` with `video.mp4`, `audio.wav`, and individual frames.

## Features

- **Text-to-video (T2V)** and **image-to-video (I2V)** generation
- Joint audio+video through 48-layer DiT transformer (22B params)
- 8-bit / 4-bit quantized inference via MLX `quantized_matmul`
- 8-step distilled Euler diffusion (LoRA-fused)
- 48kHz stereo audio (BigVGAN v2 vocoder + bandwidth extension)
- Optional 2x spatial upscaler
- Aspect-ratio-aware resolution snapping for I2V
- **LoRA fine-tuning** — on-device video style/subject adaptation, no PyTorch required

## Requirements

- Apple Silicon Mac (M-series, M1 or later)
- macOS with Metal support
- Python 3.12+, [uv](https://docs.astral.sh/uv/)
- ffmpeg
- ~14 GB unified memory (8-bit) or ~10 GB (4-bit)

## Model Weights

Download from HuggingFace and place under `models/`:

| Model | Source | Path |
|-------|--------|------|
| LTX-2.3 FP8 (29 GB) | [Lightricks/LTX-2.3-fp8](https://huggingface.co/Lightricks/LTX-2.3-fp8) | `models/LTX-2.3-fp8/ltx-2.3-22b-dev-fp8.safetensors` |
| Distilled LoRA (7.6 GB) | [Lightricks/LTX-2.3](https://huggingface.co/Lightricks/LTX-2.3) | `models/LTX-2.3/ltx-2.3-22b-distilled-lora-384.safetensors` |
| Spatial Upscaler 2x (1 GB) | [Lightricks/LTX-2.3](https://huggingface.co/Lightricks/LTX-2.3) | `models/LTX-2.3/ltx-2.3-spatial-upscaler-x2-1.0.safetensors` |
| Gemma 3 12B (~24 GB) | [google/gemma-3-12b-pt](https://huggingface.co/google/gemma-3-12b-pt) | `models/gemma-3-12b/` |

```bash
# Download with huggingface-cli
huggingface-cli download Lightricks/LTX-2.3-fp8 --local-dir models/LTX-2.3-fp8
huggingface-cli download Lightricks/LTX-2.3 --local-dir models/LTX-2.3
huggingface-cli download google/gemma-3-12b-pt --local-dir models/gemma-3-12b
```

## Fine-tuning

Adapt the model to new video styles or subjects using LoRA:

```bash
# Step 1 — precompute text/video embeddings from your dataset
uv run python train.py precompute --data-dir input/my_dataset/ --model-path models/gemma-3-12b/

# Step 2 — train LoRA adapter
uv run python train.py train \
  --model-path models/LTX-2.3-fp8/ltx-2.3-22b-dev-fp8.safetensors \
  --data-root input/my_dataset/.precomputed/768x512_25f_crop \
  --output-dir output/my_lora \
  --steps 1000 --rank 32 --bits 8

# Step 3 — generate with trained adapter
uv run python generate.py --lora output/my_lora/adapter-final.safetensors "your prompt"
```

See [docs/finetune.md](docs/finetune.md) for the full guide.

## CLI Reference

```
uv run python generate.py [prompt] [options]

  -p, --prompt-flag   Text prompt (flag form)
  -i, --image         Input image for I2V (jpg/jpeg/png)
  --strength          I2V conditioning strength 0.0-1.0 (default: 0.95)
  -f, --frames        Frame count, must be 8k+1 (default: 121)
  -H, --height        Height, divisible by 32 (default: 512)
  -W, --width         Width, divisible by 32 (default: 768)
  -b, --bits          Quantization: 4 or 8 (default: 8)
  -s, --seed          Random seed
  --fps               Frames per second (default: 25)
  --no-audio          Skip audio generation
  --upscale           2x spatial upscale
  --lora              Path to LoRA adapter (.safetensors)
  -o, --output        Output directory (default: output/)
```

## Docs

- [Usage & Examples](docs/usage.md)
- [Fine-tuning (LoRA)](docs/finetune.md)
- [Performance](docs/performance.md)
- [Architecture](docs/architecture.md)
- [Quantization](docs/quantization.md)

## License

Research use. Model weights are subject to [Lightricks LTX-Video license](https://huggingface.co/Lightricks/LTX-Video/blob/main/LICENSE).