# SenseNova-U1.5-8B-MoT

English | 简体中文

SenseNova-U1.5-8B-MoT is the U1.5 checkpoint used throughout this repository. It uses the NEO-unify architecture for text-to-image generation, image editing, interleaved text-image generation, and visual understanding. The [U1.5 paper](https://arxiv.org/abs/2609.11929) describes its spatial reconstruction, native resolutions up to 4K, and improvements in text rendering and image editing. ## Checkpoints - **U1.5 checkpoint:** [Hugging Face](https://huggingface.co/sensenova/SenseNova-U1.5-8B-MoT) · [ModelScope](https://modelscope.cn/models/SenseNova/SenseNova-U1.5-8B-MoT). Use this checkpoint for the examples in this repository. - **U1.5 Preview:** [Preview guide](u1.5_preview.md). Preview uses separate weights; use its matching model card and examples. The [model catalog](models.md) lists the rest of the U1 and U1.5 releases. ## Run U1.5 For one GPU, start with `--vram_mode balanced`. Plan for **24 GB of GPU memory and 64 GB of system RAM** on a mostly idle machine. In the 8-step runs on a 32 GB RTX 5090, `balanced` peaked at 15.17 GiB of PyTorch reserved GPU memory; process CPU RSS reached about 52.2 GiB with `balanced` and `low`. Set up the shared Python environment with the [installation guide](installation.md). From the repository root, download U1.5 and generate a first image: ```bash hf download sensenova/SenseNova-U1.5-8B-MoT \ --local-dir ./models/SenseNova-U1.5-8B-MoT python examples/t2i/inference.py \ --model_path ./models/SenseNova-U1.5-8B-MoT \ --prompt "A cinematic mountain lake at sunrise, realistic photography." \ --width 2048 --height 2048 --num_steps 50 \ --vram_mode balanced --attn_backend sdpa \ --output output.png ``` The [examples](../examples/README.md) contain the full CLI reference. The shortest U1.5 commands for the other supported tasks are below. ### Image editing ```bash python examples/editing/inference.py \ --model_path ./models/SenseNova-U1.5-8B-MoT \ --prompt "Change the jacket of the person on the left to bright yellow." \ --image examples/editing/data/images/1.webp \ --num_steps 50 --vram_mode balanced --attn_backend sdpa \ --output edited.png ``` ### Interleaved text and image generation ```bash python examples/interleave/inference.py \ --model_path ./models/SenseNova-U1.5-8B-MoT \ --prompt "Give me an illustrated tutorial for making tomato and egg stir-fry." \ --resolution "16:9" --max_images 1 \ --vram_mode balanced --attn_backend sdpa \ --output_dir outputs/interleave --stem tutorial ``` ### Visual question answering ```bash python examples/vqa/inference.py \ --model_path ./models/SenseNova-U1.5-8B-MoT \ --image examples/vqa/data/images/menu.jpg \ --question "Which dishes would make a balanced meal for two people?" \ --vram_mode balanced --attn_backend sdpa \ --output outputs/menu_answer.txt ``` ### 8-step V2 LoRA The [V2 LoRA adapter](https://huggingface.co/sensenova/SenseNova-U1.5-8B-MoT-LoRAs/blob/main/SenseNova-U1.5-8B-MoT-LoRA-8step-V2.safetensors) runs with **U1.5**; it is not a standalone checkpoint or an adapter for Preview. Download it, then pass the local file to `--lora_path`: ```bash hf download sensenova/SenseNova-U1.5-8B-MoT-LoRAs \ SenseNova-U1.5-8B-MoT-LoRA-8step-V2.safetensors \ --local-dir ./models/SenseNova-U1.5-8B-MoT-LoRAs python examples/t2i/inference.py \ --model_path ./models/SenseNova-U1.5-8B-MoT \ --lora_path ./models/SenseNova-U1.5-8B-MoT-LoRAs/SenseNova-U1.5-8B-MoT-LoRA-8step-V2.safetensors \ --prompt "A cinematic mountain lake at sunrise, realistic photography." \ --width 2048 --height 2048 --num_steps 8 --cfg_scale 1.0 \ --vram_mode balanced --attn_backend sdpa --output output_lora.png ``` For editing and interleaved generation, use the [examples](../examples/README.md) with the same LoRA path, eight steps, and CFG 1.0. ## Resolution The quick start uses 2048 × 2048. U1.5 supports native resolutions up to 4K; when setting an explicit size, pass both `--width` and `--height` as multiples of 32. Editing derives an output size from the first input image when those flags are omitted. The [inference record](u1.5_inference.md) lists the measured settings. ## Visual examples The [U1.5 showcases](u1.5_showcases.md) pair generated and edited images with their prompts. [Prompt enhancement](u1.5_best_practices.md) is available for requests that need more preparation. ## Evaluations and runtime See the [benchmark overview](assets/benchmarks/u1.5_radial.webp), [detailed results](assets/benchmarks/u1.5_combined.webp), and [performance versus speed figure](assets/u15_perform_vs_speed_6bench.webp) for the reported evaluations. The [U1.5 memory and 8-step timing results](u1.5_inference.md) cover RTX 5090 runs and allocator-cap tests on a shared H100. They include text-to-image, editing, and one-image interleaved generation. ## Known limitations U1.5 can make mistakes in dense, small, or mixed Chinese-English text. Layouts that require exact counts, alignment, or hierarchy also need checking, as do small faces, hands, and limbs. Broad, multi-turn, or multi-reference edits may change regions that were meant to stay intact. Some prompts produce excessive detail or color; try lowering `cfg_scale` gradually when that happens. For architecture and training details, read the [U1.5 technical report](pdf/SenseNOVA_U1_5.pdf) or the [paper](https://arxiv.org/abs/2609.11929).