# PSD2Live

[中文](README.md) | [日本語](README_ja.md)

PSD2Live is an automated Live2D model generation pipeline and desktop application. Given a layered PSD file, the system automatically performs semantic layer recognition, 8-connected bilateral splitting, adaptive Delaunay mesh triangulation, 9-pose facial lattice construction, multi-pendulum hair dynamics, and seamless idle loop generation, exporting both editable `.cmo3` editor projects and runtime `.moc3` file families.

> [!IMPORTANT]
> **Want to use the desktop app directly?** Windows 10/11 x64 users can download an executable from [Releases](https://github.com/tsunehimatoi/psd2live/releases/latest). Extract and run the portable ZIP, or choose the EXE or MSI installer. These packages include a Java runtime, so no source-build environment is required.

<p align="center">
  <img src="docs/imgs/use.gif" alt="PSD2Live Workflow Demo" />
  <br>
  <em>End-to-end automated modeling, real-time gaze tracking, and dynamic preview</em>
</p>

---

## Documentation Index

| Document | Description |
| :--- | :--- |
| [User Guide (docs/en/USER_GUIDE.md)](docs/en/USER_GUIDE.md) | Desktop GUI, version-history tree, independent log dock, Agent/MCP connection, shortcuts, and CLI reference |
| [Agent / MCP Product & Technical Design (Chinese)](docs/zh/AGENT_ARCHITECTURE.md) | Implemented MCP tools, persistent workspace/history model, image workflow, and delivery roadmap |
| [Live2D SDK Setup Guide (docs/en/CUBISM_SDK_SETUP.md)](docs/en/CUBISM_SDK_SETUP.md) | Official Native SDK license policy, shader extraction, and hardware-accelerated preview setup |
| [PSD Layer Specification (docs/en/PSD_LAYER_SPEC.md)](docs/en/PSD_LAYER_SPEC.md) | 31 semantic tags, side resolution rules, connected-component splitting, and layering guidelines |
| [Deformer & Math Specification (docs/en/DEFORMER_AND_PARAMETER_SPEC.md)](docs/en/DEFORMER_AND_PARAMETER_SPEC.md) | Deformer tree topology, 9-pose facial lattice math, C1 roll curve, feature warps, and physics |
| [Implementation Comparison (docs/en/IMPLEMENTATION_COMPARISON.md)](docs/en/IMPLEMENTATION_COMPARISON.md) | Technical comparison across 16 pipeline stages, coordinate invariants, and integrity verification |

---

## Core Features

- **Adaptive Mesh Generation**: Separable Gaussian alpha pre-filtering and adaptive binarization; periodic cubic Bézier fitting with physical support window corner detection and curvature-weighted adaptive resampling (up to 12x); constrained Delaunay triangulation with topology-convergent Lawson flips and overlong internal edge bisection.

  <p align="center">
    <img src="docs/imgs/mesh22.png" width="32%" alt="22 px high-density adaptive mesh" />
    <img src="docs/imgs/mesh64.png" width="32%" alt="64 px balanced adaptive mesh" />
    <img src="docs/imgs/mesh115.png" width="32%" alt="115 px low-density adaptive mesh" />
    <br>
    <em>Mesh spacing at 22 / 64 / 115 px: density comparison from detailed contours to lightweight topology</em>
  </p>
- **Deformer (Warp) Generation**:
  - **Eye & Mouth Deformation**: Shared projective plane constraints for eyes and brows, iris counter-translation against perspective compression, and eyelash alpha-weighted centerline tracking for smooth closed U-curves; centripetal compression of full-open mouth toward central seam with auto-clipped teeth and tongue.
  - **Nine-Pose Lattice Construction**: `AngleX (±45°) × AngleY (±30°)` 8×8 facial lattice combining C1-continuous horizontal roll (near-side reveal, broad plateau preservation, far-side compression), vertical V/^ pitch curvature, and diagonal $C_{xy} = \text{yaw} \times \text{pitch}$ cross-terms.
- **Animation**: Automated generation of a 6-second seamless looping `idle.motion3.json` covering breathing, subtle head/body sway, and natural eye blinks; desktop GUI supports optional integration with official Cubism 5-r.5 SDK native offscreen OpenGL rendering for **100% official rendering & physical dynamics parity (Ground Truth)** (this project does NOT include or redistribute proprietary SDK binaries, see [SDK Setup Guide](docs/en/CUBISM_SDK_SETUP.md); automatically falls back to pure CPU high-precision software rasterization when SDK is absent) with live mouse gaze tracking (Mouse Look).
- **Physics**: Decoupled front and back hair following the head container with root-pinned, $v^3$ cubic tip sway multi-pendulum dynamics; eyelid closure velocity driving second-order damped harmonic oscillators for pupil jelly squash/stretch dynamics (`ParamEyeBallForm`).
- **Editable Agent / MCP Workspace**: A bearer-authenticated local Streamable HTTP MCP lets ChatGPT/Codex, Gemini/Antigravity, and other MCP hosts inspect the project, render spatially reversible PNG Views, import transparent assets, manage parameters, and edit multidimensional keyforms on meshes, warp/rotation deformers, parts, and glue. Every mutation enters a persistent, append-only branch history with resumable task checkpoints.
- **Project & Runtime Export**: Synchronized one-click export of editable Live2D Cubism Modeler 5 `.cmo3` projects and `.moc3` runtime families (`.model3.json`, `.cdi3.json`, `physics3.json`, `idle.motion3.json`, and texture atlases); enforced three-stage geometric integrity gates (neutral pose fidelity, extreme angle bounds, and warp lattice mirror symmetry).

<p align="center">
  <img src="docs/imgs/agent.png" alt="PSD2Live AI Agent asset generation, integration, and multi-parameter render workflow" />
  <br>
  <em>An example of an Agent adding a hair clip: reading Skills and MCP tools, inspecting the model, generating and adding the asset, and checking other parameter poses. This example does not establish that the more complex tasks below are available.</em>
</p>

### Agent Capabilities and Implementation Status

#### Available

- Add other hair accessories or decorations, then check occlusion, placement, and deformation across multiple parameter poses.

#### Theoretically feasible, but Agents cannot get reliable results yet — pending implementation

> [!WARNING]
> **The following tasks are theoretically feasible, but Agents cannot reliably configure and complete them. End-to-end execution is extremely unstable, and these capabilities remain pending implementation. Unless you are debugging the program, we recommend not attempting them.** Existing MCP interfaces do not mean an Agent can complete a task; repeated generation, positioning, and correction can quickly consume large amounts of tokens and image-generation quota without producing a usable result.

Expected implementation difficulty increases in the following order:

1. Generate expression and action variants with additional parameter or animation controls, such as an `@v@` expression, waving, or crossing both arms.
2. Separate the mouth into independently editable lips, inner mouth, teeth, and tongue layers.
3. Separate the hair into front, side, back, ahoge, or other independently riggable strands, and reconstruct hidden regions.
4. Add shadows, including generating and rigging hair-shadow layers and side-of-face shadow layers.

#### Currently beyond the Agent's capabilities

- Deform parts plausibly and precisely: this requires the AI to manipulate individual Warp/Mesh points and control their deformation correctly. Agents cannot yet perform this reliably.

> [!IMPORTANT]
> These workflows require both the selected model and the Agent harness to expose a working image-generation capability. A model that can only understand text or images, but cannot generate and return an image, cannot complete asset creation and import.

Results depend on the model, image generator, Agent harness, prompt, and the quality of the original PSD layer separation. Passing regression tests for the underlying tools does not establish the reliability of the pending tasks above.

**Help wanted: Pull Requests for prompt engineering and Agent workflows.** We especially need contributors with experience in tool discovery and selection, depth and occlusion reasoning for part separation, generation constraints, positioning and correction workflows, token budgets, and stopping conditions. Reproducible cases, effective prompt or Skill improvements, workflow implementations, and evaluation cases are welcome. Where possible, include the model and host used, actual consumption, and both successful and failed results so we can assess completion rates and reduce wasted retries. A single successful demonstration is not enough to mark these capabilities as complete.

---

## Quick Start

### Prerequisites
- **Java Runtime**: Windows packages downloaded from Releases include a runtime. JDK 21 or higher is required only when building or launching from source with Gradle.
- **Operating System**: Windows 10/11 x64 (supports 100% pixel-perfect official rendering & physics parity when configured with official Native SDK), Linux / macOS (software rasterization)
- **Live2D Official SDK Notice**: Source code and release packages **do NOT include or redistribute** official Live2D proprietary SDK binaries. Full pipeline generation and CPU preview work 100% out of the box. To enable official runtime consistency verification on Windows, please refer to the [Live2D SDK Setup Guide](docs/en/CUBISM_SDK_SETUP.md).

### Launching the Desktop GUI

- **Windows Quick Launch**: Run `run-gui.bat` in the repository root.
- **Gradle Launch**:
  ```powershell
  # Windows
  .\gradlew.bat run

  # Linux / macOS
  ./gradlew run
  ```

#### Common Shortcuts

| Action | Shortcut / Gesture |
| :--- | :--- |
| **Zoom** | Mouse Wheel (`0.05x ~ 64.0x`) |
| **Pan** | Middle Click Drag / Left Click Blank Drag |
| **Center & Fit** | `F` / `Home` / `0` |
| **Select Mesh** | Left Click on Mesh |
| **Open PSD** | `Ctrl + O` |
| **Reanalyze** | `Ctrl + R` |
| **Generate & Export** | `Ctrl + G` |
| **Export To...** | `Ctrl + Shift + G` |

#### Connecting an AI Agent / MCP Host

1. Keep the PSD2Live desktop app running and open **Agent / MCP → Agent / MCP Connection & Prompts…**.
2. Copy the matching configuration: HTTP TOML for ChatGPT desktop/Codex, or HTTP JSON for Gemini/Antigravity. Other Streamable HTTP hosts use the displayed endpoint and `Authorization: Bearer <token>` header; do not change it to the legacy `/sse` endpoint.
3. Use the Stdio JSON fallback only for hosts without HTTP MCP support. It runs the repository-root `mcp_proxy.py` with Python 3 and reads `PSD2LIVE_MCP_TOKEN`; on Windows it can also read the token saved by PSD2Live.
4. For domain workflows, install `.agent/skills/psd2live-rigging` and `.agent/skills/hair-separation` in the host's documented skill directory. List tools and call `project_get_state` first.

The MCP currently exposes project/layer/parameter reads, object and keyform editing, parameter CRUD, model-data PNG Views, transparent-asset import, soft deletion, resumable tasks, and append-only branch history. Every project edit that advances `HEAD` must use the latest `expected_history_head_node_id`. After a timeout or disconnect, inspect `project_get_state` and `history_list` before deciding whether to retry.

PSD2Live provides Views, spatial mapping, and asset import; it does not expose a host-private image generator. For painted differences, part separation, occlusion completion, or pixel reconstruction, actually call Nano Banana Pro/NBP, GPT Image 2 (`gpt-image-2`), or an equivalent host-native image tool, then import the transparent PNG with `asset_import_png`. Do not draw substitutes with Python, PIL/OpenCV, SVG, or Canvas. See the [User Guide](docs/en/USER_GUIDE.md) for the complete workflow.

---

### Command Line Interface (CLI)

```powershell
# Basic export
.\gradlew.bat run --args="--input ./sample.psd --output ./output"

# Advanced configuration
.\gradlew.bat run --args="--input ./sample.psd --output ./output --atlas 8192 --mesh-spacing 48 --head-strength 1.2 --lang en"
```

| Option | Type | Default | Description |
| :--- | :---: | :---: | :--- |
| `--input <path>` | Path | *(Required)* | Input PSD file path |
| `--output <path>` | Path | `PSD_DIR/psd2live-output` | Destination output directory |
| `--lang <zh\|en\|ja>` | String | System Locale | UI and log language (`zh` / `en` / `ja`) |
| `--atlas <size>` | Int | `4096` | Texture atlas square dimension (`256 ~ 16384`) |
| `--mesh-spacing <px>` | Int | `64` | Base mesh sampling spacing in pixels |
| `--head-strength <val>`| Float | `1.0` | 9-pose facial deformer strength multiplier |
| `--body-strength <val>`| Float | `1.0` | Body kinematics and breath strength multiplier |
| `--no-physics` | Flag | `false` | Disables `physics3.json` and CMO3 physics injection |
| `--no-cmo3` | Flag | `false` | Skips `.cmo3` project export |
| `--no-moc3` | Flag | `false` | Skips `.moc3` runtime export |

---

## PSD Naming Reference

> [!TIP]
> **Key PSD Artwork & Composition Guidelines**:
> - **Mouth open with stroke outlines preferred**: Author the mouth in a fully open state; clean outline strokes along the lip contour ensure clean fusion into a crisp seam upon centripetal closure.
> - **Eyelashes upper half only**: The `eyelash` layer must strictly contain upper eyelashes (no lower lashes) to ensure proper U-shaped blink curve morphing.
> - **Initial head tilt supported**: Initial character head tilt is permitted; the pipeline automatically estimates this initial angle and uses it as the neutral origin to calibrate rotation limits.
> - **Body must remain upright (excessive tilt unsupported)**: Kinematics and breathing rely on a vertical canvas frame; severely tilted or reclining poses are unsupported.
> 
> See [PSD Layer Specification (docs/en/PSD_LAYER_SPEC.md)](docs/en/PSD_LAYER_SPEC.md) for full rules.

| Component | Recommended English | Aliases (ZH / JA) | Behavior |
| :--- | :--- | :--- | :--- |
| **Hair** | `front hair`, `back hair` | 前发, 后发, 前髪, 後ろ髪 | Head-follow Warp + $v^3$ tip multi-pendulum physics |
| **Face** | `face`, `facedetail` | 脸, 脸部, 顔, 肌, チーク | Facial baseline and details |
| **Eyes** | `eyewhite`, `eyelash`, `irides`, `eye_close` | 眼白, 睫毛, 瞳孔, 闭眼, 目, 瞳 | Auto bilateral split, iris clipping, upper-lash smooth U-curve closure |
| **Brows** | `eyebrow` | 眉, 眉毛, まゆ | Auto bilateral split and projective plane linkage |
| **Nose** | `nose` | 鼻, 鼻子 | Maximum perceived 3D depth displacement |
| **Mouth** | `mouth`, `mouth_open` | 嘴, 口, 张嘴, 口開き | Full open reference art (strokes preferred); centripetal compression |
| **Oral Parts** | `tooth-t`, `tooth-b`, `tongue` | 上牙, 下牙, 舌头, 歯, 舌 | Optional components; auto-clipped by mouth |
| **Ears** | `ears` | 耳, 耳朵 | Negative depth shift and far-side opacity attenuation |
| **Body** | `neck`, `topwear`, `bottomwear`, `legwear` | 脖子, 上衣, 裤子, 裙子, 服 | Body kinematics, tilt, and breathing (upright torso required) |
| **Accessories** | `headwear`, `earwear`, `neckwear`, `tail`, `wings` | 头饰, 耳饰, 项链, 尾巴, 翅膀 | Parented to respective containers |

---

## Deformer Hierarchy & Parameters

```text
Root (Canvas Space)
 └─ DeformBodyXY (ParamBodyAngleX, ParamBodyAngleY)
     └─ DeformBodyZBreath (ParamBodyAngleZ, ParamBreath)
         └─ DeformHeadRotation (ParamAngleZ)
             └─ DeformHeadContainer (ParamAngleX, ParamAngleY Skull Follow)
                 ├─ DeformFaceNinePose (ParamAngleX, ParamAngleY 9-Pose Lattice)
                 │   ├─ Eye / Iris / Brow / Nose / Mouth / Ear
                 │   └─ FaceDetails
                 ├─ HairFrontFollow → HairFrontPhysics (ParamHairFront)
                 ├─ HairBackFollow  → HairBackPhysics  (ParamHairBack)
                 └─ HeadAccessories
```

| Parameter ID | Name | Range | Default | Purpose |
| :--- | :--- | :---: | :---: | :--- |
| `ParamAngleX` / `Y` / `Z` | Head Angle X / Y / Z | `[-45..45]` / `[-30..30]` / `[-30..30]` | `0` | Head yaw, pitch, and planar rotation |
| `ParamEyeLOpen` / `ROpen` | Left / Right Eye Open | `[0, 1]` | `1` | Smooth eyelash U-curve, iris clipped by eye-white |
| `ParamEyeBallX` / `Y` | EyeBall X / Y | `[-1, +1]` | `0` | Gaze tracking offset |
| `ParamEyeBallForm` | EyeBall Form | `[-1, +1]` | `0` | Blink-driven 2nd order jelly dynamics |
| `ParamBrowLY` / `RY` | Brow Left / Right Y | `[-1, +1]` | `0` | Brow vertical displacement |
| `ParamMouthForm` | Mouth Form | `[-1, +1]` | `0` | Corner curvature and width |
| `ParamMouthOpenY` | Mouth Open | `[0, 1]` | `0` | Full open -> center seam closure interpolation |
| `ParamBodyAngleX` / `Y` / `Z`| Body Angle X / Y / Z | `[-10, +10]` | `0` | Torso yaw Roll, S-curve pitch, and tilt |
| `ParamBreath` | Breath | `[0, 1]` | `0` | Gaussian chest expansion |
| `ParamHairFront` / `Back` | Hair Front / Back | `[-1, +1]` | `0` | Multi-pendulum hair dynamics |

---

## Output Bundle Structure

```text
output_dir/
├── sample.cmo3                    # Editable Live2D Modeler 5 project
├── sample.moc3                    # Runtime model binary (MOC5 baseline)
├── sample.model3.json             # Runtime configuration (textures, physics, motion)
├── sample.cdi3.json               # Display names metadata
├── sample.physics3.json           # Physics configuration (hair pendulums + eye jelly)
├── sample.idle.motion3.json       # 6-second seamless looping idle motion
├── sample.4096/texture_00.png     # Texture atlas page
└── sample.psd2live.json         # Diagnostic report and mapping metadata
```

---

## Build & Verification

```powershell
# Compile and assemble standalone distribution ZIP
.\gradlew.bat clean test distZip

# Execute unit and integration tests
.\gradlew.bat test
```

---

## License & Attribution

- **License**: [GNU General Public License v3.0 (GPL-3.0)](LICENSE).
- **Third-Party Attribution**: Integrates core modules from [Umamo](THIRD_PARTY_NOTICES.md), with algorithm and semantic inspiration from [Stretchy Studio](THIRD_PARTY_NOTICES.md). See [`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md).

---

## Disclaimer

- PSD2Live is an independent open-source project and is not affiliated with, endorsed by, or sponsored by Live2D Inc. or its affiliates.
- Names and file extensions such as `Live2D`, `Cubism`, `.cmo3`, and `.moc3` are used solely for format interoperability and compatibility descriptions. All trademarks and intellectual property rights belong to their respective holders. This project does not contain or redistribute the official proprietary Live2D Cubism SDK.
- This software is provided "as is". Users should maintain backups of original PSD assets and inspect generated output in target applications prior to production use.