# 🦜 VieNeu-TTS
[](https://github.com/keon/awesome-nlp)
[](https://discord.gg/yJt8kzjzWZ)
[](https://colab.research.google.com/drive/1b9PO-lcGZX9pEkEwQmu8MfhSnjxKrALW?usp=sharing)
[](https://huggingface.co/pnnbao-ump/VieNeu-TTS-v3-Turbo)
[](https://huggingface.co/pnnbao-ump/VieNeu-TTS-v2)
[](https://huggingface.co/pnnbao-ump/VieNeu-TTS)
**VieNeu-TTS-v2** is the next generation of on-device Vietnamese TTS, featuring **10,000+ hours** of bilingual training, **instant voice cloning**, and a dedicated **Podcast/Conversation** mode.
> [!NOTE]
> **🆕 VieNeu-TTS v3 Turbo (early access) is out for preview!**
> A brand-new architecture **designed and trained from scratch by Phạm Nguyễn Ngọc Bảo** (codec: [MOSS-Audio-Tokenizer-Nano](https://huggingface.co/OpenMOSS-Team/MOSS-Audio-Tokenizer-Nano); phonemizer: [sea-g2p](https://github.com/pnnbao97/sea-g2p)):
> - **48 kHz** high-fidelity audio (up from 24 kHz).
> - **Built-in default voices** — stable and consistent, no reference clip needed.
> - **Reading styles**: natural, news, and storytelling.
> - **Emotion / non-verbal cues** *(experimental)*: drop `[cười]`, `[thở dài]`, `[hắng giọng]` straight into the text.
> - **Batched generation** (batch size up to 32), including a multi-speaker **Conversation** mode that batches the whole script regardless of speaker.
> - **Instant voice cloning** from a 3–8s clip, with automatic reference denoising.
>
> Try it in the Web UI (backbone **"VieNeu-TTS-v3-Turbo (Thử nghiệm)"**) or the SDK (`Vieneu(mode="v3turbo")`). The **full v3** release is coming in the next few weeks.
> [!IMPORTANT]
> **🚀 VieNeu-TTS-v2 is here!**
> The full high-fidelity bilingual architecture is now available with:
> - **10,000+ Hours of Data:** Unmatched naturalness in both English and Vietnamese.
> - **Podcast & Dialogue Mode:** Multi-speaker support with emotional nuances.
> - **Zero-shot Cloning:** Clone any voice in 3-5 seconds across all v2 variants.
## ✨ Key Features
- **10,000+ Hours Training**: Trained on a massive English-Vietnamese dataset for human-like prosody.
- **Bilingual (En-Vi) Code-switching**: Powered by [**sea-g2p**](https://github.com/pnnbao97/sea-g2p) for high-fidelity pronunciation and seamless transitions between Vietnamese and English.
- **Podcast & Conversation Mode**: Multi-speaker dialogue support with automatic character detection.
- **Instant Voice Cloning**: Clone any voice with just **3-5 seconds** of reference audio.
- **Ultra-Fast Performance**: Optimized for **GPU (LMDeploy)** and **CPU (GGUF/ONNX)**.
- **Production-Ready**: High-quality 24 kHz waveform generation, fully offline.
[](https://github.com/user-attachments/assets/021f6671-2d7f-4635-91fb-88b2ab0ddbcd)
## 📌 Table of Contents
1. [🦜 Installation & Web UI](#installation)
2. [📦 Using the Python SDK](#sdk)
3. [🐳 High-Quality Server (Standard Mode)](#docker-remote)
4. [🔬 Model Overview](#backbones)
5. [🚀 Roadmap](#roadmap)
6. [🤝 Support & Contact](#support)
7. [📑 Citation](#citation)
---
## 🦜 1. Installation & Web UI
### Setup with `uv` (Recommended)
`uv` is the fastest way to manage dependencies.
```bash
# Windows:
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
# Linux/macOS:
curl -LsSf https://astral.sh/uv/install.sh | sh
```
1. **Clone the Repo:**
```bash
git clone https://github.com/pnnbao97/VieNeu-TTS.git
cd VieNeu-TTS
```
2. **Install Dependencies:**
- **Option 1: CPU & macOS (minimal, torch-free) — recommended for maximum speed** — runs **v3 Turbo via ONNX**
> 💡 *No GPU required. Installs only the lightweight ONNX stack; **v3 Turbo runs on CPU (48 kHz)** with default voices, voice cloning and emotion cues. PyTorch is never installed.*
>
> ⚡ **For the fastest CPU inference, install with `uv sync` — not `pip install`.** `uv sync` reproduces the locked environment that pins the optimized ONNX Runtime build, so you get maximum speed out of the box.
>
> 🍎 **macOS users: use this option too.** For v3 Turbo the torch-free ONNX path on the CPU is *faster* than the MPS/PyTorch build (`--group gpu`), so prefer `uv sync` for top speed on Apple Silicon.
```bash
uv sync
```
- **Option 2: GPU** — **v3 Turbo on GPU (PyTorch)**
> 💡 *Requires a CUDA NVIDIA GPU (CUDA ≥ 12.8) or Apple Silicon MPS. [NVIDIA Toolkit](https://developer.nvidia.com/cuda-downloads) recommended. Adds the PyTorch stack so **v3 Turbo runs on GPU** — inference is batched automatically on CUDA (same API, no code change).*
```bash
uv sync --group gpu
```
3. **Start the Web UI:**
```bash
uv run vieneu-web
```
Access the UI at `http://127.0.0.1:7860`.
---
## 📦 2. Using the Python SDK (vieneu)
The `vieneu` SDK **defaults to VieNeu-TTS v3 Turbo (48 kHz)**. The minimal install is **torch-free**: on CPU everything runs on **ONNX Runtime** (PyTorch is never imported), and on a CUDA machine it auto-switches to the PyTorch engine — where inference is **batched automatically** (same API, no code change).
### Quick Start
**CPU (default)** — torch-free, runs v3 Turbo via ONNX Runtime. Most users want this:
> ⚡**On CPU the backbone runs `int8` by default** — ~1.6× faster and ~4× smaller than fp32, with voice quality preserved. Want maximum fidelity instead? Pass `Vieneu(precision="fp32")` (slower on CPU). `precision` only affects the CPU/ONNX path; on GPU it's ignored (PyTorch).
```bash
pip install vieneu
```
**GPU (CUDA)** — only if you have an NVIDIA GPU.
> ℹ️ **When is GPU actually worth it?** The GPU win comes from **batching**, so it
> only pays off on **long text** (many chunks generated together in one forward —
> long-form or bulk synthesis). For **short text** the torch-free **CPU/ONNX** path
> is usually *faster* (there's no batch to fill, and no kernel-launch overhead). Use
> CPU for short, interactive calls; reach for GPU for long-form or high-throughput work.
```bash
pip install torch==2.8.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128
pip install "transformers==4.57.6" # pinned — most stable transformers for the GPU SDK
pip install vieneu
```
```python
import time
from vieneu import Vieneu
# Default = v3 Turbo (48 kHz). GPU → PyTorch (auto-detected).
vieneu = Vieneu() # On a GPU machine you can still switch to ONNX/CPU if you prefer: Vieneu(backend="onnx")
# 1. Built-in voice by name — no reference clip needed
print("🔊 Generating speech...")
start_time = time.time()
audio = vieneu.infer("[cười] Trời ơi, cái giọng nó tự nhiên mà nó mượt mà dã man, nghe không khác gì người thật luôn. Giờ thì tha hồ mà quẩy content với cả kho giọng nói đa dạng, đủ mọi sắc thái biểu cảm. Mọi người bật loa lên rồi cùng trải nghiệm thử với mình nhé!", voice="Phạm Tuyên")
elapsed_time = time.time() - start_time
vieneu.save(audio, "output.wav")
print("✅ Saved to output.wav")
# Tính RTF (Real-Time Factor)
sample_rate = 48000
audio_duration = len(audio) / sample_rate
rtf = elapsed_time / audio_duration
print(f"\n⏱️ Thời gian xử lý: {elapsed_time:.3f}s")
print(f"🎵 Thời lượng audio: {audio_duration:.3f}s")
print(f"📊 RTF: {rtf:.4f} ({'nhanh hơn' if rtf < 1 else 'chậm hơn'} real-time {1/rtf:.2f}x)" if rtf > 0 else "")
# List the built-in voices
voices = vieneu.list_preset_voices()
print(f"\n🎙️ {len(voices)} built-in voices available:")
for label, voice_id in voices:
print(f" - {label} ({voice_id})")
# 2. ⚡ Batch on GPU: infer_batch() runs many texts in ONE batched forward — same API.
# On a CUDA GPU the chunks from every text share each forward step (big throughput
# win). On CPU it still WORKS (no error) — just sequentially, so there's no batch
# gain. Batch caps at max_batch_size (default 32; tune via Vieneu(max_batch_size=64)
# or infer_batch(..., batch_size=64), or batch_size=1 to disable). A single long
# infer() also auto-batches its own chunks. Uncomment to try (GPU recommended):
#
# import time
# texts = [
# "Chào cả nhà, hôm nay mình sẽ hướng dẫn các bạn cách cài đặt và sử dụng bộ giọng đọc mới.",
# "Giọng nghe cực kỳ tự nhiên và truyền cảm, lại có thể chuyển đổi biểu cảm một cách linh hoạt.",
# "Nếu thấy hữu ích, các bạn nhớ để lại một lượt thích và chia sẻ video này cho mọi người nhé!",
# ] * 10 # 30 texts — enough to fill the batch and really show the GPU throughput win
# t0 = time.time()
# audios = vieneu.infer_batch(texts, voice="Minh Đức")
# elapsed = time.time() - t0
# total_audio = sum(len(a) for a in audios) / 48_000
# print(f"⚡ {len(texts)} texts | audio {total_audio:.1f}s | wall {elapsed:.1f}s | RTF {elapsed/total_audio:.3f}")
# for i, a in enumerate(audios):
# vieneu.save(a, f"batch_{i}.wav")
```
#### Streaming (real-time) 🔊
> v3 Turbo supports **frame-level streaming**: audio starts in ~300 ms and generation stays *ahead* of playback (RTF < 1 on CPU — ~2–3× on a laptop, ~7× on Apple Silicon), so it's ideal for realtime / interactive apps. Streaming runs on the > **ONNX/CPU** engine — low first-audio latency, frame-by-frame; the GPU/PyTorch engine is built for **batch throughput**, not streaming, so pin `backend="onnx"` for realtime. Just iterate `infer_stream`:
```python
from vieneu import Vieneu
vieneu = Vieneu(backend="onnx") # force ONNX/CPU — the streaming path (int8)
for chunk in vieneu.infer_stream("Xin chào các bạn!", voice="Minh Đức"):
play(chunk) # np.float32 @ 48 kHz — play/write as it arrives
```
A complete **FastAPI web streaming demo** is in [`apps/web_stream.py`](apps/web_stream.py):
```bash
uv run python -m apps.web_stream # → http://127.0.0.1:8001
```
#### Available Voices
The v3 Turbo engine includes **14 curated preset voices** covering **3 Vietnamese regions** (North, Central, South) with diverse genders and styles:
- **Northern (Bắc)**: Natural, news, storytelling styles
- **Central (Trung)**: Natural style (Quang Sơn, Ngọc Trân)
- **Southern (Nam)**: Natural, news, storytelling styles
Each voice supports **3 reading styles**: natural (`tu_nhien`), news (`tin_tuc`), and storytelling (`doc_truyen`).
### Reading style
Pick how the text is read with `style` (default `"tu_nhien"`):
| `style` | Meaning |
| -------------- | ------------- |
| `"tu_nhien"` | Natural / conversational |
| `"tin_tuc"` | News |
| `"doc_truyen"` | Storytelling |
```python
audio = vieneu.infer("Trận Caen là một trận đánh trong Chiến tranh Trăm Năm giữa Anh và Pháp diễn ra vào ngày 26 tháng 7 năm 1346 khi quân viễn chinh Anh dưới sự chỉ huy của Edward III tấn công thành Caen do quân Pháp nắm giữ.", voice="Minh Đức", style="tin_tuc")
```
### Emotion cues (experimental)
Inline tags are supported anywhere in the text: `[cười]` (chuckle), `[thở dài]` (sigh), `[hắng giọng]` (clear throat).
```python
audio = vieneu.infer("Nghe hay quá đi [cười]. Để mình nói tiếp [hắng giọng].", voice="Trúc Ly")
```
### Voice cloning
Clone any voice from a short reference clip. The clip is cleaned up automatically
(background noise removed, and trimmed to ≤ 8 seconds) before cloning — keep
`denoise=True` unless your clip is already clean.
```python
audio = vieneu.infer(
"Đây là giọng được nhân bản tức thì.",
ref_audio="my_voice.wav", # a 3–8s reference clip
denoise=True, # default; set False if the clip is already clean
style="doc_truyen",
)
vieneu.save(audio, "cloned.wav")
```
### Save & reuse a cloned voice
Register a reference once with `add_voice`, then use it by name like a built-in voice.
```python
# Enroll a voice (denoises + extracts the speaker profile once)
vieneu.add_voice("Giọng của tôi", "my_voice.wav")
# Now reuse it anywhere, including the conversation mode
audio = vieneu.infer("Câu này dùng giọng đã lưu.", voice="Giọng của tôi")
# Persist your voices so they load next session
vieneu.save_voices() # writes to the default voices file
# vieneu.remove_voice("Giọng của tôi")
# Add a voice you already cleaned yourself → skip denoising
vieneu.add_voice("Giọng sạch", "already_clean.wav", denoise=False)
```
### Clean up a clip on its own
Get the denoised audio without synthesizing anything (e.g. to inspect or store it):
```python
wav, sr = vieneu.denoise("noisy.wav", out_path="clean.wav") # 44.1 kHz mono
```
> **Note:** `denoise`, `add_voice`, and voice cloning currently require the PyTorch
> (GPU) engine. Built-in voices work everywhere.
---
## 🐳 3. High-Quality Server (Standard Mode)
Deploy VieNeu-TTS as a high-performance API Server (powered by LMDeploy) with a single command.
### 1. Run with Docker (Recommended)
**Requirement**: [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) is required for GPU support.
**Start the Server with a Public Tunnel (No port forwarding needed):**
```bash
docker run --gpus all -p 23333:23333 -v huggingface_cache:/root/.cache/huggingface pnnbao/vieneu-tts:latest --tunnel
```
* **Default**: The server loads the `VieNeu-TTS-v2` model for maximum quality.
* **Tunneling**: The Docker image includes a built-in `bore` tunnel. Check the container logs to find your public address (e.g., `bore.pub:31631`).
### 2. Using the SDK (Remote Mode)
Once the server is running, you can connect from anywhere (Colab, Web Apps, etc.) without loading heavy models locally.
**Installation**:
```bash
pip install "vieneu[legacy]"
```
**Usage**:
```python
from vieneu import Vieneu
import os
# Configuration
REMOTE_API_BASE = 'http://your-server-ip:23333/v1' # Or bore tunnel URL
REMOTE_MODEL_ID = "pnnbao-ump/VieNeu-TTS-v2"
# Initialization (LIGHTWEIGHT - only loads small codec locally)
# Default emotion is "natural" (conversational) - set emotion="storytelling" for storytelling mode
vieneu = Vieneu(mode='remote', api_base=REMOTE_API_BASE, model_name=REMOTE_MODEL_ID, emotion="natural")
os.makedirs("outputs", exist_ok=True)
# List remote voices
available_voices = vieneu.list_preset_voices()
for desc, name in available_voices:
print(f" - {desc} (ID: {name})")
# Use specific voice (dynamically select second voice)
if available_voices:
_, my_voice_id = available_voices[1]
voice_data = vieneu.get_preset_voice(my_voice_id)
audio_spec = vieneu.infer(text="Chào bạn, tôi đang nói bằng giọng của bác sĩ Tuyên.", voice=voice_data)
vieneu.save(audio_spec, f"outputs/remote_{my_voice_id}.wav")
print(f"💾 Saved synthesis to: outputs/remote_{my_voice_id}.wav")
# Standard synthesis (uses default voice)
text_input = "Chế độ remote giúp tích hợp VieNeu vào ứng dụng Web hoặc App cực nhanh mà không cần GPU tại máy khách."
audio = vieneu.infer(text=text_input)
vieneu.save(audio, "outputs/remote_output.wav")
print("💾 Saved remote synthesis to: outputs/remote_output.wav")
# Zero-shot voice cloning (encodes audio locally, sends codes to server)
if os.path.exists("examples/audio_ref/example_ngoc_huyen.wav"):
cloned_audio = vieneu.infer(
text="Đây là giọng nói được clone và xử lý thông qua VieNeu Server.",
ref_audio="examples/audio_ref/example_ngoc_huyen.wav",
ref_text="Tác phẩm dự thi bảo đảm tính khoa học, tính đảng, tính chiến đấu, tính định hướng."
)
vieneu.save(cloned_audio, "outputs/remote_cloned_output.wav")
print("💾 Saved remote cloned voice to: outputs/remote_cloned_output.wav")
```
*For full implementation details, see: [examples/main_remote.py](examples/main_remote.py)*
### Voice Preset Specification (v1.0)
VieNeu-TTS uses the official `vieneu.voice.presets` specification to define reusable voice assets. Only `voices.json` files following this spec are guaranteed to be compatible with VieNeu-TTS SDK ≥ v1.x.
### 3. Advanced Configuration
Customize the server to run specific versions or your own fine-tuned models.
**Run the 0.3B Model (Faster):**
```bash
docker run --gpus all pnnbao/vieneu-tts:serve --model pnnbao-ump/VieNeu-TTS-0.3B --tunnel
```
**Serve a Local Fine-tuned Model:**
If you have merged a LoRA adapter, mount your output directory to the container:
```bash
# Linux / macOS
docker run --gpus all \
-v $(pwd)/finetune/output:/workspace/models \
pnnbao/vieneu-tts:serve \
--model /workspace/models/merged_model --tunnel
```
---
## 🔬 4. Model Overview
| Model | Format | Device | Bilingual | Features | Speed |
|---|---|---|---|---|---|
| **VieNeu-TTS-v3-Turbo** *(early access)* | PyTorch/ONNX | **GPU/CPU** | ✅ | **48 kHz, Default voices, Cloning, Emotion cues, Conversation** | **Fast (batched)** |
| **VieNeu-TTS-v2** | PyTorch | **GPU** | ✅ | **Podcast, En-Vi CS** | **Fast (LMDeploy)** |
| **VieNeu-v2-CPU** | GGUF/ONNX | **CPU/Edge** | ✅ | **Podcast, En-Vi CS** | **Extreme Speed** |
| **VieNeu-v2-Turbo** | GGUF/ONNX | **CPU/Edge** | ✅ | Lightweight En-Vi | **Ultra Fast** |
| **VieNeu-TTS (v1)** | PyTorch | GPU/CPU | ❌ | Stable (Vi only) | Standard |
---
## 🚀 5. Roadmap
- [x] **VieNeu-TTS-v2**: Full high-fidelity bilingual architecture with **Podcast Mode** and **Voice Cloning**.
- [x] **VieNeu-Codec**: Optimized neural codec for Vietnamese (ONNX).
- [x] **Turbo Voice Cloning**: Bringing instant cloning to the lightweight Turbo engine.
- [x] **VieNeu-TTS v3 Turbo (early access)**: New from-scratch 48 kHz architecture — built-in default voices (speaker tokens), experimental emotion cues, batched generation & multi-speaker conversation.
- [ ] **VieNeu-TTS v3 (full release)**: Complete v3 with finalized quality, stable emotion control, more default voices & streaming server.
- [ ] **Mobile SDK**: Official support for Android/iOS deployment.
---
## 🤝 6. Support & Contact
- **Hugging Face:** [pnnbao-ump](https://huggingface.co/pnnbao-ump)
- **Discord:** [Join our community](https://discord.gg/yJt8kzjzWZ)
- **Facebook:** [Pham Nguyen Ngoc Bao](https://www.facebook.com/pnnbao97)
- **License:** Apache 2.0 (Free to use).
---
## 📑 7. Citation
```bibtex
@misc{vieneutts2026,
title = {VieNeu-TTS-v2: Advanced Vietnamese Text-to-Speech with Podcast and Code-Switching Support},
author = {Pham Nguyen Ngoc Bao},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/pnnbao-ump/VieNeu-TTS}}
}
```
---
## 🌟 Star History
---
## 🤝 Contributors
Thanks to all the amazing people who have contributed to this project!
---
## 🙏 Acknowledgements
This project uses [neucodec](https://huggingface.co/neuphonic/neucodec) (v1/v2) and [MOSS-Audio-Tokenizer-Nano](https://huggingface.co/OpenMOSS-Team/MOSS-Audio-Tokenizer-Nano) (v3 Turbo) for audio coding, and [sea-g2p](https://github.com/pnnbao97/sea-g2p) for text normalization and phonemization.
**Made with ❤️ for the Vietnamese TTS community**