---
name: fish-audio-api
description: Write direct HTTP / WebSocket calls to the Fish Audio platform (TTS, ASR, voice design, voice models, wallet, real-time TTS streaming) without depending on the Python or JavaScript SDK. Use when the user asks to call Fish Audio from curl, a language without an official SDK, an edge/runtime environment that cannot install the SDK, or when they explicitly want raw REST / WebSocket code. Covers authentication, endpoint URLs, required headers, request / response schemas, MessagePack vs JSON vs multipart encoding rules, multi-speaker dialogue, voice-design candidate generation, and the WebSocket streaming protocol.
---

# Fish Audio Raw API Skill

Use this skill to generate correct, runnable Fish Audio API calls without any SDK. The canonical machine-readable sources are:

- REST: `https://docs.fish.audio/api-reference/openapi.json`
- WebSocket: `https://docs.fish.audio/api-reference/asyncapi.yml`

This file condenses those into rules an agent can apply directly.

## Global facts

- Base URL: `https://api.fish.audio`
- WebSocket base: `wss://api.fish.audio`
- Auth (all endpoints): `Authorization: Bearer <FISH_API_KEY>`
- Optional distributed tracing for inference APIs: see `https://docs.fish.audio/api-reference/observability`.
- Get API keys: `https://fish.audio/app/api-keys`
- Never hardcode keys. Read from an env var like `FISH_API_KEY`.
- Errors are JSON with at least `{status, message}` (for example 401 / 402 / 404 / 429). Speech-to-text errors from `transcribe-1-pro` also carry `code` and `request_id`. Errors from the network edge in front of the API (some 413 and 5xx responses) may not be JSON. `422` validation errors, on the endpoints that return them, are an array of `{loc, type, msg, ctx, in}`.

## Endpoint map

| Method | Path | Purpose |
| --- | --- | --- |
| POST | `/v1/tts` | Text-to-Speech (streams audio bytes) |
| POST | `/v1/asr` | Speech-to-Text (returns JSON transcript) |
| POST | `/v1/voice-design` | Voice Design (returns generated voice candidates) |
| GET | `/model` | List voice models |
| POST | `/model` | Create voice model (voice cloning) |
| GET | `/model/{id}` | Get voice model metadata |
| PATCH | `/model/{id}` | Update voice model |
| DELETE | `/model/{id}` | Delete voice model |
| GET | `/wallet/{user_id}/package` | Subscription package info (`user_id` defaults to `self`) |
| GET | `/wallet/{user_id}/api-credit` | API credit balance (`user_id` defaults to `self`) |
| WSS | `/v1/tts/live` | Real-time TTS streaming (MessagePack frames) |

## Text-to-Speech: `POST /v1/tts`

Required headers:

- `Authorization: Bearer <FISH_API_KEY>`
- `Content-Type: application/json` **or** `application/msgpack`

Optional headers:

- `model`: values `s1` (deprecated), `s2-pro`, `s2.1-pro`, `s2.1-pro-free`, `drama-3`. If omitted or unrecognized, the server falls back to `s2.1-pro` (paid). Default to `s2.1-pro` for production; use `s2.1-pro-free` for free-tier evaluation and prototyping (same model, no TTFA/DPA guarantees). Do not choose `s1` for new code: S1 is deprecated and will be retired on December 31, 2026. After that date, requests that specify `s1` are served by `s2.1-pro`. Migration: `https://docs.fish.audio/developer-guide/models-pricing/migrate-from-s1`.

Response: streaming audio bytes (`Transfer-Encoding: chunked`) in the format set by `format`. Write to a file or pipe to a player. There is **no JSON wrapper** on success.

### Request body fields (TTSRequest)

| Field | Type | Default | Notes |
| --- | --- | --- | --- |
| `text` | string | — (required) | The text to synthesize. Use speaker tags `<\|speaker:0\|>`, `<\|speaker:1\|>` for multi-speaker. |
| `reference_id` | string \| string[] \| null | null | Voice model ID. Array = multi-speaker (`s2-pro`, the S2.1-Pro family, and `drama-3`). |
| `references` | ReferenceAudio[] \| ReferenceAudio[][] \| null | null | Inline zero-shot cloning samples. **Requires `application/msgpack`** because `audio` is raw bytes. 2D array for multi-speaker. |
| `temperature` | number 0–1 | 0.7 | Expressiveness. |
| `top_p` | number 0–1 | 0.7 | Nucleus sampling. |
| `prosody.speed` | number 0.5–2 | 1 | Playback speed. |
| `prosody.volume` | number (dB) | 0 | Loudness offset. |
| `prosody.normalize_loudness` | bool | true | **`s2-pro` and the S2.1-Pro family.** |
| `chunk_length` | int 100–300 | 300 | Text segment size. |
| `min_chunk_length` | int 0–100 | 50 | Min chars before a new chunk. |
| `normalize` | bool | true | Spell out numbers, dates, units and symbols before synthesis. Uses `language` when set, otherwise detects the language per chunk. |
| `language` | string \| null | null (auto) | BCP-47 tag of the text, e.g. `en`, `es-MX`, `zh-CN`, `zh-TW`. Case-insensitive; `_` = `-`. Only affects normalization and word timestamps; never the voice, accent or audio quality. Omit, `""` or `"auto"` = detect per chunk. Tags without normalization rules skip normalization (no error); tag list: `https://docs.fish.audio/features/text-to-speech#supported-normalization-languages`. Spanish without a region uses Spain conventions; `es-MX` for Mexican readings. Leave unset for mixed-language text. With `normalize: false`, only word timestamps use it. Over 35 chars or chars outside `[A-Za-z0-9_-]` → 400. Not in the official SDKs yet; send it in the raw body. |
| `format` | `wav` \| `pcm` \| `mp3` \| `opus` | `mp3` | Output format. |
| `sample_rate` | int \| null | null (44100, or 48000 for opus) | Output sample rate. |
| `mp3_bitrate` | 64 \| 128 \| 192 | 128 | Only when `format=mp3`. |
| `opus_bitrate` | -1000 \| 24000 \| 32000 \| 48000 \| 64000 | -1000 (auto) | Opus bitrate in **bps**. Only when `format=opus`. |
| `latency` | `low` \| `normal` \| `balanced` | `normal` | Quality vs latency. |
| `max_new_tokens` | int | 1024 | Per-chunk audio token cap. |
| `repetition_penalty` | number | 1.2 | >1.0 reduces repeats. |
| `condition_on_previous_chunks` | bool | true | Cross-chunk voice consistency. |
| `early_stop_threshold` | number 0–1 | 1.0 | Batch early-stop. |

`ReferenceAudio` = `{ audio: <raw bytes>, text: <transcript string> }`. 10–30 s of clean speech works best.

### Voice source rules

1. **Library / custom voice model** → set `reference_id` to the model `_id`. Simplest path.
2. **Zero-shot from audio** → set `references` (array of `{audio, text}`) and use **MessagePack** body. JSON cannot carry raw audio bytes.
3. **Multi-speaker dialogue (`s2-pro`, the S2.1-Pro family, and `drama-3`)** → `reference_id: [id0, id1, ...]` and embed `<|speaker:0|>` / `<|speaker:1|>` markers inside `text`. For zero-shot multi-speaker, `references` is an array-of-arrays, one inner array per speaker.

### Single-speaker curl

```bash
curl --request POST https://api.fish.audio/v1/tts \
  --header "Authorization: Bearer $FISH_API_KEY" \
  --header "Content-Type: application/json" \
  --header "model: s2.1-pro" \
  --data '{
    "text": "Hello! Welcome to Fish Audio.",
    "reference_id": "<voice-model-id>",
    "format": "mp3",
    "mp3_bitrate": 128,
    "latency": "normal"
  }' \
  --output out.mp3
```

### Multi-speaker curl

```bash
curl --request POST https://api.fish.audio/v1/tts \
  --header "Authorization: Bearer $FISH_API_KEY" \
  --header "Content-Type: application/json" \
  --header "model: s2.1-pro" \
  --data '{
    "text": "<|speaker:0|>Good morning!<|speaker:1|>Good morning! How are you?",
    "reference_id": ["<speaker-0-id>", "<speaker-1-id>"],
    "format": "mp3"
  }' \
  --output dialogue.mp3
```

### Python (no SDK, streaming to file)

```python
import os, httpx

payload = {
    "text": "Hello from Fish Audio.",
    "reference_id": "<voice-model-id>",
    "format": "mp3",
    "latency": "normal",
}

headers = {
    "Authorization": f"Bearer {os.environ['FISH_API_KEY']}",
    "Content-Type": "application/json",
    "model": "s2.1-pro",
}

with httpx.stream("POST", "https://api.fish.audio/v1/tts",
                  headers=headers, json=payload, timeout=None) as r:
    r.raise_for_status()
    with open("out.mp3", "wb") as f:
        for chunk in r.iter_bytes():
            f.write(chunk)
```

### Python with inline references (MessagePack)

```python
import os, httpx, msgpack

with open("sample.wav", "rb") as f:
    ref_audio = f.read()

payload = {
    "text": "Clone this voice and say this line.",
    "references": [{"audio": ref_audio, "text": "Transcript of sample.wav."}],
    "format": "mp3",
}

headers = {
    "Authorization": f"Bearer {os.environ['FISH_API_KEY']}",
    "Content-Type": "application/msgpack",
    "model": "s2.1-pro",
}

body = msgpack.packb(payload, use_bin_type=True)
with httpx.stream("POST", "https://api.fish.audio/v1/tts",
                  headers=headers, content=body, timeout=None) as r:
    r.raise_for_status()
    with open("out.mp3", "wb") as f:
        for chunk in r.iter_bytes():
            f.write(chunk)
```

### Node.js (fetch, streaming)

```js
import { createWriteStream } from "node:fs";
import { Readable } from "node:stream";
import { pipeline } from "node:stream/promises";

const res = await fetch("https://api.fish.audio/v1/tts", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.FISH_API_KEY}`,
    "Content-Type": "application/json",
    model: "s2.1-pro",
  },
  body: JSON.stringify({
    text: "Hello from Fish Audio.",
    reference_id: "<voice-model-id>",
    format: "mp3",
    latency: "normal",
  }),
});

if (!res.ok) throw new Error(`${res.status} ${await res.text()}`);
await pipeline(Readable.fromWeb(res.body), createWriteStream("out.mp3"));
```

## Speech-to-Text: `POST /v1/asr`

Synchronous: one audio file per request, and the JSON response arrives when the whole file has been transcribed.

Use `transcribe-1-pro`, the recommended model, and select it with the `model` header on every request. This section describes `transcribe-1-pro`; for the differences on `transcribe-1`, see "If you use `transcribe-1`" at the end of this section.

Guide: `https://docs.fish.audio/features/speech-to-text`. The `/v1/asr` entry in `openapi.json` currently does not list the `transcribe-1-pro` fields, `request_id`, `speaker_turns`, or the error `code`; where it is less complete or disagrees, follow this section.

Headers:

- `Authorization: Bearer <FISH_API_KEY>` (required).
- `model: transcribe-1-pro` (recommended): recordings up to 60 minutes, including multi-speaker conversations, with speaker markers, `speaker_turns`, and emotion and vocal-event cues. Write the value exactly, in lowercase. The header is optional, but a missing or unrecognized value (for example `Transcribe-1-Pro` or `transcribe-1pro`) is served and billed as `transcribe-1`, and no error is returned; if you expected Pro but `text` has no speaker markers and you did not send `diarize=false`, check the header. `model` is a header only; a `model` form field is ignored.
- Body encoding: `multipart/form-data` (let the HTTP client set `Content-Type` and the boundary) or `Content-Type: application/msgpack`. Base64-encoded audio in a JSON body is not supported.

Request fields (the same names in multipart and MessagePack):

| Field                          | Default      | Notes                                                                                                                                                                                                                                           |
| ------------------------------ | ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `audio`                        | — (required) | Exactly one audio file: a multipart file part, or MessagePack `bin`. The format is read from the bytes, not the file name.                                                                                                                      |
| `language`                     | unset        | Optional hint, a lowercase ISO 639-1 code such as `en`, `zh`, or `ja`. Detection still runs, and the hint does not force the transcript language. Other forms, such as `en-US` or `English`, may be rejected with 400.                          |
| `ignore_timestamps`            | `true`       | `false` returns word-level `segments` and, unless `diarize=false`, `speaker_turns`; it adds processing time. Multipart: any value other than `true` (case-insensitive), including `1` or an empty value, means `false`. MessagePack: a boolean. |
| `tag_audio_events`             | `true`       | `false` removes bracketed cues such as `[laughter]` or `[高兴]` from `text` and `speaker_turns`. Timestamps, `duration`, and billing do not change.                                                                                             |
| `diarize`                      | `auto`       | `auto` or `true`: return `speaker_turns` when timestamps are requested. `false`: omit `speaker_turns` and the speaker markers in `text`; the rest of the transcript does not change.                                                                                    |
| `num_speakers`                 | unset        | Expected number of speakers, an integer ≥ 1. A best-effort hint that may have no effect on short recordings. Cannot be combined with `min_speakers` or `max_speakers`.                                                                          |
| `min_speakers`, `max_speakers` | unset        | Bounds on the number of speakers, integers ≥ 1, with `min_speakers` ≤ `max_speakers`. Same best-effort rule.                                                                                                                                    |

Multipart values: `tag_audio_events` is `true` or `false` and `diarize` is `auto`, `true`, or `false` (both case-insensitive); speaker counts are digits. MessagePack values: `tag_audio_events` is a boolean, `diarize` is a boolean or `"auto"`/`"true"`/`"false"`, and speaker counts are integers. An invalid value, `tag_audio_events`, `diarize`, or a speaker count sent twice in a multipart form, or a speaker count with `diarize=false` returns 400 `invalid_parameter`.

Limits and formats:

- `transcribe-1-pro` accepts recordings up to 60 minutes; longer audio returns 400 `audio_too_long`. Send long recordings as compressed audio (MP3, Opus, or AAC); an hour of 128 kbps MP3 is about 55 MiB. A request that is too large returns 413. Send a whole conversation as one file: speaker labels are consistent within one response, not across requests.
- Processing time grows with the length of the recording, and long `transcribe-1-pro` requests can take several minutes. Set a generous client timeout (the examples use 15 minutes); many HTTP libraries default to much less (httpx: 5 s). If a long request fails with a 5xx error or the connection drops, retry it.
- `transcribe-1-pro` accepts WAV, MP3, AAC (including M4A/MP4), FLAC, Ogg (Opus or Vorbis), WebM/Matroska, and MOV, including browser recordings, and uses the first audio track of a video file; send the original file bytes. AIFF, CAF, WMA, AMR, AC-3, and raw (headerless) PCM return 400.

Response (200), an illustrative `transcribe-1-pro` response with `ignore_timestamps=false`:

```json
{
  "text": "<|speaker:0|> Thanks for joining. <|speaker:1|> [laughter] Happy to be here.",
  "duration": 4.2,
  "segments": [
    { "text": "Thanks", "start": 0.16, "end": 0.48 },
    { "text": "for", "start": 0.48, "end": 0.64 },
    { "text": "joining", "start": 0.64, "end": 1.12 },
    { "text": "Happy", "start": 2.24, "end": 2.56 },
    { "text": "to", "start": 2.56, "end": 2.68 },
    { "text": "be", "start": 2.68, "end": 2.8 },
    { "text": "here", "start": 2.8, "end": 3.2 }
  ],
  "language": "English",
  "language_code": "en",
  "request_id": "3f6c2a1e-8b4d-4c7a-9e21-5d0b7f9a6c13",
  "speaker_turns": [
    {
      "speaker": "speaker:0",
      "text": "Thanks for joining.",
      "start": 0.16,
      "end": 1.12
    },
    {
      "speaker": "speaker:1",
      "text": "[laughter] Happy to be here.",
      "start": 2.24,
      "end": 3.2
    }
  ]
}
```

Response fields:

- `text`: The full transcript, with inline `<|speaker:N|>` markers (none with `diarize=false`), usually with a space on each side, and, unless `tag_audio_events=false`, bracketed cues. Text before the first marker belongs to the first turn; unless `diarize=false` was sent, a transcript with no marker is a single speaker (speaker 0).
- `duration`: Audio length in seconds, including silence.
- `segments`: Word-level timestamps `{ text, start, end }` in seconds. Usually one word (one or a few characters in Chinese and Japanese), with no punctuation, markers, or cues; it can be normalized (`35` for `3.5`), so it does not always match `text`. `start` can equal `end`. `[]` (never omitted) when `ignore_timestamps=true`, when no speech was found, or when timing is temporarily unavailable. Segments are not speaker turns.
- `language`: The detected language's English name, such as `English` or `Chinese`. Omitted when it cannot be determined.
- `language_code`: ISO 639-1 code for `language`, such as `en`. Omitted when unknown. If the language cannot be determined and you sent a `language` hint, it reports your hint, unchecked. Responses report one language, even for recordings that switch languages.
- `request_id`: Unique ID for the request, also in error bodies and in the `x-request-id` response header. Include it when you contact support.
- `speaker_turns`: Present only when `ignore_timestamps=false` and `diarize` is not `false`. Turns in the order they occur (`[]` when no speech was found), each `{ speaker: "speaker:N", text, start, end }`. `N` matches the `<|speaker:N|>` marker in `text`; labels identify speakers within one response only. Turn `text` has no markers and keeps cues unless `tag_audio_events=false`. If `segments` is empty, turn times are approximate and can cover the whole recording. Consecutive turns can have the same speaker; do not assume turns are contiguous or non-overlapping. Prefer `speaker_turns` over parsing `text`.

`language` and `language_code` are omitted, never `null`. Do not depend on the order of keys in the JSON.

### curl

```bash
curl --request POST https://api.fish.audio/v1/asr \
  --header "Authorization: Bearer $FISH_API_KEY" \
  --header "model: transcribe-1-pro" \
  --form "audio=@input.wav" \
  --form "ignore_timestamps=false"
```

### Python

```python
import os, httpx

with open("input.wav", "rb") as f:
    r = httpx.post(
        "https://api.fish.audio/v1/asr",
        headers={
            "Authorization": f"Bearer {os.environ['FISH_API_KEY']}",
            "model": "transcribe-1-pro",
        },
        files={"audio": f},
        data={"ignore_timestamps": "false"},
        # Long Pro recordings can take several minutes; httpx defaults to 5 s.
        timeout=httpx.Timeout(900.0, connect=10.0),
    )
if r.is_error:
    try:
        err = r.json()  # {status, message}; transcribe-1-pro adds code, request_id
    except ValueError:
        err = {"status": r.status_code, "message": r.text}  # edge errors may not be JSON
    raise RuntimeError(f"ASR failed: {err}")
result = r.json()
print(result["text"])
for turn in result.get("speaker_turns", []):
    print(f"{turn['speaker']} [{turn['start']:.2f}-{turn['end']:.2f}] {turn['text']}")
```

### Node.js (fetch)

```js
import { readFile } from "node:fs/promises";
// npm install undici@7 (Node.js 20.18.1+; undici 8 needs Node.js 22.19+)
import { Agent, setGlobalDispatcher } from "undici";

// Node's fetch waits only 300 s for response headers; long Pro requests can take longer.
setGlobalDispatcher(
  new Agent({ headersTimeout: 900_000, bodyTimeout: 900_000 })
);

const form = new FormData();
form.append("audio", new Blob([await readFile("input.wav")]), "input.wav");
form.append("ignore_timestamps", "false");

const res = await fetch("https://api.fish.audio/v1/asr", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.FISH_API_KEY}`,
    model: "transcribe-1-pro",
  },
  body: form,
  signal: AbortSignal.timeout(900_000),
});
if (!res.ok) throw new Error(`${res.status} ${await res.text()}`);
const result = await res.json();
console.log(result.text);
for (const turn of result.speaker_turns ?? []) {
  console.log(`${turn.speaker} [${turn.start}-${turn.end}] ${turn.text}`);
}
```

MessagePack instead of multipart: send `msgpack.packb({"audio": audio_bytes, "ignore_timestamps": False}, use_bin_type=True)` with `Content-Type: application/msgpack` and the same `model: transcribe-1-pro` header; values are typed (booleans, integers), and `audio` must be raw bytes (`bin`).

### If you use `transcribe-1`

`transcribe-1` is for general transcription of short recordings, and it serves every request whose `model` header is missing or not an exact match. It differs from `transcribe-1-pro` as follows:

- Fields: send only `audio`, `language`, and `ignore_timestamps`. The other fields apply to `transcribe-1-pro`; send them only with `model: transcribe-1-pro`.
- Response: `text`, `duration`, `segments`, and, when the language is known, `language` and `language_code`. Speaker markers, `speaker_turns`, and `request_id` are `transcribe-1-pro` features.
- Errors: rely only on `status` and `message`.
- Limits: up to 50 MiB per request; keep MP3 and Opus files under 25 MiB. A request over the size limit returns 413 or 400. For recordings longer than a few minutes, use `transcribe-1-pro`. A long request can fail with 503 when it exceeds the processing-time limit; retry, and if it keeps failing, use `transcribe-1-pro` or split the audio.
- Formats: WAV, MP3, AAC (including M4A/MP4), FLAC, and Ogg (Opus or Vorbis). Convert WebM recordings (for example, from a browser's MediaRecorder) to Ogg/Opus, MP3, or WAV first, or use `transcribe-1-pro`.

## Voice Design: `POST /v1/voice-design`

Required headers:

- `Authorization: Bearer <FISH_API_KEY>`
- `Content-Type: application/json`
- `model: voice-design-1` (required; currently the only public Voice Design model)

Response: JSON `{ candidates: VoiceDesignCandidate[] }`. Each candidate includes `audio_base64`; decode it to write the generated audio bytes to a file. The current candidate audio payload is WAV bytes encoded as base64.

### Request body fields (VoiceDesignRequest)

| Field                     | Type           | Default      | Notes                                                                       |
| ------------------------- | -------------- | ------------ | --------------------------------------------------------------------------- |
| `instruction`             | string         | — (required) | Voice design prompt. 1 to 2000 characters.                                  |
| `reference_text`          | string \| null | null         | Optional preview text to read in the generated voice. Up to 150 characters. |
| `language`                | string \| null | null         | Optional language hint such as `en`, `zh`, or `ja`.                         |
| `n`                       | int            | 2            | Number of candidates. Range: 1 to 4.                                        |
| `speed`                   | number         | 1.0          | Speaking speed multiplier. Must be greater than 0 and at most 3.             |
| `num_step`                | int            | 32           | Diffusion steps. Range: 1 to 128.                                           |
| `guidance_scale`          | number         | 2.0          | Prompt guidance. Must be at least 0.                                        |
| `instruct_guidance_scale` | number         | 0.0          | Instruction guidance. Must be at least 0.                                   |
| `seed`                    | int \| null    | null         | Optional deterministic seed for candidate generation.                       |

Do **not** send MessagePack, multipart form data, inline reference audio, or service-internal fields such as `features`, `features_json_file`, or `include_audio_base64`.

### curl

```bash
curl --request POST https://api.fish.audio/v1/voice-design \
  --header "Authorization: Bearer $FISH_API_KEY" \
  --header "Content-Type: application/json" \
  --header "model: voice-design-1" \
  --data '{
    "instruction": "Warm, confident studio narrator with a natural tone",
    "reference_text": "Welcome to Fish Audio.",
    "language": "en",
    "n": 2
  }' | jq -r '.candidates[0].audio_base64' | base64 --decode > voice.wav
```

### Python

```python
import base64
import os
import httpx

r = httpx.post(
    "https://api.fish.audio/v1/voice-design",
    headers={
        "Authorization": f"Bearer {os.environ['FISH_API_KEY']}",
        "Content-Type": "application/json",
        "model": "voice-design-1",
    },
    json={
        "instruction": "Warm, confident studio narrator with a natural tone",
        "reference_text": "Welcome to Fish Audio.",
        "language": "en",
        "n": 2,
    },
    timeout=120,
)
r.raise_for_status()
candidate = r.json()["candidates"][0]
with open("voice.wav", "wb") as f:
    f.write(base64.b64decode(candidate["audio_base64"]))
```

Billing: one successful generation request is charged once, even when it returns multiple candidates. Authentication, validation, balance, concurrency, and service errors are not billed.

## Voice models: `/model`

### List: `GET /model`

Query params: `page_size` (default 10), `page_number` (default 1), `title`, `tag` (string or array), `self` (bool; only your models), `author_id`, `language`, `title_language`, `sort_by` (`score` | `task_count` | `created_at`, default `score`).

Returns `{total, items: ModelEntity[]}`.

### Create: `POST /model` (multipart/form-data)

Required: `type=tts`, `title`, `train_mode=fast`, `voices` (one or more audio file uploads).

Optional: `visibility` (`public` | `unlist` | `private`, default `public`; `cover_image` is required if `public`), `description`, `cover_image`, `texts` (transcripts matching each voice; if omitted, ASR is run on the audio), `tags` (string or array), `enhance_audio_quality` (bool, default `false`).

```bash
curl --request POST https://api.fish.audio/model \
  --header "Authorization: Bearer $FISH_API_KEY" \
  --form "type=tts" \
  --form "train_mode=fast" \
  --form "title=My Voice" \
  --form "visibility=private" \
  --form "voices=@sample1.wav" \
  --form "voices=@sample2.wav" \
  --form "texts=Transcript of sample 1." \
  --form "texts=Transcript of sample 2." \
  --form "tags=en" \
  --form "tags=narration"
```

Returns 201 with the full `ModelEntity` including `_id`, `state` (`created` | `training` | `trained` | `failed`), `visibility`, `samples`, `author`, counts, timestamps. Use `_id` as `reference_id` in `/v1/tts`.

### Get / Update / Delete

- `GET /model/{id}` → `ModelEntity`
- `PATCH /model/{id}`: JSON, form-urlencoded, multipart, or msgpack. Nullable fields: `title`, `description`, `cover_image` (binary), `visibility`, `tags`.
- `DELETE /model/{id}` → 200 on success.

```bash
curl --request PATCH https://api.fish.audio/model/<id> \
  --header "Authorization: Bearer $FISH_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{"title": "Renamed", "visibility": "unlist"}'
```

## Wallet

- `GET /wallet/self/package` → `{user_id, type, total, balance, created_at, updated_at, finished_at}`
- `GET /wallet/self/api-credit` → `{_id, user_id, credit, created_at, updated_at, has_phone_sha256, has_free_credit}`. Pass `?check_free_credit=true` to also populate `has_free_credit` (default `false`; the field is `null` when not checked).

Replace `self` with a specific `user_id` if you have permission; otherwise always use `self`.

## WebSocket TTS: `wss://api.fish.audio/v1/tts/live`

For low-latency / streaming TTS (e.g. LLM token stream → speech). All frames are **MessagePack-encoded** binary messages.

### Connection headers

- `Authorization: Bearer <FISH_API_KEY>`
- `model`: optional; values `s1` (deprecated, retires December 31, 2026, then served by `s2.1-pro`), `s2-pro`, `s2.1-pro`, `s2.1-pro-free`, `drama-3` (falls back to `s2.1-pro` when omitted or unrecognized)

### Event sequence

Client → server:

1. `StartEvent` (once, first message): `{event: "start", request: <TTSRequest>}`. The `request` object is the same schema as `POST /v1/tts` above. Usually `request.text = ""` and the real text streams in `TextEvent`s. `request.language`, if set, applies to the whole session.
2. `TextEvent` (one per text chunk): `{event: "text", text: "..."}`. Send as many as needed.
3. `FlushEvent` (optional): `{event: "flush"}`. Forces the server to synthesize buffered text immediately (use for turn-taking / low-latency flushes).
4. `CloseEvent` (final): `{event: "stop"}`. **Note the literal is `stop`, not `close`.**

Server → client:

- `AudioEvent`: `{event: "audio", audio: <bytes>}`. Many of these; concatenate in order to reconstruct the audio stream in the format set by `request.format`.
- `FinishEvent`: `{event: "finish", reason: "stop" | "error"}`. Exactly one, then the server closes the socket. Ignore unknown events for forward compatibility.

### Python example (`websockets>=14` + `msgpack`)

`additional_headers` is the parameter name in `websockets` v14+. On older
releases use `extra_headers=` or import from `websockets.legacy.client`.

```python
import asyncio, os, msgpack, websockets
from websockets.exceptions import ConnectionClosed

API_KEY = os.environ["FISH_API_KEY"]
URL = "wss://api.fish.audio/v1/tts/live"

start = {
    "event": "start",
    "request": {
        "text": "",
        "reference_id": "<voice-model-id>",
        "format": "mp3",
        "latency": "normal",
    },
}

async def run(text_stream):
    headers = {"Authorization": f"Bearer {API_KEY}", "model": "s2.1-pro"}
    async with websockets.connect(URL, additional_headers=headers,
                                  max_size=None) as ws:
        await ws.send(msgpack.packb(start, use_bin_type=True))

        async def sender():
            try:
                async for chunk in text_stream:
                    await ws.send(msgpack.packb(
                        {"event": "text", "text": chunk}, use_bin_type=True))
                await ws.send(msgpack.packb({"event": "stop"}, use_bin_type=True))
            except ConnectionClosed:
                pass  # server sent finish before the text stream drained

        send_task = asyncio.create_task(sender())
        try:
            with open("out.mp3", "wb") as f:
                async for raw in ws:
                    msg = msgpack.unpackb(raw, raw=False)
                    if msg["event"] == "audio":
                        f.write(msg["audio"])
                    elif msg["event"] == "finish":
                        if msg["reason"] == "error":
                            raise RuntimeError("TTS failed")
                        break
        finally:
            send_task.cancel()
            try:
                await send_task
            except (asyncio.CancelledError, ConnectionClosed):
                pass

async def words():
    for w in ["Hello", " from", " Fish", " Audio."]:
        yield w

asyncio.run(run(words()))
```

### Node.js example (`ws` + `@msgpack/msgpack`)

```js
import WebSocket from "ws";
import { encode, decode } from "@msgpack/msgpack";
import { createWriteStream } from "node:fs";

const ws = new WebSocket("wss://api.fish.audio/v1/tts/live", {
  headers: {
    Authorization: `Bearer ${process.env.FISH_API_KEY}`,
    model: "s2.1-pro",
  },
});

const out = createWriteStream("out.mp3");

ws.on("open", () => {
  ws.send(encode({
    event: "start",
    request: { text: "", reference_id: "<voice-model-id>", format: "mp3" },
  }));
  ws.send(encode({ event: "text", text: "Hello from Fish Audio." }));
  ws.send(encode({ event: "stop" }));
});

ws.on("message", (buf) => {
  const msg = decode(buf);
  if (msg.event === "audio") out.write(Buffer.from(msg.audio));
  else if (msg.event === "finish") {
    out.end();
    ws.close();
    if (msg.reason === "error") throw new Error("TTS failed");
  }
});
```

## Emotion / expression control

S2-Pro and the S2.1-Pro models use free-form `[bracket]` natural-language tags inside `text`, e.g. `[slightly sarcastic, rising tone]`; there is no separate parameter. The deprecated S1 model uses legacy `(parenthesis)` tags, e.g. `(happy) What a day!`. The API does not convert between the two, so write new code with `[bracket]` tags (S1 retires December 31, 2026). Full list: `https://docs.fish.audio/api-reference/emotion-reference.md`.

## Encoding and content-type rules

- Use `application/json` for normal TTS requests. It's the simplest and works for `reference_id` flows.
- Use `application/msgpack` when you need to send raw audio bytes inline (inline `references`, or the WebSocket protocol).
- Use `multipart/form-data` for `/v1/asr` and `POST /model` because they upload files. `/v1/asr` also accepts `application/msgpack`; it does not accept base64 audio in JSON.
- All WebSocket frames are MessagePack binary, regardless of inner payload.

## Error handling checklist

- 400 on `POST /v1/tts` → `language` is longer than 35 characters or contains characters other than letters, digits, `-` and `_`.
- 401 → missing / bad `Authorization` header.
- 402 → out of credit. Check `/wallet/self/api-credit`.
- 404 → bad `model/{id}` (voice model doesn't exist or isn't visible to you).
- 422 → validation. The response is an array; each item's `loc` points at the offending field. Most common causes:
  - `reference_id` is an array but model is `s1` (multi-speaker requires `s2-pro`, an S2.1-Pro model, or `drama-3`).
  - `references` sent with `Content-Type: application/json` (must be msgpack).
  - Numeric param out of range (`temperature`, `top_p`, `chunk_length`, `min_chunk_length`, `prosody.speed`, `early_stop_threshold`).
  - `mp3_bitrate` / `opus_bitrate` set without matching `format`.
- WebSocket: a `finish` event with `reason: "error"` means the server failed mid-stream. Surface the message and reconnect rather than retrying on the same socket.
- `POST /v1/asr` (`transcribe-1-pro`): branch on the HTTP status and on `code`, never on `message`. New `code` values may be added; handle unknown codes by HTTP status. With `transcribe-1`, rely only on `status` and `message`.
  - 400 → fix the request; do not retry. Codes: `invalid_request` (unreadable body, missing or repeated `audio`), `invalid_parameter` (bad or conflicting field values), `invalid_audio` (undecodable or unsupported format), `audio_too_long` (over 60 minutes), `audio_too_short` (under about 0.08 s).
  - 413 → request too large (`request_too_large`). It may come from the network edge without a JSON body. Send compressed audio.
  - 415 → unsupported `Content-Type` (`unsupported_media_type`). Use `multipart/form-data` or `application/msgpack`.
  - 429 → usually your account is at its concurrency limit, shared by all its API keys; each `/v1/asr` request holds a slot until its response returns. No `Retry-After` header is sent; retry with exponential backoff.
  - 500 (`internal_error`), 502, 503 (`upstream_unavailable`, `upstream_timeout`, `excessive_repetition`, `diarization_failed`), 504 → temporary; retry with exponential backoff.
  - Any other 4xx (`upstream_rejected`, rare) → do not retry.

## Decision shortcuts

- User just wants audio from text → `POST /v1/tts` with JSON + `reference_id`.
- User has a raw voice clip and wants instant cloning → `POST /v1/tts` with MessagePack + `references`.
- User wants dialogue between multiple speakers → `POST /v1/tts` on `s2.1-pro` with `reference_id` array and `<|speaker:N|>` tags.
- User is streaming tokens from an LLM and wants speech to play as it arrives → WebSocket `/v1/tts/live`.
- User wants a persistent custom voice they can reuse → `POST /model` first, then reuse the returned `_id` as `reference_id`.
- User wants a transcript → `POST /v1/asr` with the `model: transcribe-1-pro` header (send it on every request; without it, the request runs on `transcribe-1`).