# Voice Note Transcription

> Transcribes inbound WhatsApp voice notes to text via an OpenAI-compatible speech-to-text backend and
> delivers a `message.transcription` event to your webhook — so bots and AI can read and reply to audio.

![type: extension](https://img.shields.io/badge/type-extension-blue.svg)
![license: MIT](https://img.shields.io/badge/license-MIT-green.svg)
![built for OpenWA](https://img.shields.io/badge/OpenWA-%E2%89%A5%200.8.0-25D366.svg)
[![downloads](https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Frmyndharis%2FOpenWA-plugins%2Fbadges%2Fdownloads%2Fvoice-transcription.json)](https://github.com/rmyndharis/OpenWA-plugins/releases?q=voice-transcription)

## Details

<!-- BEGIN DETAILS (generated by scripts/catalog.mjs — do not edit by hand) -->
| Field | Value |
| ----- | ----- |
| **Identifier** | `voice-transcription` |
| **Version** | 1.3.7 |
| **Released** | 2026-11-10 |
| **Status** | beta |
| **Author** | Yudhi Armyndharis |
| **License** | MIT |
| **Type** | `extension` |
| **Requires OpenWA** | ≥ 0.8.0 (tested 0.30.0) |
| **Keywords** | transcription, speech-to-text, stt, whisper, voice, audio, whatsapp, openwa |
| **Repository** | [OpenWA-plugins/voice-transcription](https://github.com/rmyndharis/OpenWA-plugins/tree/main/voice-transcription) |
<!-- END DETAILS -->

## Features

- **Voice → text, out of band.** On each inbound voice note the plugin runs speech-to-text and POSTs a
  `message.transcription` event to your webhook. It never touches the `message.received` payload and never
  replies into the contact's chat, so the transcript reaches your bot/AI without polluting the conversation
  or leaking back to the sender.
- **Never blocks delivery.** The hook returns immediately; STT runs as an un-awaited task. A slow or long
  transcription cannot delay (or drop) WhatsApp message delivery, and it is not bound by OpenWA's 5-second
  plugin-hook budget.
- **Bring your own STT.** Any OpenAI-compatible `/v1/audio/transcriptions` endpoint: self-hosted
  [Speaches](https://github.com/speaches-ai/speaches)/faster-whisper (free, local, decodes WhatsApp
  OGG/Opus natively — no transcoding), or hosted Groq / OpenAI by changing one URL.
- **Status events.** Delivers `completed` (with the transcript), `failed` (STT errored), or `skipped`
  (too large / rate-limited / empty) — a consumer always knows a voice note arrived, even when it can't be read.
- **Optional in-chat delivery.** `chatDelivery` can also post the transcript into WhatsApp (`self` notes it to
  your own number; `reply` quote-replies to the sender). Off by default; webhook delivery is optional too
  (chat-only is supported).
- **Signed webhooks.** When a delivery secret is set, the body is HMAC-SHA256 signed in `X-OpenWA-Signature`
  (same scheme as OpenWA's core webhooks), so existing verification reuses the same check.
- **Cost & abuse guards.** Exact `maxSizeBytes` skip, a best-effort per-session hourly cap, a message-type
  filter (voice only by default), a best-effort idempotency guard against engine re-fires, and an STT
  circuit breaker that backs off a degraded backend.
- **Fail-open & guarded HTTP.** Any error is logged and skipped. All outbound calls go through the host's
  SSRF-guarded `ctx.net.fetch`; the STT and delivery hosts must be allow-listed (see **Security**). The STT
  and delivery secrets are stored redacted.

## What it does

For every inbound message whose `type` is in **Message types to transcribe** (default `voice`) and that
carries inline audio, the plugin: decodes the audio, skips it if it was dropped over the inbound size cap
or exceeds `maxSizeBytes`, applies the idempotency + hourly-rate guards, calls your STT endpoint, and — if a
non-empty transcript comes back — POSTs this to your **Delivery webhook URL**:

```json
{
  "event": "message.transcription",
  "sessionId": "…",
  "messageId": "<waMessageId>",
  "chatId": "…@s.whatsapp.net",
  "status": "completed",
  "source": "speech-to-text",
  "untrusted": true,
  "timestamp": 1758000000,
  "transcription": { "text": "…", "language": "es", "provider": "faster-whisper", "model": "small" }
}
```

Correlate it to the original voice note by `messageId` (it arrives shortly **after** `message.received`,
out of order — do not assume ordering).

`timestamp` is the voice note's original send time (unix seconds). From OpenWA 0.23.6 a Baileys session
delivers, after it reconnects, the notes WhatsApp queued while it was disconnected, so an event can arrive
long after the note was sent: check it before replying. Such a late note is still transcribed and
delivered to the webhook and to `self`, but never quote-replied (see `chatDelivery`).

## Setup

1. **Run an STT backend.** Easiest local option — Speaches (faster-whisper), which exposes an
   OpenAI-compatible API and transcribes OGG/Opus directly:
   ```bash
   docker run -d --name speaches -p 8000:8000 ghcr.io/speaches-ai/speaches:latest-cpu
   ```
   Preload/keep a small model warm for low latency. Or use hosted Groq (`https://api.groq.com/openai`,
   model `whisper-large-v3-turbo`) / OpenAI with an API key.
2. **Allow the hosts.** Any **https** URL you set as **STT base URL** or **Delivery webhook URL** is
   allowed automatically — the manifest declares both as `net.allowConfigHosts`, so the host admits
   whatever you configure without you having to edit and re-package the plugin. The manifest also ships
   a static `net.allow` for `localhost`, `127.0.0.1`, `api.groq.com:443` and `api.openai.com:443`,
   which is what covers a plain-`http` local backend (config-derived hosts are https-only).
3. **For a localhost STT/delivery target** — any self-hosted Speaches/faster-whisper deployment; a hosted
   Groq/OpenAI endpoint needs nothing here — also set the environment variable `SSRF_ALLOWED_HOSTS` on the
   OpenWA host itself (not in this plugin's config; the SSRF guard blocks loopback by default), e.g.
   `SSRF_ALLOWED_HOSTS=127.0.0.1,localhost`. Prefer a literal `http://127.0.0.1:PORT` target (nothing to
   DNS-rebind).
4. **Stand up your delivery endpoint** (an n8n webhook, a Worker, your bot) and put its URL in
   **Delivery webhook URL**.

## Install

```bash
# Upload the packaged zip
curl -X POST http://localhost:2785/api/plugins/install \
  -H "X-API-Key: $OPENWA_API_KEY" -F "file=@voice-transcription.zip"

# Configure (per session, or '*' for all)
curl -X PUT http://localhost:2785/api/plugins/voice-transcription/config \
  -H "X-API-Key: $OPENWA_API_KEY" -H 'Content-Type: application/json' \
  -d '{ "config": { "sttBaseUrl": "http://127.0.0.1:8000", "model": "small", "deliveryWebhookUrl": "http://127.0.0.1:5678/webhook/transcript" } }'

# Enable
curl -X POST http://localhost:2785/api/plugins/voice-transcription/enable \
  -H "X-API-Key: $OPENWA_API_KEY"
```

## Configuration

| Key | Required | Default | Description |
| --- | -------- | ------- | ----------- |
| `sttBaseUrl` | yes | — | OpenAI-compatible STT base URL (`/v1/audio/transcriptions` is appended). An https host is allowed automatically; localhost also needs `SSRF_ALLOWED_HOSTS`. |
| `sttApiKey` | no | — | Bearer key for hosted STT (Groq/OpenAI). Stored redacted. |
| `model` | no | `small` | Whisper model name. |
| `language` | no | _(auto)_ | BCP-47 hint; blank = auto-detect. |
| `provider` | no | `faster-whisper` | Label recorded in the delivered event. |
| `timeoutMs` | no | `20000` | STT request timeout (max 25000, below the host's 30000 per-capability ceiling). |
| `enabledMessageTypes` | no | `["voice"]` | Add `audio` to also transcribe non-PTT audio (more cost). |
| `maxSizeBytes` | no | `16777216` | Skip audio larger than this (exact cost guard). |
| `maxPerHour` | no | `60` | Best-effort per-session hourly transcription cap. |
| `deliveryWebhookUrl` | cond. | — | Endpoint receiving the `message.transcription` event. An https host is allowed automatically; a plain-http one is not, so it only works for `localhost`/`127.0.0.1` (already in `net.allow`) and also needs `SSRF_ALLOWED_HOSTS`. Optional if you only use `chatDelivery`. |
| `deliverySecret` | no | — | Optional. HMAC-SHA256 signs the body in `X-OpenWA-Signature: sha256=<hex>` (same as core webhooks). Stored redacted. |
| `deliveryTimeoutMs` | no | `5000` | Delivery POST timeout. |
| `chatDelivery` | no | `off` | Also post the transcript into WhatsApp: `off` (webhook only) · `self` (note to your own number) · `reply` (quote-reply to the sender, visible to them). **In a group, `reply` posts to the whole group**, so every member reads the contents of the voice note. `reply` is skipped for a note that reached the gateway more than 5 minutes after it was sent; with no webhook such a note is not transcribed at all. |

## Compatibility

Checked against the OpenWA v0.30.0 worker runtime with isolated services. This covers plugin lifecycle, hooks or ingress dispatch, capability calls and per-session configuration. Live WhatsApp sessions, provider authentication and dashboard rendering require separate deployment checks.

- Requires OpenWA **≥ 0.8.0**, the release that introduced `net.allowConfigHosts`. Both `sttBaseUrl` and
  `deliveryWebhookUrl` are resolved through it, so on a 0.7.x host every call to a configured host is
  refused and neither transcription nor delivery works at all.
- Engine-neutral: both Baileys and whatsapp-web.js materialize the audio before the hook fires, so the
  plugin works on either.
- A backlog delivered after a reconnect (OpenWA 0.23.6+, Baileys) counts against `maxPerHour` in the
  hour it arrives.
- While another plugin (for example chatwoot-adapter) holds a chat in a `human` or `closed` handover, its
  voice notes are not dispatched to this plugin and are not transcribed; from OpenWA 0.23.6 that also
  covers the same contact's other `@lid` and phone ids.
- **Best-effort by design (no core changes).** Because a sandboxed plugin has no host-managed background
  queue, transcription runs as an un-awaited task in the worker turn: it is **at-most-once while the worker
  is alive**, has no backpressure, and the hourly/idempotency guards are best-effort (a truly simultaneous
  engine re-fire can still double-call STT). For exactly-once, structured-event delivery, a future core
  `message.transcription` event would be the upgrade path.

### Per-session config

**Supported, with a caveat.** Every config field may be overridden per WhatsApp session via the
dashboard; an override that changes a coordinator-affecting field (e.g. `sttBaseUrl`, `sttApiKey`,
`model`, `language`, `timeoutMs`, delivery fields, `chatDelivery`, `enabledMessageTypes`,
`maxSizeBytes`, `maxPerHour`, `provider`) takes effect on the next inbound message. The plugin uses
config-signature caching: the coordinator (and the STT provider's circuit breaker) is reused across
messages with the same resolved config, and rebuilt only when the signature changes.

**Caveat for multi-backend deployments:** when two sessions point at *different* STT backends, they
share one coordinator slot and clobber each other's circuit-breaker state on every message
alternation. OpenWA installs a plugin once per gateway (a second install of the same id is refused with
`409`) and gives separate instances only to plugins with ingress routes, so this plugin has no
per-session instance: keep one STT backend for all sessions, or run one OpenWA gateway per backend.

**Host overrides and secrets.** From OpenWA 0.26.0 a per-session override that points
`sttBaseUrl` or `deliveryWebhookUrl` at another origin than the base config inherits none of the
stored secrets, `sttApiKey` and `deliverySecret` alike. Set both in that override: without
`deliverySecret` the transcript delivery goes out unsigned.

## Security

- **Outbound HTTP is allow-listed.** Calls go through the host's SSRF-guarded `ctx.net.fetch`. Reachable
  hosts are the four static `net.allow` entries plus the https hosts of the STT and delivery URLs you
  configure (`net.allowConfigHosts`) — nothing else. Internal/loopback IPs stay blocked unless the operator
  opts in via `SSRF_ALLOWED_HOSTS`. This plugin never ships `net.allow: ["*"]`: it carries your voice audio
  and an API key, so egress stays pinned to the hosts you named. A config-derived host must be plain https
  with no embedded credentials, which is what keeps that convenience from widening the allowlist.
- **Secrets** are stored redacted. `sttApiKey` is sent as a Bearer token only to the STT host;
  `deliverySecret` is **never transmitted** — it HMAC-signs the body (`X-OpenWA-Signature`).
- **Treat the transcript as untrusted.** `transcription.text` is attacker-controlled speech — the event
  marks it `untrusted: true`. A downstream LLM auto-responder MUST place it in a **user** role, never a
  system/trusted context (a caller can *speak* injection instructions a typist never would).

## Changelog

See [CHANGELOG.md](./CHANGELOG.md).

## License

MIT