# computah
A local voice assistant for people who want the assistant they already use by text to answer by voice too.
computah listens for a wake word, transcribes the request, sends the transcript to a persistent assistant session, and speaks the answer back. The speech path runs locally on CPU with openWakeWord, faster-whisper, and Piper. The brain can run on the same host or behind a small file bridge on another machine.
## What it does
computah is the core of a local voice assistant. Feed it a wav file or run one of its live microphone loops and it uses the same stages:
1. Detect the wake word in the audio.
2. Transcribe the spoken request.
3. Send the transcript to an assistant session that keeps context.
4. Render the answer as a wav file.
The project is built so each stage can be replaced. The wake-word detector, transcriber, brain bridge, and text-to-speech layer have narrow boundaries in `pipeline.py`.
## Why it exists
Most voice assistants bundle wake-word detection, speech recognition, memory, and speech output into one service. That makes the assistant easy to start but hard to own. computah is shaped around different constraints:
- The wake word is configurable today, and a trainable `computah` wake word is the goal.
- Speech recognition and speech synthesis stay local.
- The voice interface talks to the same long-running assistant session used for text.
- Tests can run without a microphone by passing audio files through the pipeline.
- Model files, voices, local session paths, and hostnames stay outside git.
The name is also the intended wake word: “computah,” said in your own voice.
## Current status
v0.1.0 has a working file-based pipeline and live microphone loops. Hardware deployment remains device-specific.
| Area | Status |
| --- | --- |
| Wake-word detection | Works with installed openWakeWord models and custom `.onnx` files in `models/`. |
| Speech-to-text | Works through faster-whisper with CTranslate2 int8. |
| Brain | Supports a fallback CLI backend and the persistent file bridge. |
| Text-to-speech | Works through Piper by writing a reply wav. |
| Live loop | Implemented for desktop microphones and an ALSA/arecord hardware path. |
| Custom `computah` wake word | The training recipe and recording protocol are documented; the production model is not yet validated. |
## How the pipeline works
```text
audio in
└─▶ wake word openWakeWord, ONNX, 80 ms frames
└─▶ transcript faster-whisper, CTranslate2 int8
└─▶ brain persistent assistant bridge or CLI fallback
└─▶ wav Piper, ONNX voice model
```
| Stage | Default implementation | Boundary |
| --- | --- | --- |
| Wake word | openWakeWord | `detect_wake` returns whether the configured phrase fired. |
| Speech-to-text | faster-whisper | `transcribe` returns text from a file path or normalized 16 kHz mono array. |
| Brain | bridge or CLI | `brain` returns short spoken text. |
| Text-to-speech | Piper | `speak` writes the answer to a wav file. |
Module-level caches keep the wake-word and Whisper models warm inside one process. Wake-word detection normalizes audio to 16 kHz mono int16. File-based transcription still accepts a wav path; both live paths normalize captured int16 PCM and pass the waveform to faster-whisper in memory, without a request-side temporary file.
In a live turn, a mishear guard sits between transcription and the brain: it reads faster-whisper's own confidence signals (`avg_logprob`, `no_speech_prob`) and, when a transcript looks garbled or silence-derived, speaks a short re-prompt instead of dispatching. Because the brain acts on what it hears, this keeps a misheard command from triggering an action. Both live paths (`run_turn` and the `live_driver` hardware loop) gate through the same `guard_transcript`; the file-based `run_pipeline` reports the same signals for inspection but does not gate on them.
## The brain bridge
The bridge is the main design choice. computah does not need to create a fresh assistant call for every voice turn. It can append a user event to an inbox file and wait for the next reply block from an already-running assistant session.
`brain_bridge.py` keeps the transport injectable:
- `cli_send` and `file_reply_reader` talk to a session on the same host.
- `ssh_cli_send` and `ssh_reply_reader` use the same file contract on another host.
- `local_sim_send` and `sim_persona.py` let tests exercise bridge behavior without a live assistant.
`brain_via_bridge` gives each transcript an `event_id`. The local, SSH, and simulated transports preserve that ID, and a reply echoes it so the bridge matches each answer to its request. Replies from an older unstamped producer use a positional fallback during upgrades.
A fresh clone uses `brain_backend: "cli"` so it can run without bridge setup. To use the persistent session path, copy `config.local.example.json` to `config.local.json`, set `brain_backend` to `bridge`, and keep deployment values there. `config.local.json` is gitignored and overrides `config.json` at runtime.
## Repository layout
| Path | Purpose |
| --- | --- |
| `pipeline.py` | Pipeline stages, config loading, and CLI entry point. |
| `brain_bridge.py` | Bridge contract plus local, ssh, and simulated transports. |
| `sim_persona.py` | Test stand-in for a long-running assistant session. |
| `prep_wake_samples.py` | Converts wake-word recordings into training clips. |
| `config.json` | Committed defaults for wake word, model choices, and backend selection. |
| `config.local.example.json` | Template for gitignored local bridge settings. |
| `requirements.txt` | Python dependencies for the CPU-only speech path. |
| `docs/` | GitHub Pages site and recording notes. |
| `models/`, `voices/`, `whisper_models/` | Local model directories; generated or downloaded files stay out of git. |
| `test_*.py` | Mic-free tests for the bridge, dispatch logic, sample prep, and pipeline. |
## Setup
The project is developed on Linux/ARM64 with Python 3.13 on a Raspberry Pi 5. Other Linux hosts should work if the same dependencies are available.
```bash
python -m venv .venv
.venv/bin/pip install -e .
.venv/bin/python -m piper.download_voices en_US-lessac-medium --download-dir voices
```
The editable install reads the pinned dependencies from `requirements.txt`, adds
the `.venv/bin/computah` command, and keeps configuration and model paths anchored
to this checkout. It does not put model files in the Python package.
faster-whisper downloads its model into `whisper_models/` on first use. Piper voices live in `voices/`. Custom wake-word models live in `models/`. These files are local artifacts and are not committed.
## Usage
List available wake words:
```bash
.venv/bin/computah --list-wake-words
```
Switch the active wake word and persist it to `config.json`:
```bash
.venv/bin/computah --set-wake-word hey_jarvis
```
Use `--local` to persist the choice to the gitignored `config.local.json`
instead, so a deployment can activate its own wake word (often a gitignored
`computah` model) without dirtying the tracked `config.json` a fresh clone
runs with:
```bash
.venv/bin/computah --set-wake-word computah --local
```
`config.local.json` overrides `config.json` at runtime, so the local choice
wins while the committed default stays a built-in.
Run the pipeline on a wav file:
```bash
.venv/bin/computah clip.wav -o reply.wav
```
The output path receives the spoken reply.
Run the desktop microphone loop in the foreground:
```bash
.venv/bin/computah --listen
```
## Run at startup
`docs/computah.service` is a sample systemd user unit. It restarts the live loop
after a failure or a clean device-stream exit and applies the same 1.5 GB memory
and no-swap limits used by the project's model tests. The sample expects this
checkout at `~/computah` with the editable install in `~/computah/.venv`. If
either path differs, edit
`WorkingDirectory` and `ExecStart` in the copied unit.
```bash
mkdir -p ~/.config/systemd/user
cp docs/computah.service ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now computah.service
sudo loginctl enable-linger "$USER"
```
Linger lets the user service start at boot and continue without an active login.
Check the process and follow its logs with:
```bash
systemctl --user status computah.service
journalctl --user -u computah.service -f
```
## Choosing a microphone
The live loop needs a capture device that delivers full-band audio. Use a USB
Audio Class mic (or another wired/USB mic). **Do not use a Bluetooth mic on
Windows for continuous speech**, including a conference mic like the Anker
PowerConf paired over Bluetooth.
Windows captures from a Bluetooth mic over the hands-free profile (HFP), which
downsamples and recompresses the stream to narrowband and drops frames.
faster-whisper then receives degraded audio and returns garbled transcripts. The
same PowerConf over USB delivers its DSP-cleaned full-band stream intact and
transcribes cleanly.
This does not look like an audio problem. Wake detection keeps working over HFP --
a live "computah ..." still scores 0.998 -- because the wake model matches a short
fixed pattern that survives the degradation. Only the sentence after it comes back
wrong, so the loop appears healthy right up to the transcript.
To check a device before relying on it:
```bash
.venv/bin/python audio.py --list # flags unsuitable input devices
.venv/bin/python audio.py --test-mic "powerconf" # see the exit codes below
```
`--list` tags any input device it can tell is unsuitable and prints why. A
hands-free marker in the name works on every host. A low default rate is used only
for WASAPI, where it represents the shared-mode capture format; ALSA and CoreAudio
defaults are preferences and do not prove the device is narrowband. The live loop
prints the same warning at startup and keeps running, since the device is still
good enough for wake detection.
`--test-mic` goes further, because it has audio to look at: after capturing, it
inspects the spectrum for the cliff that upsampling leaves behind. Audio that
arrived at 16 kHz but started at 8 kHz carries almost nothing above 4 kHz, and no
amount of resampling puts it back. That catches the common Linux case where
PipeWire hides an 8 kHz transport behind a 16 kHz stream.
The spectral check has a strict boundary: energy above 4 kHz rules out that one
upsampling signature, but it does not rule out wideband HFP. A wideband HFP codec
can carry energy above 4 kHz while recompression and frame drops still garble
speech. A capture without the narrowband cliff is therefore reported as
inconclusive, as is a capture that is too short, silent, or constant.
`--test-mic` exit codes, which point at different fixes:
| Code | Meaning |
| --- | --- |
| 2 | Nothing arrived (only zeros). The mic is muted, unplugged, or the wrong source. |
| 3 | Frames arrived, but the device is unsuitable for speech. It works; pick a different one. |
| 4 | Frames arrived, but the available checks cannot establish suitability. Prefer USB or verify with transcription. |
The two layers fail differently. `--list` sees only what the device advertises:
wideband HFP negotiates 16 kHz and passes the rate test, and on Linux the
hands-free marker is usually absent from the name. `--test-mic` can flag an
upsampled narrowband stream from the audio, while leaving wideband HFP and a
resampler noisy enough to fill the empty band unresolved. A clean `--list` is not
proof a Bluetooth link is fine. Prefer USB.
## Configuration
`config.json` contains safe defaults that can be committed:
| Key | Meaning |
| --- | --- |
| `wake_word` | Active openWakeWord model name. |
| `wake_threshold` | Detection score required before the pipeline continues. |
| `log_level` | Live-loop log level: `DEBUG`, `INFO`, `WARNING`, `ERROR`, or `CRITICAL`. |
| `whisper_model` | faster-whisper model size or path. |
| `whisper_compute` | CTranslate2 compute type, usually `int8` on the target device. |
| `whisper_cpu_threads` | CTranslate2 CPU thread count. `0` lets CTranslate2 choose; set a positive count in `config.local.json` when measuring the target host. |
| `whisper_vad_filter` | Enable faster-whisper's transcription-time VAD filter. This is separate from the capture-time speech gate. |
| `stt_confidence_guard` | When true, a live turn drops a low-confidence transcript before the brain and speaks a re-prompt. |
| `stt_min_avg_logprob` | Floor for the transcript's mean per-token log-probability; below it the turn is rejected. This is the gate. |
| `stt_max_no_speech_prob` | How silence-like the audio looked. Combined with a low `avg_logprob` it labels a reject as silence; following faster-whisper's own rule, a confident decode is never rejected for this alone. |
| `voice_model` | Piper voice name/stem; `speak` loads `voices/.onnx`. |
| `brain_backend` | `cli` for standalone fallback or `bridge` for the persistent session path. |
| `claude_model` | Model name for the fallback CLI brain. |
| `claude_timeout_s` | Timeout for the fallback CLI brain. |
Use `config.local.json` for machine-specific bridge values. It is gitignored and merged over `config.json`, so private hostnames, usernames, and assistant paths do not leak into commits.
Both live loops write stage and timing records to stderr. Wake scores, spoken
transcripts, and replies appear only at `DEBUG`; the default `INFO` level does
not retain that content, and the built-in handler does not write a log file.
`live_driver.py --debug` overrides `log_level` for that process and adds
per-frame wake telemetry.
## Custom wake words
Built-in openWakeWord phrases appear in `--list-wake-words`. To add a custom wake word, place a trained `.onnx` model in `models/`. It appears as `` and can be selected the same way as a built-in model:
```bash
.venv/bin/python pipeline.py --set-wake-word
```
A custom model overrides a built-in model with the same name. Keep custom model files local unless you intend to publish them.
## Testing
Start with the fast tests. They do not load speech models:
```bash
.venv/bin/python test_brain_bridge.py
.venv/bin/python test_brain_dispatch.py
.venv/bin/python test_confidence_guard.py
.venv/bin/python test_logging.py
.venv/bin/python test_package_entry.py
.venv/bin/python test_prep_wake_samples.py
```
Run model-dependent tests when the voice and Whisper models are present:
```bash
.venv/bin/python test_pipeline_bridge.py
.venv/bin/python test_pipeline.py
```
On a memory-constrained host, cap the full bridge test:
```bash
systemd-run --user --scope -p MemoryMax=1500M -p MemorySwapMax=0 \
.venv/bin/python test_pipeline_bridge.py
```
## Latency notes
Measured on a Raspberry Pi 5 with warm models and a simulated brain:
| Stage | Approximate time |
| --- | --- |
| Wake detection | 0.8 s |
| Speech-to-text | 3.3 s |
| Text-to-speech | 3.9 s |
Those are one manual measurement. Regenerate them with `benchmark.py`, which runs the
file pipeline repeatedly on one fixed clip with warm models and prints a median/p95
table to paste over the one above, along with peak RSS:
```bash
systemd-run --user --scope -p MemoryMax=1500M -p MemorySwapMax=0 \
.venv/bin/python benchmark.py --runs 20
```
The clip is a fixed Piper utterance built from the configured wake word, synthesized
into `test_audio/benchmark_clip_.wav` on first run and reused after that, so
the input does not drift between runs. The wake word is in the filename because the clip
speaks it: switching `--wake-word` synthesizes a new clip rather than scoring the old
phrase against the new model. Only that default clip is synthesized on demand; an
explicit `--wav` that does not exist is an error rather than something to fill in, so a
typo cannot quietly benchmark generated audio in place of your sample.
The table above predates the script and was taken against a simulated brain, so it has
no Brain reply row. `benchmark.py` measures whichever backend and transport the config
names. Its Brain reply and End-to-end turn rows can therefore describe the `claude` CLI,
a live bridge session, or the configured local simulator. Say which one produced the
run; simulator rows are not assistant-session latency. Those rows depend on the model
or simulator and the load at the time.
Text-to-speech is the one to watch, because it looks like a stage row and is not
reproducible either: `run_pipeline` times `speak(reply, ...)`, so it renders the
assistant's answer, and its duration tracks how long that answer happens to be. Only
wake detection and speech-to-text are driven by the fixed clip. If you want a
comparable Text-to-speech number across runs, quote the reply length beside it or hold
the reply fixed.
So when you paste, replace the "simulated brain" caption too: say which backend and
model produced the run, and note that only the first two rows are input-driven. Keeping
the old caption over a pasted table would claim a simulated brain for rows that
measured a real one.
On a working bridge (`brain_backend: "bridge"` with its reply path set, plus the
transport's required host or inbox), every run writes the clip's transcript to the
configured endpoint, so the benchmark refuses to start and says so. This includes
`sim`: its inbox path is free-form configuration and may name a running session rather
than an isolated `SimPersona`. Pass `--live-brain` to measure the configured bridge on
purpose, or set `brain_backend` to `cli` to leave bridge inboxes alone. A rejected bridge
configuration answers locally and sends nothing, so it still runs without the flag.
It times the ssh hop to the brain host as its own row, and only when `brain_backend` is
`bridge`: a `cli` backend answers locally, so a leftover `brain_transport: "ssh"` buys no
hop. The brain stage is transport plus however long the assistant took to answer, and the
transport half is not one hop per turn: the floor is three (the reply-file read before the
send, the send, and an immediate first poll), or four when `brain_inbox_path` enables the
landing probe, which adds one post-send lookup of the turn's `event_id`. Then
`ssh_reply_reader` opens another connection for every later poll with nothing
multiplexing them, so it grows with the answer. Warming Piper or
whisper cannot reduce it.
The 3.9 s text-to-speech figure predates the resident voice: `speak()` now synthesizes
in process from a cached Piper voice, with the CLI only as a warned fallback, and
`warm_models` loads it before the first turn. Re-running the benchmark is what settles
whether text-to-speech is still the largest cost.
## Roadmap
- Train and ship a custom `computah` openWakeWord model using the [reproduction recipe](docs/training-computah.md).
- Add a live microphone loop with endpointing and playback.
- Keep Piper loaded between turns.
- Exercise the bridge against a live assistant session over the network.
- Explore a small wake-word satellite that streams audio only after detection.
## Known limitations
The bridge uses positional matching only for replies from an older producer that does not stamp an `event_id`. In that compatibility mode, a slow or dropped reply can leave the cursor out of step. Update the producer to restore identity matching before relying on multi-turn requests.
## Contributing
Keep changes small and tested:
1. Read `CLAUDE.md` before changing architecture or public docs.
2. Use sentence case for headings and user-facing text.
3. Avoid filler and hype; say what the project does in plain language.
4. Update docs when behavior changes.
5. Run the fastest relevant test first, then broader tests when local models are available.
6. Keep generated models, voices, local config, and assistant session data out of git.
## GitHub Pages
The site in `docs/` is ready for GitHub Pages. In repository settings, deploy Pages from the `docs` folder on the current branch. The page includes an SVG favicon and PNG social preview metadata.
## License
MIT. See `LICENSE`.