67 lines
2.8 KiB
Markdown
67 lines
2.8 KiB
Markdown
# The Collector
|
||
|
||
A smooth-operator conversational voice AI running on drjones' homelab. Lives on
|
||
Proxmox LXC **CT 771** @ `10.30.20.172:8766`.
|
||
|
||
## What it is
|
||
A latency-optimized voice assistant. It listens, responds naturally through a
|
||
local LLM, speaks with a natural voice, and can make outbound phone calls on
|
||
request.
|
||
|
||
## Pipeline (latency budget)
|
||
| Stage | Engine | Latency |
|
||
|-------|--------|---------|
|
||
| STT | faster-whisper `small.en` (GPU, nightmare `.29:8770`) | ~0.35s / utterance |
|
||
| LLM | `ornith-1.5:9b-64k` @ nightmare `.29:11434` (fallback `.222`) | ~600ms first token, streamed |
|
||
| TTS | Piper `lessac-high` (CPU, 22.05k → 16k) | ~400ms / sentence, overlapped w/ LLM |
|
||
|
||
Sentence-level TTS/LLM overlap: audio starts flowing after the first sentence
|
||
completes while the model keeps generating. Server-side energy VAD with barge-in.
|
||
|
||
## Interfaces
|
||
- **Browser voice** — `GET /` (WebSocket `/ws`, PCM16 16 kHz full-duplex)
|
||
- **Phone** — SignalWire (outbound via `POST /api/call {to, task}`; inbound stays
|
||
on JONES's number for now). Media stream `/signalwire/stream` (mulaw 8 kHz).
|
||
- **Text** — `POST /api/text {text}`
|
||
- **Health** — `GET /api/health`
|
||
|
||
## Public URL
|
||
Cloudflare quick tunnel (ephemeral): `https://placed-ivory-unto-effective.trycloudflare.com`
|
||
Self-registers via `POST /api/config {public_base_url}` (systemd `collector-tunnel.service`).
|
||
|
||
## Key decisions
|
||
- **LLM lane = nightmare `.29`, not `.222`**: `.222` holds ornith resident but is
|
||
saturated by ~92 cron jobs + Kalshi bots and hangs on `/api/chat`; `.29` answers
|
||
in ~1s. `keep_alive:30m` keeps ornith warm there.
|
||
- **STT on GPU (nightmare), not the CT CPU**: whisper on the CT's 2 cores was 7-8s
|
||
(the real latency killer); `faster-whisper` on nightmare's 4080S
|
||
(`collector-stt.service` @ `:8770`, CUDA) is ~0.35s — 20×. TTS stays on CPU (Piper
|
||
~400ms, not the bottleneck).
|
||
- **Phone** reuses the SignalWire account + number (`+1 254 513 9895`) JONES uses;
|
||
outbound TwiML points at this app's tunnel.
|
||
|
||
## Files
|
||
```
|
||
/opt/collector/
|
||
app.py self-contained FastAPI app (STT/TTS/LLM/phone)
|
||
config.yaml model lane, TTS voice, SignalWire creds, persona
|
||
selftest.py Piper→whisper round-trip self-test
|
||
static/index.html browser voice UI (AudioWorklet)
|
||
tunnel.sh cloudflared quick tunnel + self-registration
|
||
collector.service systemd unit (app)
|
||
collector-tunnel.service systemd unit (tunnel)
|
||
```
|
||
|
||
## Ops
|
||
```bash
|
||
# restart
|
||
ssh root@10.30.20.85 "pct exec 771 -- systemctl restart collector"
|
||
# logs
|
||
ssh root@10.30.20.85 "pct exec 771 -- journalctl -u collector -n 30"
|
||
# self-test the audio pipeline
|
||
ssh root@10.30.20.85 "pct exec 771 -- python3 /opt/collector/selftest.py"
|
||
# make a phone call
|
||
curl -s -X POST http://10.30.20.172:8766/api/call \
|
||
-H 'Content-Type: application/json' -d '{"to":"+1XXXXXXXXXX","task":"..."}'
|
||
```
|