Snapshot: full project state

README.md

@@ -0,0 +1,66 @@
+# The Collector
+
+A smooth-operator conversational voice AI running on drjones' homelab. Lives on
+Proxmox LXC **CT 771** @ `10.30.20.172:8766`.
+
+## What it is
+A latency-optimized voice assistant. It listens, responds naturally through a
+local LLM, speaks with a natural voice, and can make outbound phone calls on
+request.
+
+## Pipeline (latency budget)
+| Stage | Engine | Latency |
+|-------|--------|---------|
+| STT | faster-whisper `small.en` (GPU, nightmare `.29:8770`) | ~0.35s / utterance |
+| LLM | `ornith-1.5:9b-64k` @ nightmare `.29:11434` (fallback `.222`) | ~600ms first token, streamed |
+| TTS | Piper `lessac-high` (CPU, 22.05k → 16k) | ~400ms / sentence, overlapped w/ LLM |
+
+Sentence-level TTS/LLM overlap: audio starts flowing after the first sentence
+completes while the model keeps generating. Server-side energy VAD with barge-in.
+
+## Interfaces
+- **Browser voice** — `GET /` (WebSocket `/ws`, PCM16 16 kHz full-duplex)
+- **Phone** — SignalWire (outbound via `POST /api/call {to, task}`; inbound stays
+  on JONES's number for now). Media stream `/signalwire/stream` (mulaw 8 kHz).
+- **Text** — `POST /api/text {text}`
+- **Health** — `GET /api/health`
+
+## Public URL
+Cloudflare quick tunnel (ephemeral): `https://placed-ivory-unto-effective.trycloudflare.com`
+Self-registers via `POST /api/config {public_base_url}` (systemd `collector-tunnel.service`).
+
+## Key decisions
+- **LLM lane = nightmare `.29`, not `.222`**: `.222` holds ornith resident but is
+  saturated by ~92 cron jobs + Kalshi bots and hangs on `/api/chat`; `.29` answers
+  in ~1s. `keep_alive:30m` keeps ornith warm there.
+- **STT on GPU (nightmare), not the CT CPU**: whisper on the CT's 2 cores was 7-8s
+  (the real latency killer); `faster-whisper` on nightmare's 4080S
+  (`collector-stt.service` @ `:8770`, CUDA) is ~0.35s — 20×. TTS stays on CPU (Piper
+  ~400ms, not the bottleneck).
+- **Phone** reuses the SignalWire account + number (`+1 254 513 9895`) JONES uses;
+  outbound TwiML points at this app's tunnel.
+
+## Files
+```
+/opt/collector/
+  app.py               self-contained FastAPI app (STT/TTS/LLM/phone)
+  config.yaml          model lane, TTS voice, SignalWire creds, persona
+  selftest.py          Piper→whisper round-trip self-test
+  static/index.html    browser voice UI (AudioWorklet)
+  tunnel.sh            cloudflared quick tunnel + self-registration
+  collector.service    systemd unit (app)
+  collector-tunnel.service  systemd unit (tunnel)
+```
+
+## Ops
+```bash
+# restart
+ssh root@10.30.20.85 "pct exec 771 -- systemctl restart collector"
+# logs
+ssh root@10.30.20.85 "pct exec 771 -- journalctl -u collector -n 30"
+# self-test the audio pipeline
+ssh root@10.30.20.85 "pct exec 771 -- python3 /opt/collector/selftest.py"
+# make a phone call
+curl -s -X POST http://10.30.20.172:8766/api/call \
+  -H 'Content-Type: application/json' -d '{"to":"+1XXXXXXXXXX","task":"..."}'
+```