Add rage table-flip state; wire gateway to live Speaches; add CI pipeline
Test, Build and Push / test-gateway (push) Successful in 1m2s
Test, Build and Push / release (push) Skipped
Test, Build and Push / build-gateway (push) Skipped

- Face: new 'rage' state — 3-frame kaomoji loop (stare, flip the table,
  put it back) for in-flight request failures; 'error' stays the quiet
  persistent face for a dead link. Sim + artifact + design doc updated.
- Gateway: stt.py/tts.py are now pluggable backends. Default 'speaches'
  talks OpenAI-format HTTP to the live container on :8601
  (faster-whisper-small STT, Kokoro bm_george TTS with 24->16 kHz
  audioop resample); 'embedded' fallback kept behind the [speech] extra.
  Verified with a live TTS->STT round trip (warm: STT 0.27s, TTS 1.9s).
  Docker image is now slim (no CUDA/ML deps). Python pinned to 3.12
  (system 3.8 too old, audioop gone in 3.13).
- CI: .gitea/workflows/build.yml — lint+test on main pushes; on v* tags
  test, build gateway image, push to registry, release, and trigger
  Watchtower (tatlock pattern; needs REGISTRY_USER/REGISTRY_PASSWORD/
  WATCHTOWER_TOKEN secrets). Runtime stack in deploy/desklock-gateway.yml.
- architecture.md: measured speech latencies, deployed-Speaches status,
  CI & deployment section.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-14 17:54:30 +02:00
co-authored by Claude Fable 5
parent 42b988b7ef
commit f5df5049db
10 changed files with 283 additions and 41 deletions
+34 -10
View File
@@ -87,9 +87,6 @@ models — the container stays a slim pure-Python image with no CUDA/ML dependen
- `embedded` — in-process faster-whisper / Piper. Kept as a fallback so the gateway can
run standalone (dev on a laptop, speech container down), at the cost of a fat image.
> **Status note:** the initial scaffold implements only the `embedded` path; the
> backend switch and Speaches client are the next gateway task.
The gateway is stateless apart from in-flight conversations; it can restart freely.
### 3. Speech layer — Speaches (container, GPU)
@@ -99,9 +96,15 @@ is a self-hosted, OpenAI-API-compatible speech server: STT via faster-whisper, T
Kokoro/Piper, dynamic model load/offload with a TTL, and a `/v1/realtime` WebSocket API
we may adopt later for streaming transcription.
- **Deployment**: its own stack in `system-admin-toj/containers/stacks/`, GPU-enabled.
Proposed host port **8601** (verified free; register in `CONTAINERS.md` at deploy).
LAN-only like the Tatlock internal route — do not expose through NPM without auth.
- **Deployed 2026-07-14**: `ghcr.io/speaches-ai/speaches:latest-cuda` on host port
**8601**, with `Systran/faster-whisper-small` (STT) and
`speaches-ai/Kokoro-82M-v1.0-ONNX` (TTS, 24 kHz — the gateway resamples to the
16 kHz device contract; default voice `bm_george`, en-GB male). LAN-only like the
Tatlock internal route — do not expose through NPM without auth. Register in
`CONTAINERS.md`.
- **Measured** (live round trip through the gateway code, warm): STT ~0.3 s for a
~3 s utterance; TTS ~1.9 s for a ~3 s sentence. Cold start after model TTL offload
adds ~510 s to the first request.
- **Why a shared layer instead of models inside the gateway**: one GPU-resident model
instance serves the whole homelab. Open WebUI is currently configured with
`AUDIO_STT_ENGINE=openai` / `AUDIO_TTS_ENGINE=openai` (OpenAI *cloud*) — pointing its
@@ -137,6 +140,7 @@ ported to LVGL. The `STATES` table in the sim defines the contract:
| `pensive` | `· ·` | `~` | 7 streams | cycling `...` thought dots |
| `effort` | `> <` | `~` | 40 fast streams | **orbit arc on bezel + `[ Ns ]` elapsed counter**, face jitter |
| `speaking` | `^ ^` | cycles `o O - O = o` | 14 streams | mouth animates ~150 ms/frame |
| `rage` | — | — | 34 fast streams | 3-frame kaomoji loop through the eyes slot: `(°□°) ┬─┬``(╯°□°)╯︵ ┻━┻``┬─┬ ( º_º )` — flips the table, then composes itself and puts it back |
| `error` | `x x` | `-` | none (rain dies) | face dims to 45% |
**Wait cues are a hard requirement** (user-stated): Tatlock turns take 1025 s, so
@@ -145,8 +149,9 @@ elapsed-seconds counter, and max rain. Never a bare static face during a wait, a
fake progress bars — only honest cues.
**Protocol → face mapping**: gateway `state: thinking``effort`; transcription and
other short local waits → `pensive`; `listening`/`speaking` map 1:1; WebSocket
disconnected → `error`; otherwise `idle`.
other short local waits → `pensive`; `listening`/`speaking` map 1:1; an in-flight
request failure (STT/Tatlock/TTS error) → `rage` for a few loops, then `idle`;
WebSocket disconnected → `error` (quiet, persistent); otherwise `idle`.
**LVGL port notes** (for phase 2):
@@ -155,6 +160,7 @@ disconnected → `error`; otherwise `idle`.
- Rain: `lv_canvas` (or a pooled label grid) with per-frame fade; orbit arc = `lv_arc`.
- Fonts: generate a large monospace glyph font including the katakana subset used in
`GLYPHS` via `lv_font_conv`; the built-in `unscii` fonts are too small for 800 px.
The `rage` frames additionally need `╯ ︵ ┻ ━ ┬ ─ ノ ° □ º` in the subset.
- The sim's text glow (`text-shadow`) is browser flair — the device renders flat glyphs.
## Latency budget & streaming
@@ -164,8 +170,8 @@ GPU-resident benchmarks of 2026-07-14, gemma4:e2b at ~100 tok/s):
| Stage | Cost |
|-------|------|
| STT (whisper `small`, GPU) | a few hundred ms for a ~5 s utterance |
| TTS (Piper/Kokoro) | faster than realtime |
| STT (Speaches whisper `small`) | ~0.3 s warm (measured) |
| TTS (Speaches Kokoro) | ~1.9 s per ~3 s sentence, warm (measured) |
| Tatlock Steward analysis | ~6 s warm |
| **Tatlock, full local flow** | **1125 s end-to-end** (librarian-routed ~2025 s) |
| Tatlock cold start (>2 h idle) | +~8 s (`OLLAMA_KEEP_ALIVE=2h`) |
@@ -239,3 +245,21 @@ it is the one contract between the two halves of the repo.
gateway is also where a future second endpoint (kitchen, office) would connect.
- **Monorepo**: the WS protocol couples firmware and gateway; versioning them together
avoids contract drift.
## CI & deployment
Gitea Actions (`.gitea/workflows/build.yml`), following the tatlock/tatlock-ui pattern:
- **Every push to `main`**: lint + tests for the gateway (Python 3.12).
- **Version tags (`v0.1.0`, …)**: tests, then build `gateway/` into
`git.schweitz.internal/jpmschweitzer/desklock-gateway:{latest,tag}`, push to the
Gitea registry, create a release, and trigger Watchtower to roll the running
container.
- Required repo/org secrets: `REGISTRY_USER`, `REGISTRY_PASSWORD`,
`WATCHTOWER_TOKEN` (same trio tatlock uses).
- The runtime stack definition lives in `deploy/desklock-gateway.yml`; copy it into
`system-admin-toj/containers/stacks/` to deploy, and register port 8600 in
`CONTAINERS.md`.
Firmware is not containerized: it's flashed over USB (`idf.py flash`), with OTA planned
for phase 5.