Add rage table-flip state; wire gateway to live Speaches; add CI pipeline
- Face: new 'rage' state — 3-frame kaomoji loop (stare, flip the table, put it back) for in-flight request failures; 'error' stays the quiet persistent face for a dead link. Sim + artifact + design doc updated. - Gateway: stt.py/tts.py are now pluggable backends. Default 'speaches' talks OpenAI-format HTTP to the live container on :8601 (faster-whisper-small STT, Kokoro bm_george TTS with 24->16 kHz audioop resample); 'embedded' fallback kept behind the [speech] extra. Verified with a live TTS->STT round trip (warm: STT 0.27s, TTS 1.9s). Docker image is now slim (no CUDA/ML deps). Python pinned to 3.12 (system 3.8 too old, audioop gone in 3.13). - CI: .gitea/workflows/build.yml — lint+test on main pushes; on v* tags test, build gateway image, push to registry, release, and trigger Watchtower (tatlock pattern; needs REGISTRY_USER/REGISTRY_PASSWORD/ WATCHTOWER_TOKEN secrets). Runtime stack in deploy/desklock-gateway.yml. - architecture.md: measured speech latencies, deployed-Speaches status, CI & deployment section. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
+34
-10
@@ -87,9 +87,6 @@ models — the container stays a slim pure-Python image with no CUDA/ML dependen
|
||||
- `embedded` — in-process faster-whisper / Piper. Kept as a fallback so the gateway can
|
||||
run standalone (dev on a laptop, speech container down), at the cost of a fat image.
|
||||
|
||||
> **Status note:** the initial scaffold implements only the `embedded` path; the
|
||||
> backend switch and Speaches client are the next gateway task.
|
||||
|
||||
The gateway is stateless apart from in-flight conversations; it can restart freely.
|
||||
|
||||
### 3. Speech layer — Speaches (container, GPU)
|
||||
@@ -99,9 +96,15 @@ is a self-hosted, OpenAI-API-compatible speech server: STT via faster-whisper, T
|
||||
Kokoro/Piper, dynamic model load/offload with a TTL, and a `/v1/realtime` WebSocket API
|
||||
we may adopt later for streaming transcription.
|
||||
|
||||
- **Deployment**: its own stack in `system-admin-toj/containers/stacks/`, GPU-enabled.
|
||||
Proposed host port **8601** (verified free; register in `CONTAINERS.md` at deploy).
|
||||
LAN-only like the Tatlock internal route — do not expose through NPM without auth.
|
||||
- **Deployed 2026-07-14**: `ghcr.io/speaches-ai/speaches:latest-cuda` on host port
|
||||
**8601**, with `Systran/faster-whisper-small` (STT) and
|
||||
`speaches-ai/Kokoro-82M-v1.0-ONNX` (TTS, 24 kHz — the gateway resamples to the
|
||||
16 kHz device contract; default voice `bm_george`, en-GB male). LAN-only like the
|
||||
Tatlock internal route — do not expose through NPM without auth. Register in
|
||||
`CONTAINERS.md`.
|
||||
- **Measured** (live round trip through the gateway code, warm): STT ~0.3 s for a
|
||||
~3 s utterance; TTS ~1.9 s for a ~3 s sentence. Cold start after model TTL offload
|
||||
adds ~5–10 s to the first request.
|
||||
- **Why a shared layer instead of models inside the gateway**: one GPU-resident model
|
||||
instance serves the whole homelab. Open WebUI is currently configured with
|
||||
`AUDIO_STT_ENGINE=openai` / `AUDIO_TTS_ENGINE=openai` (OpenAI *cloud*) — pointing its
|
||||
@@ -137,6 +140,7 @@ ported to LVGL. The `STATES` table in the sim defines the contract:
|
||||
| `pensive` | `· ·` | `~` | 7 streams | cycling `...` thought dots |
|
||||
| `effort` | `> <` | `~` | 40 fast streams | **orbit arc on bezel + `[ Ns ]` elapsed counter**, face jitter |
|
||||
| `speaking` | `^ ^` | cycles `o O - O = o` | 14 streams | mouth animates ~150 ms/frame |
|
||||
| `rage` | — | — | 34 fast streams | 3-frame kaomoji loop through the eyes slot: `(°□°) ┬─┬` → `(╯°□°)╯︵ ┻━┻` → `┬─┬ ノ( º_º ノ)` — flips the table, then composes itself and puts it back |
|
||||
| `error` | `x x` | `-` | none (rain dies) | face dims to 45% |
|
||||
|
||||
**Wait cues are a hard requirement** (user-stated): Tatlock turns take 10–25 s, so
|
||||
@@ -145,8 +149,9 @@ elapsed-seconds counter, and max rain. Never a bare static face during a wait, a
|
||||
fake progress bars — only honest cues.
|
||||
|
||||
**Protocol → face mapping**: gateway `state: thinking` → `effort`; transcription and
|
||||
other short local waits → `pensive`; `listening`/`speaking` map 1:1; WebSocket
|
||||
disconnected → `error`; otherwise `idle`.
|
||||
other short local waits → `pensive`; `listening`/`speaking` map 1:1; an in-flight
|
||||
request failure (STT/Tatlock/TTS error) → `rage` for a few loops, then `idle`;
|
||||
WebSocket disconnected → `error` (quiet, persistent); otherwise `idle`.
|
||||
|
||||
**LVGL port notes** (for phase 2):
|
||||
|
||||
@@ -155,6 +160,7 @@ disconnected → `error`; otherwise `idle`.
|
||||
- Rain: `lv_canvas` (or a pooled label grid) with per-frame fade; orbit arc = `lv_arc`.
|
||||
- Fonts: generate a large monospace glyph font including the katakana subset used in
|
||||
`GLYPHS` via `lv_font_conv`; the built-in `unscii` fonts are too small for 800 px.
|
||||
The `rage` frames additionally need `╯ ︵ ┻ ━ ┬ ─ ノ ° □ º` in the subset.
|
||||
- The sim's text glow (`text-shadow`) is browser flair — the device renders flat glyphs.
|
||||
|
||||
## Latency budget & streaming
|
||||
@@ -164,8 +170,8 @@ GPU-resident benchmarks of 2026-07-14, gemma4:e2b at ~100 tok/s):
|
||||
|
||||
| Stage | Cost |
|
||||
|-------|------|
|
||||
| STT (whisper `small`, GPU) | a few hundred ms for a ~5 s utterance |
|
||||
| TTS (Piper/Kokoro) | faster than realtime |
|
||||
| STT (Speaches whisper `small`) | ~0.3 s warm (measured) |
|
||||
| TTS (Speaches Kokoro) | ~1.9 s per ~3 s sentence, warm (measured) |
|
||||
| Tatlock Steward analysis | ~6 s warm |
|
||||
| **Tatlock, full local flow** | **11–25 s end-to-end** (librarian-routed ~20–25 s) |
|
||||
| Tatlock cold start (>2 h idle) | +~8 s (`OLLAMA_KEEP_ALIVE=2h`) |
|
||||
@@ -239,3 +245,21 @@ it is the one contract between the two halves of the repo.
|
||||
gateway is also where a future second endpoint (kitchen, office) would connect.
|
||||
- **Monorepo**: the WS protocol couples firmware and gateway; versioning them together
|
||||
avoids contract drift.
|
||||
|
||||
## CI & deployment
|
||||
|
||||
Gitea Actions (`.gitea/workflows/build.yml`), following the tatlock/tatlock-ui pattern:
|
||||
|
||||
- **Every push to `main`**: lint + tests for the gateway (Python 3.12).
|
||||
- **Version tags (`v0.1.0`, …)**: tests, then build `gateway/` into
|
||||
`git.schweitz.internal/jpmschweitzer/desklock-gateway:{latest,tag}`, push to the
|
||||
Gitea registry, create a release, and trigger Watchtower to roll the running
|
||||
container.
|
||||
- Required repo/org secrets: `REGISTRY_USER`, `REGISTRY_PASSWORD`,
|
||||
`WATCHTOWER_TOKEN` (same trio tatlock uses).
|
||||
- The runtime stack definition lives in `deploy/desklock-gateway.yml`; copy it into
|
||||
`system-admin-toj/containers/stacks/` to deploy, and register port 8600 in
|
||||
`CONTAINERS.md`.
|
||||
|
||||
Firmware is not containerized: it's flashed over USB (`idf.py flash`), with OTA planned
|
||||
for phase 5.
|
||||
|
||||
Reference in New Issue
Block a user