release v0.2.2

Network migration: the gateway's default Tatlock URL is now the docker
container name (http://tatlock:8000); the retiring tatlock.schweitz.internal
domain is gone from config and docs. Deployments that set
DESKLOCK_TATLOCK_BASE_URL are unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

.claude/skills/device-screenshot/SKILL.md

@@ -0,0 +1,83 @@
+---
+name: device-screenshot
+description: >
+  Capture a screenshot of the live DeskLock ESP32-P4 display over USB and view
+  it as a PNG. Use whenever you need to SEE what's on the device screen — verify
+  a face/UI change, check the tap controls overlay, confirm a state (idle,
+  listening, effort), or debug layout remotely without the user's camera.
+  Triggers: "screenshot the device", "what's on the screen", "capture the
+  display", "show me the face", "did the UI change land".
+---
+
+# DeskLock device screenshot over USB
+
+Grabs the current LVGL screen off the device and rebuilds it as a PNG on the
+host. No camera, no gateway — it rides the USB serial that's already attached for
+flashing.
+
+## How it works (so you can debug it)
+
+The firmware (`main/face.c` `face_screenshot_dump`, gated by `DESKLOCK_DEVMODE`
+in `desklock_main.c`) runs a task that watches the **USB-serial-JTAG RX FIFO**
+for a trigger byte, renders the active screen with `lv_snapshot_take_to_draw_buf`
+into a PSRAM buffer, 2×-downscales to 400×400, and streams it as **raw binary**
+straight to the USB-serial-JTAG TX FIFO — framed by a text header
+`###SHOT_BEGIN … bytes=N crc=0x… bin=1###` + N bytes + `###SHOT_END###`. The host
+tool `firmware/tools/device_shot.py` sends the trigger, reads N bytes, checks the
+CRC (retrying on the rare dropped frame), and writes a PNG.
+
+Two things this design is deliberately built around, learned the hard way:
+- **Don't use `printf`.** The primary console is UART at 115200 baud (~11 KB/s) —
+  a frame would take ~37 s. Writing straight to the USB FIFO runs at USB speed
+  (~1 s). That's why the dump uses `usb_serial_jtag_ll_write_txfifo`, not stdout.
+- **Opening the port does NOT reset the P4** (unlike esptool), and there's no USB
+  stdin, so the trigger is a byte the firmware polls for — on-demand, no reboot,
+  captures whatever state is currently on screen.
+
+## Prerequisites
+
+- Firmware flashed with **dev mode ON** — it's OFF by default (production spends
+  no internal RAM on the watcher and nothing extra runs on the render path). Flash
+  a dev build with:
+  ```bash
+  cd firmware && sg dialout -c "bash -c 'source ~/esp-idf/export.sh && idf.py -DDESKLOCK_DEVMODE=ON -p /dev/ttyACM0 flash'"
+  ```
+  If screenshots return "no frame", the flashed build is production — reflash with
+  `-DDESKLOCK_DEVMODE=ON`. Back to production: `-DDESKLOCK_DEVMODE=OFF` (the value
+  sticks in the CMake cache until you flip it). Leave it ON for a UI-dev session
+  (you're reflashing for UI changes anyway); flip OFF for the final/production flash.
+- Device on `/dev/ttyACM0`. The port needs the `dialout` group, so run the tool
+  under `sg dialout -c '…'` (this login session predates dialout membership).
+- Nothing else holding the port (no `idf.py monitor` running) — one owner only.
+
+## Take a shot
+
+From the `desklock` repo root (`/mnt/media/Projects/desklock`):
+
+```bash
+# current screen (whatever state the device is in right now)
+sg dialout -c "python3 firmware/tools/device_shot.py /tmp/shot.png"
+
+# force the tap controls overlay in-frame (mic + volume) — for verifying it
+# remotely since you can't physically tap
+sg dialout -c "python3 firmware/tools/device_shot.py /tmp/shot.png --overlay"
+```
+
+Then **Read `/tmp/shot.png`** to view it. A clean run prints
+`[try 1] 400x400 320000B crc … OK` and takes ~5 s.
+
+Trigger bytes: `s` = current screen, `o` = force overlay. The tool retries up to
+4× on a CRC mismatch (occasional console contention), so a transient bad frame
+self-heals.
+
+## Notes / caveats
+
+- **RAM / render path:** the watcher costs one ~5 KB internal-RAM task, so it's
+  behind `DESKLOCK_DEVMODE` (off by default). Internal RAM is the scarce resource
+  on this board (it caps the rain sprite pool). Never ship a production build with
+  it on.
+- 400×400 is plenty for layout/UI checks. To change resolution, adjust `SHOT_DS`
+  in `face.c` (the downscale factor) — the host reads the size from the header.
+- If you just reflashed, wait ~7 s for boot before the first shot.
+- Only the USB console path is used; this does not touch the device↔gateway
+  WebSocket protocol.

.gitea/workflows/build.yml

@@ -49,7 +49,7 @@ jobs:
       - name: Login to Gitea Registry
         uses: docker/login-action@v3
         with:
-          registry: git.schweitz.internal
+          registry: git.schweitz.net
           username: ${{ secrets.REGISTRY_USER }}
           password: ${{ secrets.REGISTRY_PASSWORD }}
 
@@ -61,8 +61,8 @@ jobs:
           provenance: false
           sbom: false
           tags: |
-            git.schweitz.internal/jpmschweitzer/desklock-gateway:latest
+            git.schweitz.net/jpmschweitzer/desklock-gateway:latest
-            git.schweitz.internal/jpmschweitzer/desklock-gateway:${{ github.ref_name }}
+            git.schweitz.net/jpmschweitzer/desklock-gateway:${{ github.ref_name }}
 
       - name: Trigger Watchtower update
         if: success()

AGENTS.md

@@ -6,8 +6,8 @@
 ## What this project is
 
 DeskLock is the living-room visual/audio endpoint for **Tatlock**, the homelab butler
-(`/mnt/media/Projects/tatlock`, API at `http://tatlock.schweitz.internal:8000`). Two halves,
+(`/mnt/media/Projects/tatlock`, API at `http://tatlock:8000` on `docker-dataplane`).
-one repo:
+Two halves, one repo:
 
 - `firmware/` — ESP-IDF (C, LVGL 9) app for the Waveshare ESP32-P4-WIFI6-Touch-LCD-3.4C
   (3.4" round 800×800 touch display, dual mics + ES7210 AEC, ES8311 codec + speaker).
@@ -124,7 +124,8 @@ make typecheck      # mypy
 ## Homelab context
 
 - This server **is** tower-of-joy; the device, gateway, and Tatlock all share the LAN.
-- Use `tatlock.schweitz.internal:8000` (direct, no SSO) — the public
+- Machine-to-machine traffic uses container names on the `docker-dataplane` network —
-  `tatlock.schweitz.net` route sits behind Authentik and is not for machine-to-machine
+  `http://tatlock:8000` (the deployed stack overrides `DESKLOCK_TATLOCK_BASE_URL` to
-  traffic.
+  the host LAN IP while Tatlock runs on the host). Browser-facing URLs use
+  `https://<name>.schweitz.net` (NPM, Authentik SSO) — never for M2M traffic.
 - Git remote: `git.schweitz.net` (Gitea).

CHANGELOG.md

@@ -8,6 +8,38 @@ until the first tagged release.
 
 ## [Unreleased]
 
+## [0.2.2] — 2026-07-19
+
+### Changed
+
+- The gateway's default Tatlock URL is now `http://tatlock:8000` (docker
+  container name) — the retiring `tatlock.schweitz.internal` domain is gone
+  from config and docs. Deployments setting `DESKLOCK_TATLOCK_BASE_URL` are
+  unaffected.
+
+## [0.2.1] — 2026-07-15
+
+### Fixed
+
+- A spoken volume command no longer leaves the face stuck in the thinking
+  spinner — the feedback gong is a UI cue and no longer holds the busy state,
+  so the return-to-idle isn't swallowed.
+- The butler filler line reads as one phrase instead of two — reworded to "Let
+  me check on that, sir." to avoid a text-to-speech pause before "for you".
+
+## [0.2.0] — 2026-07-15
+
+### Added
+
+- After a request, the butler immediately says "Let me check on that for you,
+  sir" and keeps a spinner up while it works, so the long wait isn't dead air.
+- Spoken volume commands ("volume up/down", "mute/unmute", "set volume to N",
+  "this one goes to eleven") are handled instantly on the gateway, bypassing the
+  assistant — no waiting on a reply just to change the volume.
+- Tap the screen to reveal on-screen controls — a microphone button plus volume
+  down/up and a live level bar, using Phosphor icon glyphs. Tapping the dimmed
+  backdrop dismisses them.
+
 ### Fixed
 
 - Fixed the blue-screen flicker during voice activity — the matrix rain now
@@ -16,5 +48,7 @@ until the first tagged release.
 
 ### Changed
 
+- Volume is now an 0–11 scale (it goes to eleven) with a gong cue on each change
+  and a little "11" that floats off the bar at maximum.
 - Matrix rain is drawn from a reused pool of pre-rendered streak sprites instead
   of live text labels, and eases off while the device is listening or speaking.

README.md

@@ -38,8 +38,8 @@ happens on this server.
                                      ▼       │  Speaches  (container, GPU) │
                      ┌────────────────────┐  │  • STT: faster-whisper      │
                      │  Tatlock (butler)  │  │  • TTS: Kokoro / Piper      │
-                     │  tatlock.schweitz. │  │  also usable by Open WebUI, │
+                     │  http://tatlock    │  │  also usable by Open WebUI, │
-                     │  internal :8000    │  │  Home Assistant, …          │
+                     │  :8000             │  │  Home Assistant, …          │
                      └────────────────────┘  └─────────────────────────────┘
 ```
 

docs/architecture.md

@@ -74,7 +74,7 @@ models — the container stays a slim pure-Python image with no CUDA/ML dependen
 2. Buffers inbound PCM until end-of-utterance (client-signalled in phase 1; VAD later).
 3. **STT**: POST to Speaches `/v1/audio/transcriptions`.
 4. **Chat**: POST the transcript to Tatlock `/v1/chat/completions`
-   (`http://tatlock.schweitz.internal:8000`, OpenAI-compatible, **streaming**),
+   (`http://tatlock:8000`, OpenAI-compatible, **streaming**),
    maintaining the conversation history so follow-ups have context.
 5. **TTS**: as Tatlock's token stream completes each sentence, POST it to Speaches
    `/v1/audio/speech` and forward the PCM immediately — see
@@ -266,8 +266,17 @@ gateway → device:  {"type": "reply_text", "text": "..."}
 gateway → device:  {"type": "audio_start", "sample_rate": 16000}
 gateway → device:  <binary PCM frames>   (may arrive sentence-by-sentence; play as a stream)
 gateway → device:  {"type": "audio_end"}
+
+gateway → device:  {"type": "command", "action": "volume_up"}   (LLM-bypass; see below)
 ```
 
+`command` (gateway → device) is an **alternative to the reply path**: when the
+gateway recognizes a simple device command in the transcript (volume/mute), it
+sends a `command` instead of calling Tatlock — no `reply_text`/audio — then returns
+to `idle`. Actions: `volume_up`, `volume_down`, `mute`, `unmute`, and `volume_set`
+with an extra `"level"` field (0–11, the on-device volume scale). Matched by the
+gateway's `commands.py`; applied on the device in `gw_client.c` → `face.c`.
+
 Planned additions (documented before implemented, here first):
 
 - `reply_delta` (gateway → device): incremental reply text for on-screen streaming while
@@ -326,7 +335,7 @@ Gitea Actions (`.gitea/workflows/build.yml`), following the tatlock/tatlock-ui p
 
 - **Every push to `main`**: lint + tests for the gateway (Python 3.12).
 - **Version tags (`v0.1.0`, …)**: tests, then build `gateway/` into
-  `git.schweitz.internal/jpmschweitzer/desklock-gateway:{latest,tag}`, push to the
+  `git.schweitz.net/jpmschweitzer/desklock-gateway:{latest,tag}`, push to the
   Gitea registry, create a release, and trigger Watchtower to roll the running
   container.
 - Required repo/org secrets: `REGISTRY_USER`, `REGISTRY_PASSWORD`,

firmware/main/CMakeLists.txt

@@ -10,9 +10,21 @@ idf_component_register(
         "fonts/font_face_140.c"
         "fonts/font_rain_22.c"
         "fonts/font_rage_64.c"
+        "fonts/font_phosphor_64.c"
     INCLUDE_DIRS "."
     EMBED_FILES "c6_slave.bin"
     PRIV_REQUIRES nvs_flash esp_wifi esp_event esp_netif json esp_timer esp_app_format esp_http_server esp-sr
 )
 
 target_compile_definitions(${COMPONENT_LIB} PRIVATE LV_LVGL_H_INCLUDE_SIMPLE)
+
+# Dev-only features (screenshot watcher). OFF in production so nothing extra runs
+# on the render path / spends internal RAM. Deploy dev with:
+#   idf.py -DDESKLOCK_DEVMODE=ON build flash
+# and back to production with -DDESKLOCK_DEVMODE=OFF (the value sticks in the
+# CMake cache until you flip it).
+option(DESKLOCK_DEVMODE "Enable dev-only features (screenshot watcher)" OFF)
+if(DESKLOCK_DEVMODE)
+    target_compile_definitions(${COMPONENT_LIB} PRIVATE DESKLOCK_DEVMODE=1)
+    message(STATUS "DeskLock: DEVMODE ON — screenshot watcher enabled")
+endif()

firmware/main/audio.c

@@ -29,12 +29,20 @@ static const char *TAG = "audio";
 
 #define GONG_SECONDS 5
 #define GONG_SAMPLES (RATE * GONG_SECONDS)
-#define GONG_VOLUME 75
 #define GONG_PEAK 14000.0f
 
+/* Speaker volume is an 11-step level (0..AUDIO_VOL_MAX). This one goes to eleven. */
+#define VOL_DEFAULT_LEVEL 8               /* ~73%, matches the previous 75 default */
+
 static esp_codec_dev_handle_t s_spk;
 static esp_codec_dev_handle_t s_mic;
+static int  s_level = VOL_DEFAULT_LEVEL;  /* speaker volume, 0..AUDIO_VOL_MAX */
+static bool s_muted;
+static int level_to_pct(int level) { return level * 100 / AUDIO_VOL_MAX; }
 static int16_t *s_gong;
+#define GONG_CHUNK 1600                 /* 100 ms — cut-off granularity for a restart */
+static volatile uint32_t s_gong_gen;    /* bumped per request; a change mid-play restarts */
+static volatile bool s_gong_active;
 
 static uint8_t *s_reply;
 static volatile size_t s_reply_len;
@@ -152,7 +160,21 @@ static void audio_task(void *arg)
          * ourselves. The tail delay covers the DMA that plays after write()
          * returns, preventing a playback-boundary false wake. */
         if (job == JOB_GONG && s_gong != NULL) {
-            esp_codec_dev_write(s_spk, s_gong, GONG_SAMPLES * sizeof(int16_t));
+            /* Play in chunks so a new request (a fresh volume change) cuts the
+             * current gong off and restarts, instead of queueing another 5 s.
+             * Note: the gong is a UI cue, not "playback" — it deliberately does
+             * NOT set s_playing, so a following state:idle isn't swallowed (that
+             * guard is for reply audio) and it doesn't gate wake for 5 s. */
+            uint32_t gen;
+            do {
+                gen = s_gong_gen;
+                for (size_t off = 0; off < GONG_SAMPLES && s_gong_gen == gen; off += GONG_CHUNK) {
+                    size_t n = GONG_SAMPLES - off;
+                    if (n > GONG_CHUNK) n = GONG_CHUNK;
+                    esp_codec_dev_write(s_spk, s_gong + off, n * sizeof(int16_t));
+                }
+            } while (s_gong_gen != gen);   /* a new request arrived mid-play → restart */
+            s_gong_active = false;
         } else if (job == JOB_CHIME && s_chime != NULL) {
             s_playing = true;
             esp_codec_dev_write(s_spk, s_chime, CHIME_SAMPLES * sizeof(int16_t));
@@ -276,7 +298,7 @@ void audio_init(void)
     };
     s_spk = bsp_audio_codec_speaker_init();
     if (s_spk != NULL) {
-        esp_codec_dev_set_out_vol(s_spk, GONG_VOLUME);
+        esp_codec_dev_set_out_vol(s_spk, level_to_pct(s_level));
         esp_codec_dev_open(s_spk, &fs);
     }
     s_mic = bsp_audio_codec_microphone_init();
@@ -303,8 +325,14 @@ void audio_play_chime(void)
 
 void audio_play_gong(void)
 {
-    job_t job = JOB_GONG;
+    s_gong_gen++;                       /* (re)start; cuts any in-flight gong */
-    xQueueSend(s_jobs, &job, 0);
+    if (!s_gong_active) {
+        s_gong_active = true;
+        job_t job = JOB_GONG;
+        if (xQueueSend(s_jobs, &job, 0) != pdTRUE) {
+            s_gong_active = false;      /* queue full — nothing will run it */
+        }
+    }
 }
 
 /* Streaming = the detect_task forwards AFE audio upstream. Wake word and touch
@@ -348,3 +376,35 @@ bool audio_is_playing(void)
 {
     return s_playing;
 }
+
+void audio_level_set(int level)
+{
+    if (level < 0) {
+        level = 0;
+    } else if (level > AUDIO_VOL_MAX) {
+        level = AUDIO_VOL_MAX;
+    }
+    s_level = level;
+    s_muted = false;                      /* setting a level unmutes */
+    if (s_spk != NULL) {
+        esp_codec_dev_set_out_vol(s_spk, level_to_pct(s_level));
+    }
+}
+
+int audio_level_get(void)
+{
+    return s_level;
+}
+
+void audio_mute(bool on)
+{
+    s_muted = on;                         /* gate output; s_level is preserved */
+    if (s_spk != NULL) {
+        esp_codec_dev_set_out_vol(s_spk, on ? 0 : level_to_pct(s_level));
+    }
+}
+
+bool audio_is_muted(void)
+{
+    return s_muted;
+}

firmware/main/desklock.h

@@ -22,6 +22,8 @@ void face_set(face_state_t state);      /* safe from any task */
 face_state_t face_get(void);
 void face_activity(void);
 void face_status(const char *text);   /* bottom status line (diag/boot) */               /* reset the power ladder to active */
+void face_screenshot_dump(bool overlay);  /* dev: raw RGB565 screen dump over USB */
+void face_volume_command(const char *action, int level);  /* apply a gateway volume command */
 
 /* audio.c */
 void audio_init(void);
@@ -34,6 +36,11 @@ void audio_playback_begin(void);
 void audio_playback_feed(const uint8_t *data, size_t len);
 void audio_playback_end(void);
 bool audio_is_playing(void);
+#define AUDIO_VOL_MAX 11                /* volume level range 0..11 (goes to eleven) */
+void audio_level_set(int level);        /* 0..AUDIO_VOL_MAX; also unmutes */
+int  audio_level_get(void);
+void audio_mute(bool on);               /* gate output; keeps the set level */
+bool audio_is_muted(void);
 
 /* net.c */
 void net_start(void);

firmware/main/desklock_main.c

@@ -30,6 +30,16 @@
  * 0 = normal app. */
 #define FACE_LOADTEST 0
 
+/* Dev mode: enabled at deploy time with `idf.py -DDESKLOCK_DEVMODE=ON build flash`
+ * (see main/CMakeLists.txt), OFF by default in production. Currently gates only the
+ * screenshot watcher — a task that watches the USB-serial-JTAG RX for a trigger
+ * byte and dumps the current screen (see the device-screenshot skill /
+ * firmware/tools/device_shot.py). Off by default so production spends no internal
+ * RAM on it and nothing extra runs on the render path. */
+#ifndef DESKLOCK_DEVMODE
+#define DESKLOCK_DEVMODE 0
+#endif
+
 static const char *TAG = "desklock";
 
 /* All utterance framing (blocking WS sends, chime, face) runs on app_task, never
@@ -112,7 +122,12 @@ static void app_task(void *arg)
 
 void app_on_playback_done(void)
 {
-    face_set(FACE_IDLE);
+    /* Only fall back to idle if we're still the speaker. After the butler filler
+     * line, the gateway re-asserts "thinking" (FACE_EFFORT) while Tatlock works —
+     * don't clobber that spinner when the filler audio finishes. */
+    if (face_get() == FACE_SPEAKING) {
+        face_set(FACE_IDLE);
+    }
 }
 
 #if FACE_LOADTEST
@@ -232,6 +247,29 @@ void sdio_tx_selftest_kick(void)
 }
 #endif
 
+#if DESKLOCK_DEVMODE
+#include "hal/usb_serial_jtag_ll.h"
+/* Watch the USB-serial-JTAG RX FIFO (LL reads, no driver install -> no conflict
+ * with the secondary console's TX) for a trigger byte and dump one frame. */
+static void screenshot_task(void *arg)
+{
+    (void)arg;
+    vTaskDelay(pdMS_TO_TICKS(6000));   /* let boot + first render settle */
+    ESP_LOGI(TAG, "screenshot: ready — send 's' on USB serial to capture a frame");
+    uint8_t rx[16];
+    for (;;) {
+        if (usb_serial_jtag_ll_rxfifo_data_available()) {
+            int n = usb_serial_jtag_ll_read_rxfifo(rx, sizeof(rx));
+            for (int i = 0; i < n; i++) {
+                if (rx[i] == 's') { face_screenshot_dump(false); break; }  /* current screen */
+                if (rx[i] == 'o') { face_screenshot_dump(true);  break; }  /* force overlay */
+            }
+        }
+        vTaskDelay(pdMS_TO_TICKS(150));
+    }
+}
+#endif
+
 void app_main(void)
 {
     ESP_LOGI(TAG, "DeskLock starting");
@@ -287,4 +325,8 @@ void app_main(void)
 #if FACE_LOADTEST
     xTaskCreate(loadtest_task, "loadtest", 4096, NULL, 4, NULL);
 #endif
+
+#if DESKLOCK_DEVMODE
+    xTaskCreate(screenshot_task, "shot", 5120, NULL, 4, NULL);
+#endif
 }

firmware/main/face.c

@@ -7,8 +7,12 @@
 #include <string.h>
 #include <time.h>
 
+#include "freertos/FreeRTOS.h"
+#include "freertos/task.h"
 #include "esp_random.h"
 #include "esp_heap_caps.h"
+#include "esp_rom_crc.h"
+#include "hal/usb_serial_jtag_ll.h"
 #include "bsp/esp-bsp.h"
 #include "lvgl.h"
 
@@ -16,6 +20,17 @@
 
 LV_FONT_DECLARE(font_face_140);
 LV_FONT_DECLARE(font_rain_22);
+LV_FONT_DECLARE(font_phosphor_64);
+
+/* Phosphor icon glyphs (Private Use Area, UTF-8) baked into font_phosphor_64. */
+#define ICON_MIC       "\xEE\x8C\xA6"   /* ph-microphone         U+E326 */
+#define ICON_MIC_OFF   "\xEE\x8C\xA8"   /* ph-microphone-slash   U+E328 */
+#define ICON_VOL_UP    "\xEE\x91\x8A"   /* ph-speaker-high       U+E44A */
+#define ICON_VOL_DOWN  "\xEE\x91\x8C"   /* ph-speaker-low        U+E44C */
+#define ICON_VOL_MUTE  "\xEE\x91\x9A"   /* ph-speaker-slash      U+E45A */
+#define ICON_EQ        "\xEE\xAE\xBC"   /* ph-equalizer          U+EBBC */
+#define ICON_SLIDERS   "\xEE\x90\xB2"   /* ph-sliders            U+E432 */
+#define ICON_SLIDERS_H "\xEE\x90\xB4"   /* ph-sliders-horizontal U+E434 */
 LV_FONT_DECLARE(font_rage_64);
 
 #define SCREEN 800
@@ -111,6 +126,14 @@ static struct {
     lv_obj_t *status;
     lv_obj_t *elapsed;
     lv_obj_t *arc;
+    lv_obj_t *controls;    /* tap-to-reveal control overlay (mic + volume) */
+    lv_obj_t *vol_bar;
+    lv_obj_t *vol_lbl;     /* level number 0..11, or "MUTE" */
+    lv_timer_t *ctl_timer; /* one-shot auto-hide for the controls */
+    lv_obj_t *vol_hud;     /* lightweight volume feedback for voice commands */
+    lv_obj_t *hud_bar;
+    lv_obj_t *hud_lbl;
+    lv_timer_t *hud_timer; /* one-shot auto-hide for the HUD */
     stream_t streams[RAIN_MAX];
     int arc_rot;
     uint32_t state_since;
@@ -407,10 +430,200 @@ static void clock_cb(lv_timer_t *timer)
     }
 }
 
+/* --- tap-to-reveal controls: mic + volume down/up + level bar --- */
+
+static void controls_update_vol(void)
+{
+    int lvl = audio_level_get();
+    lv_bar_set_value(F.vol_bar, lvl, LV_ANIM_OFF);
+    if (audio_is_muted()) {
+        lv_label_set_text(F.vol_lbl, "MUTE");
+    } else {
+        char buf[8];
+        snprintf(buf, sizeof(buf), "%d", lvl);
+        lv_label_set_text(F.vol_lbl, buf);
+    }
+}
+
+static void controls_hide(void)
+{
+    lv_obj_add_flag(F.controls, LV_OBJ_FLAG_HIDDEN);
+    lv_timer_pause(F.ctl_timer);
+}
+
+static void controls_show(void)
+{
+    controls_update_vol();
+    lv_obj_clear_flag(F.controls, LV_OBJ_FLAG_HIDDEN);
+    face_activity();                 /* wake the screen from any dim state */
+    lv_timer_reset(F.ctl_timer);
+    lv_timer_resume(F.ctl_timer);
+}
+
+/* Audible feedback at the new volume: any non-silent change plays the gong. A new
+ * change cuts the in-flight gong off and restarts it (audio.c) rather than queueing. */
+static void volume_feedback(void)
+{
+    if (!audio_is_muted() && audio_level_get() > 0) {
+        audio_play_gong();
+    }
+}
+
+/* "This one goes to eleven": a big 11 drifts up and fades when max is hit. */
+static void eleven_ty_cb(void *o, int32_t v)  { lv_obj_set_style_translate_y((lv_obj_t *)o, v, 0); }
+static void eleven_opa_cb(void *o, int32_t v) { lv_obj_set_style_text_opa((lv_obj_t *)o, (lv_opa_t)v, 0); }
+static void eleven_del_cb(lv_anim_t *a)       { lv_obj_del((lv_obj_t *)a->var); }
+
+static void eleven_pop(lv_obj_t *anchor)
+{
+    lv_obj_t *l = lv_label_create(lv_layer_top());   /* floats above everything */
+    lv_label_set_text(l, "11");
+    lv_obj_set_style_text_font(l, &lv_font_montserrat_48, 0);
+    lv_obj_set_style_text_color(l, lv_color_hex(0xC3FFD7), 0);
+    lv_obj_update_layout(l);
+    lv_area_t a;                                     /* start just above the bar */
+    lv_obj_get_coords(anchor, &a);
+    lv_obj_set_pos(l, (a.x1 + a.x2) / 2 - lv_obj_get_width(l) / 2,
+                   a.y1 - lv_obj_get_height(l) - 4);
+
+    lv_anim_t up;                                    /* drift up (via translate, keeps align) */
+    lv_anim_init(&up);
+    lv_anim_set_var(&up, l);
+    lv_anim_set_exec_cb(&up, eleven_ty_cb);
+    lv_anim_set_values(&up, 0, -200);
+    lv_anim_set_duration(&up, 1100);
+    lv_anim_set_path_cb(&up, lv_anim_path_ease_out);
+    lv_anim_set_deleted_cb(&up, eleven_del_cb);      /* deletes the label (cancels the fade too) */
+    lv_anim_start(&up);
+
+    lv_anim_t fade;
+    lv_anim_init(&fade);
+    lv_anim_set_var(&fade, l);
+    lv_anim_set_exec_cb(&fade, eleven_opa_cb);
+    lv_anim_set_values(&fade, LV_OPA_COVER, LV_OPA_TRANSP);
+    lv_anim_set_duration(&fade, 1100);
+    lv_anim_start(&fade);
+}
+
+/* --- voice-command volume HUD (shown when a gateway command changes volume) --- */
+static void hud_update(void)
+{
+    int lvl = audio_level_get();
+    if (audio_is_muted()) {
+        lv_bar_set_value(F.hud_bar, 0, LV_ANIM_OFF);
+        lv_label_set_text(F.hud_lbl, "MUTE");
+    } else {
+        lv_bar_set_value(F.hud_bar, lvl, LV_ANIM_OFF);
+        char b[8];
+        snprintf(b, sizeof(b), "%d", lvl);
+        lv_label_set_text(F.hud_lbl, b);
+    }
+}
+
+static void hud_autohide_cb(lv_timer_t *t)
+{
+    (void)t;
+    lv_obj_add_flag(F.vol_hud, LV_OBJ_FLAG_HIDDEN);
+    lv_timer_pause(F.hud_timer);
+}
+
+static void hud_show(void)
+{
+    hud_update();
+    lv_obj_clear_flag(F.vol_hud, LV_OBJ_FLAG_HIDDEN);
+    face_activity();
+    lv_timer_reset(F.hud_timer);
+    lv_timer_resume(F.hud_timer);
+}
+
+/* Apply a gateway volume command (runs on the websocket task). */
+void face_volume_command(const char *action, int level)
+{
+    if (strcmp(action, "volume_up") == 0) {
+        audio_level_set(audio_level_get() + 1);
+    } else if (strcmp(action, "volume_down") == 0) {
+        audio_level_set(audio_level_get() - 1);
+    } else if (strcmp(action, "volume_set") == 0) {
+        audio_level_set(level);
+    } else if (strcmp(action, "mute") == 0) {
+        audio_mute(true);
+    } else if (strcmp(action, "unmute") == 0) {
+        audio_mute(false);
+    } else {
+        return;
+    }
+
+    bsp_display_lock(UINT32_MAX);
+    hud_show();
+    if (!audio_is_muted() && audio_level_get() == AUDIO_VOL_MAX) {
+        eleven_pop(F.hud_bar);
+    }
+    bsp_display_unlock();
+
+    volume_feedback();
+}
+
+static void ctl_autohide_cb(lv_timer_t *t) { (void)t; controls_hide(); }
+
+static void mic_cb(lv_event_t *e)
+{
+    (void)e;
+    controls_hide();
+    app_on_touch();                  /* the existing tap-to-talk action */
+}
+
+static void volup_cb(lv_event_t *e)
+{
+    (void)e;
+    audio_level_set(audio_level_get() + 1);
+    controls_update_vol();
+    if (audio_level_get() == AUDIO_VOL_MAX) {
+        eleven_pop(F.vol_bar);
+    }
+    volume_feedback();
+    lv_timer_reset(F.ctl_timer);
+}
+
+static void voldown_cb(lv_event_t *e)
+{
+    (void)e;
+    audio_level_set(audio_level_get() - 1);
+    controls_update_vol();
+    volume_feedback();
+    lv_timer_reset(F.ctl_timer);
+}
+
+/* tap anywhere (the underlying catcher, reachable only while controls are hidden)
+ * reveals the controls; a tap on the dim scrim behind them dismisses. */
 static void touch_cb(lv_event_t *e)
 {
     (void)e;
-    app_on_touch();
+    controls_show();
+}
+
+static void scrim_cb(lv_event_t *e)
+{
+    (void)e;
+    controls_hide();
+}
+
+static lv_obj_t *icon_btn(lv_obj_t *parent, const char *icon, int diam, lv_event_cb_t cb)
+{
+    lv_obj_t *b = lv_button_create(parent);
+    lv_obj_set_size(b, diam, diam);
+    lv_obj_set_style_radius(b, LV_RADIUS_CIRCLE, 0);
+    lv_obj_set_style_bg_color(b, lv_color_hex(0x0A140E), 0);
+    lv_obj_set_style_bg_opa(b, LV_OPA_COVER, 0);
+    lv_obj_set_style_border_color(b, lv_color_hex(0x00A050), 0);
+    lv_obj_set_style_border_width(b, 2, 0);
+    lv_obj_set_style_shadow_width(b, 0, 0);
+    lv_obj_t *lbl = lv_label_create(b);
+    lv_obj_set_style_text_font(lbl, &font_phosphor_64, 0);
+    lv_obj_set_style_text_color(lbl, lv_color_hex(0xC3FFD7), 0);
+    lv_label_set_text(lbl, icon);
+    lv_obj_center(lbl);
+    lv_obj_add_event_cb(b, cb, LV_EVENT_CLICKED, NULL);
+    return b;
 }
 
 void face_init(void)
@@ -507,6 +720,77 @@ void face_init(void)
     lv_obj_add_flag(touch, LV_OBJ_FLAG_CLICKABLE);
     lv_obj_add_event_cb(touch, touch_cb, LV_EVENT_CLICKED, NULL);
 
+    /* --- tap-to-reveal control overlay: dim scrim + mic + volume down/up + bar --- */
+    F.controls = lv_obj_create(scr);                      /* above the touch catcher */
+    lv_obj_remove_style_all(F.controls);
+    lv_obj_set_size(F.controls, SCREEN, SCREEN);
+    lv_obj_set_style_bg_color(F.controls, lv_color_black(), 0);
+    lv_obj_set_style_bg_opa(F.controls, 200, 0);          /* dim the face behind */
+    lv_obj_add_flag(F.controls, LV_OBJ_FLAG_CLICKABLE);
+    lv_obj_clear_flag(F.controls, LV_OBJ_FLAG_SCROLLABLE);
+    lv_obj_add_event_cb(F.controls, scrim_cb, LV_EVENT_CLICKED, NULL);
+    lv_obj_add_flag(F.controls, LV_OBJ_FLAG_HIDDEN);      /* hidden until tapped */
+
+    lv_obj_t *mic = icon_btn(F.controls, ICON_MIC, 150, mic_cb);
+    lv_obj_align(mic, LV_ALIGN_CENTER, 0, -30);
+    lv_obj_t *vdn = icon_btn(F.controls, ICON_VOL_DOWN, 110, voldown_cb);
+    lv_obj_align(vdn, LV_ALIGN_CENTER, -230, -30);
+    lv_obj_t *vup = icon_btn(F.controls, ICON_VOL_UP, 110, volup_cb);
+    lv_obj_align(vup, LV_ALIGN_CENTER, 230, -30);
+
+    F.vol_bar = lv_bar_create(F.controls);
+    lv_obj_set_size(F.vol_bar, 360, 16);
+    lv_obj_align(F.vol_bar, LV_ALIGN_CENTER, 0, 140);
+    lv_bar_set_range(F.vol_bar, 0, AUDIO_VOL_MAX);
+    lv_obj_set_style_bg_color(F.vol_bar, lv_color_hex(0x0A140E), LV_PART_MAIN);
+    lv_obj_set_style_bg_opa(F.vol_bar, LV_OPA_COVER, LV_PART_MAIN);
+    lv_obj_set_style_bg_color(F.vol_bar, lv_color_hex(0x00C860), LV_PART_INDICATOR);
+    lv_obj_set_style_radius(F.vol_bar, 8, LV_PART_MAIN);
+    lv_obj_set_style_radius(F.vol_bar, 8, LV_PART_INDICATOR);
+
+    F.vol_lbl = lv_label_create(F.controls);
+    lv_obj_set_style_text_font(F.vol_lbl, &lv_font_montserrat_28, 0);
+    lv_obj_set_style_text_color(F.vol_lbl, lv_color_hex(0xC3FFD7), 0);
+    lv_label_set_text(F.vol_lbl, "");
+    lv_obj_align(F.vol_lbl, LV_ALIGN_CENTER, 0, 180);
+
+    F.ctl_timer = lv_timer_create(ctl_autohide_cb, 5000, NULL);
+    lv_timer_pause(F.ctl_timer);
+
+    /* voice-command volume HUD: just a slim volume bar (+ level number), centered
+     * and auto-hiding — separate from the tap overlay so a spoken "volume up"
+     * doesn't pop the mic button. The "11" animation flows off this bar. */
+    F.vol_hud = lv_obj_create(scr);
+    lv_obj_remove_style_all(F.vol_hud);
+    lv_obj_set_size(F.vol_hud, 560, 64);
+    lv_obj_center(F.vol_hud);
+    lv_obj_set_style_bg_color(F.vol_hud, lv_color_black(), 0);
+    lv_obj_set_style_bg_opa(F.vol_hud, 180, 0);
+    lv_obj_set_style_radius(F.vol_hud, 16, 0);
+    lv_obj_set_style_pad_all(F.vol_hud, 10, 0);
+    lv_obj_clear_flag(F.vol_hud, LV_OBJ_FLAG_SCROLLABLE);
+    lv_obj_add_flag(F.vol_hud, LV_OBJ_FLAG_HIDDEN);
+
+    /* bar centered (symmetric on screen); level number hangs off to the right */
+    F.hud_bar = lv_bar_create(F.vol_hud);
+    lv_obj_set_size(F.hud_bar, 360, 18);
+    lv_obj_align(F.hud_bar, LV_ALIGN_CENTER, 0, 0);
+    lv_bar_set_range(F.hud_bar, 0, AUDIO_VOL_MAX);
+    lv_obj_set_style_bg_color(F.hud_bar, lv_color_hex(0x0A140E), LV_PART_MAIN);
+    lv_obj_set_style_bg_opa(F.hud_bar, LV_OPA_COVER, LV_PART_MAIN);
+    lv_obj_set_style_bg_color(F.hud_bar, lv_color_hex(0x00C860), LV_PART_INDICATOR);
+    lv_obj_set_style_radius(F.hud_bar, 9, LV_PART_MAIN);
+    lv_obj_set_style_radius(F.hud_bar, 9, LV_PART_INDICATOR);
+
+    F.hud_lbl = lv_label_create(F.vol_hud);
+    lv_obj_set_style_text_font(F.hud_lbl, &lv_font_montserrat_28, 0);
+    lv_obj_set_style_text_color(F.hud_lbl, lv_color_hex(0xC3FFD7), 0);
+    lv_label_set_text(F.hud_lbl, "");
+    lv_obj_align(F.hud_lbl, LV_ALIGN_RIGHT_MID, -6, 0);
+
+    F.hud_timer = lv_timer_create(hud_autohide_cb, 2000, NULL);
+    lv_timer_pause(F.hud_timer);
+
     /* Boot straight into the calm idle face (dry — rain gate holds it off until
      * the gateway connects). No "CONNECTING" banner churn on every recovery. */
     F.state = FACE_IDLE;
@@ -571,3 +855,69 @@ void face_activity(void)
         bsp_display_unlock();
     }
 }
+
+/* --- dev screenshot over USB (see the device-screenshot skill) --------------- */
+
+/* Blocking write straight to the USB-serial-JTAG TX FIFO. Bypasses stdout/printf
+ * — the primary console is UART at 115200 baud (~11 KB/s), far too slow for a
+ * frame; the USB FIFO runs at USB speed. Waits for FIFO space so nothing drops. */
+static void shot_usb_write(const uint8_t *data, size_t len)
+{
+    size_t sent = 0;
+    while (sent < len) {
+        if (!usb_serial_jtag_ll_txfifo_writable()) { vTaskDelay(1); continue; }
+        sent += usb_serial_jtag_ll_write_txfifo(data + sent, len - sent);
+        usb_serial_jtag_ll_txfifo_flush();
+    }
+}
+
+/* Render the live screen to an RGB565 buffer, 2x-downscale it, and stream it as
+ * RAW BINARY straight over the USB-serial-JTAG, framed by a ###SHOT_BEGIN ...###
+ * text header (carrying byte count + CRC) and ###SHOT_END###, so the host can
+ * rebuild a PNG. `overlay` forces the tap controls in-frame. Called only from the
+ * DESKLOCK_DEVMODE screenshot watcher; not part of normal operation. */
+#define SHOT_DS 2                                 /* downscale factor (2 -> 400x400) */
+void face_screenshot_dump(bool overlay)
+{
+    /* Snapshot into our OWN PSRAM buffer — the default lv_snapshot_take() allocs
+     * the 1.28 MB frame from the small LVGL heap and returns NULL. */
+    const int W = SCREEN, H = SCREEN;
+    const uint32_t stride = lv_draw_buf_width_to_stride(W, LV_COLOR_FORMAT_RGB565);
+    const size_t buf_size = (size_t)stride * H;
+    uint8_t *mem = heap_caps_malloc(buf_size, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
+    if (!mem) { printf("###SHOT_FAIL mem###\n"); return; }
+    lv_draw_buf_t dbuf;
+    lv_draw_buf_init(&dbuf, W, H, LV_COLOR_FORMAT_RGB565, stride, mem, buf_size);
+
+    bsp_display_lock(UINT32_MAX);
+    if (overlay) {                                /* force the tap overlay in-frame */
+        controls_show();
+        lv_timer_pause(F.ctl_timer);              /* don't let it auto-hide mid-capture */
+    }
+    lv_result_t rc = lv_snapshot_take_to_draw_buf(lv_screen_active(), LV_COLOR_FORMAT_RGB565, &dbuf);
+    bsp_display_unlock();
+    if (rc != LV_RESULT_OK) { printf("###SHOT_FAIL take=%d###\n", (int)rc); heap_caps_free(mem); return; }
+
+    const int ow = W / SHOT_DS, oh = H / SHOT_DS;
+    const size_t olen = (size_t)ow * oh * 2;
+    uint8_t *out = heap_caps_malloc(olen, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
+    if (!out) { printf("###SHOT_FAIL malloc###\n"); heap_caps_free(mem); return; }
+
+    for (int y = 0; y < oh; y++) {
+        const uint16_t *src = (const uint16_t *)(dbuf.data + (size_t)(y * SHOT_DS) * dbuf.header.stride);
+        uint16_t *dst = (uint16_t *)(out + (size_t)y * ow * 2);
+        for (int x = 0; x < ow; x++) dst[x] = src[x * SHOT_DS];
+    }
+    heap_caps_free(mem);
+
+    const uint32_t crc = esp_rom_crc32_le(0, out, olen);
+    char hdr[128];
+    int hlen = snprintf(hdr, sizeof(hdr),
+                        "\n###SHOT_BEGIN w=%d h=%d fmt=rgb565le bytes=%u crc=0x%08x bin=1###\n",
+                        ow, oh, (unsigned)olen, (unsigned)crc);
+    shot_usb_write((const uint8_t *)hdr, hlen);
+    shot_usb_write(out, olen);                    /* raw RGB565-LE, exactly `bytes` long */
+    static const char end[] = "\n###SHOT_END###\n";
+    shot_usb_write((const uint8_t *)end, sizeof(end) - 1);
+    heap_caps_free(out);
+}

firmware/main/gw_client.c

@@ -52,6 +52,12 @@ static void handle_text(const char *data, int len)
         audio_playback_end();
     } else if (strcmp(type->valuestring, "error") == 0) {
         face_set(FACE_RAGE);
+    } else if (strcmp(type->valuestring, "command") == 0) {
+        const cJSON *action = cJSON_GetObjectItem(root, "action");
+        const cJSON *level = cJSON_GetObjectItem(root, "level");
+        if (cJSON_IsString(action)) {
+            face_volume_command(action->valuestring, cJSON_IsNumber(level) ? level->valueint : 0);
+        }
     }
     cJSON_Delete(root);
 }

firmware/sdkconfig.defaults

@@ -32,6 +32,10 @@ CONFIG_ESP_WIFI_SOFTAP_SUPPORT=y
 # custom fonts are uncompressed, but enable the decoder as belt-and-braces
 CONFIG_LV_USE_FONT_COMPRESSED=y
 
+# lv_snapshot_take() — render the live screen to a buffer for the dev
+# screenshot-over-USB dump (face_screenshot_dump)
+CONFIG_LV_USE_SNAPSHOT=y
+
 # ESP-Hosted board variant + wifi-remote data-path tuning (from factory brookesia config;
 # without these the RPC control path works but data frames never flow)
 CONFIG_SLAVE_IDF_TARGET_ESP32C6=y

firmware/tools/device_shot.py

@@ -0,0 +1,100 @@
+#!/usr/bin/env python3
+"""Capture a screenshot from the DeskLock device over USB.
+
+The firmware (SCREENSHOT_ENABLE) watches the USB-serial-JTAG RX for a trigger
+byte and streams the current screen as a text header
+  ###SHOT_BEGIN w=.. h=.. fmt=rgb565le bytes=N crc=0xXXXX bin=1###\n
+followed by exactly N raw RGB565-LE bytes, then \n###SHOT_END###. We send the
+trigger, read the payload, verify the CRC, and write a PNG.
+
+Usage (run in the dialout group, e.g. under `sg dialout -c '...'`):
+    python3 device_shot.py [out.png] [--overlay] [--port /dev/ttyACM0]
+  --overlay  send 'o' (force the tap controls overlay) instead of 's' (current screen)
+"""
+import os, sys, termios, select, time, zlib, re
+
+port = "/dev/ttyACM0"
+out = "shot.png"
+trig = b"s"
+args = sys.argv[1:]
+while args:
+    a = args.pop(0)
+    if a == "--overlay": trig = b"o"
+    elif a == "--port": port = args.pop(0)
+    elif not a.startswith("-"): out = a
+    else: sys.stderr.write(f"[warn] ignoring {a}\n")
+
+ATTEMPTS = 4
+PER_TRY = 8.0
+begin_re = re.compile(
+    rb"###SHOT_BEGIN w=(\d+) h=(\d+) fmt=(\S+) bytes=(\d+) crc=0x([0-9a-fA-F]+) bin=1###\n")
+
+fd = os.open(port, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
+a = termios.tcgetattr(fd)
+a[0] = 0; a[1] = 0; a[3] = 0            # raw: no CR/NL translation, no canon/echo
+a[4] = termios.B115200; a[5] = termios.B115200
+termios.tcsetattr(fd, termios.TCSANOW, a)
+
+
+def drain(sec=0.3):
+    t = time.time()
+    while time.time() - t < sec:
+        r, _, _ = select.select([fd], [], [], 0.05)
+        if r:
+            try: os.read(fd, 65536)
+            except BlockingIOError: pass
+
+
+def capture_once():
+    drain(0.3)                          # clear any in-flight tail from a prior dump
+    buf = bytearray(); hdr = None; t0 = time.time(); last_trig = 0.0
+    while time.time() - t0 < PER_TRY:
+        now = time.time()
+        if hdr is None and now - last_trig >= 2.0:
+            try: os.write(fd, trig)
+            except OSError: pass
+            last_trig = now
+        r, _, _ = select.select([fd], [], [], 0.2)
+        if r:
+            try: chunk = os.read(fd, 65536)
+            except BlockingIOError: chunk = b""
+            if chunk: buf += chunk
+        if hdr is None:
+            m = begin_re.search(buf)
+            if m:
+                hdr = (int(m.group(1)), int(m.group(2)), int(m.group(4)), int(m.group(5), 16))
+                buf = bytearray(buf[m.end():])      # everything after header = raw payload
+        if hdr is not None and len(buf) >= hdr[2]:
+            return (hdr[0], hdr[1], hdr[2], hdr[3], bytes(buf[:hdr[2]]))
+    return None
+
+
+result = None
+for attempt in range(1, ATTEMPTS + 1):
+    res = capture_once()
+    if res is None:
+        sys.stderr.write(f"[try {attempt}] no frame\n"); continue
+    w, h, nbytes, crc_dev, raw = res
+    crc_host = zlib.crc32(raw) & 0xffffffff
+    ok = crc_host == crc_dev
+    sys.stderr.write(f"[try {attempt}] {w}x{h} {len(raw)}B "
+                     f"crc dev=0x{crc_dev:08x} host=0x{crc_host:08x} {'OK' if ok else 'RETRY'}\n")
+    if ok:
+        result = (w, h, raw); break
+os.close(fd)
+
+if result is None:
+    sys.stderr.write("[fail] no clean frame — SCREENSHOT_ENABLE flashed? device up?\n")
+    sys.exit(2)
+
+w, h, raw = result
+from PIL import Image
+img = Image.new("RGB", (w, h)); px = img.load(); i = 0
+for y in range(h):
+    for x in range(w):
+        v = raw[i] | (raw[i+1] << 8); i += 2
+        r5 = (v >> 11) & 0x1f; g6 = (v >> 5) & 0x3f; b5 = v & 0x1f
+        px[x, y] = ((r5 << 3) | (r5 >> 2), (g6 << 2) | (g6 >> 4), (b5 << 3) | (b5 >> 2))
+img.save(out)
+sys.stderr.write(f"[ok] wrote {out} ({w}x{h})\n")
+print(out)

gateway/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "desklock-gateway"
-version = "0.1.0"
+version = "0.2.2"
 description = "Voice gateway bridging the DeskLock device to the Tatlock butler (STT/chat/TTS)"
 requires-python = ">=3.11"
 dependencies = [

gateway/src/desklock_gateway/commands.py

@@ -0,0 +1,95 @@
+"""Voice command service.
+
+Intercepts simple device commands from the STT transcript so they bypass the LLM
+(no 10-25s Tatlock round-trip). `match()` returns a gateway->device ``command``
+message dict, or ``None`` if the transcript is not a recognized command and should
+go to Tatlock.
+
+Matching is deliberately precise, not clever: a false positive hijacks a real
+request, so we only fire on explicit volume/mute wording. Extend ``match()`` with
+new rules as the command vocabulary grows.
+"""
+
+import re
+
+VOL_MAX = 11
+
+_WORD_NUMBERS = {
+    "zero": 0,
+    "one": 1,
+    "two": 2,
+    "three": 3,
+    "four": 4,
+    "five": 5,
+    "six": 6,
+    "seven": 7,
+    "eight": 8,
+    "nine": 9,
+    "ten": 10,
+    "eleven": 11,
+}
+
+Command = dict[str, str | int]
+
+
+def _norm(text: str) -> str:
+    """Lower-case, drop punctuation, collapse whitespace."""
+    return re.sub(r"\s+", " ", re.sub(r"[^a-z0-9 ]", " ", text.lower())).strip()
+
+
+def _has(t: str, *words: str) -> bool:
+    return any(re.search(rf"\b{w}\b", t) for w in words)
+
+
+def _to_int(tok: str) -> int | None:
+    if tok.isdigit():
+        return int(tok)
+    return _WORD_NUMBERS.get(tok)
+
+
+def _cmd(action: str, level: int | None = None) -> Command:
+    msg: Command = {"type": "command", "action": action}
+    if level is not None:
+        msg["level"] = max(0, min(VOL_MAX, level))
+    return msg
+
+
+def match(transcript: str) -> Command | None:
+    """Return a ``command`` message if the transcript is a device command, else None."""
+    t = _norm(transcript)
+    if not t:
+        return None
+
+    # mute / unmute — check unmute first ("unmute" contains "mute")
+    if _has(t, "unmute"):
+        return _cmd("unmute")
+    if _has(t, "mute"):
+        return _cmd("mute")
+
+    # explicit "set volume to N" (digit or number word), requires the word "volume"
+    m = re.search(r"\bvolume\s+(?:to\s+|at\s+|is\s+)?(\w+)\b", t)
+    if m:
+        val = _to_int(m.group(1))
+        if val is not None:
+            return _cmd("volume_set", level=val)
+
+    # max / the meme — "this one goes to eleven", "crank it", "max volume"
+    if (
+        re.search(r"\bgoes to eleven\b", t)
+        or _has(t, "crank")
+        or (_has(t, "max", "maximum", "full") and _has(t, "volume", "sound"))
+    ):
+        return _cmd("volume_set", level=VOL_MAX)
+
+    # relative up / down — need explicit volume/sound context or an unambiguous verb
+    vol_ctx = _has(t, "volume", "sound")
+    if _has(t, "louder") or (vol_ctx and _has(t, "up")) or re.search(r"\bturn it up\b", t):
+        return _cmd("volume_up")
+    if (
+        _has(t, "quieter", "softer")
+        or (vol_ctx and _has(t, "down"))
+        or re.search(r"\bturn it down\b", t)
+    ):
+        return _cmd("volume_down")
+
+    return None

gateway/src/desklock_gateway/config.py

@@ -4,7 +4,7 @@ from pydantic_settings import BaseSettings
 class Settings(BaseSettings):
     """Gateway configuration, overridable via DESKLOCK_* environment variables."""
 
-    tatlock_base_url: str = "http://tatlock.schweitz.internal:8000"
+    tatlock_base_url: str = "http://tatlock:8000"
     tatlock_model: str = "Tatlock"
 
     # PCM rate of the device WebSocket contract (docs/architecture.md)
@@ -19,6 +19,10 @@ class Settings(BaseSettings):
     tts_model: str = "speaches-ai/Kokoro-82M-v1.0-ONNX"
     tts_voice: str = "bm_george"
 
+    # Spoken immediately after a query is accepted, to fill the long Tatlock wait.
+    # Avoid "for you" — Kokoro inserts an unnatural mid-phrase pause before it.
+    filler_text: str = "Let me check on that, sir."
+
     # embedded fallback only (requires the [speech] extra)
     embedded_stt_model: str = "small"
     embedded_stt_device: str = "cuda"

gateway/src/desklock_gateway/main.py

@@ -11,7 +11,7 @@ from datetime import datetime, timezone
 
 from fastapi import FastAPI, WebSocket, WebSocketDisconnect
 
-from . import stt, tts
+from . import commands, stt, tts
 from .config import settings
 from .tatlock import TatlockClient
 
@@ -24,6 +24,21 @@ PCM_CHUNK_BYTES = 4096
 # device registry: every endpoint that ever connected, keyed by client IP
 DEVICES: dict[str, dict] = {}
 
+# Butler filler audio (the "let me check…" line) is synthesized once, then replayed.
+_filler_pcm: bytes | None = None
+_filler_tried = False
+
+
+async def _ensure_filler() -> bytes | None:
+    global _filler_pcm, _filler_tried
+    if not _filler_tried:
+        _filler_tried = True
+        try:
+            _filler_pcm = await asyncio.to_thread(tts.synthesize, settings.filler_text)
+        except Exception:
+            logger.exception("filler synth failed; continuing without it")
+    return _filler_pcm
+
 
 def _now() -> str:
     return datetime.now(timezone.utc).astimezone().isoformat(timespec="seconds")
@@ -104,6 +119,24 @@ async def _handle_utterance(ws: WebSocket, tatlock: TatlockClient, pcm: bytes) -
         await ws.send_json({"type": "state", "value": "idle"})
         return
 
+    # Simple device commands (volume/mute) bypass the LLM entirely.
+    command = commands.match(transcript)
+    if command is not None:
+        await ws.send_json(command)
+        await ws.send_json({"type": "state", "value": "idle"})
+        return
+
+    # Acknowledge immediately with a canned line so the long Tatlock wait isn't dead
+    # air, then re-assert "thinking" to keep the spinner up while it works.
+    filler = await _ensure_filler()
+    if filler:
+        await ws.send_json({"type": "reply_text", "text": settings.filler_text})
+        await ws.send_json({"type": "audio_start", "sample_rate": settings.sample_rate})
+        for offset in range(0, len(filler), PCM_CHUNK_BYTES):
+            await ws.send_bytes(filler[offset : offset + PCM_CHUNK_BYTES])
+        await ws.send_json({"type": "audio_end"})
+        await ws.send_json({"type": "state", "value": "thinking"})
+
     reply = await tatlock.ask(transcript)
     await ws.send_json({"type": "reply_text", "text": reply})
 

gateway/tests/test_commands.py

@@ -0,0 +1,61 @@
+from desklock_gateway import commands
+
+
+def test_mute_and_unmute() -> None:
+    assert commands.match("mute") == {"type": "command", "action": "mute"}
+    assert commands.match("mute the volume") == {"type": "command", "action": "mute"}
+    # "unmute" contains "mute" — must resolve to unmute
+    assert commands.match("unmute") == {"type": "command", "action": "unmute"}
+
+
+def test_relative_up_down() -> None:
+    assert commands.match("volume up") == {"type": "command", "action": "volume_up"}
+    assert commands.match("louder please") == {"type": "command", "action": "volume_up"}
+    assert commands.match("turn it up") == {"type": "command", "action": "volume_up"}
+    assert commands.match("volume down") == {"type": "command", "action": "volume_down"}
+    assert commands.match("a bit quieter") == {"type": "command", "action": "volume_down"}
+    assert commands.match("turn it down") == {"type": "command", "action": "volume_down"}
+
+
+def test_set_to_number_digit_and_word() -> None:
+    assert commands.match("set volume to 7") == {
+        "type": "command",
+        "action": "volume_set",
+        "level": 7,
+    }
+    assert commands.match("volume to seven") == {
+        "type": "command",
+        "action": "volume_set",
+        "level": 7,
+    }
+    assert commands.match("volume 3") == {"type": "command", "action": "volume_set", "level": 3}
+    # clamps into 0..11
+    assert commands.match("set volume to 50") == {
+        "type": "command",
+        "action": "volume_set",
+        "level": 11,
+    }
+
+
+def test_goes_to_eleven() -> None:
+    assert commands.match("this one goes to eleven") == {
+        "type": "command",
+        "action": "volume_set",
+        "level": 11,
+    }
+    assert commands.match("set volume to eleven") == {
+        "type": "command",
+        "action": "volume_set",
+        "level": 11,
+    }
+    assert commands.match("crank it") == {"type": "command", "action": "volume_set", "level": 11}
+    assert commands.match("max volume") == {"type": "command", "action": "volume_set", "level": 11}
+
+
+def test_non_commands_fall_through_to_llm() -> None:
+    # these must NOT be hijacked from Tatlock
+    assert commands.match("what's the weather tomorrow") is None
+    assert commands.match("set an alarm for a quarter to eleven") is None
+    assert commands.match("turn up the heating in the lounge") is None
+    assert commands.match("") is None
+    assert commands.match("tell me a joke") is None