Rework the speaker volume from a 0-100 percentage to an 11-step level
(0..11, mapped to the codec's percent), with mute that remembers the
prior level so unmute restores it. Any non-silent change plays the gong
as feedback; a new change cuts the in-flight gong off and restarts it
rather than queueing another. The tap overlay shows the level number and
0..11 bar, and a "11" drifts up off the bar when you hit maximum.
Also lands the device side of gateway volume commands: gw_client routes a
"command" message to face_volume_command, which applies the change and
shows a compact auto-hiding volume HUD (a centered bar) — the gateway
half that sends these lives in a following commit.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
Adds a way to capture the live LVGL screen off the device and rebuild it
as a PNG on the host, so UI changes can be verified remotely without a
camera. A watcher task polls the USB-serial-JTAG RX for a trigger byte and
streams the current screen as raw RGB565 straight to the USB FIFO (framed
by ###SHOT_BEGIN/END### with a CRC); firmware/tools/device_shot.py and the
device-screenshot skill drive it from the host.
Writing straight to the USB FIFO bypasses the primary UART console, which
at 115200 baud would take ~37s per frame. The whole capability is behind
DESKLOCK_DEVMODE (off by default, enable at deploy time with
-DDESKLOCK_DEVMODE=ON) so production spends no internal RAM on the watcher
and nothing extra runs on the render path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
Tapping the face now brings up an overlay with a microphone button and
volume down/up, plus a live level bar showing the current output volume.
Icons are drawn from a subset of the Phosphor glyph font. Tapping the dim
scrim behind the controls dismisses them; they also auto-hide after a few
seconds of inactivity.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
Each stream periodically swaps to a different one of the pooled streak
sprites on a staggered timer, so the columns keep changing rather than
sitting statically tiled — the rain reads as alive in the corner of the
eye. The pool stays at 5: each sprite is ~15 KB of internal RAM and more
starves esp-hosted's internal-RAM task stacks into a boot loop.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
The rain was 40 live text labels; moving a label re-rasterizes its
katakana glyphs every frame, reading the font from PSRAM and blending
against the framebuffer. That per-frame load starved the DSI's
framebuffer read and flashed the panel blue while a voice reply
downloaded (confirmed: with the rain removed the flash vanished).
Pre-render a small pool of streak sprites once into internal RAM and
blit them; movement is now a cheap opaque copy that never touches the
PSRAM bus. Also drain the rain during audio (listening/speaking) rather
than freezing it. This clears the last of the load flicker while keeping
the dense look.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
Two secondary mitigations for the DSI underrun (the primary fix is the
reduced DPI clock in the BSP). Raise LV_INV_BUF_SIZE 32 -> 128 so the
busy rain states stay partial-redraw instead of collapsing into a
full-screen redraw, which dumps a ~1.3 MB PSRAM write burst that
competes with the DSI's continuous framebuffer read. Advance the matrix
rain every other tick (doubled step, visually identical) to halve how
often its ~80 labels invalidate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
TOUCH CRASH (the blue screen): app_on_touch ran a blocking WebSocket
send (up to 5s) directly in the LVGL touch callback, stalling the
MIPI-DSI flush into a garbage/blue frame + task-watchdog reboot on
every tap. Now touch_cb only gives a semaphore; a dedicated app_task
does the blocking sends, audio, and face changes off the render
thread. WS send timeout cut 5s to 1.5s as belt-and-braces.
RAIN = REACHABILITY (user request): rain now falls only while the
gateway WebSocket is live (gated on gw_connected in rain_tick). It
drains gracefully on disconnect, resumes on reconnect, a genuine
glanceable reachable signal. Idle density bumped 2 to 4 so
connected-idle reads distinctly from disconnected-black.
CALM THE WEDGE CHURN (user request): transient wifi/WS drops no longer
slam to the x_x error face or a CONNECTING banner. Boot goes straight
to the calm idle face (dry until connected). Only a sustained 30s+
outage escalates to x_x (clock_cb); the ~15s wedge-recovery just shows
a brief rain pause.
Two fixes from adversarial concurrency review before flashing:
- persistent single capture task (was xTaskCreate per utterance; a
rapid re-tap or WS-stop-vs-app-start race could put two readers on
one mic/I2S handle and corrupt the codec)
- reset s_talking on disconnect (app_on_disconnect) so the first tap
after a mid-utterance drop starts fresh, not the stop branch
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of the dead data path (assoc/scan/RPC fine, zero data
frames): esp-hosted 1.4.x is formally incompatible with IDF 5.5
(esp-hosted-mcu#47) and the factory C6 slave firmware was ancient
(couldn't even answer a version RPC). Waveshare's examples pin 1.4.* —
do not follow them.
Fix:
- host: espressif/esp_hosted ^2.12 (+ esp_wifi_remote 1.6)
- slave: 2.12.11 network_adapter.bin embedded in the app (c6_ota.c
streams it to the C6 over the SDIO RPC channel at boot when the
reported version is < 2.x; ~10s, no wires, idempotent)
- RAM diet for hosted 2.x's footprint (MEMPOOL_PREFER_SPIRAM,
reduced WIFI_RMT buffers) — without it internal SRAM famine
boot-loops in xTaskCreateStaticPinnedToCore before app_main
- conservative 20MHz SDIO clock for first verified data path
Verified on hardware: DHCP lease (even that healed), 8/8 pings to
router and tower-of-joy at 1-5ms, WebSocket to the gateway connected,
device registered at /devices as desklock-p4 with fw version.
Also: wifi_diag.c L1 SoftAP diagnostic mode (DESKLOCK-DIAG) with
review fixes, boot ping ladder in net.c, static-IP fallback,
face_status() line, docs for the whole failure taxonomy.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The device now runs the complete face contract on glass: seven states
with glyph expressions (custom 140px face font), matrix rain as pooled
label streams (22px DejaVu+Noto katakana font), rage kaomoji frames
(64px), orbit arc + elapsed counter wait cues, blink/talk/thought
animations, idle clock (SNTP, Europe/Amsterdam), and the power ladder
(active/ambient 35%/dormant 5% with rain parked).
Voice path: ES7210 mic capture task streams 16k PCM over
esp_websocket_client to the gateway; reply PCM buffers to PSRAM and
plays via the audio task; touch-to-talk (tap to speak, tap to send).
Protocol mapping per docs: thinking->effort, audio->speaking,
error event->rage (auto-composes after 2 loops), WS loss->error face.
Bring-up fixes: 8MB factory partition (fonts overflowed 1.5M),
bsp_display_lock(0) is try-lock in this adapter (use UINT32_MAX),
gong synth yields to feed IDLE0, wifi scan diagnostics (found SSID
case mismatch), esp_websocket_client pinned ~1.3 (1.4 needs IDF>5.5).
Verified on hardware: boots, connects to Wi-Fi and holds an open
WebSocket to the gateway. Sauron (iMac TTY endpoint + ops console)
planned in architecture.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
jpmschweitzerandClaude Opus 4.8