Say "Computer" -> chime + listening -> speak -> AFE VAD detects you
stopped -> auto-sends. No taps. Touch still works as a manual override.
- esp-sr 2.4.6 added; wn9_computer_tts model packed into a new "model"
flash partition (MODEL_IN_FLASH). App moved to 8M, model 4M.
- audio.c: replaced the on-demand capture_task with an AFE pipeline —
feed_task is the SOLE mic reader (-> afe->feed); detect_task fetches,
watches wakeup_state for the wake word and vad_state for end of
speech, and forwards AFE-cleaned audio upstream during an utterance.
One mic reader ever.
- short rising chime acknowledges the wake audibly.
Fixes from adversarial review before trusting it:
1. utterance framing (blocking WS sends) moved OFF the AFE fetch thread
onto an app_task event queue (EV_TOUCH/EV_WAKE/EV_SPEECH_END) — a
1.5s send could stall fetch and drop the first ~1.5s of speech.
2. app_task is now the single serializer of start/end -> no TOCTOU
double-start (was: two utterance_start on a tap during wake).
3. VAD accounting resets on every streaming (re)start (wake OR tap),
not just wake -> a tapped utterance can no longer end instantly on
stale silence.
4. chime/reply set s_playing (+DMA tail hold) and detect_task skips the
mic while s_playing -> our own audio no longer streams into STT or
false-triggers the wake at a playback boundary (no AEC yet).
5. NULL-checked AFE create + feed buffer; tasks only start if AFE is up.
Verified on hardware: model loads, AFE inits with the Computer word,
boots and connects clean, no crash/wedge.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TOUCH CRASH (the blue screen): app_on_touch ran a blocking WebSocket
send (up to 5s) directly in the LVGL touch callback, stalling the
MIPI-DSI flush into a garbage/blue frame + task-watchdog reboot on
every tap. Now touch_cb only gives a semaphore; a dedicated app_task
does the blocking sends, audio, and face changes off the render
thread. WS send timeout cut 5s to 1.5s as belt-and-braces.
RAIN = REACHABILITY (user request): rain now falls only while the
gateway WebSocket is live (gated on gw_connected in rain_tick). It
drains gracefully on disconnect, resumes on reconnect, a genuine
glanceable reachable signal. Idle density bumped 2 to 4 so
connected-idle reads distinctly from disconnected-black.
CALM THE WEDGE CHURN (user request): transient wifi/WS drops no longer
slam to the x_x error face or a CONNECTING banner. Boot goes straight
to the calm idle face (dry until connected). Only a sustained 30s+
outage escalates to x_x (clock_cb); the ~15s wedge-recovery just shows
a brief rain pause.
Two fixes from adversarial concurrency review before flashing:
- persistent single capture task (was xTaskCreate per utterance; a
rapid re-tap or WS-stop-vs-app-start race could put two readers on
one mic/I2S handle and corrupt the codec)
- reset s_talking on disconnect (app_on_disconnect) so the first tap
after a mid-utterance drop starts fresh, not the stop branch
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The device now runs the complete face contract on glass: seven states
with glyph expressions (custom 140px face font), matrix rain as pooled
label streams (22px DejaVu+Noto katakana font), rage kaomoji frames
(64px), orbit arc + elapsed counter wait cues, blink/talk/thought
animations, idle clock (SNTP, Europe/Amsterdam), and the power ladder
(active/ambient 35%/dormant 5% with rain parked).
Voice path: ES7210 mic capture task streams 16k PCM over
esp_websocket_client to the gateway; reply PCM buffers to PSRAM and
plays via the audio task; touch-to-talk (tap to speak, tap to send).
Protocol mapping per docs: thinking->effort, audio->speaking,
error event->rage (auto-composes after 2 loops), WS loss->error face.
Bring-up fixes: 8MB factory partition (fonts overflowed 1.5M),
bsp_display_lock(0) is try-lock in this adapter (use UINT32_MAX),
gong synth yields to feed IDLE0, wifi scan diagnostics (found SSID
case mismatch), esp_websocket_client pinned ~1.3 (1.4 needs IDF>5.5).
Verified on hardware: boots, connects to Wi-Fi and holds an open
WebSocket to the gateway. Sauron (iMac TTY endpoint + ops console)
planned in architecture.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
jpmschweitzerandClaude Fable 5