The stock 1500 Mbps (the P4 maximum) makes the DSI transmit each line in
a tight burst, so the framebuffer reads are bursty and lose the PSRAM
bus to the Wi-Fi SDIO DMA under load. 720 spreads the read out and
tolerates the contention. Espressif's documented underrun mitigation;
the floor at this pixel clock is ~480 Mbps.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
Comment-only. Moving the esp-hosted transport buffers off PSRAM
(MEMPOOL_PREFER_SPIRAM=n), to keep the radio's SDIO DMA off the DSI's
bus and stop it starving the framebuffer read, was tried against the
flicker — but it exhausts internal RAM and boot-loops in esp-hosted init
before Wi-Fi comes up. Documented so it isn't retried; the setting stays
=y.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
Two secondary mitigations for the DSI underrun (the primary fix is the
reduced DPI clock in the BSP). Raise LV_INV_BUF_SIZE 32 -> 128 so the
busy rain states stay partial-redraw instead of collapsing into a
full-screen redraw, which dumps a ~1.3 MB PSRAM write burst that
competes with the DSI's continuous framebuffer read. Advance the matrix
rain every other tick (doubled step, visually identical) to halve how
often its ~80 labels invalidate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
Start the display via bsp_display_start_with_config with a 16 KB LVGL
task stack. The 8 KB default overflows once LV_INV_BUF_SIZE is raised to
128 (the partial-flush path's stack frame scales with it) — the guard
fires as a "Stack protection fault" boot loop otherwise.
Add FACE_LOADTEST (gated off): a load-emulation harness that cycles
isolated load types — a render-only face ladder, light/heavy render plus
a network stream, render plus playback, and all three stacked. Each
phase logs a marker and shows an on-screen label. It pinned the blue
flicker to a DSI PSRAM-bandwidth underrun (the driver logs "underrun
happens"), which is serial-verifiable without eyes on the screen. Kept
for future bus-contention debugging.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
The blue flicker during voice activity is a MIPI-DSI framebuffer-read
underrun on the PSRAM bus, not tearing. The DSI DMA reads the 800x800
framebuffer continuously (~138 MB/s at the stock 80 MHz clock); under
load the Wi-Fi (esp-hosted SDIO) and audio-playback DMA on the same
PSRAM bus starve the DSI FIFO, which the hardware then paints blue
("lcd.dsi.dpi: can't fetch data from external memory fast enough").
Drop the DPI pixel clock to 40 MHz (~54 fps): halves the continuous
read and gives the FIFO slack to ride out bursty SDIO latency spikes.
Still smooth for the face UI, and lowers display power.
Vendored as a local component because managed_components/ is gitignored
(same pattern as the esp_hosted fork) — a local component overrides the
same-named waveshare/ managed dependency, so the fix survives a clean
build.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
Root cause (investigation): the BSP default tear-avoid mode is
TRIPLE_PARTIAL. On ESP-IDF 5.5 the MIPI-DSI driver has no
on_frame_buf_complete callback (added in IDF 6.0 -> the boot warning
"buffer-switch...may not function on MIPI DSI"), so the adapter falls
back to on_refresh_done, which fires every refresh (~60Hz) as a fake
vsync. In PARTIAL mode that release path is NOT submit-gated, so when
one LVGL frame takes >1 refresh to render it over-releases the buffer
that is still being scanned out -> LVGL draws into the live front
buffer -> tearing/flicker.
Why it only started with the wake word, and only under load: at idle
(sparse rain) a frame renders in <16.6ms so exactly one submit per
refresh -> harmless. The always-on WakeNet added constant CPU/PSRAM
load that pushed the heavy-rain frames (listening=16, thinking=40
streams) past one refresh -> triggered the over-release. User
correctly identified it as a resource starve exposing the latent bug.
Fix: switch to TRIPLE_FULL, which IS submit-gated even without the
callback (one release per actual submit). Same 3 framebuffers, zero
memory cost, app-side one-liner via bsp_display_start_with_config so
the managed component is untouched. Trade-off: full-screen redraw per
frame; if the rain gets choppy under load, cheaper rain rendering
(canvas / half-rate tick) is the follow-up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Say "Computer" -> chime + listening -> speak -> AFE VAD detects you
stopped -> auto-sends. No taps. Touch still works as a manual override.
- esp-sr 2.4.6 added; wn9_computer_tts model packed into a new "model"
flash partition (MODEL_IN_FLASH). App moved to 8M, model 4M.
- audio.c: replaced the on-demand capture_task with an AFE pipeline —
feed_task is the SOLE mic reader (-> afe->feed); detect_task fetches,
watches wakeup_state for the wake word and vad_state for end of
speech, and forwards AFE-cleaned audio upstream during an utterance.
One mic reader ever.
- short rising chime acknowledges the wake audibly.
Fixes from adversarial review before trusting it:
1. utterance framing (blocking WS sends) moved OFF the AFE fetch thread
onto an app_task event queue (EV_TOUCH/EV_WAKE/EV_SPEECH_END) — a
1.5s send could stall fetch and drop the first ~1.5s of speech.
2. app_task is now the single serializer of start/end -> no TOCTOU
double-start (was: two utterance_start on a tap during wake).
3. VAD accounting resets on every streaming (re)start (wake OR tap),
not just wake -> a tapped utterance can no longer end instantly on
stale silence.
4. chime/reply set s_playing (+DMA tail hold) and detect_task skips the
mic while s_playing -> our own audio no longer streams into STT or
false-triggers the wake at a playback boundary (no AEC yet).
5. NULL-checked AFE create + feed buffer; tasks only start if AFE is up.
Verified on hardware: model loads, AFE inits with the Computer word,
boots and connects clean, no crash/wedge.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause (verified against our exact IDF tree, not the community guess):
the "258" in "sdio_write_task: Failed to send data: 258" is NOT a timeout
(that is 263). 258 = 0x102 = ESP_ERR_INVALID_ARG. On the ESP32-P4, block-
mode CMD53 writes require the SOURCE buffer to be 64-byte (cache-line)
aligned; the IDF sdmmc driver rejects a misaligned source with INVALID_ARG
BEFORE any bus activity. esp_hosts write loop then declares "Unrecoverable
host sdio state" and reboots the whole P4. The audio TX payload is not
64-aligned, so streaming mic audio wedged on the very FIRST frame (which is
exactly what we saw: listening -> instant Failed to send -> reboot).
This also explains why buffer/queue/clock/retry tuning all did nothing: the
write never reached the bus. And why our symptom was instant, not after
~100 writes (the community block-mode-desync theory) — it is the first
misaligned buffer, every time.
Fix: vendored esp_hosted 2.12.11 as an editable local component (overrides
the registry copy) and bounce a misaligned TX payload through one aligned
DMA scratch buffer in hosted_sdio_write_block (port_esp_hosted_host_sdio.c).
TX is serialized by the bus lock so a single static bounce buffer is safe;
freed in hosted_sdio_deinit. Host-only change — no C6 reflash.
VERIFIED ON HARDWARE (autonomous self-test): 40s of continuous mic-audio
upstream streaming — the traffic that previously wedged on the first frame
— ran clean, zero timeouts, zero reboots. A guarded SDIO_TX_SELFTEST harness
is kept (compiled out) for future SDIO stress testing.
Credit: root cause + patch designed via multi-agent investigation; the
precise 258=INVALID_ARG decode (correcting the upstream community timeout
assumption) came from checking our actual esp_err.h.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of the "blue screen on tap": NOT a firmware bug. The touch
fix worked (reached "listening"), and the mic audio then streamed
upstream — which instantly triggered the upstream SDIO wedge (#167):
sdio_write_task: Failed to send data: 258 (timeout) -> host restart
i.e. the same wedge we fought overnight, but now provoked reliably by
the sustained ~32KB/s mic upload instead of waiting minutes. The wedge
is a host->slave SDIO write timeout: the slave RX buffers fill because
its Wi-Fi TX cannot drain fast enough.
Attempt: restore the slave Wi-Fi TX path shrunk during the RAM hunt and
enlarge the SDIO queues so the burst is absorbed and drained:
- WIFI_RMT_DYNAMIC_TX_BUFFER_NUM 24->64, STATIC_RX 8->16, RX 24->32
- WIFI_RMT_TX_BA_WIN 16->32
- ESP_HOSTED_SDIO_TX_Q_SIZE / RX_Q_SIZE = 32
- SDIO clock 10->20 MHz for drain headroom (still 1-bit)
Boot verified: no RAM famine, connects clean. Whether it survives the
audio burst is the open question upstream has not solved.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TOUCH CRASH (the blue screen): app_on_touch ran a blocking WebSocket
send (up to 5s) directly in the LVGL touch callback, stalling the
MIPI-DSI flush into a garbage/blue frame + task-watchdog reboot on
every tap. Now touch_cb only gives a semaphore; a dedicated app_task
does the blocking sends, audio, and face changes off the render
thread. WS send timeout cut 5s to 1.5s as belt-and-braces.
RAIN = REACHABILITY (user request): rain now falls only while the
gateway WebSocket is live (gated on gw_connected in rain_tick). It
drains gracefully on disconnect, resumes on reconnect, a genuine
glanceable reachable signal. Idle density bumped 2 to 4 so
connected-idle reads distinctly from disconnected-black.
CALM THE WEDGE CHURN (user request): transient wifi/WS drops no longer
slam to the x_x error face or a CONNECTING banner. Boot goes straight
to the calm idle face (dry until connected). Only a sustained 30s+
outage escalates to x_x (clock_cb); the ~15s wedge-recovery just shows
a brief rain pause.
Two fixes from adversarial concurrency review before flashing:
- persistent single capture task (was xTaskCreate per utterance; a
rapid re-tap or WS-stop-vs-app-start race could put two readers on
one mic/I2S handle and corrupt the codec)
- reset s_talking on disconnect (app_on_disconnect) so the first tap
after a mid-utterance drop starts fresh, not the stop branch
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Night-watch caught a 21s wedge (vs 5-13min baseline) landing exactly
on the boot ping-diagnosis + wifi-stats-dump burst, which hammered the
freshly-recovered SDIO link on every reboot. That scaffolding proved
connectivity long ago; removing it makes each recovery light and clean.
ping_target/ping_diag_task kept as no-op symbols for manual use.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Stops the RuntimeError spam ('Cannot call receive once a disconnect
message has been received') every time the device's watchdog reboots.
Redeployed to the tatlock-ui stack.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The BT-off custom slave (2.12.12) can't survive the C6's OTA rollback
protection: it flashes, the C6 boots it, reverts to 2.12.11, host
reflashes -> ~30s reboot loop, worse than the wedge. BLE-coex
hypothesis is UNTESTABLE via OTA (custom slaves won't stick; needs
direct C6 UART flash, out of scope tonight). Reverted to the
self-healing 2.12.11 baseline: ~5-13min MTBF, 15s auto-recovery.
That is the shippable overnight state.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The wedge clock is immune to every network-layer change (~300s
regardless of bus width, clock, power-save, DHCP). A slave-side
periodic starver remains; the C6's BLE controller (BT_LE_SLEEP_ENABLE=y
in stock slave, shares the radio/RF arbiter with Wi-Fi) is the prime
suspect. DeskLock uses no Bluetooth, so the stock slave carries a
coexistence scheduler we can delete outright. Custom slave 2.12.12
(CONFIG_BT_ENABLED=n, ESP_HOSTED_CP_BT=n), OTA'd over SDIO. If the
metronome stops, BLE coexistence was starving the SDIO peripheral.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Four controlled experiments characterize it: 72-byte frame times out
at CMD53 after the slave advertised credits; bus width/clock/power
save have no effect; removing dhcpc stretches MTBF ~5min -> ~13min.
Slave-side peripheral stall, upstream bug. Device self-heals in ~15s.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The wedge metronome (~300s: 297/297/322/437s across bus widths,
clocks, and power modes) matches FRITZ DHCP T1 renewal at half of a
10-min lease. Every wedging build ran an active dhcp client. Static
addressing removes the renewal TX path entirely; if the metronome
dies, root cause #2 is the DHCP renew packet through hosted's write
path. (2.9.7 experiment abandoned: slave image bootloops/rolls back
on this C6; version matrix documented for upstream.)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1-bit bus alone did not stop the sdio_write timeout wedge (recurred
at ~5min uptime). Modem power save is the new prime suspect: the C6
napping mid-SDIO-transaction matches the minutes-scale periodicity.
Always-powered appliance -> PS_NONE costs ~50mW and removes the
entire failure class if correct. SDIO clock also halved to 10MHz.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Night-watch caught the instability in the act at 7min uptime:
E H_SDIO_DRV: sdio_write_task: Failed to send data: 258 (timeout)
E H_SDIO_DRV: Unrecoverable host sdio state -> SW_CPU_RESET
i.e. esp-hosted-mcu#167. Known mitigation: 1-bit SDIO bus (#148),
still ~80x voice bandwidth. Also: recovery reboots no longer ring
the gong (esp_reset_reason gate) - a butler doesn't bong at 3am.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of the dead data path (assoc/scan/RPC fine, zero data
frames): esp-hosted 1.4.x is formally incompatible with IDF 5.5
(esp-hosted-mcu#47) and the factory C6 slave firmware was ancient
(couldn't even answer a version RPC). Waveshare's examples pin 1.4.* —
do not follow them.
Fix:
- host: espressif/esp_hosted ^2.12 (+ esp_wifi_remote 1.6)
- slave: 2.12.11 network_adapter.bin embedded in the app (c6_ota.c
streams it to the C6 over the SDIO RPC channel at boot when the
reported version is < 2.x; ~10s, no wires, idempotent)
- RAM diet for hosted 2.x's footprint (MEMPOOL_PREFER_SPIRAM,
reduced WIFI_RMT buffers) — without it internal SRAM famine
boot-loops in xTaskCreateStaticPinnedToCore before app_main
- conservative 20MHz SDIO clock for first verified data path
Verified on hardware: DHCP lease (even that healed), 8/8 pings to
router and tower-of-joy at 1-5ms, WebSocket to the gateway connected,
device registered at /devices as desklock-p4 with fw version.
Also: wifi_diag.c L1 SoftAP diagnostic mode (DESKLOCK-DIAG) with
review fixes, boot ping ladder in net.c, static-IP fallback,
face_status() line, docs for the whole failure taxonomy.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- gateway: GET /devices registry (per-IP connect count, timestamps,
device/fw from new 'hello' protocol event); graceful utterance
errors -> error event; uvicorn ws-ping-timeout 120s
- firmware: hello on WS connect; static 192.168.86.53 fallback after
15s without DHCP; scan-on-boot removed (kept for diagnostics);
ping_interval 8s; fonts regenerated --no-compress (compressed
glyphs render blank with LV_USE_FONT_COMPRESSED off); 8MB factory
partition; montserrat_28; LVGL lock timeout semantics fixed
- docs: sauron endpoint plan, power ladder, versioning policy
Known issue: 'Outside' AP association succeeds but no L2 traffic
flows (no DHCP, no ARP) — worked once at 22:05 then never again;
ToJ side verified clean. Investigation ongoing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The device now runs the complete face contract on glass: seven states
with glyph expressions (custom 140px face font), matrix rain as pooled
label streams (22px DejaVu+Noto katakana font), rage kaomoji frames
(64px), orbit arc + elapsed counter wait cues, blink/talk/thought
animations, idle clock (SNTP, Europe/Amsterdam), and the power ladder
(active/ambient 35%/dormant 5% with rain parked).
Voice path: ES7210 mic capture task streams 16k PCM over
esp_websocket_client to the gateway; reply PCM buffers to PSRAM and
plays via the audio task; touch-to-talk (tap to speak, tap to send).
Protocol mapping per docs: thinking->effort, audio->speaking,
error event->rage (auto-composes after 2 loops), WS loss->error face.
Bring-up fixes: 8MB factory partition (fonts overflowed 1.5M),
bsp_display_lock(0) is try-lock in this adapter (use UINT32_MAX),
gong synth yields to feed IDLE0, wifi scan diagnostics (found SSID
case mismatch), esp_websocket_client pinned ~1.3 (1.4 needs IDF>5.5).
Verified on hardware: boots, connects to Wi-Fi and holds an open
WebSocket to the gateway. Sauron (iMac TTY endpoint + ops console)
planned in architecture.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
User verdict: audible and butler-non-intrusive on the bare 2W speaker.
Recorded as the house sound signature in architecture.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Additive gong synthesis: seven inharmonic partials with per-partial
decay (fundamental 165Hz rings 2.6s, highs die in 0.2s), a detuned
pair for shimmer, 45ms distant attack, and three feedback-comb
reflections (95/210/370ms, net gain .93) for a dense stone-space tail.
Normalized to -12dBFS at codec volume 58: quiet but voluminous.
Synthesized into PSRAM at boot (~4s), plays 5s, blocking is fine for
bring-up — audio moves to its own task in phase 2.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
C5->E5 half-second chime with fade-out at boot, right after the face
renders. Codec opens in slave mode, esp_codec_dev write path works.
Phase 1 bring-up is now complete: display, touch, audio, 200MHz PSRAM.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- firmware boots to a rendered LVGL face in 1.57s (eyes + smile + tag)
- sdkconfig: CONFIG_IDF_EXPERIMENTAL_FEATURES=y unlocks SPIRAM 200MHz
(silently degraded to 20MHz before -> MIPI-DSI underrun -> LVGL lock
starvation -> task watchdog); mirrors official 08_lvgl_demo_v9 config
- architecture.md: power management section (user prime concern) —
active/ambient/dormant/night ladder, levers, hard edges, <=300ms wake
- AGENTS/CLAUDE: replace stale bring-up warnings with verified commands
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The gateway service now lives in system-admin-toj/containers/stacks/
tatlock-ui.yml (shared docker-dataplane network: speaches by service
name, host-run Tatlock by LAN IP). Drop deploy/desklock-gateway.yml.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Live smoke test showed /v1/chat/completions replies open with the
Steward's <think> block (Open WebUI convention) — without stripping,
the device would speak the internal monologue aloud.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Settles the Python version question: floor >=3.11, no ceiling.
Container moves to python:3.13-slim, CI tests on 3.13. The embedded
Piper fallback resamples with numpy (already present via [speech]).
Verified with a live round trip at 16 kHz.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Face: new 'rage' state — 3-frame kaomoji loop (stare, flip the table,
put it back) for in-flight request failures; 'error' stays the quiet
persistent face for a dead link. Sim + artifact + design doc updated.
- Gateway: stt.py/tts.py are now pluggable backends. Default 'speaches'
talks OpenAI-format HTTP to the live container on :8601
(faster-whisper-small STT, Kokoro bm_george TTS with 24->16 kHz
audioop resample); 'embedded' fallback kept behind the [speech] extra.
Verified with a live TTS->STT round trip (warm: STT 0.27s, TTS 1.9s).
Docker image is now slim (no CUDA/ML deps). Python pinned to 3.12
(system 3.8 too old, audioop gone in 3.13).
- CI: .gitea/workflows/build.yml — lint+test on main pushes; on v* tags
test, build gateway image, push to registry, release, and trigger
Watchtower (tatlock pattern; needs REGISTRY_USER/REGISTRY_PASSWORD/
WATCHTOWER_TOKEN secrets). Runtime stack in deploy/desklock-gateway.yml.
- architecture.md: measured speech latencies, deployed-Speaches status,
CI & deployment section.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- sim/face/index.html: browser simulator of the 800x800 round panel with
six states (idle/listening/pensive/effort/speaking/error), state-driven
rain density, blink/talk/thought animations, idle clock
- effort state gets hard-required wait cues: orbiting bezel arc, elapsed
counter, max rain (Tatlock turns run 10-25s)
- rain driven by setInterval, not rAF: renders under the screenshot
tool's --virtual-time-budget and mirrors LVGL lv_timer
- architecture.md: Face design contract (state table, protocol mapping,
LVGL port notes) + latency table updated to GPU-era benchmarks
(Steward ~6s warm, full flow 11-25s; old CPU-era figures retired)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- STT/TTS move to a shared Speaches container (OpenAI-format, GPU, port
8601 proposed); gateway becomes a thin orchestrator with pluggable
speech backends (speaches default, embedded fallback)
- Record the on-device ceiling: WakeNet wake word, VAD, ES7210 AEC,
optional MultiNet fixed commands; open-vocabulary STT permanently out
- Record the real latency bottleneck (Tatlock ~2 min full local flow):
gateway must stream chat tokens and synthesize sentence-by-sentence
- Plan reply_delta + barge-in protocol additions
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DeskLock gives the Tatlock butler a face and voice on a Waveshare
ESP32-P4-WIFI6-Touch-LCD-3.4C round display in the living room.
- firmware/: ESP-IDF project targeting esp32p4 with the Waveshare XC BSP
- gateway/: FastAPI voice bridge (faster-whisper STT, Tatlock chat, Piper TTS)
- docs/architecture.md: component design and device<->gateway WS protocol
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
