diff --git a/AGENTS.md b/AGENTS.md index f37bbc7..60a44a1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -52,27 +52,27 @@ LVGL timers). `examples/esp-idf/` — notably `08_lvgl_demo_v9` (display), `06_I2SCodec` (audio), `04_wifistation` (Wi-Fi via ESP-Hosted). When wiring a new peripheral, check the official example first; do not guess pin mappings. -- ⚠️ **Unverified scaffold**: the BSP API calls in `desklock_main.c` and the - `sdkconfig.defaults` values were written before the first successful build. Validate - against the official examples on first bring-up, then delete this warning. - ```bash -# One-time: install ESP-IDF (not yet installed on tower-of-joy) -git clone -b v5.5 --recursive https://github.com/espressif/esp-idf.git ~/esp-idf -~/esp-idf/install.sh esp32p4 - -# Every shell: +# ESP-IDF v5.5 is installed at ~/esp-idf. Every shell: source ~/esp-idf/export.sh -# Build / flash / monitor (device is on USB-C; check `ls /dev/ttyACM*`) +# Build / flash (device on USB-C at /dev/ttyACM0, CH343 bridge) cd firmware -idf.py set-target esp32p4 # once idf.py build -idf.py -p /dev/ttyACM0 flash monitor # Ctrl+] exits monitor +sg dialout -c "bash -lc 'source ~/esp-idf/export.sh >/dev/null && idf.py -p /dev/ttyACM0 flash'" ``` -Flashing requires the `dialout` group (or run with sudo once and fix the group). If the -device doesn't enumerate, hold BOOT while pressing RESET to enter download mode. +- `sg dialout -c '…'` is needed because the login session predates the user's dialout + membership; a plain `idf.py flash` works after any re-login. +- **PSRAM must run at 200 MHz** or the 800×800 MIPI-DSI framebuffer underruns + (`lcd.dsi.dpi: can't fetch data…` spam, LVGL lock never frees, task watchdog). + `CONFIG_SPIRAM_SPEED_200M` only takes effect together with + `CONFIG_IDF_EXPERIMENTAL_FEATURES=y` — otherwise it is **silently dropped** and you + get 20 MHz. `sdkconfig.defaults` mirrors the official `08_lvgl_demo_v9` config. +- Non-interactive boot-log capture (avoid `idf.py monitor`, it's interactive): open + `/dev/ttyACM0` at 115200 with pyserial, pulse RTS to reset, read ~8 s. Verified boot + is ~1.6 s from reset to `desklock: DeskLock up`. +- If the device doesn't enumerate, hold BOOT while pressing RESET for download mode. ## Gateway (`gateway/`) diff --git a/CLAUDE.md b/CLAUDE.md index 781a3d5..5717899 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -28,11 +28,14 @@ make lint typecheck ## Gotchas -- **ESP-IDF is not yet installed on this machine** — install instructions in AGENTS.md. -- The firmware scaffold has never been built; treat BSP calls and sdkconfig as - provisional until first successful `idf.py build` (see warning in AGENTS.md). -- The device flashes over USB-C on this server, but it doesn't currently enumerate - (`/dev/ttyACM*` empty) and this user lacks the `dialout` group — resolve both before - attempting to flash. +- ESP-IDF v5.5 lives at `~/esp-idf` (`source ~/esp-idf/export.sh`). Flash via + `sg dialout -c …` (see AGENTS.md) — the login session predates dialout membership. +- **PSRAM 200 MHz requires `CONFIG_IDF_EXPERIMENTAL_FEATURES=y`** — without it the + option silently degrades to 20 MHz and the DSI display underruns into a watchdog + loop. Details in AGENTS.md. - Gateway speech deps are optional extras; `make setup` alone runs the app and tests - without GPU/ML packages. + without GPU/ML packages. `make setup` uses `python3.12` (system python3 is 3.8). +- Tatlock replies open with a `` block — always strip via + `tatlock.strip_reasoning()` before TTS or display. +- Low power is a prime user requirement: see "Power management" in + docs/architecture.md before touching the face/render loop. diff --git a/docs/architecture.md b/docs/architecture.md index 8c19942..cd3b561 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -164,6 +164,54 @@ WebSocket disconnected → `error` (quiet, persistent); otherwise `idle`. The `rage` frames additionally need `╯ ︵ ┻ ━ ┬ ─ ノ ° □ º` in the subset. - The sim's text glow (`text-shadow`) is browser flair — the device renders flat glyphs. +## Power management (prime concern) + +User requirement: the device idles on a wall 95%+ of its life — low power when nothing +is happening is a first-class design goal, not a phase-5 nicety. The face state machine +is therefore built around a **power ladder** from day one: + +| Power state | Backlight | Rendering | CPU | Entered when | +|-------------|-----------|-----------|-----|--------------| +| `active` | 100% | full animation | full clock | conversation in progress (listening→speaking) | +| `ambient` | ~35% | idle face, sparse rain | DFS enabled | idle, but activity in the last few minutes | +| `dormant` | off (or ≤5%) | **no redraws** — render loop parked | min clock via DFS | no voice/touch for N min (default 10) | +| `night` | off, panel sleep | none | min clock | schedule or "goodnight" command | + +Levers, in order of impact: + +1. **Backlight** — this is an IPS LCD: black pixels still burn backlight (unlike OLED), + so brightness is the dominant lever. `bsp_display_brightness_set()` drives it. +2. **Render idleness** — rain off and animations parked means LVGL stops producing + frames, which is what lets DFS actually reach its floor. +3. **DFS / power management** (`CONFIG_PM_ENABLE`) — automatic frequency scaling when + tasks are quiet. Note the MIPI-DSI constraint below. +4. **Radio** — the C6 runs Wi-Fi modem power-save; the gateway WebSocket widens its + ping interval when the device reports `dormant`. + +Wake triggers (any → `ambient`/`active`): wake word (phase 5), touch (from phase 2), +local VAD "someone is speaking" pre-warm, a gateway-initiated event (butler wants to +say something), scheduled morning end of `night`. + +**Hard edges — what limits how low we can go:** + +- **The hands-free promise sets the power floor.** Wake word requires mics + the AFE + pipeline running continuously; deep sleep is permanently off the table while the + device promises to answer its name. The floor is "CPU lightly loaded at min clock, + radios in power-save, backlight off." +- **DSI needs clocks while the panel is active** — the deepest CPU savings only unlock + in `dormant`/`night` when the panel stops being refreshed (panel sleep / blank). +- **Wake latency budget: ≤ ~300 ms** from trigger to visible face (backlight ramp + + first render). Anything slower reads as "it's off," which kills the butler illusion. +- **Touch stays powered** in all states except possibly `night` — its idle draw is + negligible and tap-to-wake must always work. +- **No invented numbers**: actual draw gets measured with a USB power meter at each + phase; working target is `dormant` ≤ ⅓ of `active`. (Always-on device: every watt + saved ≈ 9 kWh/year.) + +Implementation order: backlight dimming + `dormant` timeout + touch wake land in +**phase 2** with the face state machine (timeout-driven); voice-linked triggers upgrade +it in phase 5. + ## Latency budget & streaming Measured/known numbers that shape the design (Tatlock figures per tatlock CLAUDE.md, diff --git a/firmware/main/desklock_main.c b/firmware/main/desklock_main.c index c266dd2..c222dee 100644 --- a/firmware/main/desklock_main.c +++ b/firmware/main/desklock_main.c @@ -6,25 +6,55 @@ * 2. Wi-Fi up via ESP32-C6 (ESP-Hosted over SDIO) * 3. WebSocket connection to the DeskLock gateway * 4. Face state machine: idle / listening / thinking / speaking + * + * Current stage: bring-up — display + a placeholder face. */ #include "esp_log.h" #include "bsp/esp-bsp.h" +#include "lvgl.h" static const char *TAG = "desklock"; +static void show_placeholder_face(void) +{ + bsp_display_lock(0); + + lv_obj_t *scr = lv_screen_active(); + lv_obj_set_style_bg_color(scr, lv_color_black(), 0); + + lv_obj_t *eyes = lv_label_create(scr); + lv_label_set_text(eyes, "- -"); + lv_obj_set_style_text_color(eyes, lv_color_hex(0xADFFC8), 0); + lv_obj_set_style_text_font(eyes, &lv_font_montserrat_48, 0); + lv_obj_align(eyes, LV_ALIGN_CENTER, 0, -80); + + lv_obj_t *mouth = lv_label_create(scr); + lv_label_set_text(mouth, "\\_/"); + lv_obj_set_style_text_color(mouth, lv_color_hex(0xADFFC8), 0); + lv_obj_set_style_text_font(mouth, &lv_font_montserrat_48, 0); + lv_obj_align(mouth, LV_ALIGN_CENTER, 0, 40); + + lv_obj_t *tag = lv_label_create(scr); + lv_label_set_text(tag, "DESKLOCK"); + lv_obj_set_style_text_color(tag, lv_color_hex(0x2F7A4B), 0); + lv_obj_align(tag, LV_ALIGN_CENTER, 0, 160); + + bsp_display_unlock(); +} + void app_main(void) { ESP_LOGI(TAG, "DeskLock starting"); bsp_display_start(); bsp_display_backlight_on(); + show_placeholder_face(); - /* TODO(bring-up): render placeholder face via LVGL - * TODO(bring-up): Wi-Fi via ESP-Hosted (C6) - * TODO(voice): mic capture (ES7210) -> WebSocket upstream - * TODO(voice): gateway PCM downstream -> ES8311 playback + ESP_LOGI(TAG, "DeskLock up (placeholder face)"); + + /* TODO(face): full LVGL face state machine per docs/architecture.md + * TODO(net): Wi-Fi via ESP-Hosted (C6), WebSocket to gateway + * TODO(voice): ES7210 capture upstream, ES8311 playback downstream */ - - ESP_LOGI(TAG, "DeskLock up (display only)"); } diff --git a/firmware/sdkconfig.defaults b/firmware/sdkconfig.defaults index f58aa73..67a23b8 100644 --- a/firmware/sdkconfig.defaults +++ b/firmware/sdkconfig.defaults @@ -5,10 +5,19 @@ CONFIG_IDF_TARGET="esp32p4" CONFIG_ESPTOOLPY_FLASHSIZE_32MB=y CONFIG_PARTITION_TABLE_SINGLE_APP_LARGE=y -# 32 MB in-package PSRAM (hex mode) — LVGL frame buffers live here +# PSRAM at 200 MHz — REQUIRED for the 800x800 MIPI-DSI framebuffer. +# 200 MHz is gated behind IDF_EXPERIMENTAL_FEATURES; without it the build +# silently falls back to 20 MHz and the DSI underruns (mirrors the official +# 08_lvgl_demo_v9 example config). CONFIG_SPIRAM=y -CONFIG_SPIRAM_MODE_HEX=y +CONFIG_IDF_EXPERIMENTAL_FEATURES=y CONFIG_SPIRAM_SPEED_200M=y +CONFIG_SPIRAM_XIP_FROM_PSRAM=y +CONFIG_CACHE_L2_CACHE_256KB=y +CONFIG_CACHE_L2_CACHE_LINE_128B=y CONFIG_COMPILER_OPTIMIZATION_PERF=y CONFIG_FREERTOS_HZ=1000 + +# LVGL font for the bring-up placeholder face +CONFIG_LV_FONT_MONTSERRAT_48=y