First successful flash: placeholder face live; fix 20MHz PSRAM trap; power ladder
- firmware boots to a rendered LVGL face in 1.57s (eyes + smile + tag) - sdkconfig: CONFIG_IDF_EXPERIMENTAL_FEATURES=y unlocks SPIRAM 200MHz (silently degraded to 20MHz before -> MIPI-DSI underrun -> LVGL lock starvation -> task watchdog); mirrors official 08_lvgl_demo_v9 config - architecture.md: power management section (user prime concern) — active/ambient/dormant/night ladder, levers, hard edges, <=300ms wake - AGENTS/CLAUDE: replace stale bring-up warnings with verified commands Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Claude Fable 5
@@ -52,27 +52,27 @@ LVGL timers).
|
||||
`examples/esp-idf/` — notably `08_lvgl_demo_v9` (display), `06_I2SCodec` (audio),
|
||||
`04_wifistation` (Wi-Fi via ESP-Hosted). When wiring a new peripheral, check the
|
||||
official example first; do not guess pin mappings.
|
||||
- ⚠️ **Unverified scaffold**: the BSP API calls in `desklock_main.c` and the
|
||||
`sdkconfig.defaults` values were written before the first successful build. Validate
|
||||
against the official examples on first bring-up, then delete this warning.
|
||||
|
||||
```bash
|
||||
# One-time: install ESP-IDF (not yet installed on tower-of-joy)
|
||||
git clone -b v5.5 --recursive https://github.com/espressif/esp-idf.git ~/esp-idf
|
||||
~/esp-idf/install.sh esp32p4
|
||||
|
||||
# Every shell:
|
||||
# ESP-IDF v5.5 is installed at ~/esp-idf. Every shell:
|
||||
source ~/esp-idf/export.sh
|
||||
|
||||
# Build / flash / monitor (device is on USB-C; check `ls /dev/ttyACM*`)
|
||||
# Build / flash (device on USB-C at /dev/ttyACM0, CH343 bridge)
|
||||
cd firmware
|
||||
idf.py set-target esp32p4 # once
|
||||
idf.py build
|
||||
idf.py -p /dev/ttyACM0 flash monitor # Ctrl+] exits monitor
|
||||
sg dialout -c "bash -lc 'source ~/esp-idf/export.sh >/dev/null && idf.py -p /dev/ttyACM0 flash'"
|
||||
```
|
||||
|
||||
Flashing requires the `dialout` group (or run with sudo once and fix the group). If the
|
||||
device doesn't enumerate, hold BOOT while pressing RESET to enter download mode.
|
||||
- `sg dialout -c '…'` is needed because the login session predates the user's dialout
|
||||
membership; a plain `idf.py flash` works after any re-login.
|
||||
- **PSRAM must run at 200 MHz** or the 800×800 MIPI-DSI framebuffer underruns
|
||||
(`lcd.dsi.dpi: can't fetch data…` spam, LVGL lock never frees, task watchdog).
|
||||
`CONFIG_SPIRAM_SPEED_200M` only takes effect together with
|
||||
`CONFIG_IDF_EXPERIMENTAL_FEATURES=y` — otherwise it is **silently dropped** and you
|
||||
get 20 MHz. `sdkconfig.defaults` mirrors the official `08_lvgl_demo_v9` config.
|
||||
- Non-interactive boot-log capture (avoid `idf.py monitor`, it's interactive): open
|
||||
`/dev/ttyACM0` at 115200 with pyserial, pulse RTS to reset, read ~8 s. Verified boot
|
||||
is ~1.6 s from reset to `desklock: DeskLock up`.
|
||||
- If the device doesn't enumerate, hold BOOT while pressing RESET for download mode.
|
||||
|
||||
## Gateway (`gateway/`)
|
||||
|
||||
|
||||
@@ -28,11 +28,14 @@ make lint typecheck
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **ESP-IDF is not yet installed on this machine** — install instructions in AGENTS.md.
|
||||
- The firmware scaffold has never been built; treat BSP calls and sdkconfig as
|
||||
provisional until first successful `idf.py build` (see warning in AGENTS.md).
|
||||
- The device flashes over USB-C on this server, but it doesn't currently enumerate
|
||||
(`/dev/ttyACM*` empty) and this user lacks the `dialout` group — resolve both before
|
||||
attempting to flash.
|
||||
- ESP-IDF v5.5 lives at `~/esp-idf` (`source ~/esp-idf/export.sh`). Flash via
|
||||
`sg dialout -c …` (see AGENTS.md) — the login session predates dialout membership.
|
||||
- **PSRAM 200 MHz requires `CONFIG_IDF_EXPERIMENTAL_FEATURES=y`** — without it the
|
||||
option silently degrades to 20 MHz and the DSI display underruns into a watchdog
|
||||
loop. Details in AGENTS.md.
|
||||
- Gateway speech deps are optional extras; `make setup` alone runs the app and tests
|
||||
without GPU/ML packages.
|
||||
without GPU/ML packages. `make setup` uses `python3.12` (system python3 is 3.8).
|
||||
- Tatlock replies open with a `<think>` block — always strip via
|
||||
`tatlock.strip_reasoning()` before TTS or display.
|
||||
- Low power is a prime user requirement: see "Power management" in
|
||||
docs/architecture.md before touching the face/render loop.
|
||||
|
||||
@@ -164,6 +164,54 @@ WebSocket disconnected → `error` (quiet, persistent); otherwise `idle`.
|
||||
The `rage` frames additionally need `╯ ︵ ┻ ━ ┬ ─ ノ ° □ º` in the subset.
|
||||
- The sim's text glow (`text-shadow`) is browser flair — the device renders flat glyphs.
|
||||
|
||||
## Power management (prime concern)
|
||||
|
||||
User requirement: the device idles on a wall 95%+ of its life — low power when nothing
|
||||
is happening is a first-class design goal, not a phase-5 nicety. The face state machine
|
||||
is therefore built around a **power ladder** from day one:
|
||||
|
||||
| Power state | Backlight | Rendering | CPU | Entered when |
|
||||
|-------------|-----------|-----------|-----|--------------|
|
||||
| `active` | 100% | full animation | full clock | conversation in progress (listening→speaking) |
|
||||
| `ambient` | ~35% | idle face, sparse rain | DFS enabled | idle, but activity in the last few minutes |
|
||||
| `dormant` | off (or ≤5%) | **no redraws** — render loop parked | min clock via DFS | no voice/touch for N min (default 10) |
|
||||
| `night` | off, panel sleep | none | min clock | schedule or "goodnight" command |
|
||||
|
||||
Levers, in order of impact:
|
||||
|
||||
1. **Backlight** — this is an IPS LCD: black pixels still burn backlight (unlike OLED),
|
||||
so brightness is the dominant lever. `bsp_display_brightness_set()` drives it.
|
||||
2. **Render idleness** — rain off and animations parked means LVGL stops producing
|
||||
frames, which is what lets DFS actually reach its floor.
|
||||
3. **DFS / power management** (`CONFIG_PM_ENABLE`) — automatic frequency scaling when
|
||||
tasks are quiet. Note the MIPI-DSI constraint below.
|
||||
4. **Radio** — the C6 runs Wi-Fi modem power-save; the gateway WebSocket widens its
|
||||
ping interval when the device reports `dormant`.
|
||||
|
||||
Wake triggers (any → `ambient`/`active`): wake word (phase 5), touch (from phase 2),
|
||||
local VAD "someone is speaking" pre-warm, a gateway-initiated event (butler wants to
|
||||
say something), scheduled morning end of `night`.
|
||||
|
||||
**Hard edges — what limits how low we can go:**
|
||||
|
||||
- **The hands-free promise sets the power floor.** Wake word requires mics + the AFE
|
||||
pipeline running continuously; deep sleep is permanently off the table while the
|
||||
device promises to answer its name. The floor is "CPU lightly loaded at min clock,
|
||||
radios in power-save, backlight off."
|
||||
- **DSI needs clocks while the panel is active** — the deepest CPU savings only unlock
|
||||
in `dormant`/`night` when the panel stops being refreshed (panel sleep / blank).
|
||||
- **Wake latency budget: ≤ ~300 ms** from trigger to visible face (backlight ramp +
|
||||
first render). Anything slower reads as "it's off," which kills the butler illusion.
|
||||
- **Touch stays powered** in all states except possibly `night` — its idle draw is
|
||||
negligible and tap-to-wake must always work.
|
||||
- **No invented numbers**: actual draw gets measured with a USB power meter at each
|
||||
phase; working target is `dormant` ≤ ⅓ of `active`. (Always-on device: every watt
|
||||
saved ≈ 9 kWh/year.)
|
||||
|
||||
Implementation order: backlight dimming + `dormant` timeout + touch wake land in
|
||||
**phase 2** with the face state machine (timeout-driven); voice-linked triggers upgrade
|
||||
it in phase 5.
|
||||
|
||||
## Latency budget & streaming
|
||||
|
||||
Measured/known numbers that shape the design (Tatlock figures per tatlock CLAUDE.md,
|
||||
|
||||
@@ -6,25 +6,55 @@
|
||||
* 2. Wi-Fi up via ESP32-C6 (ESP-Hosted over SDIO)
|
||||
* 3. WebSocket connection to the DeskLock gateway
|
||||
* 4. Face state machine: idle / listening / thinking / speaking
|
||||
*
|
||||
* Current stage: bring-up — display + a placeholder face.
|
||||
*/
|
||||
|
||||
#include "esp_log.h"
|
||||
#include "bsp/esp-bsp.h"
|
||||
#include "lvgl.h"
|
||||
|
||||
static const char *TAG = "desklock";
|
||||
|
||||
static void show_placeholder_face(void)
|
||||
{
|
||||
bsp_display_lock(0);
|
||||
|
||||
lv_obj_t *scr = lv_screen_active();
|
||||
lv_obj_set_style_bg_color(scr, lv_color_black(), 0);
|
||||
|
||||
lv_obj_t *eyes = lv_label_create(scr);
|
||||
lv_label_set_text(eyes, "- -");
|
||||
lv_obj_set_style_text_color(eyes, lv_color_hex(0xADFFC8), 0);
|
||||
lv_obj_set_style_text_font(eyes, &lv_font_montserrat_48, 0);
|
||||
lv_obj_align(eyes, LV_ALIGN_CENTER, 0, -80);
|
||||
|
||||
lv_obj_t *mouth = lv_label_create(scr);
|
||||
lv_label_set_text(mouth, "\\_/");
|
||||
lv_obj_set_style_text_color(mouth, lv_color_hex(0xADFFC8), 0);
|
||||
lv_obj_set_style_text_font(mouth, &lv_font_montserrat_48, 0);
|
||||
lv_obj_align(mouth, LV_ALIGN_CENTER, 0, 40);
|
||||
|
||||
lv_obj_t *tag = lv_label_create(scr);
|
||||
lv_label_set_text(tag, "DESKLOCK");
|
||||
lv_obj_set_style_text_color(tag, lv_color_hex(0x2F7A4B), 0);
|
||||
lv_obj_align(tag, LV_ALIGN_CENTER, 0, 160);
|
||||
|
||||
bsp_display_unlock();
|
||||
}
|
||||
|
||||
void app_main(void)
|
||||
{
|
||||
ESP_LOGI(TAG, "DeskLock starting");
|
||||
|
||||
bsp_display_start();
|
||||
bsp_display_backlight_on();
|
||||
show_placeholder_face();
|
||||
|
||||
/* TODO(bring-up): render placeholder face via LVGL
|
||||
* TODO(bring-up): Wi-Fi via ESP-Hosted (C6)
|
||||
* TODO(voice): mic capture (ES7210) -> WebSocket upstream
|
||||
* TODO(voice): gateway PCM downstream -> ES8311 playback
|
||||
ESP_LOGI(TAG, "DeskLock up (placeholder face)");
|
||||
|
||||
/* TODO(face): full LVGL face state machine per docs/architecture.md
|
||||
* TODO(net): Wi-Fi via ESP-Hosted (C6), WebSocket to gateway
|
||||
* TODO(voice): ES7210 capture upstream, ES8311 playback downstream
|
||||
*/
|
||||
|
||||
ESP_LOGI(TAG, "DeskLock up (display only)");
|
||||
}
|
||||
|
||||
@@ -5,10 +5,19 @@ CONFIG_IDF_TARGET="esp32p4"
|
||||
CONFIG_ESPTOOLPY_FLASHSIZE_32MB=y
|
||||
CONFIG_PARTITION_TABLE_SINGLE_APP_LARGE=y
|
||||
|
||||
# 32 MB in-package PSRAM (hex mode) — LVGL frame buffers live here
|
||||
# PSRAM at 200 MHz — REQUIRED for the 800x800 MIPI-DSI framebuffer.
|
||||
# 200 MHz is gated behind IDF_EXPERIMENTAL_FEATURES; without it the build
|
||||
# silently falls back to 20 MHz and the DSI underruns (mirrors the official
|
||||
# 08_lvgl_demo_v9 example config).
|
||||
CONFIG_SPIRAM=y
|
||||
CONFIG_SPIRAM_MODE_HEX=y
|
||||
CONFIG_IDF_EXPERIMENTAL_FEATURES=y
|
||||
CONFIG_SPIRAM_SPEED_200M=y
|
||||
CONFIG_SPIRAM_XIP_FROM_PSRAM=y
|
||||
CONFIG_CACHE_L2_CACHE_256KB=y
|
||||
CONFIG_CACHE_L2_CACHE_LINE_128B=y
|
||||
|
||||
CONFIG_COMPILER_OPTIMIZATION_PERF=y
|
||||
CONFIG_FREERTOS_HZ=1000
|
||||
|
||||
# LVGL font for the bring-up placeholder face
|
||||
CONFIG_LV_FONT_MONTSERRAT_48=y
|
||||
|
||||
Reference in New Issue
Block a user