diff --git a/AGENTS.md b/AGENTS.md deleted file mode 100644 index 8afcfbc..0000000 --- a/AGENTS.md +++ /dev/null @@ -1,131 +0,0 @@ -# AGENTS.md - -> Operational protocols and architecture for AI assistants working on DeskLock. -> Read [docs/architecture.md](docs/architecture.md) before making design changes. - -## What this project is - -DeskLock is the living-room visual/audio endpoint for **Tatlock**, the homelab butler -(`/mnt/media/Projects/tatlock`, API at `http://tatlock:8000` on `docker-dataplane`). -Two halves, one repo: - -- `firmware/` — ESP-IDF (C, LVGL 9) app for the Waveshare ESP32-P4-WIFI6-Touch-LCD-3.4C - (3.4" round 800×800 touch display, dual mics + ES7210 AEC, ES8311 codec + speaker). -- `gateway/` — Python FastAPI container on tower-of-joy orchestrating STT → chat - (Tatlock `/v1/chat/completions`) → TTS. Listens on port **8600**. STT/TTS models live - in the shared **Speaches** container (live on port 8601, OpenAI-format API), not in - the gateway image; `stt.py`/`tts.py` are pluggable backends (`speaches` default, - `embedded` fallback needing the `[speech]` extra). Gateway needs **Python ≥ 3.11** - (no ceiling; the container runs 3.13) — but system python3 on tower-of-joy is 3.8, - so `make setup` explicitly uses `python3.12`. No local audio resampling in the - default path: the gateway requests 16 kHz output via Speaches' `sample_rate` - extension (verified live). - -The device and gateway speak a WebSocket protocol defined in `docs/architecture.md`. -**That doc is the contract** — update it in the same change as any protocol edit on -either side. - -The face (black screen, ASCII glyph expressions, matrix rain as activity signal) is -designed in `sim/face/index.html` — the design source of truth — and specified in the -"Face design" section of `docs/architecture.md`. Change the sim and the doc together; -the LVGL implementation follows them. Verify sim changes visually with -`~/bin/claude-screenshot` (note: the tool uses `--virtual-time-budget`, which starves -`requestAnimationFrame` — drive sim animation with `setInterval`, which also mirrors -LVGL timers). - -## Hard rules - -- **Keep the firmware thin.** No STT, no TTS, no conversation logic on the device. - If a feature needs intelligence, it goes in the gateway or in Tatlock itself. -- **Never modify Tatlock from this repo.** It is a separate project with its own repo. - DeskLock consumes its public API only. -- **Secrets** (Wi-Fi credentials, any future API keys) never go in source. Firmware - gets them via a gitignored `firmware/secrets.h` (see AGENTS notes below) or NVS; - the gateway via environment variables (`DESKLOCK_*`). - -## Firmware (`firmware/`) - -- Toolchain: **ESP-IDF ≥ 5.4** (not Arduino, not PlatformIO). Target `esp32p4`. -- BSP: [`waveshare/esp32_p4_wifi6_touch_lcd_xc`](https://components.espressif.com/components/waveshare/esp32_p4_wifi6_touch_lcd_xc) - from the ESP Component Registry (pulled automatically via `main/idf_component.yml`). -- Reference implementations: [waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC](https://github.com/waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC) - `examples/esp-idf/` — notably `08_lvgl_demo_v9` (display), `06_I2SCodec` (audio), - `04_wifistation` (Wi-Fi via ESP-Hosted). When wiring a new peripheral, check the - official example first; do not guess pin mappings. -```bash -# ESP-IDF v5.5 is installed at ~/esp-idf. Every shell: -source ~/esp-idf/export.sh - -# Build / flash (device on USB-C at /dev/ttyACM0, CH343 bridge) -cd firmware -idf.py build -sg dialout -c "bash -lc 'source ~/esp-idf/export.sh >/dev/null && idf.py -p /dev/ttyACM0 flash'" -``` - -- `sg dialout -c '…'` is needed because the login session predates the user's dialout - membership; a plain `idf.py flash` works after any re-login. -- **Radio stack: esp_hosted ≥ 2.x on BOTH chips, non-negotiable.** esp-hosted 1.x is - formally incompatible with IDF 5.5 (esp-hosted-mcu#47) — symptom: RPC/scan/connect - all work, but NO data frames ever flow (no DHCP, no ARP, no ping). Waveshare's - examples pin 1.4.* and the factory C6 slave firmware is ancient — both wrong. The - host manifest pins `espressif/esp_hosted: "^2.12"`; the matching slave image is - embedded as `main/c6_slave.bin` and `c6_ota.c` flashes the C6 **over SDIO** at boot - whenever the C6 reports a version < 2.x (build a new bin from the component's - `slave/` project for esp32c6 when bumping versions). -- **Internal-RAM famine assert**: `assert failed: xTaskCreateStaticPinnedToCore … - xPortcheckValidStackMem` in a pre-app_main boot loop means static+early allocations - starved internal SRAM (hosted 2.x is hungry). Keep - `CONFIG_ESP_HOSTED_MEMPOOL_PREFER_SPIRAM=y` and the reduced `WIFI_RMT_*` buffer - counts in sdkconfig.defaults; check `heap_init:` pool lines when the binary grows. -- SDIO clock is set conservatively (`CONFIG_ESP_HOSTED_SDIO_CLOCK_FREQ_KHZ=20000`), - ample for 16 kHz voice; raising to 40 MHz is untested on this board's data path. -- **L1 driver diagnosis mode**: set `WIFI_DIAG_MODE 1` in desklock_main.c — the device - becomes AP `DESKLOCK-DIAG` (pass `desklock123`, page at http://192.168.4.1/) proving - radio+SDIO+IP with zero external network variables. Ladder: L0 SDIO control → L1 - softap data → L2 STA to any network → L3 STA to "Outside" → L4 gateway. -- **PSRAM must run at 200 MHz** or the 800×800 MIPI-DSI framebuffer underruns - (`lcd.dsi.dpi: can't fetch data…` spam, LVGL lock never frees, task watchdog). - `CONFIG_SPIRAM_SPEED_200M` only takes effect together with - `CONFIG_IDF_EXPERIMENTAL_FEATURES=y` — otherwise it is **silently dropped** and you - get 20 MHz. `sdkconfig.defaults` mirrors the official `08_lvgl_demo_v9` config. -- Non-interactive boot-log capture (avoid `idf.py monitor`, it's interactive): open - `/dev/ttyACM0` at 115200 with pyserial, pulse RTS to reset, read ~8 s. Verified boot - is ~1.6 s from reset to `desklock: DeskLock up`. -- If the device doesn't enumerate, hold BOOT while pressing RESET for download mode. - -## Gateway (`gateway/`) - -```bash -cd gateway -make setup # venv + dev deps (no ML models) -make setup-speech # additionally install faster-whisper + piper -make run # uvicorn on :8600 with reload -make test # pytest -make lint # ruff check + format check -make typecheck # mypy -``` - -- Config via `DESKLOCK_*` env vars — see `src/desklock_gateway/config.py` for the schema - and defaults. -- `stt.py` / `tts.py` defer their heavy imports so the app boots without the `speech` - extra — keep it that way so protocol tests stay fast. -- Deployment is CI-driven: pushing a `v*` tag makes Gitea Actions test, build, and push - `desklock-gateway:{latest,tag}` to the registry and trigger Watchtower - (`.gitea/workflows/build.yml`; needs `REGISTRY_USER`/`REGISTRY_PASSWORD`/ - `WATCHTOWER_HTTP_API_TOKEN` secrets). Plain pushes to `main` run lint + tests only. The - gateway deploys as part of the **`tatlock-ui` Portainer stack** — - `system-admin-toj/containers/stacks/tatlock-ui.yml` (registered in `CONTAINERS.md`, - port 8600). Stack updates go through the Portainer API on :8001 (JWT auth; recipe in - `system-admin-toj/containers/setup-new-host.md`), not by editing files on disk. -- Verify speech changes against the live Speaches container with a real round trip - (TTS → STT of a known phrase, expect the transcript back); warm timings to expect: - STT ~0.3 s, TTS ~2 s per sentence. - -## Homelab context - -- This server **is** tower-of-joy; the device, gateway, and Tatlock all share the LAN. -- Machine-to-machine traffic uses container names on the `docker-dataplane` network — - `http://tatlock:8000` (the deployed stack overrides `DESKLOCK_TATLOCK_BASE_URL` to - the host LAN IP while Tatlock runs on the host). Browser-facing URLs use - `https://.schweitz.net` (NPM, Authentik SSO) — never for M2M traffic. -- Git remote: `git.schweitz.net` (Gitea). diff --git a/CLAUDE.md b/CLAUDE.md index b53e4f1..5cbad9d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,45 +1,172 @@ -# CLAUDE.md +# CLAUDE.md — desklock -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. +Two components, one repo, coupled by a shared WebSocket protocol: -Claude Code-specific notes for this project. For architecture, hard rules, and full -command reference — see [AGENTS.md](AGENTS.md), and read it before starting work. +- `firmware/` — ESP-IDF (C, LVGL 9) app for a Waveshare ESP32-P4-WIFI6-Touch-LCD-3.4C + (3.4" round 800×800 touch display, dual mics + ES7210 AEC, ES8311 codec + speaker). + Flashed over USB; not containerized. +- `gateway/` — Python/FastAPI container `desklock-gateway`, port **8600**, part of the + **`tatlock-ui`** Portainer stack (`system-admin-toj/containers/stacks/tatlock-ui.yml`, + verified against the live container and `CONTAINERS.md`). Orchestrates STT → Tatlock + chat → TTS; carries no ML dependencies itself. -## Quick orientation +The device↔gateway protocol is specified in `docs/architecture.md` under "WebSocket +protocol (device ↔ gateway)". **Any protocol change updates that file in the same +change** — it is the contract, not a description of one side's behavior. -DeskLock = firmware for a Waveshare ESP32-P4 round-display device (`firmware/`, ESP-IDF/C/LVGL) -plus a voice gateway container (`gateway/`, Python/FastAPI, port 8600) that bridges device -audio to the Tatlock butler API. The device↔gateway WebSocket protocol lives in -`docs/architecture.md` and must stay in sync with both implementations. +**Never modify Tatlock from this repo.** desklock consumes Tatlock's public API only +(`http://tatlock:8000` on `docker-dataplane`); this is a standing cross-repo rule, not +local policy. + +**Keep the firmware thin.** No STT, no TTS, no conversation logic on the device — that +intelligence belongs in the gateway or in Tatlock itself. + +**Secrets never go in source.** Firmware gets them via gitignored `firmware/main/secrets.h` +(verified: gitignored, and `#include`d by `gw_client.c`/`net.c`) or NVS; the gateway via +`DESKLOCK_*` environment variables. + +## Gotchas — firmware + +- Toolchain: **ESP-IDF ≥ 5.4** (this box has 5.5, installed at `~/esp-idf`), not + Arduino, not PlatformIO. Target `esp32p4`. `source ~/esp-idf/export.sh` is required + every shell — `idf.py` is not on the non-interactive PATH otherwise. +- BSP: `waveshare/esp32_p4_wifi6_touch_lcd_xc` from the ESP Component Registry, pulled + automatically via `main/idf_component.yml`. Reference implementations: + [waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC](https://github.com/waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC) + `examples/esp-idf/` — `08_lvgl_demo_v9` (display), `06_I2SCodec` (audio), + `04_wifistation` (Wi-Fi). Check the official example before guessing a pin mapping. +- Flashing needs the `dialout` group; a login session started before that membership + took effect needs `sg dialout -c "bash -lc 'source ~/esp-idf/export.sh >/dev/null && + idf.py -p /dev/ttyACM0 flash'"` — a plain `idf.py flash` works after any re-login. +- **PSRAM 200 MHz requires `CONFIG_IDF_EXPERIMENTAL_FEATURES=y`.** Without it, + `CONFIG_SPIRAM_SPEED_200M` is silently dropped to 20 MHz and the 800×800 MIPI-DSI + framebuffer underruns (`lcd.dsi.dpi: can't fetch data…` spam, LVGL lock never frees, + task watchdog). Verified both settings present in `firmware/sdkconfig.defaults`. +- **Wi-Fi radio stack must be esp_hosted ≥ 2.x on both the P4 host and the C6 slave, + non-negotiable.** 1.x is formally incompatible with IDF 5.5 (esp-hosted-mcu#47) — + symptom is control-plane-only: RPC/scan/connect all work, but no data frame ever + flows (no DHCP, no ARP, no ping). Waveshare's examples and the factory C6 slave + firmware both pin the wrong (1.x-era) version. The host manifest pins + `espressif/esp_hosted: "^2.12"` (verified in `firmware/main/idf_component.yml`); the + matching slave image is embedded as `main/c6_slave.bin`, and `c6_ota.c` flashes the + C6 over SDIO at boot whenever it reports a version below 2.x. +- **Boot-loop assert `xTaskCreateStaticPinnedToCore … xPortcheckValidStackMem`** before + `app_main` means internal SRAM starvation (hosted 2.x is hungry). Keep + `CONFIG_ESP_HOSTED_MEMPOOL_PREFER_SPIRAM=y` and the reduced `WIFI_RMT_*` buffer counts + in `sdkconfig.defaults` (both verified present); check `heap_init:` pool lines in the + boot log when the binary grows. +- SDIO clock is conservative by design: `CONFIG_ESP_HOSTED_SDIO_CLOCK_FREQ_KHZ=20000` + (verified), ample for 16 kHz voice — raising it to 40 MHz is untested on this board's + data path. +- **Wi-Fi diagnosis ladder**: set `WIFI_DIAG_MODE 1` in `desklock_main.c` (verified the + macro and `#if` guard exist, currently `0`) — the device becomes AP `DESKLOCK-DIAG` + (password `desklock123`, page at `http://192.168.4.1/`, verified in `wifi_diag.c`), + proving radio+SDIO+IP with zero external network variables. Ladder: L0 SDIO control → + L1 softap data → L2 STA to any network → L3 STA to "Outside" → L4 gateway. +- Non-interactive boot-log capture: avoid `idf.py monitor` (interactive) — open + `/dev/ttyACM0` at 115200 with pyserial, pulse RTS to reset, read ~8s. Reported boot + time (~1.6s to `desklock: DeskLock up`) is carried from `AGENTS.md` and was **not** + re-timed this pass — no device was connected in this session (see Liveness below). +- If the device doesn't enumerate, hold BOOT while pressing RESET for download mode. + +## Gotchas — gateway + +- Gateway speech deps (`faster-whisper`, `piper-tts`) are an optional extra — + `make setup` alone runs the app and the test suite without them. `make setup` invokes + `python3.12` explicitly; system `python3` on tower-of-joy is 3.8. +- `ruff` and `mypy` are **not** on the non-interactive PATH — they exist only inside + `gateway/.venv/bin/` once `make setup` has run. Use `make lint` / `make typecheck`, or + invoke `.venv/bin/ruff` / `.venv/bin/mypy` directly; a bare `ruff`/`mypy` will fail to + resolve, which is why the `.claude/settings.json` allow list uses the venv-relative + paths and `make` targets rather than bare tool names. +- Tatlock replies open with a `` block — always strip it via + `tatlock.strip_reasoning()` (`gateway/src/desklock_gateway/tatlock.py`) before TTS or + display. Verified present and called at the one call site. +- Low power is a stated hardware requirement — read "Power management" in + `docs/architecture.md` before touching the face/render loop. +- Gateway health check is `GET /healthz` (verified in `main.py` and matches the + container healthcheck in `tatlock-ui.yml`), not `/health`. ## Commands ```bash -# Firmware (requires `source ~/esp-idf/export.sh` first; IDF ≥ 5.4) +# Firmware (requires `source ~/esp-idf/export.sh` first; ESP-IDF ≥ 5.4, this box has 5.5) cd firmware && idf.py build idf.py -p /dev/ttyACM0 flash monitor # Gateway -cd gateway && make setup # once -make run # dev server :8600 -make test # pytest; single test: .venv/bin/pytest tests/test_health.py -k healthz -make lint typecheck +cd gateway && make setup # venv + dev deps (no ML models) +make setup-speech # additionally install faster-whisper + piper +make run # uvicorn on :8600 with reload +make test # pytest (tests/test_health.py, test_tatlock.py, test_commands.py) +make lint typecheck # .venv/bin/ruff + .venv/bin/mypy ``` -## Gotchas +## Liveness -- ESP-IDF v5.5 lives at `~/esp-idf` (`source ~/esp-idf/export.sh`). Flash via - `sg dialout -c …` (see AGENTS.md) — the login session predates dialout membership. -- **PSRAM 200 MHz requires `CONFIG_IDF_EXPERIMENTAL_FEATURES=y`** — without it the - option silently degrades to 20 MHz and the DSI display underruns into a watchdog - loop. Details in AGENTS.md. -- **Wi-Fi = esp_hosted 2.x on BOTH chips** (host manifest + C6 slave, auto-OTA'd from - `main/c6_slave.bin`). 1.x on IDF 5.5 gives working control RPC but a dead data path - (the great July 14th debugging night). Boot-loop assert on - `xTaskCreateStaticPinnedToCore` = internal-RAM famine. Details in AGENTS.md. -- Gateway speech deps are optional extras; `make setup` alone runs the app and tests - without GPU/ML packages. `make setup` uses `python3.12` (system python3 is 3.8). -- Tatlock replies open with a `` block — always strip via - `tatlock.strip_reasoning()` before TTS or display. -- Low power is a prime user requirement: see "Power management" in - docs/architecture.md before touching the face/render loop. +- **Gateway (`desklock-gateway` container, port 8600):** confirmed live — `docker ps` + shows the container running under that name (method: direct container inspection; + blind spot: none relevant here, this confirms the process is up, not that every route + behaves correctly — that would need a request against it, not checked this pass). +- **Firmware / device:** liveness is **undetermined** and cannot be established the way + the gateway's can. No `/dev/ttyACM0` was present in this session (checked: `ls + /dev/ttyACM*` found nothing) and there is no remote telemetry — the device only proves + itself alive over a physical USB serial connection or by joining the LAN and speaking + the WebSocket protocol, neither of which this session had access to. Do not infer + device state from repo contents or from the gateway being up. +- `strip_reasoning()` reachability: confirmed by direct read of + `gateway/src/desklock_gateway/tatlock.py` (method: source read of the one call site; + blind spot: does not confirm it's exercised by a live request — that's what + `tests/test_tatlock.py` is for, not re-run this pass). + +## Work tracking + +This repo's vault is standalone — its tickets and internal decisions live in its own +`.pql/` and `governance/`, and travel with a clone (`.pql/changelog/` is committed). + +```bash +/home/jpmschweitzer/.local/bin/pql ticket list +/home/jpmschweitzer/.local/bin/pql plan whatsnext +/home/jpmschweitzer/.local/bin/pql decisions list +``` + +`pql` is not on the non-interactive PATH — use the absolute path above. From inside this +repo no `--vault` flag is needed (pql anchors at the nearest `.git/` ancestor, which is +this repo) — but that also means a bare `pql` run from the **workspace root** will not +see this repo's tickets, and a write from the workspace root would go to the wrong +vault. Cross-repo/stack-level decisions (host, network, deploy mechanics — none specific +to desklock were found at the time of writing) live in the workspace vault instead: + +```bash +/home/jpmschweitzer/.local/bin/pql --vault /mnt/media/Projects decisions list --domain desklock +``` + +## Git + +- **History is linear — no merge commits.** Work on `main`, or a short-lived branch that + is fast-forwarded and deleted. This is the workspace-wide policy; there is no + per-repo exception here. +- Conventional Commits (`feat:`, `fix:`, `refactor:`, `docs:`, `chore:`). +- Stage explicitly — never `git add -A` (denied by `.claude/settings.json` policy). +- Update `CHANGELOG.md` under `[Unreleased]` for user-facing changes. + +## Releasing (gateway only — firmware has no release flow) + +Deploy is not automatic — confirm one is wanted first. + +1. Bump `version` in `gateway/pyproject.toml`. +2. Move `[Unreleased]` entries into a dated `CHANGELOG.md` section. +3. Commit, tag `vX.Y.Z`, push with tags. +4. `.gitea/workflows/build.yml` runs lint + pytest on every push to `main`; on a `v*` + tag it additionally builds and pushes + `git.schweitz.net/jpmschweitzer/desklock-gateway:{latest,tag}` and pings Watchtower. +5. Verify: `curl http://192.168.86.149:8600/healthz`. + +## Architecture + +`docs/architecture.md` is the source of truth for system design, the face design +(`sim/face/index.html` is its visual source — change both together and verify with +`~/bin/claude-screenshot`, noting its `--virtual-time-budget` starves +`requestAnimationFrame`, so sim animation is driven by `setInterval` instead), power +budget, latency budget, and the full WebSocket protocol spec. Not restated here because +it is detailed enough to drift if duplicated — read it directly.